Background
The severity of the rapidly spreading Covid-19 pandemic — (SARS-CoV2), a new type of coronavirus belonging to the genus β— is due not only to the lack of an effective and efficient vaccine, but also in the lack of inexpensive, fast, and reliable testing. Artificial intelligence technology can help.
Current screening techniques are slow, expensive, and require trained healthcare workers
Today’s gold standard for testing are Viral cultures and RT-PCR. However, it takes hours to detect the nucleic acid and days to isolate the virus. In addition, specialized instruments and expertise are required. Even then, these screening tools are not 100% accurate. For example with a nasal swab, it’s easy to miss the RNA. If the healthcare worker doesn’t swab deep enough, or if the patient’s mucus coats the swab, the test may miss it.
Science is urgently exploring rapid testing tools including rapid antigen detection (RAD) tests, but there is not yet enough data to know how accurate these will be.
One possible answer: less random swabbing, more (and smarter) listening
Knowing that COVID-19 impacts targets the respiratory system, could cough patterns from infected individuals be sufficiently distinct to indicate the presence of absence of illness? This approach has worked well to accurately diagnose viral fever and pneumonia.
If so, a cough capture app could be useful. Based on this premise, in early March this year, Wipro research teams, working closely with Clinicians and Speech Pathologist from the HCG Hospital in Bangalore and supported by COVID Hospitals like BMCRI, Bangalore and the Bhaktivedanta Hospital in Mumbai, focused on recordings of patient as an input to a screening algorithm that leverages advanced artificial intelligence technology. The study, approved by the Ethical Committee, is a unique blend of Medical, Speech Sciences and Engineering Prowess to speed innovation in COVID-19 techniques for the greater good.
Wipro COVID-19 AI technology research had three goals:
Toward a mobile Cough Capture app
Cough signals captured using a simple ‘do it yourself’ recording would be analysed by a learning algorithm to identify audio signatures specific to people who are infected. COVID-19 cough detection, and the ongoing creation of a more robust COVID-19 cough dataset, is an ideal application for advanced artificial intelligence technology. As more data from these recordings becomes we can improve sensitivity and accuracy to further validate the concept.
Material and method:
Data:
The proposed Covid-19 do-it-yourself mobile cough test and cough detection captures audio, as well as self-reported data including gender, age and other health indicators, and uploads it to a secure cloud server for analysis using advanced artificial intelligence technology.
Here is some sample data from our early tests.
Pre-processing of data:
Before being accepted, each sample is tested to ensure that a cough has been captured at sufficient length and volume to be useful. A virtue of the proposed system is that almost any type of cough sample can be used. There are no strict requirements about how a tester must cough.
Once accepted, audio samples are used to generate Mel-spectrogram and MFCC (Mel-frequency cepstral coefficients) for further processing.
Model development and early results
A number of Machine Learning and Deep Learning COVID-19 model projections have been developed using the accepted cough audio samples. Each model examines the audio files to determine if the tester shows an audio signature that indicates if the tester is infected with COVID positive, or negative.
To train each model, a portion of samples was used for training and another set were used for validation of COVID-19 model predictions as shown in the table above.
A set of 1553 validation samples was used. 25 of these were Covid-19 Positive; 1528, mostly obtained from public domain pneumonia and normal cough data, were negative. The performance of the model is 95% with 2 false negatives (Covid positive is identified as negative) and 75 false positives (negative samples identified as positive).
In another experiment with segmented cough i.e one or more cough segment are extracted from a given recording, performance of the COVID-19 model has an accuracy rates over 85% and in one case going up to 93% for Training and 86% for Validation. In this experiment each cough segment is passed to the model for inference. Decisions are made based on majority voting among different models, as well as the confidence level of prediction by the model.
The ROC curve and improving ROC Model accuracy
ROC model evaluation (Receiver Operating Characteristic) and improving performance based on the ROC curve are critical next steps To enhance performance and reduce misclassification, we are working to establish the threshold of confidence level using ROC and the number of pure neurons responsible for both types of classification rather than relying on inference from the model.
Additionally, because the number of training samples for ROC model validation is limited, we’re pursuing an ensemble model that leverages all the models which may enhance performance significantly. Usually, for a two-class classifier, the model generates a class for which the probability (we consider as confidence level of inference) is more than 0.5. In our analysis we observed that majority of the correct classification have high confidence and false classification have value around 0.5 though there are exceptions. We define a threshold of confidence to reduce misclassification and the threshold is defined based on ROC curve. When the confidence of classification is more than the threshold in that ROC curve, we accept the class. Otherwise, we mark it as inconclusive. In some of the samples, the confidence levels from multiple models are different. In such cases, the decision is taken based on majority voting. This helps to reduce ambiguous decisions.
We also analyse neurons responsible for different classes. We take the dense layer and find which all neurons are activated only for Covid-19 positive classes and only for negative classes. We call those pure neurons. During inference, we check the percentage of pure neurons activated for the given class along with confidence score for decision making. This also helps us making better inference.
Learnings and gaps
Can society safely unlock and achieve a “new normal”?
Key use cases
We foresee two important use cases.
First, screening for medical interventions. These would be done at Hospitals by trained medical staff for treatment of infected patients.
Second, screening in public places like Airports, Malls, restaurants and beyond to arrest the spread.
A powerful advantage of AI-enabled, do-it-yourself cough testing is that it would be effective, rapid and easily reproducible: able to screen much larger numbers with far greater accuracy. In its final phases, we envision a system that can work independently and objectively, reducing risks to the entire society of unlocking and achieving a “new normal” .
There is more work ahead, but the early results are encouraging. However, the issues surrounding AI in healthcare remain the same as ever. Concerns about data quality, the absence of humans, and the overall accuracy of AI tools are perhaps even more important to bear in mind for this application than others.