Wuhan Huangpi People’s Hospital: An AI Framework to detect COVID-19 Using Chest CT

Diagnosing COVID-19

The article we’re demystifying can be found here, if you’d like to follow along.

The highly contagious nature of COVID-19 makes it critical to have efficient diagnostic procedures. You have likely heard of RT-qPCR, currently the most common lab diagnostic test for COVID-19. In an RT-qPCR test a throat or nasal swab is collected, samples are sent to the lab, and results are back within a few hours or up to 2 days.

While RT-qPCR is currently the gold standard, concerns have been raised regarding its capacity for early detection. PCR is also quite costly, requires a complicated manual process, and has a long turn-around time. Computed tomography (CT) is an alternate diagnostic method, by which cross-sectional images of the lungs are used to detect indicators of COVID-19.The use of CT presents an attractive alternative since scanners are widely available and the tests can be conducted more quickly.

The challenge arises in the subtlety of COVID-19’s visual indicators, and the radiologist’s task of interpreting these images. This article proposes a deep learning framework that automatically detects COVID-19 from chest CT scans.

What makes a good test?

First of all, let’s consider what constitutes a “good’ diagnostic test. Sensitivity and specificity are two attributes used to evaluate screening methodologies. Sensitivity is the ability to correctly identify patients who DO have the disease. Specificity refers to identifying patients who do NOT have the disease. Ideally, a reliable diagnostic would have high scores for both attributes. 

RT-qPCR (reverse transcription qualitative polymerase chain reaction) is a lab procedure used to detect the virus’s presence. To find out more about RT-qPCR, check out this article written by Ryan Chan.

A Closer Look at CT

Computed tomography (CT/CAT scan) is an imaging procedure used to visualize tissues, bone and blood vessels. X-ray emitters and detectors rotate around person’s body and measure the amount of radiation absorbed. The images are then assembled on a computer, providing radiologists with a three-dimensional view of the interior. Different body parts absorb x-rays in different amounts. For example, denser tissue like bone is more likely to scatter or absorb the x-ray, resulting in less energy being detected and generating a lighter image. In a scan, bones appear white, tissues grey, and air appears black.

Things to Look For:

Symptoms of COVID-19 include inflammation, fluid and pus in the lungs. There a number of visual indicators that radiologists look for, including;

1. Ground-glass opacification (GGO) – a hazy opaque region that indicates partial filling of lung air spaces. Air spaces would normally appear black on CT, but instead appear cloudy white due to fluid exudation or partial collapse of alveoli. These changes appear early on in the course of the infection
2. Crazy-paving pattern – a linear pattern appearing overtop a GGO, indicative of interlobular thickening. This is observable as the disease progresses (9-13 days, up to a month afterwards)
3. Consolidation – a region of normally compressible lung tissue becomes filled with liquid instead of air.

The Study

This study is a retrospective study, which means that it uses data that was collected at some point in the past for another reason besides this research. The study was conducted with a dataset of volumetric chest CT exams from 3,506 patients at participating hospitals. The researches evaluated their model’s performance by its ability to distinguish COVID-19 from community acquired pneumonia and other non-pneumonia exams. Here’s a breakdown of the patient diagnoses in their dataset:  

1. 1296 (30%) positive for COVID-19 (confirmed with PCR, acquired from Dec 2019 and Feb 2020. )      
2. 1735 (40%) positive for community-acquired pneumonia
3. 1325 (30%) positive for non-pneumonia lung conditions

How does the Machine Learn?

The authors present COVNet, a 3D deep learning framework that extracts both 2D and 3D representative features of COVID-19. Here’s how it works.

Input: a series of CT Slices

1. Preprocessing – The first step is to extract the lung region as the region of interest (ROI) – using a U-net based segmentation method
2. RestNet50 – The backbone for many computer vision tasks, RestNet is an artificial neural network that uses skip connections to learn faster and with fewer layers in the initial training. It was used to generate features for each of the corresponding CT slices.
3. Max Pooling – The extracted features are then combined by Max Pooling, an operation that calculates the largest value in each patch of a feature map, then continuously down-samples and repeats the process to highlight the most present feature in each patch.

Output: Probability Score generated for each diagnosis class (COVID-19, CAP, and non-pneumonia)

To illustrate the most important regions leading to the model’s decision, the authors created heatmaps using Grad-CAM. This heatmap illustrates the regions that the algorithm paid the most attention to in its classification process.

The red colour in these heatmaps highlights the activation region associated with the predicted class. Notice that normal regions as shown in the non-pneumonia example were ignored while the regions of ground glass opacity are emphasized in the scans for COVID-19 and CAP.

Putting COVNet to the Test

The model’s performance was evaluated using an independent testing set. An independent testing set is a set of data separate from the data used to train the framework. The table below summarizes the model’s performance against four metrics: sensitivity, specificity, AUC (area under the curve), and P-Value. A p-value is a complicated concept to explain, but this link does a good job.

From the table, we can see that the model achieved high sensitivity and specificity scores of 90% and 96% for detecting COVID-19.

ROC and AUC are additional performance metrics, ROC (receiver operating characteristic) visually illustrates the tradeoff between sensitivity and specificity. The true positive rate (the number of times that the model says something positive is positive) is plotted on the vertical axis, with the false positive rate (the number of times that the model says something negative is positive) on the horizontal axis. A sharp curve angled in the upper left hand corner indicates excellent performance. AUC (area under the curve) quantifies this same metric, with values closer to AUC=1 indicating strong performance.  Figure 3a in the paper shows the ROC curve for the model’s performance in detecting COVID-19. The model achieved an AUC of 0.96.

Conclusions and Limitations

Once the model is trained, the average processing time for each CT exam is short, requiring just 4.51 seconds on a standard workstation. While RT-PCR is considered the reference standard, automated CT analysis could be used as a reliable and rapid screening approach, with the COVNET deep learning model demonstrating high sensitivity and specificity, and an AUC of 0.96.

While the results are promising, this study is not without its limitations. The first is that COVID-19 has similar imaging characteristics as pneumonia caused by other types of viruses. To further this study, it would be ideal to test the performance of COVNet in distinguishing COVID-19 from other viral pneumonias with real time PCR confirmation. Secondly, the challenge with deep learning is the lack of transparency in the model’s process. While the heatmap visualizations are helpful, they lack the ability to pinpoint the specific imaging features used in the classification process. Furthermore, the presentation of different lung disease varies with a number of host factors including age, immune status, and underlying co-morbidities. A multidisciplinary approach to this is recommended, in conjunction with chest CT analysis.

The results of this study are encouraging, suggesting the potential of AI research to support radiologists in rapidly detecting COVID-19. The use of standard CT equipment coupled with an automated classification process could empower more widely available testing. Further steps include being able not only to detect the disease, but also to classify its severity. This application would support disease progression monitoring, recovery assessment, and individualized patient care.