The COVID-19 pandemic, caused by the SARS-CoV-2 virus, has had devastating effects in our daily lives, public health and the economy. The SARS-CoV-2 virus is easily transmittable, causing cough, fever and fatigue and it can, in some cases, evolve to a severe pulmonary infection and even cause death. Chest radiography (commonly known as x-ray) can help in the assessment of the development of the pulmonary infection caused by SARS-CoV-2 and thus aid in determining follow-up and therapy. Given the complexity of chest radiography reading, radiologists play a fundamental role. However, radiologists often have to deal with a high number of exams to assess and often do not have enough to analyse the X-ray images. Also the number of radiologists is scarce, and as result the images are analysed by clinicians or interns with less experience. Artificial intelligence technology could thus contribute to reduce the workload for radiologists by proposing an objective second opinion and help in obtaining the correct diagnosis regarding COVID-19.

X-ray as a complementary COVID-19 diagnosis method

Chest radiography is a medical imaging method that allows to visualize the inside of a patient’s body in a painless and non-invasive way. In the particular case of COVID-19 screening and patient management, it is naturally important to analyse the lungs given the pulmonary infection associated with the disease. The goal of imaging the thorax is thus to assess the degree of pulmonary infection and understand the development of the disease, determining the follow-up and therapy of a given patient. Nevertheless, many COVID-19 patients have a normal X-ray in the early stages of the disease, which means that X-ray can only be used as a complementary diagnosis method to the traditional COVID-19 swab test.

Artificial Intelligence as a second opinion

The analysis of X-ray images should always be performed by trained staff and preferentially by a radiologist. In the case of COVID-19 patients, this is even more important as COVID-19 radiological features can be particularly subtle and difficult to distinguish from pulmonary infections caused by bacteria or other viruses. However, trained radiologists often have to deal with a high number of exams to assess and the resulting fatigue, human error and uncertainty could lead to misdiagnosis.

Artificial Intelligence technology could thus contribute to improve the analysis of x-ray images by proposing an objective second opinion and help in obtaining the correct diagnosis regarding COVID-19. These techniques are based on the extraction of features associated to COVID-19 (one simple example could be the difference between the right and left lung volumes), which are then analysed with the goal of predicting the risk of the subject having COVID-19.

How does CXR_AI4COVID-19 actually work?

The algorithm developed by INESC TEC, named CXR_AI4COVID-19, is based on a set of techniques known as Deep Learning methods. These techniques have been used successfully to solve problems which depend on automatic image analysis (such as number plate identification or autonomous driving) and has recently been used in the development of decision support systems in clinical settings.

One of the greatest advantages of Deep Learning when compared to other image analysis techniques is that the most relevant features for the diagnosis are automatically learned by the system. In order to do this, a large number of images (typically in the thousands) have to be analysed. These images must be representative of the different ways that COVID-19 appears in patients but also of normal subjects and patients with other pathologies. The learning process is done by incentivizing the system to predict the correct diagnosis for as many images as possible (COVID-19, normal and other pathologies). As the system sees and compares images, it tries to predict which ones have COVID-19 (and which ones don’t) and receives information on how many correct predictions it made, and on which images. In this way, the system can learn which features are contributing to a correct diagnosis and reinforce the importance of those features. On the other hand, if more incorrect diagnoses occur, the features that were extracted are not useful and the system looks for different solutions. Given enough images, the features learned by the system become representative of the pathology, allowing for automatic diagnosis.

In spite of the extraordinary ability of Deep Learning techniques, its application also has downsides when compared to simpler technology. For one, a large amount of images is needed, which can be complicated in new or rare pathologies. Secondly, most Deep Learning techniques receive an image as input and give as output the probability of having a certain pathology. This behaviour means that it is not immediately possible to determine which features the system has learned and why they are important to determine the diagnosis. This means that one must be extremely careful in the analysis of the results - for example, is the system actually learning the radiological features of COVID-19 or only that intubated patients are more likely to have COVID-19? In fact, these kinds of shortcuts learned by the system often lead to unfounded promises of superhuman performance which then fall apart when testing begins in a clinical setting.

The approach used for the development of CXR_AI4COVID-19 was designed so that these limitations are avoided, making the system more robust and transparent. As such, the radiologist receives not only the diagnosis predicted by the system but also which regions in the image were most important for the decision made by the system (Figure 1), allowing the radiologist to determine whether the prediction is based on clinical principles.

In conversation with the radiologists involved in the developed of CXR_AI4COVID-19, they consider that this project “where Medicine and Engineering walk side by side” there is true potential for “the creation of a diagnostic tool which can have a useful and powerful application in a clinical setting by aiding clinicians in detecting COVID-19 in chest radiography”. Furthermore, they stated that these kinds of systems are, more and more, the “Radiology of the future”.

In order to understand if CXR_AI4COVID-19 is prepared for testing in a clinical setting, an initial validation of the system in X-ray images from both national and international hospitals was performed, in which images from the Centro Hospitalar de Vila Nova de Gaia e Espinho (CHVNGE) were also tested. To compare with typical human performance, two radiologists were asked to read and make a diagnosis in 1861 X-ray images. On these images, the system showed a similar performance to the radiologists, showing that relevant features for COVID-19 were learned. As such, plans are now underway to test CXR_AI4COVID-19 in a real clinical setting in CHVNGE, where it can contribute with an easily interpretable second opinion regarding the presence of COVID-19 in chest radiography, and hopefully help in the fight against the COVID-19 pandemic.