X-RAIS: The Third Eye

Download Slides

The health emergency underway worldwide has highlighted the need to strengthen the surveillance and care of the sick at home, to avoid hospital overcrowding that we have seen in recent months inevitably compromise the management of the emergency itself. X-RAIS is an AI tool, which as a third eye supports radiologists during the reporting phase of radiological images. Within this context, we extended X-RAIS capabilities with ALFABETO (ALl FAster BEtter TOgether).

ALFABETO has the main objective of assisting healthcare personnel in the initial triage phase at the patient’s home: using instrumental data, anamnestic data, etc., ALFABETO carries out an objective evaluation of the degree of severity of the pathology and a predictive analysis of the possible evolution in the short to medium term, thus providing the essential elements to decide the care strategy to be implemented (home care vs. hospitalization).

The project involves the development of a software platform – integrated with diagnostic instrumentation – based on Artificial Intelligence components and aimed at supporting the diagnosis, sharing clinical information in real-time with the various health professionals involved in the process and predicting the evolution of the pathology.

Speaker: Federico Comotto


– Hi everyone. I’m Federico Comotto, Data Scientist at Laife Reply. Thank you very much for joining us today at this session, X-Rais The Third Eye. So before deep dive into this session, I want to introduce the company where I work, which is Laife Reply. In Laife Reply, we combined the vertical expertise on healthcare and pharmaceutical, with the technological specialization on Artificial Intelligence, Machine Learning and Big Data. So basically, we develop those innovative and technological solution in order to solve real problems in the healthcare and pharmaceutical industry. Here you can see some of the use cases that we usually work on. They are, for example, medical imaging, radiomics, natural language processing, but also chat bot and behavior monitoring monitoring application. So if you want more information about us, please don’t be afraid and email us at info.laife@reply.it. Okay, let’s start by looking at the agenda for today. I’m very excited. So we will start with the context where X-Rais was born, then we will move to what is X-Rais. And then Alfabeto, which is an important platform that we developed and it is based on X-Rais. And finally, I want to make some conclusion about of course X-Rais, but also on Artificial Intelligence more in general. Okay, let’s start with the Context. So here, we can see some pain points in the healthcare industry. But more in detail, these are pain points in the mammography area. So you have to think about that, 40 million of mammograms are performed in the USA every year. This is a huge number. And you also need to think about that we don’t have enough experts radiologists to process and analyze all these images. So, I think that it’s time and we need some innovative tools to help radiologists in this process. Another important data is that the 80% of malignant tumors in mammograms are detected. This is what in machine learning we usually call accuracy. I think that this is a pretty high number, but we could and we also should improve this number, maybe by using some artificial intelligence, why not? And, last but not least, of course, is that according to the well more adaptation, sorry, 2.1 million of women suffer from breast cancer each year. This is, again, a huge number. And I think that we should help them in some way. So what is the story behind this data? The story is that, we have a lot of images, we have a lot of mammograms, but we don’t have enough radiologists, at least X ray radiologists to analyze them. We are pretty good at the detection of malignant tumours in mammograms, but we could and we should improve. And finally, as I told you, we have a lot of women that suffer from breast cancer every year and we should we should help them in some way. This is the idea underneath X-Rais. This is why X-Rais was born. Okay, but what is X-Rais? Okay, X-Rais is a medical platform. Actually the CE marking is still going on process. And the we aim at the secondary category, which is basically a medical device to support diagnosis and X-Rais is based on deep learning. And can be tumors in different modalities, such as X-Rais automography, and different anatomical details. Such as the mammography, or the chest. So, in the in the at the very beginning, we develop x rays on the anatomical details of mammography and using X-Rais. And, of course, X-Rais is cloud based, because we need a lot of computational power. But we develop x rays just by using some open framework based on Python. So, I want to mention our journey. This is very important for us. So, X-Rais was born in January 2018, with a prototype for analyzing medical images. But in March 2018, we sealed an important partnership with Healthcare Hospital in Perea, which is Maugery, to develop X-Rais, in order to identify any type of anomaly on mammography. And then in 2019, we see later another important partnership with the Treviso Hospital. In this case, to detect the suspicious anomalies in mammograpghs. And finally, in April 2020, we start with with a new project, which is Alfabeto. And of course, involved X-Rais. And it was the, the main idea it was to use X-Rais in order to help support the healthcare system in the during this pandemic. So, of course, there is a big community around X-Rais. Here you can see the just the milestone for our, for X-Rais. Okay, let’s start with the very, very first visualization. That’s, that’s mammograms. So the idea is that X-Rais is abled by using Deep Learning to take the mammography and then analyze, analyze the mammography and find some important information. This information is, first of all the type of anomaly. But also the morphology of the anomaly, which is for example, the shape. So, in this case, irregular, and the margin that could be speculated. So, by using some artificial intelligence, some deep learning algorithm, we are able to identify important features in the mammography. These features are essential, and people of algologist in the in the everyday job. So of course, these can really speed up and add the diagnosis in the work of of radiologists. So, before moving to the next part of the speech, I want to mention some important numbers about X-Rais. Because as I told you, there is a community around the X-Rais in the also important data. First of all, we train the model over more than 20,000 images, where we detected more than 19 anomalies. And we find 5000 cases. So this is a huge number, especially for some training. Because of that, as I told you, we use cloud based resources, because we need a lot of computational power. As you can read from the slide, at least 10 servers in more than 50,000 iteration for the deep learning algorithms. Because of course, we need a good accuracy. And finally, we have this community around the X-Rais that involve 10 partners, and also community of radiologists that help us in the annotation of the images. It is an important and fundamental task. Of course, I give you just the idea of X-Rais, the high level idea but if you want more information, please take a photo at this QR code. I’m going to give you a couple of seconds. Okay perfect. So, now, we are in the second part of the speech which is, Alfabeto. Okay, but what is Alfabeto? Alfabeto state for, All Faster Better Together. Alfabeto is a totally different project, which involves X-Rais, because we thought that x rays could help in the support, in support sorry, the diagnosis during this COVID 19 pandemic. So, the idea was to identify those patients that should go to the hospital or stay at home. But I don’t know, in the USA, I don’t know in the, in other country, but what we are seeing in Italy is that, the hospital are suffering, because they’re over food, they are collapsing. because they have a lot of patients, we need some way to simplify this process. So the idea was to identify the severity of the disease in the patient’s home. But what is Alfabeto? So as I told you, Alfabeto is a totally new platform, based of course, on artificial intelligence, which aims to reduce your hospitalization by supporting and speeding up diagnosis of COVID-19. The idea came by the collaboration of three important parties in the healthcare system, which are of course, Laife Reply, so the company where I work, but also the Instituti Clinici in Maugery, which is located in Pavia. It is an important health care hospital, and also the University of Pavia. So I like to say that this is the Excellence for our Better healthcare. So, but let me summarize what is the process involve, that involves Alfabeto. So, as I told you, the idea was to take care in some way of the patients at home. So think about that, we have some symptomatic patients that ask for assistance at home. At that point, a medical team, visit the patient at home, of course, with with the support of Alfabeto of the platform of Alfabeto. At that point, the patient is subjected to some test. First of all this the chest x rays, so we take the images, the X rays of the chest. And, of course, we collect other type of data. We call it clinical data. These data are, for example, the temporary tools of their patients, but also the saturation. It also information about any type of disease, this this patient could have, such as diabetes, for example. And then Alfabeto starts with the severity assessment and the progression estimate of the COVID-19 disease. This is an interactive session where the model is able to perform a classification task, but also report and summarize all the information and share this information with all the healthcare professional. We call we call this the reporting phase. At that point, the medical team, which is the support of Alfabeto and also a doctor, that is remotely connected to the platform, to the Alfabeto platform, figuring out which what is the best strategy for the patient which is, which could be hospitalization versus home stay. Okay, let’s deep dive into the technological component of Alfabeto. Of course, the machine learning process involves x rays in some way. So this is the high level pipeline. The first component is x rays, as I told you. In this case, we use the capability of X rays in some way we extend the capability of X rays in order to extract, to use it as a feature extractor. So the idea is that, by looking looking at the chest x rays, were able to identify some important feature. These features are then combined to serve clinical data, those that I told you before, tempura towards saturation in other type of disease, and feed to to another model. This model is a classifier. And the task of this model is to identify if the patient should stay at home or go to the hospital. But please mind that at the very end, there is always the doctor that take the last decision. As I told you, this is a platform and a doctor will be remotely connected to this platform. In order to analyze what Alfabeto said. And finally, choose the best strategies for the path for the patients. Okay, let’s look at the first component that of course, it is x rays. As I told you, we use x rays as a feature extractor. The deep learning model that we use is simply and let me say that DenseNet. So the algorithm was developed and trained in order to identify these six different feature, which are Consolidation, Infiltration, Edema, Effusion, Pneumonia or Lung Opacity. We also identify and select this feature with the support of our expert radiologist, which of course, approved all this feature, because said that these are the most relevant for the COVID-19 task. And also those feature that their radiologist could observe in order to distinguish between patient with COVID and patient without COVID. So as I told you, these features are then combined to the clinical data that we collect at home, temporal to situation and so on, and feed to another model, which is a classifier. So let me explain this classifier. First of all, it’s cloud based. Because even in this case, we need a lot of computational resources and computational power to train the model. In fact, we train more than 50 model by using auto ML. These techniques allows us to train a lot of model in a faster way, of course, on all the models are trained by using the cross validation techniques. And the finally model that we, the best model is an Ensembled one. So it’s a combination of different model, which performed pretty well, the AUC is 81%, that means that in the 81% of the cases, the model is able to distinguish if the patient should go at home or stay to the hospital. I think that this is a great result, especially considering that this is these are all real world data. So, you can even look at the ROC curve of this model. So you can see the performance, the average of the performance of the model is far away for the purple line, which of course, means random choice. So to summarize, we use, we feed the model to two different modern x rays. By using the chest x rays. We combine the feature that we extract from x rays with some clinical data and we train another model in order to classify is if the patient should go to the hospital or stay at home. So, let me just summarize or what are the advantages of this important platform of Alfabeto. First of all, we can take care of the patient is at home. So we can test in which we can check if the patient should go to the hospital or stay at home. Just by going to the patient’s home. So, we should not have any more of the problem of over full hospital. And then we also have a dynamic triage because there is this interaction between different components, the technology, which is Alfabeto, the platform but also the medical team, the patients and the doctor that is mortally connected. As I told you, we can reduce the access to the hospital because of course, we can distinguish the patient’s own if the patient should go to the hospital or not. Another important aspect of Alfabeto, it is an important advantage is that, the information is shared in real time with the Afghan professional and the patients. In fact, we built a platform that is able to share all these the information. So the data, the X rays, images, and also the prediction, both with the medical team but also with a doctor that is remotely connected. And finally, of course we can reduce the clinical risk and define more embedded invest in the best strategy for the patient. So we are really taking care of the patients. Okay, but now it’s time for the conclusion. Okay, first of all, I want summarize what is X-Rais. So as I told you, is an artificial intelligence solution that use deep learning to analyze the medical images in support doctors in the everyday job. So it’s very important to support in some way the doctors in everyday job. As I told you, for example, in the mammography area, we have a lot of images, we don’t have enough expert, and we should add them in some way. And this is why we developed X-Rais at the very beginning. But second, we can also inherently allows the inclusion of heterogeneous data to perform predictive analysis. So this is the case of Alfabeto, where we use, we extend the capability of X-Rais with another component, another machine learning model, in order to perform a prediction on the evolution of the pathology. And we were also able to combine structured data, such as clinical one, that we can collect it in the patient’s home with, with unstructured data, such as the chest x rays. And finally, this is very important. And I want to mention that we can improve the medical screening and reduce the diagnosis time, which means, of course, higher quality in medical treatments, and reduction in clinical risk. I think that this is very important, especially for the healthcare industry. And finally, I want to say that I think that it’s time for the AI. So, AI is saying that disease costs more than health. And this is very true. Artificial intelligence, I think that the is mature to help support the healthcare system in the diagnosis phase, but also in other process. We can in some way automatize process, we can simplify process. So, we can use big data and artificial intelligence to make the healthcare industry a better place. And in Laife Reply, we develop these innovative solutions based on artificial intelligence and big data to help the healthcare system but also the pharmaceutical one in to save lives. So I think that this is the real goal of our everyday job. So thank you very much for joining us to this session. Please, if you want more information, email us at info.laife@reply.it. Also, don’t forget to read and review this session, it is very important for us. And please, it’s time for questions. So, if you have any question, don’t be shy.

Watch more Data + AI sessions here
Try Databricks for free
« back
About Federico Comotto

Laife Reply

Data Scientist at Laife Reply.