Interpreting Medical Images with Artificial Intelligence

From Object Recognition to Medical Image Interpretation

In recent years, dramatic advances have been made in computer vision in the field of object recognition within colour images. Under optimal conditions, the object recognition rate can even approach that of human recognition.

For a machine to recognize an object, a cat for example, it must be fed thousands or even millions of different images of cats. The object of interest, the cat in this example, must be manually annotated and segmented within images in order to make the machine understand that all these objects, despite their differences, are actually cats. These images are then represented in matrix form to feed a machine-learning algorithm, as an optimization problem with large-scale variables and solutions generating millions of parameters, which will be used to recognize the object in an image that has not been annotated.

In medical imaging, however, things get tougher. The images are voluminous, in black and white, sometimes in 3D, and they may contain “noise”. Futhermore, teaching a machine to identify a tumor can be much more complex than teaching it to recognize a cat in a photo. A person without medical knowledge cannot make a diagnosis from a medical image. The same applies to a machine: the task is more complex because this medical knowledge has to be in the form of a mathematical model. For a machine to recognize an abnormality (such as a tumor) or an organ, it needs to be fed thousands of images where the area of interest was previously defined manually by trained personnel like radiologists. The number of possibilities is very large and the amount of information to compile and process is immeasurable; it is therefore necessary to develop efficient optimization algorithms (combinatorial optimization) to use the information and enable a machine to interpret these images automatically.

Ismail Ben Ayed, Professor in the Department of Manufacturing Engineering at the École de technologie supérieure (ÉTS) and Research Chair in Artificial Intelligence and Medical Imagery, a chair created on October 4, 2016, is actively involved in this aspect of data processing.

Presenting Professor Ben Ayed

Professor Ben Ayed completed his BA and MA in Tunisia. During his Master’s, he completed an internship at the National Institute of Scientific Research (INRS), at the Énergie Matériaux Télécommunications (EMT) Research Centre. This internship encouraged him to pursue a PhD at the same institute, in computer vision. In 2007, he became a researcher for GE Healthcare in London, Ontario. Some of his accomplishments include the creation of an algorithm that can automatically annotate the bones of the spinal column, and software that can evaluate the ejection fraction of the heart; both of these algorithms are now being used in hospitals. In 2015, he decided to continue his academic career at the École de technologie supérieure (ÉTS), in order to have more freedom in the direction of his research. His goal is to develop algorithms to be used extensively in the health field.

Practical Applications of Computerized Image Interpretation

These algorithms should help accelerate the analysis of X-ray, ultrasound, magnetic resonance or tomographic images, by facilitating the work of radiologists and accelerating the reporting process. They could also identify statistical trends by analyzing the images before and after surgery, which would subsequently help select patients according to the urgency of their case. However, the potential of this technology is much greater.

Indeed, imagine a program that continuously searches tons of images generated by medical imaging and analyzes them, considering all visible information on the image, not only that relating to the reason for the consultation. For example, a patient undergoes an X-ray of his spinal column because he has a sore back. The radiologist analyzing the MRI is a spinal column specialist and naturally pays attention to that part of the body. However, by looking just beside that area, the diameter of the patient’s abdominal aorta would be seen to be abnormally large, indicating a risk of rupture caused by an aneurysm, and leading to the patient’s immediate death.

This case is not unusual: it is estimated that 5% of patients undergoing MRIs of the spinal column are at risk of an abdominal aortic aneurysm. A program that can detect the abdominal aorta in all MRIs where it is visible could save lives! The same goes for a variety of abnormalities that other algorithms could identify.

Professor Ben Ayed is convinced that a unique computational model that will mimic the way humans interpret images will eventually be developed. This model will undoubtedly be complex and require powerful computers.