Learn how GE Healthcare used AWS to create a new AI model that interprets MRIs

MRI images are understandably complex and data-rich.

For this reason, developers Training large language models (LLMs) To analyze magnetic resonance imaging (MRI) I had to slice the captured images into 2D images. However, this results in only an approximation of the original image, which limits the model’s ability to analyze complex anatomical structures. This creates challenges in complex cases involving… Brain tumorsskeletal disorders or cardiovascular diseases.

but GE Healthcare It appears to have overcome this formidable hurdle, presenting the first basic model for whole-body 3D MRI research (FM) at this year’s show. Re AWS: Invented. For the first time, models can use full 3D images of the entire body.

GE Healthcare’s FM on AWS is built from the ground up — there are very few models designed specifically for medical imaging like MRI — and is based on more than 173,000 images from more than 19,000 studies. The developers say they were able to train the model with five times less computation than was previously required.

GE Healthcare has not yet commercialized the foundation model; It is still in the evolutionary research stage. Early resident General Brigham’s MassIt is scheduled to begin testing soon.

“Our vision is to put these models in the hands of technical teams working in healthcare systems, giving them powerful tools to develop research and clinical applications faster and more cost-effectively as well,” Barry Bhatia, chief AI officer at GE HealthCare, told VentureBeat. .

Enabling real-time analysis of complex 3D MRI data

While this is a groundbreaking development, generative AI and LLMs are not a new area for the company. Bhatia explained that the team has been working with advanced technologies for more than 10 years.

One of its leading products is Air Recon DLa deep learning-based reconstruction algorithm that allows radiologists to obtain clear images more quickly. The algorithm removes noise from raw images and improves the signal-to-noise ratio, reducing scanning times by up to 50%. Since 2020, 34 million patients have been scanned with AIR Recon DL.

GE Healthcare began work on MRI FM in early 2024. Because the model is multimodal, it can support image-to-text search, correlate images and words, and segment and classify diseases. The goal is to give Health care professionals There is more detail in a single scan than ever before, leading to faster and more accurate diagnosis and treatment, Bhatia said.

“The model has great potential to enable real-time analysis of 3D MRI data, which can improve medical procedures such as biopsies, radiation therapy, and robotic surgery,” Dan Sheeran, general manager of healthcare and life sciences at AWS, told VentureBeat.

It has already outperformed other publicly available research models on tasks including classification of prostate cancer and Alzheimer’s disease. It showed up to 30% accuracy in matching MRI scans with text descriptions in image retrieval – which may not sound impressive, but it represents a significant improvement over the 3% ability shown by similar models.

“It’s getting to a point where it’s giving some really strong results,” Bhatia said. “The ramifications are huge.”

Do more with data (much less).

the MRI process Bhatia explained that it takes a few different types of data sets to support different technologies that map the human body.

For example, what is known as T1-weighted imaging technology highlights adipose tissue and reduces the water signal, while T2-weighted imaging enhances the water signals. The two methods are complementary and create a complete picture of the brain to help doctors detect abnormalities such as tumors, trauma or cancer.

“MRI images come in different shapes and sizes, just like you have books in different shapes and sizes, right?” Bhatia said.

To overcome the challenges presented by diverse datasets, the developers introduced a “resize and adapt” strategy so that the model can handle and react to different variations. Additionally, data might be missing in some areas — the image might be incomplete, for example — so they simply taught the model to ignore those cases.

“Instead of faltering, the model taught us how to bridge the gaps and focus on what is available,” Bhatia said. “Think of this as solving a puzzle that has some missing pieces.”

The developers also used semi-supervised learning between students and teachers, which is especially useful when data is limited. Using this method, two different neural networks are trained on both labeled and unlabeled data, with the teacher creating labels that help the student learn and predict future labels.

“We’re now using a lot of these self-supervised techniques, which don’t require massive amounts of data or labels to train large models,” Bhatia said. “It reduces dependencies, as you can learn more from these raw images than in the past.”

This helps ensure the model performs well in hospitals with fewer resources, older machines, and different types of data sets, Bhatia explained.

He also stressed the importance of multiple paradigms. “A lot of technology in the past was monolithic,” Bhatia said. “It will just look at the image, at the text. But now it’s multimodal, where it can go from image to text, from text to image, so you can bring in a lot of things that were done with separate models in the past and really unify the workflow.”

He stressed that researchers only use data sets to which they have rights; GE Healthcare has partners that license de-identified data sets, and they are careful to adhere to compliance standards and policies.

Use AWS SageMaker to address compute and data challenges

There are undoubtedly many challenges when building such sophisticated models, such as the limited computational power of gigabyte-sized 3D images.

“It’s a huge volume of 3D data,” Bhatia said. “You have to put it in the model’s memory, and it’s a really complicated problem.”

To help overcome this problem, GE Healthcare built on this Amazon Sage Makerwhich provides high-speed networking and distributed training capabilities across multiple GPUs, and leverages Nvidia A100 GPUs and tensor core GPUs for large-scale training.

“Because of the volume of data and the size of the models, they can’t send it to a single GPU,” Bhatia explained. SageMaker allowed them to customize and scale operations across multiple GPUs that could interact with each other.

As developers used Amazon FSX in Amazon S3 Object storage, which allowed data sets to be read and written faster.

Bhatia pointed out that another challenge is cost optimization; With Amazon’s Elastic Computing Cloud (EC2), developers have been able to move unused or infrequently used data to less expensive storage tiers.

“Leveraging Sagemaker to train these large models — essentially for efficient, distributed training across high-performance multi-GPU clusters — was an important component that really helped us move faster,” Bhatia said.

He emphasized that all components were built from a data integrity and compliance perspective that took HIPAA, regulations, and other regulatory frameworks into account.

Ultimately, “these technologies can actually simplify, help us innovate faster, as well as improve overall operational efficiency by reducing administrative burden, and ultimately drive better patient care – because you are now providing more personalized care.”

Serves as a basis for other finely tuned specialized models

While the current model is limited to the field of MRI, researchers see great opportunities for expansion into other areas of medicine.

Historically, AI in medical imaging has been limited by the need to develop custom models for specific conditions in specific organs, requiring expert annotation for each image used in training, Sheeran noted.

But this approach is “inherently limited” by the different ways in which diseases manifest among individuals, and poses challenges with generalizability.

“What we really need are thousands of these models and the ability to quickly create new ones when we encounter new information,” he said. High-quality labeled datasets for each model are also essential.

Now with generative AI, instead of training separate models for each disease/organ combination, developers can pre-train one basic model that can serve as the basis for other fine-tuned specialized models.

For example, GE Healthcare’s model could be expanded to include areas such as radiation therapy, where radiologists spend significant time manually marking organs that may be at risk. It could also help reduce scan time during X-rays and other procedures that currently require patients to sit in the machine for long periods, Bhatia said.

“We’re not just expanding access to medical imaging data through cloud-based tools; we’re changing how this data is used to drive the advancement of AI in healthcare,” marveled Sheeran.

Daily insights into business use cases with VB Daily

If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from organizational transformations to hands-on deployments, so you can share insights to maximize ROI.

Read our privacy policy

Thanks for subscribing. Check more VB Newsletters are here.

An error occurred.