Data in Medicine: AI & MRIs

Facebook AI Research (FAIR) is teaming up with New York University School of Medicine to test whether artificial intelligence can make MRI scans a faster process. They believe that AI can produce clearer images than current MRI scans all while taking 1/10th the time and capturing less data.

Below is the original article from Fast Company:

If you’ve ever had an MRI, you know it can be a long, drawn-out, and very loud process. Given that you wouldn’t even be having an MRI unless you and your doctors were concerned about something, it can be stressful–not to mention claustrophobic–to be jammed into the machine’s chamber for up to an hour.

That’s why scientists at Facebook AI Research (FAIR) and the New York University School of Medicine have been developing technology that could help make MRIs as much as 10 times faster than is currently possible. Beyond that, they say the effort could result in the diagnostic technology being available to many more people, and it could eventually replace X-ray and CT machines in some cases, potentially allowing patients to avoid being subjected to radiation. It’s not clear how long it could take for the research to be implemented in hospitals.

NYU School of Medicine researchers Michael RechtYvonne Lui, and Daniel Sodickson examine an MRI. [Image: courtesy of FAIR and NYU School of Medicine]

For Facebook, this project has nothing to do with its bottom-line business or even its mission of connecting the world. It’s just about extending the capabilities of AI. The company has performed numerous research projects with third-party groups, particularly relating to AI, but this is the first time that such a project has aspired to solve a problem with such a large societal impact and been grounded in something tangible.“In FAIR, our mission is to explore the scientific problems of AI, both fundamental and practical,” said FAIR scientists and project lead Larry Zitnick. “But we can’t do this alone, which is why we publish, open source, and often collaborate with others. The FastMRI project is unique in that it forces us to explore image understanding in novel ways, and partnering with NYU offers a tangible path to bettering society.”

In a blog post about the research project, Zitnick and Dan Sodickson and Michael Recht from the NYU School of Medicine wrote about the pros and cons of MRI technology. It produces imagery that captures imagery of much higher degrees of detail in soft tissue areas like organs and blood vessels than imaging systems like X-rays or CT scans. But it comes with the downside of being far slower.

Many people, of course, will put up with the slow procedure because of the diagnostic capability, but the blog’s authors write that MRIs can be particularly challenging for children or those who suffer from claustrophobia or pain from lying down. Further, they write, many rural areas and countries with limited resources often have shortages of MRI machines. They hope their work can result in a vast increase in the number of people who can benefit from the technology.

The reason MRI scans take so long, Zitnick, Sodickson, and Recht argue, is that the machines work slowly at collecting raw data in numerous sequential views and then converting those data to the kind of images of people’s internal body structures that are useful for doctors. The larger the body area that is being scanned, the longer an MRI will take, which explains why they vary in time from 15 minutes to more than an hour.


Artificial intelligence could make it possible, the project’s researchers believe, to produce the same–or even better–MRI imagery even while capturing less data and taking less time. They argue that training neural networks to understand the images’ underlying structure could fill in gaps left by quicker scanning. The authors explain that this method is similar to the way people take in sensory information–with our brains completing incomplete pictures, such as blocked or poorly lit objects. They also say that their early work on the topic shows that properly trained neural networks are able to perform such tasks and can produce high-quality imagery using much less data that was once thought necessary.

Raw MRI data that has been under-sampled. [Image: courtesy of FAIR and NYU School of Medicine]

The big challenge is achieving high accuracy while also minimizing scan times. Mistakes could result in misdiagnosis. The authors write that they recognize that manifesting that work in real-world medical situations will require further knowledge about AI as well as large-scale computing resources.The NYU researchers began work on using AI to speed up MRIs in 2016. At around the same time, Facebook’s AI Research team (FAIR) was looking for ways to impact real-world AI applications. NYU’s project dovetailed with that of FAIR, and allowed the Facebook scientists to employ the expertise they’d developed over several years in computer vision, as well as their capabilities for training large-scale models, which they could combine with NYU’s expertise in imaging science.

For this research project, the NYU Medical School’s researchers utilized a data set of 10,000 clinical cases–about 3 million MRI images of knees, livers, and brains. They hasten to point out that in all cases, the data was stripped of identifying information about patients, and that the work is HIPAA-compliant and has been given the green light by a university review board tasked with oversight of human subject research. Further, no data from Facebook is being used in the project.

The NYU and Facebook scientists plan on open-sourcing their work in the expectation that the wider research community will be able to replicate and build on their findings.

An MRI image of a knee as subsampled data (left) and reconstructed from fully sampled raw data. (right) [Images: courtesy of FAIR and NYU School of Medicine]

And while the initial project goals revolved around MRI technology, the scientists feel that their work could be extended to other imaging technologies, including CT scans. The theory is that AI could help reduce the radiation doses required to conduct such scans–something that could be particularly helpful for vulnerable patients such as children.

Ultimately, the underlying hope for the project is to demonstrate the power of AI and machine learning to generate high-value imagery in new and more efficient ways. “With the goal of radically changing the way medical images are acquired in the first place,” the authors write, “our aim is not simply enhanced data mining with AI, but rather the generation of fundamentally new capabilities for medical visualization to benefit human health.”