Skip to cookie consent Skip to main content

Researchers Use Foundation Models to Discover New Cancer Imaging Biomarkers

Suraj Pai

Researchers at Mass General Brigham have harnessed the technology behind foundation models, which power tools like ChatGPT, to discover new cancer imaging biomarkers that could transform how patterns are identified from radiological images. Improved identification of such patterns can greatly impact the early detection and treatment of cancer.

The research team developed their foundation model using a comprehensive dataset consisting of 11,467 images of abnormal radiologic scans. Using these images, the model was able to identify patterns that predict anatomical site, malignancy, and prognosis across three different use cases in four cohorts. Compared to existing methods in the field, the team's approach remained powerful when applied to specialized tasks where only limited data are available. Results are published in Nature Machine Intelligence.

"Given that image biomarker studies are tailored to answer increasingly specific research questions, we believe that our work will enable more accurate and efficient investigations," said first author Suraj Pai from the Artificial Intelligence in Medicine (AIM) Program at Mass General Brigham.

Hugo Aerts, PhD

Despite the improved efficacy of AI methods, a key question remains their reliability and explainability (the concept that an AI’s answers can be explained in a way that “makes sense” to humans). The researchers demonstrated that their methods remained stable across inter-reader variations and differences in acquisition. Patterns identified by the foundation model also demonstrated strong associations with underlying biology, mainly correlating with immune-related pathways.

“Our findings demonstrate the efficacy of foundation models in medicine when only limited data might be available for training deep learning networks, especially when applied to identifying reliable imaging biomarkers for cancer-associated use cases,” said senior author Hugo Aerts, PhD, director of the AIM Program.

Authorship: In addition to Pai and Aerts, additional Mass General Brigham authors include Dennis Bontempi, Ibrahim Hadzic, Vasco Prudente, Tafadzwa L. Chaunzwa, Simon Bernatz, Ahmed Hosny, and Raymond H. Mak. Other authors include Mateo Sokač and Nicolai J. Birkbak.

Funding: National Institute of Health (NIH-USA U24CA194354, NIH-USA U01CA190234, NIH-USA U01CA209414, NIH-USA R35CA22052 and NIH-USA U54CA274516-01A1), the European Union, European Research Council (866504) and Deutsche Forschungsgemeinschaft, the German Research Foundation (502050303).

Paper cited: Pai, S., Bontempi, D., Hadzic, I. et al. Foundation model for cancer imaging biomarkers. Nat Mach Intell DOI:1038/s42256-024-00807-9

Media contact

Elizabeth Murphy
Program Director, External Communications

About Mass General Brigham

Mass General Brigham is an integrated academic health care system, uniting great minds to solve the hardest problems in medicine for our communities and the world. Mass General Brigham connects a full continuum of care across a system of academic medical centers, community and specialty hospitals, a health insurance plan, physician networks, community health centers, home care, and long-term care services. Mass General Brigham is a nonprofit organization committed to patient care, research, teaching, and service to the community. In addition, Mass General Brigham is one of the nation’s leading biomedical research organizations with several Harvard Medical School teaching hospitals. For more information, please visit