Using artificial intelligence (AI) to supplement radiologists’ evaluations of mammograms may improve breast-cancer screening by reducing false positives without missing cases of cancer, according to a study by researchers at Washington University School of Medicine in St. Louis and Whiterabbit.ai, a Silicon Valley-based technology startup.
The researchers developed an algorithm that identified normal mammograms with very high sensitivity. They then ran a simulation on patient data to see what would have happened if all of the very low-risk mammograms had been taken off radiologists’ plates, freeing the doctors to concentrate on the more questionable scans. The simulation revealed that fewer people would have been called back for additional testing but that the same number of cancer cases would have been detected.
“False positives are when you call a patient back for additional testing, and it turns out to be benign,” explained senior author Richard L. Wahl, MD, a professor of radiology at Washington University’s Mallinckrodt Institute of Radiology (MIR) and a professor of radiation oncology. “That causes a lot of unnecessary anxiety for patients and consumes medical resources. This simulation study showed that very low-risk mammograms can be reliably identified by AI to reduce false positives and improve workflows.”
The study is published April 10 in the journal Radiology: Artificial Intelligence.
Wahl previously collaborated with Whiterabbit.ai on an algorithm to help radiologists judge breast density on mammograms to identify people who could benefit from additional or alternative screening. That algorithm received clearance from the Food and Drug Administration (FDA) in 2020 and is now marketed by Whiterabbit.ai as WRDensity.
In this study, Wahl and colleagues at Whiterabbit.ai worked together to develop a way to rule out cancer using AI to evaluate mammograms. They trained the AI model on 123,248 2D digital mammograms (containing 6,161 showing cancer) that were largely collected and read by Washington University radiologists. Then, they validated and tested the AI model on three independent sets of mammograms, two from institutions in the U.S. and one in the United Kingdom.
First, the researchers figured out what the doctors did: how many patients were called back for secondary screening and biopsies; the results of those tests; and the final determination in each case. Then, they applied AI to the datasets to see what would have been different if AI had been used to remove negative mammograms in the initial assessments and physicians had followed standard diagnostic procedures to evaluate the rest.
For example, consider the largest dataset, which contained 11,592 mammograms. When scaled to 10,000 mammograms (to make the math simpler for the purposes of the simulation), AI identified 34.9% as negative. If those 3,485 negative mammograms had been removed from the workload, radiologists would have made 897 callbacks for diagnostic exams, a reduction of 23.7% from the 1,159 they made in reality. At the next step, 190 people would have been called in a second time for biopsies, a reduction of 6.9% from the 200 in reality. At the end of the process, both the AI rule-out and real-world standard-of-care approaches identified the same 55 cancers. In other words, this study of AI suggests that out of 10,000 people who underwent initial mammograms, 262 could have avoided diagnostic exams, and 10 could have avoided biopsies, without any cancer cases being missed.
“At the end of the day, we believe in a world where the doctor is the superhero who finds cancer and helps patients navigate their journey ahead,” said co-author Jason Su, co-founder and chief technology officer at Whiterabbit.ai. “The way AI systems can help is by being in a supporting role. By accurately assessing the negatives, it can help remove the hay from the haystack so doctors can find the needle more easily. This study demonstrates that AI can potentially be highly accurate in identifying negative exams. More importantly, the results showed that automating the detection of negatives may also lead to a tremendous benefit in the reduction of false positives without changing the cancer detection rate.”
Pedemonte S, Tsue T, Mombourquette B, Vu YNT, Matthews T, Morales Hoil R, Shah M, Ghare N, Zingman-Daniels N, Holley S, Appleton CM, Su J, Wahl RL. A Semi-autonomous Deep Learning System to Reduce False-Positives in Screening Mammography. Radiology: Artificial Intelligence. April 10, 2024. DOI: 10.1148/ryai.230033
This work was supported by funding from Whiterabbit.ai Inc.
Washington University has equity interests in Whiterabbit.ai Inc. and may receive royalty income and milestone payments from a “collaboration and license agreement” with Whiterabbit.ai to develop a technology evaluated in this research. Wahl is the principal investigator on a research contract from Whiterabbit.ai. These agreements are managed by the Washington University Institutional Conflict of Interest Committee.
About Washington University School of Medicine
WashU Medicine is a global leader in academic medicine, including biomedical research, patient care and educational programs with 2,900 faculty. Its National Institutes of Health (NIH) research funding portfolio is the second largest among U.S. medical schools and has grown 56% in the last seven years. Together with institutional investment, WashU Medicine commits well over $1 billion annually to basic and clinical research innovation and training. Its faculty practice is consistently within the top five in the country, with more than 1,900 faculty physicians practicing at 130 locations and who are also the medical staffs of Barnes-Jewish and St. Louis Children’s hospitals of BJC HealthCare. WashU Medicine has a storied history in MD/PhD training, recently dedicated $100 million to scholarships and curriculum renewal for its medical students, and is home to top-notch training programs in every medical subspecialty as well as physical therapy, occupational therapy, and audiology and communications sciences.
Originally published on the School of Medicine website
Comments and respectful dialogue are encouraged, but content will be moderated. Please, no personal attacks, obscenity or profanity, selling of commercial products, or endorsements of political candidates or positions. We reserve the right to remove any inappropriate comments. We also cannot address individual medical concerns or provide medical advice in this forum.