{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T05:34:56Z","timestamp":1792647296954,"version":"4.5.0"},"reference-count":16,"publisher":"Oxford University Press (OUP)","license":[{"start":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T00:00:00Z","timestamp":1792540800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>To compare the diagnostic performance of five contemporary vision-enabled large language models (LLMs) in mammography interpretation using a case\u2013control benchmark design.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>A total of 200 cases were analyzed using a quota-balanced case\u2013control design (100 malignant, BI-RADS 4\u20135; 100 controls, BI-RADS 1\u20132). All models (ChatGPT-5T, ChatGPT-5I, ChatGPT-4o, Gemini 2.5 Pro, and Gemini 2.5 Flash) received standard mammography images. Models were evaluated for lesion detection, laterality, quadrant localization, lesion type, and malignant\u2013benign discrimination. Performance was assessed using accuracy, sensitivity, specificity, balanced accuracy, Cohen\u2019s \u03ba, and area under the receiver operating characteristic curve (AUC). Breast density was recorded for all cases, while histopathological subtype and lesion size were documented for malignant cases.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Lesion detection accuracy ranged from 79% to 91%, with high malignant sensitivity (92%\u2013100%) but lower specificity in normal cases (24%\u201382%). ChatGPT-5T performed best in laterality (83%; \u03ba = 0.70) and quadrant localization (73%; \u03ba = 0.62). Lesion type classification was modest (74%\u201385%). In malignant\u2013benign discrimination, ChatGPT-5T achieved the highest balanced accuracy (87%), while ChatGPT-4o showed poor specificity (28%). The study cohort was predominantly composed of lower-density breasts, larger lesions, and invasive ductal carcinoma as the most common subtype.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Vision-enabled LLMs demonstrated high sensitivity for breast lesion detection and classification tasks. More deliberative models, particularly ChatGPT-5T, consistently outperformed faster counterparts, especially in spatial reasoning tasks. However, reduced specificity in normal cases underscores that current LLM outputs should remain under radiologist supervision as supportive second-reader tools rather than standalone diagnostic systems. Given the quota-balanced design and cohort composition, results should be interpreted as comparative benchmarking rather than real-world diagnostic performance.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jbi\/wbag038","type":"journal-article","created":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T11:34:49Z","timestamp":1781004889000},"source":"Crossref","is-referenced-by-count":1,"title":["Benchmark Evaluation of Vision-Enabled Large Language Models in Mammography: A Case\u2013Control Study"],"prefix":"10.1093","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2207-5850","authenticated-orcid":false,"given":"Kazim Ayberk","family":"Sinci","sequence":"first","affiliation":[{"name":"Department of Radiology, Kanuni Sultan Suleyman Education and Research Hospital , Istanbul ,","place":["T\u00fcrkiye"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-7863-8927","authenticated-orcid":false,"given":"Mehmet","family":"Aksit","sequence":"additional","affiliation":[{"name":"Department of Radiology, Izmir Katip Celebi University Ataturk Training and Research Hospital , Izmir ,","place":["T\u00fcrkiye"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1225-2526","authenticated-orcid":false,"given":"Merve","family":"Gursoy Bulut","sequence":"additional","affiliation":[{"name":"Department of Radiology, Acibadem Izmir Kent Hospital , Izmir ,","place":["T\u00fcrkiye"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,10,21]]},"reference":[{"issue":"3","key":"2026102112424234800_wbag038-B1","doi-asserted-by":"publisher","first-page":"229","DOI":"10.3322\/caac.21834","article-title":"Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"74","author":"Bray","year":"2024","journal-title":"CA Cancer J Clin"},{"issue":"24","key":"2026102112424234800_wbag038-B2","doi-asserted-by":"publisher","first-page":"2353","DOI":"10.1056\/NEJMsr1504363","article-title":"Breast-cancer screening\u2013viewpoint of the IARC Working Group","volume":"372","author":"Lauby-Secretan","year":"2015","journal-title":"N Engl J Med"},{"issue":"1","key":"2026102112424234800_wbag038-B3","doi-asserted-by":"publisher","first-page":"897","DOI":"10.1186\/s12885-017-3929-6","article-title":"Outcome of breast cancer screening in Denmark","volume":"17","author":"Lynge","year":"2017","journal-title":"BMC Cancer"},{"issue":"9","key":"2026102112424234800_wbag038-B4","doi-asserted-by":"publisher","first-page":"1123","DOI":"10.1016\/s1470-2045(15)00128-x","article-title":"Effect of mammographic screening from age 40 years on breast cancer mortality in the UK Age trial at 17 years\u2019 follow-up: a randomised controlled trial","volume":"16","author":"Moss","year":"2015","journal-title":"Lancet Oncol"},{"key":"2026102112424234800_wbag038-B5","doi-asserted-by":"publisher","first-page":"i2350","DOI":"10.1136\/bmj.i2350","article-title":"Staff shortages are putting UK breast cancer screening \u201cat risk,\u201d survey finds","volume":"353","author":"Gulland","year":"2016","journal-title":"BMJ"},{"issue":"2","key":"2026102112424234800_wbag038-B6","doi-asserted-by":"publisher","first-page":"370","DOI":"10.2214\/ajr.08.1665","article-title":"Workforce shortages in breast imaging: impact on mammography utilization","volume":"192","author":"Wing","year":"2009","journal-title":"AJR Am J Roentgenol"},{"issue":"3","key":"2026102112424234800_wbag038-B7","doi-asserted-by":"publisher","first-page":"209","DOI":"10.3857\/roj.2023.00633","article-title":"Developing prompts from large language model for extracting clinical information from pathology and ultrasound reports in breast cancer","volume":"41","author":"Choi","year":"2023","journal-title":"Radiat Oncol J"},{"issue":"1","key":"2026102112424234800_wbag038-B8","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1186\/s12911-019-0997-3","article-title":"The implementation of natural language processing to extract index lesions from breast magnetic resonance imaging reports","volume":"19","author":"Liu","year":"2019","journal-title":"BMC Med Inform Decis Mak"},{"issue":"1","key":"2026102112424234800_wbag038-B9","doi-asserted-by":"publisher","first-page":"e1","DOI":"10.1016\/j.crad.2023.09.009","article-title":"Automatic classification and prioritisation of actionable BI-RADS categories using natural language processing models","volume":"79","author":"L\u00f3pez-\u00dabeda","year":"2024","journal-title":"Clin Radiol"},{"issue":"10","key":"2026102112424234800_wbag038-B10","doi-asserted-by":"publisher","first-page":"990","DOI":"10.1016\/j.jacr.2023.05.003","article-title":"Evaluating GPT as an Adjunct for Radiologic Decision Making: GPT-4 Versus GPT-3.5 in a Breast Imaging Pilot","volume":"20","author":"Rao","year":"2023","journal-title":"J Am Coll Radiol"},{"issue":"1","key":"2026102112424234800_wbag038-B11","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1007\/s12672-025-02005-4","article-title":"A preliminary investigation into the potential, pitfalls, and limitations of large language models for mammography interpretation","volume":"16","author":"Pesapane","year":"2025","journal-title":"Discov Oncol"},{"key":"2026102112424234800_wbag038-B12","doi-asserted-by":"publisher","DOI":"10.1016\/j.acra.2025.09.008","article-title":"Performance Comparison of Cutting-Edge Large Language Models on the ACR In-Training Examination: An Update for 2025","author":"Young","year":"2025","journal-title":"Acad Radiol"},{"issue":"2","key":"2026102112424234800_wbag038-B13","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.250617","article-title":"One Year On: Assessing Progress of Multimodal Large Language Model Performance on RSNA 2024 Case of the Day Questions","volume":"316","author":"Hou","year":"2025","journal-title":"Radiology"},{"issue":"1","key":"2026102112424234800_wbag038-B14","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.232756","article-title":"Chatbots and Large Language Models in Radiology: A Practical Primer for Clinical and Research Applications","volume":"310","author":"Bhayana","year":"2024","journal-title":"Radiology"},{"issue":"5","key":"2026102112424234800_wbag038-B15","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230582","article-title":"Performance of ChatGPT on a Radiology Board-style Examination: Insights into Current Strengths and Limitations","volume":"307","author":"Bhayana","year":"2023","journal-title":"Radiology"},{"issue":"4","key":"2026102112424234800_wbag038-B16","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.240790","article-title":"Retrieval-Augmented Generation with Large Language Models in Radiology: From Theory to Practice","volume":"7","author":"Fink","year":"2025","journal-title":"Radiol Artif Intell"}],"container-title":["Journal of Breast Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jbi\/advance-article-pdf\/doi\/10.1093\/jbi\/wbag038\/72258069\/wbag038.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jbi\/advance-article-pdf\/doi\/10.1093\/jbi\/wbag038\/72258069\/wbag038.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T16:42:48Z","timestamp":1792600968000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jbi\/advance-article\/doi\/10.1093\/jbi\/wbag038\/8965263"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,21]]},"references-count":16,"URL":"https:\/\/doi.org\/10.1093\/jbi\/wbag038","relation":{},"ISSN":["2631-6110","2631-6129"],"issn-type":[{"value":"2631-6110","type":"print"},{"value":"2631-6129","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,21]]},"article-number":"wbag038"}}