{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T12:52:42Z","timestamp":1791550362760,"version":"4.3.4"},"reference-count":40,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T00:00:00Z","timestamp":1791331200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T00:00:00Z","timestamp":1791331200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: Large language models (LLMs) are increasingly used as decision-support tools in radiology, yet their influence on clinical decision-making remains a significant concern. This study evaluates how artificial intelligence (AI)-generated malignancy risk estimates influence clinical decisions in breast imaging and assesses automation bias (AB) among radiology residents at different training levels. Methods: In this simulation-based study, 20 residents reviewed radiology reports for 15 synthetic breast imaging cases generated by ChatGPT-4 Turbo, each presented in either a concordant or discordant format. Participants were stratified into two groups based on their completion of a breast imaging rotation and evaluated each case by assigning a BI-RADS score, recommending management, and rating their confidence in the AI risk estimate. Diagnostic accuracy and independent predictors of AB were analyzed using Generalized Estimating Equations (GEE) to account for reader-level clustering. Results: Overall management accuracy was significantly lower in cases with discordant AI outputs (p = 0.008). A similar, albeit non-significant, decreasing trend was observed for BI-RADS accuracy. While advanced residents maintained high management accuracy regardless of AI concordance, early-stage residents showed a sharp decline in discordant cases (77.3% vs. 45.3%; p = 0.006). This decline was most critical in malignant cases for early-stage residents, where accuracy dropped from 92.0% to 48.0% (p &lt; 0.001). In the multivariable GEE logistic regression model, training level was the only independent predictor of AB (OR: 0.09, 95% CI: 0.03\u20130.28, p &lt; 0.001). Conclusions: In this simulation study, advanced clinical training was associated with lower susceptibility to AB in breast imaging, although training level alone is unlikely to be a sufficient safeguard. Safe integration of AI tools will require caution among junior trainees alongside system-level measures such as model transparency and institutional oversight.<\/jats:p>","DOI":"10.3390\/diagnostics16193241","type":"journal-article","created":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T11:26:53Z","timestamp":1791458813000},"page":"3241","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["The Influence of Large Language Model-Generated Artificial Intelligence Risk Estimates on Clinical Decision-Making in Breast Imaging: A Simulation-Based Study"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8515-281X","authenticated-orcid":false,"given":"Ebru","family":"Dusunceli Atman","sequence":"first","affiliation":[{"name":"Department of Radiology, School of Medicine, Ankara University, 06230 Alt\u0131ndag, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6888-5616","authenticated-orcid":false,"given":"Sena","family":"Bozer Uludag","sequence":"additional","affiliation":[{"name":"Department of Radiology, School of Medicine, Ankara University, 06230 Alt\u0131ndag, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8441-2912","authenticated-orcid":false,"given":"\u00c7a\u011flar","family":"Uzun","sequence":"additional","affiliation":[{"name":"Department of Radiology, School of Medicine, Ankara University, 06230 Alt\u0131ndag, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-7794-6659","authenticated-orcid":false,"given":"Gizem","family":"Agaran","sequence":"additional","affiliation":[{"name":"Department of Radiology, School of Medicine, Ankara University, 06230 Alt\u0131ndag, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7671-5131","authenticated-orcid":false,"given":"Zeynep","family":"Eskalen","sequence":"additional","affiliation":[{"name":"Department of Radiology, School of Medicine, Ankara University, 06230 Alt\u0131ndag, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2820-1384","authenticated-orcid":false,"given":"Kazim Burak","family":"Karaca","sequence":"additional","affiliation":[{"name":"Department of Radiology, Tokat State Hospital, Almus, 60950 Tokat, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"investigation","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,10,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1038\/s41568-018-0016-5","article-title":"Artificial intelligence in radiology","volume":"18","author":"Hosny","year":"2018","journal-title":"Nat. Rev. Cancer"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Papageorgiou, P.S., Christodoulou, R.C., Pitsillos, R., Petrou, V., Vamvouras, G., Kormentza, E.V., Papagelopoulos, P.J., and Georgiou, M.F. (2025). The role of large language models in improving diagnostic-related groups assignment and clinical decision support in healthcare systems: An example from radiology and nuclear medicine. Appl. Sci., 15.","DOI":"10.3390\/app15169005"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"685","DOI":"10.1007\/s11604-024-01552-0","article-title":"The impact of large language models on radiology: A guide for radiologists on the latest innovations in AI","volume":"42","author":"Nakaura","year":"2024","journal-title":"Jpn. J. Radiol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"605","DOI":"10.12669\/pjms.39.2.7653","article-title":"ChatGPT-Reshaping medical education and clinical management","volume":"39","author":"Khan","year":"2023","journal-title":"Pak. J. Med. Sci."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Nasser, M., and Yusof, U.K. (2023). Deep learning based methods for breast cancer diagnosis: A systematic review and future direction. Diagnostics, 13.","DOI":"10.3390\/diagnostics13010161"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hussain, S., Ali, M., Naseem, U., Nezhadmoghadam, F., Jatoi, M.A., Gulliver, T.A., and Tamez-Pe\u00f1a, J.G. (2024). Breast cancer risk prediction using machine learning: A systematic review. Front. Oncol., 14.","DOI":"10.3389\/fonc.2024.1343627"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1148\/radiol.2019192471","article-title":"Harnessing the power of deep learning to assess breast cancer risk","volume":"294","author":"Bahl","year":"2020","journal-title":"Radiology"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"eaba4373","DOI":"10.1126\/scitranslmed.aba4373","article-title":"Toward robust mammography-based models for breast cancer risk","volume":"13","author":"Yala","year":"2021","journal-title":"Sci. Transl. Med."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"e222575","DOI":"10.1148\/radiol.222575","article-title":"Beyond breast density: Risk measures for breast cancer in multiple imaging modalities","volume":"306","author":"Acciavatti","year":"2023","journal-title":"Radiology"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1136\/bmjqs-2018-008370","article-title":"Artificial intelligence, bias and clinical safety","volume":"28","author":"Challen","year":"2019","journal-title":"BMJ Qual. Saf."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"e100081","DOI":"10.1136\/bmjhci-2019-100081","article-title":"Human factors challenges for the safe use of artificial intelligence in patient care","volume":"26","author":"Sujan","year":"2019","journal-title":"BMJ Health Care Inform."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1136\/amiajnl-2011-000089","article-title":"Automation bias: A systematic review of frequency, effect mediators, and mitigators","volume":"19","author":"Goddard","year":"2012","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_14","unstructured":"Newell, M.S., Destounis, S.V., Leung, J.W.T., DeMartini, W.B., Lee, C.H., and Eby, P.R. (2025). ACR BI-RADS\u00aev2025Manual, American College of Radiology."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e222176","DOI":"10.1148\/radiol.222176","article-title":"Automation bias in mammography: The Impact of artificial intelligence BI-RADS suggestions on reader performance","volume":"307","author":"Dratsch","year":"2023","journal-title":"Radiology"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1518\/001872097778543886","article-title":"Humans and Automation: Use, Misuse, Disuse, Abuse","volume":"39","author":"Parasuraman","year":"1997","journal-title":"Hum. Factors"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1038\/s41746-024-01328-w","article-title":"Impact of human and artificial intelligence collaboration on workload reduction in medical image interpretation","volume":"7","author":"Chen","year":"2024","journal-title":"NPJ Digit. Med."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1038\/s41551-021-00751-8","article-title":"Synthetic data in machine learning for medicine and healthcare","volume":"5","author":"Chen","year":"2021","journal-title":"Nat. Biomed. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"e231593","DOI":"10.1148\/radiol.231593","article-title":"Quantitative evaluation of large language models to streamline radiology report impressions: A multimodal retrospective analysis","volume":"310","author":"Doshi","year":"2024","journal-title":"Radiology"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e31284","DOI":"10.2196\/31284","article-title":"Online patient education materials related to lipoprotein(a): Readability assessment","volume":"24","author":"Pearson","year":"2022","journal-title":"J. Med. Internet Res."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e017372","DOI":"10.1161\/JAHA.120.017372","article-title":"Readability of online patient educational materials for coronary artery calcium scans and implications for health disparities","volume":"9","author":"Rodriguez","year":"2020","journal-title":"J. Am. Heart Assoc."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1007\/s00146-025-02422-7","article-title":"Exploring automation bias in human\u2013AI collaboration: A review and implications for explainable AI","volume":"41","author":"Romeo","year":"2025","journal-title":"AI Soc."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"5890","DOI":"10.3402\/meo.v16i0.5890","article-title":"An analysis of clinical reasoning through a recent and comprehensive approach: The dual-process theory","volume":"16","author":"Pelaccia","year":"2011","journal-title":"Med. Educ. Online"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1111\/j.1365-2753.2012.01900.x","article-title":"An integrated model of clinical reasoning: Dual-process theory of cognition and metacognition","volume":"18","author":"Marcum","year":"2012","journal-title":"J. Eval. Clin. Prac."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1007\/s10459-009-9182-2","article-title":"Clinical cognition and diagnostic error: Applications of a dual process model of reasoning","volume":"14","author":"Croskerry","year":"2009","journal-title":"Adv. Health Sci. Educ. Theory Pract."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"C\u00eerlig, I.A., Olaru, A.M., Ene, M.A., Domenco, A.\u0218., Teic\u0103, R.V., Ciofiac, C.M., Nica, R.-E., Novac, V.-M., Mamuleanu, M., and Florescu, L.-M. (2026). The impact of artificial intelligence on BI-RADS classification and diagnostic confidence in mammography interpretation by radiology residents. Life, 16.","DOI":"10.3390\/life16061020"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1287\/isre.2020.0980","article-title":"Augmenting medical diagnosis decisions? An investigation into physicians\u2019 decision-making process with artificial intelligence","volume":"32","author":"Jussupow","year":"2021","journal-title":"Inf. Syst. Res."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1006\/ijhc.1999.0349","article-title":"Accountability and automation bias","volume":"52","author":"Skitka","year":"2000","journal-title":"Int. J. Hum. Comput Stud."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1177\/0018720810376055","article-title":"Complacency and bias in human use of automation: An attentional integration","volume":"52","author":"Parasuraman","year":"2010","journal-title":"Hum. Factors"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1207\/s15327108ijap0801_3","article-title":"Automation bias: Decision making and performance in high-tech cockpits","volume":"81","author":"Mosier","year":"1998","journal-title":"Int. J. Aviat. Psychol."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cummings, M.L. (2004, January 20\u201322). Automation bias in intelligent time critical decision support systems. Proceedings of the AIAA 1st Intelligent Systems Technical Conference, Chicago, IL, USA. AIAA Paper 2004-6313.","DOI":"10.2514\/6.2004-6313"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Lyell, D., Magrabi, F., Raban, M., Pont, L.G., Baysari, M.T., Day, R.O., and Coiera, E. (2017). Automation bias in electronic prescribing. BMC Med. Inform. Decis. Mak., 17.","DOI":"10.1186\/s12911-017-0425-5"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"5415","DOI":"10.1007\/s00330-023-10514-5","article-title":"Reader bias in breast cancer screening related to cancer prevalence and artificial intelligence decision support\u2014A reader study","volume":"34","author":"Janicijevic","year":"2024","journal-title":"Eur. Radiol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1038\/s41746-025-01503-7","article-title":"Bias recognition and mitigation strategies in artificial intelligence healthcare applications","volume":"8","author":"Hasanzadeh","year":"2025","journal-title":"NPJ Digit. Med."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"656","DOI":"10.1518\/001872006779166334","article-title":"Supporting trust calibration and the effective use of decision aids by presenting dynamic system confidence information","volume":"48","author":"McGuirl","year":"2006","journal-title":"Hum. Factors"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1016\/S1071-5819(03)00038-7","article-title":"The role of trust in automation reliance","volume":"58","author":"Dzindolet","year":"2003","journal-title":"Int. J. Hum. Comput Stud."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1038\/s41746-023-00766-2","article-title":"Human visual explanations mitigate bias in AI-based assessment of surgeon skills","volume":"6","author":"Kiyasseh","year":"2023","journal-title":"NPJ Digit. Med."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1038\/s41746-023-00858-z","article-title":"Bias in AI-based models for medical applications: Challenges and mitigation strategies","volume":"6","author":"Mittermaier","year":"2023","journal-title":"NPJ Digit. Med."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"719","DOI":"10.1038\/s41551-023-01056-8","article-title":"Algorithmic fairness in artificial intelligence for medicine and healthcare","volume":"7","author":"Chen","year":"2023","journal-title":"Nat. Biomed. Eng."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"5695","DOI":"10.1007\/s00330-025-11498-0","article-title":"AI and human interactions in prostate cancer diagnosis using MRI","volume":"35","author":"Padhani","year":"2025","journal-title":"Eur. Radiol."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/19\/3241\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T12:10:29Z","timestamp":1791547829000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/19\/3241"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,7]]},"references-count":40,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2026,10]]}},"alternative-id":["diagnostics16193241"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16193241","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,7]]}}}