{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T19:31:19Z","timestamp":1787859079519,"version":"build-2784847793"},"reference-count":25,"publisher":"Public Library of Science (PLoS)","issue":"6","license":[{"start":{"date-parts":[[2025,6,26]],"date-time":"2025-06-26T00:00:00Z","timestamp":1750896000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["HRD-2222336"],"award-info":[{"award-number":["HRD-2222336"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006510","name":"Duke University","doi-asserted-by":"publisher","award":["MedX Internal Grant"],"award-info":[{"award-number":["MedX Internal Grant"]}],"id":[{"id":"10.13039\/100006510","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>An external validation of IAIA-BL\u2014a deep-learning based, inherently interpretable breast lesion malignancy prediction model\u2014was performed on two patient populations: 207 women ages 31 to 96, (425 mammograms) from iCAD, and 58 women (104 mammograms) from Emory University. This is the first external validation of an inherently interpretable, deep learning-based lesion classification model. IAIA-BL and black-box baseline models had lower mass margin classification performance on the external datasets than the internal dataset as measured by AUC. These losses correlated with a smaller reduction in malignancy classification performance, though AUC 95% confidence intervals overlapped for all sites. However, interpretability, as measured by model activation on relevant portions of the lesion, was maintained across all populations. Together, these results show that model interpretability can generalize even when performance does not.<\/jats:p>","DOI":"10.1371\/journal.pone.0320091","type":"journal-article","created":{"date-parts":[[2025,6,26]],"date-time":"2025-06-26T17:31:08Z","timestamp":1750959068000},"page":"e0320091","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":3,"title":["Multi-site validation of an interpretable model to analyze breast masses"],"prefix":"10.1371","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2677-0014","authenticated-orcid":true,"given":"Luke","family":"Moffett","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8247-4725","authenticated-orcid":true,"given":"Alina Jade","family":"Barnett","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3971-1075","authenticated-orcid":true,"given":"Jon","family":"Donnelly","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3598-7082","authenticated-orcid":true,"given":"Fides Regina","family":"Schwartz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hari","family":"Trivedi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joseph","family":"Lo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cynthia","family":"Rudin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,6,26]]},"reference":[{"issue":"1","key":"pone.0320091.ref001","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1186\/s13244-023-01541-3","article-title":"Developing, purchasing, implementing and monitoring AI tools in radiology: practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA","volume":"15","author":"AP Brady","year":"2024","journal-title":"Insights Imaging"},{"key":"pone.0320091.ref002","doi-asserted-by":"crossref","first-page":"106554","DOI":"10.1016\/j.compbiomed.2023.106554","article-title":"Computer-aided breast cancer detection and classification in mammography: a comprehensive review","volume":"153","author":"K Loizidou","year":"2023","journal-title":"Comput Biol Med"},{"key":"pone.0320091.ref003","unstructured":"Barnett AJ, Schwartz FR, Tao C, Chen C, Ren Y, Lo JY, et al. Interpretable mammographic image classification using case-based reasoning and deep learning. In: IJCAI-21 Workshop on Deep Learning, Case-Based Reasoning, and AutoML: Present and Future Synergies. 2021."},{"issue":"12","key":"pone.0320091.ref004","doi-asserted-by":"crossref","first-page":"1061","DOI":"10.1038\/s42256-021-00423-x","article-title":"A case-based interpretable deep learning model for classification of mass lesions in digital mammography","volume":"3","author":"AJ Barnett","year":"2021","journal-title":"Nat Mach Intell"},{"issue":"3","key":"pone.0320091.ref005","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1016\/j.diii.2017.01.001","article-title":"BI-RADS\u00ae fifth edition: a summary of changes","volume":"98","author":"DA Spak","year":"2017","journal-title":"Diagn Interv Imaging"},{"key":"pone.0320091.ref006","unstructured":"Chen C, Li O, Tao D, Barnett AJ, Rudin C, Su JK. This looks like that: deep learning for interpretable image recognition. In: Advances in Neural Information Processing Systems 32. 2019."},{"key":"pone.0320091.ref007","doi-asserted-by":"crossref","unstructured":"Li O, Liu H, Chen C, Rudin C. Deep learning for case-based reasoning through prototypes: a neural network that explains its predictions. In: Proceedings of the AAAI Conference on Artificial Intelligence. New Orleans, LA: AAAI Press; 2018.","DOI":"10.1609\/aaai.v32i1.11771"},{"issue":"1","key":"pone.0320091.ref008","doi-asserted-by":"crossref","DOI":"10.1148\/ryai.220047","article-title":"The EMory BrEast imaging Dataset (EMBED): a racially diverse, granular dataset of 3.4 million screening and diagnostic mammographic images","volume":"5","author":"JJ Jeong","year":"2023","journal-title":"Radiol Artif Intell"},{"key":"pone.0320091.ref009","article-title":"Interpretable mammographic image classification using case-based reasoning and deep learning.","author":"AJ Barnett","year":"2021"},{"issue":"12","key":"pone.0320091.ref010","doi-asserted-by":"crossref","first-page":"2860","DOI":"10.1049\/iet-ipr.2020.0070","article-title":"Deep learning algorithm for breast masses classification in mammograms","volume":"14","author":"VS Gnanasekaran","year":"2020","journal-title":"IET Image Process"},{"key":"pone.0320091.ref011","doi-asserted-by":"crossref","first-page":"105584","DOI":"10.1016\/j.cmpb.2020.105584","article-title":"Evaluation of deep learning detection and classification towards computer-aided diagnosis of breast lesions in digital X-ray mammograms","volume":"196","author":"MA Al-Antari","year":"2020","journal-title":"Comput Methods Programs Biomed"},{"key":"pone.0320091.ref012","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.cmpb.2018.01.017","article-title":"Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system","volume":"157","author":"MA Al-Masni","year":"2018","journal-title":"Comput Methods Programs Biomed"},{"key":"pone.0320091.ref013","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.eswa.2015.10.015","article-title":"Breast cancer classification using deep belief networks","volume":"46","author":"AM Abdel-Zaher","year":"2016","journal-title":"Expert Syst Appl"},{"key":"pone.0320091.ref014","article-title":"Grad-CAM: why did you say that?","author":"RR Selvaraju","year":"2016"},{"key":"pone.0320091.ref015","doi-asserted-by":"crossref","unstructured":"Chattopadhay A, Sarkar A, Howlader P, Balasubramanian VN. Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA. 2018, pp. 839\u201347. doi: 10.1109\/WACV.2018.00097","DOI":"10.1109\/WACV.2018.00097"},{"issue":"5","key":"pone.0320091.ref016","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"C Rudin","year":"2019","journal-title":"Nat Mach Intell"},{"key":"pone.0320091.ref017","unstructured":"Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B. Sanity checks for saliency maps. In: Advances in Neural Information Processing Systems 31. 2018."},{"issue":"6","key":"pone.0320091.ref018","doi-asserted-by":"crossref","DOI":"10.1148\/ryai.2021200267","article-title":"Assessing the trustworthiness of saliency maps for localizing abnormalities in medical imaging","volume":"3","author":"N Arun","year":"2021","journal-title":"Radiol Artif Intell"},{"key":"pone.0320091.ref019","article-title":"Very deep convolutional networks for large-scale image recognition.","author":"K Simonyan","year":"2014"},{"key":"pone.0320091.ref020","doi-asserted-by":"crossref","unstructured":"Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248\u201355.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"pone.0320091.ref021","unstructured":"Koh PW, Sagawa S, Marklund H, Xie SM, Zhang M, Balsubramani A, et al. Wilds: A benchmark of in-the-wild distribution shifts. In: International Conference on Machine Learning. 2021, pp. 5637\u201364."},{"key":"pone.0320091.ref022","first-page":"27","article-title":"Google\u2019s medical AI was super accurate in a lab. Real life was a different story.","volume":"4","author":"s Heaven WD o u g l a","year":"2020","journal-title":"MIT Technology Review"},{"issue":"1","key":"pone.0320091.ref023","doi-asserted-by":"crossref","first-page":"3673","DOI":"10.1038\/s41467-020-17478-w","article-title":"Causality matters in medical imaging","volume":"11","author":"DC Castro","year":"2020","journal-title":"Nat Commun"},{"issue":"4","key":"pone.0320091.ref024","doi-asserted-by":"crossref","first-page":"773","DOI":"10.2214\/ajr.166.4.8610547","article-title":"Breast imaging reporting and data system standardized mammography lexicon: observer variability in lesion description","volume":"166","author":"JA Baker","year":"1996","journal-title":"AJR Am J Roentgenol"},{"key":"pone.0320091.ref025","unstructured":"Wiles O, Gowal S, Stimberg F, Rebuffi S-A, Ktena I, Dj Dvijotham K, et al. A fine-grained analysis on distribution shift. arXiv, preprint, 2022."}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0320091","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,26]],"date-time":"2025-06-26T17:31:16Z","timestamp":1750959076000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0320091"}},"subtitle":[],"editor":[{"given":"Pascal A. T.","family":"Baltzer","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,6,26]]},"references-count":25,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6,26]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0320091","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,26]]}}}