{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T23:40:59Z","timestamp":1777938059307,"version":"3.51.4"},"reference-count":34,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T00:00:00Z","timestamp":1777507200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100020254","name":"Koios Medical, Inc.","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100020254","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: Circumscribed oval, hypoechoic masses are common on screening breast ultrasound (US), and the vast majority are benign. Triple-receptor negative invasive breast cancer (TNBC) can appear similar, resulting in both human and artificial intelligence (AI) interpretive errors. Purpose: We sought to improve AI performance in distinguishing common benign masses from TNBC through a retrospective model refinement and validation study. Materials and Methods: In an Institutional Review Board-approved HIPAA-compliant protocol, from five academic medical centers, orthogonal ultrasound images of 1771 breast masses 2 cm or smaller were acquired, consisting of cysts, complicated cysts, other benign, and malignancies. Cases were randomized, controlling for lesion class, site, and patient age, with 1446 (including 402, 27.8%, malignancies) used for training and 325 (including 95, 29.2% malignancies) for validation using Koios DS\u00ae (decision support, KDS) software version 2.0. A breast imaging radiologist from each center reviewed images and recorded BI-RADS features and assessment. Demographics, symptoms, and pathology or at least one-year follow-up was recorded. The KDS score was evaluated standalone and in combination with BI-RADS using logistic regression and ROC analysis with focus on specificity at sensitivity of 98%. Results: In training, KDS standalone performed comparably to BI-RADS, and significantly improved BI-RADS malignancy risk prediction (p &lt; 0.001). The 98%\u2013sensitivity threshold for combined KDS + BI-RADS was estimated and kept fixed during validation. In validation, KDS standalone performed similar to BI-RADS with AUC = 0.97 (CI: 0.95\u20130.98) versus 0.95 (p = 0.22), with sensitivity of 98% (93\/95, CI: 95\u2013100%) for both and specificity of 70.9% (163\/230, CI: 65.0\u201376.7%) for KDS versus 63.9% for BI-RADS (147\/230, p = 0.10). Combining KDS + BIRADS significantly improved overall performance (AUC 0.98, p &lt; 0.001) and specificity (74.4%, 171\/230, p &lt; 0.001) while maintaining sensitivity at 98% (93\/95). Conclusions: While KDS alone should not replace BI-RADS, when used in combination with BI-RADS, it can significantly improve specificity for highly accurate (98% sensitivity) triaging management of masses representative of those seen on screening US.<\/jats:p>","DOI":"10.3390\/diagnostics16091354","type":"journal-article","created":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T10:47:28Z","timestamp":1777546048000},"page":"1354","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Training AI to Improve Distinction of Triple-Negative Invasive Breast Cancer from Cysts and Fibroadenomas on Ultrasound"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8792-6185","authenticated-orcid":false,"given":"Wendie A.","family":"Berg","sequence":"first","affiliation":[{"name":"Department of Radiology, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andriy I.","family":"Bandos","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Health Data Science, University of Pittsburgh School of Public Health, Pittsburgh, PA 15261, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Linda H.","family":"Larsen","sequence":"additional","affiliation":[{"name":"Department of Radiology, Keck School of Medicine of USC, Los Angeles, CA 90033, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Samantha L.","family":"Heller","sequence":"additional","affiliation":[{"name":"Department of Radiology, NYU Grossman School of Medicine, NYU Langone Health, New York, NY 10016, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Regina J.","family":"Hooley","sequence":"additional","affiliation":[{"name":"Department of Radiology and Biomedical Imaging, Yale School of Medicine, New Haven, CT 06520, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Richard S.","family":"Ha","sequence":"additional","affiliation":[{"name":"Department of Radiology, Columbia University School of Medicine, New York, NY 10032, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maham","family":"Siddique","sequence":"additional","affiliation":[{"name":"Department of Radiology, Columbia University School of Medicine, New York, NY 10032, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jeremy M.","family":"Berg","sequence":"additional","affiliation":[{"name":"Department of Computational and Systems Biology, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7413-179X","authenticated-orcid":false,"given":"Yuying","family":"Cao","sequence":"additional","affiliation":[{"name":"School of Medicine, University of Pittsburgh, Pittsburgh, PA 15213, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-8128-036X","authenticated-orcid":false,"given":"R. Chad","family":"McClennan","sequence":"additional","affiliation":[{"name":"Koios Medical, Inc., Chicago, IL 60654, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ajit","family":"Jairaj","sequence":"additional","affiliation":[{"name":"Koios Medical, Inc., Chicago, IL 60654, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,4,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e242634","DOI":"10.1148\/radiol.242634","article-title":"Screening for Breast Cancer with Contrast-enhanced Mammography as an Alternative to MRI: SCEMAM Trial Results","volume":"315","author":"Berg","year":"2025","journal-title":"Radiology"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1093\/jbi\/wbz055","article-title":"Screening breast ultrasound using hand-held or automated technique in women with dense breasts","volume":"1","author":"Berg","year":"2019","journal-title":"J. Breast Imaging"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1394","DOI":"10.1001\/jama.2012.388","article-title":"Detection of breast cancer with addition of annual screening ultrasound or a single screening MRI to mammography in women with elevated breast cancer risk","volume":"307","author":"Berg","year":"2012","journal-title":"JAMA"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2416","DOI":"10.1200\/JCO.22.01445","article-title":"Prospective Multicenter Diagnostic Performance of Technologist-Performed Screening Breast Ultrasound After Tomosynthesis in Women With Dense Breasts (the DBTUST)","volume":"41","author":"Berg","year":"2023","journal-title":"J. Clin. Oncol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1001\/jama.2020.0572","article-title":"Comparison of Abbreviated Breast MRI vs. Digital Breast Tomosynthesis for Breast Cancer Detection Among Women With Dense Breasts Undergoing Screening","volume":"323","author":"Comstock","year":"2020","journal-title":"JAMA"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1093\/jbi\/wbae019","article-title":"Role of Supplemental Breast MRI in Screening Women with Mammographically Dense Breasts: A Systematic Review and Meta-analysis","volume":"6","author":"Faheem","year":"2024","journal-title":"J. Breast Imaging"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1935","DOI":"10.1016\/S0140-6736(25)00582-3","article-title":"Comparison of supplemental breast cancer imaging techniques-interim results from the BRAID randomised controlled trial","volume":"405","author":"Gilbert","year":"2025","journal-title":"Lancet"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"784","DOI":"10.1016\/S0140-6736(25)02319-0","article-title":"Cumulative incidence of advanced breast cancer in women aged 40\u201349 years in the Japan Strategic Anti-cancer Randomised Trial (J-START) of adjunctive ultrasonography: A prespecified secondary analysis","volume":"407","author":"Suzuki","year":"2026","journal-title":"Lancet"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"e243953","DOI":"10.1148\/radiol.243953","article-title":"Molecular Breast Imaging and Digital Breast Tomosynthesis for Dense Breast Screening: The Density MATTERS Trial","volume":"316","author":"Hruska","year":"2025","journal-title":"Radiology"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1148\/radiol.2016161444","article-title":"Supplemental Breast MR Imaging Screening of Women with Average Risk of Breast Cancer","volume":"283","author":"Kuhl","year":"2017","journal-title":"Radiology"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1148\/radiol.2021203633","article-title":"Supplemental Breast MRI for Women with Extremely Dense Breasts: Results of the Second Screening Round of the DENSE Trial","volume":"299","author":"Veenhuizen","year":"2021","journal-title":"Radiology"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1055\/s-2005-858203","article-title":"The \"EVA\" Trial: Evaluation of the Efficacy of Diagnostic Methods (Mammography, Ultrasound, MRI) in the secondary and tertiary prevention of familial breast cancer. Preliminary results after the first half of the study period","volume":"177","author":"Kuhl","year":"2005","journal-title":"Rofo"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1097\/RLI.0b013e3181f3fcdf","article-title":"Multicenter surveillance of women at high genetic breast cancer risk using mammography, ultrasonography, and contrast-enhanced magnetic resonance imaging (the high breast cancer risk Italian 1 study): Final results","volume":"46","author":"Sardanelli","year":"2011","journal-title":"Investig. Radiol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1148\/radiol.2541090953","article-title":"Reasons women at elevated risk of breast cancer refuse breast MR imaging screening: ACRIN 6666","volume":"254","author":"Berg","year":"2010","journal-title":"Radiology"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"759.e1","DOI":"10.1016\/j.crad.2018.04.002","article-title":"Reasons for (non)participation in supplemental population-based MRI breast screening for women with extremely dense breasts","volume":"73","author":"Bakker","year":"2018","journal-title":"Clin. Radiol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"6334","DOI":"10.1007\/s00330-024-10685-9","article-title":"Re-attendance in supplemental breast MRI screening rounds of the DENSE trial for women with extremely dense breasts","volume":"34","author":"Veenhuizen","year":"2024","journal-title":"Eur. Radiol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1148\/radiol.2019182660","article-title":"Performance of Dual-Energy Contrast-enhanced Digital Mammography for Screening Women at Increased Risk of Breast Cancer","volume":"293","author":"Sung","year":"2019","journal-title":"Radiology"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1177\/08465371251318578","article-title":"Cancers Detected on Supplemental Breast Ultrasound in Women With Dense Breasts: Update From a Canadian Centre","volume":"76","author":"Gordon","year":"2025","journal-title":"Can. Assoc. Radiol. J."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1093\/jbi\/wbab060","article-title":"BI-RADS 3 on Screening Breast Ultrasound: What Is It and What Is the Appropriate Management?","volume":"3","author":"Berg","year":"2021","journal-title":"J. Breast Imaging"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1148\/radiol.2020200037","article-title":"Reducing Unnecessary Biopsy and Follow-up of Benign Cystic Breast Lesions","volume":"295","author":"Berg","year":"2020","journal-title":"Radiology"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1148\/radiol.13122829","article-title":"Probably Benign Lesions at Screening Breast US in a Population with Elevated Risk: Prevalence and Rate of Malignancy in the ACRIN 6666 Trial","volume":"269","author":"Barr","year":"2013","journal-title":"Radiology"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1093\/jbi\/wbae085","article-title":"Triple-Negative Breast Cancer: Radiologic-Pathologic Correlation","volume":"7","author":"Valluri","year":"2025","journal-title":"J. Breast Imaging"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1445","DOI":"10.2214\/AJR.19.21872","article-title":"Should We Ignore, Follow, or Biopsy? Impact of Artificial Intelligence Decision Support on Breast Ultrasound Lesion Assessment","volume":"214","author":"Mango","year":"2020","journal-title":"AJR Am. J. Roentgenol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1007\/s10278-018-0132-5","article-title":"Impact of Data Presentation on Physician Performance Utilizing Artificial Intelligence-Based Computer-Aided Diagnosis and Decision Support Systems","volume":"32","author":"Barinov","year":"2019","journal-title":"J. Digit. Imaging"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1093\/jbi\/wbab013","article-title":"Impact of Original and Artificially Improved AI-Based CADx on Breast US Interpretation","volume":"3","author":"Berg","year":"2021","journal-title":"J. Breast Imaging"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1093\/jbi\/wbad080","article-title":"Artificial Intelligence Decision Support for Triple-Negative Breast Cancers on Ultrasound","volume":"6","author":"Coffey","year":"2024","journal-title":"J. Breast Imaging"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1093\/jbi\/wbaf022","article-title":"Triple-Negative Breast Cancer: Differential Imaging Features Based on Menopausal Status and Race","volume":"7","author":"Kuzmiak","year":"2025","journal-title":"J. Breast Imaging"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1054\/brst.2001.0391","article-title":"The comparative sensitivity of mammography and ultrasound in women with breast symptoms: An age-specific analysis","volume":"11","author":"Houssami","year":"2002","journal-title":"Breast"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1169","DOI":"10.2214\/AJR.12.8842","article-title":"Accuracy and value of breast ultrasound for primary imaging evaluation of symptomatic women 30\u201339 years of age","volume":"199","author":"Lehman","year":"2012","journal-title":"AJR Am. J. Roentgenol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"e223351","DOI":"10.1148\/radiol.223351","article-title":"Toward AI-supported US Triage of Women with Palpable Breast Lumps in a Low-Resource Setting","volume":"307","author":"Berg","year":"2023","journal-title":"Radiology"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1148\/radiol.11110640","article-title":"Shear-wave elastography improves the specificity of breast US: The BE1 multinational study of 939 masses","volume":"262","author":"Berg","year":"2012","journal-title":"Radiology"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3615","DOI":"10.7150\/ijms.64243","article-title":"Role of \u201cStiff Rim\u201d sign obtained by shear wave elastography in diagnosis and guiding therapy of breast cancer","volume":"18","author":"Xu","year":"2021","journal-title":"Int. J. Med. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1148\/radiol.14132443","article-title":"Added value of shear-wave elastography for evaluation of breast masses detected with screening US imaging","volume":"273","author":"Lee","year":"2014","journal-title":"Radiology"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ejca.2021.11.005","article-title":"The potential of combined shear wave and strain elastography to reduce unnecessary biopsies in breast cancer diagnostics\u2014An international, multicentre trial","volume":"161","author":"Golatta","year":"2022","journal-title":"Eur. J. Cancer"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/9\/1354\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T04:25:24Z","timestamp":1777695924000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/9\/1354"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,30]]},"references-count":34,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,5]]}},"alternative-id":["diagnostics16091354"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16091354","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,30]]}}}