{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T13:07:23Z","timestamp":1779196043509,"version":"3.51.4"},"reference-count":16,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T00:00:00Z","timestamp":1779148800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background: Artificial intelligence (AI) has shown promise in breast ultrasound image analysis, but most evidence still comes from single-dataset studies. Clinical translation requires evaluation under heterogeneous acquisition and curation conditions. This study presents a patient-leakage-aware, reproducible benchmark for breast ultrasound AI under dataset shift, with emphasis on external generalization, calibration, and confidence-related behavior. Methods: A reproducible benchmark framework was developed using patient-level splitting, internal testing, pairwise cross-dataset evaluation, whole-image and region-of-interest (ROI) input strategies, calibration analysis, targeted ROI-margin sensitivity analysis, representative explainable AI visualization, and an auxiliary lesion-versus-normal confidence-based analysis. Four public breast ultrasound datasets (BUSI, BUS-UCLM, BUS-BRA, and BrEaST) were harmonized for a primary benign-versus-malignant lesion classification task. Normal images were excluded from the primary endpoint and used only in auxiliary analyses when sufficient numbers were available. Results: Cross-dataset testing was weaker on average than internal testing, with mean raw AUROC decreasing from 0.801 to 0.719 and mean balanced accuracy from 0.723 to 0.635. ROI input improved external performance, especially for the vision transformer, increasing mean external AUROC from 0.666 to 0.805 and mean external balanced accuracy from 0.594 to 0.713 relative to whole-image input. Temperature scaling improved calibration-related metrics, reducing mean external expected calibration error from 0.180 to 0.150 and mean external negative log-likelihood from 0.848 to 0.682. Conclusions: This study establishes a reproducible benchmark for evaluating breast ultrasound AI under dataset shift, with explicit attention to patient-level leakage control, external validity, and reliability of predicted probabilities.<\/jats:p>","DOI":"10.3390\/diagnostics16101537","type":"journal-article","created":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T11:53:10Z","timestamp":1779191590000},"page":"1537","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Breast Ultrasound AI Under Dataset Shift: A Patient-Leakage-Aware Benchmark"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7466-9522","authenticated-orcid":false,"given":"Lulu","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Engineering, Reykjavik University, 102 Reykjavik, Iceland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,5,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Bunnell, A., Valdez, D., Strand, F., Glaser, Y., Sadowski, P., and Shepherd, J.A. (2025). Artificial intelligence-enhanced handheld breast ultrasound for screening: A systematic review of diagnostic test accuracy. PLoS Digit. Health, 4.","DOI":"10.1371\/journal.pdig.0001019"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"299","DOI":"10.4048\/jbc.2025.0123","article-title":"The Clinical Application of Artificial Intelligence in Breast Imaging: Current Insights, Challenges, and Future Directions","volume":"28","author":"Sohn","year":"2025","journal-title":"J. Breast Cancer"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.semcancer.2023.09.001","article-title":"Recent advancements in artificial intelligence for breast cancer: Image augmentation, segmentation, diagnosis, and prognosis approaches","volume":"96","author":"Zhang","year":"2023","journal-title":"Semin. Cancer Biol."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Liu, J., Pian, L., Chen, J., Zhao, J., Liu, Y., Meng, F., and Zeng, C. (2025). Artificial intelligence in breast ultrasound: A systematic review of research advances. Front. Oncol., 15.","DOI":"10.3389\/fonc.2025.1619364"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"22754","DOI":"10.1038\/s41598-024-74316-5","article-title":"Role of inter- and extra-lesion tissue, transfer learning, and fine-tuning in the robust classification of breast lesions","volume":"14","author":"Nastase","year":"2024","journal-title":"Sci. Rep."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6840","DOI":"10.1109\/JBHI.2024.3454000","article-title":"A Multi-Task Transformer with Local-Global Feature Interaction and Multiple Tumoral Region Guidance for Breast Cancer Diagnosis","volume":"28","author":"Zhang","year":"2024","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1111\/1754-9485.13612","article-title":"Developing, purchasing, implementing and monitoring AI tools in radiology: Practical considerations. A multi-society statement from the ACR, CAR, ESR, RANZCR & RSNA","volume":"68","author":"Brady","year":"2024","journal-title":"J. Med. Imaging Radiat. Oncol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1016\/j.arthro.2024.01.043","article-title":"Medical Imaging Applications Developed Using Artificial Intelligence Demonstrate High Internal Validity Yet Are Limited in Scope and Lack External Validation","volume":"41","author":"Oeding","year":"2025","journal-title":"Arthroscopy"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3110","DOI":"10.1002\/mp.16812","article-title":"BUS-BRA: A breast ultrasound dataset for assessing computer-aided diagnosis systems","volume":"51","author":"Pereira","year":"2024","journal-title":"Med. Phys."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1038\/s41597-024-02984-z","article-title":"Curated benchmark dataset for ultrasound based breast lesion analysis","volume":"11","author":"Domalik","year":"2024","journal-title":"Sci. Data"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1038\/s41597-025-04562-3","article-title":"BUS-UCLM: Breast ultrasound lesion segmentation dataset","volume":"12","author":"Bueno","year":"2025","journal-title":"Sci. Data"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"104863","DOI":"10.1016\/j.dib.2019.104863","article-title":"Dataset of breast ultrasound images","volume":"28","author":"Gomaa","year":"2020","journal-title":"Data Brief."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1038\/s41746-025-02162-4","article-title":"On the public dissemination and open sourcing of ultrasound resources, datasets and deep learning models","volume":"8","author":"Alsharid","year":"2025","journal-title":"npj Digit. Med."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"102830","DOI":"10.1016\/j.artmed.2024.102830","article-title":"Trustworthy clinical AI solutions: A unified review of uncertainty quantification in deep learning models for medical image analysis","volume":"151","author":"Lambert","year":"2024","journal-title":"Artif. Intell. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"103223","DOI":"10.1016\/j.media.2024.103223","article-title":"A review of uncertainty quantification in medical image analysis: Probabilistic and non-probabilistic methods","volume":"97","author":"Huang","year":"2024","journal-title":"Med. Image Anal."},{"key":"ref_16","first-page":"ubae006","article-title":"AI and machine learning in medical imaging: Key points from development to translation","volume":"1","author":"Samala","year":"2024","journal-title":"BJR Artif. Intell."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/10\/1537\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T12:16:56Z","timestamp":1779193016000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/10\/1537"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,19]]},"references-count":16,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2026,5]]}},"alternative-id":["diagnostics16101537"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16101537","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,19]]}}}