{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,31]],"date-time":"2026-10-31T17:27:39Z","timestamp":1793467659272,"version":"4.5.0"},"reference-count":69,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,9,24]],"date-time":"2021-09-24T00:00:00Z","timestamp":1632441600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,9,24]],"date-time":"2021-09-24T00:00:00Z","timestamp":1632441600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100014434","name":"Polish National Agency for Academic Exchange","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100014434","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000002","name":"U.S. Department of Health & Human Services | National Institutes of Health","doi-asserted-by":"publisher","award":["P41EB017183"],"award-info":[{"award-number":["P41EB017183"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"U.S. Department of Health & Human Services | National Institutes of Health","doi-asserted-by":"publisher","award":["R21CA225175"],"award-info":[{"award-number":["R21CA225175"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"name":"U.S. Department of Health & Human Services | National Institutes of Health"},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1922658"],"award-info":[{"award-number":["1922658"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000936","name":"Gordon and Betty Moore Foundation","doi-asserted-by":"publisher","award":["9683"],"award-info":[{"award-number":["9683"]}],"id":[{"id":"10.13039\/100000936","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Nat Commun"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Though consistently shown to detect mammographically occult cancers, breast ultrasound has been noted to have high false-positive rates. In this work, we present an AI system that achieves radiologist-level accuracy in identifying breast cancer in ultrasound images. Developed on 288,767 exams, consisting of 5,442,907 B-mode and Color Doppler images, the AI achieves an area under the receiver operating characteristic curve (AUROC) of 0.976 on a test set consisting of 44,755 exams. In a retrospective reader study, the AI achieves a higher AUROC than the average of ten board-certified breast radiologists (AUROC: 0.962 AI, 0.924\u2009\u00b1\u20090.02 radiologists). With the help of the AI, radiologists decrease their false positive rates by 37.3% and reduce requested biopsies by 27.8%, while maintaining the same level of sensitivity. This highlights the potential of AI in improving the accuracy, consistency, and efficiency of breast ultrasound diagnosis.<\/jats:p>","DOI":"10.1038\/s41467-021-26023-2","type":"journal-article","created":{"date-parts":[[2021,10,12]],"date-time":"2021-10-12T08:05:45Z","timestamp":1634025945000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":259,"title":["Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams"],"prefix":"10.1038","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7726-2514","authenticated-orcid":false,"given":"Yiqiu","family":"Shen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6076-725X","authenticated-orcid":false,"given":"Farah E.","family":"Shamout","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jamie R.","family":"Oliver","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9284-4830","authenticated-orcid":false,"given":"Jan","family":"Witowski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kawshik","family":"Kannan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5117-9651","authenticated-orcid":false,"given":"Jungkyu","family":"Park","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nan","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Connor","family":"Huddleston","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stacey","family":"Wolfson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexandra","family":"Millet","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robin","family":"Ehrenpreis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Divya","family":"Awal","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cathy","family":"Tyma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Naziya","family":"Samreen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yiming","family":"Gao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chloe","family":"Chhor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stacey","family":"Gandhi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cindy","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sheila","family":"Kumari-Subaiya","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cindy","family":"Leonard","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Reyhan","family":"Mohammed","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"Moczulski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jaime","family":"Altabet","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James","family":"Babb","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alana","family":"Lewin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6577-8601","authenticated-orcid":false,"given":"Beatriu","family":"Reig","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Linda","family":"Moy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laura","family":"Heacock","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0549-1446","authenticated-orcid":false,"given":"Krzysztof J.","family":"Geras","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,9,24]]},"reference":[{"key":"26023_CR1","doi-asserted-by":"publisher","first-page":"209","DOI":"10.3322\/caac.21660","volume":"71","author":"H Sung","year":"2021","unstructured":"Sung, H. et al. Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 71, 209\u2013249 (2021).","journal-title":"CA Cancer J. Clin."},{"key":"26023_CR2","doi-asserted-by":"publisher","first-page":"7","DOI":"10.3322\/caac.21654","volume":"71","author":"RL Siegel","year":"2021","unstructured":"Siegel, R. L., Miller, K. D., Fuchs, H. E. & Jemal, A. Cancer statistics, 2021. CA Cancer J. Clin. 71, 7\u201333 (2021).","journal-title":"CA Cancer J. Clin."},{"key":"26023_CR3","doi-asserted-by":"publisher","first-page":"3673","DOI":"10.1002\/cncr.30842","volume":"123","author":"EK Arleo","year":"2017","unstructured":"Arleo, E. K., Hendrick, R. E., Helvie, M. A. & Sickles, E. A. Comparison of recommendations for screening mammography using cisnet models. Cancer 123, 3673\u20133680 (2017).","journal-title":"Cancer"},{"key":"26023_CR4","doi-asserted-by":"publisher","first-page":"879","DOI":"10.1016\/j.rcl.2010.06.002","volume":"48","author":"S Feig","year":"2010","unstructured":"Feig, S. Cost-effectiveness of mammography, MRI, and ultrasonography for breast cancer screening. Radiol. Clin. 48, 879\u2013891 (2010).","journal-title":"Radiol. Clin."},{"key":"26023_CR5","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1148\/radiol.2251011667","volume":"225","author":"TM Kolb","year":"2002","unstructured":"Kolb, T. M., Lichy, J. & Newhouse, J. H. Comparison of the performance of screening mammography, physical examination, and breast us and evaluation of factors that influence them: an analysis of 27,825 patient evaluations. Radiology 225, 165\u2013175 (2002).","journal-title":"Radiology"},{"key":"26023_CR6","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1056\/NEJMoa062790","volume":"356","author":"NF Boyd","year":"2007","unstructured":"Boyd, N. F. et al. Mammographic density and the risk and detection of breast cancer. N. Engl. J. Med. 356, 227\u2013236 (2007).","journal-title":"N. Engl. J. Med."},{"key":"26023_CR7","doi-asserted-by":"publisher","first-page":"djv367","DOI":"10.1093\/jnci\/djv367","volume":"108","author":"WA Berg","year":"2016","unstructured":"Berg, W. A. et al. Ultrasound as the primary screening test for breast cancer: analysis from ACRIN 6666. J. Natl Cancer Inst. 108, djv367 (2016).","journal-title":"J. Natl Cancer Inst."},{"key":"26023_CR8","doi-asserted-by":"publisher","first-page":"887","DOI":"10.7863\/jum.2004.23.7.887","volume":"23","author":"PJ Dempsey","year":"2004","unstructured":"Dempsey, P. J. The history of breast ultrasound. J. Ultrasound Med. 23, 887\u2013894 (2004).","journal-title":"J. Ultrasound Med."},{"key":"26023_CR9","doi-asserted-by":"publisher","first-page":"316","DOI":"10.1148\/radiol.2020201036","volume":"297","author":"M Chung","year":"2020","unstructured":"Chung, M. et al. US as the primary imaging modality in the evaluation of palpable breast masses in breastfeeding women, including those of advanced maternal age. Radiology 297, 316\u2013324 (2020).","journal-title":"Radiology"},{"key":"26023_CR10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1200\/JGO.19.11000","volume":"5","author":"R Sood","year":"2019","unstructured":"Sood, R. et al. Ultrasound for breast cancer detection globally: a systematic review and meta-analysis. J. Global Oncol. 5, 1\u201317 (2019).","journal-title":"J. Global Oncol."},{"key":"26023_CR11","doi-asserted-by":"publisher","first-page":"2151","DOI":"10.1001\/jama.299.18.2151","volume":"299","author":"WA Berg","year":"2008","unstructured":"Berg, W. A. et al. Combined screening with ultrasound and mammography vs mammography alone in women at elevated risk of breast cancer. JAMA 299, 2151\u20132163 (2008).","journal-title":"JAMA"},{"key":"26023_CR12","unstructured":"Sickles, E. A. et al. ACR BI-RADS\u00ae atlas, breast imaging reporting and data system. Reston, VA: American College of Radiology 39\u201348 (2013)."},{"key":"26023_CR13","doi-asserted-by":"publisher","first-page":"177","DOI":"10.2214\/ajr.181.1.1810177","volume":"181","author":"P Crystal","year":"2003","unstructured":"Crystal, P., Strano, S. D., Shcharynski, S. & Koretz, M. J. Using sonography to screen women with mammographically dense breasts. Am. J. Roentgenol. 181, 177\u2013182 (2003).","journal-title":"Am. J. Roentgenol."},{"key":"26023_CR14","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1148\/radiol.2392042127","volume":"239","author":"E Lazarus","year":"2006","unstructured":"Lazarus, E., Mainiero, M. B., Schepps, B., Koelliker, S. L. & Livingston, L. S. BI-RADS lexicon for us and mammography: interobserver variability and positive predictive value. Radiology 239, 385\u2013391 (2006).","journal-title":"Radiology"},{"key":"26023_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12885-019-6169-0","volume":"20","author":"L Yang","year":"2020","unstructured":"Yang, L. et al. Performance of ultrasonography screening for breast cancer: a systematic review and meta-analysis. BMC Cancer 20, 1\u201315 (2020).","journal-title":"BMC Cancer"},{"key":"26023_CR16","doi-asserted-by":"publisher","first-page":"1394","DOI":"10.1001\/jama.2012.388","volume":"307","author":"WA Berg","year":"2012","unstructured":"Berg, W. A. et al. Detection of breast cancer with addition of annual screening ultrasound or a single screening MRI to mammography in women with elevated breast cancer risk. JAMA 307, 1394\u20131404 (2012).","journal-title":"JAMA"},{"key":"26023_CR17","doi-asserted-by":"publisher","first-page":"1021","DOI":"10.1016\/j.ejca.2010.12.002","volume":"47","author":"V Corsetti","year":"2011","unstructured":"Corsetti, V. et al. Evidence of the effect of adjunct ultrasound screening in women with mammography-negative dense breasts: interval breast cancers at 1 year follow-up. Eur. J. Cancer 47, 1021\u20131026 (2011).","journal-title":"Eur. J. Cancer"},{"key":"26023_CR18","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1016\/S0929-6441(08)60005-3","volume":"16","author":"D-R Chen","year":"2008","unstructured":"Chen, D.-R. & Hsiao, Y.-H. Computer-aided diagnosis in breast ultrasound. J. Med. Ultrasound 16, 46\u201356 (2008).","journal-title":"J. Med. Ultrasound"},{"key":"26023_CR19","doi-asserted-by":"publisher","first-page":"928","DOI":"10.1016\/j.acra.2007.04.016","volume":"14","author":"W-C Shen","year":"2007","unstructured":"Shen, W.-C., Chang, R.-F., Moon, W. K., Chou, Y.-H. & Huang, C.-S. Breast ultrasound computer-aided diagnosis using BI-RADS features. Acad. Radiol. 14, 928\u2013939 (2007).","journal-title":"Acad. Radiol."},{"key":"26023_CR20","doi-asserted-by":"crossref","unstructured":"Lee, J.-H. et al. Fourier-based shape feature extraction technique for computer-aided b-mode ultrasound diagnosis of breast tumor. In Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 6551\u20136554 (IEEE, 2012).","DOI":"10.1109\/EMBC.2012.6347495"},{"key":"26023_CR21","doi-asserted-by":"publisher","first-page":"620","DOI":"10.1007\/s10278-012-9499-x","volume":"25","author":"J Ding","year":"2012","unstructured":"Ding, J., Cheng, H.-D., Huang, J., Liu, J. & Zhang, Y. Breast ultrasound image classification based on multiple-instance learning. J. Digital Imaging 25, 620\u2013627 (2012).","journal-title":"J. Digital Imaging"},{"key":"26023_CR22","doi-asserted-by":"publisher","DOI":"10.1155\/2017\/7894705","volume":"2017","author":"L Bing","year":"2017","unstructured":"Bing, L. & Wang, W. Sparse representation based multi-instance learning for breast ultrasound image classification. Comput. Math. Methods Med. 2017, 7894705 https:\/\/doi.org\/10.1155\/2017\/7894705 (2017).","journal-title":"Comput. Math. Methods Med."},{"key":"26023_CR23","doi-asserted-by":"crossref","unstructured":"Prabhakar, T. & Poonguzhali, S. Automatic detection and classification of benign and malignant lesions in breast ultrasound images using texture morphological and fractal features. In\u20092017 10th Biomedical Engineering International Conference (BMEiCON),\u20091\u20135 (IEEE, 2017).","DOI":"10.1109\/BMEiCON.2017.8229114"},{"key":"26023_CR24","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1016\/j.ejrad.2017.07.027","volume":"95","author":"Q Zhang","year":"2017","unstructured":"Zhang, Q., Suo, J., Chang, W., Shi, J. & Chen, M. Dual-modal computer-assisted evaluation of axillary lymph node metastasis in breast cancer patients on both real-time elastography and b-mode ultrasound. Eur. J. Radiol. 95, 66\u201374 (2017).","journal-title":"Eur. J. Radiol."},{"key":"26023_CR25","doi-asserted-by":"publisher","first-page":"300","DOI":"10.2214\/AJR.18.20392","volume":"212","author":"Y Gao","year":"2019","unstructured":"Gao, Y., Geras, K. J., Lewin, A. A. & Moy, L. New frontiers: an update on computer-aided diagnosis for breast imaging in the age of artificial intelligence. Am. J. Roentgenol. 212, 300\u2013307 (2019).","journal-title":"Am. J. Roentgenol."},{"key":"26023_CR26","doi-asserted-by":"publisher","first-page":"246","DOI":"10.1148\/radiol.2019182627","volume":"293","author":"KJ Geras","year":"2019","unstructured":"Geras, K. J., Mann, R. M. & Moy, L. Artificial intelligence for mammography and digital breast tomosynthesis: current concepts and future perspectives. Radiology 293, 246\u2013259 (2019).","journal-title":"Radiology"},{"key":"26023_CR27","doi-asserted-by":"publisher","first-page":"1055","DOI":"10.3390\/diagnostics10121055","volume":"10","author":"T Fujioka","year":"2020","unstructured":"Fujioka, T. et al. The utility of deep learning in breast ultrasonic imaging: a review. Diagnostics 10, 1055 (2020).","journal-title":"Diagnostics"},{"key":"26023_CR28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-016-0001-8","volume":"6","author":"J-Z Cheng","year":"2016","unstructured":"Cheng, J.-Z. et al. Computer-aided diagnosis with deep learning architecture: applications to breast lesions in us images and pulmonary nodules in ct scans. Sci. Rep. 6, 1\u201313 (2016).","journal-title":"Sci. Rep."},{"key":"26023_CR29","doi-asserted-by":"publisher","first-page":"1218","DOI":"10.1109\/JBHI.2017.2731873","volume":"22","author":"MH Yap","year":"2017","unstructured":"Yap, M. H. et al. Automated breast ultrasound lesions detection using convolutional neural networks. IEEE J. Biomed. Health Informatics 22, 1218\u20131226 (2017).","journal-title":"IEEE J. Biomed. Health Informatics"},{"key":"26023_CR30","first-page":"1","volume":"10","author":"W Al-Dhabyani","year":"2019","unstructured":"Al-Dhabyani, W., Gomaa, M., Khaled, H. & Aly, F. Deep learning approaches for data augmentation and classification of breast masses using ultrasound images. Int. J. Adv. Computer Sci. Appl. 10, 1\u201311 (2019).","journal-title":"Int. J. Adv. Computer Sci. Appl."},{"key":"26023_CR31","doi-asserted-by":"publisher","DOI":"10.1186\/s41747-019-0112-7","volume":"3","author":"E Fleury","year":"2019","unstructured":"Fleury, E. & Marcomini, K. Performance of machine learning software to classify breast lesions using BI-RADS radiomic features on ultrasound images. Eur. Radiol. Exp. 3, 34 (2019).","journal-title":"Eur. Radiol. Exp."},{"key":"26023_CR32","doi-asserted-by":"publisher","first-page":"235013","DOI":"10.1088\/1361-6560\/ab5093","volume":"64","author":"H Tanaka","year":"2019","unstructured":"Tanaka, H., Chiu, S.-W., Watanabe, T., Kaoku, S. & Yamaguchi, T. Computer-aided diagnosis system for breast ultrasound images using deep learning. Phys. Med. Biol. 64, 235013 (2019).","journal-title":"Phys. Med. Biol."},{"key":"26023_CR33","doi-asserted-by":"publisher","DOI":"10.1186\/s12880-019-0349-x","volume":"19","author":"Z Cao","year":"2019","unstructured":"Cao, Z., Duan, L., Yang, G., Yue, T. & Chen, Q. An experimental study on breast lesion detection and classification from ultrasound images using deep learning architectures. BMC Med. Imaging 19, 51 (2019).","journal-title":"BMC Med. Imaging"},{"key":"26023_CR34","doi-asserted-by":"publisher","first-page":"7714","DOI":"10.1088\/1361-6560\/aa82ec","volume":"62","author":"S Han","year":"2017","unstructured":"Han, S. et al. A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys. Med. Biol. 62, 7714 (2017).","journal-title":"Phys. Med. Biol."},{"key":"26023_CR35","doi-asserted-by":"crossref","first-page":"20170576","DOI":"10.1259\/bjr.20170576","volume":"91","author":"AS Becker","year":"2018","unstructured":"Becker, A. S. et al. Classification of breast cancer in ultrasound imaging using a generic deep learning analysis software: a pilot study. Br. J. Radiol. 91, 20170576 (2018).","journal-title":"Br. J. Radiol."},{"key":"26023_CR36","doi-asserted-by":"publisher","DOI":"10.1155\/2018\/4605191","volume":"2018","author":"T Xiao","year":"2018","unstructured":"Xiao, T. et al. Comparison of transferred deep neural networks in ultrasonic breast masses discrimination. BioMed Res. Int. 2018, 4605191 https:\/\/doi.org\/10.1155\/2018\/4605191 (2018).","journal-title":"BioMed Res. Int."},{"key":"26023_CR37","doi-asserted-by":"crossref","unstructured":"Oquab, M., Bottou, L., Laptev, I. & Sivic, J. Is object localization for free?-weakly-supervised learning with convolutional neural networks. In\u2009Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,\u2009685\u2013694 (2015).","DOI":"10.1109\/CVPR.2015.7298668"},{"key":"26023_CR38","doi-asserted-by":"crossref","unstructured":"Zhou, B., Khosla, A., Lapedriza, A., Oliva, A. & Torralba, A. Learning deep features for discriminative localization. In\u2009Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,\u20092921\u20132929 (2016).","DOI":"10.1109\/CVPR.2016.319"},{"key":"26023_CR39","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1093\/nsr\/nwx106","volume":"5","author":"Z-H Zhou","year":"2018","unstructured":"Zhou, Z.-H. A brief introduction to weakly supervised learning. Natl Sci. Rev. 5, 44\u201353 (2018).","journal-title":"Natl Sci. Rev."},{"key":"26023_CR40","doi-asserted-by":"publisher","first-page":"104863","DOI":"10.1016\/j.dib.2019.104863","volume":"28","author":"W Al-Dhabyani","year":"2020","unstructured":"Al-Dhabyani, W., Gomaa, M., Khaled, H. & Fahmy, A. Dataset of breast ultrasound images. Data in Brief 28, 104863 (2020).","journal-title":"Data in Brief"},{"key":"26023_CR41","unstructured":"Shamout, F. et al. The NYU breast ultrasound dataset v1.0. Tech. Rep. (2021). Available at https:\/\/cs.nyu.edu\/~kgeras\/reports\/ultrasound_datav1.0.pdf."},{"key":"26023_CR42","first-page":"10040","volume":"8","author":"H-Y Du","year":"2015","unstructured":"Du, H.-Y., Lin, B.-R. & Huang, D.-P. Ultrasonographic findings of triple-negative breast cancer. Int. J. Clin. Exp. Med. 8, 10040 (2015).","journal-title":"Int. J. Clin. Exp. Med."},{"key":"26023_CR43","doi-asserted-by":"publisher","first-page":"5458","DOI":"10.1007\/s00330-019-06118-7","volume":"29","author":"A Ciritsis","year":"2019","unstructured":"Ciritsis, A. et al. Automatic classification of ultrasound breast lesions using a deep convolutional neural network mimicking human decision-making. Eur. Radiol. 29, 5458\u20135468 (2019).","journal-title":"Eur. Radiol."},{"key":"26023_CR44","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1054\/brst.2001.0391","volume":"11","author":"N Houssami","year":"2002","unstructured":"Houssami, N., Ciatto, S., Irwig, L., Simpson, J. & Macaskill, P. The comparative sensitivity of mammography and ultrasound in women with breast symptoms: an age-specific analysis. Breast 11, 125\u2013130 (2002).","journal-title":"Breast"},{"key":"26023_CR45","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1109\/TPAMI.2018.2798607","volume":"41","author":"T Baltru\u0161aitis","year":"2018","unstructured":"Baltru\u0161aitis, T., Ahuja, C. & Morency, L.-P. Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intelligence 41, 423\u2013443 (2018).","journal-title":"IEEE Trans. Pattern Anal. Mach. Intelligence"},{"key":"26023_CR46","doi-asserted-by":"publisher","first-page":"100004","DOI":"10.1016\/j.array.2019.100004","volume":"3","author":"T Zhou","year":"2019","unstructured":"Zhou, T., Ruan, S. & Canu, S. A review: deep learning for medical image segmentation using multi-modality fusion. Array 3, 100004 (2019).","journal-title":"Array"},{"key":"26023_CR47","doi-asserted-by":"publisher","first-page":"408","DOI":"10.1007\/s10278-018-0132-5","volume":"32","author":"L Barinov","year":"2019","unstructured":"Barinov, L. et al. Impact of data presentation on physician performance utilizing artificial intelligence-based computer-aided diagnosis and decision support systems. J. Digital Imaging 32, 408\u2013416 (2019).","journal-title":"J. Digital Imaging"},{"key":"26023_CR48","doi-asserted-by":"crossref","unstructured":"Dong, F. et al. One step further into the blackbox: a pilot study of how to build more confidence around an ai-based decision system of breast nodule assessment in 2d ultrasound. Eur. Radiol. 1\u201310 (2021).","DOI":"10.1007\/s00330-020-07561-7"},{"key":"26023_CR49","doi-asserted-by":"publisher","first-page":"e035757","DOI":"10.1136\/bmjopen-2019-035757","volume":"10","author":"C Zhao","year":"2020","unstructured":"Zhao, C. et al. Reducing the number of unnecessary biopsies of US-BI-RADS 4a lesions through a deep learning method for residents-in-training: a cross-sectional study. BMJ Open 10, e035757 (2020).","journal-title":"BMJ Open"},{"key":"26023_CR50","doi-asserted-by":"publisher","first-page":"466","DOI":"10.1007\/s11604-019-00831-5","volume":"37","author":"T Fujioka","year":"2019","unstructured":"Fujioka, T. et al. Distinction between benign and malignant breast masses at breast ultrasound using deep learning method with convolutional neural network. Jpn. J. Radiol. 37, 466\u2013472 (2019).","journal-title":"Jpn. J. Radiol."},{"key":"26023_CR51","doi-asserted-by":"publisher","first-page":"1445","DOI":"10.2214\/AJR.19.21872","volume":"214","author":"VL Mango","year":"2020","unstructured":"Mango, V. L., Sun, M., Wynn, R. T. & Ha, R. Should we ignore, follow, or biopsy? impact of artificial intelligence decision support on breast ultrasound lesion assessment. Am. J. Roentgenol. 214, 1445\u20131452 (2020).","journal-title":"Am. J. Roentgenol."},{"key":"26023_CR52","doi-asserted-by":"publisher","first-page":"522","DOI":"10.1038\/s41551-021-00711-2","volume":"5","author":"X Qian","year":"2021","unstructured":"Qian, X. et al. Prospective assessment of breast cancer risk from multimodal multiview ultrasound images via clinically applicable deep learning. Nat. Biomed. Eng. 5, 522\u2013532 (2021).","journal-title":"Nat. Biomed. Eng."},{"key":"26023_CR53","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1161\/CIRCULATIONAHA.114.014508","volume":"131","author":"GS Collins","year":"2015","unstructured":"Collins, G. S., Reitsma, J. B., Altman, D. G. & Moons, K. G. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (tripod) the tripod statement. Circulation 131, 211\u2013219 (2015).","journal-title":"Circulation"},{"key":"26023_CR54","first-page":"1995","volume":"3361","author":"Y LeCun","year":"1995","unstructured":"LeCun, Y. & Bengio, Y. et al. Convolutional networks for images, speech, and time series. Handbook of Brain Theory and Neural Networks 3361, 1995 (1995).","journal-title":"Handbook of Brain Theory and Neural Networks"},{"key":"26023_CR55","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S. & Sun, J. Identity mappings in deep residual networks. In\u2009European Conference on Computer Vision,\u2009630\u2013645 (Springer, 2016).","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"26023_CR56","unstructured":"Ilse, M., Tomczak, J. M. & Welling, M. Attention-based deep multiple instance learning. Proceedings of the 35th International Conference on Machine Learning, In Proceedings of Machine Learning Research (eds Dy, J. & Krause, A.) 80, 2127\u20132136 (PMLR, 2018)."},{"key":"26023_CR57","doi-asserted-by":"crossref","unstructured":"Shen, Y. et al. Globally-aware multiple instance classifier for breast cancer screening. In\u2009International Workshop on Machine Learning in Medical Imaging,\u200918\u201326 (Springer, 2019).","DOI":"10.1007\/978-3-030-32692-0_3"},{"key":"26023_CR58","doi-asserted-by":"publisher","first-page":"101908","DOI":"10.1016\/j.media.2020.101908","volume":"68","author":"Y Shen","year":"2021","unstructured":"Shen, Y. et al. An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization. Med. Image Anal. 68, 101908 (2021).","journal-title":"Med. Image Anal."},{"key":"26023_CR59","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-021-00453-0","volume":"4","author":"FE Shamout","year":"2021","unstructured":"Shamout, F. E. et al. An artificial intelligence system for predicting the deterioration of covid-19 patients in the emergency department. NPJ Digital Med. 4, 1\u201311 (2021).","journal-title":"NPJ Digital Med."},{"key":"26023_CR60","unstructured":"Kingma, D. P. & Ba, J. Adam: A method for stochastic optimization. Preprint at https:\/\/arxiv.org\/abs\/1412.6980 (2014)."},{"key":"26023_CR61","doi-asserted-by":"crossref","unstructured":"Caruana, R. Multitask learning: a knowledge-based source of inductive bias. In Proceedings of the Tenth International Conference on Machine Learning 41\u201348 (1993).","DOI":"10.1016\/B978-1-55860-307-3.50012-5"},{"key":"26023_CR62","first-page":"2","volume":"13","author":"J Bergstra","year":"2012","unstructured":"Bergstra, J. & Bengio, Y. Random search for hyper-parameter optimization. J. Mach. Learning Res. 13, 2 (2012).","journal-title":"J. Mach. Learning Res."},{"key":"26023_CR63","doi-asserted-by":"crossref","unstructured":"Dietterich, T. G. Ensemble methods in machine learning. In\u2009International Workshop on Multiple Classifier Systems,\u20091\u201315 (Springer, 2000).","DOI":"10.1007\/3-540-45014-9_1"},{"key":"26023_CR64","unstructured":"Shanmugam, D., Blalock, D., Balakrishnan, G. & Guttag, J. When and why test-time augmentation works. Preprint at https:\/\/arxiv.org\/abs\/2011.11156 (2020)."},{"key":"26023_CR65","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1111\/j.1466-8238.2007.00358.x","volume":"17","author":"JM Lobo","year":"2008","unstructured":"Lobo, J. M., Jim\u00e9nez-Valverde, A. & Real, R. Auc: a misleading measure of the performance of predictive distribution models. Global Ecol. Biogeography 17, 145\u2013151 (2008).","journal-title":"Global Ecol. Biogeography"},{"key":"26023_CR66","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F. et al. Scikit-learn: Machine learning in python. J. Mach. Learning Res. 12, 2825\u20132830 (2011).","journal-title":"J. Mach. Learning Res."},{"key":"26023_CR67","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1111\/1467-9639.00050","volume":"23","author":"RW Johnson","year":"2001","unstructured":"Johnson, R. W. An introduction to the bootstrap. Teaching Stat. 23, 49\u201354 (2001).","journal-title":"Teaching Stat."},{"key":"26023_CR68","unstructured":"Chihara, L. & Hesterberg, T.\u2009Mathematical Statistics with Resampling and R\u2009(Wiley Online Library, 2011)."},{"key":"26023_CR69","doi-asserted-by":"publisher","first-page":"409","DOI":"10.1016\/S0033-8389(01)00017-3","volume":"40","author":"L Liberman","year":"2002","unstructured":"Liberman, L. & Menell, J. H. Breast imaging reporting and data system (BI-RADS). Radiol. Clin. 40, 409\u2013430 (2002).","journal-title":"Radiol. Clin."}],"container-title":["Nature Communications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41467-021-26023-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41467-021-26023-2","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41467-021-26023-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,9]],"date-time":"2024-09-09T17:10:12Z","timestamp":1725901812000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41467-021-26023-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,24]]},"references-count":69,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["26023"],"URL":"https:\/\/doi.org\/10.1038\/s41467-021-26023-2","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.04.28.21256203","asserted-by":"object"}]},"ISSN":["2041-1723"],"issn-type":[{"value":"2041-1723","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,24]]},"assertion":[{"value":"22 July 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 September 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 September 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"5645"}}