{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T14:57:07Z","timestamp":1782313027615,"version":"3.54.5"},"reference-count":49,"publisher":"Public Library of Science (PLoS)","issue":"3","license":[{"start":{"date-parts":[[2022,3,24]],"date-time":"2022-03-24T00:00:00Z","timestamp":1648080000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Wellness Open Living Labs, LLC"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:sec id=\"sec001\">\n<jats:title>Objectives<\/jats:title>\n<jats:p>The objective of this study was to develop and validate a state-of-the-art, deep learning (DL)-based model for detecting breast cancers on mammography.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec002\">\n<jats:title>Methods<\/jats:title>\n<jats:p>Mammograms in a hospital development dataset, a hospital test dataset, and a clinic test dataset were retrospectively collected from January 2006 through December 2017 in Osaka City University Hospital and Medcity21 Clinic. The hospital development dataset and a publicly available digital database for screening mammography (DDSM) dataset were used to train and to validate the RetinaNet, one type of DL-based model, with five-fold cross-validation. The model\u2019s sensitivity and mean false positive indications per image (mFPI) and partial area under the curve (AUC) with 1.0 mFPI for both test datasets were externally assessed with the test datasets.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec003\">\n<jats:title>Results<\/jats:title>\n<jats:p>The hospital development dataset, hospital test dataset, clinic test dataset, and DDSM development dataset included a total of 3179 images (1448 malignant images), 491 images (225 malignant images), 2821 images (37 malignant images), and 1457 malignant images, respectively. The proposed model detected all cancers with a 0.45\u20130.47 mFPI and had partial AUCs of 0.93 in both test datasets.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec004\">\n<jats:title>Conclusions<\/jats:title>\n<jats:p>The DL-based model developed for this study was able to detect all breast cancers with a very low mFPI. Our DL-based model achieved the highest performance to date, which might lead to improved diagnosis for breast cancer.<\/jats:p>\n<\/jats:sec>","DOI":"10.1371\/journal.pone.0265751","type":"journal-article","created":{"date-parts":[[2022,3,24]],"date-time":"2022-03-24T21:33:32Z","timestamp":1648157612000},"page":"e0265751","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":25,"title":["Development and validation of a deep learning model for detection of breast cancers in mammography from multi-institutional datasets"],"prefix":"10.1371","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3878-3616","authenticated-orcid":true,"given":"Daiju","family":"Ueda","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Akira","family":"Yamamoto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Naoyoshi","family":"Onoda","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tsutomu","family":"Takashima","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8150-5152","authenticated-orcid":true,"given":"Satoru","family":"Noda","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shinichiro","family":"Kashiwagi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tamami","family":"Morisaki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shinya","family":"Fukumoto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Masatsugu","family":"Shiba","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mina","family":"Morimura","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5847-6587","authenticated-orcid":true,"given":"Taro","family":"Shimono","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ken","family":"Kageyama","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8050-4895","authenticated-orcid":true,"given":"Hiroyuki","family":"Tatekawa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2738-8781","authenticated-orcid":true,"given":"Kazuki","family":"Murai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7961-1266","authenticated-orcid":true,"given":"Takashi","family":"Honjo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Akitoshi","family":"Shimazaki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daijiro","family":"Kabata","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yukio","family":"Miki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2022,3,24]]},"reference":[{"issue":"6","key":"pone.0265751.ref001","doi-asserted-by":"crossref","first-page":"394","DOI":"10.3322\/caac.21492","article-title":"Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.","volume":"68","author":"F Bray","year":"2018","journal-title":"CA Cancer J Clin."},{"issue":"4","key":"pone.0265751.ref002","doi-asserted-by":"crossref","first-page":"279","DOI":"10.7326\/M15-2886","article-title":"Screening for Breast Cancer: U.S. Preventive Services Task Force Recommendation Statement","volume":"164","author":"AL Siu","year":"2016","journal-title":"Ann Intern Med"},{"issue":"7464","key":"pone.0265751.ref003","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1136\/bmj.329.7464.477","article-title":"Influence of personal characteristics of individual women on sensitivity and specificity of mammography in the Million Women Study: cohort study","volume":"329","author":"E Banks","year":"2004","journal-title":"BMJ"},{"issue":"11","key":"pone.0265751.ref004","doi-asserted-by":"crossref","first-page":"1828","DOI":"10.1001\/jamainternmed.2015.5231","article-title":"Diagnostic accuracy of digital screening mammography with and without computer-aided detection","volume":"175","author":"CD Lehman","year":"2015","journal-title":"JAMA Intern Med"},{"issue":"2","key":"pone.0265751.ref005","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1148\/radiol.13130724","article-title":"Breast cancer detected with screening US: reasons for nondetection at mammography","volume":"270","author":"MS Bae","year":"2014","journal-title":"Radiology"},{"issue":"3","key":"pone.0265751.ref006","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1148\/radiology.184.3.1509041","article-title":"Analysis of cancers missed at screening mammography.","volume":"184","author":"RE Bird","year":"1992","journal-title":"Radiology"},{"issue":"12","key":"pone.0265751.ref007","doi-asserted-by":"crossref","first-page":"1770","DOI":"10.1016\/S0959-8049(03)00311-3","article-title":"Use of previous screening mammograms to identify features indicating cases that would have a possible gain in prognosis following earlier detection","volume":"39","author":"MJ Broeders","year":"2003","journal-title":"Eur J Cancer"},{"issue":"4","key":"pone.0265751.ref008","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1148\/rg.234025083","article-title":"Missed breast carcinoma: pitfalls and pearls.","volume":"23","author":"AS Majid","year":"2003","journal-title":"Radiographics"},{"issue":"3","key":"pone.0265751.ref009","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1007\/s10549-016-3882-0","article-title":"Characteristics and prognosis of interval cancers after biennial screen-film or full-field digital screening mammography","volume":"158","author":"RJ Weber","year":"2016","journal-title":"Breast Cancer Res Treat"},{"issue":"11","key":"pone.0265751.ref010","doi-asserted-by":"crossref","first-page":"2019","DOI":"10.1016\/j.ejrad.2014.08.010","article-title":"Single reading with computer-aided detection performed by selected radiologists in a breast cancer screening program","volume":"83","author":"X Bargallo","year":"2014","journal-title":"Eur J Radiol"},{"issue":"8","key":"pone.0265751.ref011","doi-asserted-by":"crossref","first-page":"580","DOI":"10.7326\/0003-4819-158-8-201304160-00002","article-title":"Short-term outcomes of screening mammography using computer-aided detection: a population-based study of medicare enrollees","volume":"158","author":"JJ Fenton","year":"2013","journal-title":"Ann Intern Med"},{"issue":"16","key":"pone.0265751.ref012","doi-asserted-by":"crossref","first-page":"1675","DOI":"10.1056\/NEJMoa0803545","article-title":"Single reading with computer-aided detection for screening mammography","volume":"359","author":"FJ Gilbert","year":"2008","journal-title":"N Engl J Med"},{"issue":"4","key":"pone.0265751.ref013","doi-asserted-by":"crossref","first-page":"854","DOI":"10.2214\/AJR.07.2812","article-title":"Comparison of computer-aided detection to double reading of screening mammograms: review of 231,221 mammograms.","volume":"190","author":"M. Gromet","year":"2008","journal-title":"AJR Am J Roentgenol"},{"key":"pone.0265751.ref014","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1186\/1471-2342-12-22","article-title":"Is single reading with computer-aided detection (CAD) as good as double reading in mammography screening? A systematic review.","volume":"12","author":"E Azavedo","year":"2012","journal-title":"BMC Med Imaging"},{"issue":"14","key":"pone.0265751.ref015","doi-asserted-by":"crossref","first-page":"1399","DOI":"10.1056\/NEJMoa066099","article-title":"Influence of computer-aided detection on performance of screening mammography","volume":"356","author":"JJ Fenton","year":"2007","journal-title":"N Engl J Med"},{"issue":"7553","key":"pone.0265751.ref016","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning.","volume":"521","author":"Y LeCun","year":"2015","journal-title":"Nature"},{"issue":"1","key":"pone.0265751.ref017","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1007\/s11604-018-0795-3","article-title":"Technical and clinical overview of deep learning in radiology.","volume":"37","author":"D Ueda","year":"2019","journal-title":"Jpn J Radiol"},{"key":"pone.0265751.ref018","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks.","author":"A Krizhevsky","year":"2012","journal-title":"Adv Neural Inf Process Syst."},{"key":"pone.0265751.ref019","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-67558-9_37","volume-title":"Deep Learning for Automatic Detection of Abnormal Findings in Breast Mammography","author":"A Akselrod-Ballin","year":"2017"},{"key":"pone.0265751.ref020","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.cmpb.2018.01.017","article-title":"Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system.","volume":"157","author":"MA Al-Masni","year":"2018","journal-title":"Comput Methods Programs Biomed"},{"issue":"7","key":"pone.0265751.ref021","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1097\/RLI.0000000000000358","article-title":"Deep Learning in mammography: diagnostic accuracy of a multipurpose image analysis software in the detection of breast cancer","volume":"52","author":"AS Becker","year":"2017","journal-title":"Invest Radiol"},{"key":"pone.0265751.ref022","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-00946-5_10","volume-title":"Improving breast cancer detection using symmetry information with deep learning","author":"Y Brhane Hagos","year":"2018"},{"key":"pone.0265751.ref023","unstructured":"Choukroun Y, Bakalo R, Ben-Ari R, Askelrod-Ballin A, Barkan E, Kisilev P. Mammogram classification and abnormality detection from nonlocal labels using deep multiple instance neural network. Proceedings of the Eurographics Workshop on Visual Computing for Biology and Medicine; Bremen, Germany. 3309886: Eurographics Association; 2017. p. 11\u20139."},{"key":"pone.0265751.ref024","doi-asserted-by":"crossref","unstructured":"Dhungel N, Carneiro G, Bradley AP, editors. Automated mass detection in mammograms using cascaded deep learning and random forests. 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA); 2015 23\u201325 Nov. 2015.","DOI":"10.1109\/DICTA.2015.7371234"},{"key":"pone.0265751.ref025","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-46723-8_13","volume-title":"The automated learning of deep features for breast mass classification from mammograms","author":"N Dhungel","year":"2016"},{"key":"pone.0265751.ref026","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/j.media.2017.01.009","article-title":"A deep learning approach for the analysis of masses in mammograms with minimal user intervention","volume":"37","author":"N Dhungel","year":"2017","journal-title":"Med Image Anal"},{"issue":"9","key":"pone.0265751.ref027","doi-asserted-by":"crossref","first-page":"e0203355","DOI":"10.1371\/journal.pone.0203355","article-title":"Detection of masses in mammograms using a one-stage object detector based on a deep convolutional neural network.","volume":"13","author":"H Jung","year":"2018","journal-title":"PLoS One."},{"key":"pone.0265751.ref028","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1016\/j.media.2016.07.007","article-title":"Large scale deep learning for computer aided detection of mammographic lesions","volume":"35","author":"T Kooi","year":"2017","journal-title":"Med Image Anal"},{"issue":"7788","key":"pone.0265751.ref029","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1038\/s41586-019-1799-6","article-title":"International evaluation of an AI system for breast cancer screening","volume":"577","author":"SM McKinney","year":"2020","journal-title":"Nature"},{"issue":"1","key":"pone.0265751.ref030","doi-asserted-by":"crossref","first-page":"4165","DOI":"10.1038\/s41598-018-22437-z","article-title":"Detecting and classifying lesions in mammograms with Deep Learning.","volume":"8","author":"D Ribli","year":"2018","journal-title":"Sci Rep."},{"issue":"2","key":"pone.0265751.ref031","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1148\/radiol.2018181371","article-title":"Detection of breast cancer with mammography: effect of an artificial intelligence support system","volume":"290","author":"A Rodriguez-Ruiz","year":"2019","journal-title":"Radiology"},{"key":"pone.0265751.ref032","author":"J Teuwen","year":"2018","journal-title":"Soft tissue lesion detection in mammography using deep neural networks for object detection"},{"issue":"2","key":"pone.0265751.ref033","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1148\/radiol.2019182622","article-title":"Predicting Breast Cancer by Applying Deep Learning to Linked Health Records and Mammograms.","volume":"292","author":"A Akselrod-Ballin","year":"2019","journal-title":"Radiology"},{"issue":"2","key":"pone.0265751.ref034","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1148\/radiol.2019190872","article-title":"Comparison of a Deep Learning Risk Score and Standard Mammographic Density Score for Breast Cancer Risk Prediction.","volume":"294","author":"K Dembrower","year":"2020","journal-title":"Radiology"},{"issue":"1","key":"pone.0265751.ref035","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1148\/radiol.2019182716","article-title":"A Deep Learning Mammography-based Model for Improved Breast Cancer Risk Prediction.","volume":"292","author":"A Yala","year":"2019","journal-title":"Radiology"},{"issue":"5","key":"pone.0265751.ref036","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1200\/PO.20.00176","article-title":"Training, Validation, and Test of Deep Learning Models for Classification of Receptor Expressions in Breast Cancers From Mammograms.","author":"D Ueda","year":"2021","journal-title":"JCO Precision Oncology"},{"issue":"4","key":"pone.0265751.ref037","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1007\/s11604-020-01070-9","article-title":"Visualizing \u201cfeatureless\u201d regions on mammograms classified as invasive ductal carcinomas by a deep learning algorithm: the promise of AI support in radiology.","volume":"39","author":"D Ueda","year":"2021","journal-title":"Japanese Journal of Radiology"},{"key":"pone.0265751.ref038","doi-asserted-by":"crossref","first-page":"g7594","DOI":"10.1136\/bmj.g7594","article-title":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ","volume":"350","author":"GS Collins","year":"2015","journal-title":"British Medical Journal"},{"key":"pone.0265751.ref039","first-page":"39","article-title":"ACR BI-RADS\u00ae Atlas, Breast imaging reporting and data system.","author":"Sickles EA","year":"2013","journal-title":"American College of Radiology."},{"issue":"3","key":"pone.0265751.ref040","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1007\/s10549-010-0770-x","article-title":"Predictors of interobserver agreement in breast imaging using the Breast Imaging Reporting and Data System","volume":"120","author":"ALM Antonio","year":"2010","journal-title":"Breast Cancer Res Treat"},{"issue":"6","key":"pone.0265751.ref041","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1007\/s10278-013-9622-7","article-title":"The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository.","volume":"26","author":"K Clark","year":"2013","journal-title":"J Digit Imaging."},{"key":"pone.0265751.ref042","doi-asserted-by":"crossref","first-page":"170177","DOI":"10.1038\/sdata.2017.177","article-title":"A curated mammography data set for use in computer-aided detection and diagnosis research.","volume":"4","author":"RS Lee","year":"2017","journal-title":"Sci Data."},{"key":"pone.0265751.ref043","article-title":"Curated Breast Imaging Subset of DDSM (CBIS-DDSM).","author":"RS Lee","year":"2016","journal-title":"The Cancer Imaging Archive"},{"key":"pone.0265751.ref044","article-title":"Focal loss for dense object detection","author":"TY Lin","year":"2018","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"pone.0265751.ref045","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J, editors. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition; 2016.","DOI":"10.1109\/CVPR.2016.90"},{"key":"pone.0265751.ref046","doi-asserted-by":"crossref","unstructured":"Lin T, Doll\u00e1r P, Girshick R, He K, Hariharan B, Belongie S, editors. Feature pyramid networks for object detection. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017 21\u201326 July 2017.","DOI":"10.1109\/CVPR.2017.106"},{"key":"pone.0265751.ref047","unstructured":"Github [2020\/03\/28]. Available from: https:\/\/github.com\/detection-mammography."},{"key":"pone.0265751.ref048","unstructured":"TensorFlow [2020\/03\/28]. Available from: https:\/\/www.tensorflow.org."},{"issue":"2","key":"pone.0265751.ref049","first-page":"382","article-title":"Free response approach to measurement and characterization of radiographic observer performance.","volume":"130","author":"PC Bunch","year":"1978","journal-title":"AJR Am J Roentgenol"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0265751","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,3,24]],"date-time":"2022-03-24T21:34:31Z","timestamp":1648157671000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0265751"}},"subtitle":[],"editor":[{"given":"Pascal A. T.","family":"Baltzer","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2022,3,24]]},"references-count":49,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,3,24]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0265751","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,24]]}}}