{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,19]],"date-time":"2026-09-19T04:31:10Z","timestamp":1789792270575,"version":"4.0.1"},"reference-count":37,"publisher":"Public Library of Science (PLoS)","issue":"11","license":[{"start":{"date-parts":[[2020,11,11]],"date-time":"2020-11-11T00:00:00Z","timestamp":1605052800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"crossref","award":["2017R1C1B5076240"],"award-info":[{"award-number":["2017R1C1B5076240"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:sec id=\"sec001\">\n<jats:title>Objectives<\/jats:title>\n<jats:p>This study aimed to compare the diagnostic performance of deep learning algorithm trained by single view (anterior-posterior (AP) or lateral view) with that trained by multiple views (both views together) in diagnosis of mastoiditis on mastoid series and compare the diagnostic performance between the algorithm and radiologists.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec002\">\n<jats:title>Methods<\/jats:title>\n<jats:p>Total 9,988 mastoid series (AP and lateral views) were classified as normal or abnormal (mastoiditis) based on radiographic findings. Among them 792 image sets with temporal bone CT were classified as the gold standard test set and remaining sets were randomly divided into training (n = 8,276) and validation (n = 920) sets by 9:1 for developing a deep learning algorithm. Temporal (n = 294) and geographic (n = 308) external test sets were also collected. Diagnostic performance of deep learning algorithm trained by single view was compared with that trained by multiple views. Diagnostic performance of the algorithm and two radiologists was assessed. Inter-observer agreement between the algorithm and radiologists and between two radiologists was calculated.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec003\">\n<jats:title>Results<\/jats:title>\n<jats:p>Area under the receiver operating characteristic curves of algorithm using multiple views (0.971, 0.978, and 0.965 for gold standard, temporal, and geographic external test sets, respectively) showed higher values than those using single view (0.964\/0.953, 0.952\/0.961, and 0.961\/0.942 for AP view\/lateral view of gold standard, temporal external, and geographic external test sets, respectively) in all test sets. The algorithm showed statistically significant higher specificity compared with radiologists (p = 0.018 and 0.012). There was substantial agreement between the algorithm and two radiologists and between two radiologists (\u03ba = 0.79, 0.8, and 0.76).<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec004\">\n<jats:title>Conclusion<\/jats:title>\n<jats:p>The deep learning algorithm trained by multiple views showed better performance than that trained by single view. The diagnostic performance of the algorithm for detecting mastoiditis on mastoid series was similar to or higher than that of radiologists.<\/jats:p>\n<\/jats:sec>","DOI":"10.1371\/journal.pone.0241796","type":"journal-article","created":{"date-parts":[[2020,11,11]],"date-time":"2020-11-11T18:30:55Z","timestamp":1605119455000},"page":"e0241796","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":13,"title":["Performance of deep learning to detect mastoiditis using multiple conventional radiographs of mastoid"],"prefix":"10.1371","volume":"15","author":[{"given":"Kyong Joon","family":"Lee","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6866-7926","authenticated-orcid":true,"given":"Inseon","family":"Ryoo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dongjun","family":"Choi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0374-8658","authenticated-orcid":true,"given":"Leonard","family":"Sunwoo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sung-Hye","family":"You","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hye Na","family":"Jung","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2020,11,11]]},"reference":[{"key":"pone.0241796.ref001","doi-asserted-by":"crossref","first-page":"925","DOI":"10.1007\/s00405-013-2454-8","article-title":"Mastoiditis in adults: a 19-year retrospective study","volume":"271","author":"S Palma","year":"2014","journal-title":"Eur Arch Otorhinolaryngol"},{"key":"pone.0241796.ref002","first-page":"453","article-title":"Clinical and histopathological aspects in otomastoiditis","volume":"50","author":"C Popescu","year":"2009","journal-title":"Rom J Morphol Embryol"},{"key":"pone.0241796.ref003","doi-asserted-by":"crossref","first-page":"751","DOI":"10.1097\/MAO.0b013e31817f736b","article-title":"A systematic review of diagnostic criteria for acute mastoiditis in children","volume":"29","author":"MT van den Aardweg","year":"2008","journal-title":"Otol Neurotol"},{"key":"pone.0241796.ref004","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1148\/radiology.155.2.3983389","article-title":"Acute otomastoiditis and its complications: role of CT","volume":"155","author":"MF Mafee","year":"1985","journal-title":"Radiology"},{"key":"pone.0241796.ref005","first-page":"493","article-title":"Tuberculous otomastoiditis on high-resolution temporal bone CT: comparison with nontuberculous otomastoiditis with and without cholesteatoma","volume":"28","author":"MH Rho","year":"2007","journal-title":"AJNR Am J Neuroradiol"},{"key":"pone.0241796.ref006","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1177\/000348940511400212","article-title":"Complications of mastoiditis in children at the onset of a new millennium","volume":"114","author":"Y Oestreicher-Kedem","year":"2005","journal-title":"Ann Otol Rhinol Laryngol"},{"key":"pone.0241796.ref007","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1016\/j.jacr.2016.09.039","article-title":"Utilization Trends in Noncardiac Thoracic Imaging, 2002\u20132014","volume":"14","author":"SI Kamel","year":"2017","journal-title":"J Am Coll Radiol"},{"key":"pone.0241796.ref008","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1097\/RTI.0000000000000387","article-title":"Deep Learning Applications in Chest Radiography and Computed Tomography: Current State of the Art","volume":"34","author":"SM Lee","year":"2019","journal-title":"J Thorac Imaging"},{"key":"pone.0241796.ref009","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1148\/radiol.2019191116","article-title":"Trends in Diagnostic Imaging Utilization among Medicare and Commercially Insured Adults from 2003 through 2016","volume":"294","author":"AS Hong","year":"2020","journal-title":"Radiology"},{"key":"pone.0241796.ref010","doi-asserted-by":"crossref","first-page":"942","DOI":"10.21037\/qims.2019.05.15","article-title":"Improvement diagnostic accuracy of sinusitis recognition in paranasal sinus X-ray using multiple deep learning models","volume":"9","author":"HG Kim","year":"2019","journal-title":"Quant Imaging Med Surg"},{"key":"pone.0241796.ref011","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1097\/RLI.0000000000000503","article-title":"Deep Learning in Diagnosis of Maxillary Sinusitis Using Conventional Radiography","volume":"54","author":"Y Kim","year":"2019","journal-title":"Invest Radiol"},{"key":"pone.0241796.ref012","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1148\/radiol.2017162326","article-title":"Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks","volume":"284","author":"P Lakhani","year":"2017","journal-title":"Radiology"},{"issue":"10","key":"pone.0241796.ref013","doi-asserted-by":"crossref","first-page":"e0204155","DOI":"10.1371\/journal.pone.0204155","article-title":"Deep learning in chest radiography: Detection of findings and presence of change","volume":"13","author":"R Singh","year":"2018","journal-title":"PLoS One"},{"key":"pone.0241796.ref014","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1148\/radiol.2019182908","article-title":"A Deep Learning Model to Triage Screening Mammograms: A Simulation Study","volume":"293","author":"A Yala","year":"2019","journal-title":"Radiology"},{"key":"pone.0241796.ref015","article-title":"editors. Squeeze-andexcitation networks","author":"J Hu","year":"2018","journal-title":"IEEE conference on computer vision and pattern recognition"},{"key":"pone.0241796.ref016","first-page":"180307703","article-title":"Weakly supervised medical diagnosis and localization from multiple resolutions","author":"L Yao","year":"2018","journal-title":"arXiv preprint arXiv."},{"key":"pone.0241796.ref017","article-title":"Understanding the difficulty of training deep feedforward neural networks","author":"X Glorot","year":"2010","journal-title":"the thirteenth international conference on artificial intelligence and statistics"},{"key":"pone.0241796.ref018","author":"G Hinton","year":"2012","journal-title":"Neural networks for machine learning lecture 6a overview of mini-batch gradient descent"},{"key":"pone.0241796.ref019","first-page":"3320","article-title":"How transferable are features in deep neural networks?","volume":"27","author":"J Yosinski","year":"2014","journal-title":"Advances in neural information processing systems"},{"key":"pone.0241796.ref020","doi-asserted-by":"crossref","first-page":"837","DOI":"10.2307\/2531595","article-title":"Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach","volume":"44","author":"ER DeLong","year":"1988","journal-title":"Biometrics"},{"key":"pone.0241796.ref021","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1093\/biomet\/26.4.404","article-title":"The use of confidence or fiducial limits illustrated in the case of the binomial","volume":"26","author":"CJ Clopper","year":"1934","journal-title":"Biometrika"},{"key":"pone.0241796.ref022","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: the kappa statistic","volume":"22","author":"ML McHugh","year":"2012","journal-title":"Biochem Medica"},{"key":"pone.0241796.ref023","article-title":"editors. Learning deep features for discriminative localization","author":"B Zhou","year":"2016","journal-title":"IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"pone.0241796.ref024","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs","volume":"316","author":"V Gulshan","year":"2016","journal-title":"JAMA"},{"key":"pone.0241796.ref025","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","article-title":"Deep Neural Networks for Acoustic Modeling in Speech Recognition","volume":"29","author":"G Hinton","year":"2012","journal-title":"Ieee Signal Proc Mag"},{"key":"pone.0241796.ref026","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1093\/bioinformatics\/btu277","article-title":"Deep learning of the tissue-regulated splicing code","volume":"30","author":"MKK Leung","year":"2014","journal-title":"Bioinformatics"},{"key":"pone.0241796.ref027","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"SQ Ren","year":"2017","journal-title":"Ieee T Pattern Anal"},{"key":"pone.0241796.ref028","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1016\/j.ymssp.2018.05.050","article-title":"Deep learning and its applications to machine health monitoring","volume":"115","author":"R Zhao","year":"2019","journal-title":"Mech Syst Signal Pr"},{"key":"pone.0241796.ref029","doi-asserted-by":"crossref","first-page":"1243","DOI":"10.1007\/s00330-019-06417-z","article-title":"Semi-automatic classification of prostate cancer on multi-parametric MR imaging using a multi-channel 3D convolutional neural network","volume":"30","author":"N Aldoj","year":"2020","journal-title":"Eur Radiol."},{"key":"pone.0241796.ref030","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1002\/jmri.26766","article-title":"Deep learning enables automatic detection and segmentation of brain metastases on multisequence MRI","volume":"51","author":"E Grovik","year":"2020","journal-title":"J Magn Reson Imaging"},{"key":"pone.0241796.ref031","doi-asserted-by":"crossref","first-page":"6891","DOI":"10.1007\/s00330-019-06327-0","article-title":"Deep learning to differentiate parkinsonian disorders separately using single midsagittal MR imaging: a proof of concept study","volume":"29","author":"S Kiryu","year":"2019","journal-title":"Eur Radiol"},{"key":"pone.0241796.ref032","doi-asserted-by":"crossref","first-page":"885","DOI":"10.1002\/hed.25415","article-title":"Deep convolutional neural network for the diagnosis of thyroid nodules on ultrasound","volume":"41","author":"SY Ko","year":"2019","journal-title":"Head Neck"},{"key":"pone.0241796.ref033","doi-asserted-by":"crossref","first-page":"1332","DOI":"10.1089\/thy.2018.0082","article-title":"Deep Learning-Based Computer-Aided Diagnosis System for Localization and Diagnosis of Metastatic Lymph Nodes on Ultrasound: A Pilot Study","volume":"28","author":"JH Lee","year":"2018","journal-title":"Thyroid"},{"key":"pone.0241796.ref034","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1258\/002221503768199906","article-title":"Evaluation of the mastoid air cell system by high resolution computed tomography: three-dimensional multiplanar volume rendering technique","volume":"117","author":"A Koc","year":"2003","journal-title":"J Laryngol Otol"},{"key":"pone.0241796.ref035","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1007\/s00276-004-0274-7","article-title":"Volume variation of mastoid pneumatization in different age groups: a study by three-dimensional reconstruction based on computed tomography images","volume":"27","author":"DH Lee","year":"2005","journal-title":"Surg Radiol Anat"},{"issue":"5 Pt 1","key":"pone.0241796.ref036","doi-asserted-by":"crossref","first-page":"486","DOI":"10.1177\/000348940111000516","article-title":"Volume of mastoid pneumatization: three-dimensional reconstruction with ultrahigh-resolution computed tomography","volume":"110","author":"M Luntz","year":"2001","journal-title":"Ann Otol Rhinol Laryngol"},{"key":"pone.0241796.ref037","doi-asserted-by":"crossref","first-page":"226","DOI":"10.3109\/00016488709107277","article-title":"Mastoid size determined with lateral radiographs and computerized tomography","volume":"103","author":"NW Todd","year":"1987","journal-title":"Acta Otolaryngol"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0241796","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,11]],"date-time":"2020-11-11T18:31:10Z","timestamp":1605119470000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0241796"}},"subtitle":[],"editor":[{"given":"Yuchen","family":"Qiu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2020,11,11]]},"references-count":37,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2020,11,11]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0241796","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,11]]}}}