{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,29]],"date-time":"2026-10-29T03:03:27Z","timestamp":1793243007569,"version":"4.5.0"},"reference-count":57,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2024,9,6]],"date-time":"2024-09-06T00:00:00Z","timestamp":1725580800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["No. 2020R1F1A1070070"],"award-info":[{"award-number":["No. 2020R1F1A1070070"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002649","name":"Yeungnam University","doi-asserted-by":"publisher","award":["222A580015"],"award-info":[{"award-number":["222A580015"]}],"id":[{"id":"10.13039\/501100002649","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>Camera image-based deep learning (DL) techniques have achieved promising results in dental caries screening. To apply the intraoral camera image-based DL technique for dental caries detection and assess its diagnostic performance, we employed the ensemble technique in the image classification task. 2,682 intraoral camera images were used as the dataset for image classification according to dental caries presence and caries-lesion localization using DL models such as ResNet-50, Inception-v3, Inception-ResNet-v2, and Faster R-convolutional neural network according to diagnostic study design. 534 participants whose mean age [SD] was 47.67 [\u00b113.94] years were enrolled. The dataset was divided into training (56.0%), validation (14.0%), and test subset (30.0%) annotated by one experienced dentist as a reference standard about dental caries detection and lesion location. The confusion matrix, area under the receiver operating characteristic curve (AUROC), and average precision (AP) were evaluated for performance analysis. In the end-to-end dental caries image classification, the ensemble DL models had consistently improved performance, in which as the best results, the ensemble model of Inception-ResNet-v2 achieved 0.94 of AUROC and 0.97 of AP. On the other hand, the explainable model achieved 0.91 of AUROC and 0.96 of AP after the ensemble application. For dental caries classification using intraoral camera images, the application of ensemble techniques exhibited consistently improved performance regardless of the DL models. Furthermore, the trial to create an explainable DL model based on carious lesion detection yielded favorable results.<\/jats:p>","DOI":"10.1371\/journal.pone.0310004","type":"journal-article","created":{"date-parts":[[2024,9,6]],"date-time":"2024-09-06T14:07:27Z","timestamp":1725631647000},"page":"e0310004","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":25,"title":["Diagnostic accuracy of dental caries detection using ensemble techniques in deep learning with intraoral camera images"],"prefix":"10.1371","volume":"19","author":[{"given":"Sohee","family":"Kang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Byungeun","family":"Shon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eun Young","family":"Park","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sungmoon","family":"Jeong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9582-1415","authenticated-orcid":true,"given":"Eun-Kyong","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2024,9,6]]},"reference":[{"issue":"10159","key":"pone.0310004.ref001","doi-asserted-by":"crossref","first-page":"1789","DOI":"10.1016\/S0140-6736(18)32279-7","article-title":"Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990\u20132017: A systematic analysis for the Global Burden of Disease Study 2017","volume":"392","author":"SL James","year":"2018","journal-title":"Lancet"},{"issue":"7","key":"pone.0310004.ref002","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.17219\/acem\/77022","article-title":"Evidence-based strategies for the minimally invasive treatment of carious lesions: Review of the literature.","volume":"27","author":"RA Giacaman","year":"2018","journal-title":"Adv Clin Exp Med"},{"issue":"1","key":"pone.0310004.ref003","doi-asserted-by":"crossref","first-page":"12444","DOI":"10.1038\/s41598-024-63289-0","article-title":"Equivalence study of the resin-dentine interface of internal tunnel restorations when using an enamel infiltrant resin with ethanol-wet dentine bonding.","volume":"14","author":"AM Kielbassa","year":"2024","journal-title":"Sci Rep"},{"issue":"1","key":"pone.0310004.ref004","doi-asserted-by":"crossref","first-page":"e0228249","DOI":"10.1371\/journal.pone.0228249","article-title":"Ex vivo investigation on internal tunnel approach\/internal resin infiltration and external nanosilver-modified resin infiltration of proximal caries exceeding into dentin.","volume":"15","author":"AM Kielbassa","year":"2020","journal-title":"PLoS ONE."},{"issue":"4","key":"pone.0310004.ref005","first-page":"357","article-title":"External and internal resin infiltration of natural proximal subsurface caries lesions: a valuable enhancement of the internal tunnel restoration.","volume":"48","author":"AM Kielbassa","year":"2017","journal-title":"Quintessence Int."},{"issue":"6","key":"pone.0310004.ref006","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"S Ren","year":"2017","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"pone.0310004.ref007","first-page":"770","article-title":"Deep residual learning for image recognition","volume":"2016","author":"K He","year":"2016","journal-title":"Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit"},{"key":"pone.0310004.ref008","volume-title":"Deep learning","author":"I Goodfellow","year":"2016"},{"issue":"2","key":"pone.0310004.ref009","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1080\/13645706.2019.1575882","article-title":"Introduction to artificial intelligence in medicine.","volume":"28","author":"Y Mintz","year":"2019","journal-title":"Minim Invasive Ther Allied Technol."},{"key":"pone.0310004.ref010","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.jdent.2018.07.015","article-title":"Detection and diagnosis of dental caries using a deep learning-based convolutional neural network algorithm.","volume":"77","author":"JH Lee","year":"2018","journal-title":"J Dent."},{"issue":"1","key":"pone.0310004.ref011","doi-asserted-by":"crossref","first-page":"16807","DOI":"10.1038\/s41598-021-96368-7","article-title":"Deep learning for early dental caries detection in bitewing radiographs.","volume":"11","author":"S Lee","year":"2021","journal-title":"Sci Rep"},{"key":"pone.0310004.ref012","doi-asserted-by":"crossref","first-page":"103425","DOI":"10.1016\/j.jdent.2020.103425","article-title":"Detecting caries lesions of different radiographic extension on bitewings using deep learning.","volume":"100","author":"AG Cantu","year":"2020","journal-title":"J Dent"},{"issue":"1","key":"pone.0310004.ref013","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1007\/s13755-019-0096-y","article-title":"Dental caries diagnosis in digital radiographs using back-propagation neural network.","volume":"8","author":"V Geetha","year":"2020","journal-title":"Health Inf Sci Syst"},{"issue":"6","key":"pone.0310004.ref014","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1016\/j.oooo.2020.08.024","article-title":"Automated feature detection in dental periapical radiographs by using deep learning.","volume":"131","author":"HA Khan","year":"2021","journal-title":"Oral Surg Oral Med Oral Pathol Oral Radiol"},{"issue":"13","key":"pone.0310004.ref015","doi-asserted-by":"crossref","first-page":"4613","DOI":"10.3390\/s21134613","article-title":"Caries and restoration detection using bitewing film based on transfer learning with CNNs.","volume":"21","author":"YC Mao","year":"2021","journal-title":"Sensors (Basel)."},{"issue":"5","key":"pone.0310004.ref016","doi-asserted-by":"crossref","first-page":"20210010","DOI":"10.1259\/dmfr.20210010","article-title":"Radiographic modalities for diagnosis of caries in a historical perspective: From film to machine-intelligence supported systems","volume":"50","author":"A. Wenzel","year":"2021","journal-title":"Dentomaxillofac Radiol."},{"issue":"5","key":"pone.0310004.ref017","doi-asserted-by":"crossref","first-page":"1047","DOI":"10.3390\/diagnostics12051047","article-title":"Detecting proximal caries on periapical radiographs using convolutional neural networks with different training strategies on small datasets.","volume":"12","author":"X Lin","year":"2022","journal-title":"Diagnostics (Basel)."},{"issue":"4","key":"pone.0310004.ref018","doi-asserted-by":"crossref","first-page":"5255","DOI":"10.1007\/s11042-020-09891-6","article-title":"G.V Black dental caries classification and preparation technique using optimal CNN-LSTM classifier.","volume":"80","author":"P Singh","year":"2021","journal-title":"Multimed Tools Appl"},{"issue":"11","key":"pone.0310004.ref019","doi-asserted-by":"crossref","first-page":"1227","DOI":"10.1177\/0022034519871884","article-title":"Caries detection with near-infrared transillumination using deep learning","volume":"98","author":"F Casalegno","year":"2019","journal-title":"J Dent Res"},{"key":"pone.0310004.ref020","doi-asserted-by":"crossref","first-page":"103260","DOI":"10.1016\/j.jdent.2019.103260","article-title":"Deep learning for caries lesion detection in near-infrared light transillumination images: A pilot study.","volume":"92","author":"F Schwendicke","year":"2020","journal-title":"J Dent"},{"key":"pone.0310004.ref021","doi-asserted-by":"crossref","DOI":"10.1101\/2021.05.04.21256502","volume-title":"Deep learning for caries detection using optical coherence tomography","author":"YP Huang","year":"2021"},{"issue":"5","key":"pone.0310004.ref022","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1111\/iej.13265","article-title":"Evaluation of artificial intelligence for detecting periapical pathosis on cone-beam computed tomography scans","volume":"53","author":"K Orhan","year":"2020","journal-title":"Int Endod J"},{"key":"pone.0310004.ref023","doi-asserted-by":"crossref","first-page":"103615","DOI":"10.1016\/j.jdent.2021.103615","article-title":"Detecting white spot lesions on dental photography using deep learning: A pilot study.","volume":"107","author":"H Askar","year":"2021","journal-title":"J Dent"},{"issue":"2","key":"pone.0310004.ref024","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1177\/00220345211032524","article-title":"Caries detection on intraoral images using artificial intelligence","volume":"101","author":"J K\u00fchnisch","year":"2022","journal-title":"J Dent Res"},{"issue":"2","key":"pone.0310004.ref025","first-page":"1","article-title":"A smart home dental care system: Integration of deep learning, image sensors, and mobile controller.","volume":"14","author":"D Kim","year":"2023","journal-title":"J Ambient Intell Humaniz Comput"},{"key":"pone.0310004.ref026","first-page":"1617","article-title":"Recognizing occlusal caries in dental intraoral images using deep learning","volume":"2019","author":"K Moutselos","year":"2019","journal-title":"Annu Int Conf IEEE Eng Med Biol Soc"},{"issue":"1","key":"pone.0310004.ref027","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1111\/odi.13735","article-title":"Development and evaluation of deep learning for screening dental caries from oral photographs","volume":"28","author":"X Zhang","year":"2022","journal-title":"Oral Dis"},{"issue":"11","key":"pone.0310004.ref028","doi-asserted-by":"crossref","first-page":"5504","DOI":"10.3390\/app12115504","article-title":"Deep learning application in dental caries detection using intraoral photos taken by smartphones.","volume":"12","author":"MT Thanh","year":"2022","journal-title":"Applied Sciences"},{"issue":"2","key":"pone.0310004.ref029","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1016\/j.oooo.2022.03.008","article-title":"Evaluation of a deep learning system for automatic detection of proximal surface dental caries on bitewing radiographs.","volume":"134","author":"M Estai","year":"2022","journal-title":"Oral Surg Oral Med Oral Pathol Oral Radiol"},{"issue":"1","key":"pone.0310004.ref030","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1186\/s12903-022-02589-1","article-title":"Caries detection with tooth surface segmentation on intraoral photographic images using deep learning.","volume":"22","author":"EY Park","year":"2022","journal-title":"BMC Oral Health"},{"issue":"4","key":"pone.0310004.ref031","doi-asserted-by":"crossref","first-page":"1765","DOI":"10.1111\/odi.14659","article-title":"Detecting dental caries on oral photographs using artificial intelligence: A systematic review","volume":"30","author":"M Moharrami","year":"2024","journal-title":"Oral Dis"},{"issue":"2","key":"pone.0310004.ref032","first-page":"757","article-title":"A comprehensive review on ensemble deep learning: Opportunities and challenges.","volume":"35","author":"A Mohammed","year":"2023","journal-title":"J King Saud Univ Comput Inf Sci"},{"key":"pone.0310004.ref033","doi-asserted-by":"crossref","first-page":"105151","DOI":"10.1016\/j.engappai.2022.105151","article-title":"Ensemble deep learning: A review.","volume":"115","author":"MA Ganaie","year":"2022","journal-title":"Eng Appl Artif Intell"},{"issue":"6","key":"pone.0310004.ref034","doi-asserted-by":"crossref","first-page":"5545","DOI":"10.1007\/s10462-022-10283-5","article-title":"A survey on ensemble learning under the era of deep learning.","volume":"56","author":"Y Yang","year":"2023","journal-title":"Artif Intell Rev"},{"key":"pone.0310004.ref035","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cmpb.2017.09.005","article-title":"A deep learning-based multi-model ensemble method for cancer prediction","volume":"153","author":"Y Xiao","year":"2018","journal-title":"Comput Methods Programs Biomed"},{"key":"pone.0310004.ref036","doi-asserted-by":"crossref","first-page":"100203","DOI":"10.1016\/j.imu.2019.100203","article-title":"Improving the accuracy of prediction of heart disease risk based on ensemble classification techniques","volume":"16","author":"CB Latha","year":"2019","journal-title":"Inform Med Unlocked"},{"issue":"1","key":"pone.0310004.ref037","doi-asserted-by":"crossref","first-page":"150","DOI":"10.11607\/jomi.9818","article-title":"Identification of 130 dental implant types using ensemble deep learning.","volume":"38","author":"HJ Kong","year":"2023","journal-title":"Int J Oral Maxillofac Implants"},{"issue":"6","key":"pone.0310004.ref038","doi-asserted-by":"crossref","first-page":"52","DOI":"10.3390\/jimaging6060052","article-title":"Explainable deep learning models in medical image analysis.","volume":"6","author":"A Singh","year":"2020","journal-title":"J imaging."},{"key":"pone.0310004.ref039","doi-asserted-by":"crossref","first-page":"875","DOI":"10.1007\/978-3-030-63416-2_660","article-title":"Object detection.","author":"Y Amit","year":"2021","journal-title":"Computer Vision: A Reference Guide"},{"key":"pone.0310004.ref040","first-page":"15849","article-title":"End-to-end object detection with fully convolutional network","volume":"2021","author":"J Wang","year":"2021","journal-title":"Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit"},{"issue":"3","key":"pone.0310004.ref041","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1109\/JPROC.2023.3238524","article-title":"Object detection in 20 years: A survey","volume":"111","author":"Z Zou","year":"2023","journal-title":"Proc IEEE"},{"issue":"11","key":"pone.0310004.ref042","doi-asserted-by":"crossref","first-page":"e012799","DOI":"10.1136\/bmjopen-2016-012799","article-title":"STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration","volume":"6","author":"JF Cohen","year":"2016","journal-title":"BMJ Open"},{"issue":"3","key":"pone.0310004.ref043","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1111\/j.1600-0528.2007.00347.x","article-title":"The international caries detection and assessment system (ICDAS): An integrated system for measuring dental caries.","volume":"35","author":"AI Ismail","year":"2007","journal-title":"Community Dent Oral Epidemiol"},{"issue":"1","key":"pone.0310004.ref044","doi-asserted-by":"crossref","first-page":"e41","DOI":"10.1111\/cdoe.12025","article-title":"International caries detection and assessment system (ICDAS) and its international caries classification and management system (ICCMS)\u2014Methods for staging of the caries process and enabling dentists to manage caries.","volume":"41","author":"NB Pitts","year":"2013","journal-title":"Community Dent Oral Epidemiol"},{"key":"pone.0310004.ref045","first-page":"2818","article-title":"Rethinking the inception architecture for computer vision","volume":"2016","author":"C Szegedy","year":"2016","journal-title":"Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit"},{"issue":"1","key":"pone.0310004.ref046","first-page":"4278","article-title":"Inception-v4, inception-ResNet and the impact of residual connections on learning","volume":"31","author":"C Szegedy","year":"2017","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"pone.0310004.ref047","first-page":"248","article-title":"Imagenet: A large-scale hierarchical image database","volume":"2009","author":"J Deng","year":"2009","journal-title":"Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit"},{"key":"pone.0310004.ref048","volume-title":"Decoupled weight decay regularization.","author":"I Loshchilov","year":"2017"},{"key":"pone.0310004.ref049","volume-title":"Computer Vision\u2013ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8693","author":"TY Lin"},{"key":"pone.0310004.ref050","first-page":"2117","article-title":"Feature pyramid networks for object detection","volume":"2017","author":"TY Lin","year":"2017","journal-title":"Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit"},{"issue":"1","key":"pone.0310004.ref051","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1002\/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3","article-title":"Index for rating diagnostic tests","volume":"3","author":"WJ Youden","year":"1950","journal-title":"Cancer"},{"key":"pone.0310004.ref052","first-page":"233","article-title":"The relationship between precision-recall and ROC curves","volume":"2006","author":"J Davis","year":"2006","journal-title":"Proc Int Conf Mach Learn"},{"issue":"1","key":"pone.0310004.ref053","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1007\/s40368-015-0208-6","article-title":"Detection and monitoring of early caries lesions: a review","volume":"17","author":"IA Pretty","year":"2016","journal-title":"Eur Arch Paediatr Dent"},{"key":"pone.0310004.ref054","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1186\/1472-6831-12-10","article-title":"Comparison of photographic and visual assessment of occlusal caries with histology as the reference standard.","volume":"12","author":"U Boye","year":"2012","journal-title":"BMC Oral Health"},{"key":"pone.0310004.ref055","first-page":"105","article-title":"Traditional and novel caries detection methods.","author":"MB Diniz","year":"2012","journal-title":"Contemporary Approach to Dental Caries"},{"issue":"7","key":"pone.0310004.ref056","first-page":"e41694","article-title":"Transforming dental caries diagnosis through artificial intelligence-based techniques.","volume":"15","author":"S Anil","year":"2023","journal-title":"Cureus"},{"issue":"12","key":"pone.0310004.ref057","first-page":"ZC09","article-title":"Caries detection with ICDAS and the WHO criteria: a comparative study.","volume":"11","author":"PH Bhoopathi","year":"2017","journal-title":"J Clin Diagn Res."}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0310004","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,6]],"date-time":"2024-09-06T14:08:06Z","timestamp":1725631686000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0310004"}},"subtitle":[],"editor":[{"given":"Andrej M.","family":"Kielbassa","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2024,9,6]]},"references-count":57,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2024,9,6]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0310004","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,6]]}}}