{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T17:56:10Z","timestamp":1793382970038,"version":"4.5.0"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,3,30]],"date-time":"2023-03-30T00:00:00Z","timestamp":1680134400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,30]],"date-time":"2023-03-30T00:00:00Z","timestamp":1680134400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Determining the severity of dental crowding and the necessity of tooth extraction for orthodontic treatment planning are time-consuming processes and there are no firm criteria. Thus, automated assistance would be useful to clinicians. This study aimed to construct and evaluate artificial intelligence (AI) systems to assist with such treatment planning. A total of 3,136 orthodontic occlusal photographs with annotations by two orthodontists were obtained. Four convolutional neural network (CNN) models, namely ResNet50, ResNet101, VGG16, and VGG19, were adopted for the AI process. Using the intraoral photographs as input, the crowding group and the necessity of tooth extraction were obtained. Arch length discrepancy analysis with AI-detected landmarks was used for crowding categorization. Various statistical and visual analyses were conducted to evaluate the performance. The maxillary and mandibular VGG19 models showed minimum mean errors of 0.84\u00a0mm and 1.06\u00a0mm for teeth landmark detection, respectively. Analysis of Cohen\u2019s weighted kappa coefficient indicated that crowding categorization performance was best in VGG19 (0.73), decreasing in the order of VGG16, ResNet101, and ResNet50. For tooth extraction, the maxillary VGG19 model showed the highest accuracy (0.922) and AUC (0.961). By utilizing deep learning with orthodontic photographs, dental crowding categorization and diagnosis of orthodontic extraction were successfully determined. This suggests that AI can assist clinicians in the diagnosis and decision making of treatment plans.<\/jats:p>","DOI":"10.1038\/s41598-023-32514-7","type":"journal-article","created":{"date-parts":[[2023,3,30]],"date-time":"2023-03-30T10:03:56Z","timestamp":1680170636000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":77,"title":["Evaluation of artificial intelligence model for crowding categorization and extraction diagnosis using intraoral photographs"],"prefix":"10.1038","volume":"13","author":[{"given":"Jiho","family":"Ryu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ye-Hyun","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tae-Woo","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seok-Ki","family":"Jung","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,3,30]]},"reference":[{"key":"32514_CR1","doi-asserted-by":"publisher","first-page":"384","DOI":"10.1016\/j.ajodo.2015.08.019","volume":"149","author":"SH Choi","year":"2016","unstructured":"Choi, S. H., Kim, J. S., Cha, J. Y. & Hwang, C. J. Effect of malocclusion severity on oral health-related quality of life and food intake ability in a Korean population. Am. J. Orthod. Dentofac. Orthop. 149, 384\u2013390 (2016).","journal-title":"Am. J. Orthod. Dentofac. Orthop."},{"key":"32514_CR2","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1038\/s41368-018-0012-3","volume":"10","author":"J Zou","year":"2018","unstructured":"Zou, J., Meng, M., Law, C. S., Rao, Y. & Zhou, X. Common dental diseases in children and malocclusion. Int. J. Oral Sci. 10, 7. https:\/\/doi.org\/10.1038\/s41368-018-0012-3 (2018).","journal-title":"Int. J. Oral Sci."},{"key":"32514_CR3","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1093\/ortho\/26.3.195","volume":"26","author":"PA Mossey","year":"1999","unstructured":"Mossey, P. A. The heritability of malocclusion: Part 2. The influence of genetics in malocclusion. Br. J. Orthod. 26, 195\u2013203 (1999).","journal-title":"Br. J. Orthod."},{"key":"32514_CR4","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1016\/0002-9416(83)90320-2","volume":"83","author":"RP Howe","year":"1983","unstructured":"Howe, R. P., McNamara, J. A. Jr. & O\u2019connor, K. An examination of dental crowding and its relationship to tooth size and arch dimension. Am. J. Orthod. 83, 363\u2013373 (1983).","journal-title":"Am. J. Orthod."},{"key":"32514_CR5","first-page":"280","volume":"16","author":"L Bailey","year":"2001","unstructured":"Bailey, L., Haltiwanger, L., Blakey, G. & Proffit, W. R. Who seeks surgical-orthodontic treatment: A current review. Int. J. Adult Orthodon. Orthognath. Surg. 16, 280\u2013292 (2001).","journal-title":"Int. J. Adult Orthodon. Orthognath. Surg."},{"key":"32514_CR6","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1016\/0096-6347(47)90123-3","volume":"33","author":"HN Nance","year":"1947","unstructured":"Nance, H. N. The limitations of orthodontic treatment II Diagnosis and treatment in the permanent dentition. Am. J. Orthod. Oral Surg. 33, 253\u2013301 (1947).","journal-title":"Am. J. Orthod. Oral Surg."},{"key":"32514_CR7","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1016\/j.ajodo.2005.10.003","volume":"129","author":"KJ Skidmore","year":"2006","unstructured":"Skidmore, K. J., Brook, K. J., Thomson, W. M. & Harding, W. J. Factors influencing treatment time in orthodontic patients. Am. J. Orthod. Dentofac. Orthop. 129, 230\u2013238 (2006).","journal-title":"Am. J. Orthod. Dentofac. Orthop."},{"key":"32514_CR8","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1053\/j.sodo.2010.08.008","volume":"17","author":"CH Kau","year":"2011","unstructured":"Kau, C. H., Olim, S. & Nguyen, J. T. The future of orthodontic diagnostic records. Semin. Orthod. 17, 39\u201345 (2011).","journal-title":"Semin. Orthod."},{"key":"32514_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1111\/j.1601-6343.2010.01503.x","volume":"14","author":"PS Fleming","year":"2011","unstructured":"Fleming, P. S., Marinho, V. & Johal, A. Orthodontic measurements on digital study models compared with plaster models: A systematic review. Orthod. Craniofac. Res. 14, 1\u201316 (2011).","journal-title":"Orthod. Craniofac. Res."},{"key":"32514_CR10","doi-asserted-by":"publisher","first-page":"212","DOI":"10.1016\/0889-5406(91)70058-5","volume":"100","author":"UK Han","year":"1991","unstructured":"Han, U. K., Vig, K. W., Weintraub, J. A., Vig, P. S. & Kowalski, C. J. Consistency of orthodontic treatment decisions relative to diagnostic records. Am. J. Orthod. Dentofac. Orthop. 100, 212\u2013219 (1991).","journal-title":"Am. J. Orthod. Dentofac. Orthop."},{"key":"32514_CR11","doi-asserted-by":"publisher","first-page":"885","DOI":"10.2319\/081908-436.1","volume":"79","author":"K Takada","year":"2009","unstructured":"Takada, K., Yagi, M. & Horiguchi, E. Computational formulation of orthodontic tooth-extraction decisions. Part I: To extract or not to extract. Angle Orthod. 79, 885\u2013891 (2009).","journal-title":"Angle Orthod."},{"key":"32514_CR12","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1016\/j.ajodo.2015.07.030","volume":"149","author":"SK Jung","year":"2016","unstructured":"Jung, S. K. & Kim, T. W. New approach for the diagnosis of extractions with neural network machine learning. Am. J. Orthod. Dentofac. Orthop. 149, 127\u2013133 (2016).","journal-title":"Am. J. Orthod. Dentofac. Orthop."},{"key":"32514_CR13","doi-asserted-by":"publisher","first-page":"2037","DOI":"10.1038\/s41598-018-38439-w","volume":"9","author":"P Li","year":"2019","unstructured":"Li, P. et al. Orthodontic treatment planning based on artificial neural networks. Sci. Rep. 9, 2037. https:\/\/doi.org\/10.1038\/s41598-018-38439-w (2019).","journal-title":"Sci. Rep."},{"issue":"Suppl 2","key":"32514_CR14","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1111\/ocr.12502","volume":"24","author":"L Etemad","year":"2021","unstructured":"Etemad, L. et al. Machine learning from clinical data sets of a contemporary decision for orthodontic tooth extraction. Orthod. Craniofac. Res. 24(Suppl 2), 193\u2013200 (2021).","journal-title":"Orthod. Craniofac. Res."},{"key":"32514_CR15","doi-asserted-by":"publisher","first-page":"130","DOI":"10.1186\/s12903-021-01513-3","volume":"21","author":"W Shin","year":"2021","unstructured":"Shin, W. et al. Deep learning based prediction of necessity for orthognathic surgery of skeletal malocclusion using cephalogram in Korean individuals. BMC Oral Health 21, 130. https:\/\/doi.org\/10.1186\/s12903-021-01513-3 (2021).","journal-title":"BMC Oral Health"},{"key":"32514_CR16","doi-asserted-by":"publisher","first-page":"1986","DOI":"10.1097\/SCS.0000000000005650","volume":"30","author":"HI Choi","year":"2019","unstructured":"Choi, H. I. et al. Artificial intelligent model with neural network machine learning for the diagnosis of orthognathic surgery. J. Craniofac. Surg. 30, 1986\u20131989 (2019).","journal-title":"J. Craniofac. Surg."},{"key":"32514_CR17","doi-asserted-by":"publisher","first-page":"2124","DOI":"10.3390\/app10062124","volume":"10","author":"KS Lee","year":"2020","unstructured":"Lee, K. S., Ryu, J. J., Jang, H. S., Lee, D. Y. & Jung, S. K. Deep convolutional neural networks based analysis of cephalometric radiographs for differential diagnosis of orthognathic surgery indications. Appl. Sci. 10, 2124. https:\/\/doi.org\/10.3390\/app10062124 (2020).","journal-title":"Appl. Sci."},{"key":"32514_CR18","doi-asserted-by":"publisher","first-page":"903","DOI":"10.2319\/022019-127.1","volume":"89","author":"JH Park","year":"2019","unstructured":"Park, J. H. et al. Automated identification of cephalometric landmarks: Part 1\u2014comparisons between the latest deep-learning methods YOLOV3 and SSD. Angle Orthod. 89, 903\u2013909 (2019).","journal-title":"Angle Orthod."},{"key":"32514_CR19","doi-asserted-by":"publisher","first-page":"33581","DOI":"10.1038\/srep33581","volume":"6","author":"C Lindner","year":"2016","unstructured":"Lindner, C. et al. Fully automatic system for accurate localisation and analysis of cephalometric landmarks in lateral cephalograms. Sci. Rep. 6, 33581. https:\/\/doi.org\/10.1038\/srep33581 (2016).","journal-title":"Sci. Rep."},{"key":"32514_CR20","doi-asserted-by":"publisher","first-page":"449","DOI":"10.1016\/j.ajodo.2017.06.028","volume":"153","author":"J Mont\u00fafar","year":"2018","unstructured":"Mont\u00fafar, J., Romero, M. & Scougall-Vilchis, R. J. Automatic 3-dimensional cephalometric landmarking based on active shape models in related projections. Am. J. Orthod. Dentofacial Orthop. 153, 449\u2013458 (2018).","journal-title":"Am. J. Orthod. Dentofacial Orthop."},{"key":"32514_CR21","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1177\/0022034520901715","volume":"99","author":"HJ Yu","year":"2020","unstructured":"Yu, H. J. et al. Automated skeletal classification with lateral cephalometry based on artificial intelligence. J. Dent. Res. 99, 249\u2013256 (2020).","journal-title":"J. Dent. Res."},{"key":"32514_CR22","doi-asserted-by":"publisher","first-page":"454","DOI":"10.1186\/s12903-022-02466-x","volume":"22","author":"J Ryu","year":"2022","unstructured":"Ryu, J. et al. Application of deep learning artificial intelligence technique to the classification of clinical orthodontic photos. BMC Oral Health 22, 454. https:\/\/doi.org\/10.1186\/s12903-022-02466-x (2022).","journal-title":"BMC Oral Health"},{"key":"32514_CR23","doi-asserted-by":"publisher","first-page":"3840","DOI":"10.1038\/s41598-019-40414-y","volume":"9","author":"H Chen","year":"2019","unstructured":"Chen, H. et al. A deep learning approach to automatic teeth detection and numbering based on object detection in dental periapical films. Sci. Rep. 9, 3840. https:\/\/doi.org\/10.1038\/s41598-019-40414-y (2019).","journal-title":"Sci. Rep."},{"key":"32514_CR24","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"Y LeCun","year":"2015","unstructured":"LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436\u2013444 (2015).","journal-title":"Nature"},{"key":"32514_CR25","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","volume":"42","author":"G Litjens","year":"2017","unstructured":"Litjens, G. et al. A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60\u201388 (2017).","journal-title":"Med. Image Anal."},{"key":"32514_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.5624\/isd.2019.49.1.1","volume":"49","author":"JJ Hwang","year":"2019","unstructured":"Hwang, J. J., Jung, Y. H., Cho, B. H. & Heo, M. S. An overview of deep learning in the field of dentistry. Imaging Sci. Dent. 49, 1\u20137 (2019).","journal-title":"Imaging Sci. Dent."},{"key":"32514_CR27","doi-asserted-by":"publisher","first-page":"2017","DOI":"10.1162\/jocn_a_01544","volume":"33","author":"GW Lindsay","year":"2021","unstructured":"Lindsay, G. W. Convolutional neural networks as a model of the visual system: Past, present, and future. J. Cogn. Neurosci. 33, 2017\u20132031 (2021).","journal-title":"J. Cogn. Neurosci."},{"key":"32514_CR28","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","volume":"61","author":"J Schmidhuber","year":"2015","unstructured":"Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw. 61, 85\u2013117 (2015).","journal-title":"Neural Netw."},{"key":"32514_CR29","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","volume":"542","author":"A Esteva","year":"2017","unstructured":"Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115\u2013118 (2017).","journal-title":"Nature"},{"key":"32514_CR30","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1016\/j.ajodo.2007.01.018","volume":"132","author":"N Puri","year":"2007","unstructured":"Puri, N., Pradhan, K. L., Chandna, A., Sehgal, V. & Gupta, R. Biometric study of tooth size in normal, crowded, and spaced permanent dentitions. Am. J. Orthod. Dentofacial Orthop. 132, 279 (2007).","journal-title":"Am. J. Orthod. Dentofacial Orthop."},{"key":"32514_CR31","doi-asserted-by":"publisher","first-page":"762","DOI":"10.1016\/0002-9416(49)90148-7","volume":"35","author":"CW Carey","year":"1949","unstructured":"Carey, C. W. Linear arch dimension and tooth size: An evaluation of the bone and dental structures in cases involving the possible reduction of dental units in treatment. Am. J. Orthod. 35, 762\u2013775 (1949).","journal-title":"Am. J. Orthod."},{"key":"32514_CR32","doi-asserted-by":"publisher","first-page":"641","DOI":"10.2319\/020117-80.1","volume":"87","author":"N Saghafi","year":"2017","unstructured":"Saghafi, N. et al. Influence of clinicians\u2019 experience and gender on extraction decision in orthodontics. Angle Orthod. 87, 641\u2013650 (2017).","journal-title":"Angle Orthod."},{"key":"32514_CR33","doi-asserted-by":"publisher","first-page":"e432","DOI":"10.4317\/jced.55709","volume":"11","author":"A Evrard","year":"2019","unstructured":"Evrard, A., Tepedino, M., Cattaneo, P. M. & Cornelis, M. A. Which factors influence orthodontists in their decision to extract? A questionnaire survey. J. Clin. Exp. Dent. 11, e432-438 (2019).","journal-title":"J. Clin. Exp. Dent."},{"key":"32514_CR34","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1093\/ejo\/18.1.77","volume":"18","author":"R Ribarevski","year":"1996","unstructured":"Ribarevski, R., Vig, P., Vig, K. D., Weyant, R. & O\u2019Brien, K. Consistency of orthodontic extraction decisions. Eur. J. Orthod. 18, 77\u201380 (1996).","journal-title":"Eur. J. Orthod."},{"key":"32514_CR35","doi-asserted-by":"crossref","unstructured":"Miko\u0142ajczyk, A. & Grochowski, M. Data augmentation for improving deep learning in image classification problem. In 2018 International Interdisciplinary PhD Workshop (IIPhDW) 117\u2013122 (2018).","DOI":"10.1109\/IIPHDW.2018.8388338"},{"key":"32514_CR36","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 770\u2013778 (2016).","DOI":"10.1109\/CVPR.2016.90"},{"key":"32514_CR37","unstructured":"Simonyan K. & Zisserman A. Very deep convolutional networks for large-scale image recognition. Preprint at https:\/\/arxiv.org\/abs\/1409.1556 (2014)."},{"key":"32514_CR38","first-page":"473","volume":"34","author":"SJ Lee","year":"2004","unstructured":"Lee, S. J., Moon, S. C., Kim, T. W., Nahm, D. S. & Chang, Y. I. Tooth size and arch parameters of normal occlusion in a large Korean sample. Korean J. Orthod. 34, 473\u2013480 (2004).","journal-title":"Korean J. Orthod."},{"key":"32514_CR39","doi-asserted-by":"crossref","unstructured":"Selvaraju, R. R. et al. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision. 618\u2013626 (2017).","DOI":"10.1109\/ICCV.2017.74"},{"key":"32514_CR40","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1111\/j.2517-6161.1974.tb00994.x","volume":"36","author":"M Stone","year":"1974","unstructured":"Stone, M. Cross-validatory choice and assessment of statistical predictions. J. R Stat. Soc. Ser. B Stat. Methodol. 36, 111\u2013133 (1974).","journal-title":"J. R Stat. Soc. Ser. B Stat. Methodol."},{"key":"32514_CR41","doi-asserted-by":"publisher","first-page":"356","DOI":"10.3390\/jpm11050356","volume":"11","author":"YH Kim","year":"2021","unstructured":"Kim, Y. H. et al. Influence of the depth of the convolutional neural networks on an artificial intelligence model for diagnosis of orthognathic surgery. J. Pers. Med. 11, 356. https:\/\/doi.org\/10.3390\/jpm11050356 (2021).","journal-title":"J. Pers. Med."},{"key":"32514_CR42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3295748","volume":"51","author":"S Pouyanfar","year":"2019","unstructured":"Pouyanfar, S. et al. A survey on deep learning. ACM Comput. Surv. 51, 1\u201336 (2019).","journal-title":"ACM Comput. Surv."},{"key":"32514_CR43","doi-asserted-by":"publisher","first-page":"393","DOI":"10.1016\/S0889-5406(96)70121-X","volume":"109","author":"S Baumrind","year":"1996","unstructured":"Baumrind, S., Korn, E. L., Boyd, R. L. & Maxwell, R. The decision to extract: part II. Analysis of clinicians\u2019 stated reasons for extraction. Am. J. Orthod. Dentofac. Orthop. 109, 393\u2013402 (1996).","journal-title":"Am. J. Orthod. Dentofac. Orthop."},{"key":"32514_CR44","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1590\/2176-9451.19.4.107-113.oar","volume":"19","author":"GD Correia","year":"2014","unstructured":"Correia, G. D., Habib, F. A. & Vogel, C. J. Tooth-size discrepancy: A comparison between manual and digital methods. Dental Press J. Orthod. 19, 107\u2013113 (2014).","journal-title":"Dental Press J. Orthod."},{"key":"32514_CR45","doi-asserted-by":"publisher","first-page":"657","DOI":"10.1016\/j.ajodo.2007.04.038","volume":"135","author":"J Sandler","year":"2009","unstructured":"Sandler, J. et al. Quality of clinical photographs taken by orthodontists, professional photographers, and orthodontic auxiliaries. Am. J. Orthod. Dentofac. Orthop. 135, 657\u2013662 (2009).","journal-title":"Am. J. Orthod. Dentofac. Orthop."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-32514-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-32514-7","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-32514-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T10:03:50Z","timestamp":1729159430000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-023-32514-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,30]]},"references-count":45,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["32514"],"URL":"https:\/\/doi.org\/10.1038\/s41598-023-32514-7","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,30]]},"assertion":[{"value":"31 January 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 March 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"5177"}}