{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T11:56:53Z","timestamp":1790769413423,"version":"4.1.0"},"reference-count":31,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T00:00:00Z","timestamp":1790726400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>(1) Background: Cervical vertebral maturation (CVM) staging on lateral cephalograms informs the timing of orthodontic treatment, and automated deep-learning approaches are widely studied. A recurring but under-examined obstacle is that public CVM datasets are severely imbalanced across the six stages. Using a public dataset and a rigorous eight-seed protocol, we quantify how strongly imbalance constrains performance, whether common imbalance-handling and ordinal-aware loss functions overcome it, and whether a clinically motivated three-group reformulation is more tractable than the native six-stage task. (2) Methods: Using the public Aariz dataset (1000 lateral cephalograms with expert CVM-stage labels; 700\/150\/150 train\/validation\/test, one radiograph per patient), we trained an ImageNet-pretrained ResNet-18 under four strategies: unweighted cross-entropy (baseline), class-weighted cross-entropy with balanced sampling, focal loss, and a rank-consistent ordinal (CORAL) head. Each configuration was trained with eight random seeds. Both tasks were evaluated over this multi-seed protocol, and sensitivity analyses examined the ordinal model\u2019s sampling scheme and the focal-loss focusing parameter. Performance was assessed by accuracy, macro-F1, mean absolute error in stages (MAE), and quadratic weighted kappa (QWK), and compared across strategies with the Kruskal\u2013Wallis test; secondary pairwise contrasts were Holm-corrected, and Wilson binomial confidence intervals (accuracy), seed-level confidence intervals (macro-F1), and analytically derived majority and frequency-weighted random classifiers were added as trivial reference baselines. Both the native six-stage task and a three-group scheme (pre-peak = CS1-3, peak = CS4, post-peak = CS5-6) were evaluated. (3) Results: On the six-stage task the extreme imbalance (CS1 n = 18 vs. CS5 n = 311 in training) produced poor minority-stage recognition (macro-F1 \u2248 0.22\u20130.25), with the rarest stages essentially unclassified even after imbalance handling. Reframing into three clinically meaningful groups roughly doubled macro-F1 (to \u22480.47\u20130.51) across all strategies. Critically, in the three-group setting no loss function significantly outperformed the others: across eight seeds the Kruskal\u2013Wallis test was non-significant for every metric (macro-F1 p = 0.09, QWK p = 0.05, MAE p = 0.07, accuracy p = 0.41), and the unweighted baseline matched the best specialized loss on macro-F1 (0.507 \u00b1 0.045 vs. ordinal 0.505 \u00b1 0.044). In secondary pairwise comparisons the ordinal loss nominally exceeded the weighted and focal losses (uncorrected p = 0.04 and p = 0.01), but after Holm correction only the comparison with focal loss remained significant (adjusted p = 0.03), and no pairwise difference survived a conservative six-comparison Bonferroni bound; focal loss was the least stable. All strategies clearly exceeded trivial baselines on macro-F1 (majority classifier 0.259; frequency-weighted random \u22480.333), although the majority classifier\u2019s accuracy (0.633) exceeded that of every trained model. (4) Conclusions: On this imbalanced public dataset, the dominant factor associated with CVM classification performance was the problem formulation rather than the loss function: a clinically grounded three-group scheme was far more tractable than six-stage classification, whereas specialized imbalance and ordinal losses did not reliably outperform a simple baseline. These findings, established under a fixed backbone, training budget, and a single public dataset, caution against assuming that loss-level fixes resolve CVM imbalance, and highlight data scale and problem formulation as the primary levers. Multi-dataset external validation is a necessary next step.<\/jats:p>","DOI":"10.3390\/diagnostics16193183","type":"journal-article","created":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T10:59:25Z","timestamp":1790765965000},"page":"3183","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Problem Formulation Outweighs Loss-Function Choice in Deep-Learning Cervical Vertebral Maturation Staging: A Multi-Seed Study on an Imbalanced Public Benchmark"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9840-4211","authenticated-orcid":false,"given":"Nazl\u0131","family":"Tokatl\u0131","sequence":"first","affiliation":[{"name":"Department of Computer Engineering, Faculty of Engineering and Natural Sciences, Istanbul Health and Technology University, 34275 Istanbul, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1053\/j.sodo.2005.04.005","article-title":"The cervical vertebral maturation (CVM) method for the assessment of optimal treatment timing in dentofacial orthopedics","volume":"11","author":"Baccetti","year":"2005","journal-title":"Semin. Orthod."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"133","DOI":"10.2319\/111517-787.1","article-title":"The cervical vertebral maturation method: A user\u2019s guide","volume":"88","author":"McNamara","year":"2018","journal-title":"Angle Orthod."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"100874","DOI":"10.1016\/j.ortho.2024.100874","article-title":"Inter-observer and intra-observer agreement of cervical vertebral maturation staging: A systematic review and meta-analysis","volume":"22","author":"Hussain","year":"2024","journal-title":"Int. Orthod."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1016\/j.ajodo.2015.12.013","article-title":"Reliability of cervical vertebral maturation staging","volume":"150","author":"Rainey","year":"2016","journal-title":"Am. J. Orthod. Dentofac. Orthop."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"642","DOI":"10.1016\/j.jobcr.2023.08.005","article-title":"Cervical vertebral maturation assessment using various machine learning techniques on lateral cephalogram: A systematic literature review","volume":"13","author":"Rana","year":"2023","journal-title":"J. Oral Biol. Craniofacial Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"629","DOI":"10.1007\/s11282-023-00678-7","article-title":"Deep convolutional neural network\u2014The evaluation of cervical vertebrae maturation","volume":"39","author":"Akay","year":"2023","journal-title":"Oral Radiol."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Atici, S.F., Ansari, R., Allareddy, V., Suhaym, O., Cetin, A.E., and Elnagar, M.H. (2022). Fully automated determination of the cervical vertebrae maturation stages using deep learning with directional filters. PLoS ONE, 17.","DOI":"10.1371\/journal.pone.0269198"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1111\/ocr.12644","article-title":"AggregateNet: A deep learning model for automated classification of cervical vertebrae maturation stages","volume":"26","author":"Atici","year":"2023","journal-title":"Orthod. Craniofacial Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"232","DOI":"10.34172\/joddd.41114","article-title":"Determination of cervical vertebral maturation using machine learning in lateral cephalograms","volume":"18","author":"Kavousinejad","year":"2024","journal-title":"J. Dent. Res. Dent. Clin. Dent. Prospect."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kim, E.G., Oh, I.S., So, J.E., Kang, J., Le, V.N.T., Tak, M.K., and Lee, D.W. (2021). Estimating cervical vertebral maturation with a lateral cephalogram using the convolutional neural network. J. Clin. Med., 10.","DOI":"10.3390\/jcm10225400"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"20220070","DOI":"10.1259\/dmfr.20220070","article-title":"Convolutional neural network-based automatic cervical vertebral maturation classification method","volume":"51","author":"Li","year":"2022","journal-title":"Dentomaxillofac. Radiol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"108","DOI":"10.25259\/APOS_322_2024","article-title":"Automated cervical vertebral maturation staging using deep learning: Enhancing accuracy through random oversampling and memory optimization","volume":"16","author":"Norman","year":"2024","journal-title":"APOS Trends Orthod."},{"key":"ref_13","first-page":"e175","article-title":"Artificial intelligence-based determination of the pubertal growth spurt by cervical vertebrae maturation in lateral cephalometric radiographs","volume":"137","author":"Radwan","year":"2024","journal-title":"Oral Surg. Oral Med. Oral Pathol. Oral Radiol."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Seo, H., Hwang, J., Jeong, T., and Shin, J. (2021). Comparison of deep learning models for cervical vertebral maturation stage classification on lateral cephalometric radiographs. J. Clin. Med., 10.","DOI":"10.3390\/jcm10163591"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhou, J., Zhou, H., Pu, L., Gao, Y., Tang, Z., Yang, Y., You, M., Yang, Z., Lai, W., and Long, H. (2021). Development of an artificial intelligence system for the automatic evaluation of cervical vertebral maturation status. Diagnostics, 11.","DOI":"10.3390\/diagnostics11122200"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1016\/j.neunet.2018.07.011","article-title":"A systematic study of the class imbalance problem in convolutional neural networks","volume":"106","author":"Buda","year":"2018","journal-title":"Neural Netw."},{"key":"ref_17","first-page":"79","article-title":"A systematic review on imbalanced data challenges in machine learning: Applications and solutions","volume":"52","author":"Kaur","year":"2019","journal-title":"ACM Comput. Surv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Cui, Y., Jia, M., Lin, T.Y., Song, Y., and Belongie, S. (2019). Class-balanced loss based on effective number of samples. 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.","DOI":"10.1109\/CVPR.2019.00949"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1613\/jair.953","article-title":"SMOTE: Synthetic minority over-sampling technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J. Artif. Intell. Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1336","DOI":"10.1038\/s41597-025-05542-3","article-title":"A benchmark dataset for automatic cephalometric landmark detection and CVM stage classification","volume":"12","author":"Khalid","year":"2025","journal-title":"Sci. Data"},{"key":"ref_21","unstructured":"Khalid, M.A., Zulfiqar, K., Bashir, U., Shaheen, A., Iqbal, R., Rizwan, Z., Rizwan, G., and Fraz, M.M. (2026, July 15). Aariz Cephalometric Dataset [Dataset]. Figshare, 2025. Available online: https:\/\/figshare.com\/articles\/dataset\/Aariz_Cephalometric_Dataset\/27986417."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on image data augmentation for deep learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J. Big Data"},{"key":"ref_25","first-page":"8024","article-title":"PyTorch: An imperative style, high-performance deep learning library","volume":"32","author":"Paszke","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst. NeurIPS"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollar, P. (2017). Focal loss for dense object detection. 2017 IEEE International Conference on Computer Vision (ICCV), IEEE.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1016\/j.patrec.2020.11.008","article-title":"Rank consistent ordinal regression for neural networks with application to age estimation","volume":"140","author":"Cao","year":"2020","journal-title":"Pattern Recognit. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Niu, Z., Zhou, M., Wang, L., Gao, X., and Hua, G. (2016). Ordinal regression with multiple output CNN for age estimation. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.","DOI":"10.1109\/CVPR.2016.532"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"941","DOI":"10.1007\/s10044-023-01181-9","article-title":"Deep neural networks for rank-consistent ordinal regression based on conditional probabilities","volume":"26","author":"Shi","year":"2023","journal-title":"Pattern Anal. Appl."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1037\/h0026256","article-title":"Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit","volume":"70","author":"Cohen","year":"1968","journal-title":"Psychol. Bull."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Shetty, S., Talaat, W., Al-Rawi, N., AlKawas, S., Sadek, M., Rego, R., Gaballah, K., Narasimhan, S., Alsaegh, M.A., and Hamdoon, Z. (2026). Assessment of pre-trained deep learning models in the detection of metal artifacts in axial cone-beam tomography slices. Dentomaxillofac. Radiol., twag037.","DOI":"10.1093\/dmfr\/twag037"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/19\/3183\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T11:12:16Z","timestamp":1790766736000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/19\/3183"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,30]]},"references-count":31,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2026,10]]}},"alternative-id":["diagnostics16193183"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16193183","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,30]]}}}