{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T04:38:59Z","timestamp":1787027939606,"version":"build-2736575974"},"reference-count":30,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T00:00:00Z","timestamp":1770940800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: Vitamin B12 deficiency is a prevalent yet frequently underdiagnosed condition, largely due to the limited diagnostic accuracy of serum total B12 and the restricted availability of confirmatory biomarkers such as holotranscobalamin and methylmalonic acid. This study aimed to develop and validate explainable machine learning (ML) models capable of predicting vitamin B12 deficiency using only routinely available laboratory examinations, thereby supporting early detection within standard diagnostic workflows. Methods: This retrospective study included 51,630 adult patients who underwent concurrent vitamin B12 testing and routine laboratory evaluation between 2015 and 2025. An independent temporal validation cohort of 34,744 patients was used to assess generalizability. Eight supervised ML algorithms were developed within a four-stage experimental framework incorporating default modeling, probability-threshold optimization, hyperparameter tuning, and feature engineering. Model performance was evaluated using AUC-ROC, AUC-PR, sensitivity, specificity, F1 score, accuracy, Matthews correlation coefficient, and likelihood ratios. Model explainability and clinical utility were assessed using SHAP, LIME, and decision curve analysis. Results: Among all algorithms, CatBoost demonstrated the most balanced and clinically relevant performance. In the threshold-optimized configuration, the model achieved a sensitivity of 0.92, specificity of 0.67, F1 score of 0.82, AUC-ROC of 0.88, and AUC-PR of 0.86 in the test set. Temporal validation confirmed robust generalizability, with improved discrimination (AUC-ROC 0.90; AUC-PR 0.91) and stable calibration. Explainability analyses identified hematologic indices (MCV, HGB, HCT, RDW), iron-related markers, inflammatory measurands, and age as the most influential contributors, consistent with known pathophysiology. Conclusions: This study presents a large-scale, explainable, and temporally validated ML framework for predicting vitamin B12 deficiency using routine laboratory data alone. The model demonstrates strong diagnostic performance, biological plausibility, and potential for seamless integration into laboratory and clinical decision-support systems, enabling cost-effective and early identification of patients at risk.<\/jats:p>","DOI":"10.3390\/diagnostics16040563","type":"journal-article","created":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T16:09:32Z","timestamp":1770998972000},"page":"563","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Development and Temporal Validation of Explainable Machine Learning Models for Predicting Vitamin B12 Deficiency Using Routine Laboratory Analytes"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5999-3399","authenticated-orcid":false,"given":"Ferhat","family":"Demirci","sequence":"first","affiliation":[{"name":"Department of Medical Biochemistry, \u0130zmir Tepecik Training and Research Hospital, University of Health Sciences T\u00fcrkiye, Gaziler Street 468, Yeni\u015fehir, Konak, 35120 \u0130zmir, T\u00fcrkiye"},{"name":"Department of Neurosciences, Institute of Health Sciences, Dokuz Eyl\u00fcl University, 35340 \u0130zmir, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Oktay","family":"Y\u0131ld\u0131r\u0131m","sequence":"additional","affiliation":[{"name":"Information Technology Department, Dokuz Eyl\u00fcl University Rectorate, 35340 \u0130zmir, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aylin","family":"Demirci","sequence":"additional","affiliation":[{"name":"Department of Family Medicine, \u0130zmir Tepecik Training and Research Hospital, University of Health Sciences T\u00fcrkiye, Gaziler Street 468, Yeni\u015fehir, Konak, 35120 \u0130zmir, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"P\u0131nar","family":"Akan","sequence":"additional","affiliation":[{"name":"Department of Neurosciences, Institute of Health Sciences, Dokuz Eyl\u00fcl University, 35340 \u0130zmir, T\u00fcrkiye"},{"name":"Department of Medical Biochemistry, Faculty of Medicine, Dokuz Eyl\u00fcl University, 35340 \u0130zmir, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"299","DOI":"10.3390\/nu2030299","article-title":"Vitamin B12 in Health and Disease","volume":"2","author":"Samman","year":"2010","journal-title":"Nutrients"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1056\/NEJMcp1113996","article-title":"Vitamin B12 Deficiency","volume":"368","author":"Stabler","year":"2013","journal-title":"N. Engl. J. Med."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hannibal, L., Lysne, V., Bj\u00f8rke-Monsen, A.-L., Behringer, S., Gr\u00fcnert, S.C., Spiekerkoetter, U., Jacobsen, D.W., and Blom, H.J. (2016). Biomarkers and Algorithms for the Diagnosis of Vitamin B12 Deficiency. Front. Mol. Biosci., 3.","DOI":"10.3389\/fmolb.2016.00027"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"S17","DOI":"10.1177\/15648265080292S104","article-title":"Discussion: Folate and Vitamin B12 Metabolism: Overview and Interaction with Riboflavin, Vitamin B6, and Polymorphisms","volume":"29","author":"Stover","year":"2008","journal-title":"Food Nutr. Bull."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Tamune, H., Ukita, J., Hamamoto, Y., Tanaka, H., Narushima, K., and Yamamoto, N. (2019). Efficient Prediction of Vitamin B Deficiencies via Machine-Learning Using Routine Blood Test Results in Patients With Intense Psychiatric Episode. Front. Psychiatry, 10.","DOI":"10.3389\/fpsyt.2019.01029"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"100382","DOI":"10.1016\/j.imu.2020.100382","article-title":"Support vectors machine-based model for diagnosis of multiple sclerosis using the plasma levels of selenium, vitamin B12, and vitamin D3","volume":"20","author":"Sharifmousavi","year":"2020","journal-title":"Inform. Med. Unlocked"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Sancar, N., and Tabrizi, S.S. (2023). Machine learning approach for the detection of vitamin D level: A comparative study. BMC Med. Inform. Decis. Mak., 23.","DOI":"10.1186\/s12911-023-02323-z"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1177\/00045632211046805","article-title":"Machine learning predictive models of LDL-C in the population of eastern India and its comparison with directly measured and calculated LDL-C","volume":"59","author":"Anudeep","year":"2022","journal-title":"Ann. Clin. Biochem. Int. J. Lab. Med."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Alkhanbouli, R., Almadhaani, H.M.A., Alhosani, F., and Simsekler, M.C.E. (2025). The role of explainable artificial intelligence in disease prediction: A systematic literature review and future research directions. BMC Med. Inform. Decis. Mak., 25.","DOI":"10.1186\/s12911-025-02944-6"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"84","DOI":"10.18590\/mjm.2018.vol4.iss2.9","article-title":"Effect of Removing Outliers on Statistical Inference: Implications to Interpretation of Experimental Data in Medical Research","volume":"4","author":"Gress","year":"2018","journal-title":"Marshall J. Med."},{"key":"ref_11","unstructured":"NICE Guideline (2025, October 04). Vitamin B12 Deficiency in over 16s: Diagnosis and Management. Available online: https:\/\/www.nice.org.uk\/guidance\/ng239."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1007\/s11739-022-03101-x","article-title":"A comparison of machine learning algorithms in predicting COVID-19 prognostics","volume":"18","author":"Ustebay","year":"2023","journal-title":"Intern. Emerg. Med."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lipton, Z.C., Elkan, C., and Naryanaswamy, B. (2014, January 15\u201319). Optimal Thresholding of Classifiers to Maximize F1 Measure. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Nancy, France.","DOI":"10.1007\/978-3-662-44851-9_15"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Unal, I. (2017). Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach. Comput. Math. Methods Med., 2017.","DOI":"10.1155\/2017\/3762651"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Hutter, F., Hoos, H.H., and Leyton-Brown, K. (2011, January 17\u201321). Sequential Model-Based Optimization for General Algorithm Configuration. Proceedings of the International Conference on Learning and Intelligent Optimization, Rome, Italy.","DOI":"10.1007\/978-3-642-25566-3_40"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Paplomatas, P., Kantartzi, M., Anagnostopoulos, K., and Vrahatis, A. (2025). An Artificial Intelligence and Biochemical Modelling Approach for LDL Cholesterol Prediction. Medinformatics.","DOI":"10.47852\/bonviewMEDIN52025919"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1192\/bjp.bp.107.039511","article-title":"Predictive value of folate, vitamin B 12 and homocysteine levels in late-life depression","volume":"192","author":"Kim","year":"2008","journal-title":"Br. J. Psychiatry"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mohammed, S.H., Ahamad, S., and Hameed, M.A. (2024). Enhancements in Random Forest Algorithms for Improving Healthcare Applications. Proceedings of the International Conference on Industrial Engineering and Operations Management, IEOM Society International.","DOI":"10.46254\/IN04.20240192"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"21639","DOI":"10.1038\/s41598-021-00218-5","article-title":"Explainable machine learning model for predicting spontaneous bacterial peritonitis in cirrhotic patients with ascites","volume":"11","author":"Hu","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_20","unstructured":"Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., and Gulin, A. (2018, January 3\u20138). CatBoost: Unbiased boosting with categorical features. Proceedings of the 32nd Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e70057","DOI":"10.1002\/msc.70057","article-title":"Differences in Ferritin, Vitamin D, and Vitamin B12 Between Fibromyalgia Patients and Healthy In-dividuals: A Systematic Review and Meta-Analysis","volume":"23","author":"Ismail","year":"2025","journal-title":"Musculoskelet. Care"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jensen, H.R., Laursen, M.F., Lildballe, D.L., Andersen, J.B., Nex\u00f8, E., and Licht, T.R. (2011). Effect of the vitamin B12-binding protein haptocorrin present in human milk on a panel of commensal and pathogenic bacteria. BMC Res. Notes, 4.","DOI":"10.1186\/1756-0500-4-208"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1016\/S1474-4422(06)70598-1","article-title":"Vitamin B12, folic acid, and the nervous system","volume":"5","author":"Reynolds","year":"2006","journal-title":"Lancet Neurol."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Simonenko, S.Y., Bogdanova, D.A., and Kuldyushev, N.A. (2024). Emerging Roles of Vitamin B12 in Aging and Inflammation. Int. J. Mol. Sci., 25.","DOI":"10.3390\/ijms25095044"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1097\/EDE.0000000000001027","article-title":"Can Hyperparameter Tuning Improve the Performance of a Super Learner?","volume":"30","author":"Wong","year":"2019","journal-title":"Epidemiology"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1038\/s43856-025-01043-x","article-title":"Estimating effects of serum vitamin B12 levels on psychiatric disorders and cognitive impairment: A Mendelian randomization study","volume":"5","author":"Lu","year":"2025","journal-title":"Commun. Med."},{"key":"ref_27","first-page":"39","article-title":"Retrospective Analysis of Pediatric Patients with Vitamin B12 Deficiency: Vitamin B12 and Children","volume":"5","author":"Duman","year":"2024","journal-title":"Chron. Precis. Med. Res."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Dastidar, R., and Sikder, K. (2022). Diagnostic reliability of serum active B12 (holo-transcobalamin) in true evaluation of vitamin B12 deficiency: Relevance in current perspective. BMC Res. Notes, 15.","DOI":"10.1186\/s13104-022-06224-8"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Gallina, A.L., Otay, S., de Frutos-Lucas, J., Buso, M., Martinez, P.M., Cashman, K.D., Kiely, M.E., and Astley, S. (2025). Hidden hunger in Europe: A review on determinants, fragmented policy responses, and implementation barriers. Front. Nutr., 12.","DOI":"10.3389\/fnut.2025.1669008"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"11654","DOI":"10.1038\/s41598-022-15245-z","article-title":"Temporal quality degradation in AI models","volume":"12","author":"Vela","year":"2022","journal-title":"Sci. Rep."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/4\/563\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T16:21:33Z","timestamp":1770999693000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/4\/563"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,13]]},"references-count":30,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["diagnostics16040563"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16040563","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,13]]}}}