{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T11:05:48Z","timestamp":1780052748411,"version":"3.54.0"},"reference-count":22,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T00:00:00Z","timestamp":1780012800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background: Machine learning models are increasingly applied in clinical practice. However, conventional performance metrics such as sensitivity, specificity, and accuracy do not capture the structural characteristics of predictive probability distributions, limiting their utility for optimizing decision thresholds and supporting clinical decision-making. Methods: We propose the D-index, a novel metric designed to quantify asymmetry in predicted probability distributions by measuring the difference in skewness between positive and negative predictions. The D-index was evaluated using simulated datasets with skewness-dominant and kurtosis-dominant distributions at three performance levels (AUROC = 0.7, 0.8, and 0.9). Its practical utility was further validated using real-world MALDI-TOF mass spectrometry data for predicting vancomycin resistance in Enterococcus faecium, comparing three machine learning algorithms: deep learning, XGBoost, and random forest. Results: In simulated data, the D-index increased with model performance only in skewness-dominant distributions, demonstrating selective sensitivity to distributional asymmetry. In real-world data, the D-index distinguished models with similar conventional performance but different distributional structures. Models with higher D-index values retained a larger proportion of confidently classified cases while achieving meaningful accuracy improvements under a gray-zone strategy. In contrast, models with lower D-index values required exclusion of more cases to achieve comparable or greater accuracy gains. Conclusions: The D-index provides complementary information beyond traditional performance metrics by capturing structural properties of predictive distributions. It enables identification of models that are better suited for gray-zone strategies and supports more reliable clinical decision-making. This distribution-aware approach offers a practical tool for improving the safety, efficiency, and clinical utility of machine learning models in biomedical applications.<\/jats:p>","DOI":"10.3390\/diagnostics16111671","type":"journal-article","created":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T10:06:56Z","timestamp":1780049216000},"page":"1671","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Discriminative Index: A Novel Indicator for Evaluating Machine Learning Algorithms in Laboratory Medicine"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5581-6793","authenticated-orcid":false,"given":"Hsin-Yao","family":"Wang","sequence":"first","affiliation":[{"name":"Microbiology and Infection Diagnostics, Bruker Scientific, Billerica, MA 01821, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wan-Ying","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Medicine, University at Buffalo-Catholic Health System, Buffalo, NY 14220, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Sp\u00fcler","sequence":"additional","affiliation":[{"name":"Microbiology and Infection Diagnostics, Bruker Daltonics GmbH & Co. KG, 28359 Bremen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sven","family":"Brand","sequence":"additional","affiliation":[{"name":"Microbiology and Infection Diagnostics, Bruker Daltonics GmbH & Co. KG, 28359 Bremen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3260-4269","authenticated-orcid":false,"given":"Jia-Ruei","family":"Yu","sequence":"additional","affiliation":[{"name":"Department of Laboratory Medicine, Chang Gung Memorial Hospital at Linkou, Taoyuan 333423, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0501-6215","authenticated-orcid":false,"given":"Markus","family":"Kostrzewa","sequence":"additional","affiliation":[{"name":"Microbiology and Infection Diagnostics, Bruker Daltonics GmbH & Co. KG, 28359 Bremen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,5,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1201","DOI":"10.1056\/NEJMra2302038","article-title":"Artificial Intelligence and Machine Learning in Clinical Medicine, 2023","volume":"388","author":"Haug","year":"2023","journal-title":"N. Engl. J. Med."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.clinbiochem.2022.02.011","article-title":"Applications of machine learning in routine laboratory medicine: Current state and future directions","volume":"103","author":"Rabbani","year":"2022","journal-title":"Clin. Biochem."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"5979","DOI":"10.1038\/s41598-022-09954-8","article-title":"On evaluation metrics for medical applications of artificial intelligence","volume":"12","author":"Hicks","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"10444","DOI":"10.3389\/ebm.2025.10444","article-title":"Artificial intelligence in the diagnosis of uveal melanoma: Advances and applications","volume":"250","author":"Dadzie","year":"2025","journal-title":"Exp. Biol. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.jclinepi.2019.02.004","article-title":"A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models","volume":"110","author":"Christodoulou","year":"2019","journal-title":"J. Clin. Epidemiol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1093\/ije\/dyg054","article-title":"A grey zone for quantitative diagnostic and screening tests","volume":"32","author":"Coste","year":"2003","journal-title":"Int. J. Epidemiol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1373\/clinchem.2014.232629","article-title":"Current evidence and future perspectives on the effective practice of patient-centered laboratory medicine","volume":"61","author":"Hallworth","year":"2015","journal-title":"Clin. Chem."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, H.-Y., Liu, Y.-H., Tseng, Y.-J., Chung, C.-R., Lin, T.-W., Yu, J.-R., Huang, Y.-C., and Lu, J.-J. (2022). Investigating Unfavorable Factors That Impede MALDI-TOF-Based AI in Predicting Antibiotic Resistance. Diagnostics, 12.","DOI":"10.20944\/preprints202201.0131.v1"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Arevalillo, J.M., and Navarro, H. (2021). Skewness-Kurtosis Model-Based Projection Pursuit with Application to Summarizing Gene Expression Data. Mathematics, 9.","DOI":"10.3390\/math9090954"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"White, N., Parsons, R., Collins, G., and Barnett, A. (2023). Evidence of questionable research practices in clinical prediction models. BMC Med., 21.","DOI":"10.1186\/s12916-023-03048-6"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chou, Y.Y., Wang, M.S., Lin, C.F., Lee, Y.S., Lee, P.H., Huang, S.M., Wu, C.-L., and Lin, S.-Y. (2024). The application of machine learning for identifying frailty in older patients during hospital admission. BMC Med. Inform. Decis. Mak., 24.","DOI":"10.1186\/s12911-024-02684-z"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, H.Y., Hsieh, T.T., Chung, C.R., Chang, H.C., Horng, J.T., Lu, J.J., and Huang, J.-H. (2022). Efficiently Predicting Vancomycin Resistance of Enterococcus Faecium from MALDI-TOF MS Spectra Using a Deep Learning-Based Approach. Front. Microbiol., 13.","DOI":"10.3389\/fmicb.2022.821233"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"e0091321","DOI":"10.1128\/Spectrum.00913-21","article-title":"Clinically Applicable System for Rapidly Predicting Enterococcus faecium Susceptibility to Vancomycin","volume":"9","author":"Wang","year":"2021","journal-title":"Microbiol. Spectr."},{"key":"ref_14","unstructured":"Machado, A.F., Charpentier, A., Flachaire, E., Gallic, E., and Hu, F. (2024). Probabilistic Scores of Classifiers, Calibration is not Enough. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1177\/0272989X06295361","article-title":"Decision curve analysis: A novel method for evaluating prediction models","volume":"26","author":"Vickers","year":"2006","journal-title":"Med. Decis. Mak."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1093\/jamia\/ocz228","article-title":"A tutorial on calibration measurements and calibration models for clinical prediction models","volume":"27","author":"Huang","year":"2020","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1038\/s42256-024-00817-7","article-title":"The curious case of the test set AUROC","volume":"6","author":"Roberts","year":"2024","journal-title":"Nat. Mach. Intell."},{"key":"ref_18","unstructured":"Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J., Lakshminarayanan, B., and Snoek, J. (2019). Can you trust your model\u2019s uncertainty? evaluating predictive uncertainty under dataset shift. Proceedings of the 33rd International Conference on Neural Information Processing Systems [Internet], Curran Associates Inc.. Available online: https:\/\/dL.acm.org\/doi\/10.5555\/3454287.3455541."},{"key":"ref_19","unstructured":"Guo, C., Pleiss, G., Sun, Y., and Weinberger, K.Q. (2017). On Calibration of Modern Neural Networks. arXiv."},{"key":"ref_20","unstructured":"Nixon, J., Dusenberry, M.W., Zhang, L., Jerfel, G., and Tran, D. (2026, May 13). Measuring Calibration in Deep Learning. Available online: https:\/\/openaccess.thecvf.com\/content_CVPRW_2019\/html\/Uncertainty_and_Robustness_in_Deep_Visual_Learning\/Nixon_Measuring_Calibration_in_Deep_Learning_CVPRW_2019_paper.html."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"(2018). Methods for Evaluation of medical prediction Models, Tests And Biomarkers (MEMTAB) 2018 Symposium. Diagn. Progn. Res., 2, 12.","DOI":"10.1186\/s41512-018-0036-3"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1177\/0272989X211050909","article-title":"Model-Based ROC Curve: Examining the Effect of Case Mix and Model Calibration on the ROC Plot","volume":"42","author":"Sadatsafavi","year":"2022","journal-title":"Med. Decis. Mak."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/11\/1671\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T10:18:43Z","timestamp":1780049923000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/11\/1671"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,29]]},"references-count":22,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2026,6]]}},"alternative-id":["diagnostics16111671"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16111671","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,29]]}}}