{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T11:14:12Z","timestamp":1791976452972,"version":"4.4.0"},"reference-count":26,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Several studies have used electronic health records (EHRs) to build machine learning models predicting the likelihood of developing gestational diabetes mellitus (GDM) later in pregnancy, but none have described validation of the GDM \u201clabel\u201d within the EHRs.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study examines the accuracy of GDM diagnoses in EHRs compared with a clinical team database (CTD) and their impact on machine learning models.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n EHRs from 2018 to 2022 were validated against CTD data to identify true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). Logistic regression models were trained and tested using both EHR and validated labels, whereafter simulated label noise was introduced to increase FP and FN rates. Model performance was assessed using the area under the receiver operating characteristic curve (ROC AUC) and average precision (AP).<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Among 3952 patients, 3388 (85.7%) were correctly identified with GDM in both databases, while 564 cases lacked a GDM label in EHRs, and 771 were missing a corresponding CTD label. Overall, 32,928 (87.5%) of cases were TN, 3388 (9%) TP, 771 (2%) FP, and 564 (1.5%) FN. The model trained and tested with validated labels achieved an ROC AUC of 0.817 and an AP of 0.450, whereas the same model tested using EHR labels achieved 0.814 and 0.395, respectively. Increased label noise during training led to gradual declines in ROC AUC and AP, while noise in the test set, especially elevated FP rates, resulted in marked performance drops.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Discrepancies between EHR and CTD diagnoses had a limited impact on model training but significantly affected performance evaluation when present in the test set, emphasizing the importance of accurate data validation.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/72938","type":"journal-article","created":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T09:30:07Z","timestamp":1750152607000},"page":"e72938-e72938","source":"Crossref","is-referenced-by-count":7,"title":["Label Accuracy in Electronic Health Records and Its Impact on Machine Learning Models for Early Prediction of Gestational Diabetes: 3-Step Retrospective Validation Study"],"prefix":"10.2196","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7862-7714","authenticated-orcid":false,"given":"Mark","family":"Germaine","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2020-1585","authenticated-orcid":false,"given":"Amy C","family":"O'Higgins","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8327-9016","authenticated-orcid":false,"given":"Brendan","family":"Egan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6429-6339","authenticated-orcid":false,"given":"Graham","family":"Healy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,8,21]]},"reference":[{"issue":"5","key":"R1","doi-asserted-by":"publisher","first-page":"326","DOI":"10.1111\/j.1524-4733.2007.00186.x","article-title":"Using real-world data for coverage and payment decisions: the ISPOR real-world data task force report","volume":"10","author":"Garrison","journal-title":"Value Health"},{"issue":"9","key":"R2","doi-asserted-by":"publisher","first-page":"1033","DOI":"10.1002\/pds.4297","article-title":"Good practices for real-world data studies of treatment and\/or comparative effectiveness: recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making","volume":"26","author":"Berger","journal-title":"Pharmacoepidemiol Drug Saf"},{"issue":"4","key":"R3","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1007\/s40471-018-0165-9","article-title":"Using machine learning to identify health outcomes from electronic health record data","volume":"5","author":"Wong","journal-title":"Curr Epidemiol Rep"},{"issue":"5","key":"R4","doi-asserted-by":"publisher","first-page":"1007","DOI":"10.1093\/jamia\/ocv180","article-title":"Extracting information from the text of electronic medical records to improve case detection: a systematic review","volume":"23","author":"Ford","journal-title":"J Am Med Inform Assoc"},{"issue":"Fall","key":"R5","first-page":"1c","volume":"10","author":"Bowman","journal-title":"Perspect Health Inf Manag"},{"key":"R6","unstructured":"Fr\u00e9nay B Kaban A . A comprehensive introduction to label noise. 2014. URL: http:\/\/hdl.handle.net\/2078.1\/156597 [Accessed 09-08-2025]"},{"issue":"1","key":"R7","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-024-02581-5","article-title":"Addressing label noise for electronic health records: insights from computer vision for tabular data","volume":"24","author":"Yang","journal-title":"BMC Med Inform Decis Mak"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1183\/20734735.0344-2018","article-title":"How to validate a diagnosis recorded in electronic health records","volume":"15","author":"Nissen","journal-title":"Breathe (Sheff)"},{"issue":"3","key":"R9","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1145\/3446776","article-title":"Understanding deep learning (still) requires rethinking generalization","volume":"64","author":"Zhang","journal-title":"Commun ACM"},{"issue":"9","key":"R10","doi-asserted-by":"publisher","first-page":"825","DOI":"10.1056\/NEJMsa1102519","article-title":"Electronic health records and quality of diabetes care","volume":"365","author":"Cebul","journal-title":"N Engl J Med"},{"key":"R11","doi-asserted-by":"crossref","unstructured":"Veinot TC Zheng K Lowery JC Souden M Keith R . Using electronic health record systems in diabetes care. Presented at: IHI \u201910; Nov 11-12, 2010:240-249; Arlington, VA. Nov 11, 2010.[doi: 10.1145\/1882992.1883026]","DOI":"10.1145\/1882992.1883026"},{"issue":"11","key":"R12","doi-asserted-by":"publisher","first-page":"1544","DOI":"10.1001\/jamainternmed.2018.3763","article-title":"Potential biases in machine learning algorithms using electronic health record data","volume":"178","author":"Gianfrancesco","journal-title":"JAMA Intern Med"},{"key":"R13","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2022.102378","article-title":"Machine learning-based models for gestational diabetes mellitus prediction before 24\u201328 weeks of pregnancy: a review","volume":"132","author":"Mennickent","journal-title":"Artif Intell Med"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-021-00549-7","article-title":"Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review","volume":"5","author":"de Hond","journal-title":"NPJ Digit Med"},{"issue":"3","key":"R15","doi-asserted-by":"publisher","DOI":"10.2196\/26634","article-title":"Machine learning prediction models for gestational diabetes mellitus: meta-analysis","volume":"24","author":"Zhang","journal-title":"J Med Internet Res"},{"issue":"8","key":"R16","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","article-title":"An introduction to ROC analysis","volume":"27","author":"Fawcett","journal-title":"Pattern Recognit Lett"},{"key":"R17","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2021.104510","article-title":"The need to separate the wheat from the chaff in medical informatics: introducing a comprehensive checklist for the (self)-assessment of medical AI studies","volume":"153","author":"Cabitza","journal-title":"Int J Med Inform"},{"issue":"4","key":"R18","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1001\/jamaoncol.2023.6969","article-title":"Undiagnosed cancer cases in the US during the first 10 months of the COVID-19 pandemic","volume":"10","author":"Burus","journal-title":"JAMA Oncol"},{"issue":"10","key":"R19","doi-asserted-by":"publisher","first-page":"e78","DOI":"10.2337\/dc24-1483","article-title":"Lack of data sharing despite data availability statements in studies using machine learning models for prediction of gestational diabetes mellitus","volume":"47","author":"Germaine","journal-title":"Diabetes Care"},{"issue":"6","key":"R20","doi-asserted-by":"publisher","first-page":"1987","DOI":"10.1111\/jog.14785","article-title":"Assessing the concordance and accuracy between hospital discharge data, electronic health records, and register books for diagnosis of inpatient admissions of miscarriage: a retrospective linked data study","volume":"47","author":"San Lazaro Campillo","journal-title":"J Obstet Gynaecol Res"},{"key":"R21","doi-asserted-by":"publisher","DOI":"10.2147\/CLEP.S265619","article-title":"Validity of acute cardiovascular outcome diagnoses recorded in European electronic health records: a systematic review","volume":"12","author":"Davidson","journal-title":"Clin Epidemiol"},{"issue":"3","key":"R22","doi-asserted-by":"publisher","first-page":"1097","DOI":"10.1007\/s11845-025-03926-3","article-title":"Impact of changes in gestational diabetes mellitus diagnostic criteria during the COVID-19 pandemic","volume":"194","author":"Neville","journal-title":"Ir J Med Sci"},{"issue":"1","key":"R23","doi-asserted-by":"publisher","first-page":"1161","DOI":"10.1038\/s41467-022-28818-3","article-title":"Active label cleaning for improved dataset quality under resource constraints","volume":"13","author":"Bernhardt","journal-title":"Nat Commun"},{"issue":"6","key":"R24","doi-asserted-by":"publisher","first-page":"1533","DOI":"10.1109\/TMI.2022.3141425","article-title":"Improving medical images classification with label noise using dual-uncertainty estimation","volume":"41","author":"Ju","journal-title":"IEEE Trans Med Imaging"},{"issue":"20","key":"R25","doi-asserted-by":"publisher","first-page":"3736","DOI":"10.3390\/math10203736","article-title":"Noise models in classification: unified nomenclature, extended taxonomy and pragmatic categorization","volume":"10","author":"S\u00e1ez","journal-title":"Mathematics"},{"key":"R26","unstructured":"Gestational\u2010Diabetes label noise study. GitHub. URL: https:\/\/github.com\/MarkGermaine\/Label_Noise [Accessed 25-07-2025]"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e72938","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:03:23Z","timestamp":1782842603000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e72938"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,21]]},"references-count":26,"URL":"https:\/\/doi.org\/10.2196\/72938","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,21]]},"article-number":"v13i5e72938"}}