{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,20]],"date-time":"2026-09-20T02:59:26Z","timestamp":1789873166625,"version":"4.0.1"},"reference-count":46,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR AI"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n A major challenge in using electronic health records (EHR) is the inconsistency of patient follow-up, resulting in right-censored outcomes. This becomes particularly problematic in long-horizon event predictions, such as autism and attention-deficit\/hyperactivity disorder (ADHD) diagnoses, where a significant number of patients are lost to follow-up before the outcome can be observed. Consequently, fully supervised methods such as binary classification (BC), which are trained to predict observed diagnoses, are substantially affected by the probability of sufficient follow-up, leading to biased results.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This empirical analysis aims to characterize BC\u2019s inherent limitations for long-horizon diagnosis prediction from EHR; and quantify the benefits of a specific time-to-event (TTE) approach, the discrete-time neural network (DTNN).<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n Records within the Duke University Health System EHR were analyzed, extracting features such as\n ICD-10<\/jats:italic>\n (\n International Classification of Diseases, Tenth Revision<\/jats:italic>\n ) diagnosis codes, medications, laboratories, and procedures. We compared a DTNN to 3 BC approaches and a deep Cox proportional hazards model across 4 clinical conditions to examine distributional patterns across various subgroups. Time-varying\n \n area under the receiving operating characteristic curve (AUC\n t<\/jats:sub>\n )\n \n and\n \n time-varying average precision (AP\n t<\/jats:sub>\n )\n \n were our primary evaluation metrics.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n TTE models consistently had comparable or higher\n \n AUC\n t<\/jats:sub>\n and\n \n AP\n t<\/jats:sub>\n than BC for all conditions. At clinically relevant operating time points, the area under the receiving operating characteristic curve (AUC) values for DTNN\n YOB\u22642020<\/jats:sub>\n (year-of-birth) and DCPH\n YOB\u22642020<\/jats:sub>\n (deep Cox proportional hazard) were 0.70 (95% CI 0.66\u20100.77) and 0.72 (95% CI 0.66\u20100.78) at\n t<\/jats:italic>\n =5 for autism, 0.72 (95% CI 0.65\u20100.76) and 0.68 (95% CI 0.62\u20100.74) at\n t<\/jats:italic>\n =7 for ADHD, 0.72 (95% CI 0.70\u20100.75) and 0.71 (95% CI 0.69\u20100.74) at\n t<\/jats:italic>\n =1 for recurrent otitis media, and 0.74 (95% CI 0.68\u20100.82) and 0.71 (95% CI 0.63\u20100.77) at\n t<\/jats:italic>\n =1 for food allergy, compared to 0.6 (95% CI 0.55\u20100.66), 0.47 (95% CI 0.40\u20100.54), 0.73 (95% CI 0.70\u20100.75), and 0.77 (95% CI 0.71\u20100.82) for BC\n YOB\u22642020<\/jats:sub>\n , respectively. The probabilities predicted by BC models were positively correlated with censoring times, particularly for autism and ADHD prediction. Filtering strategies based on YOB or length of follow-up only partially corrected these biases. In subgroup analyses, only DTNN predicted diagnosis probabilities that accurately reflect actual clinical prevalence and temporal trends.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n BC models substantially underpredicted diagnosis likelihood and inappropriately assigned lower probability scores to individuals with earlier censoring. Common filtering strategies did not adequately address this limitation. TTE approaches, particularly DTNN, effectively mitigated bias from the censoring distribution, resulting in superior discrimination and calibration performance and more accurate prediction of clinical prevalence. Machine learning practitioners should recognize the limitations of BC for long-horizon diagnosis prediction and adopt TTE approaches. The DTNN in particular is well-suited to mitigate the effects of right-censoring and maximize prediction performance in this setting.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/62985","type":"journal-article","created":{"date-parts":[[2025,3,27]],"date-time":"2025-03-27T16:19:54Z","timestamp":1743092394000},"page":"e62985-e62985","source":"Crossref","is-referenced-by-count":4,"title":["Limitations of Binary Classification for Long-Horizon Diagnosis Prediction and Advantages of a Discrete-Time Time-to-Event Approach: Empirical Analysis"],"prefix":"10.2196","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4090-8649","authenticated-orcid":false,"given":"De Rong","family":"Loh","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-1987-3749","authenticated-orcid":false,"given":"Elliot D","family":"Hill","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3610-4883","authenticated-orcid":false,"given":"Nan","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1410-2764","authenticated-orcid":false,"given":"Geraldine","family":"Dawson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4112-9639","authenticated-orcid":false,"given":"Matthew M","family":"Engelhard","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,3,27]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-018-0029-1","article-title":"Scalable and accurate deep learning with electronic health records","volume":"1","author":"Rajkomar","journal-title":"NPJ Digit Med"},{"issue":"3","key":"R2","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1016\/j.artmed.2009.08.001","article-title":"Impact of censoring on learning Bayesian networks in survival modelling","volume":"47","author":"Stajduhar","journal-title":"Artif Intell Med"},{"issue":"6","key":"R3","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1093\/jamia\/ocx071","article-title":"Biases introduced by filtering electronic health records for patients with \u201ccomplete data\u201d","volume":"24","author":"Weber","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"R4","doi-asserted-by":"publisher","first-page":"841","DOI":"10.1214\/08-AOAS169","article-title":"Random survival forests","volume":"2","author":"Ishwaran","journal-title":"Ann Appl Stat"},{"key":"R5","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.1078975","article-title":"Decision tree for competing risks survival probability in breast cancer study","volume":"3","author":"Ibrahim","journal-title":"Int J Biol Med Sci"},{"issue":"4","key":"R6","doi-asserted-by":"publisher","first-page":"1033","DOI":"10.1007\/s10618-014-0386-6","article-title":"Data mining for censored time-to-event data: a Bayesian network model for predicting cardiovascular risk from electronic health record data","volume":"29","author":"Bandyopadhyay","journal-title":"Data Min Knowl Disc"},{"issue":"1","key":"R7","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1111\/biom.13280","article-title":"Bayesian analysis of survival data with missing censoring indicators","volume":"77","author":"Brownstein","journal-title":"Biometrics"},{"issue":"2","key":"R8","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1111\/j.2517-6161.1972.tb00899.x","article-title":"Regression models and life-tables","volume":"34","author":"Cox","journal-title":"J R Stat Soc Ser B"},{"issue":"10","key":"R9","doi-asserted-by":"publisher","first-page":"1169","DOI":"10.1002\/(sici)1097-0258(19980530)17:10<1169::aid-sim796>3.0.co;2-d","article-title":"Feed forward neural networks for the analysis of censored survival data: a partial logistic regression approach","volume":"17","author":"Biganzoli","journal-title":"Stat Med"},{"key":"R10","doi-asserted-by":"publisher","DOI":"10.7717\/peerj.6257","article-title":"A scalable discrete-time survival model for neural networks","volume":"7","author":"Gensheimer","journal-title":"PeerJ"},{"key":"R11","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2016.03.009","article-title":"Adapting machine learning techniques to censored time-to-event health record data: a general-purpose approach using inverse probability of censoring weighting","volume":"61","author":"Vock","journal-title":"J Biomed Inform"},{"key":"R12","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2019.103337","article-title":"Deep learning for electronic health records: a comparative review of multiple deep neural architectures","volume":"101","author":"Solares","journal-title":"J Biomed Inform"},{"issue":"1","key":"R13","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-018-0482-1","article-title":"DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network","volume":"18","author":"Katzman","journal-title":"BMC Med Res Methodol"},{"key":"R14","volume":"151","author":"Engelhard","journal-title":"Proceedings of The 25th Int Conf Artif Intell Stat, PMLR"},{"key":"R15","unstructured":"LongHorizonDiagnosis. GitHub. URL: https:\/\/github.com\/engelhard-lab\/LongHorizonDiagnosis [Accessed 12-03-2025]"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.1016\/j.socscimed.2022.114759","article-title":"Using electronic health records to understand the population of local children captured in a large health system in Durham County, NC, USA, and implications for population health research","volume":"296","author":"Stolte","journal-title":"Soc Sci Med"},{"issue":"2","key":"R17","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2022.54303","article-title":"Predictive value of early autism detection models based on electronic health record data collected before age 1 year","volume":"6","author":"Engelhard","journal-title":"JAMA Netw Open"},{"key":"R18","unstructured":"RxNorm. US National Library of Medicine. URL: https:\/\/www.nlm.nih.gov\/research\/umls\/rxnorm\/index.html [Accessed 12-03-2025]"},{"key":"R19","unstructured":"CPT. American Medical Association. Aug23, 2024. URL: https:\/\/www.ama-assn.org\/practice-management\/cpt [Accessed 12-03-2025]"},{"key":"R20","unstructured":"LOINC. Regenstrief institute. URL: https:\/\/loinc.org\/ [Accessed 12-03-2025]"},{"key":"R21","unstructured":"Kvamme H Hart B Pati S Sellereite N . pycox. GitHub. Jan2022. URL: https:\/\/github.com\/havakv\/pycox.git [Accessed 12-03-2025]"},{"key":"R22","volume":"12","author":"Pedregosa","journal-title":"J Mach Learn Res"},{"key":"R23","doi-asserted-by":"crossref","unstructured":"D\u2019Agostino RB Nam BH . Evaluation of the performance of survival analysis models: discrimination and calibration measures. In: Handbook of Statistics. Vol2003. Elsevier; :1-25. [doi: 10.1016\/S0169-7161(03)23001-7]","DOI":"10.1016\/S0169-7161(03)23001-7"},{"issue":"24","key":"R24","doi-asserted-by":"publisher","first-page":"3927","DOI":"10.1002\/sim.2427","article-title":"A time-dependent discrimination index for survival data","volume":"24","author":"Antolini","journal-title":"Stat Med"},{"key":"R25","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2020.103496","article-title":"A practical perspective on the concordance index for the evaluation and selection of prognostic time-to-event models","volume":"108","author":"Longato","journal-title":"J Biomed Inform"},{"key":"R26","doi-asserted-by":"publisher","DOI":"10.2196\/52508","article-title":"Consolidated Reporting Guidelines for Prognostic and Diagnostic Machine Learning Models (CREMLS)","volume":"26","author":"El Emam","journal-title":"J Med Internet Res"},{"issue":"15_suppl","key":"R27","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2019.37.15_suppl.e18188","article-title":"Effect of right censoring bias on survival analysis","volume":"37","author":"Barrajon","journal-title":"JCO"},{"issue":"1","key":"R28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1214\/ss\/998929473","article-title":"Understanding the shape of the hazard rate: a process point of view (with comments and a rejoinder by the authors)","volume":"16","author":"Aalen","journal-title":"Statist Sci"},{"issue":"3","key":"R29","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.0771","article-title":"Racial\/ethnic disparities in the prevalence and trends of autism spectrum disorder in US children and adolescents","volume":"4","author":"Yuan","journal-title":"JAMA Netw Open"},{"issue":"2","key":"R30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.15585\/mmwr.ss7202a1","article-title":"Prevalence and characteristics of autism spectrum disorder among children aged 8 years \u2014 autism and developmental disabilities monitoring network, 11 sites, United States, 2020","volume":"72","author":"Maenner","journal-title":"MMWR Surveill Summ"},{"issue":"4","key":"R31","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2018.1471","article-title":"Twenty-year trends in diagnosed attention-deficit\/hyperactivity disorder among us children and adolescents, 1997-2016","volume":"1","author":"Xu","journal-title":"JAMA Netw Open"},{"issue":"3","key":"R32","doi-asserted-by":"publisher","DOI":"10.1542\/peds.2017-0181","article-title":"Epidemiology of acute otitis media in the postpneumococcal conjugate vaccine era","volume":"140","author":"Kaur","journal-title":"Pediatrics"},{"issue":"2","key":"R33","doi-asserted-by":"publisher","first-page":"103369","DOI":"10.1016\/j.amjoto.2021.103369","article-title":"Impact of COVID-19 on nationwide pediatric otolaryngology: otitis media and myringotomy tube trends","volume":"43","author":"Allen","journal-title":"Am J Otolaryngol"},{"issue":"11","key":"R34","doi-asserted-by":"publisher","first-page":"2306","DOI":"10.1002\/lary.21090","article-title":"Racial\/ethnic and socioeconomic disparities in the prevalence and treatment of otitis media in children in the United States","volume":"120","author":"Smith","journal-title":"Laryngoscope"},{"issue":"1","key":"R35","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1177\/0194599810391428","article-title":"Disparities in children with otitis media: the effect of insurance status","volume":"144","author":"Patel","journal-title":"Otolaryngol Neck Surg"},{"key":"R36","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijporl.2016.06.029","article-title":"Do race\/ethnicity or socioeconomic status affect why we place ear tubes in children?","volume":"88","author":"Nieman","journal-title":"Int J Pediatr Otorhinolaryngol"},{"issue":"1","key":"R37","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1016\/j.iac.2017.09.002","article-title":"Epidemiology of food allergy","volume":"38","author":"Dunlop","journal-title":"Immunol Allergy Clin North Am"},{"issue":"3","key":"R38","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1111\/imj.13362","article-title":"Food allergy: is prevalence increasing?","volume":"47","author":"Tang","journal-title":"Intern Med J"},{"issue":"3","key":"R39","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1016\/j.anai.2013.12.007","article-title":"Temporal trends and racial\/ethnic disparity in self-reported pediatric food allergy in the United States","volume":"112","author":"Keet","journal-title":"Ann Allergy Asthma Immunol"},{"issue":"5","key":"R40","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1542\/peds.79.5.683","article-title":"Prospective appraisal of complaints of adverse reactions to foods in children during the first 3 years of life","volume":"79","author":"Bock","journal-title":"Pediatrics"},{"issue":"5","key":"R41","doi-asserted-by":"publisher","first-page":"393","DOI":"10.1034\/j.1398-9995.2001.056005393.x","article-title":"The prevalence of CMA\/CMPI in young children: the validity of parentally perceived reactions in a population-based study","volume":"56","author":"Eggesb\u00f8","journal-title":"Allergy"},{"issue":"3","key":"R42","doi-asserted-by":"publisher","first-page":"668","DOI":"10.1016\/j.jaci.2011.01.039","article-title":"Prevalence of challenge-proven IgE-mediated food allergy using population-based sampling and predetermined challenge criteria in infants","volume":"127","author":"Osborne","journal-title":"J Allergy Clin Immunol"},{"issue":"3","key":"R43","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1097\/ACI.0000000000000529","article-title":"Gender aspects in food allergy","volume":"19","author":"Pali-Sch\u00f6ll","journal-title":"Curr Opin Allergy Clin Immunol"},{"issue":"6","key":"R44","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.18162","article-title":"Racial, ethnic, and socioeconomic differences in food allergies in the US","volume":"6","author":"Jiang","journal-title":"JAMA Netw Open"},{"issue":"1","key":"R45","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1016\/j.acap.2020.03.005","article-title":"Prevalence and correlates of food allergy among Medicaid-enrolled United States children","volume":"21","author":"Bilaver","journal-title":"Acad Pediatr"},{"issue":"12","key":"R46","doi-asserted-by":"publisher","first-page":"4721","DOI":"10.1007\/s10803-019-04188-6","article-title":"Race\/ethnicity-resolved time trends in United States ASD prevalence estimates from IDEA and ADDM","volume":"49","author":"Nevison","journal-title":"J Autism Dev Disord"}],"container-title":["JMIR AI"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ai.jmir.org\/2025\/1\/e62985","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:31:21Z","timestamp":1785537081000},"score":1,"resource":{"primary":{"URL":"https:\/\/ai.jmir.org\/2025\/1\/e62985"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,27]]},"references-count":46,"URL":"https:\/\/doi.org\/10.2196\/62985","relation":{},"ISSN":["2817-1705"],"issn-type":[{"value":"2817-1705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,27]]},"article-number":"v4i1e62985"}}