{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T20:07:25Z","timestamp":1787947645328,"version":"build-2784847793"},"reference-count":87,"publisher":"American Speech Language Hearing Association","issue":"9","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Speech Lang Hear Res"],"published-print":{"date-parts":[[2020,9,15]]},"abstract":"<jats:sec><jats:title>Purpose<\/jats:title><jats:p>Data mining algorithms using electronic health records (EHRs) are useful in large-scale population-wide studies to classify etiology and comorbidities (<jats:xref ref-type=\"bibr\">Casey et al., 2016<\/jats:xref>). Here, we apply this approach to developmental language disorder (DLD), a prevalent communication disorder whose risk factors and epidemiology remain largely undiscovered.<\/jats:p><\/jats:sec><jats:sec><jats:title>Method<\/jats:title><jats:p>We first created a reliable system for manually identifying DLD in EHRs based on speech-language pathologist (SLP) diagnostic expertise. We then developed and validated an automated algorithmic procedure, called, Automated Phenotyping Tool for identifying DLD cases in health systems data (APT-DLD), that classifies a DLD status for patients within EHRs on the basis of ICD (International Statistical Classification of Diseases and Related Health Problems) codes. APT-DLD was validated in a discovery sample (<jats:italic>N<\/jats:italic>= 973) using expert SLP manual phenotype coding as a gold-standard comparison and then applied and further validated in a replication sample of<jats:italic>N<\/jats:italic>= 13,652 EHRs.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>In the discovery sample, the APT-DLD algorithm correctly classified 98% (concordance) of DLD cases in concordance with manually coded records in the training set, indicating that APT-DLD successfully mimics a comprehensive chart review. The output of APT-DLD was also validated in relation to independently conducted SLP clinician coding in a subset of records, with a positive predictive value of 95% of cases correctly classified as DLD. We also applied APT-DLD to the replication sample, where it achieved a positive predictive value of 90% in relation to SLP clinician classification of DLD.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>APT-DLD is a reliable, valid, and scalable tool for identifying DLD cohorts in EHRs. This new method has promising public health implications for future large-scale epidemiological investigations of DLD and may inform EHR data mining algorithms for other communication disorders.<\/jats:p><\/jats:sec><jats:sec><jats:title>Supplemental Material<\/jats:title><jats:p><jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/doi.org\/10.23641\/asha.12753578\">https:\/\/doi.org\/10.23641\/asha.12753578<\/jats:ext-link><\/jats:p><\/jats:sec>","DOI":"10.1044\/2020_jslhr-19-00397","type":"journal-article","created":{"date-parts":[[2020,8,13]],"date-time":"2020-08-13T22:50:43Z","timestamp":1597359043000},"page":"3019-3035","source":"Crossref","is-referenced-by-count":14,"title":["Automated Phenotyping Tool for Identifying Developmental Language Disorder Cases in Health Systems Data (APT-DLD): A New Research Algorithm for Deployment in Large-Scale Electronic Health Record Systems"],"prefix":"10.1044","volume":"63","author":[{"suffix":"Jr.","given":"Courtney E.","family":"Walters","sequence":"first","affiliation":[{"name":"Department of Otolaryngology, Vanderbilt University Medical Center, Nashville, TN"},{"name":"Neuroscience Program, College of Arts and Science, Vanderbilt University, Nashville, TN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3331-1580","authenticated-orcid":false,"given":"Rachana","family":"Nitin","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology, Vanderbilt University Medical Center, Nashville, TN"},{"name":"Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Katherine","family":"Margulis","sequence":"additional","affiliation":[{"name":"Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN"},{"name":"Kennedy Krieger Institute, Baltimore, MD"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Olivia","family":"Boorom","sequence":"additional","affiliation":[{"name":"Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1470-4928","authenticated-orcid":false,"given":"Daniel E.","family":"Gustavson","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology, Vanderbilt University Medical Center, Nashville, TN"},{"name":"Vanderbilt Genetics Institute, Vanderbilt University Medical Center, Nashville, TN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Catherine T.","family":"Bush","sequence":"additional","affiliation":[{"name":"Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5143-2282","authenticated-orcid":false,"given":"Lea K.","family":"Davis","sequence":"additional","affiliation":[{"name":"Vanderbilt Genetics Institute, Vanderbilt University Medical Center, Nashville, TN"},{"name":"Department of Medicine, Vanderbilt University Medical Center, Nashville, TN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6372-6402","authenticated-orcid":false,"given":"Jennifer E.","family":"Below","sequence":"additional","affiliation":[{"name":"Vanderbilt Genetics Institute, Vanderbilt University Medical Center, Nashville, TN"},{"name":"Department of Medicine, Vanderbilt University Medical Center, Nashville, TN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9315-0830","authenticated-orcid":false,"given":"Nancy J.","family":"Cox","sequence":"additional","affiliation":[{"name":"Vanderbilt Genetics Institute, Vanderbilt University Medical Center, Nashville, TN"},{"name":"Department of Medicine, Vanderbilt University Medical Center, Nashville, TN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3342-1747","authenticated-orcid":false,"given":"Stephen M.","family":"Camarata","sequence":"additional","affiliation":[{"name":"Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1643-6979","authenticated-orcid":false,"given":"Reyna L.","family":"Gordon","sequence":"additional","affiliation":[{"name":"Department of Otolaryngology, Vanderbilt University Medical Center, Nashville, TN"},{"name":"Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN"},{"name":"Vanderbilt Genetics Institute, Vanderbilt University Medical Center, Nashville, TN"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1310","reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2018.01.004"},{"key":"e_1_3_2_3_1","unstructured":"American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.)."},{"key":"e_1_3_2_4_1","unstructured":"American Speech-Language-Hearing Association. (2019). 2020 ICD-10-CM diagnosis codes related to speech language and swallowing disorders."},{"key":"e_1_3_2_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2015.12.006"},{"key":"e_1_3_2_6_1","doi-asserted-by":"publisher","DOI":"10.1002\/acr.22989"},{"key":"e_1_3_2_7_1","doi-asserted-by":"publisher","DOI":"10.1192\/bjp.126.2.127"},{"key":"e_1_3_2_8_1","article-title":"KNIME: The Konstanz Information Miner","author":"Berthold M. R.","year":"2007","journal-title":"Studies in Classification, Data Analysis, and Knowledge Organization (GfKL 2007)"},{"key":"e_1_3_2_9_1","doi-asserted-by":"publisher","DOI":"10.1111\/1460-6984.12335"},{"key":"e_1_3_2_10_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0158753"},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.1111\/jcpp.12721"},{"issue":"2","key":"e_1_3_2_12_1","first-page":"186","article-title":"Is there really a difference? Distinguishing mild intellectual disability from similar disability categories","volume":"50","author":"Bouck E. C.","year":"2015","journal-title":"Education and Training in Autism and Developmental Disabilities"},{"key":"e_1_3_2_13_1","doi-asserted-by":"publisher","DOI":"10.1111\/cts.12514"},{"key":"e_1_3_2_14_1","doi-asserted-by":"publisher","DOI":"10.4338\/ACI-2017-02-RA-0029"},{"key":"e_1_3_2_15_1","doi-asserted-by":"publisher","DOI":"10.1038\/nbt1150"},{"key":"e_1_3_2_16_1","doi-asserted-by":"publisher","DOI":"10.1044\/jshr.3301.205"},{"key":"e_1_3_2_17_1","first-page":"19","article-title":"Assessment tools: Evaluating our measurements","author":"Camarata S.","year":"2014","journal-title":"Assessing listening and spoken language in children with hearing loss"},{"key":"e_1_3_2_18_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-publhealth-032315-021353"},{"key":"e_1_3_2_19_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41398-018-0133-7"},{"key":"e_1_3_2_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2017.04.009"},{"key":"e_1_3_2_21_1","author":"Connolly J.","year":"2013","journal-title":"CHOP. ADHD phenotype algorithm"},{"key":"e_1_3_2_22_1","doi-asserted-by":"publisher","DOI":"10.1159\/000444750"},{"key":"e_1_3_2_23_1","doi-asserted-by":"publisher","DOI":"10.1111\/1460-6984.12338"},{"key":"e_1_3_2_24_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooy037"},{"key":"e_1_3_2_25_1","doi-asserted-by":"publisher","DOI":"10.1089\/neu.2018.5916"},{"key":"e_1_3_2_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tig.2017.07.002"},{"key":"e_1_3_2_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ridd.2011.06.001"},{"key":"e_1_3_2_28_1","doi-asserted-by":"publisher","DOI":"10.1111\/1460-6984.12385"},{"key":"e_1_3_2_29_1","doi-asserted-by":"publisher","DOI":"10.1177\/0022219415617165"},{"key":"e_1_3_2_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2017.09.009"},{"key":"e_1_3_2_31_1","doi-asserted-by":"publisher","DOI":"10.1111\/gbb.12223"},{"key":"e_1_3_2_32_1","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCULATIONAHA.118.038814"},{"key":"e_1_3_2_33_1","first-page":"145","volume-title":"Pacific Symposium on Biocomputing","author":"Glicksberg B. S.","year":"2018"},{"key":"e_1_3_2_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcomdis.2018.10.001"},{"key":"e_1_3_2_35_1","doi-asserted-by":"publisher","DOI":"10.1044\/2013_JSLHR-L-12-0240"},{"issue":"1","key":"e_1_3_2_36_1","first-page":"117","article-title":"Next-generation phenotyping of electronic health records","volume":"20","author":"Hripcsak G.","year":"2013","journal-title":"JAMIA"},{"key":"e_1_3_2_37_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0166541"},{"key":"e_1_3_2_38_1","doi-asserted-by":"publisher","DOI":"10.1177\/0022219419845070"},{"key":"e_1_3_2_39_1","doi-asserted-by":"publisher","DOI":"10.1044\/0161-1461.2901.35"},{"key":"e_1_3_2_40_1","doi-asserted-by":"publisher","DOI":"10.1182\/blood-2015-05-643601"},{"issue":"6","key":"e_1_3_2_41_1","first-page":"1046","article-title":"PheKB: A catalog and workflow for creating electronic phenotype algorithms for transportability","volume":"23","author":"Kirby J. C.","year":"2016","journal-title":"JAMIA"},{"key":"e_1_3_2_42_1","doi-asserted-by":"publisher","DOI":"10.1542\/peds.2015-2469"},{"key":"e_1_3_2_43_1","doi-asserted-by":"publisher","DOI":"10.1186\/s40359-018-0222-7"},{"issue":"4","key":"e_1_3_2_44_1","first-page":"191","article-title":"Using local norms when identifying PLD: Utility of sample specific parameters","volume":"4","author":"Lancaster H. S.","year":"2016","journal-title":"International Journal of Behavioral Research & Psychology"},{"key":"e_1_3_2_45_1","doi-asserted-by":"publisher","DOI":"10.1111\/1460-6984.12433"},{"key":"e_1_3_2_46_1","doi-asserted-by":"publisher","DOI":"10.1080\/14417040210001669261"},{"key":"e_1_3_2_47_1","doi-asserted-by":"publisher","DOI":"10.1044\/1092-4388(2009\/08-0142)"},{"key":"e_1_3_2_48_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0305000914000130"},{"key":"e_1_3_2_49_1","doi-asserted-by":"publisher","DOI":"10.1111\/cdep.12053"},{"key":"e_1_3_2_50_1","doi-asserted-by":"publisher","DOI":"10.2196\/10497"},{"key":"e_1_3_2_51_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0159621"},{"issue":"2","key":"e_1_3_2_52_1","first-page":"306","article-title":"Use of computerized algorithm to identify individuals in need of testing for celiac disease","volume":"20","author":"Ludvigsson J. F.","year":"2013","journal-title":"JAMIA"},{"issue":"2","key":"e_1_3_2_53_1","first-page":"e297","article-title":"Identifying phenotypic signatures of neuropsychiatric disorders from electronic medical records","volume":"20","author":"Lyalina S.","year":"2013","journal-title":"JAMIA"},{"key":"e_1_3_2_54_1","first-page":"376","volume-title":"Pacific Symposium on Biocomputing","author":"Malinowski J.","year":"2014"},{"key":"e_1_3_2_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chc.2017.03.001"},{"key":"e_1_3_2_56_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.biopsych.2018.01.011"},{"key":"e_1_3_2_57_1","doi-asserted-by":"publisher","DOI":"10.1044\/2019_PERSP-19-00083"},{"key":"e_1_3_2_58_1","doi-asserted-by":"publisher","DOI":"10.1044\/1092-4388(2009\/08-0085)"},{"key":"e_1_3_2_59_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1475-6773.2004.00315.x"},{"key":"e_1_3_2_60_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10519-010-9424-3"},{"issue":"2","key":"e_1_3_2_61_1","first-page":"e206","article-title":"Electronic health records-driven phenotyping: Challenges, recent advances, and perspectives","volume":"20","author":"Pathak J.","year":"2013","journal-title":"JAMIA"},{"issue":"2","key":"e_1_3_2_62_1","first-page":"150","article-title":"Development and validation of an electronic medical record (EMR)-based computed phenotype of HIV-1 infection","volume":"25","author":"Paul D. W.","year":"2018","journal-title":"JAMIA"},{"issue":"1","key":"e_1_3_2_63_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/cphg.80","article-title":"Using electronic health records to generate phenotypes for research","volume":"100","author":"Pendergrass S. A.","year":"2019","journal-title":"Current Protocols in Human Genetics"},{"key":"e_1_3_2_64_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.psych.60.110707.163548"},{"key":"e_1_3_2_65_1","doi-asserted-by":"publisher","DOI":"10.1044\/jslhr.4104.951"},{"key":"e_1_3_2_66_1","doi-asserted-by":"publisher","DOI":"10.1038\/tp.2016.138"},{"key":"e_1_3_2_67_1","doi-asserted-by":"publisher","DOI":"10.1097\/MPG.0000000000002020"},{"key":"e_1_3_2_68_1","doi-asserted-by":"publisher","DOI":"10.1044\/2018_JSLHR-L-16-0459"},{"key":"e_1_3_2_69_1","doi-asserted-by":"publisher","DOI":"10.1038\/clpt.2014.137"},{"key":"e_1_3_2_70_1","doi-asserted-by":"publisher","DOI":"10.1111\/1460-6984.12102"},{"key":"e_1_3_2_71_1","doi-asserted-by":"publisher","DOI":"10.1044\/2015_JSLHR-L-15-0255"},{"key":"e_1_3_2_72_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2016.05.005"},{"key":"e_1_3_2_73_1","doi-asserted-by":"crossref","unstructured":"Safarova M. S. Liu H. & Kullo I. J. (2016). Rapid identification of familial hypercholesterolemia from electronic health records: The SEARCH study. https:\/\/doi.org\/10.1016\/j.jacl.2016.08.001","DOI":"10.1016\/j.jacl.2016.08.001"},{"key":"e_1_3_2_74_1","doi-asserted-by":"publisher","DOI":"10.1542\/pir.36-8-355"},{"key":"e_1_3_2_75_1","doi-asserted-by":"publisher","DOI":"10.1086\/338649"},{"key":"e_1_3_2_76_1","doi-asserted-by":"publisher","DOI":"10.1002\/ajmg.b.32548"},{"key":"e_1_3_2_77_1","doi-asserted-by":"publisher","DOI":"10.1044\/jshd.4602.114"},{"key":"e_1_3_2_78_1","doi-asserted-by":"crossref","unstructured":"The National Academies of Sciences Engineering and Medicine. (2016). Speech and language disorders in children: Implications for the Social Security Administration's Supplemental Security Income program. https:\/\/doi.org\/10.17226\/21872","DOI":"10.17226\/21872"},{"key":"e_1_3_2_79_1","doi-asserted-by":"publisher","DOI":"10.1044\/jslhr.4006.1245"},{"key":"e_1_3_2_80_1","doi-asserted-by":"publisher","DOI":"10.1044\/jshr.3906.1284"},{"key":"e_1_3_2_81_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0160573"},{"key":"e_1_3_2_82_1","doi-asserted-by":"publisher","DOI":"10.1044\/leader.FTR1.23122018.44"},{"key":"e_1_3_2_83_1","doi-asserted-by":"publisher","DOI":"10.1044\/leader.FTR2.23122018.54"},{"key":"e_1_3_2_84_1","first-page":"112","article-title":"Creation and validation of an EMR-based algorithm for identifying major adverse cardiac events while on statins","volume":"2014","author":"Wei W.-Q.","year":"2014","journal-title":"AMIA Joint Summits on Translational Science Proceedings"},{"issue":"1","key":"e_1_3_2_85_1","first-page":"e20","article-title":"Combining billing codes, clinical notes, and medications from electronic health records provides superior phenotyping performance","volume":"23","author":"Wei W.-Q.","year":"2016","journal-title":"JAMIA"},{"key":"e_1_3_2_86_1","doi-asserted-by":"publisher","DOI":"10.1158\/1078-0432.CCR-16-2809"},{"key":"e_1_3_2_87_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.gde.2008.07.006"},{"issue":"6","key":"e_1_3_2_88_1","first-page":"1060","article-title":"An efficient approach for surveillance of childhood diabetes by type derived from electronic health record data: The SEARCH for Diabetes in Youth study","volume":"23","author":"Zhong V. W.","year":"2016","journal-title":"JAMIA"}],"container-title":["Journal of Speech, Language, and Hearing Research"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/pubs.asha.org\/doi\/pdf\/10.1044\/2020_JSLHR-19-00397","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,5]],"date-time":"2023-10-05T18:59:40Z","timestamp":1696532380000},"score":1,"resource":{"primary":{"URL":"http:\/\/pubs.asha.org\/doi\/10.1044\/2020_JSLHR-19-00397"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,15]]},"references-count":87,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2020,9,15]]}},"alternative-id":["10.1044\/2020_JSLHR-19-00397"],"URL":"https:\/\/doi.org\/10.1044\/2020_jslhr-19-00397","relation":{},"ISSN":["1092-4388","1558-9102"],"issn-type":[{"value":"1092-4388","type":"print"},{"value":"1558-9102","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,15]]}}}