{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T11:02:06Z","timestamp":1782471726914,"version":"3.54.5"},"reference-count":49,"publisher":"Oxford University Press (OUP)","issue":"10","license":[{"start":{"date-parts":[[2018,8,17]],"date-time":"2018-08-17T00:00:00Z","timestamp":1534464000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/academic.oup.com\/journals\/pages\/about_us\/legal\/notices"}],"funder":[{"name":"Department of Veterans Affairs, Office of Academic Affiliations, Advanced Fellowship Program in Medical Informatics"},{"name":"Vanderbilt University Medical Center Department of Biomedical Informatics"},{"name":"Veterans Health Administration Health Services Research & Development"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>Standards such as the Logical Observation Identifiers Names and Codes (LOINC\u00ae) are critical for interoperability and integrating data into common data models, but are inconsistently used. Without consistent mapping to standards, clinical data cannot be harmonized, shared, or interpreted in a meaningful context. We sought to develop an automated machine learning pipeline that leverages noisy labels to map laboratory data to LOINC codes.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>Across 130 sites in the Department of Veterans Affairs Corporate Data Warehouse, we selected the 150 most commonly used laboratory tests with numeric results per site from 2000 through 2016. Using source data text and numeric fields, we developed a machine learning model and manually validated random samples from both labeled and unlabeled datasets.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The raw laboratory data consisted of &amp;gt;6.5 billion test results, with 2215 distinct LOINC codes. The model predicted the correct LOINC code in 85% of the unlabeled data and 96% of the labeled data by test frequency. In the subset of labeled data where the original and model-predicted LOINC codes disagreed, the model-predicted LOINC code was correct in 83% of the data by test frequency.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>Using a completely automated process, we are able to assign LOINC codes to unlabeled data with high accuracy. When the model-predicted LOINC code differed from the original LOINC code, the model prediction was correct in the vast majority of cases. This scalable, automated algorithm may improve data quality and interoperability, while substantially reducing the manual effort currently needed to accurately map laboratory data.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocy110","type":"journal-article","created":{"date-parts":[[2018,7,24]],"date-time":"2018-07-24T23:38:51Z","timestamp":1532475531000},"page":"1292-1300","source":"Crossref","is-referenced-by-count":31,"title":["Automated mapping of laboratory tests to LOINC codes using noisy labels in a national electronic health record system database"],"prefix":"10.1093","volume":"25","author":[{"given":"Sharidan K","family":"Parr","sequence":"first","affiliation":[{"name":"Geriatric Research Education and Clinical Center (GRECC), Tennessee Valley Health System Veterans Administration Medical Center, Nashville, Tennessee, USA"},{"name":"Division of Nephrology and Hypertension, Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, USA"},{"name":"Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew S","family":"Shotwell","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alvin D","family":"Jeffery","sequence":"additional","affiliation":[{"name":"Geriatric Research Education and Clinical Center (GRECC), Tennessee Valley Health System Veterans Administration Medical Center, Nashville, Tennessee, USA"},{"name":"Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas A","family":"Lasko","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael E","family":"Matheny","sequence":"additional","affiliation":[{"name":"Geriatric Research Education and Clinical Center (GRECC), Tennessee Valley Health System Veterans Administration Medical Center, Nashville, Tennessee, USA"},{"name":"Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee, USA"},{"name":"Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee, USA"},{"name":"Division of General Internal Medicine and Public Health, Vanderbilt University Medical Center, Nashville, Tennessee, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2018,8,17]]},"reference":[{"issue":"1","key":"2020110612230509300_ocy110-B1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1197\/jamia.M2273","article-title":"Toward a national framework for the secondary use of health data: an American Medical Informatics Association White Paper","volume":"14","author":"Safran","year":"2007","journal-title":"J Am Med Inform Assoc"},{"issue":"13","key":"2020110612230509300_ocy110-B2","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1001\/jama.2013.393","article-title":"The inevitable application of big data to health care","volume":"309","author":"Murdoch","year":"2013","journal-title":"JAMA"},{"issue":"6","key":"2020110612230509300_ocy110-B3","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1136\/jamia.1998.0050503","article-title":"A framework for comprehensive health terminology systems in the United States: development guidelines, criteria for selection, and public policy implications. ANSI Healthcare Informatics Standards Board Vocabulary Working Group and the Computer-Based Patient Records Institute Working Group on Codes and Structures","volume":"5","author":"Chute","year":"1998","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2020110612230509300_ocy110-B4","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.ijmedinf.2010.11.006","article-title":"The role of standardized data and terminological systems in computerized clinical decision support systems: literature review and survey","volume":"80","author":"Ahmadian","year":"2011","journal-title":"Int J Med Inform"},{"key":"2020110612230509300_ocy110-B5"},{"issue":"1","key":"2020110612230509300_ocy110-B6","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/S1386-5056(98)00089-6","article-title":"Combining laboratory data sets from multiple institutions using the logical observation identifier names and codes (LOINC)","volume":"51","author":"Baorto","year":"1998","journal-title":"Int J Med Inform"},{"key":"2020110612230509300_ocy110-B7","first-page":"447","article-title":"Correctness of voluntary LOINC mapping for laboratory tests in three large institutions","volume":"2010","author":"Lin","year":"2010","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110612230509300_ocy110-B8","first-page":"805","article-title":"Investigating the semantic interoperability of laboratory data exchanged using LOINC codes in three large institutions","volume":"2011","author":"Lin","year":"2011","journal-title":"AMIA Annu Symp Proc"},{"issue":"6 Part 1","key":"2020110612230509300_ocy110-B9","first-page":"277","article-title":"Adding value to the electronic health record through secondary use of data for quality assurance, research, and surveillance","volume":"13","author":"Hersh","year":"2007","journal-title":"Am J Manag Care"},{"issue":"1","key":"2020110612230509300_ocy110-B10","doi-asserted-by":"crossref","first-page":"38","DOI":"10.15265\/IY-2017-007","article-title":"Clinical data reuse or secondary use: current status and potential future progress","volume":"26","author":"Meystre","year":"2017","journal-title":"Yearb Med Inform"},{"key":"2020110612230509300_ocy110-B11"},{"key":"2020110612230509300_ocy110-B12","year":"2010"},{"key":"2020110612230509300_ocy110-B13","year":"2017"},{"key":"2020110612230509300_ocy110-B14","author":"Regenstrief Institute Inc"},{"issue":"4","key":"2020110612230509300_ocy110-B15","doi-asserted-by":"crossref","first-page":"642","DOI":"10.1016\/j.jbi.2012.04.012","article-title":"Standardizing clinical laboratory data for secondary use","volume":"45","author":"Abhyankar","year":"2012","journal-title":"J Biomed Inform"},{"issue":"03","key":"2020110612230509300_ocy110-B16","doi-asserted-by":"crossref","first-page":"536","DOI":"10.4338\/ACI-2014-12-CR-0121","article-title":"Creating a common data model for comparative effectiveness with the observational medical outcomes partnership","volume":"06","author":"FitzHenry","year":"2015","journal-title":"Appl Clin Inform"},{"issue":"1","key":"2020110612230509300_ocy110-B17","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1136\/amiajnl-2012-001159","article-title":"A corpus-based approach for automated LOINC mapping","volume":"21","author":"Fidahussein","year":"2014","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2020110612230509300_ocy110-B18","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1197\/jamia.M1935","article-title":"Standardizing laboratory data by mapping to LOINC","volume":"13","author":"Khan","year":"2006","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2020110612230509300_ocy110-B19","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1197\/jamia.M1823","article-title":"A system for automated lexical mapping","volume":"13","author":"Sun","year":"2006","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"2020110612230509300_ocy110-B20","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1093\/jamia\/ocw028","article-title":"Learning statistical models of phenotypes using noisy labeled training data","volume":"23","author":"Agarwal","year":"2016","journal-title":"J Am Med Inform Assoc"},{"key":"2020110612230509300_ocy110-B21","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.jbi.2017.04.009","article-title":"EHR-based phenotyping: bulk learning and evaluation","volume":"70","author":"Chiu","year":"2017","journal-title":"J Biomed Inform"},{"issue":"2","key":"2020110612230509300_ocy110-B22","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1006\/jcss.1996.0019","article-title":"General bounds on the number of examples needed for learning probabilistic concepts","volume":"52","author":"Simon","year":"1996","journal-title":"J Comput Syst Sci"},{"issue":"4","key":"2020110612230509300_ocy110-B23","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/0020-0190(96)00006-3","article-title":"On the sample complexity of noise-tolerant learning","volume":"57","author":"Aslam","year":"1996","journal-title":"Inf Process Lett"},{"key":"2020110612230509300_ocy110-B24","first-page":"2080","article-title":"Learning from noisy labels with deep neural networks","volume":"1406","author":"Sukhbaatar","year":"2014","journal-title":"arXiv Preprint arXiv"},{"key":"2020110612230509300_ocy110-B25","first-page":"10694","article-title":"Deep learning is robust to massive label noise","volume":"1705","author":"Rolnick","year":"2017","journal-title":"arXiv Preprint arXiv"},{"key":"2020110612230509300_ocy110-B26","author":"Natarajan","year":"2013"},{"key":"2020110612230509300_ocy110-B27","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-540-25966-4_29","volume-title":"Experiments on Ensembles with Missing and Noisy Data","author":"Melville","year":"2004"},{"key":"2020110612230509300_ocy110-B28","volume-title":"VIReC Factbook: Corporate Data Warehouse (CDW) Consult 2.1 Domain","author":"Center VIR","year":"2014"},{"key":"2020110612230509300_ocy110-B29","volume-title":"VIReC Resource Guide: VistA","author":"Center VIR","year":"2012"},{"key":"2020110612230509300_ocy110-B30"},{"key":"2020110612230509300_ocy110-B31"},{"issue":"406","key":"2020110612230509300_ocy110-B32","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1080\/01621459.1989.10478785","article-title":"Advances in record-linkage methodology as applied to matching the 1985 census of Tampa, Florida","volume":"84","author":"Jaro","year":"1989","journal-title":"J Am Stat Assoc"},{"key":"2020110612230509300_ocy110-B33","first-page":"354","author":"Winkler","year":"1990"},{"issue":"5-7","key":"2020110612230509300_ocy110-B34","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1002\/sim.4780140510","article-title":"Probabilistic linkage of large public health data files","volume":"14","author":"Jaro","year":"1995","journal-title":"Stat Med"},{"issue":"8","key":"2020110612230509300_ocy110-B35","first-page":"707","article-title":"Binary codes capable of correcting deletions, insertions and reversals","volume":"10","author":"Levenshtein","year":"1966","journal-title":"Soviet Phys Doklady"},{"issue":"4","key":"2020110612230509300_ocy110-B36","doi-asserted-by":"crossref","first-page":"658","DOI":"10.1016\/j.jbi.2012.01.008","article-title":"Auditing consistency and usefulness of LOINC use among three large institutions\u2014using version spaces for grouping LOINC codes","volume":"45","author":"Lin","year":"2012","journal-title":"J Biomed Inform"},{"issue":"1","key":"2020110612230509300_ocy110-B37","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J R Stat Soc Ser B"},{"issue":"1","key":"2020110612230509300_ocy110-B38","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1080\/00401706.1970.10488634","article-title":"Ridge regression: biased estimation for nonorthogonal problems","volume":"12","author":"Hoerl","year":"1970","journal-title":"Technometrics"},{"issue":"2","key":"2020110612230509300_ocy110-B39","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and variable selection via the elastic net","volume":"67","author":"Zou","year":"2005","journal-title":"J R Stat Soc B"},{"issue":"1","key":"2020110612230509300_ocy110-B40","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn"},{"key":"2020110612230509300_ocy110-B41","first-page":"2825","article-title":"Scikit-learn: machine learning in python","volume":"8","author":"Pedregosa","year":"2014","journal-title":"Front Neuroinform"},{"key":"2020110612230509300_ocy110-B42","author":"Chinchor","year":"1992"},{"key":"2020110612230509300_ocy110-B43","volume-title":"The Design of Experiments","author":"Fisher","year":"1935"},{"issue":"1","key":"2020110612230509300_ocy110-B44","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2307\/2331554","article-title":"The probable error of a mean","volume":"6","author":"Student","year":"1908","journal-title":"Biometrika"},{"issue":"1","key":"2020110612230509300_ocy110-B45","doi-asserted-by":"crossref","first-page":"111","DOI":"10.32614\/RJ-2014-011","article-title":"The stringdist package for approximate string matching","volume":"6","author":"van der Loo","year":"2014","journal-title":"R J"},{"key":"2020110612230509300_ocy110-B46","author":"R Core Team","year":"2015"},{"key":"2020110612230509300_ocy110-B47"},{"key":"2020110612230509300_ocy110-B48"},{"issue":"2","key":"2020110612230509300_ocy110-B49","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1093\/jamia\/ocx046","article-title":"LabRS: a rosetta stone for retrospective standardization of clinical laboratory test results","volume":"25","author":"Hauser","year":"2018","journal-title":"J Am Med Inform Assoc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/25\/10\/1292\/34150281\/ocy110.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/25\/10\/1292\/34150281\/ocy110.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T15:58:38Z","timestamp":1720454318000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/25\/10\/1292\/5075874"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8,17]]},"references-count":49,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2018,8,17]]},"published-print":{"date-parts":[[2018,10,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocy110","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2018,10]]},"published":{"date-parts":[[2018,8,17]]}}}