{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2022,6,7]],"date-time":"2022-06-07T10:15:39Z","timestamp":1654596939277},"reference-count":0,"publisher":"IOS Press","license":[{"start":{"date-parts":[[2022,6,6]],"date-time":"2022-06-06T00:00:00Z","timestamp":1654473600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,6,6]]},"abstract":"<jats:p>Supervised predictive models require labeled data for training purposes. Complete and accurate labeled data is not always available, and imperfectly labeled data may need to serve as an alternative. An important question is if the accuracy of the labeled data creates a performance ceiling for the trained model. In this study, we trained several models to recognize the presence of delirium in clinical documents using data with annotations that are not completely accurate. In the external evaluation, the support vector machine model with a linear kernel performed best, achieving an area under the curve of 89.3% and accuracy of 88%, surpassing the 80% accuracy of the training sample. We then generated a set of simulated data and carried out a series of experiments which demonstrated that models trained on imperfect data can (but do not always) outperform the accuracy of the training data.<\/jats:p>","DOI":"10.3233\/shti220161","type":"book-chapter","created":{"date-parts":[[2022,6,7]],"date-time":"2022-06-07T09:33:17Z","timestamp":1654594397000},"source":"Crossref","is-referenced-by-count":0,"title":["Accuracies of Training Labels and Machine Learning Models: Experiments on Delirium and Simulated Data"],"prefix":"10.3233","author":[{"given":"Yan","family":"Cheng","sequence":"first","affiliation":[{"name":"George Washington University, Washington, DC, USA"},{"name":"Washington DC VA Medical Center, Washington, DC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yijun","family":"Shao","sequence":"additional","affiliation":[{"name":"George Washington University, Washington, DC, USA"},{"name":"Washington DC VA Medical Center, Washington, DC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James","family":"Rudolph","sequence":"additional","affiliation":[{"name":"Providence VA Medical Center, Providence, RI, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Charlene R.","family":"Weir","sequence":"additional","affiliation":[{"name":"Salt Lake City VA Medical Center, Salt Lake City, UT, USA"},{"name":"University of Utah, Salt Lake City, UT, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Beth","family":"Sahlmann","sequence":"additional","affiliation":[{"name":"Office of Analytics and Performance Integration, Veterans Health Administration, Fort Myers, FL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qing","family":"Zeng-Treitler","sequence":"additional","affiliation":[{"name":"George Washington University, Washington, DC, USA"},{"name":"Washington DC VA Medical Center, Washington, DC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"7437","container-title":["Studies in Health Technology and Informatics","MEDINFO 2021: One World, One Health \u2013 Global Partnership for Digital Innovation"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/SHTI220161","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,7]],"date-time":"2022-06-07T09:33:17Z","timestamp":1654594397000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/SHTI220161"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,6]]},"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/shti220161","relation":{},"ISSN":["0926-9630","1879-8365"],"issn-type":[{"value":"0926-9630","type":"print"},{"value":"1879-8365","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,6]]}}}