{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T02:45:36Z","timestamp":1787021136675,"version":"build-2736575974"},"reference-count":43,"publisher":"American Society of Clinical Oncology (ASCO)","issue":"7","content-domain":{"domain":["ascopubs.org"],"crossmark-restriction":true},"short-container-title":["JCO Clin Cancer Inform"],"published-print":{"date-parts":[[2023,9]]},"abstract":"<jats:sec>\n                    <jats:title>PURPOSE<\/jats:title>\n                    <jats:p>To develop and validate natural language processing (NLP)\u2013assisted machine learning (ML)\u2013based classification models to confirm diagnoses of monoclonal gammopathy of undetermined significance (MGUS) and multiple myeloma (MM) from electronic health records (EHRs) in the Veterans Health Administration (VHA).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>MATERIALS AND METHODS<\/jats:title>\n                    <jats:p>We developed precompiled lexicons and classification rules as features for the following ML classifiers: logistic regression, random forest, and support vector machines (SVMs). These features were trained on 36,044 EHR documents from a random sample of 400 patients with at least one International Classification of Disease code for MGUS diagnosis from 1999 to 2021. The best-performing feature combination was calibrated in the validation set (17,826 documents\/200 patients) and evaluated in the testing set (9,250 documents\/100 patients). Model performance in diagnosis confirmation was compared with manual chart review results (gold standard) using recall, precision, accuracy, and F1 score. For patients correctly labeled as disease-positive, the difference between model-identified diagnosis dates and the gold standard was also computed.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>RESULTS<\/jats:title>\n                    <jats:p>In the testing set, the NLP-assisted classification model using SVMs achieved best performance in both MGUS and MM confirmation with recall\/precision\/accuracy\/F1 of 98.8%\/93.3%\/93.0%\/96.0% for MGUS and 100.0%\/92.3%\/99.0%\/96.0% for MM. Dates of diagnoses matched (\u00b145 days) with those of gold standard in 73.0% of model-confirmed MGUS and 84.6% of model-confirmed MM.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>CONCLUSION<\/jats:title>\n                    <jats:p>An NLP-assisted classification model can reliably confirm MGUS and MM diagnoses and dates and extract laboratory results using automated interpretation of EHR data. This algorithm has the potential to be adapted to other disease areas in VHA EHR system.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1200\/cci.23.00081","type":"journal-article","created":{"date-parts":[[2023,12,4]],"date-time":"2023-12-04T16:06:23Z","timestamp":1701705983000},"update-policy":"https:\/\/doi.org\/10.1200\/crossmark","source":"Crossref","is-referenced-by-count":12,"title":["Natural Language Processing\u2013Assisted Classification Models to Confirm Monoclonal Gammopathy of Undetermined Significance and Progression in Veterans' Electronic Health Records"],"prefix":"10.1200","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3560-9460","authenticated-orcid":true,"given":"Mei","family":"Wang","sequence":"first","affiliation":[{"name":"Research Service, St Louis Veterans Affairs Medical Center, St Louis, MO"},{"name":"Department of Surgery, Washington University School of Medicine, St Louis, MO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yao-Chi","family":"Yu","sequence":"additional","affiliation":[{"name":"Research Service, St Louis Veterans Affairs Medical Center, St Louis, MO"},{"name":"Department of Electrical and Systems Engineering, Washington University in St Louis, St Louis, MO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8559-7739","authenticated-orcid":true,"given":"Lawrence","family":"Liu","sequence":"additional","affiliation":[{"name":"Research Service, St Louis Veterans Affairs Medical Center, St Louis, MO"},{"name":"City of Hope National Comprehensive Cancer Center, Duarte, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6388-5553","authenticated-orcid":true,"given":"Martin W.","family":"Schoen","sequence":"additional","affiliation":[{"name":"Research Service, St Louis Veterans Affairs Medical Center, St Louis, MO"},{"name":"Department of Medicine, Saint Louis University School of Medicine, St Louis, MO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Akhil","family":"Kumar","sequence":"additional","affiliation":[{"name":"Research Service, St Louis Veterans Affairs Medical Center, St Louis, MO"},{"name":"Department of Surgery, Washington University School of Medicine, St Louis, MO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristin","family":"Vargo","sequence":"additional","affiliation":[{"name":"Research Service, St Louis Veterans Affairs Medical Center, St Louis, MO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7307-0291","authenticated-orcid":true,"given":"Graham","family":"Colditz","sequence":"additional","affiliation":[{"name":"Department of Surgery, Washington University School of Medicine, St Louis, MO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Theodore","family":"Thomas","sequence":"additional","affiliation":[{"name":"Research Service, St Louis Veterans Affairs Medical Center, St Louis, MO"},{"name":"Department of Medicine, Washington University School of Medicine, St Louis, MO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5872-9556","authenticated-orcid":true,"given":"Su-Hsin","family":"Chang","sequence":"additional","affiliation":[{"name":"Research Service, St Louis Veterans Affairs Medical Center, St Louis, MO"},{"name":"Department of Surgery, Washington University School of Medicine, St Louis, MO"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"233","reference":[{"key":"b1","volume-title":"Monoclonal Gammopathy of Undetermined Significance","author":"Kaseb H","year":"2022"},{"key":"b3","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa01133202"},{"key":"b4","doi-asserted-by":"publisher","DOI":"10.1182\/blood-2005-03-1038"},{"key":"b5","doi-asserted-by":"publisher","DOI":"10.4065\/79.7.859"},{"key":"b6","doi-asserted-by":"publisher","DOI":"10.1056\/NEJM198006123022405"},{"key":"b7","doi-asserted-by":"publisher","DOI":"10.1038\/leu.2010.60"},{"key":"b8","doi-asserted-by":"publisher","DOI":"10.1007\/s00277-021-04400-7"},{"key":"b9","doi-asserted-by":"publisher","DOI":"10.1016\/j.clml.2015.06.003"},{"key":"b10","doi-asserted-by":"publisher","DOI":"10.3390\/cancers14225637"},{"key":"b11","doi-asserted-by":"publisher","DOI":"10.1016\/j.juro.2013.04.048"},{"key":"b12","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2016.00224"},{"key":"b13","volume":"40","author":"Wang M","year":"2022","journal-title":"J Clin Oncol"},{"key":"b14","doi-asserted-by":"publisher","DOI":"10.1002\/pds.5579"},{"key":"b16","doi-asserted-by":"publisher","DOI":"10.3390\/cancers13123053"},{"key":"b17","doi-asserted-by":"publisher","DOI":"10.1093\/jnci\/djab007"},{"key":"b18","doi-asserted-by":"publisher","DOI":"10.1002\/ajh.25603"},{"key":"b19","author":"Fiala MA","journal-title":"JCO Clin Cancer Inform"},{"key":"b20","doi-asserted-by":"publisher","DOI":"10.1158\/1055-9965.EPI-19-0132"},{"key":"b21","doi-asserted-by":"publisher","DOI":"10.1016\/S2352-3026(14)00037-4"},{"key":"b22","doi-asserted-by":"publisher","DOI":"10.1093\/jnci\/djw264"},{"key":"b23","doi-asserted-by":"publisher","DOI":"10.4065\/78.1.21"},{"key":"b24","doi-asserted-by":"publisher","DOI":"10.1016\/j.anai.2013.07.022"},{"key":"b25","first-page":"233","volume":"2020","author":"Blackley SV","year":"2020","journal-title":"AMIA Annu Symp Proc"},{"key":"b26","doi-asserted-by":"publisher","DOI":"10.1200\/CCI.21.00030"},{"key":"b27","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000431"},{"key":"b28","doi-asserted-by":"publisher","DOI":"10.1055\/s-0038-1626725"},{"key":"b29","doi-asserted-by":"publisher","DOI":"10.1111\/jnu.12631"},{"key":"b31","doi-asserted-by":"publisher","DOI":"10.1016\/S1386-5056(02)00131-4"},{"key":"b32","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-017-0501-x"},{"key":"b33","doi-asserted-by":"publisher","DOI":"10.1016\/S1470-2045(14)70442-5"},{"key":"b35","doi-asserted-by":"publisher","DOI":"10.1006\/jbin.2001.1029"},{"key":"b36","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2009.05.002"},{"key":"b37","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2005.07.002"},{"key":"b38","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2007.07.002"},{"key":"b39","first-page":"189","volume":"2011","author":"Carroll RJ","year":"2011","journal-title":"AMIA Annu Symp Proc"},{"key":"b40","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511809071"},{"key":"b41","doi-asserted-by":"publisher","DOI":"10.1007\/BF00994018"},{"key":"b42","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"b43","doi-asserted-by":"publisher","DOI":"10.1186\/cc3045"},{"key":"b44","doi-asserted-by":"publisher","DOI":"10.1002\/pds.4711"},{"key":"b45","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2009.04.001"},{"key":"b46","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.7476"},{"key":"b47","doi-asserted-by":"publisher","DOI":"10.1200\/EDBK_159009"}],"container-title":["JCO Clinical Cancer Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascopubs.org\/doi\/pdfdirect\/10.1200\/CCI.23.00081","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,4]],"date-time":"2023-12-04T16:06:28Z","timestamp":1701705988000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascopubs.org\/doi\/10.1200\/CCI.23.00081"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9]]},"references-count":43,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2023,9]]}},"alternative-id":["10.1200\/CCI.23.00081"],"URL":"https:\/\/doi.org\/10.1200\/cci.23.00081","relation":{},"ISSN":["2473-4276"],"issn-type":[{"value":"2473-4276","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9]]},"assertion":[{"value":"2023-05-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-08-15","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-04","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2300081"}}