{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T21:13:52Z","timestamp":1780089232721,"version":"3.54.0"},"reference-count":36,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2020,12,23]],"date-time":"2020-12-23T00:00:00Z","timestamp":1608681600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["5R01LM009886-11"],"award-info":[{"award-number":["5R01LM009886-11"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,3,18]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>The study sought to develop and evaluate a knowledge-based data augmentation method to improve the performance of deep learning models for biomedical natural language processing by overcoming training data scarcity.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>We extended the easy data augmentation (EDA) method for biomedical named entity recognition (NER) by incorporating the Unified Medical Language System (UMLS) knowledge and called this method UMLS-EDA. We designed experiments to systematically evaluate the effect of UMLS-EDA on popular deep learning architectures for both NER and classification. We also compared UMLS-EDA to BERT.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>UMLS-EDA enables substantial improvement for NER tasks from the original long short-term memory conditional random fields (LSTM-CRF) model (micro-F1 score: +5%, + 17%, and +15%), helps the LSTM-CRF model (micro-F1 score: 0.66) outperform LSTM-CRF with transfer learning by BERT (0.63), and improves the performance of the state-of-the-art sentence classification model. The largest gain on micro-F1 score is 9%, from 0.75 to 0.84, better than classifiers with BERT pretraining (0.82).<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>This study presents a UMLS-based data augmentation method, UMLS-EDA. It is effective at improving deep learning models for both NER and sentence classification, and contributes original insights for designing new, superior deep learning approaches for low-resource biomedical domains.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocaa309","type":"journal-article","created":{"date-parts":[[2020,11,23]],"date-time":"2020-11-23T22:22:02Z","timestamp":1606170122000},"page":"812-823","source":"Crossref","is-referenced-by-count":49,"title":["UMLS-based data augmentation for natural language processing of clinical research literature"],"prefix":"10.1093","volume":"28","author":[{"given":"Tian","family":"Kang","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adler","family":"Perotte","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Youlan","family":"Tang","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Casey","family":"Ta","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9624-0214","authenticated-orcid":false,"given":"Chunhua","family":"Weng","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,12,23]]},"reference":[{"issue":"1","key":"2021031906033881300_ocaa309-B1","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1055\/s-0039-1677937","article-title":"A year of papers using biomedical texts: findings from the section on natural language processing of the IMIA yearbook","volume":"28","author":"Grabar","year":"2019","journal-title":"Yearb Med Inform"},{"issue":"8","key":"2021031906033881300_ocaa309-B2","doi-asserted-by":"crossref","first-page":"890","DOI":"10.1097\/00001888-199908000-00012","article-title":"Natural language processing and its future in medicine","volume":"74","author":"Friedman","year":"1999","journal-title":"Acad Med"},{"issue":"1","key":"2021031906033881300_ocaa309-B3","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1038\/s41591-018-0316-z","article-title":"A guide to deep learning in healthcare","volume":"25","author":"Esteva","year":"2019","journal-title":"Nat Med"},{"key":"2021031906033881300_ocaa309-B4","author":"Joulin","year":"2015"},{"key":"2021031906033881300_ocaa309-B5","first-page":"853","author":"Sun","year":"2017"},{"key":"2021031906033881300_ocaa309-B6","author":"Hestness","year":"2017"},{"issue":"1","key":"2021031906033881300_ocaa309-B7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12911-018-0723-6","article-title":"A clinical text classification paradigm using weak supervision and deep representation","volume":"19","author":"Wang","year":"2019","journal-title":"BMC Med Inform Decis Mak"},{"issue":"1","key":"2021031906033881300_ocaa309-B8","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1038\/s41746-019-0122-0","article-title":"Deep learning and alternative learning strategies for retrospective real-world clinical data","volume":"2","author":"Chen","year":"2019","journal-title":"NPJ Digit Med"},{"issue":"5","key":"2021031906033881300_ocaa309-B9","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1016\/j.crad.2017.11.015","article-title":"Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks","volume":"73","author":"Kim","year":"2018","journal-title":"Clin Radiol"},{"key":"2021031906033881300_ocaa309-B10","doi-asserted-by":"crossref","first-page":"117822261771299","DOI":"10.1177\/1178222617712994","article-title":"Using transfer learning for improved mortality prediction in a data-scarce hospital setting","volume":"9","author":"Desautels","year":"2017","journal-title":"Biomed Inform Insights"},{"key":"2021031906033881300_ocaa309-B11","author":"Devlin","year":"2018"},{"key":"2021031906033881300_ocaa309-B12","first-page":"5753","author":"Yang","year":"2019"},{"key":"2021031906033881300_ocaa309-B13","author":"Lee","year":"2019"},{"key":"2021031906033881300_ocaa309-B14","author":"Peng","year":"2019"},{"key":"2021031906033881300_ocaa309-B15","author":"Adhikari","year":"2019"},{"key":"2021031906033881300_ocaa309-B16","first-page":"649","volume-title":"Advances in Neural Information Processing Systems 28 (NIPS 2015)","author":"Zhang","year":"2015"},{"key":"2021031906033881300_ocaa309-B17","author":"Sennrich","year":"2015"},{"key":"2021031906033881300_ocaa309-B18","author":"Wei","year":"2019"},{"key":"2021031906033881300_ocaa309-B19","author":"Skreta","year":"2019"},{"issue":"7","key":"2021031906033881300_ocaa309-B20","doi-asserted-by":"crossref","first-page":"e0216913","DOI":"10.1371\/journal.pone.0216913","article-title":"Using distant supervision to augment manually annotated data for relation extraction","volume":"14","author":"Su","year":"2019","journal-title":"PLoS One"},{"key":"2021031906033881300_ocaa309-B21","author":"Kumar","year":"2019"},{"issue":"1","key":"2021031906033881300_ocaa309-B22","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/S0146-0005(97)80013-4","article-title":"Evidence-based medicine","volume":"21","author":"Sackett","year":"1997","journal-title":"Semin Perinatol"},{"issue":"2","key":"2021031906033881300_ocaa309-B23","doi-asserted-by":"crossref","first-page":"e012545","DOI":"10.1136\/bmjopen-2016-012545","article-title":"Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry","volume":"7","author":"Borah","year":"2017","journal-title":"BMJ Open"},{"issue":"9","key":"2021031906033881300_ocaa309-B24","doi-asserted-by":"crossref","first-page":"e1000326","DOI":"10.1371\/journal.pmed.1000326","article-title":"Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?","volume":"7","author":"Bastian","year":"2010","journal-title":"PLoS Med"},{"issue":"5","key":"2021031906033881300_ocaa309-B25","doi-asserted-by":"crossref","first-page":"548","DOI":"10.1002\/pmrj.12116","article-title":"Asking structured, answerable clinical questions using the Population, Intervention\/Comparator, Outcome (PICO) framework","volume":"11","author":"Speckman","year":"2019","journal-title":"PM R"},{"issue":"2","key":"2021031906033881300_ocaa309-B26","doi-asserted-by":"crossref","first-page":"75","DOI":"10.18438\/B8WS5N","article-title":"Formulating the evidence based practice question: a review of the frameworks","volume":"6","author":"Davies","year":"2011","journal-title":"Evid Based Libr Inf Pract"},{"issue":"3","key":"2021031906033881300_ocaa309-B27","doi-asserted-by":"crossref","first-page":"A12","DOI":"10.7326\/ACPJC-1995-123-3-A12","article-title":"The well-built clinical question: a key to evidence-based decisions","volume":"123","author":"Richardson","year":"1995","journal-title":"ACP J Club"},{"issue":"1","key":"2021031906033881300_ocaa309-B28","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1162\/coli.2007.33.1.63","article-title":"Answering clinical questions with knowledge-based and statistical techniques","volume":"33","author":"Demner-Fushman","year":"2007","journal-title":"Comput Linguist"},{"key":"2021031906033881300_ocaa309-B29","author":"Huang","year":"2011"},{"issue":"1","key":"2021031906033881300_ocaa309-B30","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1186\/1472-6947-9-10","article-title":"Sentence retrieval for abstracts of randomized controlled trials","volume":"9","author":"Chung","year":"2009","journal-title":"BMC Med Inform Decis Mak"},{"key":"2021031906033881300_ocaa309-B31","article-title":"Automatic classification of sentences to support evidence based medicine","volume":"12 (Suppl 2","year":"2011","journal-title":"BMC Bioinformatics"},{"key":"2021031906033881300_ocaa309-B32","author":"Nye","year":"2018"},{"key":"2021031906033881300_ocaa309-B33","year":"2016"},{"key":"2021031906033881300_ocaa309-B34","first-page":"188","article-title":"Pretraining to recognize PICO elements from randomized controlled trial literature","volume":"264","author":"Kang","year":"2019","journal-title":"Stud Health Technol Inform"},{"key":"2021031906033881300_ocaa309-B35","first-page":"67","author":"Jin","year":"2018"},{"key":"2021031906033881300_ocaa309-B36","author":"Abadi","year":"2016"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/4\/812\/36642182\/ocaa309.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/4\/812\/36642182\/ocaa309.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,29]],"date-time":"2022-11-29T11:39:49Z","timestamp":1669721989000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/4\/812\/6046153"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,23]]},"references-count":36,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,12,23]]},"published-print":{"date-parts":[[2021,3,18]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa309","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,4,1]]},"published":{"date-parts":[[2020,12,23]]}}}