{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T19:57:41Z","timestamp":1767988661379,"version":"3.49.0"},"reference-count":34,"publisher":"Oxford University Press (OUP)","issue":"9","license":[{"start":{"date-parts":[[2021,6,21]],"date-time":"2021-06-21T00:00:00Z","timestamp":1624233600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["T15 LM0007079"],"award-info":[{"award-number":["T15 LM0007079"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000057","name":"National Institute of General Medical Sciences","doi-asserted-by":"publisher","award":["R01 GM114355"],"award-info":[{"award-number":["R01 GM114355"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,8,13]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Clinical notes present a wealth of information for applications in the clinical domain, but heterogeneity across clinical institutions and settings presents challenges for their processing. The clinical natural language processing field has made strides in overcoming domain heterogeneity, while pretrained deep learning models present opportunities to transfer knowledge from one task to another. Pretrained models have performed well when transferred to new tasks; however, it is not well understood if these models generalize across differences in institutions and settings within the clinical domain. We explore if institution or setting specific pretraining is necessary for pretrained models to perform well when transferred to new tasks. We find no significant performance difference between models pretrained across institutions and settings, indicating that clinically pretrained models transfer well across such boundaries. Given a clinically pretrained model, clinical natural language processing researchers may forgo the time-consuming pretraining step without a significant performance drop.<\/jats:p>","DOI":"10.1093\/jamia\/ocab086","type":"journal-article","created":{"date-parts":[[2021,5,4]],"date-time":"2021-05-04T11:40:24Z","timestamp":1620128424000},"page":"1970-1976","source":"Crossref","is-referenced-by-count":12,"title":["Clinically relevant pretraining is all you need"],"prefix":"10.1093","volume":"28","author":[{"given":"Oliver J","family":"Bear Don\u2019t Walk IV","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tony","family":"Sun","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6695-0282","authenticated-orcid":false,"given":"Adler","family":"Perotte","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"No\u00e9mie","family":"Elhadad","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University, New York, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2021,6,21]]},"reference":[{"key":"2021081407010440800_ocab086-B1","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1002\/pds.1981","article-title":"Automated data capture from free-text radiology reports to enhance accuracy of hospital inpatient stroke codes","volume":"19","author":"Flynn","year":"2010","journal-title":"Pharmacoepidemiol Drug Saf"},{"key":"2021081407010440800_ocab086-B2","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1197\/jamia.M3096","article-title":"A text mining approach to the prediction of disease status from clinical discharge summaries","volume":"16","author":"Yang","year":"2009","journal-title":"J Am Med Inform Assoc"},{"key":"2021081407010440800_ocab086-B3","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1136\/jamia.1994.95236146","article-title":"A general natural-language text processor for clinical radiology","volume":"1","author":"Friedman","year":"1994","journal-title":"J Am Med Inform Assoc"},{"key":"2021081407010440800_ocab086-B4","article-title":"Automatic structured reporting from narrative cancer pathology reports","volume":"8","author":"Ou","year":"2014","journal-title":"Electron J Health Inform"},{"issue":"7","key":"2021081407010440800_ocab086-B5","doi-asserted-by":"crossref","first-page":"1130","DOI":"10.1016\/j.cgh.2013.11.025","article-title":"Clinical decision support with natural language processing facilitates determination of colonoscopy surveillance intervals","volume":"12","author":"Imler","year":"2014","journal-title":"Clin Gastroenterol Hepatol"},{"issue":"5","key":"2021081407010440800_ocab086-B6","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1016\/j.ipm.2015.04.006","article-title":"MEANS: a medical question-answering system combining NLP techniques and semantic Web technologies","volume":"51","author":"Ben Abacha","year":"2015","journal-title":"Inf Process Manag"},{"key":"2021081407010440800_ocab086-B7","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1093\/jamia\/ocx138","article-title":"Clinical documentation variations and NLP system portability: a case study in asthma birth cohorts across institutions","volume":"25","author":"Sohn","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"2021081407010440800_ocab086-B8","first-page":"577","article-title":"A study of transportability of an existing smoking status detection module across institutions","volume":"2012","author":"Liu","year":"2012","journal-title":"AMIA Annu Symp Proc"},{"key":"2021081407010440800_ocab086-B9","first-page":"472","article-title":"A comparison of semantic categories of the ISO reference terminology models for nursing and the MedLEE natural language processing system","volume":"107","author":"Bakken","year":"2004","journal-title":"Stud Health Technol Inform"},{"key":"2021081407010440800_ocab086-B10","first-page":"742","article-title":"The sublanguage of cross-coverage","author":"Stetson","year":"2002","journal-title":"Proc AMIA Symp"},{"key":"2021081407010440800_ocab086-B11","first-page":"270","article-title":"A broad-coverage natural language processing system","author":"Friedman","year":"2000","journal-title":"Proc AMIA Symp"},{"issue":"4","key":"2021081407010440800_ocab086-B12","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1016\/S1532-0464(03)00012-1","article-title":"Two biomedical sublanguages: a description based on the theories of Zellig Harris","volume":"35","author":"Friedman","year":"2002","journal-title":"J Biomed Inform"},{"key":"2021081407010440800_ocab086-B13","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1197\/jamia.M2927","article-title":"Methods for building sense inventories of abbreviations in clinical notes","volume":"16","author":"Xu","year":"2009","journal-title":"J Am Med Inform Assoc"},{"key":"2021081407010440800_ocab086-B14","doi-asserted-by":"crossref","first-page":"e79\u201386","DOI":"10.1093\/jamia\/ocw109","article-title":"A long journey to short abbreviations: developing an open-source framework for clinical abbreviation recognition and disambiguation (CARD)","volume":"24","author":"Wu","year":"2017","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2021081407010440800_ocab086-B15","doi-asserted-by":"crossref","first-page":"967","DOI":"10.1093\/jamia\/ocu048","article-title":"Domain adaptation for semantic role labeling of clinical text","volume":"22","author":"Zhang","year":"2015","journal-title":"J Am Med Inform Assoc"},{"key":"2021081407010440800_ocab086-B16","first-page":"4171","author":"Devlin","year":"2019"},{"key":"2021081407010440800_ocab086-B17","first-page":"328","author":"Howard","year":"2018"},{"key":"2021081407010440800_ocab086-B18","author":"Radford","year":"2018"},{"key":"2021081407010440800_ocab086-B19","first-page":"2227","author":"Peters","year":"2018"},{"key":"2021081407010440800_ocab086-B20","author":"Liu"},{"key":"2021081407010440800_ocab086-B21","first-page":"479","author":"Mou","year":"2016"},{"key":"2021081407010440800_ocab086-B22","first-page":"3111","author":"Mikolov","year":"2013"},{"key":"2021081407010440800_ocab086-B23","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3115\/v1\/D14-1162","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Pennington","year":"2014"},{"key":"2021081407010440800_ocab086-B24","author":"Joulin"},{"key":"2021081407010440800_ocab086-B25","first-page":"4465","author":"Wang","year":"2019"},{"key":"2021081407010440800_ocab086-B26","first-page":"72","author":"Alsentzer","year":"2019"},{"key":"2021081407010440800_ocab086-B27","author":"Huang"},{"key":"2021081407010440800_ocab086-B28","first-page":"58","author":"Peng","year":"2019"},{"key":"2021081407010440800_ocab086-B29","author":"Valmianski"},{"key":"2021081407010440800_ocab086-B30","author":"Selvaraj"},{"key":"2021081407010440800_ocab086-B31","doi-asserted-by":"crossref","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"key":"2021081407010440800_ocab086-B32","first-page":"422","article-title":"Towards the inference of social and behavioral determinants of sexual health: Development of a gold-standard corpus with semi-supervised learning","volume":"2018","author":"Feller","year":"2018","journal-title":"AMIA Annu Symp Proc"},{"key":"2021081407010440800_ocab086-B33","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1093\/jamia\/ocx059","article-title":"Mining 100 million notes to find homelessness and adverse childhood experiences: 2 case studies of rare and severe social determinants of health in electronic health records","volume":"25","author":"Bejan","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"2021081407010440800_ocab086-B34","first-page":"349","author":"Su","year":"2015"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/9\/1970\/39731673\/ocab086.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/9\/1970\/39731673\/ocab086.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,14]],"date-time":"2021-08-14T07:04:25Z","timestamp":1628924665000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/9\/1970\/6307175"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,21]]},"references-count":34,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,6,21]]},"published-print":{"date-parts":[[2021,8,13]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab086","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,9,1]]},"published":{"date-parts":[[2021,6,21]]}}}