{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T05:36:22Z","timestamp":1776922582230,"version":"3.51.2"},"reference-count":29,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2023,9,19]],"date-time":"2023-09-19T00:00:00Z","timestamp":1695081600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01LM012973"],"award-info":[{"award-number":["R01LM012973"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01LM012918"],"award-info":[{"award-number":["R01LM012918"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01LM013486"],"award-info":[{"award-number":["R01LM013486"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,12,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>The classification of clinical note sections is a critical step before doing more fine-grained natural language processing tasks such as social determinants of health extraction and temporal information extraction. Often, clinical note section classification models that achieve high accuracy for 1 institution experience a large drop of accuracy when transferred to another institution. The objective of this study is to develop methods that classify clinical note sections under the SOAP (\u201cSubjective,\u201d \u201cObject,\u201d \u201cAssessment,\u201d and \u201cPlan\u201d) framework with improved transferability.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and methods<\/jats:title>\n                  <jats:p>We trained the baseline models by fine-tuning BERT-based models, and enhanced their transferability with continued pretraining, including domain-adaptive pretraining and task-adaptive pretraining. We added in-domain annotated samples during fine-tuning and observed model performance over a varying number of annotated sample size. Finally, we quantified the impact of continued pretraining in equivalence of the number of in-domain annotated samples added.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We found continued pretraining improved models only when combined with in-domain annotated samples, improving the F1 score from 0.756 to 0.808, averaged across 3 datasets. This improvement was equivalent to adding 35 in-domain annotated samples.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Although considered a straightforward task when performing in-domain, section classification is still a considerably difficult task when performing cross-domain, even using highly sophisticated neural network-based methods.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>Continued pretraining improved model transferability for cross-domain clinical note section classification in the presence of a small amount of in-domain labeled samples.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocad190","type":"journal-article","created":{"date-parts":[[2023,9,19]],"date-time":"2023-09-19T22:14:26Z","timestamp":1695161666000},"page":"89-97","source":"Crossref","is-referenced-by-count":6,"title":["Improving model transferability for clinical note section classification models using continued pretraining"],"prefix":"10.1093","volume":"31","author":[{"given":"Weipeng","family":"Zhou","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics and Medical Education, School of Medicine, University of Washington-Seattle , Seattle, WA, United States"}]},{"given":"Meliha","family":"Yetisgen","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics and Medical Education, School of Medicine, University of Washington-Seattle , Seattle, WA, United States"}]},{"given":"Majid","family":"Afshar","sequence":"additional","affiliation":[{"name":"Department of Medicine, School of Medicine and Public Health, University of Wisconsin-Madison , Madison, WI, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9341-7360","authenticated-orcid":false,"given":"Yanjun","family":"Gao","sequence":"additional","affiliation":[{"name":"Department of Medicine, School of Medicine and Public Health, University of Wisconsin-Madison , Madison, WI, United States"}]},{"given":"Guergana","family":"Savova","sequence":"additional","affiliation":[{"name":"Computational Health Informatics Program, Boston Children\u2019s Hospital, Department of Pediatrics, Harvard Medical School , Boston, MA, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4513-403X","authenticated-orcid":false,"given":"Timothy A","family":"Miller","sequence":"additional","affiliation":[{"name":"Computational Health Informatics Program, Boston Children\u2019s Hospital, Department of Pediatrics, Harvard Medical School , Boston, MA, United States"}]}],"member":"286","published-online":{"date-parts":[[2023,9,19]]},"reference":[{"issue":"5","key":"2023122220301306100_ocad190-B1","doi-asserted-by":"crossref","first-page":"808","DOI":"10.1136\/amiajnl-2013-002381","article-title":"A comprehensive study of named entity recognition in Chinese clinical text","volume":"21","author":"Lei","year":"2014","journal-title":"J Am Med Inform Assoc"},{"key":"2023122220301306100_ocad190-B2","author":"Zweigenbaum","year":"2013"},{"key":"2023122220301306100_ocad190-B3","first-page":"660","article-title":"Evaluation of clinical text segmentation to facilitate cohort retrieval","volume":"2017","author":"Edinger","year":"2018","journal-title":"AMIA Annu Symp Proc"},{"issue":"3","key":"2023122220301306100_ocad190-B4","doi-asserted-by":"crossref","first-page":"230","DOI":"10.3414\/ME16-01-0073","article-title":"Structuring legacy pathology reports by openEHR archetypes to enable semantic querying","volume":"56","author":"Kropf","year":"2017","journal-title":"Methods Inf Med"},{"key":"2023122220301306100_ocad190-B5","first-page":"2001","author":"Tepper","year":"2012"},{"key":"2023122220301306100_ocad190-B6","first-page":"4864","author":"Rosenthal","year":"2019"},{"key":"2023122220301306100_ocad190-B7","first-page":"4171","author":"Devlin","year":"2019"},{"key":"2023122220301306100_ocad190-B8","volume-title":"SOAP Notes in StatPearls","author":"Podder","year":"2022"},{"issue":"1","key":"2023122220301306100_ocad190-B9","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.jbi.2011.08.020","article-title":"Building an automated SOAP classifier for emergency department reports","volume":"45","author":"Mowery","year":"2012","journal-title":"J Biomed Inform"},{"issue":"6","key":"2023122220301306100_ocad190-B10","doi-asserted-by":"crossref","first-page":"964","DOI":"10.1136\/amiajnl-2014-002776","article-title":"Bringing science to medicine: an interview with Larry Weed, inventor of the problem-oriented medical record","volume":"21","author":"Wright","year":"2014","journal-title":"J Am Med Inform Assoc"},{"key":"2023122220301306100_ocad190-B11","first-page":"8342","author":"Gururangan","year":"2020"},{"key":"2023122220301306100_ocad190-B12","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1162\/tacl_a_00172","article-title":"Temporal annotation in the clinical domain","volume":"2","author":"Styler","year":"2014","journal-title":"Trans Assoc Comput Linguist"},{"issue":"1","key":"2023122220301306100_ocad190-B13","doi-asserted-by":"crossref","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"key":"2023122220301306100_ocad190-B14","article-title":"Hierarchical annotation for building a suite of clinical natural language processing tasks: progress note understanding","author":"Gao"},{"key":"2023122220301306100_ocad190-B15","article-title":"Tasks 1 and 3 from progress note understanding suite of tasks: SOAP note tagging and problem list summarization (version 1.0.0)","author":"Gao","journal-title":"PhysioNet"},{"issue":"5","key":"2023122220301306100_ocad190-B16","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1016\/j.ijmedinf.2007.09.001","article-title":"Definition, structure, content, use and impacts of electronic health records: A review of the research literature","volume":"77","author":"H\u00e4yrinen","year":"2008","journal-title":"Int J Med Inform"},{"issue":"4","key":"2023122220301306100_ocad190-B17","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"2023122220301306100_ocad190-B18","first-page":"49","author":"Yu","year":"2019"},{"key":"2023122220301306100_ocad190-B19","doi-asserted-by":"crossref","first-page":"101139","DOI":"10.1016\/j.imu.2022.101139","article-title":"Comparison of BERT implementations for natural language processing of narrative medical documents","volume":"36","author":"Turchin","year":"2023","journal-title":"Inform Med Unlocked"},{"issue":"7","key":"2023122220301306100_ocad190-B20","doi-asserted-by":"crossref","first-page":"e27527","DOI":"10.2196\/27527","article-title":"Relation classification for bleeding events from electronic health records using deep learning systems: an empirical study","volume":"9","author":"Mitra","year":"2021","journal-title":"JMIR Med Inform"},{"key":"2023122220301306100_ocad190-B21","first-page":"72","author":"Alsentzer","year":"2019"},{"issue":"1","key":"2023122220301306100_ocad190-B22","doi-asserted-by":"crossref","first-page":"7155","DOI":"10.1038\/s41598-020-62922-y","article-title":"BEHRT: transformer for electronic health records","volume":"10","author":"Li","year":"2020","journal-title":"Sci Rep"},{"issue":"1","key":"2023122220301306100_ocad190-B23","doi-asserted-by":"crossref","DOI":"10.1038\/s41746-022-00742-2","article-title":"A large language model for electronic health records","volume":"5","author":"Yang","year":"2022","journal-title":"NPJ Digit Med"},{"key":"2023122220301306100_ocad190-B24","author":"Wolf","year":"2020"},{"issue":"1","key":"2023122220301306100_ocad190-B25","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1186\/1472-6947-12-8","article-title":"Predicting sample size required for classification performance","volume":"12","author":"Figueroa","year":"2012","journal-title":"BMC Med Inform Decis Mak"},{"issue":"1","key":"2023122220301306100_ocad190-B26","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1186\/s12859-021-04163-y","article-title":"Learning curves for drug response prediction in cancer cell lines","volume":"22","author":"Partin","year":"2021","journal-title":"BMC Bioinformatics"},{"key":"2023122220301306100_ocad190-B27","first-page":"2314","article-title":"Machine learning model validation for early stage studies with small sample sizes","volume":"2021","author":"Larracy","year":"2021","journal-title":"Ann Int Conf IEEE Eng Med Biol Soc"},{"issue":"11","key":"2023122220301306100_ocad190-B28","doi-asserted-by":"crossref","first-page":"e112774","DOI":"10.1371\/journal.pone.0112774","article-title":"Negation\u2019s not solved: generalizability versus optimizability in clinical natural language processing","volume":"9","author":"Wu","year":"2014","journal-title":"PLoS One"},{"key":"2023122220301306100_ocad190-B29","author":"Lehman","year":"2023"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/1\/89\/54762101\/ocad190.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/1\/89\/54762101\/ocad190.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,22]],"date-time":"2023-12-22T20:30:42Z","timestamp":1703277042000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/1\/89\/7277369"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,19]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,9,19]]},"published-print":{"date-parts":[[2023,12,22]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocad190","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,1,1]]},"published":{"date-parts":[[2023,9,19]]}}}