{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,22]],"date-time":"2025-03-22T11:34:50Z","timestamp":1742643290427},"reference-count":36,"publisher":"Oxford University Press (OUP)","issue":"10","license":[{"start":{"date-parts":[[2021,8,7]],"date-time":"2021-08-07T00:00:00Z","timestamp":1628294400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,9,18]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>De-identification is a fundamental task in electronic health records to remove protected health information entities. Deep learning models have proven to be promising tools to automate de-identification processes. However, when the target domain (where the model is applied) is different from the source domain (where the model is trained), the model often suffers a significant performance drop, commonly referred to as domain adaptation issue. In de-identification, domain adaptation issues can make the model vulnerable for deployment. In this work, we aim to close the domain gap by leveraging unlabeled data from the target domain.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We introduce a self-training framework to address the domain adaptation issue by leveraging unlabeled data from the target domain. We validate the effectiveness on 4 standard de-identification datasets. In each experiment, we use a pair of datasets: labeled data from the source domain and unlabeled data from the target domain. We compare the proposed self-training framework with supervised learning that directly deploys the model trained on the source domain.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>In summary, our proposed framework improves the F1-score by 5.38 (on average) when compared with direct deployment. For example, using i2b2-2014 as the training dataset and i2b2-2006 as the test, the proposed framework increases the F1-score from 76.61 to 85.41 (+8.8). The method also increases the F1-score by 10.86 for mimic-radiology and mimic-discharge.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>Our work demonstrates an effective self-training framework to boost the domain adaptation performance for the de-identification task for electronic health records.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocab128","type":"journal-article","created":{"date-parts":[[2021,6,5]],"date-time":"2021-06-05T11:20:07Z","timestamp":1622892007000},"page":"2093-2100","source":"Crossref","is-referenced-by-count":4,"title":["Improving domain adaptation in de-identification of electronic health records through self-training"],"prefix":"10.1093","volume":"28","author":[{"given":"Shun","family":"Liao","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Toronto, Toronto, Ontario, Canada"},{"name":"Donnelly Centre for Cellular and Biomoleular Research, University of Toronto, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jamie","family":"Kiros","sequence":"additional","affiliation":[{"name":"Google, Toronto, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiyang","family":"Chen","sequence":"additional","affiliation":[{"name":"Google, Toronto, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhaolei","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Toronto, Toronto, Ontario, Canada"},{"name":"Donnelly Centre for Cellular and Biomoleular Research, University of Toronto, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ting","family":"Chen","sequence":"additional","affiliation":[{"name":"Google, Toronto, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2021,8,7]]},"reference":[{"key":"2021091817522855800_ocab128-B1","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","article-title":"Scalable and accurate deep learning with electronic health records","volume":"1","author":"Rajkomar","year":"2018","journal-title":"NPJ Digit Med"},{"key":"2021091817522855800_ocab128-B2","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1186\/1472-6947-8-32","article-title":"Automated de-identification of free-text medical records","volume":"8","author":"Neamatullah","year":"2008","journal-title":"BMC Med Inform Decis Mak"},{"issue":"3","key":"2021091817522855800_ocab128-B3","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1001\/jama.2018.5630","article-title":"HIPAA and protecting health information in the 21st Century","volume":"320","author":"Cohen","year":"2018","journal-title":"JAMA"},{"issue":"3","key":"2021091817522855800_ocab128-B4","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1093\/jamia\/ocw156","article-title":"De-identification of patient notes with recurrent neural networks","volume":"24","author":"Dernoncourt","year":"2017","journal-title":"J Am Med Inform Assoc"},{"key":"2021091817522855800_ocab128-B5","doi-asserted-by":"crossref","first-page":"S34","DOI":"10.1016\/j.jbi.2017.05.023","article-title":"De-identification of clinical notes via recurrent neural network and conditional random field","volume":"75S","author":"Liu","year":"2017","journal-title":"J Biomed Inform"},{"key":"2021091817522855800_ocab128-B6","first-page":"1070","article-title":"Leveraging existing corpora for de-identification of psychiatric notes using domain adaptation","volume":"2017","author":"Lee","year":"2017","journal-title":"AMIA Annu Symp Proc"},{"key":"2021091817522855800_ocab128-B7","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1186\/s12911-020-1026-2","article-title":"Customization scenarios for de-identification of clinical notes","volume":"20","author":"Hartman","year":"2020","journal-title":"BMC Med Inform Decis Mak"},{"issue":"2","key":"2021091817522855800_ocab128-B8","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1136\/jamia.2009.000893","article-title":"Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2)","volume":"17","author":"Murphy","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2021091817522855800_ocab128-B9","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1080\/08839514.2020.1718343","article-title":"A review of automatic end-to-end de-identification: is high accuracy the only metric?","volume":"34","author":"Yogarajan","year":"2020","journal-title":"Appl Artif Intell"},{"key":"2021091817522855800_ocab128-B10","first-page":"139","volume-title":"ALT 2012: Algorithmic Learning Theory. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7568","author":"Ben-David","year":"2012"},{"key":"2021091817522855800_ocab128-B11","article-title":"Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks","author":"Lee","year":"2013","journal-title":"ICML 2013 Workshop on Challenges in Representation Learning"},{"key":"2021091817522855800_ocab128-B12","doi-asserted-by":"crossref","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"key":"2021091817522855800_ocab128-B13","author":"Ren"},{"key":"2021091817522855800_ocab128-B14","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2019","journal-title":"NAACL HLT 2019 - Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies"},{"issue":"5","key":"2021091817522855800_ocab128-B15","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1197\/jamia.M2444","article-title":"Evaluating the state-of-the-art in automatic de-identification","volume":"14","author":"Uzuner","year":"2007","journal-title":"J Am Med Inform Assoc"},{"key":"2021091817522855800_ocab128-B16","first-page":"476","article-title":"Automatic de-identification of French clinical records: comparison of rule-based and machine-learning approaches","volume":"192","author":"Grouin","year":"2013","journal-title":"Stud Health Technol Inform"},{"key":"2021091817522855800_ocab128-B17","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1186\/1471-2288-10-70","article-title":"Automatic de-identification of textual documents in the electronic health record: a review of recent research","volume":"10","author":"Meystre","year":"2010","journal-title":"BMC Med Res Methodol"},{"key":"2021091817522855800_ocab128-B18","first-page":"32","article-title":"Deep learning architecture for patient data de-identification in clinical records","author":"Yadav","year":"2016","journal-title":"Proceedings of the Clinical Natural Language Processing Workshop (Clinical NLP);"},{"key":"2021091817522855800_ocab128-B20","doi-asserted-by":"crossref","first-page":"S19","DOI":"10.1016\/j.jbi.2017.06.006","article-title":"A hybrid approach to automatic de-identification of psychiatric notes","volume":"75S","author":"Lee","year":"2017","journal-title":"J Biomed Inform"},{"key":"2021091817522855800_ocab128-B21","author":"Kouw","year":"2019"},{"key":"2021091817522855800_ocab128-B22","author":"Xie","year":"2020"},{"key":"2021091817522855800_ocab128-B23","author":"Xie","year":"2020"},{"key":"2021091817522855800_ocab128-B24","first-page":"5001","author":"Inoue","year":"2018"},{"key":"2021091817522855800_ocab128-B25","author":"Raghunathan","year":"2020"},{"key":"2021091817522855800_ocab128-B26","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.cosrev.2018.06.001","article-title":"Recent named entity recognition and classification techniques: a systematic review","volume":"29","author":"Goyal","year":"2018","journal-title":"Comput Sci Rev"},{"issue":"1","key":"2021091817522855800_ocab128-B27","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1136\/amiajnl-2012-001012","article-title":"Large-scale evaluation of automated clinical note de-identification and its impact on information extraction","volume":"20","author":"Deleger","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2021091817522855800_ocab128-B28","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","author":"Srivastava","year":"2014","journal-title":"J Mach Learn Res"},{"key":"2021091817522855800_ocab128-B29","first-page":"54","author":"Akbik","year":"2019"},{"key":"2021091817522855800_ocab128-B30","first-page":"1223","author":"Dean","year":"2012"},{"key":"2021091817522855800_ocab128-B31","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.neucom.2015.09.116","article-title":"Deep learning for visual understanding: a review","volume":"187","author":"Guo","year":"2016","journal-title":"Neurocomputing"},{"key":"2021091817522855800_ocab128-B32","first-page":"857","article-title":"De-identification of clinical text via Bi-LSTM-CRF with neural language models","volume":"2019","author":"Tang","year":"2019","journal-title":"AMIA Annu Symp Proc"},{"key":"2021091817522855800_ocab128-B33","author":"Carmon","year":"2019"},{"key":"2021091817522855800_ocab128-B34","author":"Brown","year":"2020"},{"key":"2021091817522855800_ocab128-B35","author":"Zhao","year":"2019"},{"key":"2021091817522855800_ocab128-B36","author":"Ramponi","year":"2020"},{"key":"2021091817522855800_ocab128-B37","author":"Wei","year":"2021"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/10\/2093\/40408836\/ocab128.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/10\/2093\/40408836\/ocab128.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,18]],"date-time":"2021-09-18T17:53:15Z","timestamp":1631987595000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/10\/2093\/6345434"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,7]]},"references-count":36,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,8,7]]},"published-print":{"date-parts":[[2021,9,18]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab128","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,10,1]]},"published":{"date-parts":[[2021,8,7]]}}}