{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T16:30:00Z","timestamp":1781109000539,"version":"3.54.1"},"reference-count":53,"publisher":"MIT Press","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Transactions of the Association for Computational Linguistics"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:p>Pivot-based neural representation models have led to significant progress in domain adaptation for NLP. However, previous research following this approach utilize only labeled data from the source domain and unlabeled data from the source and target domains, but neglect to incorporate massive unlabeled corpora that are not necessarily drawn from these domains. To alleviate this, we propose PERL: A representation learning model that extends contextualized word embedding models such as BERT (Devlin et al., 2019 ) with pivot-based fine-tuning. PERL outperforms strong baselines across 22 sentiment classification domain adaptation setups, improves in-domain model performance, yields effective reduced-size models, and increases model stability.<jats:sup>1<\/jats:sup><\/jats:p>","DOI":"10.1162\/tacl_a_00328","type":"journal-article","created":{"date-parts":[[2020,9,3]],"date-time":"2020-09-03T15:05:10Z","timestamp":1599145510000},"page":"504-521","source":"Crossref","is-referenced-by-count":23,"title":["PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models"],"prefix":"10.1162","volume":"8","author":[{"given":"Eyal","family":"Ben-David","sequence":"first","affiliation":[{"name":"Technion, Israel Institute of Technology."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carmel","family":"Rabinovitz","sequence":"additional","affiliation":[{"name":"Technion, Israel Institute of Technology."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roi","family":"Reichart","sequence":"additional","affiliation":[{"name":"Technion, Israel Institute of Technology."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","reference":[{"issue":"1","key":"bib1","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/s10994-009-5152-4","volume":"79","author":"Ben-David Shai","year":"2010","journal-title":"Machine Learning"},{"key":"bib2","volume-title":"ACL 2007, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23\u201330, 2007, Prague, Czech Republic","author":"Blitzer John","year":"2007"},{"key":"bib3","first-page":"120","volume-title":"EMNLP 2006, Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 22-23 July 2006, Sydney, Australia","author":"Blitzer John","year":"2006"},{"key":"bib4","first-page":"730","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26\u201331, 2015, Beijing, China, Volume 1: Long Papers","author":"Bollegala Danushka","year":"2015"},{"key":"bib5","first-page":"953","volume-title":"Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 \u2013 July 2, 2011","author":"Chen Minmin","year":"2011"},{"key":"bib6","volume-title":"Proceedings of the 29th International Conference on Machine Learning, ICML 2012, Edinburgh, Scotland, UK, June 26 \u2013 July 1, 2012","author":"Chen Minmin","year":"2012"},{"key":"bib7","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7\u201312, 2016, Berlin, Germany, Volume 2: Short Papers","author":"Clinchant St\u00e9phane","year":"2016"},{"key":"bib8","volume-title":"7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6\u20139, 2019","author":"Cui Wanyun","year":"2019"},{"key":"bib9","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2\u20137, 2019, Volume 1 (Long and Short Papers)","author":"Devlin Jacob","year":"2019"},{"key":"bib10","volume-title":"5th International Conference on Learning Representations, ICLR 2017","author":"Dozat Timothy","year":"2017"},{"key":"bib11","first-page":"489","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 \u2013 November 4, 2018","author":"Edunov Sergey","year":"2018"},{"key":"bib12","first-page":"59:1","volume":"17","author":"Ganin Yaroslav","year":"2016","journal-title":"Journal of Machine Learning Research"},{"key":"bib13","first-page":"513","volume-title":"Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 \u2013 July 2, 2011","author":"Glorot Xavier","year":"2011"},{"key":"bib14","volume-title":"Proceedings of the xLite Workshop on Cross-Lingual Technologies, NIPS","author":"Gouws Stephan","year":"2012"},{"key":"bib15","author":"Han Xiaochuang","year":"2019","journal-title":"CoRR"},{"key":"bib16","first-page":"4141","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3\u20137, 2019","author":"Hao Yaru","year":"2019"},{"key":"bib17","first-page":"601","volume-title":"Advances in Neural Information Processing Systems 19, Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 4\u20137, 2006","author":"Huang Jiayuan","year":"2006"},{"key":"bib18","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1613\/jair.1872","volume":"26","author":"Hal Daum\u00e9","year":"2006","journal-title":"Journal of Artificial Intelligence Research"},{"key":"bib19","volume-title":"ACL 2007, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23\u201330, 2007, Prague, Czech Republic","author":"Jiang Jing","year":"2007"},{"key":"bib20","volume-title":"8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\u201330, 2020","author":"Lan Zhenzhong","year":"2020"},{"issue":"4","key":"bib21","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","volume":"36","author":"Lee Jinhyuk","year":"2020","journal-title":"Bioinformatics"},{"key":"bib22","first-page":"5852","volume-title":"Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2\u20137, 2018","author":"Li Zheng","year":"2018"},{"key":"bib23","first-page":"2237","volume-title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19\u201325, 2017","author":"Li Zheng","year":"2017"},{"key":"bib24","author":"Liu Yinhan","year":"2019","journal-title":"CoRR"},{"key":"bib25","volume-title":"4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2\u20134, 2016, Conference Track Proceedings","author":"Louizos Christos","year":"2016"},{"key":"bib26","first-page":"142","volume-title":"The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 19\u201324 June, 2011, Portland, Oregon, USA","author":"Maas Andrew L.","year":"2011"},{"key":"bib27","first-page":"1041","volume-title":"Advances in Neural Information Processing Systems 21, Proceedings of the Twenty-Second Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 8\u201311, 2008","author":"Mansour Yishay","year":"2008"},{"key":"bib28","first-page":"28","volume-title":"Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 2\u20134, 2010, Los Angeles, California, USA","author":"McClosky David","year":"2010"},{"key":"bib29","first-page":"414","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2\u20137, 2019, Volume 1 (Long and Short Papers)","author":"Miller Timothy A.","year":"2019"},{"key":"bib30","unstructured":"Quang Nguyen. 2015. The airline review dataset."},{"key":"bib31","first-page":"635","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26\u201331, 2015, Beijing, China, Volume 1: Long Papers","author":"Nguyen Thien Huu","year":"2015"},{"key":"bib32","first-page":"751","volume-title":"Proceedings of the 19th International Conference on World Wide Web, WWW 2010, Raleigh, North Carolina, USA, April 26\u201330, 2010","author":"Pan Sinno Jialin","year":"2010"},{"key":"bib33","first-page":"2227","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1\u20136, 2018, Volume 1 (Long Papers)","author":"Peters Matthew E.","year":"2018"},{"key":"bib34","first-page":"1498","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, ACL 2013, 4\u20139 August 2013, Sofia, Bulgaria, Volume 1: Long Papers","author":"Plank Barbara","year":"2013"},{"key":"bib35","unstructured":"Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre- training.https:\/\/s3-us-west-2.amazonaws.com\/openai-assets\/researchcovers\/languageunsupervised\/language understandingpaper.pdf."},{"issue":"8","key":"bib36","volume":"1","author":"Radford Alec","year":"2019","journal-title":"OpenAI Blog"},{"key":"bib37","volume-title":"Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, HLT-NAACL 2003, Edmonton, Canada, May 27 \u2013 June 1, 2003","author":"Roark Brian","year":"2003"},{"key":"bib38","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00294"},{"key":"bib39","first-page":"1434","volume-title":"Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL 2012, July 12\u201314, 2012","author":"Rush Alexander M.","year":"2012"},{"key":"bib40","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00162"},{"key":"bib41","first-page":"1433","volume-title":"Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 3\u20136, 2007","author":"Sugiyama Masashi","year":"2007"},{"key":"bib42","doi-asserted-by":"crossref","first-page":"32578","DOI":"10.1109\/ACCESS.2019.2901929","volume":"7","author":"Manshu Tu","year":"2019","journal-title":"IEEE Access"},{"key":"bib43","first-page":"1096","volume-title":"Machine Learning, Proceedings of the Twenty-Fifth International Conference (ICML 2008), Helsinki, Finland, June 5\u20139, 2008, volume 307 of ACM International Conference Proceeding Series","author":"Vincent Pascal","year":"2008"},{"key":"bib44","author":"Wolf Thomas","year":"2019","journal-title":"CoRR"},{"key":"bib45","author":"Yonghui Wu","year":"2016","journal-title":"CoRR"},{"key":"bib46","first-page":"627","volume-title":"Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2008, Singapore, July 20\u201324, 2008","author":"Xue Gui-Rong","year":"2008"},{"key":"bib47","first-page":"538","volume-title":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, June 22\u201327, 2014, Baltimore, MD, USA, Volume 2: Short Papers","author":"Yi Yang","year":"2014"},{"key":"bib48","first-page":"5754","volume-title":"Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8-14 December 2019, Vancouver, BC, Canada","author":"Yang Zhilin","year":"2019"},{"key":"bib49","first-page":"236","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016","author":"Jianfei Yu","year":"2016"},{"key":"bib50","first-page":"1441","volume-title":"Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28\u2013August 2, 2019, Volume 1: Long Papers","author":"Zhang Zhengyan","year":"2019"},{"key":"bib51","first-page":"400","volume-title":"Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), Vancouver, Canada, August 3\u20134, 2017","author":"Ziser Yftah","year":"2017"},{"key":"bib52","first-page":"1241","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1\u20136, 2018, Volume 1 (Long Papers)","author":"Ziser Yftah","year":"2018"},{"key":"bib53","first-page":"5895","volume-title":"Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28\u2013August 2, 2019, Volume 1: Long Papers","author":"Ziser Yftah","year":"2019"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/tacl_a_00328","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,6]],"date-time":"2023-10-06T20:43:18Z","timestamp":1696624998000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/96489"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12]]},"references-count":53,"alternative-id":["10.1162\/tacl_a_00328"],"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00328","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12]]}}}