{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T17:14:00Z","timestamp":1771953240476,"version":"3.50.1"},"reference-count":60,"publisher":"MIT Press - Journals","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Transactions of the Association for Computational Linguistics"],"published-print":{"date-parts":[[2019,11]]},"abstract":"<jats:p> Neural dependency parsing has proven very effective, achieving state-of-the-art results on numerous domains and languages. Unfortunately, it requires large amounts of labeled data, which is costly and laborious to create. In this paper we propose a self-training algorithm that alleviates this annotation bottleneck by training a parser on its own output. Our Deep Contextualized Self-training (DCST) algorithm utilizes representation models trained on sequence labeling tasks that are derived from the parser\u2019s output when applied to unlabeled data, and integrates these models with the base parser through a gating mechanism. We conduct experiments across multiple languages, both in low resource in-domain and in cross-domain setups, and demonstrate that DCST substantially outperforms traditional self-training as well as recent semi-supervised training methods. <jats:sup>1<\/jats:sup> <\/jats:p>","DOI":"10.1162\/tacl_a_00294","type":"journal-article","created":{"date-parts":[[2019,12,12]],"date-time":"2019-12-12T20:06:03Z","timestamp":1576181163000},"page":"695-713","source":"Crossref","is-referenced-by-count":15,"title":["Deep Contextualized Self-training for Low Resource Dependency                     Parsing"],"prefix":"10.1162","volume":"7","author":[{"given":"Guy","family":"Rotman","sequence":"first","affiliation":[{"name":"Faculty of Industrial Engineering and Management, Technion, IIT."}]},{"given":"Roi","family":"Reichart","sequence":"additional","affiliation":[{"name":"Faculty of Industrial Engineering and Management, Technion, IIT."}]}],"member":"281","reference":[{"key":"bib1","doi-asserted-by":"publisher","DOI":"10.1162\/0891201041850876"},{"key":"bib2","volume-title":"Proceedings of ACL","author":"Angeli Gabor","year":"2015"},{"key":"bib3","doi-asserted-by":"crossref","first-page":"789","DOI":"10.18653\/v1\/P18-1073","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Artetxe Mikel","year":"2018"},{"key":"bib4","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1145\/279943.279962","volume-title":"Proceedings of the Eleventh Annual Conference on Computational Learning Theory","author":"Blum Avrim","year":"1998"},{"key":"bib5","first-page":"55","volume-title":"Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies","author":"Che Wanxiang","year":"2018"},{"key":"bib6","first-page":"113","volume-title":"Proceedings of the 22nd International Conference on Computational Linguistics-Volume 1","author":"Chen Wenliang","year":"2008"},{"key":"bib7","first-page":"816","volume-title":"Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers","author":"Chen Wenliang","year":"2014"},{"key":"bib8","doi-asserted-by":"crossref","first-page":"1914","DOI":"10.18653\/v1\/D18-1217","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Clark Kevin","year":"2018"},{"key":"bib9","volume-title":"4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, Conference Track Proceedings","author":"Clevert Djork-Arn\u00e9","year":"2016"},{"key":"bib10","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin Jacob","year":"2019"},{"key":"bib11","volume-title":"5th International Conference on Learning Representations, ICLR 2017, Toulon, France, Conference Track Proceedings","author":"Dozat Timothy","year":"2017"},{"key":"bib12","doi-asserted-by":"crossref","first-page":"1383","DOI":"10.18653\/v1\/P18-1128","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Dror Rotem","year":"2018"},{"issue":"4","key":"bib13","doi-asserted-by":"crossref","first-page":"233","DOI":"10.6028\/jres.071B.032","volume":"71","author":"Edmonds Jack","year":"1967","journal-title":"Journal of Research of the National Bureau of Standards B"},{"key":"bib14","first-page":"1486","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","author":"Goldwasser Dan","year":"2011"},{"key":"bib15","volume-title":"Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018)","author":"Grave Edouard","year":"2018"},{"key":"bib16","volume-title":"Thirty-First AAAI Conference on Artificial Intelligence","author":"Hadiwinoto Christian","year":"2017"},{"issue":"4","key":"bib17","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1016\/j.ipm.2010.11.003","volume":"47","author":"He Yulan","year":"2011","journal-title":"Information Processing & Management"},{"key":"bib18","doi-asserted-by":"crossref","first-page":"1127","DOI":"10.18653\/v1\/P17-1104","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","volume":"1","author":"Hershcovich Daniel","year":"2017"},{"key":"bib19","volume-title":"Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers","author":"Hovy Eduard","year":"2006"},{"key":"bib20","doi-asserted-by":"crossref","first-page":"110","DOI":"10.18653\/v1\/W18-2713","volume-title":"Proceedings of the 2nd Workshop on Neural Machine Translation and Generation","author":"Imamura Kenji","year":"2018"},{"key":"bib21","volume-title":"Proceedings of ICLR","author":"Kingma Diederik P.","year":"2015"},{"key":"bib22","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00017"},{"key":"bib23","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00101"},{"key":"bib24","doi-asserted-by":"crossref","first-page":"302","DOI":"10.3115\/v1\/P14-2050","volume-title":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","volume":"2","author":"Levy Omer","year":"2014"},{"key":"bib25","first-page":"1403","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Ma Xuezhe","year":"2018"},{"key":"bib26","volume-title":"Proceedings of CoNLL","author":"Marcheggiani Diego","year":"2017"},{"key":"bib27","first-page":"6294","volume-title":"Advances in Neural Information Processing Systems","author":"McCann Bryan","year":"2017"},{"key":"bib28","first-page":"152","volume-title":"Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics","author":"McClosky David","year":"2006"},{"key":"bib29","first-page":"337","volume-title":"Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics","author":"McClosky David","year":"2006"},{"key":"bib30","first-page":"28","volume-title":"Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics","author":"McClosky David","year":"2010"},{"key":"bib31","first-page":"92","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","volume":"2","author":"McDonald Ryan","year":"2013"},{"key":"bib32","volume-title":"Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004","author":"Mihalcea Rada","year":"2004"},{"key":"bib34","volume-title":"LREC","author":"Nivre Joakim","year":"2016"},{"key":"bib35","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3115\/v1\/D14-1162","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Pennington Jeffrey","year":"2014"},{"key":"bib36","first-page":"2227","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Peters Matthew","year":"2018"},{"key":"bib37","doi-asserted-by":"crossref","first-page":"614","DOI":"10.18653\/v1\/D18-1061","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Plank Barbara","year":"2018"},{"key":"bib38","first-page":"1566","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1","author":"Plank Barbara","year":"2011"},{"key":"bib39","first-page":"616","volume-title":"Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics","author":"Reichart Roi","year":"2007"},{"key":"bib40","volume-title":"The 56th Annual Meeting of the Association for Computational LinguisticsMeeting of the Association for Computational Linguistics","author":"Ruder Sebastian","year":"2018"},{"key":"bib41","first-page":"1434","volume-title":"Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning","author":"Rush Alexander M.","year":"2012"},{"key":"bib42","first-page":"45","volume-title":"Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies","author":"Rybak Piotr","year":"2018"},{"key":"bib43","doi-asserted-by":"crossref","first-page":"71","DOI":"10.18653\/v1\/K17-3007","volume-title":"Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies","author":"Sato Motoki","year":"2017"},{"key":"bib44","first-page":"3509","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Shareghi Ehsan","year":"2019"},{"key":"bib45","first-page":"205","volume-title":"Proceedings of the ACL 2010 Conference Short Papers","author":"S\u00f8gaard Anders","year":"2010"},{"key":"bib46","first-page":"7","volume":"94","author":"Spoustov\u00e1 Drahom\u00edra","year":"2010","journal-title":"Prague Bulletin of Mathematical Linguistics"},{"issue":"1","key":"bib47","first-page":"1929","volume":"15","author":"Srivastava Nitish","year":"2014","journal-title":"Journal of Machine Learning Research"},{"key":"bib48","first-page":"331","volume-title":"Proceedings of the Tenth Conference on European Chapter of the Association for Computational Linguistics-Volume 1","author":"Steedman Mark","year":"2003"},{"key":"bib49","first-page":"717","volume-title":"Proceedings of NAACL-HLT","author":"Strzyz Michalina","year":"2019"},{"key":"bib50","volume-title":"Proceedings of ICLR","author":"Tenney Ian","year":"2019"},{"key":"bib51","doi-asserted-by":"crossref","first-page":"1434","DOI":"10.18653\/v1\/P16-1136","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","volume":"1","author":"Toutanova Kristina","year":"2016"},{"key":"bib52","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017"},{"key":"bib53","first-page":"2773","volume-title":"Advances in Neural Information Processing Systems","author":"Vinyals Oriol","year":"2015"},{"key":"bib54","first-page":"4465","volume-title":"Proceedings of the 57th Conference of the Association for Computational Linguistics","author":"Wang Alex","year":"2019"},{"key":"bib55","volume-title":"Proceedings of ICLR","author":"Wieting John","year":"2019"},{"key":"bib56","first-page":"2145","volume-title":"Proceedings of the 27th International Conference on Computational Linguistics","author":"Yadav Vikas","year":"2018"},{"key":"bib57","volume-title":"33rd Annual Meeting of the Association for Computational Linguistics","author":"Yarowsky David","year":"1995"},{"key":"bib58","doi-asserted-by":"crossref","first-page":"359","DOI":"10.18653\/v1\/W18-5448","volume-title":"Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP","author":"Zhang Kelly W.","year":"2018"},{"key":"bib59","first-page":"649","volume-title":"Advances in Neural Information Processing Systems","author":"Zhang Xiang","year":"2015"},{"issue":"11","key":"bib60","doi-asserted-by":"crossref","first-page":"1529","DOI":"10.1109\/TKDE.2005.186","author":"Zhou Zhi-Hua","year":"2005","journal-title":"IEEE Transactions on Knowledge & Data Engineering"},{"key":"bib61","first-page":"1241","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Ziser Yftah","year":"2018"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/tacl_a_00294","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:39:30Z","timestamp":1615585170000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/43533"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11]]},"references-count":60,"alternative-id":["10.1162\/tacl_a_00294"],"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00294","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11]]}}}