{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,25]],"date-time":"2025-11-25T06:54:48Z","timestamp":1764053688730},"reference-count":53,"publisher":"MIT Press","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Transactions of the Association for Computational Linguistics"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:p>In this paper we demonstrate that context free grammar (CFG) based methods for grammar induction benefit from modeling lexical dependencies. This contrasts to the most popular current methods for grammar induction, which focus on discovering either constituents or dependencies. Previous approaches to marry these two disparate syntactic formalisms (e.g., lexicalized PCFGs) have been plagued by sparsity, making them unsuitable for unsupervised grammar induction. However, in this work, we present novel neural models of lexicalized PCFGs that allow us to overcome sparsity problems and effectively induce both constituents and dependencies within a single model. Experiments demonstrate that this unified framework results in stronger results on both representations than achieved when modeling either formalism alone.<jats:sup>1<\/jats:sup><\/jats:p>","DOI":"10.1162\/tacl_a_00337","type":"journal-article","created":{"date-parts":[[2020,11,12]],"date-time":"2020-11-12T20:07:06Z","timestamp":1605211626000},"page":"647-661","source":"Crossref","is-referenced-by-count":4,"title":["The Return of Lexical Dependencies: Neural Lexicalized PCFGs"],"prefix":"10.1162","volume":"8","author":[{"given":"Hao","family":"Zhu","sequence":"first","affiliation":[{"name":"Language Technologies Institute, Carnegie Mellon University."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yonatan","family":"Bisk","sequence":"additional","affiliation":[{"name":"Language Technologies Institute, Carnegie Mellon University."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Graham","family":"Neubig","sequence":"additional","affiliation":[{"name":"Language Technologies Institute, Carnegie Mellon University."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"bib1","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1145\/1553374.1553380","volume-title":"Proceedings of the 26th Annual International Conference on Machine Learning","author":"Bengio Yoshua","year":"2009"},{"key":"bib2","doi-asserted-by":"publisher","DOI":"10.1162\/0891201042544929"},{"key":"bib3","first-page":"1643","volume-title":"Proceedings of the Twenty-Sixth Conference on Artificial Intelligence (AAAI-12)","author":"Bisk Yonatan","year":"2012"},{"key":"bib4","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1162\/tacl_a_00211","author":"Bisk Yonatan","year":"2013","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"bib5","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Bisk Yonatan","year":"2015"},{"key":"bib6","first-page":"865","volume-title":"Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics","author":"Bod Rens","year":"2006"},{"key":"bib7","volume-title":"Two experiments on learning probabilistic dependency grammars from corpora","author":"Carroll Glenn","year":"1992"},{"key":"bib8","author":"Carroll Glenn","year":"1998","journal-title":"arXiv preprint cmp-lg\/9805001"},{"key":"bib9","first-page":"1031","volume-title":"Proceedings of the National Conference on Artificial Intelligence","author":"Charniak Eugene","year":"1996"},{"key":"bib10","first-page":"173","volume-title":"Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics","author":"Charniak Eugene","year":"2005"},{"issue":"3","key":"bib11","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1109\/TIT.1956.1056813","volume":"2","author":"Chomsky Noam","year":"1956","journal-title":"IRE Transactions on Information Theory"},{"key":"bib12","first-page":"13","volume-title":"Proceedings of the 2001 Workshop on Computational Natural Language Learning-Volume 7","author":"Clark Alexander","year":"2001"},{"key":"bib13","doi-asserted-by":"publisher","DOI":"10.1162\/089120103322753356"},{"key":"bib14","doi-asserted-by":"crossref","unstructured":"Marie-Catherine De Marneffe and Christopher D. Manning. 2008. Stanford typed dependencies manual, Technical report, Stanford University.DOI:\u00a0https:\/\/doi.org\/10.3115\/1608858.1608859","DOI":"10.3115\/1608858.1608859"},{"key":"bib15","first-page":"249","volume-title":"Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics","author":"Glorot Xavier","year":"2010"},{"key":"bib16","first-page":"17","volume-title":"Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Golland Dave","year":"2012"},{"key":"bib17","first-page":"19","volume-title":"Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data","author":"Green Nathan David","year":"2012"},{"key":"bib18","doi-asserted-by":"crossref","first-page":"208","DOI":"10.3115\/v1\/P14-1020","volume-title":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Hall David","year":"2014"},{"key":"bib19","first-page":"5315","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Han Wenjuan","year":"2019"},{"key":"bib20","author":"He Junxian","year":"2018","journal-title":"arXiv preprint arXiv:1808.09111"},{"key":"bib21","first-page":"770","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"He Kaiming","year":"2016"},{"key":"bib22","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"bib23","first-page":"335","volume-title":"Proceedings of 40th Annual Meeting of the Association for Computational Linguistics","author":"Hockenmaier Julia","year":"2002"},{"key":"bib24","doi-asserted-by":"crossref","first-page":"763","DOI":"10.18653\/v1\/D16-1073","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Jiang Yong","year":"2016"},{"key":"bib25","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00016"},{"key":"bib26","first-page":"2442","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Jin Lifeng","year":"2019"},{"key":"bib27","first-page":"2369","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Kim Yoon","year":"2019"},{"key":"bib28","volume-title":"Proceedings of ICLR","author":"Kingma Diederik P.","year":"2014"},{"key":"bib29","first-page":"128","volume-title":"Proceedings of the 40th Annual Meeting on Association for Computational Linguistics","author":"Klein Dan","year":"2002"},{"key":"bib30","first-page":"478","volume-title":"Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics","author":"Klein Dan","year":"2004"},{"issue":"2","key":"bib31","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1109\/69.917562","volume":"13","author":"Lane Peter C.R.","year":"2001","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"issue":"1","key":"bib32","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/0885-2308(90)90022-X","volume":"4","author":"Lari Karim","year":"1990","journal-title":"Computer Speech & Language"},{"key":"bib33","first-page":"2579","volume":"9","author":"van der Maaten Laurens","year":"2008","journal-title":"Journal of Machine Learning Research"},{"key":"bib34","first-page":"281","volume-title":"Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability","volume":"1","author":"MacQueen James","year":"1967"},{"issue":"2","key":"bib35","first-page":"313","volume":"19","author":"Marcus Mitchell","year":"1993","journal-title":"Computational Linguistics"},{"key":"bib36","first-page":"281","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Mare\u010dek David","year":"2013"},{"key":"bib37","first-page":"91","volume-title":"Advances in Neural Information Processing Systems","author":"Paskin Mark A.","year":"2002"},{"key":"bib38","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3115\/v1\/D14-1162","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Pennington Jeffrey","year":"2014"},{"key":"bib39","first-page":"1077","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","author":"Ponvert Elias","year":"2011"},{"key":"bib40","doi-asserted-by":"crossref","first-page":"115","DOI":"10.3115\/1654494.1654506","volume-title":"Proceedings of the Ninth International Workshop on Parsing Technology","author":"Prescher Detlef","year":"2005"},{"issue":"4","key":"bib41","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1007\/BF00360802","volume":"4","author":"Pullum Geoffrey K.","year":"1982","journal-title":"Linguistics and Philosophy"},{"key":"bib42","volume-title":"Twenty-Third International Joint Conference on Artificial Intelligence","author":"Ren Xiaona","year":"2013"},{"key":"bib43","volume-title":"Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability","author":"Robbins Herbert","year":"1951"},{"key":"bib44","volume":"13","author":"Sag Ivan A.","year":"1987","journal-title":"CSLI Lecture Notes"},{"key":"bib45","first-page":"78","volume-title":"Proceedings of the 12th Conference on Computational Linguistics-Volume 2","author":"Schabes Yves","year":"1988"},{"key":"bib46","first-page":"384","volume-title":"Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics","author":"Seginer Yoav","year":"2007"},{"key":"bib47","unstructured":"Noah Smith. 2006. Novel estimation methods for unsupervised discovery of latent structure in natural language text. Ph.D. thesis, Johns Hopkins University."},{"key":"bib48","first-page":"1281","volume-title":"Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing","author":"Spitkovsky Valentin I.","year":"2011"},{"key":"bib49","first-page":"751","volume-title":"Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics","author":"Spitkovsky Valentin I.","year":"2010"},{"key":"bib50","volume-title":"Empirical Methods in Natural Language Processing","author":"Spitkovsky Valentin I.","year":"2013"},{"key":"bib51","volume-title":"Proceedings of the 2nd Workshop on Neural Machine Translation","author":"Ke Tran","year":"2018"},{"key":"bib52","doi-asserted-by":"crossref","first-page":"277","DOI":"10.18653\/v1\/P17-1026","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Yoshikawa Masashi","year":"2017"},{"key":"bib53","author":"Yuret Deniz","year":"1998","journal-title":"arXiv preprint cmp-lg\/9805009"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/tacl_a_00337","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,12]],"date-time":"2023-10-12T06:10:14Z","timestamp":1697091014000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/96468"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12]]},"references-count":53,"alternative-id":["10.1162\/tacl_a_00337"],"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00337","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12]]}}}