{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2023,10,13]],"date-time":"2023-10-13T07:52:20Z","timestamp":1697183540717},"reference-count":62,"publisher":"MIT Press - Journals","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Transactions of the Association for Computational Linguistics"],"published-print":{"date-parts":[[2019,11]]},"abstract":"<jats:p> Treebanks traditionally treat punctuation marks as ordinary words, but linguists have suggested that a tree\u2019s \u201ctrue\u201d punctuation marks are not observed (Nunberg, 1990). These latent \u201cunderlying\u201d marks serve to delimit or separate constituents in the syntax tree. When the tree\u2019s yield is rendered as a written sentence, a string rewriting mechanism transduces the underlying marks into \u201csurface\u201d marks, which are part of the observed (surface) string but should not be regarded as part of the tree. We formalize this idea in a generative model of punctuation that admits efficient dynamic programming. We train it without observing the underlying marks, by locally maximizing the incomplete data likelihood (similarly to the EM algorithm). When we use the trained model to reconstruct the tree\u2019s underlying punctuation, the results appear plausible across 5 languages, and in particular are consistent with Nunberg\u2019s analysis of English. We show that our generative model can be used to beat baselines on punctuation restoration. Also, our reconstruction of a sentence\u2019s underlying punctuation lets us appropriately render the surface punctuation (via our trained underlying-to-surface mechanism) when we syntactically transform the sentence. <\/jats:p>","DOI":"10.1162\/tacl_a_00273","type":"journal-article","created":{"date-parts":[[2019,7,12]],"date-time":"2019-07-12T18:24:05Z","timestamp":1562955845000},"page":"357-373","source":"Crossref","is-referenced-by-count":3,"title":["A Generative Model for Punctuation in Dependency Trees"],"prefix":"10.1162","volume":"7","author":[{"given":"Xiang Lisa","family":"Li","sequence":"first","affiliation":[{"name":"Department of Computer Science, Johns Hopkins University."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dingquan","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Johns Hopkins University."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jason","family":"Eisner","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Johns Hopkins University."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"bib1","first-page":"30","volume-title":"Proceedings of the Workshop on Language in Social Media (LSM 2011)","author":"Agarwal Apoorv","year":"2011"},{"key":"bib2","doi-asserted-by":"crossref","first-page":"1048","DOI":"10.18653\/v1\/D16-1111","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Ballesteros Miguel","year":"2016"},{"key":"bib3","first-page":"143","volume":"14","author":"Bar-Hillel Yehoshua","year":"1961","journal-title":"Zeitschrift f\u00fcr Phonetik, Sprachwissenschaft und Kommunikationsforschung"},{"key":"bib4","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-73235-5","volume-title":"Rational Series and their Languages","author":"Berstel Jean","year":"1988"},{"key":"bib5","unstructured":"Ann Bies, Mark Ferguson, Karen Katz, Robert MacIntyre, Victoria Tredinnick, Grace Kim, Mary Ann Marcinkiewicz, and Britta Schasberger. 1995. Bracketing guidelines for Treebank II style: Penn Treebank project. Technical Report MS-CIS-95-06, University of Pennsylvania."},{"key":"bib6","unstructured":"Ted Briscoe. 1994. Parsing (with) punctuation, etc., Technical report, Xerox European Research Laboratory."},{"key":"bib7","doi-asserted-by":"crossref","first-page":"740","DOI":"10.3115\/v1\/D14-1082","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Chen Danqi","year":"2014"},{"key":"bib8","volume-title":"The Sound Pattern of English","author":"Chomsky Noam","year":"1968"},{"key":"bib9","doi-asserted-by":"crossref","first-page":"625","DOI":"10.3115\/v1\/P14-2102","volume-title":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Cotterell Ryan","year":"2014"},{"key":"bib10","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00149"},{"key":"bib11","first-page":"664","volume-title":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)","author":"Cotterell Ryan","year":"2016"},{"key":"bib12","doi-asserted-by":"crossref","first-page":"232","DOI":"10.3115\/1610075.1610110","volume-title":"Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Cowan Brooke","year":"2006"},{"key":"bib13","volume-title":"Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Culotta Aron","year":"2004"},{"key":"bib14","volume-title":"Proceedings of the 5th International Conference on Learning Representations (ICLR)","author":"Dozat Timothy","year":"2017"},{"key":"bib15","first-page":"199","volume-title":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)","author":"Dyer Chris","year":"2016"},{"key":"bib16","doi-asserted-by":"crossref","first-page":"340","DOI":"10.3115\/992628.992688","volume-title":"Proceedings of the 16th International Conference on Computational Linguistics (COLING)","author":"Eisner Jason","year":"1996"},{"key":"bib17","volume-title":"Proceedings of the EMNLP Workshop on Structured Prediction for NLP","author":"Eisner Jason","year":"2016"},{"key":"bib18","first-page":"191","volume-title":"Symbol Manipulation Languages and Techniques","author":"Caracciolo di Forino A.","year":"1968"},{"key":"bib19","first-page":"2001","volume":"11","author":"Ganchev Kuzman","year":"2010","journal-title":"Journal of Machine Learning Research"},{"key":"bib21","first-page":"742","volume-title":"Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT)","author":"Goldberg Yoav","year":"2010"},{"issue":"4","key":"bib22","first-page":"573","volume":"25","author":"Goodman Joshua","year":"1999","journal-title":"Computational Linguistics"},{"key":"bib23","first-page":"181","volume-title":"Handbook of Natural Language Processing","author":"Heidorn George","year":"2000"},{"key":"bib24","doi-asserted-by":"crossref","DOI":"10.1515\/9783110876000","volume-title":"Formal Aspects of Phonological Description","author":"Douglas Johnson C.","year":"1972"},{"key":"bib25","volume-title":"COLING 1994 Volume 1: The 15th International Conference on Computational Linguistics","author":"Jones Bernard E. M.","year":"1994"},{"key":"bib26","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Kingma Diederik","year":"2014"},{"key":"bib27","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00101"},{"issue":"1","key":"bib28","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/S0004-3702(02)00222-9","volume":"139","author":"Knight Kevin","year":"2002","journal-title":"Artificial Intelligence"},{"key":"bib29","first-page":"599","volume-title":"Proceedings of the 23rd International Conference on Computational Linguistics (COLING)","author":"Kong Fang","year":"2010"},{"key":"bib30","first-page":"1","volume-title":"Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Koo Terry","year":"2010"},{"key":"bib31","unstructured":"Albert E. Krahn. 2014. A New Paradigm for Punctuation. Ph.D. thesis, The University of Wisconsin-Milwaukee."},{"key":"bib32","first-page":"1381","volume-title":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Lei Tao","year":"2014"},{"key":"bib33","first-page":"234","volume-title":"Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Levy Roger","year":"2008"},{"key":"bib34","volume-title":"Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP)","author":"Li Xing","year":"2005"},{"key":"bib35","first-page":"40","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Li Zhifei","year":"2009"},{"key":"bib36","first-page":"177","volume-title":"Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Wei Lu","year":"2010"},{"key":"bib37","first-page":"137","volume-title":"Proceedings of the Australasian Language Technology Association Workshop (ALTA)","author":"Lui Marco","year":"2013"},{"key":"bib38","first-page":"791","volume-title":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Ji Ma","year":"2014"},{"key":"bib39","first-page":"955","volume":"11","author":"Mann Gideon S.","year":"2010","journal-title":"Journal of Machine Learning Research"},{"issue":"15","key":"bib40","series-title":"series 2","first-page":"1","volume-title":"American Mathematical Society Translations","author":"Markov Andrey Andreevich","year":"1960"},{"key":"bib41","first-page":"3456","volume-title":"Proceedings of the 27th International Conference on Computational Linguistics (COLING)","author":"Markov Ilia","year":"2018"},{"key":"bib42","author":"Mikolov Tomas","year":"2013","journal-title":"Computing Research Repository (CoRR)"},{"key":"bib43","first-page":"137","volume-title":"8th International Workshop on Parsing Technologies (IWPT)","author":"Nederhof Mark-Jan","year":"2003"},{"key":"bib44","first-page":"1","volume-title":"Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task","author":"Ng Hwee Tou","year":"2014"},{"key":"bib46","first-page":"915","volume-title":"Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007","author":"Nivre Joakim","year":"2007"},{"issue":"2","key":"bib47","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1017\/S1351324906004505","volume":"13","author":"Nivre Joakim","year":"2007","journal-title":"Natural Language Engineering"},{"key":"bib49","volume-title":"The Linguistics of Punctuation","author":"Nunberg Geoffrey","year":"1990"},{"key":"bib50","volume-title":"Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP 2002)","author":"Bo Pang","year":"2002"},{"key":"bib51","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3115\/v1\/D14-1162","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Pennington Jeffrey","year":"2014"},{"key":"bib52","author":"Pereira Fernando C. N.","year":"1996","journal-title":"Computing Research Repository (CoRR)"},{"issue":"2","key":"bib53","volume":"1503","author":"Rasooli Mohammad Sadegh","year":"2015","journal-title":"Computing Research Repository"},{"key":"bib54","first-page":"45","volume-title":"Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks","author":"\u0158eh\u016f\u0159ek Radim","year":"2010"},{"issue":"11","key":"bib55","first-page":"19","volume-title":"Proceedings of the Fifteenth Conference on Computational Natural Language Learning","author":"Spitkovsky Valentin I.","year":"2011"},{"key":"bib56","first-page":"1556","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-COLING)","author":"Tai Kai Sheng","year":"2015"},{"key":"bib57","doi-asserted-by":"crossref","first-page":"3047","DOI":"10.21437\/Interspeech.2016-1517","volume-title":"Interspeech","author":"Tilk Ottokar","year":"2016"},{"key":"bib58","first-page":"63","volume-title":"Proceedings of the Workshop on Structured Prediction for NLP","author":"Tran Ke M.","year":"2010"},{"key":"bib65","volume-title":"The Chicago Manual of Style","author":"University of Chicago","year":"2010"},{"key":"bib59","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00113"},{"key":"bib60","first-page":"17","volume-title":"Proceedings of the COLING 2008 Workshop on Grammar Engineering Across Frameworks","author":"White Michael","year":"2008"},{"key":"bib61","first-page":"1","volume-title":"2016 10th International Symposium on Chinese Spoken Language Processing (ISCSLP)","author":"Xu K.","year":"2016"},{"key":"bib62","first-page":"18","volume-title":"Annual Conference on Artificial Intelligence","author":"Zens Richard","year":"2002"},{"key":"bib63","first-page":"752","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Zhang Dongdong","year":"2013"},{"key":"bib64","first-page":"188","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)","author":"Zhang Yue","year":"2011"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/tacl_a_00273","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:39:24Z","timestamp":1615585164000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/43507"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11]]},"references-count":62,"alternative-id":["10.1162\/tacl_a_00273"],"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00273","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11]]}}}