{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2022,12,5]],"date-time":"2022-12-05T16:27:03Z","timestamp":1670257623737},"reference-count":37,"publisher":"MIT Press - Journals","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational Linguistics"],"published-print":{"date-parts":[[2021,2]]},"abstract":"<jats:p> There is a growing interest in investigating what neural NLP models learn about language. A prominent open question is the question of whether or not it is necessary to model hierarchical structure. We present a linguistic investigation of a neural parser adding insights to this question. We look at transitivity and agreement information of auxiliary verb constructions (AVCs) in comparison to finite main verbs (FMVs). This comparison is motivated by theoretical work in dependency grammar and in particular the work of Tesni\u00e8re ( 1959 ), where AVCs and FMVs are both instances of a nucleus, the basic unit of syntax. An AVC is a dissociated nucleus; it consists of at least two words, and an FMV is its non-dissociated counterpart, consisting of exactly one word. We suggest that the representation of AVCs and FMVs should capture similar information. We use diagnostic classifiers to probe agreement and transitivity information in vectors learned by a transition-based neural parser in four typologically different languages. We find that the parser learns different information about AVCs and FMVs if only sequential models (BiLSTMs) are used in the architecture but similar information when a recursive layer is used. We find explanations for why this is the case by looking closely at how information is learned in the network and looking at what happens with different dependency representations of AVCs. We conclude that there may be benefits to using a recursive layer in dependency parsing and that we have not yet found the best way to integrate it in our parsers. <\/jats:p>","DOI":"10.1162\/coli_a_00392","type":"journal-article","created":{"date-parts":[[2020,10,22]],"date-time":"2020-10-22T19:51:46Z","timestamp":1603396306000},"page":"763-784","source":"Crossref","is-referenced-by-count":1,"title":["What Should\/Do\/Can LSTMs Learn When Parsing Auxiliary Verb                     Constructions?"],"prefix":"10.1162","volume":"46","author":[{"given":"Miryam","family":"de Lhoneux","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Copenhagen."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sara","family":"Stymne","sequence":"additional","affiliation":[{"name":"Department of Linguistics and Philology, Uppsala University."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joakim","family":"Nivre","sequence":"additional","affiliation":[{"name":"Department of Linguistics and Philology, Uppsala University."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"bib1","volume-title":"Changing valency: Case studies in transitivity","author":"Aikhenvald Alexandra Yurievna","year":"2000"},{"key":"bib2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1749-818X.2011.00311.x"},{"key":"bib3","unstructured":"Belinkov,\n                                Yonatan.                         2018. On Internal Language Representations in Deep                         Learning: An Analysis of Machine Translation and Speech                     Recognition. Ph.D. thesis,                         Massachusetts Institute of                     Technology."},{"key":"bib4","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-2003"},{"key":"bib5","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin Jacob","year":"2019"},{"key":"bib6","volume-title":"Proceedings of the 5th International Conference on Learning Representations","author":"Dozat Timothy","year":"2017"},{"key":"bib7","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1033"},{"key":"bib8","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K17-1003"},{"key":"bib9","unstructured":"Graves,\n                                Alex.                         2008. Supervised Sequence Labelling with Recurrent                         Neural Networks. Ph.D. thesis,                         Technical University                     Munich."},{"key":"bib10","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1108"},{"key":"bib11","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"bib12","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.11196"},{"issue":"7","key":"bib13","volume":"6","author":"Johnson Mark","year":"2011","journal-title":"Linguistic Issues in Language Technology"},{"key":"bib14","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00101"},{"key":"bib15","first-page":"673","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Kuhlmann Marco","year":"2011"},{"key":"bib16","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1132"},{"key":"bib17","first-page":"1566","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"de Lhoneux Miryam","year":"2019"},{"key":"bib18","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-1202"},{"key":"bib19","first-page":"99","volume-title":"Proceedings of the 15th International Conference on Parsing Technologies","author":"de Lhoneux Miryam","year":"2017"},{"key":"bib20","first-page":"99","volume-title":"Proceedings of the 15th Treebanks and Linguistic Theories Workshop (TLT)","author":"de Lhoneux Miryam","year":"2017"},{"key":"bib21","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00115"},{"key":"bib22","volume-title":"Dependency Syntax: Theory and Practice","author":"Mel\u2019\u010duk Igor","year":"1988"},{"key":"bib23","first-page":"3111","volume-title":"Advances in Neural Information Processing Systems 26","author":"Mikolov Tomas","year":"2013"},{"key":"bib24","doi-asserted-by":"publisher","DOI":"10.1162\/coli.07-056-R1-07-027"},{"key":"bib25","doi-asserted-by":"publisher","DOI":"10.3115\/1687878.1687929"},{"key":"bib26","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-18111-0_1"},{"key":"bib27","first-page":"2216","volume-title":"Proceedings of the 5th International Confer-ence on Language Resources and Evaluation (LREC)","author":"Nivre Joakim","year":"2006"},{"key":"bib28","first-page":"1659","volume-title":"Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016)","author":"Nivre Joakim","year":"2016"},{"key":"bib29","unstructured":"Nivre,\n                                Joakim,                                 Mitchell Abrams,                                 \u017eeljko Agi\u0107,  2018. Universal Dependencies 2.2.                         LINDAT\/CLARIN digital library at the Institute of Formal and                         Applied Linguistics, Charles University,                     Prague."},{"key":"bib30","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202"},{"key":"bib31","first-page":"3532","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Ravfogel Shauli","year":"2019"},{"key":"bib32","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-5412"},{"key":"bib33","first-page":"45","volume-title":"Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks","author":"\u0158eh\u016f\u0159ek Radim","year":"2010"},{"key":"bib34","volume-title":"The Meaning of the Sentence in Its Pragmatic Aspects","author":"Sgall Petr","year":"1986"},{"key":"bib35","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K18-2011"},{"key":"bib36","volume-title":"\u00c9l\u00e9ments de syntaxe structurale","author":"Tesni\u00e8re Lucien","year":"1959"},{"key":"bib37","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00019"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/coli_a_00392","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:28:38Z","timestamp":1615584518000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/46\/4\/763-784\/97325"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2]]},"references-count":37,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,2]]}},"alternative-id":["10.1162\/coli_a_00392"],"URL":"https:\/\/doi.org\/10.1162\/coli_a_00392","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2]]}}}