{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:28:50Z","timestamp":1750220930652,"version":"3.41.0"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2019,7,13]],"date-time":"2019-07-13T00:00:00Z","timestamp":1562976000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61632011, 61772156, and 61772153."],"award-info":[{"award-number":["61632011, 61772156, and 61772153."]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2020,1,31]]},"abstract":"<jats:p>Deep contextualized word embeddings (Embeddings from Language Model, short for ELMo), as an emerging and effective replacement for the static word embeddings, have achieved success on a bunch of syntactic and semantic NLP problems. However, little is known about what is responsible for the improvements. In this article, we focus on the effect of ELMo for a typical syntax problem\u2014universal POS tagging and dependency parsing. We incorporate ELMo as additional word embeddings into the state-of-the-art POS tagger and dependency parser, and it leads to consistent performance improvements. Experimental results show the model using ELMo outperforms the state-of-the-art baseline by an average of 0.91 for POS tagging and 1.11 for dependency parsing. Further analysis reveals that the improvements mainly result from the ELMo\u2019s better abstraction ability on the out-of-vocabulary (OOV) words, and the character-level word representation in ELMo contributes a lot to the abstraction. Based on ELMo\u2019s advantage on OOV, experiments that simulate low-resource settings are conducted and the results show that deep contextualized word embeddings are effective for data-insufficient tasks where the OOV problem is severe.<\/jats:p>","DOI":"10.1145\/3326497","type":"journal-article","created":{"date-parts":[[2019,7,15]],"date-time":"2019-07-15T12:15:40Z","timestamp":1563192940000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Deep Contextualized Word Embeddings for Universal Dependency Parsing"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6311-9955","authenticated-orcid":false,"given":"Yijia","family":"Liu","sequence":"first","affiliation":[{"name":"Harbin Institute of Technology, Harbin, HeiLongJiang, China"}]},{"given":"Wanxiang","family":"Che","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology, Harbin, HeiLongJiang, China"}]},{"given":"Yuxuan","family":"Wang","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology, Harbin, HeiLongJiang, China"}]},{"given":"Bo","family":"Zheng","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology, Harbin, HeiLongJiang, China"}]},{"given":"Bing","family":"Qin","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology, Harbin, HeiLongJiang, China"}]},{"given":"Ting","family":"Liu","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology, Harbin, HeiLongJiang, China"}]}],"member":"320","published-online":{"date-parts":[[2019,7,13]]},"reference":[{"volume-title":"Proc. of Coling. http:\/\/www.aclweb.org\/anthology\/C18-1139","year":"2018","author":"Akbik Alan","key":"e_1_2_1_1_1"},{"volume-title":"Smith","year":"2015","author":"Ballesteros Miguel","key":"e_1_2_1_2_1"},{"volume-title":"Proc. of the Workshop on Computational Linguistics for Linguistic Complexity (CL4LC\u201916)","year":"2016","author":"Bentz Christian","key":"e_1_2_1_3_1"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"volume-title":"Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, and Berlin Chen.","year":"2018","author":"Bowman Samuel R.","key":"e_1_2_1_5_1"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/1699571.1699587"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2014.2365359"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.3115\/1118693.1118694"},{"key":"e_1_2_1_9_1","volume-title":"Proc. of ICML","volume":"70","author":"Dauphin Yann N.","year":"2017"},{"volume-title":"BERT: Pre-training of deep bidirectional transformers for language understanding. CoRR abs\/1810.04805","year":"2018","author":"Devlin Jacob","key":"e_1_2_1_10_1"},{"volume-title":"Manning","year":"2016","author":"Dozat Timothy","key":"e_1_2_1_11_1"},{"volume-title":"Manning","year":"2017","author":"Dozat Timothy","key":"e_1_2_1_12_1"},{"volume-title":"Dryer and Martin Haspelmath (Eds.)","year":"2013","author":"Matthew","key":"e_1_2_1_13_1"},{"volume-title":"Austin Matthews, and Noah A. Smith","year":"2015","author":"Dyer Chris","key":"e_1_2_1_14_1"},{"volume-title":"Deep residual learning for image recognition. CoRR abs\/1512.03385","year":"2015","author":"He Kaiming","key":"e_1_2_1_15_1"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1031"},{"volume-title":"Weinberger","year":"2016","author":"Huang Gao","key":"e_1_2_1_18_1"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1001"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1110"},{"volume-title":"75 languages, 1 model: Parsing universal dependencies universally. CoRR abs\/1904.02099","year":"2019","author":"Kondratyuk Daniel","key":"e_1_2_1_21_1"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-2108"},{"key":"e_1_2_1_23_1","unstructured":"Percy Liang. 2005. Semi-supervised Learning for Natural Language. Master's Thesis. MIT.  Percy Liang. 2005. Semi-supervised Learning for Natural Language. Master's Thesis. MIT."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390231"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1130"},{"volume-title":"Visualizing data using t-SNE. Journal of Machine Learning Research 9","year":"2008","author":"van der Maaten Laurens","key":"e_1_2_1_26_1"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/972470.972475"},{"key":"e_1_2_1_28_1","unstructured":"Bryan McCann James Bradbury Caiming Xiong and Richard Socher. 2017. Learned in translation: Contextualized word vectors. In NIPS 30. 6294--6305.   Bryan McCann James Bradbury Caiming Xiong and Richard Socher. 2017. Learned in translation: Contextualized word vectors. In NIPS 30. 6294--6305."},{"volume-title":"Proc. of EMNLP.","year":"2007","author":"McDonald Ryan","key":"e_1_2_1_29_1"},{"volume-title":"Pereira","year":"2006","author":"McDonald Ryan T.","key":"e_1_2_1_30_1"},{"volume-title":"Distributed representations of words and phrases and their compositionality. CoRR abs\/1310.4546","year":"2013","author":"Mikolov Tomas","key":"e_1_2_1_31_1"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli.07-056-R1-07-027"},{"volume-title":"Proc. of LREC-2016","year":"2016","author":"Nivre Joakim","key":"e_1_2_1_33_1"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1179"},{"volume-title":"Smith","year":"2019","author":"Peters Matthew","key":"e_1_2_1_37_1"},{"volume-title":"Proc. of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. https:\/\/www.aclweb.org\/anthology\/K18-2020","year":"2018","author":"Straka Milan","key":"e_1_2_1_38_1"},{"volume-title":"International Conference on Learning Representations. https:\/\/openreview.net\/forum?id&equals;SJzSgnRcKX.","year":"2019","author":"Tenney Ian","key":"e_1_2_1_39_1"},{"key":"e_1_2_1_40_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In NIPS 30. 5998--6008.   Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In NIPS 30. 5998--6008."},{"volume-title":"Proc. of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies.","year":"2018","author":"Zeman Daniel","key":"e_1_2_1_41_1"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-5448"},{"volume-title":"Proc. of ACL.","year":"2011","author":"Zhang Yue","key":"e_1_2_1_43_1"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3326497","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3326497","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:53:19Z","timestamp":1750204399000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3326497"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,13]]},"references-count":43,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,1,31]]}},"alternative-id":["10.1145\/3326497"],"URL":"https:\/\/doi.org\/10.1145\/3326497","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"type":"print","value":"2375-4699"},{"type":"electronic","value":"2375-4702"}],"subject":[],"published":{"date-parts":[[2019,7,13]]},"assertion":[{"value":"2019-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-07-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}