{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T06:54:33Z","timestamp":1782284073388,"version":"3.54.5"},"publisher-location":"New York, NY, USA","reference-count":20,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,2,4]],"date-time":"2020-02-04T00:00:00Z","timestamp":1580774400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,2,4]]},"DOI":"10.1145\/3373017.3373028","type":"proceedings-article","created":{"date-parts":[[2020,1,30]],"date-time":"2020-01-30T12:25:32Z","timestamp":1580387132000},"page":"1-4","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":51,"title":["A Short Survey of Pre-trained Language Models for Conversational AI-A New Age in NLP"],"prefix":"10.1145","author":[{"given":"Munazza","family":"Zaib","sequence":"first","affiliation":[{"name":"Macquarie University, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Quan Z.","family":"Sheng","sequence":"additional","affiliation":[{"name":"Macquarie University, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Emma Zhang","sequence":"additional","affiliation":[{"name":"The University of Adelaide, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,2,4]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"Pawel Budzianowski and Ivan Vulic. 2019. Hello It\u2019s GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems. CoRR abs\/1907.05774(2019). arxiv:1907.05774  Pawel Budzianowski and Ivan Vulic. 2019. Hello It\u2019s GPT-2 - How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems. CoRR abs\/1907.05774(2019). arxiv:1907.05774"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1547"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2019-1355"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1241"},{"key":"e_1_3_2_1_5_1","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019","volume":"1","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019 , Minneapolis, MN, USA , June 2-7, 2019, Volume 1 (Long and Short Papers). 4171\u20134186. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers). 4171\u20134186."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210183"},{"key":"e_1_3_2_1_7_1","volume-title":"Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008","author":"Larochelle Hugo","year":"2008","unstructured":"Hugo Larochelle , Dumitru Erhan , and Yoshua Bengio . 2008 . Zero-data Learning of New Tasks . In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008 , Chicago, Illinois, USA , July 13-17, 2008. 646\u2013651. Hugo Larochelle, Dumitru Erhan, and Yoshua Bengio. 2008. Zero-data Learning of New Tasks. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, July 13-17, 2008. 646\u2013651."},{"key":"e_1_3_2_1_8_1","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR abs\/1907.11692(2019). arxiv:1907.11692  Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR abs\/1907.11692(2019). arxiv:1907.11692"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-4102"},{"key":"e_1_3_2_1_10_1","volume-title":"Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems","author":"Palatucci Mark","year":"2009","unstructured":"Mark Palatucci , Dean Pomerleau , Geoffrey\u00a0 E. Hinton , and Tom\u00a0 M. Mitchell . 2009. Zero-shot Learning with Semantic Output Codes . In Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009 . Proceedings of a meeting held 7-10 December 2009, Vancouver, British Columbia, Canada .1410\u20131418. Mark Palatucci, Dean Pomerleau, Geoffrey\u00a0E. Hinton, and Tom\u00a0M. Mitchell. 2009. Zero-shot Learning with Semantic Output Codes. In Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7-10 December 2009, Vancouver, British Columbia, Canada.1410\u20131418."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331341"},{"key":"e_1_3_2_1_13_1","volume-title":"Improving language understanding by generative pre-training. URL https:\/\/s3-us-west-2. amazonaws. com\/openai-assets\/researchcovers\/languageunsupervised\/language understanding paper. pdf","author":"Radford Alec","year":"2018","unstructured":"Alec Radford , Karthik Narasimhan , Tim Salimans , and Ilya Sutskever . 2018. Improving language understanding by generative pre-training. URL https:\/\/s3-us-west-2. amazonaws. com\/openai-assets\/researchcovers\/languageunsupervised\/language understanding paper. pdf ( 2018 ). Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL https:\/\/s3-us-west-2. amazonaws. com\/openai-assets\/researchcovers\/languageunsupervised\/language understanding paper. pdf (2018)."},{"key":"e_1_3_2_1_14_1","volume-title":"Language models are unsupervised multitask learners. OpenAI Blog 1, 8","author":"Radford Alec","year":"2019","unstructured":"Alec Radford , Jeffrey Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 ( 2019 ). Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019)."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1264"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00266"},{"key":"e_1_3_2_1_17_1","volume-title":"Bidirectional Attention Flow for Machine Comprehension. abs\/1611.01603","author":"Seo Minjoon","year":"2016","unstructured":"Minjoon Seo , Aniruddha Kembhavi , Ali Farhadi , and Hannaneh Hajishirzi . 2016. Bidirectional Attention Flow for Machine Comprehension. abs\/1611.01603 ( 2016 ). Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Bidirectional Attention Flow for Machine Comprehension. abs\/1611.01603 (2016)."},{"key":"e_1_3_2_1_18_1","unstructured":"Zhilin Yang Zihang Dai Yiming Yang Jaime\u00a0G. Carbonell Ruslan Salakhutdinov and Quoc\u00a0V. Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. CoRR abs\/1906.08237(2019). arxiv:1906.08237  Zhilin Yang Zihang Dai Yiming Yang Jaime\u00a0G. Carbonell Ruslan Salakhutdinov and Quoc\u00a0V. Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. CoRR abs\/1906.08237(2019). arxiv:1906.08237"},{"key":"e_1_3_2_1_19_1","unstructured":"Yi\u00a0Ting Yeh and Yun-Nung Chen. 2019. FlowDelta: Modeling Flow Information Gain in Reasoning for Conversational Machine Comprehension. CoRR abs\/1908.05117(2019). arxiv:1908.05117  Yi\u00a0Ting Yeh and Yun-Nung Chen. 2019. FlowDelta: Modeling Flow Information Gain in Reasoning for Conversational Machine Comprehension. CoRR abs\/1908.05117(2019). arxiv:1908.05117"},{"key":"e_1_3_2_1_20_1","unstructured":"Zhuosheng Zhang Yuwei Wu Hai Zhao Zuchao Li Shuailiang Zhang Xi Zhou and Xiang Zhou. 2019. Semantics-aware BERT for Language Understanding. CoRR abs\/1909.02209(2019). arxiv:1909.02209  Zhuosheng Zhang Yuwei Wu Hai Zhao Zuchao Li Shuailiang Zhang Xi Zhou and Xiang Zhou. 2019. Semantics-aware BERT for Language Understanding. CoRR abs\/1909.02209(2019). arxiv:1909.02209"}],"event":{"name":"ACSW '20: Australasian Computer Science Week 2020","location":"Melbourne VIC Australia","acronym":"ACSW '20"},"container-title":["Proceedings of the Australasian Computer Science Week Multiconference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3373017.3373028","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3373017.3373028","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:41:25Z","timestamp":1750200085000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3373017.3373028"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,4]]},"references-count":20,"alternative-id":["10.1145\/3373017.3373028","10.1145\/3373017"],"URL":"https:\/\/doi.org\/10.1145\/3373017.3373028","relation":{},"subject":[],"published":{"date-parts":[[2020,2,4]]},"assertion":[{"value":"2020-02-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}