{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T04:49:53Z","timestamp":1784609393026,"version":"3.55.0"},"reference-count":46,"publisher":"MIT Press - Journals","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Transactions of the Association for Computational Linguistics"],"published-print":{"date-parts":[[2021,2]]},"abstract":"<jats:p> Various machine learning tasks can benefit from access to external information of different modalities, such as text and images. Recent work has focused on learning architectures with large memories capable of storing this knowledge. We propose augmenting generative Transformer neural networks with KNN-based Information Fetching (KIF) modules. Each KIF module learns a read operation to access fixed external knowledge. We apply these modules to generative dialog modeling, a challenging task where information must be flexibly retrieved and incorporated to maintain the topic and flow of conversation. We demonstrate the effectiveness of our approach by identifying relevant knowledge required for knowledgeable but engaging dialog from Wikipedia, images, and human-written dialog utterances, and show that leveraging this retrieved information improves model performance, measured by automatic and human evaluation. <\/jats:p>","DOI":"10.1162\/tacl_a_00356","type":"journal-article","created":{"date-parts":[[2021,3,3]],"date-time":"2021-03-03T19:45:18Z","timestamp":1614800718000},"page":"82-99","source":"Crossref","is-referenced-by-count":16,"title":["Augmenting Transformers with KNN-Based Composite Memory for Dialog"],"prefix":"10.1162","volume":"9","author":[{"given":"Angela","family":"Fan","sequence":"first","affiliation":[{"name":"Facebook AI Research, Universit\u00e9 de Lorraine LORIA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Claire","family":"Gardent","sequence":"additional","affiliation":[{"name":"CNRS\/LORIA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chlo\u00e9","family":"Braud","sequence":"additional","affiliation":[{"name":"CNRS\/IRIT."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antoine","family":"Bordes","sequence":"additional","affiliation":[{"name":"Facebook AI Research."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","reference":[{"key":"bib1","volume-title":"5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24\u201326, 2017, Conference Track Proceedings","author":"Bordes Antoine","year":"2017"},{"key":"bib2","first-page":"1866","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Cai Deng","year":"2019"},{"key":"bib3","volume":"1605","author":"Chandar Sarath","year":"2016","journal-title":"CoRR"},{"key":"bib4","doi-asserted-by":"crossref","first-page":"1870","DOI":"10.18653\/v1\/P17-1171","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Chen Danqi","year":"2017"},{"key":"bib5","doi-asserted-by":"crossref","first-page":"1975","DOI":"10.18653\/v1\/P16-1186","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Chen Wenlin","year":"2016"},{"key":"bib6","volume-title":"International Conference on Learning Representations","author":"Dinan Emily","year":"2018"},{"key":"bib7","first-page":"4177","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Fan Angela","year":"2019"},{"key":"bib8","first-page":"45","volume-title":"Proceedings of the 2nd Workshop on Neural Machine Translation and Generation","author":"Fan Angela","year":"2018"},{"key":"bib9","first-page":"889","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Fan Angela","year":"2018"},{"key":"bib10","first-page":"1302","volume-title":"Proceedings of the 34th International Conference on Machine Learning-Volume 70","author":"Grave Edouard","year":"2017"},{"key":"bib11","volume-title":"5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24\u201326, 2017, Conference Track Proceedings","author":"Grave Edouard","year":"2017"},{"key":"bib12","author":"Graves Alex","year":"2014","journal-title":"arXiv preprint arXiv:1410.5401"},{"key":"bib13","first-page":"5695","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Guu Kelvin","year":"2020"},{"key":"bib14","volume-title":"International Conference on Learning Representations","author":"Humeau Samuel","year":"2019"},{"key":"bib15","author":"Johnson Jeff","year":"2019","journal-title":"IEEE Transactions on Big Data"},{"key":"bib16","first-page":"190","volume-title":"Advances in Neural Information Processing Systems","author":"Joulin Armand","year":"2015"},{"key":"bib17","volume-title":"International Conference on Learning Representations","author":"Khandelwal Urvashi","year":"2019"},{"key":"bib18","volume-title":"3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7\u20139, 2015, Conference Track Proceedings","author":"Kingma Diederik P."},{"key":"bib19","first-page":"8548","volume-title":"Advances in Neural Information Processing Systems","author":"Lample Guillaume","year":"2019"},{"key":"bib20","author":"Li Margaret","year":"2019","journal-title":"arXiv preprint arXiv:1909.03087"},{"key":"bib21","first-page":"5081","volume-title":"Proceedings of the 28th International Joint Conference on Artificial Intelligence","author":"Lian Rongzhong","year":"2019"},{"key":"bib22","first-page":"181","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV)","author":"Mahajan Dhruv","year":"2018"},{"key":"bib23","first-page":"6294","volume-title":"Advances in Neural Information Processing Systems","author":"McCann Bryan","year":"2017"},{"key":"bib24","doi-asserted-by":"crossref","unstructured":"Alexander Miller, Will Feng, Dhruv Batra, Antoine Bordes, Adam Fisch, Jiasen Lu, Devi Parikh, and Jason Weston. 2017. parl.ai: A dialog research software platform. pages 79\u201384. https:\/\/arxiv.org\/abs\/1705.06476, DOI:\u00a0https:\/\/doi.org\/10.18653\/v1\/D17-2014","DOI":"10.18653\/v1\/D17-2014"},{"key":"bib25","first-page":"1081","volume-title":"Advances in Neural Information Processing Systems","author":"Mnih Andriy","year":"2009"},{"key":"bib26","doi-asserted-by":"crossref","first-page":"2463","DOI":"10.18653\/v1\/D19-1250","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Petroni Fabio","year":"2019"},{"key":"bib27","first-page":"1087","volume-title":"Advances in Neural Information Processing Systems","author":"Pl\u00f6tz Tobias","year":"2018"},{"key":"bib28","first-page":"5427","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Qin Lianhui","year":"2019"},{"key":"bib29","first-page":"3621","volume-title":"Advances in Neural Information Processing Systems","author":"Rae Jack","year":"2016"},{"key":"bib30","doi-asserted-by":"crossref","first-page":"1715","DOI":"10.18653\/v1\/P16-1162","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Sennrich Rico","year":"2016"},{"key":"bib31","first-page":"4430","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Seo Minjoon","year":"2019"},{"key":"bib32","author":"Serban Iulian V.","year":"2016","journal-title":"arXiv preprint arXiv:1611.06216"},{"key":"bib33","volume-title":"Thirtieth AAAI Conference on Artificial Intelligence","author":"Serban Iulian V.","year":"2016"},{"key":"bib34","doi-asserted-by":"crossref","first-page":"2414","DOI":"10.18653\/v1\/2020.acl-main.219","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Shuster Kurt","year":"2020"},{"key":"bib35","first-page":"12516","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Shuster Kurt","year":"2019"},{"key":"bib36","author":"Song Haoyu","year":"2020","journal-title":"arXiv preprint arXiv:2004.07672"},{"key":"bib37","author":"Song Yiping","year":"2016","journal-title":"arXiv preprint arXiv:1610.07149"},{"key":"bib38","unstructured":"Sainbayar Sukhbaatar, Edouard Grave, Guillaume Lample, Herve Jegou, and Armand Joulin. 2019. Augmenting self-attention with persistent memory. https:\/\/arxiv.org\/abs\/1907.01470"},{"key":"bib39","first-page":"2440","volume-title":"Advances in Neural Information Processing Systems","author":"Sukhbaatar Sainbayar","year":"2015"},{"issue":"2","key":"bib40","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1145\/2812802","volume":"59","author":"Thomee Bart","year":"2016","journal-title":"Communications of the ACM"},{"key":"bib41","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017"},{"key":"bib42","volume-title":"3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7\u20139, 2015, Conference Track Proceedings","author":"Weston Jason","year":"2015"},{"key":"bib43","doi-asserted-by":"crossref","first-page":"87","DOI":"10.18653\/v1\/W18-5713","volume-title":"Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI","author":"Weston Jason","year":"2018"},{"key":"bib44","first-page":"1492","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"Xie Saining","year":"2017"},{"key":"bib45","doi-asserted-by":"crossref","first-page":"2204","DOI":"10.18653\/v1\/P18-1205","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Zhang Saizheng","year":"2018"},{"issue":"1","key":"bib46","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1007\/s10791-019-09364-x","volume":"23","author":"Zhu Yutao","year":"2020","journal-title":"Information Retrieval Journal"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/tacl_a_00356","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:39:51Z","timestamp":1615585191000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/97779"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2]]},"references-count":46,"alternative-id":["10.1162\/tacl_a_00356"],"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00356","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2]]}}}