{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T23:10:54Z","timestamp":1776121854917,"version":"3.50.1"},"reference-count":37,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2023,4,26]],"date-time":"2023-04-26T00:00:00Z","timestamp":1682467200000},"content-version":"vor","delay-in-days":115,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,4,20]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Most work on modeling the conversation history in Conversational Question Answering (CQA) reports a single main result on a common CQA benchmark. While existing models show impressive results on CQA leaderboards, it remains unclear whether they are robust to shifts in setting (sometimes to more realistic ones), training data size (e.g., from large to small sets) and domain. In this work, we design and conduct the first large-scale robustness study of history modeling approaches for CQA. We find that high benchmark scores do not necessarily translate to strong robustness, and that various methods can perform extremely differently under different settings. Equipped with the insights from our study, we design a novel prompt-based history modeling approach and demonstrate its strong robustness across various settings. Our approach is inspired by existing methods that highlight historic answers in the passage. However, instead of highlighting by modifying the passage token embeddings, we add textual prompts directly in the passage text. Our approach is simple, easy to plug into practically any model, and highly effective, thus we recommend it as a starting point for future model developers. We also hope that our study and insights will raise awareness to the importance of robustness-focused evaluation, in addition to obtaining high leaderboard scores, leading to better CQA systems.1<\/jats:p>","DOI":"10.1162\/tacl_a_00549","type":"journal-article","created":{"date-parts":[[2023,4,26]],"date-time":"2023-04-26T20:05:57Z","timestamp":1682539557000},"page":"351-366","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":7,"title":["On the Robustness of Dialogue History Representation in\n                    Conversational Question Answering: A Comprehensive Study and a New Prompt-based\n                    Method"],"prefix":"10.1162","volume":"11","author":[{"given":"Zorik","family":"Gekhman","sequence":"first","affiliation":[{"name":"Technion - Israel Institute of Technology, Israel. zorik@campus.technion.ac.il"}]},{"given":"Nadav","family":"Oved","sequence":"additional","affiliation":[{"name":"Technion - Israel Institute of Technology, Israel. nadavo@campus.technion.ac.il"}]},{"given":"Orgad","family":"Keller","sequence":"additional","affiliation":[{"name":"Google Research, Israel. orgad@technion.ac.il"}]},{"given":"Idan","family":"Szpektor","sequence":"additional","affiliation":[{"name":"Google Research, Israel. szpektor@google.com"}]},{"given":"Roi","family":"Reichart","sequence":"additional","affiliation":[{"name":"Technion - Israel Institute of Technology, Israel. roiri@technion.ac.il"}]}],"member":"281","published-online":{"date-parts":[[2023,4,20]]},"reference":[{"key":"2023071720231306100_bib1","doi-asserted-by":"publisher","first-page":"468","DOI":"10.1162\/tacl_a_00471","article-title":"Topiocqa: Open-domain conversational question\n                        answering with topic switching","volume":"10","author":"Adlakha","year":"2022","journal-title":"Transactions of the\n                        Association for Computational Linguistics"},{"key":"2023071720231306100_bib2","article-title":"HTLM: Hyper-text pre-training and\n                        prompting of language models","volume-title":"International\n                        Conference on Learning Representations","author":"Aghajanyan","year":"2022"},{"key":"2023071720231306100_bib3","doi-asserted-by":"publisher","first-page":"520","DOI":"10.18653\/v1\/2021.naacl-main.44","article-title":"Open-domain question answering goes\n                        conversational via question rewriting","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of\n                        the Association for Computational Linguistics: Human Language Technologies,\n                        NAACL-HLT","author":"Anantha","year":"2021"},{"key":"2023071720231306100_bib4","article-title":"Longformer: The long-document\n                        transformer","author":"Iz","year":"2020","journal-title":"CoRR"},{"key":"2023071720231306100_bib5","doi-asserted-by":"publisher","first-page":"414","DOI":"10.1162\/tacl_a_00468","article-title":"PADA: Example-based prompt learning for\n                        on-the-fly adaptation to unseen domains","volume":"10","author":"Ben-David","year":"2022","journal-title":"Transactions of the Association for Computational\n                        Linguistics"},{"key":"2023071720231306100_bib6","first-page":"1877","article-title":"Language models are few-shot\n                        learners","volume-title":"Advances in Neural Information\n                        Processing Systems","author":"Brown","year":"2020"},{"key":"2023071720231306100_bib7","first-page":"7302","article-title":"Doqa - accessing domain-specific faqs via\n                        conversational QA","volume-title":"Proceedings of the 58th Annual\n                        Meeting of the Association for Computational Linguistics, ACL 2020, Online,\n                        July 5\u201310, 2020","author":"Campos","year":"2020"},{"key":"2023071720231306100_bib8","doi-asserted-by":"publisher","first-page":"1230","DOI":"10.24963\/ijcai.2020\/171","article-title":"Graphflow: Exploiting conversation flow with graph neural\n                        networks for conversational machine comprehension","volume-title":"Proceedings of the Twenty-Ninth International Joint Conference on\n                        Artificial Intelligence, IJCAI","author":"Chen","year":"2020"},{"key":"2023071720231306100_bib9","doi-asserted-by":"publisher","first-page":"2174","DOI":"10.18653\/v1\/D18-1241","article-title":"Quac: Question answering in\n                        context","volume-title":"Proceedings of the 2018 Conference on\n                        Empirical Methods in Natural Language Processing, Brussels, Belgium, October\n                        31 \u2013 November 4, 2018","author":"Choi","year":"2018"},{"key":"2023071720231306100_bib10","doi-asserted-by":"publisher","first-page":"1383","DOI":"10.18653\/v1\/P18-1128","article-title":"The hitchhiker\u2019s guide to testing\n                        statistical significance in natural language processing","volume-title":"Proceedings of the 56th Annual Meeting of the Association for\n                        Computational Linguistics, ACL 2018, Melbourne, Australia, July\n                        15\u201320, 2018, Volume 1: Long Papers","author":"Dror","year":"2018"},{"key":"2023071720231306100_bib11","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-02174-9","volume-title":"Statistical Significance Testing for Natural\n                        Language Processing","author":"Dror","year":"2020"},{"key":"2023071720231306100_bib12","doi-asserted-by":"publisher","first-page":"5917","DOI":"10.18653\/v1\/D19-1605","article-title":"Can you unpack that? Learning to rewrite\n                        questions-in-context","volume-title":"Proceedings of the 2019\n                        Conference on Empirical Methods in Natural Language Processing and the 9th\n                        International Joint Conference on Natural Language Processing, EMNLP-IJCNLP\n                        2019, Hong Kong, China, November 3\u20137, 2019","author":"Elgohary","year":"2019"},{"key":"2023071720231306100_bib13","doi-asserted-by":"publisher","first-page":"2739","DOI":"10.18653\/v1\/2020.coling-main.247","article-title":"Conversational machine comprehension: A\n                        literature review","volume-title":"Proceedings of the 28th\n                        International Conference on Computational Linguistics, COLING","author":"Gupta","year":"2020"},{"key":"2023071720231306100_bib14","article-title":"Flowqa: Grasping flow in history for conversational machine\n                        comprehension","volume-title":"7th International Conference on\n                        Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6\u20139,\n                        2019","author":"Huang","year":"2019"},{"key":"2023071720231306100_bib15","doi-asserted-by":"publisher","first-page":"6130","DOI":"10.18653\/v1\/2021.acl-long.478","article-title":"Learn to resolve conversational dependency: A consistency\n                        training framework for conversational question answering","volume-title":"Proceedings of the 59th Annual Meeting of the Association for\n                        Computational Linguistics and the 11th International Joint Conference on\n                        Natural Language Processing, ACL\/IJCNLP 2021, (Volume 1: Long Papers),\n                        Virtual Event, August 1\u20136, 2021","author":"Kim","year":"2021"},{"key":"2023071720231306100_bib16","article-title":"Adam: A method for stochastic optimization","volume-title":"3rd International Conference on Learning Representations, ICLR 2015,\n                        San Diego, CA, USA, May 7\u20139, 2015, Conference Track\n                        Proceedings","author":"Kingma","year":"2015"},{"key":"2023071720231306100_bib17","doi-asserted-by":"publisher","first-page":"2627","DOI":"10.18653\/v1\/2021.naacl-main.208","article-title":"How many data points is a prompt worth?","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of\n                        the Association for Computational Linguistics: Human Language\n                        Technologies","author":"Scao","year":"2021"},{"key":"2023071720231306100_bib18","doi-asserted-by":"publisher","first-page":"8074","DOI":"10.18653\/v1\/2022.acl-long.555","article-title":"Ditch the gold standard: Re-evaluating conversational\n                        question answering","volume-title":"Proceedings of the 60th\n                        Annual Meeting of the Association for Computational Linguistics (Volume 1:\n                        Long Papers), ACL 2022, Dublin, Ireland, May 22\u201327, 2022","author":"Li","year":"2022"},{"key":"2023071720231306100_bib19","article-title":"Conversational question reformulation via\n                        sequence-to-sequence architectures and pretrained language\n                        models","author":"Lin","year":"2020","journal-title":"CoRR"},{"key":"2023071720231306100_bib20","article-title":"Pre-train, prompt, and predict: A\n                        systematic survey of prompting methods in natural language\n                        processing","author":"Liu","year":"2021","journal-title":"CoRR"},{"key":"2023071720231306100_bib21","article-title":"RoBERTa: A robustly optimized BERT\n                        pretraining approach","author":"Liu","year":"2019","journal-title":"CoRR"},{"key":"2023071720231306100_bib22","article-title":"Decoupled weight decay\n                        regularization","volume-title":"7th International Conference on\n                        Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6\u20139,\n                        2019","author":"Loshchilov","year":"2019"},{"key":"2023071720231306100_bib23","doi-asserted-by":"publisher","first-page":"2071","DOI":"10.18653\/v1\/2022.findings-naacl.159","article-title":"Capturing conversational interaction for question answering\n                        via global history reasoning","volume-title":"Findings of the\n                        Association for Computational Linguistics: NAACL 2022, Seattle, WA, United\n                        States, July 10\u201315, 2022","author":"Qian","year":"2022"},{"key":"2023071720231306100_bib24","article-title":"Open-retrieval conversational question\n                        answering","author":"Chen","year":"2020","journal-title":"Proceedings of the 43rd International ACM\n                        SIGIR Conference on Research and Development in Information\n                        Retrieval"},{"key":"2023071720231306100_bib25","first-page":"1133","article-title":"BERT with history answer embedding for\n                        conversational question answering","volume-title":"Proceedings of\n                        the 42nd International ACM SIGIR Conference on Research andDevelopment in\n                        Information Retrieval, SIGIR 2019, Paris, France, July 21\u201325,\n                        2019","author":"Chen","year":"2019"},{"key":"2023071720231306100_bib26","first-page":"1391","article-title":"Attentive history selection for\n                        conversational question answering","volume-title":"Proceedings of\n                        the 28th ACM International Conference on Information and Knowledge\n                        Management, CIKM 2019, Beijing, China, November 3\u20137, 2019","author":"Chen","year":"2019"},{"key":"2023071720231306100_bib27","first-page":"140:1\u2013140:67","article-title":"Exploring the limits of transfer learning with a unified\n                        text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"Journal of Machine\n                        Learning Research"},{"key":"2023071720231306100_bib28","doi-asserted-by":"publisher","first-page":"784","DOI":"10.18653\/v1\/P18-2124","article-title":"Know what you don\u2019t know: Unanswerable\n                        questions for squad","volume-title":"Proceedings of the 56th\n                        Annual Meeting of the Association for Computational Linguistics, ACL 2018,\n                        Melbourne, Australia, July 15\u201320, 2018, Volume 2: Short\n                        Papers","author":"Rajpurkar","year":"2018"},{"key":"2023071720231306100_bib29","doi-asserted-by":"publisher","first-page":"2383","DOI":"10.18653\/v1\/D16-1264","article-title":"Squad: 100, 000+ questions for machine\n                        comprehension of text","volume-title":"Proceedings of the 2016\n                        Conference on Empirical Methods in Natural Language Processing, EMNLP 2016,\n                        Austin, Texas, USA, November 1\u20134, 2016","author":"Rajpurkar","year":"2016"},{"key":"2023071720231306100_bib30","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1162\/tacl_a_00266","article-title":"Coqa: A conversational question answering\n                        challenge","volume":"7","author":"Reddy","year":"2019","journal-title":"Transactions of the Association for\n                        Computational Linguistics"},{"key":"2023071720231306100_bib31","first-page":"355","article-title":"Question rewriting for conversational\n                        question answering","volume-title":"WSDM \u201921, The\n                        Fourteenth ACM International Conference on Web Search and Data Mining,\n                        Virtual Event, Israel, March 8\u201312, 2021","author":"Vakulenko","year":"2021"},{"key":"2023071720231306100_bib32","first-page":"5998","article-title":"Attention is all you need","volume-title":"Advances in Neural Information Processing Systems 30: Annual\n                        Conference on Neural Information Processing Systems 2017, December\n                        4\u20139, 2017, Long Beach, CA, USA","author":"Vaswani","year":"2017"},{"key":"2023071720231306100_bib33","doi-asserted-by":"publisher","first-page":"38","DOI":"10.18653\/v1\/2020.emnlp-demos.6","article-title":"Transformers: State-of-the-art natural language\n                        processing","volume-title":"Proceedings of the 2020 Conference on\n                        Empirical Methods in Natural Language Processing: System\n                        Demonstrations","author":"Wolf","year":"2020"},{"key":"2023071720231306100_bib34","first-page":"86","article-title":"Flowdelta: Modeling flow information gain in reasoning for\n                        conversational machine comprehension","volume-title":"Proceedings\n                        of the 2nd Workshop on Machine Reading for Question Answering,\n                        MRQA@EMNLP 2019, Hong Kong, China, November 4, 2019","author":"Yeh","year":"2019"},{"key":"2023071720231306100_bib35","article-title":"Big bird: Transformers for longer\n                        sequences","volume-title":"Advances in Neural Information\n                        Processing Systems 33: Annual Conference on Neural Information Processing\n                        Systems 2020, NeurIPS 2020, December 6\u201312, 2020,\n                    virtual","author":"Zaheer","year":"2020"},{"key":"2023071720231306100_bib36","doi-asserted-by":"publisher","first-page":"1862","DOI":"10.18653\/v1\/2021.findings-emnlp.160","article-title":"Ror: Read-over-read for long document machine reading\n                        comprehension","volume-title":"Findings of the Association for\n                        Computational Linguistics: EMNLP 2021, Virtual Event \/ Punta Cana, Dominican\n                        Republic, 16\u201320 November, 2021","author":"Zhao","year":"2021"},{"key":"2023071720231306100_bib37","article-title":"Sdnet: Contextualized attention-based deep\n                        network for conversational question answering","author":"Zhu","year":"2018","journal-title":"CoRR"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00549\/2145199\/tacl_a_00549.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00549\/2145199\/tacl_a_00549.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,17]],"date-time":"2023-07-17T20:23:28Z","timestamp":1689625408000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00549\/115768\/On-the-Robustness-of-Dialogue-History"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":37,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00549","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}