{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T22:11:07Z","timestamp":1782339067496,"version":"3.54.5"},"reference-count":45,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,3,24]],"date-time":"2022-03-24T00:00:00Z","timestamp":1648080000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100020321","name":"School of Computing, University of Utah","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100020321","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2022,10,31]]},"abstract":"<jats:p>In conversational search, agents can interact with users by asking clarifying questions to increase their chance of finding better results. Many recent works and shared tasks in both natural language processing and information retrieval communities have focused on identifying the need to ask clarifying questions and methodologies of generating them. These works assume that asking a clarifying question is a safe alternative to retrieving results. As existing conversational search models are far from perfect, it is possible and common that they could retrieve\/generate bad clarifying questions. Asking too many clarifying questions can also drain a user\u2019s patience when the user prefers searching efficiency over correctness. Hence, these models can backfire and harm a user\u2019s search experience due to these risks from asking clarifying questions.<\/jats:p>\n          <jats:p\/>\n          <jats:p>\u00a0\u00a0In this work, we propose a simulation framework to simulate the risk of asking questions in conversational search and further revise a risk-aware conversational search model to control the risk. We show the model\u2019s robustness and effectiveness through extensive experiments on three conversational datasets \u2014 MSDialog, Ubuntu Dialog Corpus, and Opendialkg \u2014 in which we compare it with multiple baselines. We show that the risk-control module can work with two different re-ranker models and outperform all of the baselines in most of our experiments.<\/jats:p>","DOI":"10.1145\/3507357","type":"journal-article","created":{"date-parts":[[2022,2,1]],"date-time":"2022-02-01T18:09:23Z","timestamp":1643738963000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Simulating and Modeling the Risk of Conversational Search"],"prefix":"10.1145","volume":"40","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1228-7080","authenticated-orcid":false,"given":"Zhenduo","family":"Wang","sequence":"first","affiliation":[{"name":"University of Utah, Salt Lake City, Utah, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5030-709X","authenticated-orcid":false,"given":"Qingyao","family":"Ai","sequence":"additional","affiliation":[{"name":"University of Utah, Salt Lake City, Utah, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,3,24]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3343413.3377968"},{"key":"e_1_3_2_3_2","article-title":"ConvAI3: Generating clarifying questions for open-domain dialogue systems (ClariQ)","author":"Aliannejadi Mohammad","year":"2020","unstructured":"Mohammad Aliannejadi, Julia Kiseleva, Aleksandr Chuklin, Jeff Dalton, and Mikhail Burtsev. 2020. ConvAI3: Generating clarifying questions for open-domain dialogue systems (ClariQ). arXiv:2009.11352","journal-title":"arXiv:2009.11352"},{"key":"e_1_3_2_4_2","article-title":"Asking clarifying questions in open-domain information-seeking conversations","volume":"1907","author":"Aliannejadi Mohammad","year":"2019","unstructured":"Mohammad Aliannejadi, Hamed Zamani, Fabio Crestani, and W. Bruce Croft. 2019. Asking clarifying questions in open-domain information-seeking conversations. CoRR abs\/1907.06554 arXiv:1907.06554 http:\/\/arxiv.org\/abs\/1907.06554.","journal-title":"CoRR"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.21236\/ADA439446"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/1963405.1963424"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3357939"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458163"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/1507509.1507518"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3020165.3022149"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401995"},{"key":"e_1_3_2_12_2","article-title":"Sequential attention-based network for noetic end-to-end response selection","author":"Chen Qian","year":"2019","unstructured":"Qian Chen and Wen Wang. 2019. Sequential attention-based network for noetic end-to-end response selection. arXiv:1901.02609","journal-title":"arXiv:1901.02609"},{"key":"e_1_3_2_13_2","article-title":"Generating a common question from multiple documents using multi-source encoder-decoder models","author":"Cho Woon Sang","year":"2019","unstructured":"Woon Sang Cho, Yizhe Zhang, Sudha Rao, Chris Brockett, and Sungjin Lee. 2019. Generating a common question from multiple documents using multi-source encoder-decoder models. arXiv:1910.11483","journal-title":"arXiv:1910.11483"},{"key":"e_1_3_2_14_2","volume-title":"SumPre-HSWI@ ESWC","author":"Coden Anni","year":"2015","unstructured":"Anni Coden, Daniel Gruhl, Neal Lewis, and Pablo N. Mendes. 2015. Did you mean A or B? Supporting clarification dialog for entity disambiguation.. In SumPre-HSWI@ ESWC."},{"key":"e_1_3_2_15_2","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805","journal-title":"arXiv:1810.04805"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1525\/aa.1968.70.6.02a00030"},{"key":"e_1_3_2_17_2","article-title":"Convert: Efficient and accurate conversational representations from transformers","author":"Henderson Matthew","year":"2019","unstructured":"Matthew Henderson, Inigo Casanueva, Nikola Mrk\u0161i\u0107, Pei-Hao Su, Tsung-Hsien Wen, and Ivan Vuli\u0107. 2019. Convert: Efficient and accurate conversational representations from transformers. arXiv:1911.03688","journal-title":"arXiv:1911.03688"},{"key":"e_1_3_2_18_2","unstructured":"Wenyan Hu Xiaodi Zhang Alvaro Bolivar and Randall Scott Shoup. 2019. Search box auto-complete. US Patent App. 16\/237 245."},{"key":"e_1_3_2_19_2","article-title":"Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring","author":"Humeau Samuel","year":"2019","unstructured":"Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2019. Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring. arXiv:1905.01969","journal-title":"arXiv:1905.01969"},{"key":"e_1_3_2_20_2","article-title":"Conversational question answering over passages by leveraging word proximity networks","author":"Kaiser Magdalena","year":"2020","unstructured":"Magdalena Kaiser, Rishiraj Saha Roy, and Gerhard Weikum. 2020. Conversational question answering over passages by leveraging word proximity networks. arXiv:2004.13117","journal-title":"arXiv:2004.13117"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/1571941.1572006"},{"key":"e_1_3_2_22_2","article-title":"The Ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems","author":"Lowe Ryan","year":"2015","unstructured":"Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. 2015. The Ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. arXiv:1506.08909","journal-title":"arXiv:1506.08909"},{"key":"e_1_3_2_23_2","article-title":"Training millions of personalized dialogue agents","author":"Mazar\u00e9 Pierre-Emmanuel","year":"2018","unstructured":"Pierre-Emmanuel Mazar\u00e9, Samuel Humeau, Martin Raison, and Antoine Bordes. 2018. Training millions of personalized dialogue agents. arXiv:1809.01984","journal-title":"arXiv:1809.01984"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458145"},{"key":"e_1_3_2_25_2","article-title":"Topic propagation in conversational search","author":"Mele Ida","year":"2020","unstructured":"Ida Mele, Cristina Ioana Muntean, Franco Maria Nardini, Raffaele Perego, Nicola Tonellotto, and Ophir Frieder. 2020. Topic propagation in conversational search. arXiv:2004.14054","journal-title":"arXiv:2004.14054"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1081"},{"key":"e_1_3_2_27_2","article-title":"Open-retrieval conversational question answering","author":"Qu Chen","year":"2020","unstructured":"Chen Qu, Liu Yang, Cen Chen, Minghui Qiu, W. Bruce Croft, and Mohit Iyyer. 2020. Open-retrieval conversational question answering. arXiv:2005.11364","journal-title":"arXiv:2005.11364"},{"key":"e_1_3_2_28_2","volume-title":"SIGIR\u201918","author":"Qu C.","year":"2018","unstructured":"C. Qu, L. Yang, W. B. Croft, J. Trippas, Y. Zhang, and M. Qiu. 2018. Analyzing and characterizing user intent in information-seeking conversations.. In SIGIR\u201918."},{"key":"e_1_3_2_29_2","volume-title":"CHIIR\u201919","author":"Qu C.","year":"2019","unstructured":"C. Qu, L. Yang, W. B. Croft, Y. Zhang, J. Trippas, and M. Qiu. 2019. User intent prediction in information-seeking conversations. In CHIIR\u201919."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3020165.3020183"},{"key":"e_1_3_2_31_2","article-title":"Learning to ask good questions: Ranking clarification questions using neural expected value of perfect information","author":"Rao Sudha","year":"2018","unstructured":"Sudha Rao and Hal Daum\u00e9 III. 2018. Learning to ask good questions: Ranking clarification questions using neural expected value of perfect information. arXiv:1805.04655","journal-title":"arXiv:1805.04655"},{"key":"e_1_3_2_32_2","article-title":"Answer-based adversarial training for generating clarification questions","volume":"1904","author":"Rao Sudha","year":"2019","unstructured":"Sudha Rao and Hal Daum\u00e9 III. 2019. Answer-based adversarial training for generating clarification questions. CoRR abs\/1904.02281 (2019). arXiv:1904.02281 http:\/\/arxiv.org\/abs\/1904.02281.","journal-title":"CoRR"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380193"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/2806416.2806493"},{"key":"e_1_3_2_35_2","article-title":"Controlling risk of web question answering","volume":"1905","author":"Su Lixin","year":"2019","unstructured":"Lixin Su, Jiafeng Guo, Yixing Fan, Yanyan Lan, and Xueqi Cheng. 2019. Controlling risk of web question answering. CoRR abs\/1905.10077 (2019). arXiv:1905.10077 http:\/\/arxiv.org\/abs\/1905.10077.","journal-title":"CoRR"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210002"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-15712-8_18"},{"key":"e_1_3_2_38_2","article-title":"Query resolution for conversational search with limited supervision","author":"Voskarides Nikos","year":"2020","unstructured":"Nikos Voskarides, Dan Li, Pengjie Ren, Evangelos Kanoulas, and Maarten de Rijke. 2020. Query resolution for conversational search with limited supervision. arXiv:2005.11723","journal-title":"arXiv:2005.11723"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3449893"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1172"},{"key":"e_1_3_2_41_2","volume-title":"SIGIR\u201918","author":"Yang L.","year":"2018","unstructured":"L. Yang, M. Qiu, C. Qu, J. Guo, Y. Zhang, W. B. Croft, J. Huang, and H. Chen. 2018. Response ranking with deep matching networks and external knowledge in information-seeking conversation systems. In SIGIR\u201918."},{"key":"e_1_3_2_42_2","article-title":"Neural matching models for question retrieval and next question prediction in conversation","author":"Yang Liu","year":"2017","unstructured":"Liu Yang, Hamed Zamani, Yongfeng Zhang, Jiafeng Guo, and W. Bruce Croft. 2017. Neural matching models for question retrieval and next question prediction in conversation. arXiv:1707.05409","journal-title":"arXiv:1707.05409"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380126"},{"key":"e_1_3_2_44_2","article-title":"Mimics: A large-scale data collection for search clarification","author":"Zamani Hamed","year":"2020","unstructured":"Hamed Zamani, Gord Lueck, Everest Chen, Rodolfo Quispe, Flint Luu, and Nick Craswell. 2020. Mimics: A large-scale data collection for search clarification. arXiv:2006.10174","journal-title":"arXiv:2006.10174"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401160"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271776"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3507357","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3507357","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:31:11Z","timestamp":1750188671000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3507357"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,24]]},"references-count":45,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,10,31]]}},"alternative-id":["10.1145\/3507357"],"URL":"https:\/\/doi.org\/10.1145\/3507357","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,24]]},"assertion":[{"value":"2021-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-12-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-03-24","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}