{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T05:05:10Z","timestamp":1784178310354,"version":"3.55.0"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,9,1]],"date-time":"2021-09-01T00:00:00Z","timestamp":1630454400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2021,10,31]]},"abstract":"<jats:p>Conversational search is a relatively young area of research that aims at automating an information-seeking dialogue. In this article, we help to position it with respect to other research areas within conversational artificial intelligence (AI) by analysing the structural properties of an information-seeking dialogue. To this end, we perform a large-scale dialogue analysis of more than 150K transcripts from 16 publicly available dialogue datasets. These datasets were collected to inform different dialogue-based tasks including conversational search. We extract different patterns of mixed initiative from these dialogue transcripts and use them to compare dialogues of different types. Moreover, we contrast the patterns found in information-seeking dialogues that are being used for research purposes with the patterns found in virtual reference interviews that were conducted by professional librarians. The insights we provide (1) establish close relations between conversational search and other conversational AI tasks and (2) uncover limitations of existing conversational datasets to inform future data collection tasks.<\/jats:p>","DOI":"10.1145\/3466796","type":"journal-article","created":{"date-parts":[[2021,9,1]],"date-time":"2021-09-01T19:44:40Z","timestamp":1630525480000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":17,"title":["A Large-scale Analysis of Mixed Initiative in Information-Seeking Dialogues for Conversational Search"],"prefix":"10.1145","volume":"39","author":[{"given":"Svitlana","family":"Vakulenko","sequence":"first","affiliation":[{"name":"University of Amsterdam, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Evangelos","family":"Kanoulas","sequence":"additional","affiliation":[{"name":"University of Amsterdam, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maarten","family":"De Rijke","sequence":"additional","affiliation":[{"name":"University of Amsterdam &amp; Ahold Delhaize, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,9]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Le","author":"Adiwardana Daniel","year":"2020","unstructured":"Daniel Adiwardana , Minh-Thang Luong , David R. So , Jamie Hall , Noah Fiedel , Romal Thoppilan , Zi Yang , Apoorv Kulshreshtha , Gaurav Nemade , Yifeng Lu , and Quoc V . Le . 2020 . Towards a human-like open-domain chatbot. arxiv:2001.09977. Retrieved from https:\/\/arxiv.org\/abs\/2001.09977. Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. 2020. Towards a human-like open-domain chatbot. arxiv:2001.09977. Retrieved from https:\/\/arxiv.org\/abs\/2001.09977."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331265"},{"key":"e_1_2_1_3_1","first-page":"34","article-title":"Conversational Search (Dagstuhl Seminar 19461)","volume":"9","author":"Anand Avishek","year":"2020","unstructured":"Avishek Anand , Lawrence Cavedon , Hideo Joho , Mark Sanderson , and Benno Stein . 2020 . Conversational Search (Dagstuhl Seminar 19461) . Dagstuhl Rep. 9 , 11 (2020), 34 \u2013 83 . Avishek Anand, Lawrence Cavedon, Hideo Joho, Mark Sanderson, and Benno Stein. 2020. Conversational Search (Dagstuhl Seminar 19461). Dagstuhl Rep. 9, 11 (2020), 34\u201383.","journal-title":"Dagstuhl Rep."},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the 2nd International Workshop on Conversational Approaches to Information Retrieval.","author":"Azzopardi Leif","year":"2018","unstructured":"Leif Azzopardi , Mateusz Dubiel , Martin Halvey , and Jeffery Dalton . 2018 . Conceptualizing agent-human interactions during the conversational search process . In Proceedings of the 2nd International Workshop on Conversational Approaches to Information Retrieval. Leif Azzopardi, Mateusz Dubiel, Martin Halvey, and Jeffery Dalton. 2018. Conceptualizing agent-human interactions during the conversational search process. In Proceedings of the 2nd International Workshop on Conversational Approaches to Information Retrieval."},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 85\u201396","author":"Bao Siqi","year":"2019","unstructured":"Siqi Bao , Huang He , Fan Wang , Hua Wu , and Haifeng Wang . 2019 . Plato: Pre-trained dialogue generation model with discrete latent variable . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 85\u201396 . Siqi Bao, Huang He, Fan Wang, Hua Wu, and Haifeng Wang. 2019. Plato: Pre-trained dialogue generation model with discrete latent variable. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 85\u201396."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/0957-4174(95)00011-W"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb026722"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the International Society for the Study of Argumentation","author":"Birrer Frans A. J.","year":"1998","unstructured":"Frans A. J. Birrer . 1998 . Asymmetry in the dialogue between expert and non-expert . In Proceedings of the International Society for the Study of Argumentation (1998). Frans A. J. Birrer. 1998. Asymmetry in the dialogue between expert and non-expert. In Proceedings of the International Society for the Study of Argumentation (1998)."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1547"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1241"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008205526147"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008398023083"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/3157680.3157726"},{"key":"e_1_2_1_14_1","volume-title":"The NeurIPS\u201918 Competition","author":"Dinan Emily","unstructured":"Emily Dinan , Varvara Logacheva , Valentin Malykh , Alexander Miller , Kurt Shuster , Jack Urbanek , Douwe Kiela , Arthur Szlam , Iulian Serban , Ryan Lowe , Shrimai Prabhumoye , Alan W. Black , Alexander Rudnicky , Jason Williams , Joelle Pineau , Mikhail Burtsev , and Jason Weston . 2020. The Second Conversational Intelligence Challenge (ConvAI2) . In The NeurIPS\u201918 Competition . Springer , 187\u2013208. Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek, Douwe Kiela, Arthur Szlam, Iulian Serban, Ryan Lowe, Shrimai Prabhumoye, Alan W. Black, Alexander Rudnicky, Jason Williams, Joelle Pineau, Mikhail Burtsev, and Jason Weston. 2020. The Second Conversational Intelligence Challenge (ConvAI2). In The NeurIPS\u201918 Competition. Springer, 187\u2013208."},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the 7th International Conference on Learning Representations (ICLR\u201919)","author":"Dinan Emily","year":"2019","unstructured":"Emily Dinan , Stephen Roller , Kurt Shuster , Angela Fan , Michael Auli , and Jason Weston . 2019 . Wizard of Wikipedia: Knowledge-powered conversational agents . In Proceedings of the 7th International Conference on Learning Representations (ICLR\u201919) . Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard of Wikipedia: Knowledge-powered conversational agents. In Proceedings of the 7th International Conference on Learning Representations (ICLR\u201919)."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSC.2007.54"},{"key":"e_1_2_1_17_1","unstructured":"Chongming Gao Wenqiang Lei Xiangnan He Maarten de Rijke and Tat-Seng Chua. 2021. Advances and challenges in conversational recommender systems: A survey. arxiv:2101.09459. Retrieved from https:\/\/arxiv.org\/abs\/2101.09459.  Chongming Gao Wenqiang Lei Xiangnan He Maarten de Rijke and Tat-Seng Chua. 2021. Advances and challenges in conversational recommender systems: A survey. arxiv:2101.09459. Retrieved from https:\/\/arxiv.org\/abs\/2101.09459."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000074"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3453154"},{"key":"e_1_2_1_20_1","unstructured":"Geoffrey Leech and Martin Weisser. 2013. The SPAADIA Annotation Scheme. Retreived from http:\/\/martinweisser.org\/publications\/SPAADIA_Annotation_Scheme.pdf.  Geoffrey Leech and Martin Weisser. 2013. The SPAADIA Annotation Scheme. Retreived from http:\/\/martinweisser.org\/publications\/SPAADIA_Annotation_Scheme.pdf."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/3327546.3327641"},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the 8th International Joint Conference on Natural Language Processing (IJCNLP\u201917)","volume":"995","author":"Li Yanran","year":"2017","unstructured":"Yanran Li , Hui Su , Xiaoyu Shen , Wenjie Li , Ziqiang Cao , and Shuzi Niu . 2017 . DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset . In Proceedings of the 8th International Joint Conference on Natural Language Processing (IJCNLP\u201917) , Volume 1: Long Papers. 986\u2013 995 . Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset. In Proceedings of the 8th International Joint Conference on Natural Language Processing (IJCNLP\u201917), Volume 1: Long Papers. 986\u2013995."},{"key":"e_1_2_1_23_1","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arxiv:1907.11692. Retrieved from https:\/\/arxiv.org\/abs\/1907.11692.  Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arxiv:1907.11692. Retrieved from https:\/\/arxiv.org\/abs\/1907.11692."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W15-4640"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1081"},{"key":"e_1_2_1_26_1","unstructured":"Gustavo Penha Alexandru Balan and Claudia Hauff. 2019. Introducing MANtIS: A novel multi-domain information seeking dialogues dataset. arxiv:1912.04639. Retrieved from https:\/\/arxiv.org\/abs\/1912.04639.  Gustavo Penha Alexandru Balan and Claudia Hauff. 2019. Introducing MANtIS: A novel multi-domain information seeking dialogues dataset. arxiv:1912.04639. Retrieved from https:\/\/arxiv.org\/abs\/1912.04639."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210124"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.lisr.2012.08.001"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-5941"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3020165.3020183"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00266"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2019.2946162"},{"key":"e_1_2_1_33_1","volume-title":"Users and intermediaries in information retrieval: What are they talking about? In User Modeling","author":"Saracevic Tefko","unstructured":"Tefko Saracevic , Amanda Spink , and Mei-Mei Wu. 1997. Users and intermediaries in information retrieval: What are they talking about? In User Modeling . Springer , 43\u201354. Tefko Saracevic, Amanda Spink, and Mei-Mei Wu. 1997. Users and intermediaries in information retrieval: What are they talking about? In User Modeling. Springer, 43\u201354."},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL\u201919)","author":"Roller Stephen","year":"2019","unstructured":"Abigail See, Stephen Roller , Douwe Kiela , and Jason Weston . 2019 . What Makes a Good Conversation? How Controllable Attributes Affect Human Judgments . In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL\u201919) . 1702\u20131723. Abigail See, Stephen Roller, Douwe Kiela, and Jason Weston. 2019. What Makes a Good Conversation? How Controllable Attributes Affect Human Judgments. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL\u201919). 1702\u20131723."},{"key":"e_1_2_1_35_1","doi-asserted-by":"crossref","unstructured":"Koustuv Sinha Prasanna Parthasarathi Jasmine Wang Ryan Lowe William L. Hamilton and Joelle Pineau. 2020. Learning an unreferenced metric for online dialogue evaluation. (2020) 2430\u20132441.  Koustuv Sinha Prasanna Parthasarathi Jasmine Wang Ryan Lowe William L. Hamilton and Joelle Pineau. 2020. Learning an unreferenced metric for online dialogue evaluation. (2020) 2430\u20132441.","DOI":"10.18653\/v1\/2020.acl-main.220"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(92)90044-Z"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008303312758"},{"key":"e_1_2_1_38_1","volume-title":"Question-negotiation and information seeking in libraries. Coll. Res. Libr. 29, 3","author":"Taylor Robert S.","year":"1968","unstructured":"Robert S. Taylor . 1968. Question-negotiation and information seeking in libraries. Coll. Res. Libr. 29, 3 ( 1968 ). Robert S. Taylor. 1968. Question-negotiation and information seeking in libraries. Coll. Res. Libr. 29, 3 (1968)."},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the SIGIR 3th International Workshop on Conversational Approaches to Information Retrieval (CAIR\u201920)","author":"Thomas Paul","year":"2020","unstructured":"Paul Thomas , Mary Czerwinski , Daniel McDuff , and Nick Craswell . 2020 . Theories of Conversation for Conversational IR . In Proceedings of the SIGIR 3th International Workshop on Conversational Approaches to Information Retrieval (CAIR\u201920) . Paul Thomas, Mary Czerwinski, Daniel McDuff, and Nick Craswell. 2020. Theories of Conversation for Conversational IR. In Proceedings of the SIGIR 3th International Workshop on Conversational Approaches to Information Retrieval (CAIR\u201920)."},{"key":"e_1_2_1_40_1","volume-title":"Proceedings of the 1st International Workshop on Conversational Approaches to Information Retrieval.","author":"Thomas Paul","year":"2017","unstructured":"Paul Thomas , Daniel McDuff , Mary Czerwinski , and Nick Craswell . 2017 . MISC: A data set of information-seeking conversations . In Proceedings of the 1st International Workshop on Conversational Approaches to Information Retrieval. Paul Thomas, Daniel McDuff, Mary Czerwinski, and Nick Craswell. 2017. MISC: A data set of information-seeking conversations. In Proceedings of the 1st International Workshop on Conversational Approaches to Information Retrieval."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3176349.3176387"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2019.102162"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401297"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-15712-8_35"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.3115\/981823.981833"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.3115\/982023.982038"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380126"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3466796","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3466796","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:28:09Z","timestamp":1750195689000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3466796"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9]]},"references-count":47,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,10,31]]}},"alternative-id":["10.1145\/3466796"],"URL":"https:\/\/doi.org\/10.1145\/3466796","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9]]},"assertion":[{"value":"2020-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-09-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}