{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T11:48:20Z","timestamp":1774352900785,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":50,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,8,23]],"date-time":"2022-08-23T00:00:00Z","timestamp":1661212800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,8,23]]},"DOI":"10.1145\/3539813.3545136","type":"proceedings-article","created":{"date-parts":[[2022,8,25]],"date-time":"2022-08-25T22:18:32Z","timestamp":1661465912000},"page":"51-60","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["PRE: A Precision-Recall-Effort Optimization Framework for Query Simulation"],"prefix":"10.1145","author":[{"given":"Sahiti","family":"Labhishetty","sequence":"first","affiliation":[{"name":"University of Illinois Urbana-Champaign, Urbana, IL, USA"}]},{"given":"ChengXiang","family":"Zhai","sequence":"additional","affiliation":[{"name":"University of Illinois Urbana-Champaign, Urbana, IL, USA"}]}],"member":"320","published-online":{"date-parts":[[2022,8,25]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2009923"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2600428.2609574"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277820"},{"key":"e_1_3_2_1_4_1","volume-title":"ACM SIGIR Forum","volume":"55","author":"Balog Krisztian","year":"2022","unstructured":"Krisztian Balog , David Maxwell , Paul Thomas , and Shuo Zhang . 2022 . Report on the 1st simulation for information retrieval workshop (Sim4IR 2021) at SIGIR 2021 . In ACM SIGIR Forum , Vol. 55 . ACM New York, NY, USA, 1--16. Krisztian Balog, David Maxwell, Paul Thomas, and Shuo Zhang. 2022. Report on the 1st simulation for information retrieval workshop (Sim4IR 2021) at SIGIR 2021. In ACM SIGIR Forum, Vol. 55. ACM New York, NY, USA, 1--16."},{"key":"e_1_3_2_1_6_1","volume-title":"Proceedings of ECIR.","author":"Baskaya F.","unstructured":"F. Baskaya , H. Keskustalo , and K. Jarvelin . 2011. Simulating simple and fallible relevance feedback . In Proceedings of ECIR. F. Baskaya, H. Keskustalo, and K. Jarvelin. 2011. Simulating simple and fallible relevance feedback. In Proceedings of ECIR."},{"key":"e_1_3_2_1_7_1","volume-title":"Proceedings of the 22nd ACM international conference on Information & Knowledge Management. ACM, 2297--2302","author":"Baskaya Feza","year":"2013","unstructured":"Feza Baskaya , Heikki Keskustalo , and Kalervo J\u00e4rvelin . 2013 . Modeling behavioral factors in interactive information retrieval . In Proceedings of the 22nd ACM international conference on Information & Knowledge Management. ACM, 2297--2302 . Feza Baskaya, Heikki Keskustalo, and Kalervo J\u00e4rvelin. 2013. Modeling behavioral factors in interactive information retrieval. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management. ACM, 2297--2302."},{"key":"e_1_3_2_1_8_1","volume-title":"The design of browsing and berrypicking techniques for the online search interface. Online review","author":"Bates Marcia J","year":"1989","unstructured":"Marcia J Bates . 1989. The design of browsing and berrypicking techniques for the online search interface. Online review ( 1989 ). Marcia J Bates. 1989. The design of browsing and berrypicking techniques for the online search interface. Online review (1989)."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb026722"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3204493.3204579"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883033"},{"key":"e_1_3_2_1_12_1","volume-title":"Advances in Information Retrieval, Matthias Hagen, Suzan Verberne, Craig Macdonald, Christin Seifert, Krisztian Balog, Kjetil N\u00f8rv\u00e5g","author":"Breuer Timo","unstructured":"Timo Breuer , Norbert Fuhr , and Philipp Schaer . 2022. Validating Simulations of User Query Variants . In Advances in Information Retrieval, Matthias Hagen, Suzan Verberne, Craig Macdonald, Christin Seifert, Krisztian Balog, Kjetil N\u00f8rv\u00e5g , and Vinay Setty (Eds.). Springer International Publishing , Cham , 80--94. Timo Breuer, Norbert Fuhr, and Philipp Schaer. 2022. Validating Simulations of User Query Variants. In Advances in Information Retrieval, Matthias Hagen, Suzan Verberne, Craig Macdonald, Christin Seifert, Krisztian Balog, Kjetil N\u00f8rv\u00e5g, and Vinay Setty (Eds.). Springer International Publishing, Cham, 80--94."},{"key":"e_1_3_2_1_13_1","volume-title":"Validating Simulations of User Query Variants. arXiv preprint arXiv:2201.07620","author":"Breuer Timo","year":"2022","unstructured":"Timo Breuer , Norbert Fuhr , and Philipp Schaer . 2022. Validating Simulations of User Query Variants. arXiv preprint arXiv:2201.07620 ( 2022 ). Timo Breuer, Norbert Fuhr, and Philipp Schaer. 2022. Validating Simulations of User Query Variants. arXiv preprint arXiv:2201.07620 (2022)."},{"key":"e_1_3_2_1_14_1","volume-title":"Learning, and Subtopic Ordering: A Simulation-based Analysis. arXiv preprint arXiv:2201.11181","author":"C\u00e2mara Arthur","year":"2022","unstructured":"Arthur C\u00e2mara , David Maxwell , and Claudia Hauff . 2022. Searching , Learning, and Subtopic Ordering: A Simulation-based Analysis. arXiv preprint arXiv:2201.11181 ( 2022 ). Arthur C\u00e2mara, David Maxwell, and Claudia Hauff. 2022. Searching, Learning, and Subtopic Ordering: A Simulation-based Analysis. arXiv preprint arXiv:2201.11181 (2022)."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2808194.2809470"},{"key":"e_1_3_2_1_16_1","volume-title":"Overview of the TREC 2014 session track. Technical Report. DELAWARE UNIV NEWARK DEPT OF COMPUTER AND INFORMATION SCIENCES.","author":"Carterette Ben","year":"2014","unstructured":"Ben Carterette , Evangelos Kanoulas , Mark Hall , and Paul Clough . 2014 . Overview of the TREC 2014 session track. Technical Report. DELAWARE UNIV NEWARK DEPT OF COMPUTER AND INFORMATION SCIENCES. Ben Carterette, Evangelos Kanoulas, Mark Hall, and Paul Clough. 2014. Overview of the TREC 2014 session track. Technical Report. DELAWARE UNIV NEWARK DEPT OF COMPUTER AND INFORMATION SCIENCES."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.2200\/S00654ED1V01Y201507ICR043"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/0020-0271(73)90004-1"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556195.2556217"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1086\/602622"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24027-5_7"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484055"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1498759.1498818"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983323.2983835"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835498"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb026960"},{"key":"e_1_3_2_1_27_1","unstructured":"Peter Ingwersen. 2005. Integrative framework for information seeking and interactive information retrieval. na.  Peter Ingwersen. 2005. Integrative framework for information seeking and interactive information retrieval. na."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1135777.1135835"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141753.1141818"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04769-5_6"},{"key":"e_1_3_2_1_31_1","volume-title":"Developing a model of the library search process: Cognitive and affective aspects. Rq","author":"Kuhlthau Carol Collier","year":"1988","unstructured":"Carol Collier Kuhlthau . 1988. Developing a model of the library search process: Cognitive and affective aspects. Rq ( 1988 ), 232--242. Carol Collier Kuhlthau. 1988. Developing a model of the library search process: Cognitive and affective aspects. Rq (1988), 232--242."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463091"},{"key":"e_1_3_2_1_33_1","volume-title":"RATE: A Reliability-Aware Tester-Based Evaluation Framework of User Simulators. In European Conference on Information Retrieval. Springer, 336--350","author":"Labhishetty Sahiti","year":"2022","unstructured":"Sahiti Labhishetty and ChengXiang Zhai . 2022 . RATE: A Reliability-Aware Tester-Based Evaluation Framework of User Simulators. In European Conference on Information Retrieval. Springer, 336--350 . Sahiti Labhishetty and ChengXiang Zhai. 2022. RATE: A Reliability-Aware Tester-Based Evaluation Framework of User Simulators. In European Conference on Information Retrieval. Springer, 336--350."},{"key":"e_1_3_2_1_34_1","volume-title":"Proceedings of the Data Science for Retail and E-Commerce Workshop.","author":"Labhishetty Sahiti","year":"2020","unstructured":"Sahiti Labhishetty , Chengxiang Zhai , Suhas Ranganath , and Pradeep Ranganathan . 2020 . A Cognitive User Model for E-Commerce Search . In Proceedings of the Data Science for Retail and E-Commerce Workshop. Sahiti Labhishetty, Chengxiang Zhai, Suhas Ranganath, and Pradeep Ranganathan. 2020. A Cognitive User Model for E-Commerce Search. In Proceedings of the Data Science for Retail and E-Commerce Workshop."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983323.2983805"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911469"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271676"},{"key":"e_1_3_2_1_38_1","unstructured":"Peter Norvig. 2008. Natural Language Corpus Data: Beautiful Data. http:\/\/norvig.com\/ngrams\/  Peter Norvig. 2008. Natural Language Corpus Data: Beautiful Data. http:\/\/norvig.com\/ngrams\/"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3343413.3377989"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-017-9301-2"},{"key":"e_1_3_2_1_41_1","volume-title":"Information foraging. Psychological review 106, 4","author":"Pirolli Peter","year":"1999","unstructured":"Peter Pirolli and Stuart Card . 1999. Information foraging. Psychological review 106, 4 ( 1999 ), 643. Peter Pirolli and Stuart Card. 1999. Information foraging. Psychological review 106, 4 (1999), 643."},{"key":"e_1_3_2_1_42_1","volume-title":"Studying the Effectiveness of Conversational Search Refinement Through User Simulation. In European Conference on Information Retrieval. Springer, 587--602","author":"Salle Alexandre","year":"2021","unstructured":"Alexandre Salle , Shervin Malmasi , Oleg Rokhlenko , and Eugene Agichtein . 2021 . Studying the Effectiveness of Conversational Search Refinement Through User Simulation. In European Conference on Information Retrieval. Springer, 587--602 . Alexandre Salle, Shervin Malmasi, Oleg Rokhlenko, and Eugene Agichtein. 2021. Studying the Effectiveness of Conversational Search Refinement Through User Simulation. In European Conference on Information Retrieval. Springer, 587--602."},{"key":"e_1_3_2_1_43_1","volume-title":"Human Language Technologies 2007: The Conference of the North American","author":"Schatzmann Jost","unstructured":"Jost Schatzmann , Blaise Thomson , Karl Weilhammer , Hui Ye , and Steve Young . 2007. Agenda-based user simulation for bootstrapping a POMDP dialogue system . In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers . 149--152. Jost Schatzmann, Blaise Thomson, Karl Weilhammer, Hui Ye, and Steve Young. 2007. Agenda-based user simulation for bootstrapping a POMDP dialogue system. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers. 149--152."},{"key":"e_1_3_2_1_44_1","volume-title":"BLEURT: Learning robust metrics for text generation. arXiv preprint arXiv:2004.04696","author":"Sellam Thibault","year":"2020","unstructured":"Thibault Sellam , Dipanjan Das , and Ankur P Parikh . 2020 . BLEURT: Learning robust metrics for text generation. arXiv preprint arXiv:2004.04696 (2020). Thibault Sellam, Dipanjan Das, and Ankur P Parikh. 2020. BLEURT: Learning robust metrics for text generation. arXiv preprint arXiv:2004.04696 (2020)."},{"key":"e_1_3_2_1_45_1","volume-title":"Proceedings of the 3rd annual ACM conference on Research and development in information retrieval. 236--255","author":"Tague Jean","year":"1980","unstructured":"Jean Tague , Michael Nelson , and Harry Wu . 1980 . Problems in the simulation of bibliographic retrieval systems . In Proceedings of the 3rd annual ACM conference on Research and development in information retrieval. 236--255 . Jean Tague, Michael Nelson, and Harry Wu. 1980. Problems in the simulation of bibliographic retrieval systems. In Proceedings of the 3rd annual ACM conference on Research and development in information retrieval. 236--255."},{"key":"e_1_3_2_1_46_1","volume-title":"ECIR","volume":"2017","author":"Gysel Christophe Van","year":"2017","unstructured":"Christophe Van Gysel , Evangelos Kanoulas , and Maarten de Rijke . 2017 . Pyndri: a Python Interface to the Indri Search Engine . In ECIR , Vol. 2017 . Springer. Christophe Van Gysel, Evangelos Kanoulas, and Maarten de Rijke. 2017. Pyndri: a Python Interface to the Indri Search Engine. In ECIR, Vol. 2017. Springer."},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-16354-3_75"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.10367"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2747874","article-title":"The query change model: Modeling session search as a markov decision process","volume":"33","author":"Yang Hui","year":"2015","unstructured":"Hui Yang , Dongyi Guan , and Sicong Zhang . 2015 . The query change model: Modeling session search as a markov decision process . ACM Transactions on Information Systems (TOIS) 33 , 4 (2015), 1 -- 33 . Hui Yang, Dongyi Guan, and Sicong Zhang. 2015. The query change model: Modeling session search as a markov decision process. ACM Transactions on Information Systems (TOIS) 33, 4 (2015), 1--33.","journal-title":"ACM Transactions on Information Systems (TOIS)"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403202"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"crossref","unstructured":"Yinan Zhang Xueqing Liu and ChengXiang Zhai. 2017. Information retrieval evaluation as search simulation: A general formal framework for ir evaluation. In ACM ICTIR. ACM 193--200.  Yinan Zhang Xueqing Liu and ChengXiang Zhai. 2017. Information retrieval evaluation as search simulation: A general formal framework for ir evaluation. In ACM ICTIR. ACM 193--200.","DOI":"10.1145\/3121050.3121070"}],"event":{"name":"ICTIR '22: The 2022 ACM SIGIR International Conference on the Theory of Information Retrieval","location":"Madrid Spain","acronym":"ICTIR '22","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3539813.3545136","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3539813.3545136","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:10:03Z","timestamp":1750183803000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3539813.3545136"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,23]]},"references-count":50,"alternative-id":["10.1145\/3539813.3545136","10.1145\/3539813"],"URL":"https:\/\/doi.org\/10.1145\/3539813.3545136","relation":{},"subject":[],"published":{"date-parts":[[2022,8,23]]},"assertion":[{"value":"2022-08-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}