{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,12]],"date-time":"2025-12-12T13:09:01Z","timestamp":1765544941425,"version":"3.41.0"},"reference-count":23,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,6,1]],"date-time":"2023-06-01T00:00:00Z","timestamp":1685577600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGIR Forum"],"published-print":{"date-parts":[[2023,6]]},"abstract":"<jats:p>Query Performance Prediction (QPP) is currently primarily applied to ad-hoc retrieval tasks. The Information Retrieval (IR) field is reaching new heights thanks to recent advances in large language models and neural networks, as well as emerging new ways of searching, such as conversational search. Such advancements are quickly spreading to adjacent research areas, including QPP, necessitating a reconsideration of how we perform and evaluate QPP. This workshop sought to elicit discussion on three topics related to the future of QPP: exploiting advances in IR to improve QPP, instantiating QPP on new search paradigms, and evaluating QPP on new tasks.<\/jats:p>\n          <jats:p>\n            <jats:bold>Date:<\/jats:bold>\n            6 April 2023.\n          <\/jats:p>\n          <jats:p>\n            <jats:bold>Website:<\/jats:bold>\n            https:\/\/qpp.dei.unipd.it\/.\n          <\/jats:p>","DOI":"10.1145\/3636341.3636356","type":"journal-article","created":{"date-parts":[[2023,12,4]],"date-time":"2023-12-04T17:07:47Z","timestamp":1701709667000},"page":"1-7","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Report on the 1st Workshop on Query Performance Prediction and Its Evaluation in New Tasks (QPP++ 2023) at ECIR 2023"],"prefix":"10.1145","volume":"57","author":[{"given":"Guglielmo","family":"Faggioli","sequence":"first","affiliation":[{"name":"University of Padua, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nicola","family":"Ferro","sequence":"additional","affiliation":[{"name":"University of Padua, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Josiane","family":"Mothe","sequence":"additional","affiliation":[{"name":"INSPE, Universit\u00e9 de Toulouse, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fiana","family":"Raiber","sequence":"additional","affiliation":[{"name":"Yahoo Research, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maik","family":"Fr\u00f6be","sequence":"additional","affiliation":[{"name":"Friedrich-Schiller-Universit\u00e4t Jena, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,12,4]]},"reference":[{"issue":"1","key":"e_1_2_1_1_1","first-page":"1","volume":"2","author":"Carmel David","year":"2010","unstructured":"David Carmel and Elad Yom-Tov . Estimating the query difficulty for information retrieval. Synthesis Lectures on Information Concepts , Retrieval, and Services , 2 ( 1 ): 1 -- 89 , 2010 . David Carmel and Elad Yom-Tov. Estimating the query difficulty for information retrieval. Synthesis Lectures on Information Concepts, Retrieval, and Services, 2(1):1--89, 2010.","journal-title":"Services"},{"key":"e_1_2_1_2_1","first-page":"903","volume-title":"Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2011","author":"Carterette Ben","year":"2011","unstructured":"Ben Carterette . System effectiveness, user models, and user utility: a conceptual framework for investigation . In Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2011 , pages 903 -- 912 , 2011 . Ben Carterette. System effectiveness, user models, and user utility: a conceptual framework for investigation. In Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2011, pages 903--912, 2011."},{"key":"e_1_2_1_3_1","first-page":"1","volume-title":"Faggioli et al. [2023b]","author":"Datta Suchana","unstructured":"Suchana Datta , Debasis Ganguly , Derek Greene , and Mandar Mitra . On the feasibility and robustness of pointwise evaluation of query performance prediction . In Faggioli et al. [2023b] , pages 1 -- 6 . URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-01. Suchana Datta, Debasis Ganguly, Derek Greene, and Mandar Mitra. On the feasibility and robustness of pointwise evaluation of query performance prediction. In Faggioli et al. [2023b], pages 1--6. URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-01."},{"key":"e_1_2_1_4_1","first-page":"7","volume-title":"Faggioli et al. [2023b]","author":"Datta Suchana","unstructured":"Suchana Datta , Debasis Ganguly , Josiane Mothe , and Md Zia Ullah . Combining word embedding interactions and LETOR feature evidences for supervised QPP . In Faggioli et al. [2023b] , pages 7 -- 14 . URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-02. Suchana Datta, Debasis Ganguly, Josiane Mothe, and Md Zia Ullah. Combining word embedding interactions and LETOR feature evidences for supervised QPP. In Faggioli et al. [2023b], pages 7--14. URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-02."},{"key":"e_1_2_1_5_1","volume-title":"Md Zia Ullah, and Jian-Yun Nie. Learning to adaptively rank document retrieval system configurations. ACM Transactions on Information Systems (TOIS), 37(1):1--41","author":"Deveaud Romain","year":"2018","unstructured":"Romain Deveaud , Josiane Mothe , Md Zia Ullah, and Jian-Yun Nie. Learning to adaptively rank document retrieval system configurations. ACM Transactions on Information Systems (TOIS), 37(1):1--41 , 2018 . Romain Deveaud, Josiane Mothe, Md Zia Ullah, and Jian-Yun Nie. Learning to adaptively rank document retrieval system configurations. ACM Transactions on Information Systems (TOIS), 37(1):1--41, 2018."},{"key":"e_1_2_1_6_1","volume-title":"ECIR 2021","volume":"12656","author":"Faggioli Guglielmo","year":"2021","unstructured":"Guglielmo Faggioli , Oleg Zendel , J. Shane Culpepper , Nicola Ferro , and Falk Scholer . An Enhanced Evaluation Framework for Query Performance Prediction. In Advances in Information Retrieval - 43rd European Conference on IR Research , ECIR 2021 , volume 12656 of Lecture Notes in Computer Science, pages 115--129 , 2021 . Guglielmo Faggioli, Oleg Zendel, J. Shane Culpepper, Nicola Ferro, and Falk Scholer. An Enhanced Evaluation Framework for Query Performance Prediction. In Advances in Information Retrieval - 43rd European Conference on IR Research, ECIR 2021, volume 12656 of Lecture Notes in Computer Science, pages 115--129, 2021."},{"key":"e_1_2_1_7_1","first-page":"388","volume-title":"ECIR 2023","volume":"13982","author":"Faggioli Guglielmo","year":"2023","unstructured":"Guglielmo Faggioli , Nicola Ferro , Josiane Mothe , and Fiana Raiber . QPP++ 2023 : Query-performance prediction and its evaluation in new tasks. In Advances in Information Retrieval - 45th European Conference on Information Retrieval , ECIR 2023 , volume 13982 , pages 388 -- 391 , 2023a. Guglielmo Faggioli, Nicola Ferro, Josiane Mothe, and Fiana Raiber. QPP++ 2023: Query-performance prediction and its evaluation in new tasks. In Advances in Information Retrieval - 45th European Conference on Information Retrieval, ECIR 2023, volume 13982, pages 388--391, 2023a."},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the The QPP++ 2023: Query Performance Prediction and Its Evaluation in New Tasks Workshop (QPP++), number 3366 in CEUR Workshop Proceedings","author":"Faggioli Guglielmo","year":"2023","unstructured":"Guglielmo Faggioli , Nicola Ferro , Josiane Mothe , and Fiana Raiber , editors. Proceedings of the The QPP++ 2023: Query Performance Prediction and Its Evaluation in New Tasks Workshop (QPP++), number 3366 in CEUR Workshop Proceedings , Aachen , 2023 b. URL http:\/\/ceur-ws.org\/Vol-3366\/. Guglielmo Faggioli, Nicola Ferro, Josiane Mothe, and Fiana Raiber, editors. Proceedings of the The QPP++ 2023: Query Performance Prediction and Its Evaluation in New Tasks Workshop (QPP++), number 3366 in CEUR Workshop Proceedings, Aachen, 2023b. URL http:\/\/ceur-ws.org\/Vol-3366\/."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591625"},{"key":"e_1_2_1_10_1","volume-title":"ECIR 2023","volume":"13980","author":"Faggioli Guglielmo","year":"2023","unstructured":"Guglielmo Faggioli , Thibault Formal , Stefano Marchesin , St\u00e9phane Clinchant , Nicola Ferro , and Benjamin Piwowarski . Query Performance Prediction for Neural IR: Are We There Yet? In Advances in Information Retrieval - 45th European Conference on Information Retrieval , ECIR 2023 , volume 13980 of Lecture Notes in Computer Science, pages 232--248 , 2023 d. Guglielmo Faggioli, Thibault Formal, Stefano Marchesin, St\u00e9phane Clinchant, Nicola Ferro, and Benjamin Piwowarski. Query Performance Prediction for Neural IR: Are We There Yet? In Advances in Information Retrieval - 45th European Conference on Information Retrieval, ECIR 2023, volume 13980 of Lecture Notes in Computer Science, pages 232--248, 2023d."},{"key":"e_1_2_1_11_1","first-page":"15","volume-title":"Faggioli et al. [2023b]","author":"Fr\u00f6be Maik","unstructured":"Maik Fr\u00f6be , Eric Oliver Schmidt, and Matthias Hagen. A large-scale dataset for known-item question performance prediction . In Faggioli et al. [2023b] , pages 15 -- 19 . URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-03. Maik Fr\u00f6be, Eric Oliver Schmidt, and Matthias Hagen. A large-scale dataset for known-item question performance prediction. In Faggioli et al. [2023b], pages 15--19. URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-03."},{"key":"e_1_2_1_12_1","first-page":"20","volume-title":"Faggioli et al. [2023b]","author":"Gonz\u00e1lez-S\u00e1ez Gabriela","unstructured":"Gabriela Gonz\u00e1lez-S\u00e1ez , Alaa El-Ebshihy , Petra Galu\u0161\u010d\u00e1kov\u00e1 , David Iommi , Florina Piroi , Lorraine Goeuriot , and Philippe Mulhem . Towards result delta prediction based on knowledge deltas for continuous IR evaluation . In Faggioli et al. [2023b] , pages 20 -- 24 . URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-04. Gabriela Gonz\u00e1lez-S\u00e1ez, Alaa El-Ebshihy, Petra Galu\u0161\u010d\u00e1kov\u00e1, David Iommi, Florina Piroi, Lorraine Goeuriot, and Philippe Mulhem. Towards result delta prediction based on knowledge deltas for continuous IR evaluation. In Faggioli et al. [2023b], pages 20--24. URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-04."},{"key":"e_1_2_1_14_1","first-page":"301","volume-title":"31th European Conference on IR Research, ECIR 2009","author":"Hauff Claudia","year":"2009","unstructured":"Claudia Hauff , Leif Azzopardi , and Djoerd Hiemstra . The combination and evaluation of query performance prediction methods. In Advances in Information Retrieval , 31th European Conference on IR Research, ECIR 2009 , pages 301 -- 312 , 2009 . Claudia Hauff, Leif Azzopardi, and Djoerd Hiemstra. The combination and evaluation of query performance prediction methods. In Advances in Information Retrieval, 31th European Conference on IR Research, ECIR 2009, pages 301--312, 2009."},{"key":"e_1_2_1_15_1","first-page":"25","volume-title":"Faggioli et al. [2023b]","author":"Meng Chuan","unstructured":"Chuan Meng , Mohammad Aliannejadi , and Maarten de Rijke . Performance prediction for conversational search using perplexities of query rewrites . In Faggioli et al. [2023b] , pages 25 -- 28 . URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-05. Chuan Meng, Mohammad Aliannejadi, and Maarten de Rijke. Performance prediction for conversational search using perplexities of query rewrites. In Faggioli et al. [2023b], pages 25--28. URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-05."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000061"},{"key":"e_1_2_1_17_1","first-page":"29","volume-title":"Faggioli et al. [2023b]","author":"Mothe Josiane","unstructured":"Josiane Mothe . On correlation to evaluate QPP . In Faggioli et al. [2023b] , pages 29 -- 36 . URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-06. Josiane Mothe. On correlation to evaluate QPP. In Faggioli et al. [2023b], pages 29--36. URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-06."},{"key":"e_1_2_1_18_1","first-page":"7","volume-title":"Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2005","author":"Mothe Josiane","year":"2005","unstructured":"Josiane Mothe and Ludovic Tanguy . Linguistic Features to Predict Query Difficulty . In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2005 , pages 7 -- 10 , 2005 . Josiane Mothe and Ludovic Tanguy. Linguistic Features to Predict Query Difficulty. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2005, pages 7--10, 2005."},{"key":"e_1_2_1_19_1","first-page":"13","volume-title":"Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR","author":"Raiber Fiana","year":"2014","unstructured":"Fiana Raiber and Oren Kurland . Query-Performance Prediction: Setting the Expectations Straight . In Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2014 , pages 13 -- 22 , 2014. Fiana Raiber and Oren Kurland. Query-Performance Prediction: Setting the Expectations Straight. In Proceedings of the 37th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2014, pages 13--22, 2014."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3234944.3234950"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1571941.1572055"},{"key":"e_1_2_1_22_1","first-page":"259","volume-title":"Proceedings of the 33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR","author":"Shtok Anna","year":"2010","unstructured":"Anna Shtok , Oren Kurland , and David Carmel . Using Statistical Decision Theory and Relevance Models for Query-Performance Prediction . In Proceedings of the 33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010 , pages 259 -- 266 , 2010. Anna Shtok, Oren Kurland, and David Carmel. Using Statistical Decision Theory and Relevance Models for Query-Performance Prediction. In Proceedings of the 33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010, pages 259--266, 2010."},{"key":"e_1_2_1_23_1","first-page":"1","volume-title":"Proceedings of the 22nd Australasian Document Computing Symposium, ADCS 2017","author":"Thomas Paul","year":"2017","unstructured":"Paul Thomas , Falk Scholer , Peter Bailey , and Alistair Moffat . Tasks, queries , and rankers in pre-retrieval performance prediction . Proceedings of the 22nd Australasian Document Computing Symposium, ADCS 2017 , pages 1 -- 4 , 2017 . Paul Thomas, Falk Scholer, Peter Bailey, and Alistair Moffat. Tasks, queries, and rankers in pre-retrieval performance prediction. Proceedings of the 22nd Australasian Document Computing Symposium, ADCS 2017, pages 1--4, 2017."},{"key":"e_1_2_1_24_1","first-page":"37","volume-title":"Faggioli et al. [2023b]","author":"Zendel Oleg","unstructured":"Oleg Zendel , Binsheng Liu , J. Shane Culpepper , and Falk Scholer . Entropy-based query performance prediction for neural information retrieval systems . In Faggioli et al. [2023b] , pages 37 -- 44 . URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-07. Oleg Zendel, Binsheng Liu, J. Shane Culpepper, and Falk Scholer. Entropy-based query performance prediction for neural information retrieval systems. In Faggioli et al. [2023b], pages 37--44. URL http:\/\/ceur-ws.org\/Vol-3366\/#paper-07."}],"container-title":["ACM SIGIR Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3636341.3636356","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3636341.3636356","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:35:41Z","timestamp":1750178141000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3636341.3636356"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6]]},"references-count":23,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,6]]}},"alternative-id":["10.1145\/3636341.3636356"],"URL":"https:\/\/doi.org\/10.1145\/3636341.3636356","relation":{},"ISSN":["0163-5840"],"issn-type":[{"type":"print","value":"0163-5840"}],"subject":[],"published":{"date-parts":[[2023,6]]},"assertion":[{"value":"2023-12-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}