{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T05:14:51Z","timestamp":1784178891317,"version":"3.55.0"},"reference-count":70,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council Canada","doi-asserted-by":"crossref","award":["RGPIN-2024-04733"],"award-info":[{"award-number":["RGPIN-2024-04733"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2026,2,28]]},"abstract":"<jats:p>\n                    The varying performance of information retrieval (IR) methods, including state-of-the-art transformer-based neural retrievers, across diverse queries poses a significant challenge for achieving robust and reliable retrieval effectiveness.\n                    <jats:bold>Query Performance Prediction (QPP)<\/jats:bold>\n                    seeks to estimate the effectiveness of a retrieval method for individual queries, enabling adaptive strategies to improve retrieval outcomes, particularly for challenging queries. However, existing QPP approaches face fundamental challenges: pre-retrieval methods often rely on surface-level query features that fail to capture the nuanced relationship between queries and retrieval effectiveness, while post-retrieval methods depend heavily on the quality of retrieved documents, which can be unreliable for difficult queries. To this end, we propose the\n                    <jats:italic toggle=\"yes\">\n                      Query Space Distance-Based QPP (\n                      <jats:monospace>QSD-QPP<\/jats:monospace>\n                      )\n                    <\/jats:italic>\n                    framework, which leverages the deterministic and consistent behavior of retrieval methods to estimate query performance by referencing historical queries with known effectiveness. The approach is motivated by the observation that semantically or syntactically similar queries often exhibit consistent retrieval performance, a property that can be exploited to make reliable predictions for unseen queries.\n                    <jats:monospace>QSD-QPP<\/jats:monospace>\n                    operates in two modes: (1) a\n                    <jats:italic toggle=\"yes\">lightweight pre-retrieval<\/jats:italic>\n                    instantiation that dynamically constructs a query subspace based on embedding distances to interpolate the performance of proximate historical queries, and (2) an\n                    <jats:italic toggle=\"yes\">enriched post-retrieval<\/jats:italic>\n                    instantiation that incorporates contextualized embeddings, document interactions, and historical query associations to enhance prediction accuracy. By utilizing large-scale contextualized embeddings derived from pre-trained language models,\n                    <jats:monospace>QSD-QPP<\/jats:monospace>\n                    efficiently identifies semantically similar queries and leverages their performance for robust predictions. By addressing the inherent limitations of prior approaches,\n                    <jats:monospace>QSD-QPP<\/jats:monospace>\n                    achieves a balanced trade-off between computational efficiency, prediction accuracy, and scalability. We evaluate\n                    <jats:monospace>QSD-QPP<\/jats:monospace>\n                    on four benchmark datasets, including MS MARCO Dev and TREC Deep Learning tracks (2019, 2020, and DL-Hard), demonstrating its superior accuracy and robustness compared to state-of-the-art baselines in both pre-retrieval and post-retrieval QPP tasks. To ensure reproducibility and encourage further research, we publicly release the implementation of our work.\n                  <\/jats:p>","DOI":"10.1145\/3762197","type":"journal-article","created":{"date-parts":[[2025,9,12]],"date-time":"2025-09-12T16:36:05Z","timestamp":1757694965000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Query Performance Prediction Using Neural Query Space\u00a0Proximity"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-8977-9312","authenticated-orcid":false,"given":"Amin","family":"Bigdeli","sequence":"first","affiliation":[{"name":"University of Waterloo, Waterloo, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-1630-3938","authenticated-orcid":false,"given":"Sajad","family":"Ebrahimi","sequence":"additional","affiliation":[{"name":"University of Guelph, Guelph, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4411-7089","authenticated-orcid":false,"given":"Negar","family":"Arabzadeh","sequence":"additional","affiliation":[{"name":"University of California, Berkeley, California, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-3676-6023","authenticated-orcid":false,"given":"Sara","family":"Salamat","sequence":"additional","affiliation":[{"name":"Toronto Metropolitan University, Toronto, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-1467-1659","authenticated-orcid":false,"given":"Shirin","family":"Seyedsalehi","sequence":"additional","affiliation":[{"name":"Toronto Metropolitan University, Toronto, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8631-8872","authenticated-orcid":false,"given":"Maryam","family":"Khodabakhsh","sequence":"additional","affiliation":[{"name":"Shahrood University of Technology, Shahroud, Iran"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2102-9190","authenticated-orcid":false,"given":"Fattane","family":"Zarrinkalam","sequence":"additional","affiliation":[{"name":"University of Guelph, Guelph, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5148-6237","authenticated-orcid":false,"given":"Ebrahim","family":"Bagheri","sequence":"additional","affiliation":[{"name":"University of Toronto, Toronto, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,11,18]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.367"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3583780.3615270"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3482063"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3482011"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3511808.3557719"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2020.102248"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3358152"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-45442-5_10"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-02272-2"},{"key":"e_1_3_2_11_2","unstructured":"Yihua Chen Eric K. Garcia Maya R. Gupta Ali Rahimi and Luca Cazzanti. 2009. Similarity-based classification: Concepts and algorithms. Journal of Machine Learning Research 10 (2009) 747\u2013776. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:8559164"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022664626993"},{"key":"e_1_3_2_13_2","doi-asserted-by":"crossref","unstructured":"Nick Craswell Bhaskar Mitra Emine Yilmaz and Daniel Campos. 2021a. Overview of the TREC 2020 deep learning track. arXiv:2102.07662. Retrieved from https:\/\/arxiv.org\/abs\/2102.07662","DOI":"10.6028\/NIST.SP.1266.deep-overview"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462804"},{"key":"e_1_3_2_15_2","doi-asserted-by":"crossref","unstructured":"Nick Craswell Bhaskar Mitra Emine Yilmaz Daniel Campos and Ellen M Voorhees. 2020. Overview of the TREC 2019 deep learning track. arXiv:2003.07820. Retrieved from https:\/\/arxiv.org\/abs\/2003.07820","DOI":"10.6028\/NIST.SP.1266.deep-overview"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/564376.564429"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2010063"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3488560.3498491"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3545112"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531821"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531821"},{"key":"e_1_3_2_22_2","unstructured":"Jacob Devlin Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. Retrieved from https:\/\/arxiv.org\/abs\/1810.04805"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-56066-8_6"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591625"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-28244-7_15"},{"key":"e_1_3_2_26_2","volume-title":"International Conference on Machine Learning","author":"Goldman Sally A.","year":"2000","unstructured":"Sally A. Goldman and Yan Zhou. 2000. Enhancing Supervised Learning with Unlabeled Data. In Proceedings of the International Conference on Machine Learning. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:1215747"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","unstructured":"Jiafeng Guo Yixing Fan Liang Pang Liu Yang Qingyao Ai Hamed Zamani Chen Wu W. Bruce Croft and Xueqi Cheng. 2020. A deep look into neural ranking models for information retrieval. Information Processing & Management 57 6 (2020) 102067. DOI: 10.1016\/J.IPM.2019.102067","DOI":"10.1016\/J.IPM.2019.102067"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341981.3344249"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/1842890.1842906"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30213-1_5"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2005.11.003"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-78646-7_80"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2011.01.019"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.57"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TBDATA.2019.2921572"},{"key":"e_1_3_2_36_2","unstructured":"Urvashi Khandelwal Omer Levy Dan Jurafsky Luke Zettlemoyer and Mike Lewis. 2019. Generalization through memorization: Nearest neighbor language models. arXiv:1911.00172. Retrieved from https:\/\/arxiv.org\/abs\/1911.00172"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","unstructured":"Maryam Khodabakhsh and Ebrahim Bagheri. 2021. Semantics-enabled query performance prediction for ad hoc table retrieval. Information Processing & Management 58 1 (2021) 102399. DOI: 10.1016\/J.IPM.2020.102399","DOI":"10.1016\/J.IPM.2020.102399"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2023.119015"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-56063-7_27"},{"issue":"2","key":"e_1_3_2_40_2","first-page":"1483","article-title":"Sotiris Kotsiantis, and Omiros Ragos. 2018. Semi-supervised regression: A recent review","volume":"35","author":"Kostopoulos Georgios","year":"2018","unstructured":"Georgios Kostopoulos, Stamatis Karlos, Sotiris Kotsiantis, and Omiros Ragos. 2018. Semi-supervised regression: A recent review. Journal of Intelligent & Fuzzy Systems 35, 2 (2018), 1483\u20131500.","journal-title":"Journal of Intelligent & Fuzzy Systems"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/243199.243266"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-024-05686-6"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463262"},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","unstructured":"Chuan Meng Negar Arabzadeh Mohammad Aliannejadi and Maarten de Rijke. 2023. Query performance prediction: From ad-hoc to conversational search. arXiv:2305.10923. Retrieved from https:\/\/arxiv.org\/abs\/2305.10923","DOI":"10.1145\/3539618.3591919"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3736402"},{"key":"e_1_3_2_46_2","volume-title":"Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS \u201816)","author":"Nguyen Tri","year":"2016","unstructured":"Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated machine reading comprehension dataset. In Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS \u201816) (CEUR Workshop Proceedings, Vol. 1773). Tarek Richard Besold, Antoine Bordes, Artur S. d\u2019Avila Garcez, and Greg Wayne (Eds.), CEUR-WS.org. Retrieved from https:\/\/ceur-ws.org\/Vol-1773\/CoCoNIPS_2016_paper9.pdf"},{"key":"e_1_3_2_47_2","volume-title":"CoCo@ NIPS","author":"Nguyen Tri","year":"2016","unstructured":"Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated machine reading comprehension dataset. In CoCo@ NIPS."},{"key":"e_1_3_2_48_2","unstructured":"Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage re-ranking with BERT. arXiv:1901.04085. Retrieved from https:\/\/arxiv.org\/abs\/1901.04085"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2013.10.021"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-99736-6_27"},{"key":"e_1_3_2_51_2","volume-title":"TREC","author":"Plachouras Vassilis","year":"2004","unstructured":"Vassilis Plachouras, Ben He, and Iadh Ounis. 2004. University of glasgow at TREC 2004: Experiments in web, robust, and terabyte tracks with Terrier. In TREC."},{"key":"e_1_3_2_52_2","unstructured":"Ronak Pradeep Rodrigo Nogueira and Jimmy Lin. 2021. The expando-mono-duo design pattern for text ranking with pretrained sequence-to-sequence models. arXiv:2101.05667 [cs.IR]. Retrieved from https:\/\/arxiv.org\/abs\/2101.05667"},{"issue":"232","key":"e_1_3_2_53_2","first-page":"1","article-title":"A cluster elastic net for multivariate regression","volume":"18","author":"Price Bradley S.","year":"2018","unstructured":"Bradley S. Price and Ben Sherwood. 2018. A cluster elastic net for multivariate regression. Journal of Machine Learning Research 18, 232 (2018), 1\u201339.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_54_2","doi-asserted-by":"crossref","unstructured":"Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv:1908.10084. Retrieved from https:\/\/arxiv.org\/abs\/1908.10084","DOI":"10.18653\/v1\/D19-1410"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","unstructured":"Stephen E. Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends\u00ae in Information Retrieval 3 4 (2009) 333\u2013389. DOI: 10.1561\/1500000019","DOI":"10.1561\/1500000019"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341981.3344219"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3121050.3121087"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2018.10.009"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23072"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/1835449.1835494"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/2180868.2180873"},{"key":"e_1_3_2_62_2","first-page":"2","article-title":"Unsupervised query performance prediction for neural models utilising pairwise rank preferences","volume":"1","author":"Singh Ashutosh","year":"2023","unstructured":"Ashutosh Singh, Debasis Ganguly, Suchana Datta, and Craig Macdonald. 2023. Unsupervised query performance prediction for neural models utilising pairwise rank preferences. def 1 (2023), 2.","journal-title":"def"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-019-09682-y"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/2661829.2661906"},{"key":"e_1_3_2_65_2","article-title":"Distance metric learning with application to clustering with side-information","author":"Xing Eric P.","year":"2002","unstructured":"Eric P. Xing, A. Ng, Michael I. Jordan, and Stuart J. Russell. 2002. Distance metric learning with application to clustering with side-information. In Neural Information Processing Systems. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:2643381","journal-title":"Neural Information Processing Systems"},{"key":"e_1_3_2_66_2","volume-title":"9th International Conference on Learning Representations (ICLR \u201921)","author":"Xiong Lee","year":"2021","unstructured":"Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate nearest neighbor negative contrastive learning for dense text retrieval. In 9th International Conference on Learning Representations (ICLR \u201921). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=zeFrfgyZln"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591740"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210041"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-78646-7_8"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277835"},{"key":"e_1_3_2_71_2","article-title":"Introduction to semi-supervised learning","author":"Zhu Xiaojin","year":"2009","unstructured":"Xiaojin Zhu and Andrew B. Goldberg. 2009. Introduction to semi-supervised learning. In Synthesis Lectures on Artificial Intelligence and Machine Learning. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:40097546","journal-title":"Synthesis Lectures on Artificial Intelligence and Machine Learning"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3762197","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T21:23:16Z","timestamp":1763500996000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3762197"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,18]]},"references-count":70,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,2,28]]}},"alternative-id":["10.1145\/3762197"],"URL":"https:\/\/doi.org\/10.1145\/3762197","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,18]]},"assertion":[{"value":"2024-12-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}