{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,24]],"date-time":"2025-08-24T01:10:15Z","timestamp":1755997815520,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":28,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,7,11]],"date-time":"2021-07-11T00:00:00Z","timestamp":1625961600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"BMBF","award":["01DD20003"],"award-info":[{"award-number":["01DD20003"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,7,11]]},"DOI":"10.1145\/3471158.3472241","type":"proceedings-article","created":{"date-parts":[[2021,8,31]],"date-time":"2021-08-31T20:38:17Z","timestamp":1630442297000},"page":"203-210","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Extracting per Query Valid Explanations for Blackbox Learning-to-Rank Models"],"prefix":"10.1145","author":[{"given":"Jaspreet","family":"Singh","sequence":"first","affiliation":[{"name":"Amazon, Berlin, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Megha","family":"Khosla","sequence":"additional","affiliation":[{"name":"L3S Research Center, Hannover, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wang","family":"Zhenye","sequence":"additional","affiliation":[{"name":"L3S Research Center, Hannover, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Avishek","family":"Anand","sequence":"additional","affiliation":[{"name":"L3S Research Center, Hannover, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,8,31]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"Ahmed M Alaa and Mihaela van der Schaar. 2019. Demystifying Black-box Models with Symbolic Metamodels. In Advances in Neural Information Processing Systems. 11304--11314.  Ahmed M Alaa and Mihaela van der Schaar. 2019. Demystifying Black-box Models with Symbolic Metamodels. In Advances in Neural Information Processing Systems. 11304--11314."},{"key":"e_1_3_2_1_2_1","unstructured":"Isabelle Alvarez. 2004. Explaining the result of a decision tree to the end-user. In ECAI.  Isabelle Alvarez. 2004. Explaining the result of a decision tree to the end-user. In ECAI."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"crossref","unstructured":"David Alvarez-Melis and Tommi Jaakkola. 2017. A causal framework for explaining the predictions of black-box sequence-to-sequence models. In EMNLP. ACL.  David Alvarez-Melis and Tommi Jaakkola. 2017. A causal framework for explaining the predictions of black-box sequence-to-sequence models. In EMNLP. ACL.","DOI":"10.18653\/v1\/D17-1042"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"crossref","unstructured":"Chris Burges Tal Shaked Erin Renshaw Ari Lazier Matt Deeds Nicole Hamilton and Greg Hullender. 2005. Learning to rank using gradient descent. In ICML.  Chris Burges Tal Shaked Erin Renshaw Ari Lazier Matt Deeds Nicole Hamilton and Greg Hullender. 2005. Learning to rank using gradient descent. In ICML.","DOI":"10.1145\/1102351.1102363"},{"key":"e_1_3_2_1_5_1","volume-title":"Proceedings of Machine Learning Research (Proceedings of Machine Learning Research), Kamalika Chaudhuri and Masashi Sugiyama (Eds.)","volume":"89","author":"Carter Brandon","year":"2019","unstructured":"Brandon Carter , Jonas Mueller , Siddhartha Jain , and David Gifford . 2019 . What made you do this? Understanding black-box decisions with sufficient input subsets . In Proceedings of Machine Learning Research (Proceedings of Machine Learning Research), Kamalika Chaudhuri and Masashi Sugiyama (Eds.) , Vol. 89 . 567--576. http:\/\/proceedings.mlr.press\/v89\/carter19a.html Brandon Carter, Jonas Mueller, Siddhartha Jain, and David Gifford. 2019. What made you do this? Understanding black-box decisions with sufficient input subsets. In Proceedings of Machine Learning Research (Proceedings of Machine Learning Research), Kamalika Chaudhuri and Masashi Sugiyama (Eds.), Vol. 89. 567--576. http:\/\/proceedings.mlr.press\/v89\/carter19a.html"},{"key":"e_1_3_2_1_6_1","unstructured":"Abhimanyu Das and David Kempe. 2011. Submodular Meets Spectral: Greedy Algorithms for Subset Selection Sparse Approximation and Dictionary Selection. In ICML.  Abhimanyu Das and David Kempe. 2011. Submodular Meets Spectral: Greedy Algorithms for Subset Selection Sparse Approximation and Dictionary Selection. In ICML."},{"key":"e_1_3_2_1_7_1","volume-title":"Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace.","author":"DeYoung Jay","year":"2019","unstructured":"Jay DeYoung , Sarthak Jain , Nazneen Fatema Rajani , Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. 2019 . ERASER : A Benchmark to Evaluate Rationalized NLP Models . arXiv preprint arXiv:1911.03429 (2019). Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. 2019. ERASER: A Benchmark to Evaluate Rationalized NLP Models. arXiv preprint arXiv:1911.03429 (2019)."},{"key":"e_1_3_2_1_8_1","unstructured":"Ethan R. Elenberg Alexandros G. Dimakis Moran Feldman and Amin Karbasi. 2017. Streaming Weak Submodularity: Interpreting Neural Networks on the Fly. In NeurIPS.  Ethan R. Elenberg Alexandros G. Dimakis Moran Feldman and Amin Karbasi. 2017. Streaming Weak Submodularity: Interpreting Neural Networks on the Fly. In NeurIPS."},{"key":"e_1_3_2_1_9_1","volume-title":"An efficient boosting algorithm for combining preferences. Journal of machine learning research","author":"Freund Yoav","year":"2003","unstructured":"Yoav Freund , Raj Iyer , Robert E Schapire , and Yoram Singer . 2003. An efficient boosting algorithm for combining preferences. Journal of machine learning research ( 2003 ). Yoav Freund, Raj Iyer, Robert E Schapire, and Yoram Singer. 2003. An efficient boosting algorithm for combining preferences. Journal of machine learning research (2003)."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2019.2957223"},{"key":"e_1_3_2_1_11_1","volume-title":"A survey of methods for explaining black box models. ACM CSUR","author":"Guidotti Riccardo","year":"2018","unstructured":"Riccardo Guidotti , Anna Monreale , Salvatore Ruggieri , Franco Turini , Fosca Giannotti , and Dino Pedreschi . 2018. A survey of methods for explaining black box models. ACM CSUR ( 2018 ). Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM CSUR (2018)."},{"key":"e_1_3_2_1_12_1","volume-title":"Towards Benchmarking the Utility of Explanations for Model Debugging. arXiv preprint arXiv:2105.04505","author":"Idahl Maximilian","year":"2021","unstructured":"Maximilian Idahl , Lijun Lyu , Ujwal Gadiraju , and Avishek Anand . 2021. Towards Benchmarking the Utility of Explanations for Model Debugging. arXiv preprint arXiv:2105.04505 ( 2021 ). Maximilian Idahl, Lijun Lyu, Ujwal Gadiraju, and Avishek Anand. 2021. Towards Benchmarking the Utility of Explanations for Model Debugging. arXiv preprint arXiv:2105.04505 (2021)."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Isaac Lage Emily Chen Jeffrey He Menaka Narayanan Been Kim Samuel J Gershman and Finale Doshi-Velez. 2019. Human Evaluation of Models Built for Interpretability. In AAAI.  Isaac Lage Emily Chen Jeffrey He Menaka Narayanan Been Kim Samuel J Gershman and Finale Doshi-Velez. 2019. Human Evaluation of Models Built for Interpretability. In AAAI.","DOI":"10.1609\/hcomp.v7i1.5280"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"crossref","unstructured":"Benjamin Letham Cynthia Rudin Tyler H McCormick David Madigan etal 2015. Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics (2015).  Benjamin Letham Cynthia Rudin Tyler H McCormick David Madigan et al. 2015. Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics (2015).","DOI":"10.1214\/15-AOAS848"},{"key":"e_1_3_2_1_15_1","volume-title":"Feature selection: A data perspective. ACM Computing Surveys (CSUR)","author":"Li Jundong","year":"2017","unstructured":"Jundong Li , Kewei Cheng , Suhang Wang , Fred Morstatter , Robert P Trevino , Jiliang Tang , and Huan Liu . 2017. Feature selection: A data perspective. ACM Computing Surveys (CSUR) ( 2017 ). Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P Trevino, Jiliang Tang, and Huan Liu. 2017. Feature selection: A data perspective. ACM Computing Surveys (CSUR) (2017)."},{"key":"e_1_3_2_1_16_1","unstructured":"Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in NeurIPS.  Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in NeurIPS."},{"key":"e_1_3_2_1_17_1","volume-title":"Explaining nonlinear classification decisions with deep taylor decomposition. Pattern Recognition","author":"Montavon Gr\u00e9goire","year":"2017","unstructured":"Gr\u00e9goire Montavon , Sebastian Lapuschkin , Alexander Binder , Wojciech Samek , and Klaus-Robert M\u00fcller . 2017. Explaining nonlinear classification decisions with deep taylor decomposition. Pattern Recognition ( 2017 ). Gr\u00e9goire Montavon, Sebastian Lapuschkin, Alexander Binder, Wojciech Samek, and Klaus-Robert M\u00fcller. 2017. Explaining nonlinear classification decisions with deep taylor decomposition. Pattern Recognition (2017)."},{"key":"e_1_3_2_1_18_1","unstructured":"Tao Qin and Tie-Yan Liu. 2013. Introducing LETOR 4.0 Datasets. CoRR (2013).  Tao Qin and Tie-Yan Liu. 2013. Introducing LETOR 4.0 Datasets. CoRR (2013)."},{"volume-title":"Advances in Information Retrieval","author":"Rennings Dani\u00ebl","key":"e_1_3_2_1_19_1","unstructured":"Dani\u00ebl Rennings , Felipe Moraes , and Claudia Hauff . 2019. An Axiomatic Approach to Diagnosing Neural IR Models . In Advances in Information Retrieval , Leif Azzopardi, Benno Stein, Norbert Fuhr, Philipp Mayr, Claudia Hauff, and Djoerd Hiemstra (Eds.). Springer International Publishing , Cham , 489--503. Dani\u00ebl Rennings, Felipe Moraes, and Claudia Hauff. 2019. An Axiomatic Approach to Diagnosing Neural IR Models. In Advances in Information Retrieval, Leif Azzopardi, Benno Stein, Norbert Fuhr, Philipp Mayr, Claudia Hauff, and Djoerd Hiemstra (Eds.). Springer International Publishing, Cham, 489--503."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"crossref","unstructured":"Marco Tulio Ribeiro Sameer Singh and Carlos Guestrin. 2016. Why Should I Trust You?: Explaining the Predictions of Any Classifier. In ACM SIGKDD.  Marco Tulio Ribeiro Sameer Singh and Carlos Guestrin. 2016. Why Should I Trust You?: Explaining the Predictions of Any Classifier. In ACM SIGKDD.","DOI":"10.18653\/v1\/N16-3020"},{"key":"e_1_3_2_1_21_1","volume-title":"Anchors: High-precision model-agnostic explanations. In AAAI.","author":"Ribeiro Marco Tulio","year":"2018","unstructured":"Marco Tulio Ribeiro , Sameer Singh , and Carlos Guestrin . 2018 . Anchors: High-precision model-agnostic explanations. In AAAI. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High-precision model-agnostic explanations. In AAAI."},{"key":"e_1_3_2_1_22_1","volume-title":"Posthoc Interpretability of Learning to Rank Models using Secondary Training Data. arXiv preprint arXiv:1806.11330","author":"Singh Jaspreet","year":"2018","unstructured":"Jaspreet Singh and Avishek Anand . 2018. Posthoc Interpretability of Learning to Rank Models using Secondary Training Data. arXiv preprint arXiv:1806.11330 ( 2018 ). Jaspreet Singh and Avishek Anand. 2018. Posthoc Interpretability of Learning to Rank Models using Secondary Training Data. arXiv preprint arXiv:1806.11330 (2018)."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3290620"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3375234"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331377"},{"key":"e_1_3_2_1_26_1","volume-title":"Towards Axiomatic Explanations for Neural Ranking Models. arXiv preprint arXiv:2106.08019","author":"V\u00f6lske Michael","year":"2021","unstructured":"Michael V\u00f6lske , Alexander Bondarenko , Maik Fr\u00f6be , Matthias Hagen , Benno Stein , Jaspreet Singh , and Avishek Anand . 2021. Towards Axiomatic Explanations for Neural Ranking Models. arXiv preprint arXiv:2106.08019 ( 2021 ). Michael V\u00f6lske, Alexander Bondarenko, Maik Fr\u00f6be, Matthias Hagen, Benno Stein, Jaspreet Singh, and Avishek Anand. 2021. Towards Axiomatic Explanations for Neural Ranking Models. arXiv preprint arXiv:2106.08019 (2021)."},{"key":"e_1_3_2_1_27_1","unstructured":"Kelvin Xu Jimmy Ba Ryan Kiros Kyunghyun Cho Aaron Courville Ruslan Salakhudinov Rich Zemel and Yoshua Bengio. 2015. Show attend and tell: Neural image caption generation with visual attention. In ICML.  Kelvin Xu Jimmy Ba Ryan Kiros Kyunghyun Cho Aaron Courville Ruslan Salakhudinov Rich Zemel and Yoshua Bengio. 2015. Show attend and tell: Neural image caption generation with visual attention. In ICML."},{"volume-title":"International Conference on Learning Representations.","author":"Yoon Jinsung","key":"e_1_3_2_1_28_1","unstructured":"Jinsung Yoon , James Jordon , and Mihaela van der Schaar. 2018. INVASE: Instance-wise variable selection using neural networks . In International Conference on Learning Representations. Jinsung Yoon, James Jordon, and Mihaela van der Schaar. 2018. INVASE: Instance-wise variable selection using neural networks. In International Conference on Learning Representations."}],"event":{"name":"ICTIR '21: The 2021 ACM SIGIR International Conference on the Theory of Information Retrieval","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"],"location":"Virtual Event Canada","acronym":"ICTIR '21"},"container-title":["Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3471158.3472241","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3471158.3472241","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T17:49:29Z","timestamp":1750268969000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3471158.3472241"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,11]]},"references-count":28,"alternative-id":["10.1145\/3471158.3472241","10.1145\/3471158"],"URL":"https:\/\/doi.org\/10.1145\/3471158.3472241","relation":{},"subject":[],"published":{"date-parts":[[2021,7,11]]},"assertion":[{"value":"2021-08-31","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}