{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T15:47:38Z","timestamp":1769096858215,"version":"3.49.0"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2019,2,14]],"date-time":"2019-02-14T00:00:00Z","timestamp":1550102400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"MASWeb","award":["FAPEMIG\/PRONEX APQ-01400-14"],"award-info":[{"award-number":["FAPEMIG\/PRONEX APQ-01400-14"]}]},{"name":"projects InWeb","award":["MCT\/CNPq 573871\/2008-6"],"award-info":[{"award-number":["MCT\/CNPq 573871\/2008-6"]}]},{"name":"CNPq, CAPES, FAPEMIG, FAPEGO, and IFG"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2019,4,30]]},"abstract":"<jats:p>\n            Learning to Rank (L2R) is one of the main research lines in Information Retrieval.\n            <jats:italic>Risk-sensitive L2R<\/jats:italic>\n            is a sub-area of L2R that tries to learn models that are good on average while at the same time reducing the risk of performing poorly in a few but important queries (e.g., medical or legal queries). One way of reducing risk in learned models is by selecting and removing noisy, redundant features, or features that promote some queries to the detriment of others. This is exacerbated by learning methods that usually maximize an average metric (e.g., mean average precision (MAP) or Normalized Discounted Cumulative Gain (NDCG)). However, historically,\n            <jats:italic>feature selection<\/jats:italic>\n            (FS) methods have focused only on effectiveness and feature reduction as the main objectives. Accordingly, in this work, we propose to evaluate FS for L2R with an additional objective in mind, namely\n            <jats:bold>risk-sensitiveness<\/jats:bold>\n            . We present novel single and multi-objective criteria to optimize feature reduction, effectiveness, and risk-sensitiveness, all at the same time. We also introduce a new methodology to explore the search space, suggesting effective and efficient extensions of a well-known Evolutionary Algorithm (SPEA2) for FS applied to L2R. Our experiments show that explicitly including risk as an objective criterion is crucial to achieving a more effective and risk-sensitive performance. We also provide a thorough analysis of our methodology and experimental results.\n          <\/jats:p>","DOI":"10.1145\/3300196","type":"journal-article","created":{"date-parts":[[2019,2,14]],"date-time":"2019-02-14T19:36:17Z","timestamp":1550172977000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Risk-Sensitive Learning to Rank with Evolutionary Multi-Objective Feature Selection"],"prefix":"10.1145","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9426-9988","authenticated-orcid":false,"given":"Daniel Xavier","family":"Sousa","sequence":"first","affiliation":[{"name":"Federal University of Minas Gerais, Minas Gerais, Brazil"}]},{"given":"S\u00e9rgio","family":"Canuto","sequence":"additional","affiliation":[{"name":"Federal University of Minas Gerais, Minas Gerais, Brazil"}]},{"given":"Marcos Andr\u00e9","family":"Gon\u00e7alves","sequence":"additional","affiliation":[{"name":"Federal University of Minas Gerais, Minas Gerais, Brazil"}]},{"given":"Thierson Couto","family":"Rosa","sequence":"additional","affiliation":[{"name":"Federal University of Goi\u00e1s, Goi\u00e2nia, Goi\u00e1s, Brazil"}]},{"given":"Wellington Santos","family":"Martins","sequence":"additional","affiliation":[{"name":"Federal University of Goi\u00e1s, Goi\u00e2nia, Goi\u00e1s, Brazil"}]}],"member":"320","published-online":{"date-parts":[[2019,2,14]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2007.900837"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the 6th Italian Information Retrieval Workshop. 1--8.","author":"Capannini Gabriele","year":"2015","unstructured":"Gabriele Capannini , Domenico Dato , Claudio Lucchese , Monica Mori , Franco Maria Nardini , Salvatore Orlando , Raffaele Perego , and Nicola Tonellotto . 2015 . QuickRank: A C++ suite of learning to rank algorithms . In Proceedings of the 6th Italian Information Retrieval Workshop. 1--8. Gabriele Capannini, Domenico Dato, Claudio Lucchese, Monica Mori, Franco Maria Nardini, Salvatore Orlando, Raffaele Perego, and Nicola Tonellotto. 2015. QuickRank: A C++ suite of learning to rank algorithms. In Proceedings of the 6th Italian Information Retrieval Workshop. 1--8."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2016.05.004"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the 2010 International Conference on Yahoo&excl; Learning to Rank Challenge. 129--136","author":"Chapelle Oliver","year":"2011","unstructured":"Oliver Chapelle , Chang Yi , and Tie-Yan Liu . 2011 . Future directions in learning to rank . In Proceedings of the 2010 International Conference on Yahoo&excl; Learning to Rank Challenge. 129--136 . Oliver Chapelle, Chang Yi, and Tie-Yan Liu. 2011. Future directions in learning to rank. In Proceedings of the 2010 International Conference on Yahoo&excl; Learning to Rank Challenge. 129--136."},{"key":"e_1_2_1_6_1","volume-title":"TREC 2013 web track overview. In Proceedings of the 22nd Text REtrieval Conference. 1--15","author":"Collins-Thompson Kevyn","unstructured":"Kevyn Collins-Thompson , Craig Macdonald , Paul Bennett , Fernando Diaz , and E.M. Voorhees . 2014 . TREC 2013 web track overview. In Proceedings of the 22nd Text REtrieval Conference. 1--15 . Kevyn Collins-Thompson, Craig Macdonald, Paul Bennett, Fernando Diaz, and E.M. Voorhees. 2014. TREC 2013 web track overview. In Proceedings of the 22nd Text REtrieval Conference. 1--15."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/2740769.2740804"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 18th International Conference on Machine Learning. 74--81","author":"Das Sanmay","year":"2001","unstructured":"Sanmay Das . 2001 . Filters, wrappers and a boosting-based hybrid for feature selection . In Proceedings of the 18th International Conference on Machine Learning. 74--81 . Sanmay Das. 2001. Filters, wrappers and a boosting-based hybrid for feature selection. In Proceedings of the 18th International Conference on Machine Learning. 74--81."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/4235.996017"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2600428.2609625"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911511"},{"key":"e_1_2_1_12_1","volume-title":"Proceeding of the 36th European Conference on Information Retrieval. 26--38","author":"Din\u00e7er B. Taner","year":"2014","unstructured":"B. Taner Din\u00e7er , Iadh Ounis , and Craig Macdonald . 2014 b. Tackling biased baselines in the risk-sensitive evaluation of retrieval systems . In Proceeding of the 36th European Conference on Information Retrieval. 26--38 . B. Taner Din\u00e7er, Iadh Ounis, and Craig Macdonald. 2014b. Tackling biased baselines in the risk-sensitive evaluation of retrieval systems. In Proceeding of the 36th European Conference on Information Retrieval. 26--38."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277811"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2970398.2970433"},{"key":"e_1_2_1_15_1","first-page":"57","article-title":"Is learning to rank worth it? A statistical analysis of learning to rank methods in the LETOR benchmarks","volume":"1","author":"Gomes Guilherme","year":"2013","unstructured":"Guilherme Gomes , Vitor Oliveira , Jussara Almeida , and Marcos Gon\u00e7alves . 2013 . Is learning to rank worth it? A statistical analysis of learning to rank methods in the LETOR benchmarks . J. Inf. Data Manage. 1 , 1 (2013), 57 -- 66 . Guilherme Gomes, Vitor Oliveira, Jussara Almeida, and Marcos Gon\u00e7alves. 2013. Is learning to rank worth it? A statistical analysis of learning to rank methods in the LETOR benchmarks. J. Inf. Data Manage. 1, 1 (2013), 57--66.","journal-title":"J. Inf. Data Manage."},{"key":"e_1_2_1_16_1","first-page":"1","article-title":"An encoding technique for multiobjective evolutionary algorithms applied to power distribution system reconfiguration. Sci","volume":"1","author":"Guardado J.","year":"2014","unstructured":"J. Guardado , F. Rivas-Davalos , J. Torres , S. Maximov , and E. Melgoza . 2014 . An encoding technique for multiobjective evolutionary algorithms applied to power distribution system reconfiguration. Sci . World J. 2014, 1 (2014), 1 -- 10 . J. Guardado, F. Rivas-Davalos, J. Torres, S. Maximov, and E. Melgoza. 2014. An encoding technique for multiobjective evolutionary algorithms applied to power distribution system reconfiguration. Sci. World J. 2014, 1 (2014), 1--10.","journal-title":"World"},{"key":"e_1_2_1_17_1","volume-title":"The Elements of Statistical Learning","author":"Hastie Trevor","unstructured":"Trevor Hastie , Robert Tibshirani , and Jerome Friedman . 2009. The Elements of Statistical Learning . Springer , New York, NY . 83--85 pages. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2009. The Elements of Statistical Learning. Springer, New York, NY. 83--85 pages."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2007.904035"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775067"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-011-9118-4"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2013.2247628"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2013.2286696"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/647889.739241"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2792984"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1066677.1066918"},{"key":"e_1_2_1_26_1","volume-title":"Learning to Rank for Information Retrieval","author":"Liu Tie-Yan","unstructured":"Tie-Yan Liu . 2011. Learning to Rank for Information Retrieval . Springer , New York . Tie-Yan Liu. 2011. Learning to Rank for Information Retrieval. Springer, New York."},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the 2010 International Conference on Yahoo&excl; Learning to Rank Challenge. 77--89","author":"Mohan Ananth","year":"2011","unstructured":"Ananth Mohan , Zheng Chen , and Kq Weinberger . 2011 . Web-search ranking with initialized gradient boosted regression trees . In Proceedings of the 2010 International Conference on Yahoo&excl; Learning to Rank Challenge. 77--89 . Ananth Mohan, Zheng Chen, and Kq Weinberger. 2011. Web-search ranking with initialized gradient boosted regression trees. In Proceedings of the 2010 International Conference on Yahoo&excl; Learning to Rank Challenge. 77--89."},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 36th European Conference on Information Retrieval. 455--461","author":"Naini Kaweh Djafari","year":"2014","unstructured":"Kaweh Djafari Naini and Ismail Sengor Altingovde . 2014 . Exploiting result diversification methods for feature selection in learning to rank . In Proceedings of the 36th European Conference on Information Retrieval. 455--461 . Kaweh Djafari Naini and Ismail Sengor Altingovde. 2014. Exploiting result diversification methods for feature selection in learning to rank. In Proceedings of the 36th European Conference on Information Retrieval. 455--461."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/CIT.2011.16"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2005.159"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2011.181"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2641383.2641385"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767738"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1864708.1864764"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/RIOS.2015.7270735"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983323.2983792"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.294849"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1002\/atr.5670310106"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2348283.2348385"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2015.2453952"},{"key":"e_1_2_1_41_1","first-page":"512","article-title":"Pruning and ranking the Pareto optimal set, application for the dynamic multi-objective network design problem","volume":"48","author":"Wismans Luc","year":"2011","unstructured":"Luc Wismans , T. Brands , Berkum Erik , and Michiel Bliemer . 2011 . Pruning and ranking the Pareto optimal set, application for the dynamic multi-objective network design problem . J. Adv. Transp. 48 , 6 (2011), 512 -- 525 . Luc Wismans, T. Brands, Berkum Erik, and Michiel Bliemer. 2011. Pruning and ranking the Pareto optimal set, application for the dynamic multi-objective network design problem. J. Adv. Transp. 48, 6 (2011), 512--525.","journal-title":"J. Adv. Transp."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661829.2661934"},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of Evolutionary Methods for Design, Optimization and Control with Applications to Industrial Problems. 12--19","author":"Zitzler Eckart","year":"2001","unstructured":"Eckart Zitzler , Marco Laumanns , and Lothar Thiele . 2001 . SPEA2: Improving the strength Pareto evolutionary algorithm . In Proceedings of Evolutionary Methods for Design, Optimization and Control with Applications to Industrial Problems. 12--19 . Eckart Zitzler, Marco Laumanns, and Lothar Thiele. 2001. SPEA2: Improving the strength Pareto evolutionary algorithm. In Proceedings of Evolutionary Methods for Design, Optimization and Control with Applications to Industrial Problems. 12--19."}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3300196","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3300196","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:25:23Z","timestamp":1750206323000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3300196"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,2,14]]},"references-count":43,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2019,4,30]]}},"alternative-id":["10.1145\/3300196"],"URL":"https:\/\/doi.org\/10.1145\/3300196","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,2,14]]},"assertion":[{"value":"2018-02-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-12-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-02-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}