{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T06:54:14Z","timestamp":1771484054196,"version":"3.50.1"},"reference-count":57,"publisher":"Springer Science and Business Media LLC","issue":"1-2","license":[{"start":{"date-parts":[[2023,8,11]],"date-time":"2023-08-11T00:00:00Z","timestamp":1691712000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,11]],"date-time":"2023-08-11T00:00:00Z","timestamp":1691712000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002428","name":"Austrian Science Fund","doi-asserted-by":"publisher","award":["P31890"],"award-info":[{"award-number":["P31890"]}],"id":[{"id":"10.13039\/501100002428","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002428","name":"Austrian Science Fund","doi-asserted-by":"publisher","award":["J4581"],"award-info":[{"award-number":["J4581"]}],"id":[{"id":"10.13039\/501100002428","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Austrian Science Fund"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Soc Choice Welf"],"published-print":{"date-parts":[[2025,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Shortlisting is the task of reducing a long list of alternatives to a (smaller) set of best or most suitable alternatives. Shortlisting is often used in the nomination process of awards or in recommender systems to display featured objects. In this paper, we analyze shortlisting methods that are based on approval data, a common type of preferences. Furthermore, we assume that the size of the shortlist, i.e., the number of best or most suitable alternatives, is not fixed but determined by the shortlisting method. We axiomatically analyze established and new shortlisting methods and complement this analysis with an experimental evaluation based on synthetic and real-world data. Our results lead to recommendations which shortlisting methods to use, depending on the desired properties.<\/jats:p>","DOI":"10.1007\/s00355-023-01482-2","type":"journal-article","created":{"date-parts":[[2023,8,11]],"date-time":"2023-08-11T18:02:22Z","timestamp":1691776942000},"page":"97-142","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Approval-based shortlisting"],"prefix":"10.1007","volume":"64","author":[{"given":"Martin","family":"Lackner","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3288-7462","authenticated-orcid":false,"given":"Jan","family":"Maly","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,8,11]]},"reference":[{"key":"1482_CR1","unstructured":"Allouche T, Lang J, Yger F (2022a) Multi-winner approval voting goes epistemic. In: The 38th conference on uncertainty in artificial intelligence (UAI 2022)"},{"key":"1482_CR2","doi-asserted-by":"crossref","unstructured":"Allouche T, Lang J, Yger F (2022b) Truth-tracking via approval voting: size matters. In: Proceedings of the 36th conference on artificial intelligence (AAAI-2022), pp 4768\u20134775","DOI":"10.1609\/aaai.v36i5.20403"},{"issue":"1\u20132","key":"1482_CR3","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1023\/A:1018376614624","volume":"99","author":"JA Amegashie","year":"1999","unstructured":"Amegashie JA (1999) The design of rent-seeking competitions: committees, preliminary and final contests. Public Choice 99(1\u20132):63\u201376","journal-title":"Public Choice"},{"issue":"2","key":"1482_CR4","doi-asserted-by":"publisher","first-page":"461","DOI":"10.1007\/s00355-016-1019-3","volume":"48","author":"H Aziz","year":"2017","unstructured":"Aziz H, Brill M, Conitzer V et al (2017a) Justified representation in approval-based committee voting. Soc Choice Welfare 48(2):461\u2013485","journal-title":"Soc Choice Welfare"},{"key":"1482_CR5","doi-asserted-by":"crossref","unstructured":"Aziz H, Elkind E, Faliszewski P et\u00a0al (2017b) The Condorcet principle for multiwinner elections: from shortlisting to proportionality. In: Proceedings of the 26th international joint conference on artificial intelligence (IJCAI-2017), pp 84\u201390","DOI":"10.24963\/ijcai.2017\/13"},{"key":"1482_CR6","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1016\/j.geb.2015.10.005","volume":"101","author":"S Barber\u00e0","year":"2017","unstructured":"Barber\u00e0 S, Coelho D (2017) Balancing the power to appoint officers. Games Econom Behav 101:189\u2013203","journal-title":"Games Econom Behav"},{"issue":"1","key":"1482_CR7","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1111\/1756-2171.12401","volume":"53","author":"S Barber\u00e0","year":"2022","unstructured":"Barber\u00e0 S, Coelho D (2022) Compromising on compromise rules. Rand J Econ 53(1):95\u2013112","journal-title":"Rand J Econ"},{"issue":"498","key":"1482_CR8","doi-asserted-by":"publisher","first-page":"421","DOI":"10.1093\/mind\/fzv157","volume":"125","author":"L Bovens","year":"2016","unstructured":"Bovens L (2016) Selection under uncertainty: affirmative action at shortlisting stage. Mind 125(498):421\u2013437","journal-title":"Mind"},{"issue":"3","key":"1482_CR9","doi-asserted-by":"publisher","first-page":"831","DOI":"10.2307\/1955105","volume":"72","author":"SJ Brams","year":"1978","unstructured":"Brams SJ, Fishburn PC (1978) Approval voting. Am Polit Sci Rev 72(3):831\u2013847","journal-title":"Am Polit Sci Rev"},{"issue":"4","key":"1482_CR10","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1177\/0951629811429993","volume":"24","author":"SJ Brams","year":"2012","unstructured":"Brams SJ, Kilgour M (2012) Narrowing the field in elections: the next-two rule. J Theor Polit 24(4):507\u2013525","journal-title":"J Theor Polit"},{"key":"1482_CR11","doi-asserted-by":"crossref","unstructured":"Brams SJ, Kilgour M (2015) Satisfaction Approval Voting. In: Melnik R (ed) Mathematical and Computational Modeling, pp 273\u2013298","DOI":"10.1002\/9781118853887.ch11"},{"issue":"4","key":"1482_CR12","doi-asserted-by":"publisher","first-page":"685","DOI":"10.1007\/s00355-018-1167-8","volume":"52","author":"F Brandl","year":"2019","unstructured":"Brandl F, Peters D (2019) An axiomatic characterization of the Borda mean rule. Soc Choice Welfare 52(4):685\u2013707","journal-title":"Soc Choice Welfare"},{"key":"1482_CR13","doi-asserted-by":"crossref","unstructured":"Bredereck R, Kaczmarczyk A, Niedermeier R (2017) On coalitional manipulation for multiwinner elections: shortlisting. In: Proceedings of the 26th international joint conference on artificial intelligence (IJCAI-2017), pp 887\u2013893","DOI":"10.24963\/ijcai.2017\/123"},{"key":"1482_CR14","doi-asserted-by":"crossref","unstructured":"Bredereck R, Faliszewski P, Kaczmarczyk A et\u00a0al (2019) An experimental view on committees providing justified representation. In: Proceedings of the 28th international joint conference on artificial intelligence (IJCAI-2019), pp 109\u2013115","DOI":"10.24963\/ijcai.2019\/16"},{"key":"1482_CR15","unstructured":"BWAA B (2019) BBWAA election rules. https:\/\/baseballhall.org\/hall-of-famers\/rules\/bbwaa-rules-for-election. Accessed 12 November 2019"},{"key":"1482_CR16","doi-asserted-by":"publisher","first-page":"130698","DOI":"10.1109\/ACCESS.2021.3114099","volume":"9","author":"A Chhabra","year":"2021","unstructured":"Chhabra A, Masalkovaite K, Mohapatra P (2021) An overview of fairness in clustering. IEEE Access 9:130698\u2013130720","journal-title":"IEEE Access"},{"issue":"11","key":"1482_CR17","doi-asserted-by":"publisher","first-page":"3434","DOI":"10.1257\/aer.104.11.3434","volume":"104","author":"G de Clippel","year":"2014","unstructured":"de Clippel G, Eliaz K, Knight B (2014) On the selection of arbitrators. Am Econ Rev 104(11):3434\u20133458","journal-title":"Am Econ Rev"},{"key":"1482_CR18","unstructured":"Duddy C, Houy N, Lang J et\u00a0al (2014) Social dichotomy functions, extended abstract for presentation at the 2014 meeting of the Society for Social Choice and Welfare"},{"issue":"2","key":"1482_CR19","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1007\/s00355-015-0914-3","volume":"46","author":"C Duddy","year":"2016","unstructured":"Duddy C, Piggins A, Zwicker WS (2016) Aggregation of binary evaluations: a Borda-like approach. Soc Choice Welfare 46(2):301\u2013333","journal-title":"Soc Choice Welfare"},{"issue":"4","key":"1482_CR20","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1257\/mic.20130118","volume":"7","author":"R Dutta","year":"2015","unstructured":"Dutta R, Horan S (2015) Inferring rationales from choice: identification for rational shortlist methods. Am Econ J Microecon 7(4):179\u2013201","journal-title":"Am Econ J Microecon"},{"key":"1482_CR21","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1017\/CBO9781107446984.009","volume-title":"Handbook of computational social choice","author":"E Elkind","year":"2016","unstructured":"Elkind E, Slinko A (2016) Rationalizations of voting rules. In: Brandt F, Conitzer V, Endriss U et al (eds) Handbook of computational social choice, 1st edn. Cambridge University Press, Cambridge, pp 169\u2013196","edition":"1"},{"key":"1482_CR22","doi-asserted-by":"crossref","unstructured":"Elkind E, Faliszewski P, Laslier JF, et\u00a0al (2017a) What do multiwinner voting rules do? An experiment over the two-dimensional Euclidean domain. In: Proceedings of the 31st conference on artificial intelligence (AAAI-2017). AAAI Press, pp 494\u2013501","DOI":"10.1609\/aaai.v31i1.10612"},{"issue":"3","key":"1482_CR23","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1007\/s00355-017-1026-z","volume":"48","author":"E Elkind","year":"2017","unstructured":"Elkind E, Faliszewski P, Skowron P et al (2017b) Properties of multiwinner voting rules. Soc Choice Welfare 48(3):599\u2013632","journal-title":"Soc Choice Welfare"},{"key":"1482_CR24","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1017\/CBO9781107446984.018","volume-title":"Handbook of computational social choice","author":"U Endriss","year":"2016","unstructured":"Endriss U (2016) Judgment aggregation. In: Brandt F, Conitzer V, Endriss U et al (eds) Handbook of computational social choice, 1st edn. Cambridge University Press, Cambridge, pp 399\u2013426","edition":"1"},{"key":"1482_CR25","first-page":"14345","volume":"34","author":"S Esmaeili","year":"2021","unstructured":"Esmaeili S, Brubach B, Srinivasan A et al (2021) Fair clustering under a bounded cost. Adv Neural Inf Process Syst 34:14345\u201314357","journal-title":"Adv Neural Inf Process Syst"},{"key":"1482_CR26","unstructured":"Faliszewski P, Skowron P, Slinko A et\u00a0al (2017) Multiwinner voting: a new challenge for social choice theory. In: Endriss U (ed) Trends in computational social choice, chap 2. AI Access, pp 27\u201347"},{"key":"1482_CR27","unstructured":"Faliszewski P, Slinko A, Talmon N (2020) The complexity of multiwinner voting rules with variable number of winners. In: Proceedings of the 24th European conference on artificial intelligence (ECAI-2020)"},{"key":"1482_CR28","doi-asserted-by":"crossref","unstructured":"Fern\u00e1ndez LS, Elkind E, Lackner M et\u00a0al (2017) Proportional justified representation. In: Proceedings of the 31st conference on artificial intelligence (AAAI-2017). AAAI Press, pp 670\u2013676","DOI":"10.1609\/aaai.v31i1.10611"},{"key":"1482_CR29","doi-asserted-by":"crossref","unstructured":"Freeman R, Kahng A, Pennock DM (2020) Proportionality in approval-based elections with a variable number of winners. In: Proceedings of the 29th international joint conference on artificial intelligence (IJCAI-2020), pp 132\u2013138","DOI":"10.24963\/ijcai.2020\/19"},{"key":"1482_CR30","unstructured":"Gangl C, Lackner M, Maly J et\u00a0al (2019) Aggregating expert opinions in support of medical diagnostic decision-making. In: Knowledge representation for health care\/prohealth (KR4HC), pp 56\u201362"},{"key":"1482_CR31","doi-asserted-by":"publisher","first-page":"372","DOI":"10.1016\/j.jet.2016.01.002","volume":"162","author":"S Horan","year":"2016","unstructured":"Horan S (2016) A simple model of two-stage choice. J Econ Theory 162:372\u2013406","journal-title":"J Econ Theory"},{"key":"1482_CR32","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1007\/978-3-642-02839-7_6","volume-title":"Handbook on approval voting","author":"M Kilgour","year":"2010","unstructured":"Kilgour M (2010) Approval balloting for multi-winner elections. In: Laslier JF, Sanver R (eds) Handbook on approval voting. Springer, Berlin, pp 105\u2013124"},{"key":"1482_CR33","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1007\/s11238-016-9535-2","volume":"81","author":"M Kilgour","year":"2016","unstructured":"Kilgour M (2016) Approval elections with a variable number of winners. Theory Decision 81:199\u2013211","journal-title":"Theory Decision"},{"key":"1482_CR34","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1007\/978-3-642-20441-8_12","volume-title":"Electoral systems","author":"M Kilgour","year":"2012","unstructured":"Kilgour M, Marshall E (2012) Approval balloting for fixed-size committees. Electoral systems. Springer, Berlin, pp 305\u2013326"},{"issue":"3","key":"1482_CR35","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1111\/1467-6419.00083","volume":"13","author":"P Klemperer","year":"1999","unstructured":"Klemperer P (1999) Auction theory: a guide to the literature. J Econ Surv 13(3):227\u2013286","journal-title":"J Econ Surv"},{"issue":"1","key":"1482_CR36","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1007\/s00199-016-1006-z","volume":"65","author":"C Kops","year":"2018","unstructured":"Kops C (2018) (F)lexicographic shortlist method. Econ Theor 65(1):79\u201397","journal-title":"Econ Theor"},{"key":"1482_CR37","unstructured":"Lackner M, Maly J (2021) Approval-based shortlisting. In: Proceedings of the 20th international conference on autonomous agents and multiagent systems (AAMAS-2021). IFAAMAS, pp 1566\u20131568"},{"key":"1482_CR38","doi-asserted-by":"crossref","unstructured":"Lackner M, Maly J (2022) Python code for \u201dapproval-based shortlisting\u201d. https:\/\/doi.org\/10.5281\/zenodo.3821983","DOI":"10.1007\/s00355-023-01482-2"},{"issue":"105","key":"1482_CR39","first-page":"173","volume":"192","author":"M Lackner","year":"2021","unstructured":"Lackner M, Skowron P (2021) Consistent approval-based multi-winner rules. J Econ Theory 192(105):173","journal-title":"J Econ Theory"},{"key":"1482_CR40","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-09016-5","volume-title":"Multi-winner voting with approval preferences","author":"M Lackner","year":"2023","unstructured":"Lackner M, Skowron P (2023) Multi-winner voting with approval preferences. Springer, Berlin"},{"issue":"1","key":"1482_CR41","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1007\/s11229-011-0025-3","volume":"187","author":"C List","year":"2012","unstructured":"List C (2012) The theory of judgment aggregation: an introductory review. Synthese 187(1):179\u2013207","journal-title":"Synthese"},{"issue":"5","key":"1482_CR42","doi-asserted-by":"publisher","first-page":"1824","DOI":"10.1257\/aer.97.5.1824","volume":"97","author":"P Manzini","year":"2007","unstructured":"Manzini P, Mariotti M (2007) Sequentially rationalizable choice. Am Econ Rev 97(5):1824\u20131839","journal-title":"Am Econ Rev"},{"key":"1482_CR43","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1016\/j.jmateco.2015.06.015","volume":"60","author":"M N\u00fa\u00f1ez","year":"2015","unstructured":"N\u00fa\u00f1ez M, Laslier JF (2015) Bargaining through approval. J Math Econ 60:63\u201373","journal-title":"J Math Econ"},{"key":"1482_CR44","unstructured":"Procaccia AD, Shah N (2015) Is approval voting optimal given approval votes? In: Advances in neural information processing systems, pp 1801\u20131809"},{"key":"1482_CR45","unstructured":"Quinn J, Schneier B (2016) A proportional voting system for awards nominations resistant to voting blocs. Preprint per https:\/\/www.schneier.com\/academic\/archives\/2016\/05\/a_proportional_votin.html. Accessed 14 November 2019"},{"key":"1482_CR46","doi-asserted-by":"crossref","unstructured":"Rey S, Endriss U, de\u00a0Haan R (2021) Shortlisting rules and incentives in an end-to-end model for participatory budgeting. In: Proceedings of the 30th international joint conference on artificial intelligence (IJCAI-2021), pp 370\u2013376","DOI":"10.24963\/ijcai.2021\/52"},{"key":"1482_CR47","unstructured":"S\u00e1nchez-Fern\u00e1ndez L, Fisteus JA (2019) Monotonicity axioms in approval-based multi-winner voting rules. In: Proceedings of the 18th international conference on autonomous agents and multiagent systems (AAMAS-2019). International Foundation for Autonomous Agents and Multiagent Systems, pp 485\u2013493"},{"issue":"3","key":"1482_CR48","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3396863","volume":"8","author":"NB Shah","year":"2020","unstructured":"Shah NB, Zhou D (2020) Approval voting and incentives in crowdsourcing. ACM Trans Econ Comput (TEAC) 8(3):1\u201340","journal-title":"ACM Trans Econ Comput (TEAC)"},{"key":"1482_CR49","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781107298019","volume-title":"Understanding machine learning: from theory to algorithms","author":"S Shalev-Shwartz","year":"2014","unstructured":"Shalev-Shwartz S, Ben-David S (2014) Understanding machine learning: from theory to algorithms. Cambridge University Press, Cambridge"},{"key":"1482_CR50","volume-title":"Multiagent systems\u2014algorithmic, game-theoretic, and logical foundations","author":"Y Shoham","year":"2009","unstructured":"Shoham Y, Leyton-Brown K (2009) Multiagent systems\u2014algorithmic, game-theoretic, and logical foundations. Cambridge University Press, Cambridge"},{"key":"1482_CR51","doi-asserted-by":"crossref","unstructured":"Singh A, Rose C, Visweswariah K et\u00a0al (2010) Prospect: a system for screening candidates for recruitment. In: Proceedings of the 19th ACM international conference on information and knowledge management. ACM, pp 659\u2013668","DOI":"10.1145\/1871437.1871523"},{"key":"1482_CR52","unstructured":"The Hugo Awards (2019) The voting system. http:\/\/www.thehugoawards.org\/the-voting-system. Accessed 12 November 2019"},{"key":"1482_CR53","unstructured":"The Man Booker Prize (2018) Rules & entry form. https:\/\/thebookerprizes.com\/sites\/manbosamjo\/files\/uploadedfiles\/files\/ManBookerPrize2018RulesAndEntryForm.pdf. Accessed 13 November 2019"},{"issue":"3","key":"1482_CR54","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1007\/BF00433944","volume":"4","author":"TN Tideman","year":"1987","unstructured":"Tideman TN (1987) Independence of clones as a criterion for voting rules. Soc Choice Welfare 4(3):185\u2013206","journal-title":"Soc Choice Welfare"},{"issue":"5","key":"1482_CR55","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1016\/0950-4230(92)80030-C","volume":"5","author":"HM Tweeddale","year":"1992","unstructured":"Tweeddale HM, Cameron RF, Sylvester SS (1992) Some experiences in hazard identification and risk shortlisting. J Loss Prev Process Ind 5(5):279\u2013288","journal-title":"J Loss Prev Process Ind"},{"issue":"4","key":"1482_CR56","doi-asserted-by":"publisher","first-page":"941","DOI":"10.1007\/s00355-012-0704-0","volume":"41","author":"CJ Tyson","year":"2013","unstructured":"Tyson CJ (2013) Behavioral implications of shortlisting procedures. Soc Choice Welfare 41(4):941\u2013963","journal-title":"Soc Choice Welfare"},{"key":"1482_CR57","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1017\/CBO9781107446984.003","volume-title":"Handbook of computational social choice","author":"WS Zwicker","year":"2016","unstructured":"Zwicker WS, Moulin H (2016) Introduction to the theory of voting. In: Brandt F, Conitzer V, Endriss U et al (eds) Handbook of computational social choice. Cambridge University Press, Cambridge, pp 23\u201356"}],"container-title":["Social Choice and Welfare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00355-023-01482-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00355-023-01482-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00355-023-01482-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,27]],"date-time":"2025-01-27T15:04:22Z","timestamp":1737990262000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00355-023-01482-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,11]]},"references-count":57,"journal-issue":{"issue":"1-2","published-print":{"date-parts":[[2025,2]]}},"alternative-id":["1482"],"URL":"https:\/\/doi.org\/10.1007\/s00355-023-01482-2","relation":{},"ISSN":["0176-1714","1432-217X"],"issn-type":[{"value":"0176-1714","type":"print"},{"value":"1432-217X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,11]]},"assertion":[{"value":"1 May 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 July 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 August 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}