{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T04:30:56Z","timestamp":1778128256933,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":64,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,6,3]],"date-time":"2024-06-03T00:00:00Z","timestamp":1717372800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Federal Statistical Office of Germany"},{"name":"Munich Center for Machine Learning"},{"name":"Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry of Education and Research"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,6,3]]},"DOI":"10.1145\/3630106.3658974","type":"proceedings-article","created":{"date-parts":[[2024,6,5]],"date-time":"2024-06-05T13:14:21Z","timestamp":1717593261000},"page":"1305-1320","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["One Model Many Scores: Using Multiverse Analysis to Prevent Fairness Hacking and Evaluate the Influence of Model Design Decisions"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9406-7761","authenticated-orcid":false,"given":"Jan","family":"Simson","sequence":"first","affiliation":[{"name":"Institute of Statistics, LMU Munich, Germany and Munich Center for Machine Learning (MCML), Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8867-762X","authenticated-orcid":false,"given":"Florian","family":"Pfisterer","sequence":"additional","affiliation":[{"name":"Institute of Statistics, LMU Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7363-4299","authenticated-orcid":false,"given":"Christoph","family":"Kern","sequence":"additional","affiliation":[{"name":"Institute of Statistics, LMU Munich, Germany and Munich Center for Machine Learning (MCML), Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,6,5]]},"reference":[{"key":"e_1_3_2_3_1_1","unstructured":"Alekh Agarwal Alina Beygelzimer Miroslav Dud\u00edk John Langford and Hanna Wallach. 2018. A Reductions Approach to Fair Classification. (2018)."},{"key":"e_1_3_2_3_2_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2011.02407"},{"key":"e_1_3_2_3_3_1","volume-title":"International Conference on Machine Learning. PMLR, 161\u2013170","author":"A\u00efvodji Ulrich","year":"2019","unstructured":"Ulrich A\u00efvodji, Hiromi Arai, Olivier Fortineau, S\u00e9bastien Gambs, Satoshi Hara, and Alain Tapp. 2019. Fairwashing: the risk of rationalization. In International Conference on Machine Learning. PMLR, 161\u2013170."},{"key":"e_1_3_2_3_4_1","doi-asserted-by":"publisher","unstructured":"J.J. Allaire Charles Teague Carlos Scheidegger Yihui Xie and Christophe Dervieux. 2022. Quarto. https:\/\/doi.org\/10.5281\/zenodo.5960048 DOI: 10.5281\/zenodo.5960048.","DOI":"10.5281\/zenodo.5960048"},{"key":"e_1_3_2_3_5_1","volume-title":"Machine bias. ProPublica (05","author":"Angwin Julia","year":"2016","unstructured":"Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica (05 2016), 254\u2013264. https:\/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing"},{"key":"e_1_3_2_3_6_1","doi-asserted-by":"publisher","unstructured":"Michelle Bao Angela Zhou Samantha Zottola Brian Brubach Sarah Desmarais Aaron Horowitz Kristian Lum and Suresh Venkatasubramanian. 2022. It\u2019s COMPASlicated: The Messy Relationship between RAI Datasets and Algorithmic Fairness Benchmarks. (2022). https:\/\/doi.org\/10.48550\/arXiv.2106.05498","DOI":"10.48550\/arXiv.2106.05498"},{"key":"e_1_3_2_3_7_1","volume-title":"Fairness and Machine Learning: Limitations and Opportunities","author":"Barocas Solon","unstructured":"Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2023. Classification - No Fairness through Unawareness. In Fairness and Machine Learning: Limitations and Opportunities. The MIT Press, Cambridge, Massachusetts."},{"key":"e_1_3_2_3_8_1","doi-asserted-by":"publisher","unstructured":"Samuel\u00a0J. Bell Onno\u00a0P. Kampman Jesse Dodge and Neil\u00a0D. Lawrence. 2022. Modeling the Machine Learning Multiverse. (2022). https:\/\/doi.org\/10.48550\/arXiv.2206.05985","DOI":"10.48550\/arXiv.2206.05985"},{"key":"e_1_3_2_3_9_1","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1484"},{"key":"e_1_3_2_3_10_1","volume-title":"Model Multiplicity: Opportunities, Concerns, and Solutions.","author":"Black Emily","year":"2022","unstructured":"Emily Black, Manish Raghavan, and Solon Barocas. 2022. Model Multiplicity: Opportunities, Concerns, and Solutions. (2022)."},{"key":"e_1_3_2_3_11_1","doi-asserted-by":"publisher","unstructured":"Nate Breznau et al. 2022. Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty. Proceedings of the National Academy of Sciences 119 44 (11 2022) e2203150119. https:\/\/doi.org\/10.1073\/pnas.2203150119 Publisher: Proceedings of the National Academy of Sciences.","DOI":"10.1073\/pnas.2203150119"},{"key":"e_1_3_2_3_12_1","unstructured":"US\u00a0Census Bureau. 2021. ACS Health Insurance Coverage Recoding Programming Code. https:\/\/www.census.gov\/topics\/health\/health-insurance\/guidance\/programming-code\/acs-recoding.html Section: Government."},{"key":"e_1_3_2_3_13_1","unstructured":"US\u00a0Census Bureau. 2021. Understanding and using the American Community Survey public use microdata sample files: What data users need to know."},{"key":"e_1_3_2_3_14_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.13197"},{"key":"e_1_3_2_3_15_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.aac4716"},{"key":"e_1_3_2_3_16_1","unstructured":"nteract contributors. 2017. papermill: Parametrize and run Jupyter and nteract Notebooks. https:\/\/github.com\/nteract\/papermill"},{"key":"e_1_3_2_3_17_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i20.30203"},{"key":"e_1_3_2_3_18_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1958.tb00292.x"},{"key":"e_1_3_2_3_19_1","volume-title":"Retiring Adult: New Datasets for Fair Machine Learning.","author":"Ding Frances","year":"2021","unstructured":"Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. 2021. Retiring Adult: New Datasets for Fair Machine Learning. (2021), 13."},{"key":"e_1_3_2_3_20_1","volume-title":"Rethinking bias mitigation: Fairer architectures make for fairer face recognition. Advances in Neural Information Processing Systems 36","author":"Dooley Samuel","year":"2024","unstructured":"Samuel Dooley, Rhea Sukthanker, John Dickerson, Colin White, Frank Hutter, and Micah Goldblum. 2024. Rethinking bias mitigation: Fairer architectures make for fairer face recognition. Advances in Neural Information Processing Systems 36 (2024)."},{"key":"e_1_3_2_3_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-022-00854-z"},{"key":"e_1_3_2_3_22_1","unstructured":"Evanthia Faliagka Kostas Ramantas and Giannis Tzimas. 2012. Application of Machine Learning Algorithms to an online Recruitment System. (2012)."},{"key":"e_1_3_2_3_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-05318-5_1"},{"key":"e_1_3_2_3_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287589"},{"key":"e_1_3_2_3_25_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos"},{"key":"e_1_3_2_3_26_1","doi-asserted-by":"publisher","DOI":"10.1511\/2014.111.460"},{"key":"e_1_3_2_3_27_1","unstructured":"Moritz Hardt Eric Price Eric Price and Nati Srebro. 2016. Equality of Opportunity in Supervised Learning. (2016)."},{"key":"e_1_3_2_3_28_1","volume-title":"Robodebt: five years of lies, mistakes and failures that caused a $1.8bn scandal. The Guardian (03","author":"Henriques-Gomes Luke","year":"2023","unstructured":"Luke Henriques-Gomes. 2023. Robodebt: five years of lies, mistakes and failures that caused a $1.8bn scandal. The Guardian (03 2023). https:\/\/www.theguardian.com\/australia-news\/2023\/mar\/11\/robodebt-five-years-of-lies-mistakes-and-failures-that-caused-a-18bn-scandal"},{"key":"e_1_3_2_3_29_1","volume-title":"Vol.\u00a01","author":"Tin\u00a0Kam Ho.","unstructured":"Tin\u00a0Kam Ho. 1995. Random decision forests, Vol.\u00a01. IEEE, 278\u2013282."},{"key":"e_1_3_2_3_30_1","doi-asserted-by":"publisher","DOI":"10.1198\/106186007X237892"},{"key":"e_1_3_2_3_31_1","volume-title":"International Conference on Machine Learning. PMLR, 754\u2013762","author":"Hutter Frank","year":"2014","unstructured":"Frank Hutter, Holger Hoos, and Kevin Leyton-Brown. 2014. International Conference on Machine Learning. PMLR, 754\u2013762. https:\/\/proceedings.mlr.press\/v32\/hutter14.html ISSN: 1938-7228."},{"key":"e_1_3_2_3_32_1","unstructured":"Amnesty International. 2021. Xenophobic Machines. Technical Report. https:\/\/www.amnesty.org\/en\/wp-content\/uploads\/2021\/10\/EUR3546862021ENGLISH.pdf"},{"key":"e_1_3_2_3_33_1","doi-asserted-by":"publisher","unstructured":"Rashidul Islam Shimei Pan and James\u00a0R. Foulds. 2021. AIES \u201921: AAAI\/ACM Conference on AI Ethics and Society. ACM Virtual Event USA 586\u2013596. https:\/\/doi.org\/10.1145\/3461702.3462614","DOI":"10.1145\/3461702.3462614"},{"key":"e_1_3_2_3_34_1","unstructured":"Alboukadel Kassambara. 2023. ggpubr: \u2019ggplot2\u2019 Based Publication Ready Plots. https:\/\/CRAN.R-project.org\/package=ggpubr"},{"key":"e_1_3_2_3_36_1","doi-asserted-by":"publisher","unstructured":"Christoph Kern Ruben\u00a0L. Bach Hannah Mautner and Frauke Kreuter. 2021. Fairness in Algorithmic Profiling: A German Case Study. (2021). https:\/\/doi.org\/10.48550\/arXiv.2108.04134","DOI":"10.48550\/arXiv.2108.04134"},{"key":"e_1_3_2_3_37_1","volume-title":"Adult data set. UCI machine learning repository 5","author":"Kohavi Ronny","year":"1996","unstructured":"Ronny Kohavi and Barry Becker. 1996. Adult data set. UCI machine learning repository 5 (1996), 2093."},{"key":"e_1_3_2_3_38_1","volume-title":"Feature engineering and selection: a practical approach for predictive models","author":"Kuhn Max","unstructured":"Max Kuhn and Kjell Johnson. 2020. Feature engineering and selection: a practical approach for predictive models. CRC Press, Taylor & Francis Group, Boca Raton London New York. www.feat.engineering"},{"key":"e_1_3_2_3_39_1","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1452"},{"key":"e_1_3_2_3_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-023-00679-8"},{"key":"e_1_3_2_3_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3457607"},{"key":"e_1_3_2_3_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3593013.3593988"},{"key":"e_1_3_2_3_43_1","doi-asserted-by":"publisher","DOI":"10.1111\/1475-3995.00375"},{"key":"e_1_3_2_3_44_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.aax2342"},{"key":"e_1_3_2_3_45_1","volume-title":"Our World in Data (06","author":"Ortiz-Ospina Esteban","year":"2017","unstructured":"Esteban Ortiz-Ospina and Max Roser. 2017. Healthcare Spending. Our World in Data (06 2017). https:\/\/ourworldindata.org\/financing-healthcare"},{"key":"e_1_3_2_3_46_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_3_2_3_47_1","doi-asserted-by":"publisher","unstructured":"Valerio Perrone Michele Donini Muhammad\u00a0Bilal Zafar Robin Schmucker Krishnaram Kenthapadi and C\u00e9dric Archambeau. 2021. AIES \u201921: AAAI\/ACM Conference on AI Ethics and Society. ACM Virtual Event USA 854\u2013863. https:\/\/doi.org\/10.1145\/3461702.3462629","DOI":"10.1145\/3461702.3462629"},{"key":"e_1_3_2_3_48_1","unstructured":"F. Pfisterer S. Coors J. Thomas and B. Bischl. 2019. Multi-Objective Automatic Machine Learning with AutoxgboostMC. arXiv 1908.10796 [stat.ML] (2019)."},{"key":"e_1_3_2_3_49_1","doi-asserted-by":"crossref","unstructured":"Kit\u00a0T. Rodolfa Pedro Saleiro and Rayid Ghani. 2020. Bias and Fairness (2 ed.). Chapman and Hall\/CRC. Num Pages: 32.","DOI":"10.1201\/9780429324383-11"},{"key":"e_1_3_2_3_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3593013.3593985"},{"key":"e_1_3_2_3_51_1","doi-asserted-by":"publisher","DOI":"10.1177\/0956797611417632"},{"key":"e_1_3_2_3_52_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0033242"},{"key":"e_1_3_2_3_53_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-020-0912-z"},{"key":"e_1_3_2_3_54_1","volume-title":"Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems 25","author":"Snoek Jasper","year":"2012","unstructured":"Jasper Snoek, Hugo Larochelle, and Ryan\u00a0P Adams. 2012. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems 25 (2012)."},{"key":"e_1_3_2_3_55_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMsb1706645"},{"key":"e_1_3_2_3_56_1","doi-asserted-by":"publisher","DOI":"10.1177\/1745691616658637"},{"key":"e_1_3_2_3_57_1","unstructured":"Pipenv\u00a0Maintainer Team. 2017. pipenv: Python Development Workflow for Humans.https:\/\/github.com\/pypa\/pipenv"},{"key":"e_1_3_2_3_58_1","volume-title":"R: A Language and Environment for Statistical Computing","author":"Team R\u00a0Core","year":"2022","unstructured":"R\u00a0Core Team. 2022. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https:\/\/www.R-project.org\/"},{"key":"e_1_3_2_3_59_1","doi-asserted-by":"publisher","unstructured":"The pandas\u00a0development team. 2020. pandas-dev\/pandas: Pandas. Zenodo. https:\/\/doi.org\/10.5281\/zenodo.3509134 DOI: 10.5281\/zenodo.3509134.","DOI":"10.5281\/zenodo.3509134"},{"key":"e_1_3_2_3_60_1","volume-title":"Python 3 Reference Manual. CreateSpace","author":"Van\u00a0Rossum Guido","unstructured":"Guido Van\u00a0Rossum and Fred\u00a0L. Drake. 2009. Python 3 Reference Manual. CreateSpace, Scotts Valley, CA."},{"key":"e_1_3_2_3_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3593013.3593998"},{"key":"e_1_3_2_3_62_1","first-page":"1","article-title":"Fairlearn: Assessing and Improving Fairness of AI Systems","volume":"24","author":"Weerts Hilde","year":"2023","unstructured":"Hilde Weerts, Miroslav Dud\u00edk, Richard Edgar, Adrin Jalali, Roman Lutz, and Michael Madaio. 2023. Fairlearn: Assessing and Improving Fairness of AI Systems. Journal of Machine Learning Research 24, 257 (2023), 1\u20138. http:\/\/jmlr.org\/papers\/v24\/23-0389.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_3_63_1","doi-asserted-by":"publisher","unstructured":"Hilde J.\u00a0P. Weerts. 2021. An Introduction to Algorithmic Fairness. (2021). https:\/\/doi.org\/10.48550\/arXiv.2105.05595","DOI":"10.48550\/arXiv.2105.05595"},{"key":"e_1_3_2_3_64_1","doi-asserted-by":"publisher","unstructured":"Hadley Wickham Mara Averick Jennifer Bryan Winston Chang Lucy McGowan Romain Fran\u00e7ois Garrett Grolemund Alex Hayes Lionel Henry Jim Hester Max Kuhn Thomas Pedersen Evan Miller Stephan Bache Kirill M\u00fcller Jeroen Ooms David Robinson Dana Seidel Vitalie Spinu Kohske Takahashi Davis Vaughan Claus Wilke Kara Woo and Hiroaki Yutani. 2019. Welcome to the Tidyverse. https:\/\/joss.theoj.org DOI: 10.21105\/joss.01686.","DOI":"10.21105\/joss.01686"},{"key":"e_1_3_2_3_65_1","series-title":"Series B (Statistical Methodology) 67, 2","volume-title":"Regularization and Variable Selection via the Elastic Net. Journal of the Royal Statistical Society","author":"Zou Hui","year":"2005","unstructured":"Hui Zou and Trevor Hastie. 2005. Regularization and Variable Selection via the Elastic Net. Journal of the Royal Statistical Society. Series B (Statistical Methodology) 67, 2 (2005), 301\u2013320. https:\/\/www.jstor.org\/stable\/3647580 Publisher: [Royal Statistical Society, Wiley]."}],"event":{"name":"FAccT '24: The 2024 ACM Conference on Fairness, Accountability, and Transparency","location":"Rio de Janeiro Brazil","acronym":"FAccT '24"},"container-title":["The 2024 ACM Conference on Fairness, Accountability, and Transparency"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3630106.3658974","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3630106.3658974","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:58Z","timestamp":1750287058000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3630106.3658974"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,3]]},"references-count":64,"alternative-id":["10.1145\/3630106.3658974","10.1145\/3630106"],"URL":"https:\/\/doi.org\/10.1145\/3630106.3658974","relation":{},"subject":[],"published":{"date-parts":[[2024,6,3]]},"assertion":[{"value":"2024-06-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}