{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T19:16:12Z","timestamp":1772738172049,"version":"3.50.1"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,7,29]],"date-time":"2024-07-29T00:00:00Z","timestamp":1722211200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"AFOSR, ONR, Minerva, NSF","award":["#2318461"],"award-info":[{"award-number":["#2318461"]}]},{"name":"Pitt Cyber Institute\u2019s PCAG"},{"name":"University of Pittsburgh Center for Research Computing, RRID:SCR","award":["022735"],"award-info":[{"award-number":["022735"]}]},{"name":"NIH","award":["#S10OD028483"],"award-info":[{"award-number":["#S10OD028483"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2024,8,31]]},"abstract":"<jats:p>Despite the benefits of personalizing items and information tailored to users\u2019 needs, it has been found that recommender systems tend to introduce biases that favor popular items or certain categories of items and dominant user groups. In this study, we aim to characterize the systematic errors of a recommendation system and how they manifest in various accountability issues, such as stereotypes, biases, and miscalibration. We propose a unified framework that distinguishes the sources of prediction errors into a set of key measures that quantify the various types of system-induced effects, at both the individual and collective levels. Based on our measuring framework, we examine the most widely adopted algorithms in the context of movie recommendation. Our research reveals three important findings: (1) Differences between algorithms: recommendations generated by simpler algorithms tend to be more stereotypical but less biased than those generated by more complex algorithms. (2) Disparate impact on groups and individuals: system-induced biases and stereotypes have a disproportionate effect on atypical users and minority groups (e.g., women and older users). (3) Mitigation opportunity: using structural equation modeling, we identify the interactions between user characteristics (typicality and diversity), system-induced effects, and miscalibration. We further investigate the possibility of mitigating system-induced effects by oversampling underrepresented groups and individuals, which was found to be effective in reducing stereotypes and improving recommendation quality. Our research is the first systematic examination of not only system-induced effects and miscalibration but also the stereotyping issue in recommender systems.<\/jats:p>","DOI":"10.1145\/3650044","type":"journal-article","created":{"date-parts":[[2024,2,29]],"date-time":"2024-02-29T12:26:44Z","timestamp":1709209604000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Break Out of a Pigeonhole: A Unified Framework for Examining Miscalibration, Bias, and Stereotype in Recommender Systems"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5797-5445","authenticated-orcid":false,"given":"Yongsu","family":"Ahn","sequence":"first","affiliation":[{"name":"University of Pittsburgh, Pittsburgh, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8497-3015","authenticated-orcid":false,"given":"Yu-Ru","family":"Lin","sequence":"additional","affiliation":[{"name":"University of Pittsburgh, Pittsburgh, USA"}]}],"member":"320","published-online":{"date-parts":[[2024,7,29]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"crossref","unstructured":"Mohsen Abbasi Sorelle A. Friedler Carlos Scheidegger and Suresh Venkatasubramanian. 2019. Fairness in representation: Quantifying stereotyping as a representational harm. arXiv:1901.09565 [cs stat]. http:\/\/arxiv.org\/abs\/1901.09565","DOI":"10.1137\/1.9781611975673.90"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3109859.3109912"},{"key":"e_1_3_2_4_2","unstructured":"Himan Abdollahpouri Masoud Mansoury Robin Burke and Bamshad Mobasher. 2019. The Impact of Popularity Bias on Fairness and Calibration in Recommendation. arXiv:1910.05755 [cs]. http:\/\/arxiv.org\/abs\/1910.05755"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383313.3418487"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934262"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3484509"},{"key":"e_1_3_2_8_2","first-page":"23","volume-title":"KaRS@ RecSys","author":"ALRossais Nourah A.","year":"2018","unstructured":"Nourah A. ALRossais and Daniel Kudenko. 2018. Evaluating stereotype and non-stereotype recommender systems.. In KaRS@ RecSys. 23\u201328."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aaa1160"},{"key":"e_1_3_2_10_2","unstructured":"Shilong Bao Qianqian Xu Zhiyong Yang Yuan He Xiaochun Cao and Qingming Huang. 2022. The Minority Matters: A Diversity-Promoting Collaborative Metric Learning Algorithm. arXiv:2209.15292 [cs]. http:\/\/arxiv.org\/abs\/2209.15292"},{"key":"e_1_3_2_11_2","volume-title":"Stereotype and Most-Popular Recommendations in the Digital Library Sowiport","author":"Beel Joeran","year":"2017","unstructured":"Joeran Beel, Siddharth Dinesh, Philipp Mayr, Zeljko Carevic, and Jain Raghvendra. 2017. Stereotype and Most-Popular Recommendations in the Digital Library Sowiport. Humboldt-Universit\u00e4t zu Berlin."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1080\/05568640409485143"},{"key":"e_1_3_2_13_2","unstructured":"James Brofos Rui Shu and Roy R. Lederman. [n. d.]. A bias-variance decomposition for Bayesian deep learning."},{"key":"e_1_3_2_14_2","volume-title":"NeurIPS 2019 Workshop on Bayesian Deep Learning","author":"Brofos James","year":"2019","unstructured":"James Brofos, Rui Shu, and Roy R. Lederman. 2019. A bias-variance decomposition for Bayesian deep learning. In NeurIPS 2019 Workshop on Bayesian Deep Learning."},{"key":"e_1_3_2_15_2","article-title":"FairVis: Visual analytics for discovering intersectional Bias in machine learning","author":"Cabrera \u00c1ngel Alexander","year":"2019","unstructured":"\u00c1ngel Alexander Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, and Duen Horng Chau. 2019. FairVis: Visual analytics for discovering intersectional Bias in machine learning. arXiv:1904.05419 [cs, stat].http:\/\/arxiv.org\/abs\/1904.05419","journal-title":"arXiv:1904.05419 [cs, stat]."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/1722149.1722154"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240370"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052585"},{"key":"e_1_3_2_19_2","first-page":"172","volume-title":"Proceedings of the 1st Conference on Fairness, Accountability and Transparency","author":"Ekstrand Michael D.","year":"2018","unstructured":"Michael D. Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D. Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All the cool kids, how do they fit in? Popularity and demographic biases in recommender evaluation and effectiveness. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency. PMLR, 172\u2013186. https:\/\/proceedings.mlr.press\/v81\/ekstrand18b.htmlISSN: 2640-3498."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240373"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2020.102377"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401431"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401431"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3465416.3483298"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052569"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1162\/089976698300017232"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.22"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022899518027"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702520"},{"key":"e_1_3_2_31_2","first-page":"275","volume-title":"ICML","author":"Kohavi Ron","year":"1996","unstructured":"Ron Kohavi, David H Wolpert, et\u00a0al. 1996. Bias plus variance decomposition for zero-one loss functions. In ICML, Vol. 96. Citeseer, 275\u2013283."},{"key":"e_1_3_2_32_2","unstructured":"Kun Lin Nasim Sonboli Bamshad Mobasher and Robin Burke. 2019. Crank up the Volume: Preference Bias Amplification in Collaborative Recommendation. arXiv:1909.06362 [cs stat]. http:\/\/arxiv.org\/abs\/1909.06362"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3372923.3404793"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2003.1167344"},{"key":"e_1_3_2_35_2","unstructured":"Masoud Mansoury Himan Abdollahpouri Jessie Smith Arman Dehpanah Mykola Pechenizkiy and Bamshad Mobasher. 2020. Investigating Potential Factors Associated with Gender Discrimination in Collaborative Recommender Systems. arXiv:2002.07786 [cs]. http:\/\/arxiv.org\/abs\/2002.07786"},{"key":"e_1_3_2_36_2","first-page":"11","volume-title":"AMIR@ ECIR","author":"Mansoury Masoud","year":"2019","unstructured":"Masoud Mansoury and Robin Burke. 2019. Algorithm selection with librec-auto. In AMIR@ ECIR. 11\u201317."},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3241614"},{"key":"e_1_3_2_38_2","unstructured":"Masoud Mansoury Bamshad Mobasher Robin Burke and Mykola Pechenizkiy. 2019. Bias Disparity in Collaborative Recommendation: Algorithmic Evaluation and Comparison. arXiv:1908.00831 [cs]. http:\/\/arxiv.org\/abs\/1908.00831"},{"key":"e_1_3_2_39_2","article-title":"Umap: Uniform manifold approximation and projection for dimension reduction","author":"McInnes Leland","year":"2018","unstructured":"Leland McInnes, John Healy, and James Melville. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426.","journal-title":"arXiv preprint arXiv:1802.03426"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2022.2067936"},{"key":"e_1_3_2_41_2","doi-asserted-by":"crossref","unstructured":"Yuri Nakao Simone Stumpf Subeida Ahmed Aisha Naseer and Lorenzo Strappelli. 2022. Towards Involving End-users in Interactive Human-in-the-loop AI Fairness. arXiv:2204.10464 [cs]. http:\/\/arxiv.org\/abs\/2204.10464","DOI":"10.1145\/3514258"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/1297231.1297251"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/2566486.2568012"},{"key":"e_1_3_2_44_2","article-title":"BPR: Bayesian personalized ranking from implicit feedback","author":"Rendle Steffen","year":"2012","unstructured":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618.","journal-title":"arXiv preprint arXiv:1205.2618"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/192844.192905"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240372"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780195378016.001.0001"},{"key":"e_1_3_2_48_2","article-title":"Bias disparity in recommendation systems","author":"Tsintzou Virginia","year":"2018","unstructured":"Virginia Tsintzou, Evaggelia Pitoura, and Panayiotis Tsaparas. 2018. Bias disparity in recommendation systems. arXiv:1811.01461 [cs]. http:\/\/arxiv.org\/abs\/1811.01461","journal-title":"arXiv:1811.01461 [cs]"},{"key":"e_1_3_2_49_2","article-title":"Structural equation modeling","volume":"2","author":"Ullman Jodie B.","year":"2012","unstructured":"Jodie B. Ullman and Peter M. Bentler. 2012. Structural equation modeling. In Handbook of Psychology. (2nd ed.).","journal-title":"Handbook of Psychology."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467249"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3532075"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2017.00428"},{"key":"e_1_3_2_53_2","unstructured":"Hongzhi Yin Bin Cui Jing Li Junjie Yao and Chen Chen. 2012. Challenging the Long Tail Recommendation. arXiv:1205.6700 [cs]. http:\/\/arxiv.org\/abs\/1205.6700"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467340"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/1060745.1060754"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3650044","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3650044","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:43Z","timestamp":1750291423000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3650044"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,29]]},"references-count":54,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,8,31]]}},"alternative-id":["10.1145\/3650044"],"URL":"https:\/\/doi.org\/10.1145\/3650044","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,29]]},"assertion":[{"value":"2023-06-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}