{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T22:58:35Z","timestamp":1775861915881,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":52,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,11,8]],"date-time":"2020-11-08T00:00:00Z","timestamp":1604793600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1934884, 1513263"],"award-info":[{"award-number":["1934884, 1513263"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,11,8]]},"DOI":"10.1145\/3368089.3409704","type":"proceedings-article","created":{"date-parts":[[2020,11,10]],"date-time":"2020-11-10T21:08:44Z","timestamp":1605042524000},"page":"642-653","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":73,"title":["Do the machine learning models on a crowd sourced platform exhibit bias? an empirical study on model fairness"],"prefix":"10.1145","author":[{"given":"Sumon","family":"Biswas","sequence":"first","affiliation":[{"name":"Iowa State University, USA"}]},{"given":"Hridesh","family":"Rajan","sequence":"additional","affiliation":[{"name":"Iowa State University, USA"}]}],"member":"320","published-online":{"date-parts":[[2020,11,8]]},"reference":[{"key":"e_1_3_2_2_1_1","unstructured":"Julius Adebayo and Lalana Kagal. 2016. Iterative orthogonal feature projection for diagnosing bias in black-box models. arXiv preprint arXiv:1611.04967 ( 2016 ).  Julius Adebayo and Lalana Kagal. 2016. Iterative orthogonal feature projection for diagnosing bias in black-box models. arXiv preprint arXiv:1611.04967 ( 2016 )."},{"key":"e_1_3_2_2_2_1","unstructured":"Julius A Adebayo etal 2016. FairML: ToolBox for diagnosing bias in predictive modeling. Ph.D. Dissertation. Massachusetts Institute of Technology.  Julius A Adebayo et al. 2016. FairML: ToolBox for diagnosing bias in predictive modeling. Ph.D. Dissertation. Massachusetts Institute of Technology."},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338937"},{"key":"e_1_3_2_2_4_1","unstructured":"Rachel KE Bellamy Kuntal Dey Michael Hind Samuel C Hofman Stephanie Houde Kalapriya Kannan Pranay Lohia Jacquelyn Martino Sameep Mehta Aleksandra Mojsilovic etal 2018. AI Fairness 360 : An extensible toolkit for detecting understanding and mitigating unwanted algorithmic bias. arXiv preprint arXiv: 1810. 01943 ( 2018 ).  Rachel KE Bellamy Kuntal Dey Michael Hind Samuel C Hofman Stephanie Houde Kalapriya Kannan Pranay Lohia Jacquelyn Martino Sameep Mehta Aleksandra Mojsilovic et al. 2018. AI Fairness 360 : An extensible toolkit for detecting understanding and mitigating unwanted algorithmic bias. arXiv preprint arXiv: 1810. 01943 ( 2018 )."},{"key":"e_1_3_2_2_5_1","unstructured":"Richard Berk Hoda Heidari Shahin Jabbari Michael Kearns and Aaron Roth. 2018. Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research ( 2018 ) 0049124118782533.  Richard Berk Hoda Heidari Shahin Jabbari Michael Kearns and Aaron Roth. 2018. Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research ( 2018 ) 0049124118782533."},{"key":"e_1_3_2_2_6_1","unstructured":"Reuben Binns. 2017. Fairness in machine learning: Lessons from political philosophy. arXiv preprint arXiv:1712.03586 ( 2017 ).  Reuben Binns. 2017. Fairness in machine learning: Lessons from political philosophy. arXiv preprint arXiv:1712.03586 ( 2017 )."},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.3912064"},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"crossref","unstructured":"Toon Calders and Sicco Verwer. 2010. Three naive Bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery 21 2 ( 2010 ) 277-292.  Toon Calders and Sicco Verwer. 2010. Three naive Bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery 21 2 ( 2010 ) 277-292.","DOI":"10.1007\/s10618-010-0190-x"},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"crossref","volume-title":"May the best analyst win","author":"Carpenter Jennifer","DOI":"10.1126\/science.331.6018.698"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287594"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"crossref","unstructured":"Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5 2 ( 2017 ) 153-163.  Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5 2 ( 2017 ) 153-163.","DOI":"10.1089\/big.2016.0047"},{"key":"e_1_3_2_2_12_1","unstructured":"George E Dahl Navdeep Jaitly and Ruslan Salakhutdinov. 2014. Multi-task neural networks for QSAR predictions. arXiv preprint arXiv:1406.1231 ( 2014 ).  George E Dahl Navdeep Jaitly and Ruslan Salakhutdinov. 2014. Multi-task neural networks for QSAR predictions. arXiv preprint arXiv:1406.1231 ( 2014 )."},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3278721.3278729"},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2090236.2090255"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783311"},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287589"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106277"},{"key":"e_1_3_2_2_18_1","first-page":"2415","article-title":"Satisfying real-world goals with dataset constraints","author":"Goh Gabriel","year":"2016","journal-title":"Advances in Neural Information Processing Systems."},{"key":"e_1_3_2_2_19_1","unstructured":"Divya Gopinath Kaiyuan Wang Mengshi Zhang Corina S Pasareanu and Sarfraz Khurshid. 2018. Symbolic execution for deep neural networks. arXiv preprint arXiv:1807. 10439 ( 2018 ).  Divya Gopinath Kaiyuan Wang Mengshi Zhang Corina S Pasareanu and Sarfraz Khurshid. 2018. Symbolic execution for deep neural networks. arXiv preprint arXiv:1807. 10439 ( 2018 )."},{"key":"e_1_3_2_2_20_1","unstructured":"Moritz Hardt Eric Price and Nati Srebro. 2016. Equality of opportunity in supervised learning. In Advances in neural information processing systems. 3315-3323.  Moritz Hardt Eric Price and Nati Srebro. 2016. Equality of opportunity in supervised learning. In Advances in neural information processing systems. 3315-3323."},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372831"},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300830"},{"key":"e_1_3_2_2_23_1","unstructured":"Kathryn Jepsen. 2014. The machine learning community takes on the Higgs. https:\/\/www.symmetrymagazine.org\/article\/july-2014\/the-machinelearning-community-takes-on-the-higgs.  Kathryn Jepsen. 2014. The machine learning community takes on the Higgs. https:\/\/www.symmetrymagazine.org\/article\/july-2014\/the-machinelearning-community-takes-on-the-higgs."},{"key":"e_1_3_2_2_24_1","volume-title":"Artificial intelligence with python","author":"Joshi Prateek"},{"key":"e_1_3_2_2_25_1","unstructured":"Kaggle. 2010. The world's largest data science community with powerful tools and resources to help you achieve your data science goals. www.kaggle.com.  Kaggle. 2010. The world's largest data science community with powerful tools and resources to help you achieve your data science goals. www.kaggle.com."},{"key":"e_1_3_2_2_26_1","unstructured":"Kaggle. 2017. Adult Census Dataset. https:\/\/www.kaggle.com\/uciml\/adultcensus-income.  Kaggle. 2017. Adult Census Dataset. https:\/\/www.kaggle.com\/uciml\/adultcensus-income."},{"key":"e_1_3_2_2_27_1","unstructured":"Kaggle. 2017. Bank Marketing Dataset. https:\/\/www.kaggle.com\/c\/bankmarketing-uci.  Kaggle. 2017. Bank Marketing Dataset. https:\/\/www.kaggle.com\/c\/bankmarketing-uci."},{"key":"e_1_3_2_2_28_1","volume-title":"Competition: Santander Product Recommendation. https:\/\/www. kaggle.com\/c\/santander-product-recommendation\/overview.","year":"2017"},{"key":"e_1_3_2_2_29_1","unstructured":"Kaggle. 2017. German Credit Dataset. https:\/\/www.kaggle.com\/uciml\/germancredit.  Kaggle. 2017. German Credit Dataset. https:\/\/www.kaggle.com\/uciml\/germancredit."},{"key":"e_1_3_2_2_30_1","unstructured":"Kaggle. 2017. Home Credit Dataset. https:\/\/www.kaggle.com\/c\/home-creditdefault-risk.  Kaggle. 2017. Home Credit Dataset. https:\/\/www.kaggle.com\/c\/home-creditdefault-risk."},{"key":"e_1_3_2_2_31_1","unstructured":"Kaggle. 2017. Titanic ML Dataset. https:\/\/www.kaggle.com\/c\/titanic\/data.  Kaggle. 2017. Titanic ML Dataset. https:\/\/www.kaggle.com\/c\/titanic\/data."},{"key":"e_1_3_2_2_32_1","unstructured":"Kaggle. 2019. Adult Census Kernel: Multiple ML Techniques and Analysis. https:\/\/www.kaggle.com\/bananuhbeatdown\/multiple-ml-techniques-andanalysis-of-dataset.  Kaggle. 2019. Adult Census Kernel: Multiple ML Techniques and Analysis. https:\/\/www.kaggle.com\/bananuhbeatdown\/multiple-ml-techniques-andanalysis-of-dataset."},{"key":"e_1_3_2_2_33_1","volume-title":"Kernel: German Credit Risk Analysis. https:\/\/www.kaggle.com\/ pahulpreet\/german-credit-risk-analysis-beginner-s-guide.","year":"2019"},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"crossref","unstructured":"Faisal Kamiran and Toon Calders. 2012. Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems 33 1 ( 2012 ) 1-33.  Faisal Kamiran and Toon Calders. 2012. Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems 33 1 ( 2012 ) 1-33.","DOI":"10.1007\/s10115-011-0463-8"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2012.45"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33486-3_3"},{"key":"e_1_3_2_2_37_1","unstructured":"Jon Kleinberg Sendhil Mullainathan and Manish Raghavan. 2016. Inherent trade-ofs in the determination of risk scores. arXiv preprint arXiv:1609.05807 ( 2016 ).  Jon Kleinberg Sendhil Mullainathan and Manish Raghavan. 2016. Inherent trade-ofs in the determination of risk scores. arXiv preprint arXiv:1609.05807 ( 2016 )."},{"key":"e_1_3_2_2_38_1","doi-asserted-by":"crossref","unstructured":"Judea Pearl et al. 2009. Causal inference in statistics: An overview. Statistics surveys 3 ( 2009 ) 96-146.  Judea Pearl et al. 2009. Causal inference in statistics: An overview. Statistics surveys 3 ( 2009 ) 96-146.","DOI":"10.1214\/09-SS057"},{"key":"e_1_3_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401959"},{"key":"e_1_3_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132785"},{"key":"e_1_3_2_2_41_1","first-page":"5680","article-title":"On fairness and calibration","author":"Pleiss Geof","year":"2017","journal-title":"Advances in Neural Information Processing Systems."},{"key":"e_1_3_2_2_42_1","unstructured":"Scikit Learn. 2019. LightGBM API Documentation. https:\/\/lightgbm.readthedocs. io\/en\/latest\/pythonapi\/lightgbm.LGBMClassifier.html.  Scikit Learn. 2019. LightGBM API Documentation. https:\/\/lightgbm.readthedocs. io\/en\/latest\/pythonapi\/lightgbm.LGBMClassifier.html."},{"key":"e_1_3_2_2_43_1","unstructured":"Scikit Learn. 2019. SVC API Documentation. https:\/\/scikit-learn.org\/stable\/ modules\/generated\/sklearn.preprocessing.StandardScaler.html.  Scikit Learn. 2019. SVC API Documentation. https:\/\/scikit-learn.org\/stable\/ modules\/generated\/sklearn.preprocessing.StandardScaler.html."},{"key":"e_1_3_2_2_44_1","volume-title":"FAT Forensics: A Python Toolbox for Algorithmic Fairness, Accountability and Transparency. arXiv preprint arXiv","author":"Sokol Kacper","year":"1909"},{"key":"e_1_3_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220046"},{"key":"e_1_3_2_2_46_1","unstructured":"Stack Overflow. 2016. How does the class_weight parameter in scikit-learn work ? https:\/\/stackoverflow.com\/questions\/30972029\/how-does-the-classweight-parameter-in-scikit-learn-work.  Stack Overflow. 2016. How does the class_weight parameter in scikit-learn work ? https:\/\/stackoverflow.com\/questions\/30972029\/how-does-the-classweight-parameter-in-scikit-learn-work."},{"key":"e_1_3_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_3_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP.2017.29"},{"key":"e_1_3_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238165"},{"key":"e_1_3_2_2_50_1","volume-title":"Manuel Gomez Rodriguez, and Krishna P Gummadi","author":"Zafar Muhammad Bilal","year":"2015"},{"key":"e_1_3_2_2_51_1","volume-title":"International Conference on Machine Learning. 325-333","author":"Zemel Rich","year":"2013"},{"key":"e_1_3_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3278721.3278779"}],"event":{"name":"ESEC\/FSE '20: 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering","location":"Virtual Event USA","acronym":"ESEC\/FSE '20","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"]},"container-title":["Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3368089.3409704","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3368089.3409704","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:44:40Z","timestamp":1750203880000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3368089.3409704"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,8]]},"references-count":52,"alternative-id":["10.1145\/3368089.3409704","10.1145\/3368089"],"URL":"https:\/\/doi.org\/10.1145\/3368089.3409704","relation":{},"subject":[],"published":{"date-parts":[[2020,11,8]]},"assertion":[{"value":"2020-11-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}