{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T22:51:08Z","timestamp":1786143068884,"version":"3.56.0"},"publisher-location":"New York, NY, USA","reference-count":43,"publisher":"ACM","license":[{"start":{"date-parts":[[2019,6,17]],"date-time":"2019-06-17T00:00:00Z","timestamp":1560729600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2019,6,17]]},"DOI":"10.1145\/3322640.3326705","type":"proceedings-article","created":{"date-parts":[[2019,7,23]],"date-time":"2019-07-23T12:18:16Z","timestamp":1563884296000},"page":"83-92","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":66,"title":["Why Machine Learning May Lead to Unfairness"],"prefix":"10.1145","author":[{"given":"Song\u00fcl","family":"Tolan","sequence":"first","affiliation":[{"name":"Joint Research Centre, European Commission"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marius","family":"Miron","sequence":"additional","affiliation":[{"name":"Joint Research Centre, European Commission"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emilia","family":"G\u00f3mez","sequence":"additional","affiliation":[{"name":"Joint Research Centre, European Commission, Universitat Pompeu Fabra, Barcelona"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carlos","family":"Castillo","sequence":"additional","affiliation":[{"name":"Universitat Pompeu Fabra, Barcelona"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,6,17]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1177\/0011000005285875"},{"key":"e_1_3_2_1_2_1","volume-title":"A reductions approach to fair classification. arXiv preprint arXiv:1803.02453","author":"Agarwal A","year":"2018","unstructured":"A Agarwal , A Beygelzimer , M Dud\u00edk , J Langford , and H Wallach . 2018. A reductions approach to fair classification. arXiv preprint arXiv:1803.02453 ( 2018 ). A Agarwal, A Beygelzimer, M Dud\u00edk, J Langford, and H Wallach. 2018. A reductions approach to fair classification. arXiv preprint arXiv:1803.02453 (2018)."},{"key":"e_1_3_2_1_3_1","volume-title":"ProPublica","author":"Angwin J","year":"2016","unstructured":"J Angwin , J Larson , S Mattu , and L Kirchner . 2016 . Machine bias . ProPublica , May 23 (2016). J Angwin, J Larson, S Mattu, and L Kirchner. 2016. Machine bias. ProPublica, May 23 (2016)."},{"key":"e_1_3_2_1_4_1","volume-title":"Big Data's Disparate Impact. California law review 104, 1","author":"Barocas S","year":"2016","unstructured":"S Barocas and A Selbst . 2016. Big Data's Disparate Impact. California law review 104, 1 ( 2016 ), 671--729. S Barocas and A Selbst. 2016. Big Data's Disparate Impact. California law review 104, 1 (2016), 671--729."},{"key":"e_1_3_2_1_5_1","unstructured":"R Berk H Heidari S Jabbari M Kearns and A Roth. 2017. Fairness in Criminal Justice Risk Assessments: The State of the Art. (2017) 1--42. arXiv:1703.09207 http:\/\/arxiv.org\/abs\/1703.09207  R Berk H Heidari S Jabbari M Kearns and A Roth. 2017. Fairness in Criminal Justice Risk Assessments: The State of the Art. (2017) 1--42. arXiv:1703.09207 http:\/\/arxiv.org\/abs\/1703.09207"},{"key":"e_1_3_2_1_6_1","volume-title":"Do girls manipulate and boys fight? Developmental trends in regard to direct and indirect aggression. Aggressive behavior 18, 2","author":"Bj\u00f6rkqvist K","year":"1992","unstructured":"K Bj\u00f6rkqvist , KM Lagerspetz , and A Kaukiainen . 1992. Do girls manipulate and boys fight? Developmental trends in regard to direct and indirect aggression. Aggressive behavior 18, 2 ( 1992 ), 117--127. K Bj\u00f6rkqvist, KM Lagerspetz, and A Kaukiainen. 1992. Do girls manipulate and boys fight? Developmental trends in regard to direct and indirect aggression. Aggressive behavior 18, 2 (1992), 117--127."},{"key":"e_1_3_2_1_7_1","unstructured":"M Blanch M Capdevila M Ferrer B Framis \u00da Ruiz J Mora A Batlle and B L\u00f3pez. 2017. La reincid\u00e8ncia en la just\u00edcia de menors. CEJFE.  M Blanch M Capdevila M Ferrer B Framis \u00da Ruiz J Mora A Batlle and B L\u00f3pez. 2017. La reincid\u00e8ncia en la just\u00edcia de menors. CEJFE."},{"key":"e_1_3_2_1_8_1","unstructured":"R Borum. 2006. Manual for the structured assessment of violence risk in youth (SAVRY). (2006).  R Borum. 2006. Manual for the structured assessment of violence risk in youth (SAVRY). (2006)."},{"key":"e_1_3_2_1_9_1","volume-title":"Structured assessment of violence risk in youth. Mental health screening and assessment in juvenile justice","author":"Borum Randy","year":"2005","unstructured":"Randy Borum , Patrick A Bartel , and Adelle E Forth . 2005. Structured assessment of violence risk in youth. Mental health screening and assessment in juvenile justice ( 2005 ), 311--323. Randy Borum, Patrick A Bartel, and Adelle E Forth. 2005. Structured assessment of violence risk in youth. Mental health screening and assessment in juvenile justice (2005), 311--323."},{"key":"e_1_3_2_1_10_1","volume-title":"Correctional Offender Management Profiles for Alternative Sanctions (COMPAS). Handbook of Recidivism Risk\/Needs Assessment Tools","author":"Brennan T","year":"2018","unstructured":"T Brennan and W Dieterich . 2018. Correctional Offender Management Profiles for Alternative Sanctions (COMPAS). Handbook of Recidivism Risk\/Needs Assessment Tools ( 2018 ), 49. T Brennan and W Dieterich. 2018. Correctional Offender Management Profiles for Alternative Sanctions (COMPAS). Handbook of Recidivism Risk\/Needs Assessment Tools (2018), 49."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1111\/1745-9133.12055"},{"key":"e_1_3_2_1_13_1","volume-title":"Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2","author":"Chouldechova A","year":"2017","unstructured":"A Chouldechova . 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2 ( 2017 ), 153--163. A Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2 (2017), 153--163."},{"key":"e_1_3_2_1_14_1","volume-title":"The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023","author":"Corbett-Davies S","year":"2018","unstructured":"S Corbett-Davies and S Goel . 2018. The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023 ( 2018 ). S Corbett-Davies and S Goel. 2018. The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023 (2018)."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098095"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1018033108"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"crossref","unstructured":"MDeMichele P Baumgartner M Wenger K Barrick M Comfort and S Misra. 2018. The Public Safety Assessment: A Re-Validation and Assessment of Predictive Utility and Differential Prediction by Race and Gender in Kentucky. (2018).  MDeMichele P Baumgartner M Wenger K Barrick M Comfort and S Misra. 2018. The Public Safety Assessment: A Re-Validation and Assessment of Predictive Utility and Differential Prediction by Race and Gender in Kentucky. (2018).","DOI":"10.2139\/ssrn.3168452"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1037\/ser0000075"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2090236.2090255"},{"key":"e_1_3_2_1_20_1","volume-title":"A comparative study of fairness-enhancing interventions in machine learning. arXiv preprint arXiv:1802.04422","author":"Friedler S A","year":"2018","unstructured":"S A Friedler , C Scheidegger , S Venkatasubramanian , S Choudhary , EP Hamilton , and D Roth . 2018. A comparative study of fairness-enhancing interventions in machine learning. arXiv preprint arXiv:1802.04422 ( 2018 ). S A Friedler, C Scheidegger, S Venkatasubramanian, S Choudhary, EP Hamilton, and D Roth. 2018. A comparative study of fairness-enhancing interventions in machine learning. arXiv preprint arXiv:1802.04422 (2018)."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287563"},{"key":"e_1_3_2_1_22_1","volume-title":"Proceedings of the Machine Learning: The Debates Workshop.","author":"Green B","year":"2018","unstructured":"B Green and L Hu . 2018 . The myth in the methodology: towards a recontextualization of fairness in machine learning . In Proceedings of the Machine Learning: The Debates Workshop. B Green and L Hu. 2018. The myth in the methodology: towards a recontextualization of fairness in machine learning. In Proceedings of the Machine Learning: The Debates Workshop."},{"key":"e_1_3_2_1_24_1","unstructured":"M Hardt E Price N Srebro etal 2016. Equality of opportunity in supervised learning. In Advances in neural information processing systems. 3315--3323.   M Hardt E Price N Srebro et al. 2016. Equality of opportunity in supervised learning. In Advances in neural information processing systems. 3315--3323."},{"key":"e_1_3_2_1_25_1","volume-title":"Residual Unfairness in Fair Machine Learning from Prejudiced Data. arXiv preprint arXiv:1806.02887","author":"Kallus N","year":"2018","unstructured":"N Kallus and A Zhou . 2018. Residual Unfairness in Fair Machine Learning from Prejudiced Data. arXiv preprint arXiv:1806.02887 ( 2018 ). N Kallus and A Zhou. 2018. Residual Unfairness in Fair Machine Learning from Prejudiced Data. arXiv preprint arXiv:1806.02887 (2018)."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"crossref","DOI":"10.3386\/w23180","volume-title":"Human decisions and machine predictions. The quarterly journal of economics 133, 1","author":"Kleinberg J","year":"2017","unstructured":"J Kleinberg , H Lakkaraju , J Leskovec , J Ludwig , and S Mullainathan . 2017. Human decisions and machine predictions. The quarterly journal of economics 133, 1 ( 2017 ), 237--293. J Kleinberg, H Lakkaraju, J Leskovec, J Ludwig, and S Mullainathan. 2017. Human decisions and machine predictions. The quarterly journal of economics 133, 1 (2017), 237--293."},{"key":"e_1_3_2_1_27_1","volume-title":"Proceedings of Innovations in Theoretical Computer Science (ITCS).","author":"Kleinberg J","year":"2016","unstructured":"J Kleinberg , S Mullainathan , and M Raghavan . 2016 . Inherent trade-offs in the fair determination of risk scores . In Proceedings of Innovations in Theoretical Computer Science (ITCS). J Kleinberg, S Mullainathan, and M Raghavan. 2016. Inherent trade-offs in the fair determination of risk scores. In Proceedings of Innovations in Theoretical Computer Science (ITCS)."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/219717.219768"},{"key":"e_1_3_2_1_29_1","volume-title":"The mythos of model interpretability. arXiv preprint arXiv:1606.03490","author":"Lipton ZC","year":"2016","unstructured":"ZC Lipton . 2016. The mythos of model interpretability. arXiv preprint arXiv:1606.03490 ( 2016 ). ZC Lipton. 2016. The mythos of model interpretability. arXiv preprint arXiv:1606.03490 (2016)."},{"key":"e_1_3_2_1_30_1","unstructured":"Za Lipton J McAuley and A Chouldechova. 2018. Does mitigating ML's impact disparity require treatment disparity?. In Advances in Neural Information Processing Systems. 8136--8146.   Za Lipton J McAuley and A Chouldechova. 2018. Does mitigating ML's impact disparity require treatment disparity?. In Advances in Neural Information Processing Systems. 8136--8146."},{"key":"e_1_3_2_1_31_1","volume-title":"Delayed impact of fair machine learning. arXiv preprint arXiv:1803.04383","author":"Liu LT","year":"2018","unstructured":"LT Liu , S Dean , E Rolf , M Simchowitz , and M Hardt . 2018. Delayed impact of fair machine learning. arXiv preprint arXiv:1803.04383 ( 2018 ). LT Liu, S Dean, E Rolf, M Simchowitz, and M Hardt. 2018. Delayed impact of fair machine learning. arXiv preprint arXiv:1803.04383 (2018)."},{"key":"e_1_3_2_1_33_1","unstructured":"Northpoint Inc. 2012. COMPAS Risk and Need Assessment System. Technical Report. Northpoint Inc.  Northpoint Inc. 2012. COMPAS Risk and Need Assessment System. Technical Report. Northpoint Inc."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1177\/0093854809331457"},{"key":"e_1_3_2_1_35_1","volume-title":"Are risk assessments racially biased?: Field study of the SAVRY and YLS\/CMI in probation. Psychological assessment 29, 6","author":"Perrault R T","year":"2017","unstructured":"R T Perrault , G M Vincent , and L S Guy . 2017. Are risk assessments racially biased?: Field study of the SAVRY and YLS\/CMI in probation. Psychological assessment 29, 6 ( 2017 ), 664. R T Perrault, G M Vincent, and L S Guy. 2017. Are risk assessments racially biased?: Field study of the SAVRY and YLS\/CMI in probation. Psychological assessment 29, 6 (2017), 664."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"crossref","unstructured":"C Robert. 2014. Machine learning a probabilistic perspective.  C Robert. 2014. Machine learning a probabilistic perspective.","DOI":"10.1080\/09332480.2014.914768"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1177\/0022427895032001004"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1080\/14999013.2014.922141"},{"key":"e_1_3_2_1_40_1","volume-title":"risk assessment, and sanctioning: The cost of treating women like men. Law and human behavior 40, 5","author":"Skeem J","year":"2016","unstructured":"J Skeem , J Monahan , and C Lowenkamp . 2016. Gender , risk assessment, and sanctioning: The cost of treating women like men. Law and human behavior 40, 5 ( 2016 ), 580. J Skeem, J Monahan, and C Lowenkamp. 2016. Gender, risk assessment, and sanctioning: The cost of treating women like men. Law and human behavior 40, 5 (2016), 580."},{"key":"e_1_3_2_1_41_1","volume-title":"Fair and Unbiased Algorithmic Decision Making: Current State and Future Challenges. JRC Digital Economy Working Paper 10","author":"Tolan S","year":"2018","unstructured":"S Tolan . 2018. Fair and Unbiased Algorithmic Decision Making: Current State and Future Challenges. JRC Digital Economy Working Paper 10 ( 2018 ). S Tolan. 2018. Fair and Unbiased Algorithmic Decision Making: Current State and Future Challenges. JRC Digital Economy Working Paper 10 (2018)."},{"key":"e_1_3_2_1_42_1","volume-title":"Does risk assessment make a difference? Results of implementing the SAVRY in juvenile probation. Behavioral sciences & the law 30, 4","author":"Vincent G M","year":"2012","unstructured":"G M Vincent , L S Guy , B G Gershenson , and P McCabe . 2012. Does risk assessment make a difference? Results of implementing the SAVRY in juvenile probation. Behavioral sciences & the law 30, 4 ( 2012 ), 384--405. G M Vincent, L S Guy, B G Gershenson, and P McCabe. 2012. Does risk assessment make a difference? Results of implementing the SAVRY in juvenile probation. Behavioral sciences & the law 30, 4 (2012), 384--405."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1177\/1043986207309595"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052660"},{"key":"e_1_3_2_1_45_1","volume-title":"On the relation between accuracy and fairness in binary classification. arXiv preprint arXiv:1505.05723","author":"\u017dliobait\u0117 I","year":"2015","unstructured":"I \u017dliobait\u0117 . 2015. On the relation between accuracy and fairness in binary classification. arXiv preprint arXiv:1505.05723 ( 2015 ). I \u017dliobait\u0117. 2015. On the relation between accuracy and fairness in binary classification. arXiv preprint arXiv:1505.05723 (2015)."},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-016-9182-5"}],"event":{"name":"ICAIL '19: Seventeenth International Conference on Artificial Intelligence and Law","location":"Montreal QC Canada","acronym":"ICAIL '19","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence","Univ. of Montreal University of Montreal","AAAI","IAAIL Intl Asso for Artifical Intel & Law"]},"container-title":["Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3322640.3326705","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3322640.3326705","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:53:58Z","timestamp":1750204438000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3322640.3326705"}},"subtitle":["Evidence from Risk Assessment for Juvenile Justice in Catalonia"],"short-title":[],"issued":{"date-parts":[[2019,6,17]]},"references-count":43,"alternative-id":["10.1145\/3322640.3326705","10.1145\/3322640"],"URL":"https:\/\/doi.org\/10.1145\/3322640.3326705","relation":{},"subject":[],"published":{"date-parts":[[2019,6,17]]},"assertion":[{"value":"2019-06-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}