{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T19:18:11Z","timestamp":1776885491201,"version":"3.51.2"},"publisher-location":"New York, NY, USA","reference-count":51,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,7,11]],"date-time":"2021-07-11T00:00:00Z","timestamp":1625961600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 61672012"],"award-info":[{"award-number":["No. 61672012"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2020AAA0107800"],"award-info":[{"award-number":["2020AAA0107800"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,7,11]]},"DOI":"10.1145\/3460319.3464820","type":"proceedings-article","created":{"date-parts":[[2021,7,8]],"date-time":"2021-07-08T22:18:43Z","timestamp":1625782723000},"page":"103-114","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":50,"title":["Efficient white-box fairness testing through gradient search"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6427-9587","authenticated-orcid":false,"given":"Lingfeng","family":"Zhang","sequence":"first","affiliation":[{"name":"East China Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5290-5691","authenticated-orcid":false,"given":"Yueling","family":"Zhang","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3152-4347","authenticated-orcid":false,"given":"Min","family":"Zhang","sequence":"additional","affiliation":[{"name":"East China Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,7,11]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201916)","author":"Abadi Mart\u00edn","year":"2016","unstructured":"Mart\u00edn Abadi , Paul Barham , Jianmin Chen , Zhifeng Chen , Andy Davis , Jeffrey Dean , Matthieu Devin , Sanjay Ghemawat , Geoffrey Irving , and Michael Isard . 2016 . Tensorflow: A system for large-scale machine learning . In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201916) . USENIX Association, USA. 265\u2013283. isbn:978 1931971331 https:\/\/dl.acm.org\/doi\/10.5555\/3026877.3026899 Mart\u00edn Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, and Michael Isard. 2016. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201916). USENIX Association, USA. 265\u2013283. isbn:9781931971331 https:\/\/dl.acm.org\/doi\/10.5555\/3026877.3026899"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338937"},{"key":"e_1_3_2_1_3_1","volume-title":"Machine bias: there\u2019s software used across the country to predict future criminals. and it\u2019s biased against blacks. propublica","author":"Angwin Julia","year":"2016","unstructured":"Julia Angwin , Jeff Larson , Surya Mattu , and Lauren Kirchner . 2016. Machine bias: there\u2019s software used across the country to predict future criminals. and it\u2019s biased against blacks. propublica 2016 . https:\/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias: there\u2019s software used across the country to predict future criminals. and it\u2019s biased against blacks. propublica 2016. https:\/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing"},{"key":"e_1_3_2_1_4_1","unstructured":"Richard Berk Hoda Heidari Shahin Jabbari Matthew Joseph Michael Kearns Jamie Morgenstern Seth Neel and Aaron Roth. 2017. A convex framework for fair regression. arXiv preprint arXiv:1706.02409 arxiv:1706.02409  Richard Berk Hoda Heidari Shahin Jabbari Matthew Joseph Michael Kearns Jamie Morgenstern Seth Neel and Aaron Roth. 2017. A convex framework for fair regression. arXiv preprint arXiv:1706.02409 arxiv:1706.02409"},{"key":"e_1_3_2_1_5_1","volume-title":"Davide Del Testa","author":"Bojarski Mariusz","year":"2016","unstructured":"Mariusz Bojarski , Davide Del Testa , Daniel Dworakowski, Bernhard Firner , Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, and Jiakai Zhang. 2016 . End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316, arxiv:1604.07316 Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, and Jiakai Zhang. 2016. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316, arxiv:1604.07316"},{"key":"e_1_3_2_1_6_1","volume-title":"Conference on fairness, accountability and transparency. 77\u201391","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru . 2018 . Gender shades: Intersectional accuracy disparities in commercial gender classification . In Conference on fairness, accountability and transparency. 77\u201391 . http:\/\/proceedings.mlr.press\/v81\/buolamwini18a.html Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. 77\u201391. http:\/\/proceedings.mlr.press\/v81\/buolamwini18a.html"},{"key":"e_1_3_2_1_7_1","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems. 3995\u20134004","author":"Calmon Flavio P","year":"2017","unstructured":"Flavio P Calmon , Dennis Wei , Bhanukiran Vinzamuri , Karthikeyan Natesan Ramamurthy , and Kush R Varshney . 2017 . Optimized pre-processing for discrimination prevention . In Proceedings of the 31st International Conference on Neural Information Processing Systems. 3995\u20134004 . https:\/\/dl.acm.org\/doi\/10.5555\/3294996.3295155 Flavio P Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. 2017. Optimized pre-processing for discrimination prevention. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 3995\u20134004. https:\/\/dl.acm.org\/doi\/10.5555\/3294996.3295155"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.49"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022627411411"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1089\/big.2016.0048"},{"key":"e_1_3_2_1_11_1","volume-title":"Amazon scraps secret AI recruiting tool that showed bias against women","author":"Dastin Jeffrey","year":"2018","unstructured":"Jeffrey Dastin . 2018. Amazon scraps secret AI recruiting tool that showed bias against women . San Fransico, CA : Reuters . Retrieved on October , 9 ( 2018 ), 2018. https:\/\/www.reuters.com\/article\/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G Jeffrey Dastin. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. San Fransico, CA: Reuters. Retrieved on October, 9 (2018), 2018. https:\/\/www.reuters.com\/article\/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1977.tb01600.x"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00957"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2020.3000681"},{"key":"e_1_3_2_1_15_1","volume-title":"Optimization and global minimization methods suitable for neural networks. Neural computing surveys, 2","author":"Duch W\u0142","year":"1998","unstructured":"W\u0142 odzis\u0142 aw Duch and Jerzy Korczak . 1998. Optimization and global minimization methods suitable for neural networks. Neural computing surveys, 2 ( 1998 ), 163\u2013212. W\u0142 odzis\u0142 aw Duch and Jerzy Korczak. 1998. Optimization and global minimization methods suitable for neural networks. Neural computing surveys, 2 (1998), 163\u2013212."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2090236.2090255"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-017-5663-3"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.2307\/1403797"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46675-0_53"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106277"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSPEC.2016.7473149"},{"key":"e_1_3_2_1_22_1","unstructured":"Ian J Goodfellow Jonathon Shlens and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 arxiv:1412.6572  Ian J Goodfellow Jonathon Shlens and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 arxiv:1412.6572"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-31865-1_25"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.5555\/3157382.3157469"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02985802"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.709601"},{"key":"e_1_3_2_1_27_1","unstructured":"Andrew Ilyas Logan Engstrom and Aleksander Madry. 2018. Prior convictions: Black-box adversarial attacks with bandits and priors. arXiv preprint arXiv:1807.07978 arxiv:1807.07978  Andrew Ilyas Logan Engstrom and Aleksander Madry. 2018. Prior convictions: Black-box adversarial attacks with bandits and priors. arXiv preprint arXiv:1807.07978 arxiv:1807.07978"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/DASC.2016.7778091"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33486-3_3"},{"key":"e_1_3_2_1_30_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, arxiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, arxiv:1412.6980 Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, arxiv:1412.6980"},{"key":"e_1_3_2_1_31_1","unstructured":"Alexey Kurakin Ian Goodfellow and Samy Bengio. 2016. Adversarial examples in the physical world. arxiv:1607.02533. arxiv:1607.02533  Alexey Kurakin Ian Goodfellow and Samy Bengio. 2016. Adversarial examples in the physical world. arxiv:1607.02533. arxiv:1607.02533"},{"key":"e_1_3_2_1_32_1","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems. 4069\u20134079","author":"Kusner Matt J","year":"2017","unstructured":"Matt J Kusner , Joshua R Loftus , Chris Russell , and Ricardo Silva . 2017 . Counterfactual fairness . In Proceedings of the 31st International Conference on Neural Information Processing Systems. 4069\u20134079 . https:\/\/dl.acm.org\/doi\/10.5555\/3294996.3295162 Matt J Kusner, Joshua R Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual fairness. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 4069\u20134079. https:\/\/dl.acm.org\/doi\/10.5555\/3294996.3295162"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/3468.618255"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2017.07.005"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1982.1056489"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2020\/64"},{"key":"e_1_3_2_1_37_1","unstructured":"Ninareh Mehrabi Fred Morstatter Nripsuta Saxena Kristina Lerman and Aram Galstyan. 2019. A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635 arxiv:1908.09635  Ninareh Mehrabi Fred Morstatter Nripsuta Saxena Kristina Lerman and Aram Galstyan. 2019. A survey on bias and fairness in machine learning. arXiv preprint arXiv:1908.09635 arxiv:1908.09635"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.282"},{"key":"e_1_3_2_1_39_1","unstructured":"Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Icml. https:\/\/dl.acm.org\/doi\/10.5555\/3104322.3104425  Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Icml. https:\/\/dl.acm.org\/doi\/10.5555\/3104322.3104425"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP.2016.36"},{"key":"e_1_3_2_1_41_1","unstructured":"Geoff Pleiss Manish Raghavan Felix Wu Jon Kleinberg and Kilian Q Weinberger. 2017. On fairness and calibration. 5684\u20135693. isbn:9781510860964 https:\/\/dl.acm.org\/doi\/10.5555\/3295222.3295319  Geoff Pleiss Manish Raghavan Felix Wu Jon Kleinberg and Kilian Q Weinberger. 2017. On fairness and calibration. 5684\u20135693. isbn:9781510860964 https:\/\/dl.acm.org\/doi\/10.5555\/3295222.3295319"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/0041-5553(64)90137-5"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3314248"},{"key":"e_1_3_2_1_44_1","volume-title":"Astraea: Grammar-based Fairness Testing. arXiv preprint arXiv:2010.02542, arxiv:2010.02542","author":"Soremekun Ezekiel","year":"2020","unstructured":"Ezekiel Soremekun , Sakshi Udeshi , and Sudipta Chattopadhyay . 2020 . Astraea: Grammar-based Fairness Testing. arXiv preprint arXiv:2010.02542, arxiv:2010.02542 Ezekiel Soremekun, Sakshi Udeshi, and Sudipta Chattopadhyay. 2020. Astraea: Grammar-based Fairness Testing. arXiv preprint arXiv:2010.02542, arxiv:2010.02542"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.2307\/1412107"},{"key":"e_1_3_2_1_46_1","volume-title":"Calculus with analytic geometry","author":"Swokowski Earl William","unstructured":"Earl William Swokowski . 1979. Calculus with analytic geometry . Taylor & Francis . Earl William Swokowski. 1979. Calculus with analytic geometry. Taylor & Francis."},{"key":"e_1_3_2_1_47_1","unstructured":"Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna Dumitru Erhan Ian Goodfellow and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 arxiv:1312.6199  Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna Dumitru Erhan Ian Goodfellow and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 arxiv:1312.6199"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380400"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238165"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380331"}],"event":{"name":"ISSTA '21: 30th ACM SIGSOFT International Symposium on Software Testing and Analysis","location":"Virtual Denmark","acronym":"ISSTA '21","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"]},"container-title":["Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460319.3464820","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3460319.3464820","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:48:31Z","timestamp":1750193311000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460319.3464820"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,11]]},"references-count":51,"alternative-id":["10.1145\/3460319.3464820","10.1145\/3460319"],"URL":"https:\/\/doi.org\/10.1145\/3460319.3464820","relation":{},"subject":[],"published":{"date-parts":[[2021,7,11]]},"assertion":[{"value":"2021-07-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}