{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T23:22:53Z","timestamp":1783466573220,"version":"3.55.0"},"reference-count":88,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2022,3,30]],"date-time":"2022-03-30T00:00:00Z","timestamp":1648598400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,30]],"date-time":"2022-03-30T00:00:00Z","timestamp":1648598400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"University of Oulu including Oulu University Hospital"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2022,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Context<\/jats:title>\n                <jats:p>Machine learning (ML) software systems are permeating many aspects of our life, such as healthcare, transportation, banking, and recruitment. These systems are trained with data that is often biased, resulting in biased behaviour. To address this issue, fairness testing approaches have been proposed to test ML systems for fairness, which predominantly focus on assessing classification-based ML systems. These methods are not applicable to regression-based systems, for example, they do not quantify the magnitude of the disparity in predicted outcomes, which we identify as important in the context of regression-based ML systems.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Method:<\/jats:title>\n                <jats:p>We conduct this study as design science research. We identify the problem instance in the context of emergency department (ED) wait-time prediction. In this paper, we develop an effective and efficient fairness testing approach to evaluate the fairness of regression-based ML systems. We propose fairness degree, which is a new fairness measure for regression-based ML systems, and a novel search-based fairness testing (SBFT) approach for testing regression-based machine learning systems. We apply the proposed solutions to ED wait-time prediction software.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results:<\/jats:title>\n                <jats:p>We experimentally evaluate the effectiveness and efficiency of the proposed approach with ML systems trained on real observational data from the healthcare domain. We demonstrate that SBFT significantly outperforms existing fairness testing approaches, with up to 111% and 190% increase in effectiveness and efficiency of SBFT compared to the best performing existing approaches.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion:<\/jats:title>\n                <jats:p>These findings indicate that our novel fairness measure and the new approach for fairness testing of regression-based ML systems can identify the degree of fairness in predictions, which can help software teams to make data-informed decisions about whether such software systems are ready to deploy. The scientific knowledge gained from our work can be phrased as a technological rule; to measure the fairness of the regression-based ML systems in the context of emergency department wait-time prediction use fairness degree and search-based techniques to approximate it.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s10664-022-10116-7","type":"journal-article","created":{"date-parts":[[2022,3,30]],"date-time":"2022-03-30T11:02:38Z","timestamp":1648638158000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Search-based fairness testing for regression-based machine learning systems"],"prefix":"10.1007","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5080-9276","authenticated-orcid":false,"given":"Anjana","family":"Perera","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aldeida","family":"Aleti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chakkrit","family":"Tantithamthavorn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jirayus","family":"Jiarpakdee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Burak","family":"Turhan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lisa","family":"Kuhn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Katie","family":"Walker","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,3,30]]},"reference":[{"key":"10116_CR1","doi-asserted-by":"crossref","unstructured":"Aggarwal A, Lohia P, Nagar S, Dey K, Saha D (2019) Black box fairness testing of machine learning models. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp 625\u2013635","DOI":"10.1145\/3338906.3338937"},{"key":"10116_CR2","doi-asserted-by":"crossref","unstructured":"Alshahwan N, Gao X, Harman M, Jia Y, Mao K, Mols A, Tei T, Zorin I (2018) Deploying search based software engineering with sapienz at facebook. In: International Symposium on Search Based Software Engineering. Springer, pp 3\u201345","DOI":"10.1007\/978-3-319-99241-9_1"},{"key":"10116_CR3","unstructured":"Angwin J, Larson J, Mattu S, Kirchner L (2016) Machine bias. Propublica"},{"issue":"3","key":"10116_CR4","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1002\/stvr.1486","volume":"24","author":"A Arcuri","year":"2014","unstructured":"Arcuri A, Briand L (2014) A hitchhiker\u2019s guide to statistical tests for assessing randomized algorithms in software engineering. Softw Test Verif Reliab 24 (3):219\u2013250","journal-title":"Softw Test Verif Reliab"},{"issue":"2","key":"10116_CR5","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1016\/j.jacc.2017.05.024","volume":"70","author":"CN Bairey Merz","year":"2017","unstructured":"Bairey Merz C N, Andersen H, Sprague E, Burns A, Keida M, Walsh M N, Greenberger P, Campbell S, Pollin I, McCullough C, Brown N, Jenkins M, Redberg R, Johnson P, Robinson B (2017) Knowledge, attitudes, and beliefs regarding cardiovascular disease in women: The women\u2019s heart alliance. J Am Coll Cardiol 70(2):123\u2013132. [Online]. Available: https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0735109717374077","journal-title":"J Am Coll Cardiol"},{"key":"10116_CR6","unstructured":"Barocas S, Hardt M, Narayanan A (2018) Fairness and machine learning. fairmlbook.org"},{"issue":"4\/5","key":"10116_CR7","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1147\/JRD.2019.2942287","volume":"63","author":"RKE Bellamy","year":"2019","unstructured":"Bellamy RKE, Dey K, Hind M, Hoffman S C, Houde S, Kannan K, Lohia P, Martino J, Mehta S, Mojsilovi\u0107 A et al (2019) Ai fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM J Res Dev 63(4\/5):4\u20131","journal-title":"IBM J Res Dev"},{"key":"10116_CR8","doi-asserted-by":"crossref","unstructured":"Bendick M (2007) Situation Testing for Employment Discrimination in the United States of America. [Online; accessed 29-November-2021]. [Online]. Available: https:\/\/www.cairn.info\/revue-horizons-strategiques-2007-3-page-17.htmhttps:\/\/www.cairn.info\/revue-horizons-strategiques-2007-3-page-17.htm","DOI":"10.3917\/hori.005.0017"},{"key":"10116_CR9","unstructured":"Berk R, Heidari H, Jabbari S, Joseph M, Kearns M, Morgenstern J, Neel S, Roth A (2017) A convex framework for fair regression. arXiv:1706.02409"},{"key":"10116_CR10","unstructured":"Binns R (2018) Fairness in machine learning: Lessons from political philosophy. In: Conference on Fairness, Accountability and Transparency. PMLR, pp 149\u2013159"},{"key":"10116_CR11","unstructured":"Bishop P G, Bloomfield R E (2002) Worst case reliability prediction based on a prior estimate of residual defects. In: 13th International Symposium on Software Reliability Engineering, 2002. Proceedings. IEEE, pp 295\u2013303"},{"key":"10116_CR12","doi-asserted-by":"crossref","unstructured":"Biswas S, Rajan H (2020) Do the machine learning models on a crowd sourced platform exhibit bias? an empirical study on model fairness. arXiv:2005.12379","DOI":"10.1145\/3368089.3409704"},{"key":"10116_CR13","unstructured":"Calmon F, Wei D, Vinzamuri B, Ramamurthy K N, Varshney K R (2017) Optimized pre-processing for discrimination prevention. In: Advances in Neural Information Processing Systems, pp 3992\u20134001"},{"key":"10116_CR14","unstructured":"Caton S, Haas C (2020) Fairness in machine learning: A survey"},{"key":"10116_CR15","doi-asserted-by":"crossref","unstructured":"Chakraborty J, Majumder S, Menzies T (2021) Bias in machine learning software: Why? how? what to do? arXiv:2105.12195","DOI":"10.1145\/3468264.3468537"},{"key":"10116_CR16","doi-asserted-by":"crossref","unstructured":"Chakraborty J, Majumder S, Yu Z, Menzies T (2020) Fairway: A way to build fair ml software. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp 654\u2013665","DOI":"10.1145\/3368089.3409697"},{"key":"10116_CR17","doi-asserted-by":"crossref","unstructured":"Chiappa S (2019) Path-specific counterfactual fairness. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 33, pp 7801\u20137808","DOI":"10.1609\/aaai.v33i01.33017801"},{"key":"10116_CR18","unstructured":"Chouldechova A, Benavides-Prado D, Fialko O, Vaithianathan R (2018) A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In: Conference on Fairness, Accountability and Transparency. PMLR, pp 134\u2013148"},{"key":"10116_CR19","unstructured":"Corbett-Davies S, Goel S (2018) The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv:1808.00023"},{"key":"10116_CR20","doi-asserted-by":"crossref","unstructured":"Corbett-Davies S, Pierson E, Feller A, Goel S, Huq A (2017) Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining, pp 797\u2013806","DOI":"10.1145\/3097983.3098095"},{"issue":"1","key":"10116_CR21","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1109\/TSE.2005.12","volume":"31","author":"V Cortellessa","year":"2005","unstructured":"Cortellessa V, Goseva-Popstojanova K, Appukkutty K, Guedem AR, Hassan A, Elnaggar R, Abdelmoez W, Ammar HH (2005) Model-based performance risk analysis. IEEE Trans Softw Eng 31(1):3\u201320","journal-title":"IEEE Trans Softw Eng"},{"key":"10116_CR22","doi-asserted-by":"crossref","unstructured":"Del Grosso C, Antoniol G, Di Penta M, Galinier P, Merlo E (2005) Improving network applications security: a new heuristic to generate stress testing data. In: Proceedings of the 7th annual conference on Genetic and evolutionary computation, pp 1037\u20131043","DOI":"10.1145\/1068009.1068185"},{"issue":"2","key":"10116_CR23","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1007\/s11739-014-1154-8","volume":"10","author":"S Di Somma","year":"2015","unstructured":"Di Somma S, Paladino L, Vaughan L, Lalle I, Magrini L, Magnanti M (2015) Overcrowding in emergency department: an international issue. Internal Emerg Med 10(2):171\u2013175. [Online]. Available: https:\/\/doi.org\/10.1007\/s11739-014-1154-8","journal-title":"Internal Emerg Med"},{"issue":"7.4","key":"10116_CR24","first-page":"1","volume":"7","author":"W Dieterich","year":"2016","unstructured":"Dieterich W, Mendoza C, Brennan T (2016) Compas risk scales: Demonstrating accuracy equity and predictive parity. Northpoint Inc 7(7.4):1","journal-title":"Northpoint Inc"},{"key":"10116_CR25","doi-asserted-by":"crossref","unstructured":"Dwork C, Hardt M, Pitassi T, Reingold O, Zemel R (2012) Fairness through awareness. In: Proceedings of the 3rd innovations in theoretical computer science conference, pp 214\u2013226","DOI":"10.1145\/2090236.2090255"},{"key":"10116_CR26","doi-asserted-by":"crossref","unstructured":"Feldman M, Friedler S A, Moeller J, Scheidegger C, Venkatasubramanian S (2015) Certifying and removing disparate impact. In: proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp 259\u2013268","DOI":"10.1145\/2783258.2783311"},{"key":"10116_CR27","unstructured":"Ferral K Wisconsin supreme court allows state to continue using computer program to assist in sentencing. the capital times. [Online; accessed 9-January- 2020]. [Online]. Available: http:\/\/host.madison.com\/ct\/news\/local\/govt-and-politics\/wisconsin-supreme-court-allows-state-to-continue-using-computer-program\/article7eb67874-bf40-59e3-b62a-923d1626fa0f.htmlhttp:\/\/host.madison.com\/ct\/news\/local\/govt-and-politics\/wisconsin-supreme-court-allows-state-to-continue-using-computer-program\/article7eb67874-bf40-59e3-b62a-923d1626fa0f.htmlhttp:\/\/host.madison.com\/ct\/news\/local\/govt-and-politics\/wisconsin-supreme-court-allows-state-to-continue-using-computer-program\/article7eb67874-bf40-59e3-b62a-923d1626fa0f.html"},{"key":"10116_CR28","doi-asserted-by":"crossref","unstructured":"Fraser G, Arcuri A (2013) Evosuite: On the challenges of test case generation in the real world. In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation. IEEE, pp 362\u2013369","DOI":"10.1109\/ICST.2013.51"},{"issue":"3","key":"10116_CR29","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1145\/230538.230561","volume":"14","author":"B Friedman","year":"1996","unstructured":"Friedman B, Nissenbaum H (1996) Bias in computer systems. ACM Trans Inf Syst 14(3):330\u2013347","journal-title":"ACM Trans Inf Syst"},{"key":"10116_CR30","doi-asserted-by":"crossref","unstructured":"Galhotra S, Brun Y, Meliou A (2017) Fairness testing: testing software for discrimination. In: Joint Meeting on Foundations of Software Engineering (FSE). ACM, pp 498\u2013510","DOI":"10.1145\/3106237.3106277"},{"key":"10116_CR31","unstructured":"Ghaffary S (2019) The algorithms that detect hate speech online are biased against black people. [Online; accessed 14-October-2020]. [Online]. Available: https:\/\/www.vox.com\/recode\/2019\/8\/15\/20806384\/social-media-hate-speech-bias-black-african-american-facebook-twitterhttps:\/\/www.vox.com\/recode\/2019\/8\/15\/20806384\/social-media-hate-speech-bias-black-african-american-facebook-twitter"},{"key":"10116_CR32","unstructured":"Grgic-Hlaca N, Zafar M B, Gummadi K P, Weller A (2016) The case for process fairness in learning: Feature selection for fair decision making. In: NIPS Symposium on Machine Learning and the Law, vol 1, p 2"},{"key":"10116_CR33","unstructured":"Hardawar D (2012) Staples, home depot, and other online stores change prices based on your location. [Online; accessed 14-October-2020]. [Online]. Available: https:\/\/venturebeat.com\/2012\/12\/24\/staples-online-stores-price-changes\/https:\/\/venturebeat.com\/2012\/12\/24\/staples-online-stores-price-changes\/"},{"key":"10116_CR34","first-page":"3315","volume":"29","author":"M Hardt","year":"2016","unstructured":"Hardt M, Price E, Srebro N (2016) Equality of opportunity in supervised learning. Adv Neural Inf Process Syst 29:3315\u20133323","journal-title":"Adv Neural Inf Process Syst"},{"key":"10116_CR35","unstructured":"Hern A (2020) Twitter apologises for \u2019racist\u2019 image-cropping algorithm. [Online; accessed 7-August-2021]. [Online]. Available: https:\/\/www.theguardian.com\/technology\/2020\/sep\/21\/twitter-apologises-for-racist-image-cropping-algorithmhttps:\/\/www.theguardian.com\/technology\/2020\/sep\/21\/twitter-apologises-for-racist-image-cropping-algorithm"},{"key":"10116_CR36","doi-asserted-by":"crossref","unstructured":"Horkoff J (2019) Non-functional requirements for machine learning: Challenges and new directions. In: 2019 IEEE 27th International Requirements Engineering Conference (RE). IEEE, pp 386\u2013391","DOI":"10.1109\/RE.2019.00050"},{"key":"10116_CR37","unstructured":"Ingold D, Soper S (2016) Amazon doesn\u2019t consider the race of its customers. should it? [Online; accessed 14-October-2020]. [Online]. Available: https:\/\/www.bloomberg.com\/graphics\/2016-amazon-same-day\/"},{"key":"10116_CR38","unstructured":"Ingold D, Soper S (2016) Amazon doesn\u2019t consider the race of its customers. should it? Bloomberg News"},{"key":"10116_CR39","unstructured":"Johnson B, Bartola J, Angell R, Keith K, Witty S, Giguere S J, Brun Y (2020) Fairkit, fairkit, on the wall, who\u2019s the fairest of them all? supporting data scientists in training fair models. arXiv:2012.09951"},{"issue":"6","key":"10116_CR40","doi-asserted-by":"publisher","first-page":"239","DOI":"10.5694\/mja15.00812","volume":"204","author":"CP Juergens","year":"2016","unstructured":"Juergens C P, Dabin B, French J K, Kritharides L, Hyun K, Kilian J, Chew Derek PB, Brieger D (2016) English as a second language and outcomes of patients presenting with acute coronary syndromes: results from the concordance registry. Med J Aust 204(6):239\u2013239. [Online]. Available: https:\/\/doi.org\/10.5694\/mja15.00812","journal-title":"Med J Aust"},{"issue":"1","key":"10116_CR41","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10115-011-0463-8","volume":"33","author":"F Kamiran","year":"2012","unstructured":"Kamiran F, Calders T (2012) Data preprocessing techniques for classification without discrimination. Knowl Inf Syst 33(1):1\u201333","journal-title":"Knowl Inf Syst"},{"key":"10116_CR42","doi-asserted-by":"crossref","unstructured":"Kamishima T, Akaho S, Sakuma J (2011) Fairness-aware learning through regularization approach. In: 2011 IEEE 11th International Conference on Data Mining Workshops. IEEE, pp 643\u2013650","DOI":"10.1109\/ICDMW.2011.83"},{"issue":"6","key":"10116_CR43","doi-asserted-by":"publisher","first-page":"1789","DOI":"10.1109\/TIFS.2012.2214212","volume":"7","author":"BF Klare","year":"2012","unstructured":"Klare B F, Burge M J, Klontz J C, Bruegge R W V, Jain A K (2012) Face recognition performance: Role of demographic information. IEEE Trans Inf Forensic Secur 7(6):1789\u20131801","journal-title":"IEEE Trans Inf Forensic Secur"},{"issue":"1","key":"10116_CR44","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1016\/j.infsof.2012.03.009","volume":"55","author":"K Lakhotia","year":"2013","unstructured":"Lakhotia K, Harman M, Gross H (2013) Austin: An open source tool for search based software testing of c programs. Inf Softw Technol 55(1):112\u2013125","journal-title":"Inf Softw Technol"},{"key":"10116_CR45","doi-asserted-by":"crossref","unstructured":"Ledford H (2019) Millions of black people affected by racial bias in health-care algorithms. [Online; accessed 7-August-2021]. [Online]. Available: https:\/\/www.nature.com\/articles\/d41586-019-03228-6","DOI":"10.1038\/d41586-019-03228-6"},{"key":"10116_CR46","first-page":"43","volume":"3","author":"M L\u00f3pez-Ib\u00e1nez","year":"2016","unstructured":"L\u00f3pez-Ib\u00e1nez M, Dubois-Lacoste J, C\u00e1ceres L P, Birattari M, St\u00fctzle T (2016) The irace package: Iterated racing for automatic algorithm configuration. Oper Res Perspect 3:43\u201358","journal-title":"Oper Res Perspect"},{"key":"10116_CR47","doi-asserted-by":"crossref","unstructured":"Luong B T, Ruggieri S, Turini F (2011) k-nn as an implementation of situation testing for discrimination discovery and prevention. In: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 502\u2013510","DOI":"10.1145\/2020408.2020488"},{"key":"10116_CR48","unstructured":"Mahdawi A (2018) Is your friend getting a cheaper Uber fare than you are? [Online; accessed 7-August-2021]. [Online]. Available: https:\/\/www.theguardian.com\/commentisfree\/2018\/apr\/13\/uber-lyft-prices-personalized-datahttps:\/\/www.theguardian.com\/commentisfree\/2018\/apr\/13\/uber-lyft-prices-personalized-data"},{"key":"10116_CR49","doi-asserted-by":"crossref","unstructured":"Mao K, Harman M, Jia Y (2016) Sapienz: Multi-objective automated testing for android applications. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, pp 94\u2013105","DOI":"10.1145\/2931037.2931054"},{"key":"10116_CR50","unstructured":"Mattioli D (2012) On Orbitz, Mac Users Steered to Pricier Hotels. [Online; accessed 9-January-2020]. [Online]. Available: http:\/\/www.wsj.com\/articles\/SB10001424052702304458604577488822667325882"},{"issue":"2","key":"10116_CR51","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1002\/stvr.294","volume":"14","author":"P McMinn","year":"2004","unstructured":"McMinn P (2004) Search-based software test data generation: a survey. Softw Test Verif Reliab 14(2):105\u2013156","journal-title":"Softw Test Verif Reliab"},{"issue":"13","key":"10116_CR52","doi-asserted-by":"publisher","first-page":"1302","DOI":"10.1161\/CIR.0000000000000381","volume":"133","author":"JC McSweeney","year":"2016","unstructured":"McSweeney J C, Rosenfeld A G, Abel W M, Braun L T, Burke L E, Daugherty S L, Fletcher G F, Gulati M, Mehta L S, Pettey C, Reckelhoff J F (2016) Preventing and experiencing ischemic heart disease as a woman: State of the science. Circulation 133(13):1302\u20131331. [Online]. Available: https:\/\/doi.org\/10.1161\/CIR.0000000000000381","journal-title":"Circulation"},{"key":"10116_CR53","unstructured":"Mehrabi N, Morstatter F, Saxena N, Lerman K, Galstyan A (2019) A survey on bias and fairness in machine learning. arXiv:1908.09635"},{"issue":"9","key":"10116_CR54","doi-asserted-by":"publisher","first-page":"916","DOI":"10.1161\/CIR.0000000000000351","volume":"133","author":"LS Mehta","year":"2016","unstructured":"Mehta L S, Beckie T M, DeVon H A, Grines C L, Krumholz H M, Johnson M N, Lindley K J, Vaccarino V, Wang T Y, Watson K E, Wenger N K (2016) Acute myocardial infarction in women. Circulation 133(9):916\u2013947. [Online]. Available: https:\/\/doi.org\/10.1161\/CIR.0000000000000351","journal-title":"Circulation"},{"key":"10116_CR55","unstructured":"Mullainathan S (2019) Biased algorithms are easier to fix than biased people, www.nytimes.com\/2019\/12\/06\/business\/algorithm-bias-fix.html, accessed: 10\/12\/2019."},{"key":"10116_CR56","unstructured":"Olson P (2011) CNN Money, The algorithm that beats your bank manager. http:\/\/www.forbes.com\/sites\/parmyolson\/2011\/03\/15\/the-algorithm-that-beats-your-bank-manager\/#cd84e4f77ca8, [Accessed 9\/11\/2020]"},{"key":"10116_CR57","doi-asserted-by":"crossref","unstructured":"Panichella A, Kifetew F M, Tonella P (2015) Reformulating branch coverage as a many-objective optimization problem. In: 2015 IEEE 8th international conference on software testing, verification and validation (ICST). IEEE, pp 1\u201310","DOI":"10.1109\/ICST.2015.7102604"},{"key":"10116_CR58","doi-asserted-by":"publisher","first-page":"236","DOI":"10.1016\/j.infsof.2018.08.009","volume":"104","author":"A Panichella","year":"2018","unstructured":"Panichella A, Kifetew F M, Tonella P (2018) A large scale empirical comparison of state-of-the-art search-based test case generators. Inf Softw Technol 104:236\u2013256","journal-title":"Inf Softw Technol"},{"key":"10116_CR59","doi-asserted-by":"crossref","unstructured":"Perera A, Aleti A, B\u00f6hme M, Turhan B (2020) Defect prediction guided search-based software testing. In: Proceedings of the 35th IEEE\/ACM International Conference on Automated Software Engineering. ACM","DOI":"10.1145\/3324884.3416612"},{"issue":"2-3","key":"10116_CR60","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1023\/A:1008119029962","volume":"18","author":"P Puschner","year":"2000","unstructured":"Puschner P, Burns A (2000) Guest editorial: A review of worst-case execution-time analysis. Real-Time Syst 18(2-3):115\u2013128","journal-title":"Real-Time Syst"},{"key":"10116_CR61","doi-asserted-by":"publisher","first-page":"440","DOI":"10.1109\/TSE.1980.230492","volume":"5","author":"CV Ramamoorthy","year":"1980","unstructured":"Ramamoorthy CV, Ho G S (1980) Performance evaluation of asynchronous concurrent systems using petri nets. IEEE Trans Softw Eng 5:440\u2013449","journal-title":"IEEE Trans Softw Eng"},{"key":"10116_CR62","doi-asserted-by":"crossref","unstructured":"Ribeiro M T, Singh S, Guestrin C (2016) \u201dwhy should i trust you?\u201d explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135\u20131144","DOI":"10.1145\/2939672.2939778"},{"key":"10116_CR63","doi-asserted-by":"crossref","unstructured":"Runeson P, Engstr\u00f6m E, Storey M-A (2020) The design science paradigm as a frame for empirical software engineering. In: Contemporary empirical methods in software engineering. Springer, pp 127\u2013147","DOI":"10.1007\/978-3-030-32489-6_5"},{"key":"10116_CR64","doi-asserted-by":"crossref","unstructured":"Selbst A D, Boyd D, Friedler S A, Venkatasubramanian S, Vertesi J (2019) Fairness and abstraction in sociotechnical systems. In: Proceedings of the conference on fairness, accountability, and transparency, pp 59\u201368","DOI":"10.1145\/3287560.3287598"},{"issue":"2","key":"10116_CR65","doi-asserted-by":"publisher","first-page":"31","DOI":"10.35680\/2372-0247.1090","volume":"2","author":"S Shah","year":"2015","unstructured":"Shah S, Patel A, Rumoro D P, Hohmann S, Fullam F (2015) Managing patient expectations at emergency department triage. Patient Exper J 2 (2):31\u201344","journal-title":"Patient Exper J"},{"key":"10116_CR66","unstructured":"Sharkey A (2020) Care robots for the elderly are dangerous. [Online; accessed 14-October-2020]. [Online]. Available: https:\/\/www.telegraph.co.uk\/science\/2016\/05\/30\/care-bots-for-the-elderly-are-dangerous-warns-artificial-intelli"},{"key":"10116_CR67","doi-asserted-by":"crossref","unstructured":"Sharma A, Wehrheim H (2019) Testing machine learning algorithms for balanced data usage. In: 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST). IEEE, pp 125\u2013135","DOI":"10.1109\/ICST.2019.00022"},{"key":"10116_CR68","doi-asserted-by":"publisher","first-page":"1470","DOI":"10.2307\/4093259","volume":"117","author":"RB Siegel","year":"2003","unstructured":"Siegel R B (2003) Equality talk: Antisubordination and anticlassification values in constitutional struggles over brown. Harv L Rev 117:1470","journal-title":"Harv L Rev"},{"issue":"6","key":"10116_CR69","doi-asserted-by":"publisher","first-page":"686","DOI":"10.1016\/j.jemermed.2011.01.018","volume":"41","author":"OA Soremekun","year":"2011","unstructured":"Soremekun O A, Takayesu J K, Bohan S J (2011) Framework for analyzing wait times and other factors that impact patient satisfaction in the emergency department. J Emerg Med 41(6):686\u2013692","journal-title":"J Emerg Med"},{"issue":"1","key":"10116_CR70","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/j.hlc.2020.06.025","volume":"30","author":"J Stehli","year":"2021","unstructured":"Stehli J, Duffy S J, Burgess S, Kuhn L, Gulati M, Chow C, Zaman S (2021) Sex disparities in myocardial infarction: biology or bias?. Heart Lung Circul 30(1):18\u201326","journal-title":"Heart Lung Circul"},{"issue":"6","key":"10116_CR71","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1109\/MSPEC.2016.7473150","volume":"53","author":"E Strickland","year":"2016","unstructured":"Strickland E (2016) Doc bot preps for the or. IEEE Spectr 53 (6):32\u201360","journal-title":"IEEE Spectr"},{"key":"10116_CR72","doi-asserted-by":"crossref","unstructured":"Strobel S, Ren K Y, Dragoman A, Pettit C, Stancati A, Kallergis D, Smith M, Sidhu K, Rutledge G, Mondoux S (2021) Do patients respond to posted emergency department wait times: Time-series evidence from the implementation of a wait time publication system in hamilton, canada. Ann Emerg Med","DOI":"10.1016\/j.annemergmed.2021.04.009"},{"issue":"1","key":"10116_CR73","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12889-016-3954-4","volume":"17","author":"J Sun","year":"2017","unstructured":"Sun J, Lin Q, Zhao P, Zhang Q, Xu K, Chen H, Hu C J, Stuntz M, Li H, Liu Y (2017) Reducing waiting time and raising outpatient satisfaction in a chinese public tertiary general hospital-an interrupted time series study. BMC Public Health 17(1):1\u201311","journal-title":"BMC Public Health"},{"key":"10116_CR74","doi-asserted-by":"crossref","unstructured":"Tatman R (2017) Gender and dialect bias in youtube\u2019s automatic captions. In: Proceedings of the First ACL Workshop on Ethics in Natural Language Processing, pp 53\u201359","DOI":"10.18653\/v1\/W17-1606"},{"key":"10116_CR75","doi-asserted-by":"crossref","unstructured":"Tramer F, Atlidakis V, Geambasu R, Hsu D, Hubaux J-P, Humbert M, Juels A, Lin H (2017) Fairtest: Discovering unwarranted associations in data-driven applications. In: 2017 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, pp 401\u2013416","DOI":"10.1109\/EuroSP.2017.29"},{"issue":"6","key":"10116_CR76","doi-asserted-by":"publisher","first-page":"758","DOI":"10.1002\/clc.22938","volume":"41","author":"JA Udell","year":"2018","unstructured":"Udell J A, Fonarow G C, Maddox T M, Cannon C P, Frank Peacock W, Laskey W K, Grau-Sepulveda M V, Smith E E, Hernandez A F, Peterson E D et al (2018) Sustained sex-based treatment differences in acute coronary syndrome care: insights from the american heart association get with the guidelines coronary artery disease registry. Clin Cardiol 41(6):758\u2013768","journal-title":"Clin Cardiol"},{"key":"10116_CR77","doi-asserted-by":"crossref","unstructured":"Udeshi S, Arora P, Chattopadhyay S (2018) Automated directed fairness testing. In: Proceedings of the 33rd ACM\/IEEE International Conference on Automated Software Engineering, pp 98\u2013108","DOI":"10.1145\/3238147.3238165"},{"issue":"2","key":"10116_CR78","first-page":"101","volume":"25","author":"A Vargha","year":"2000","unstructured":"Vargha A, Delaney H D (2000) A critique and improvement of the cl common language effect size statistics of mcgraw and wong. J Educ Behav Stat 25 (2):101\u2013132","journal-title":"J Educ Behav Stat"},{"key":"10116_CR79","doi-asserted-by":"crossref","unstructured":"Vogel B, Acevedo M, Appelman Y, Merz C N B, Chieffo A, Figtree G A, Guerrero M, Kunadian V, Lam CSP, Maas AHEM et al (2021) The lancet women and cardiovascular disease commission: reducing the global burden by 2030. The Lancet","DOI":"10.1016\/S0140-6736(21)00684-X"},{"key":"10116_CR80","doi-asserted-by":"crossref","unstructured":"Walker K, Jiarpakdee J, Loupis A, Tantithamthavorn C, Joe K, Ben-Meir M, Akhlaghi H, Hutton J, Wang W, Stephenson M, Blecher G, Buntine P, Sweeny A, Turhan B (2021) On behalf of the Australasian College for Emergency Medicine, Emergency medicine patient wait time multivariable prediction models: a multicentre derivation and validation study. medRxiv, [Online]. Available: https:\/\/www.medrxiv.org\/content\/early\/2021\/03\/24\/2021.03.19.21253921","DOI":"10.1101\/2021.03.19.21253921"},{"key":"10116_CR81","doi-asserted-by":"crossref","unstructured":"Walker K, Stephenson M, Loupis A, Ben-Meir M, Joe K, Stephenson M, Lowthian J, Yip B, Wu E, Hansen K et al (2020) Displaying emergency patient estimated wait times: A multi-centre, qualitative study of patient, community, paramedic and health administrator perspectives. Emergency Medicine Australasia","DOI":"10.1111\/1742-6723.13640"},{"key":"10116_CR82","doi-asserted-by":"publisher","first-page":"901","DOI":"10.1016\/j.ijcard.2016.06.244","volume":"220","author":"K Wechkunanukul","year":"2016","unstructured":"Wechkunanukul K, Grantham H, Teubner D, Hyun K K, Clark R A (2016) Presenting characteristics and processing times for culturally and linguistically diverse (cald) patients with chest pain in an emergency department: Time, ethnicity, and delay (ted) study ii. Int J Cardiol 220:901\u2013908","journal-title":"Int J Cardiol"},{"key":"10116_CR83","doi-asserted-by":"crossref","unstructured":"Wegener J, B\u00fchler O (2004) Evaluation of different fitness functions for the evolutionary testing of an autonomous parking system. In: Genetic and Evolutionary Computation Conference. Springer, pp 1400\u20131412","DOI":"10.1007\/978-3-540-24855-2_160"},{"key":"10116_CR84","doi-asserted-by":"crossref","unstructured":"Zhang B H, Lemoine B, Mitchell M (2018) Mitigating unwanted biases with adversarial learning. In: Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society, pp 335\u2013340","DOI":"10.1145\/3278721.3278779"},{"key":"10116_CR85","doi-asserted-by":"crossref","unstructured":"Zhang J M, Harman M (2021) \u2019ignorance and prejudice\u2019in software fairness. In: 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, pp 1436\u20131447","DOI":"10.1109\/ICSE43902.2021.00129"},{"key":"10116_CR86","unstructured":"Zhang J M, Harman M, Ma L, Liu Y (2020) Machine learning testing: Survey, landscapes and horizons. IEEE Trans Softw Eng"},{"key":"10116_CR87","doi-asserted-by":"crossref","unstructured":"Zhang L, Wu Y, Wu X (2016) Situation testing-based discrimination discovery: A causal inference approach","DOI":"10.1007\/978-3-319-39931-7_9"},{"key":"10116_CR88","doi-asserted-by":"crossref","unstructured":"Zhang P, Wang J, Sun J, Dong G, Wang X, Wang X, Dong J S, Dai T (2020) White-box fairness testing through adversarial sampling. In: Proceedings of the ACM\/IEEE 42nd International Conference on Software Engineering, pp 949\u2013960","DOI":"10.1145\/3377811.3380331"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10116-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-022-10116-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10116-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,23]],"date-time":"2022-04-23T10:22:42Z","timestamp":1650709362000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-022-10116-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,30]]},"references-count":88,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,5]]}},"alternative-id":["10116"],"URL":"https:\/\/doi.org\/10.1007\/s10664-022-10116-7","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,30]]},"assertion":[{"value":"29 December 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 March 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The study received Monash Health ethics committee approval (RES-19-0000-763A).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}},{"value":"The authors have no conflicts of interest to declare.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Conflict of Interests"}}],"article-number":"79"}}