{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T16:07:46Z","timestamp":1782317266073,"version":"3.54.5"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,10,18]],"date-time":"2022-10-18T00:00:00Z","timestamp":1666051200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,18]],"date-time":"2022-10-18T00:00:00Z","timestamp":1666051200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["AI &amp; Soc"],"published-print":{"date-parts":[[2023,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In recent years, the work of organizations in the area of digitization has intensified significantly. This trend is also evident in the field of recruitment where job application tracking systems (ATS) have been developed to allow job advertisements to be published online. However, recent studies have shown that recruiting in most organizations is not inclusive, being subject to human biases and prejudices. Most discrimination activities appear early but subtly in the hiring process, for instance, exclusive phrasing in job advertisement discourages qualified applicants from minority groups from applying. The existing works are limited to analyzing, categorizing and highlighting the occurrence of bias in the recruitment process. In this paper, we go beyond this and develop machine learning models for identifying and classifying biased and discriminatory language in job descriptions. We develop and evaluate a machine learning system for identifying five major categories of biased and discriminatory language in job advertisements, i.e., masculine-coded, feminine-coded, exclusive, LGBTQ-coded, demographic and racial language. We utilized the combination of linguistic features with recent state-of-the-art word embeddings representations as input features for various machine learning classifiers. Our results show that the machine learning classifiers were able to identify all the five categories of biased and discriminatory language with a decent accuracy. The Random Forest classifier with FastText word embeddings achieved the best performance with tenfolds cross-validation. Our system directly addresses the bias in the attraction phase of hiring by identifying and classifying biased and discriminatory language and thus encouraging recruiters to write more inclusive job advertisements.<\/jats:p>","DOI":"10.1007\/s00146-022-01574-0","type":"journal-article","created":{"date-parts":[[2022,10,18]],"date-time":"2022-10-18T06:02:39Z","timestamp":1666072959000},"page":"1025-1038","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":38,"title":["A machine learning approach to recognize bias and discrimination in job advertisements"],"prefix":"10.1007","volume":"38","author":[{"given":"Richard","family":"Frissen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kolawole John","family":"Adebayo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rohan","family":"Nanda","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,10,18]]},"reference":[{"key":"1574_CR1","unstructured":"Adebayo KJ, Di Caro L, Boella G (2017) Siamese network with soft attention for semantic text understanding. In: Proceedings of the 13th international conference on semantic systems, pp 160\u2013167"},{"key":"1574_CR2","unstructured":"Akbik A, Blythe D, Vollgraf R (2018) Contextual string embeddings for sequence labeling. In: Proceedings of the 27th international conference on computational linguistics"},{"key":"1574_CR3","first-page":"813","volume":"6","author":"S Amin","year":"2020","unstructured":"Amin S, Jayakar N, Kiruthika M, Gurjar A (2020) Best fit resume predictor. Int J Eng Technol 6:813\u2013820","journal-title":"Int J Eng Technol"},{"key":"1574_CR4","doi-asserted-by":"publisher","first-page":"238","DOI":"10.1111\/j.1540-4560.2012.01747.x","volume":"2","author":"M Bendick","year":"2012","unstructured":"Bendick M, Nunes AP (2012) Developing the research basis for controlling bias in hiring. J Soc Issues 2:238\u2013262","journal-title":"J Soc Issues"},{"issue":"4","key":"1574_CR5","doi-asserted-by":"publisher","first-page":"991","DOI":"10.1257\/0002828042002561","volume":"94","author":"M Bertrand","year":"2004","unstructured":"Bertrand M, Mullainathan S (2004) Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. Am Econ Rev 94(4):991\u20131013","journal-title":"Am Econ Rev"},{"key":"1574_CR6","doi-asserted-by":"crossref","unstructured":"B\u00f6hm S, Linnyk O, Kohl J, Weber T, Teetz I, Bandurka K, Kersting M (2020) Analysing gender bias in IT job postings. In: Proceedings of the 2020 on computers and people research conference","DOI":"10.1145\/3378539.3393862"},{"key":"1574_CR7","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1162\/tacl_a_00051","volume":"5","author":"P Bojanowski","year":"2017","unstructured":"Bojanowski P, Grave E, Joulin A, Mikolov T (2017) Enriching word vectors with subword information. Trans Assoc Comput Linguist 5:135\u2013146","journal-title":"Trans Assoc Comput Linguist"},{"issue":"04","key":"1574_CR8","doi-asserted-by":"publisher","first-page":"669","DOI":"10.1142\/S0218001493000339","volume":"07","author":"J Bromley","year":"1993","unstructured":"Bromley J, Bentz JW, Bottou L, Guyon I, Lecun Y, Moore C, Shah R (1993) Signature verification using a \u201cSiamese\u201d time delay neural network. Int J Pattern Recognit Artif Intell 07(04):669\u2013688","journal-title":"Int J Pattern Recognit Artif Intell"},{"key":"1574_CR9","unstructured":"Chopra S, Hadsell R, LeCun Y (2005) Learning a similarity metric discriminatively, with application to face verification. In: 2005 IEEE Computer Society conference on computer vision and pattern recognition (CVPR\u201905)"},{"key":"1574_CR10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fpsyg.2017.01321","volume":"8","author":"E Derous","year":"2017","unstructured":"Derous E, Decoster J (2017) Implicit age cues in resumes: subtle effects on hiring discrimination. Front Psychol 8:1\u201315","journal-title":"Front Psychol"},{"issue":"2","key":"1574_CR11","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1111\/1748-8583.12217","volume":"29","author":"E Derous","year":"2018","unstructured":"Derous E, Ryan AM (2018) When your resume is (not) turning you down: modelling ethnic bias in resume screening. Hum Resour Manag J 29(2):113\u2013130","journal-title":"Hum Resour Manag J"},{"key":"1574_CR12","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the north"},{"key":"1574_CR13","first-page":"799","volume":"14","author":"I Dnvsls","year":"2016","unstructured":"Dnvsls I, Kiran R (2016) Profile screening and recommending using natural language processing (NLP) and leverage Hadoop framework for Bigdata. Int J Comput Sci Inf Secur 14:799\u2013811","journal-title":"Int J Comput Sci Inf Secur"},{"issue":"1","key":"1574_CR14","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.athoracsur.2020.07.005","volume":"111","author":"CP Erkmen","year":"2021","unstructured":"Erkmen CP, Kane L, Cooke DT (2021) Bias mitigation in cardiothoracic recruitment. Ann Thorac Surg 111(1):12\u201315","journal-title":"Ann Thorac Surg"},{"issue":"1","key":"1574_CR15","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1037\/a0022530","volume":"101","author":"D Gaucher","year":"2011","unstructured":"Gaucher D, Friesen J, Kay AC (2011) Evidence that gendered wording in job advertisements exists and sustains gender inequality. J Pers Soc Psychol 101(1):109\u2013128","journal-title":"J Pers Soc Psychol"},{"key":"1574_CR16","first-page":"1","volume":"20","author":"WH Gravett","year":"2017","unstructured":"Gravett WH (2017) The myth of objectivity: implicit racial bias and the law (part 1). Potchefstroom Electron Law J 20:1\u201325","journal-title":"Potchefstroom Electron Law J"},{"key":"1574_CR17","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1016\/j.eswa.2016.04.013","volume":"60","author":"S Guo","year":"2016","unstructured":"Guo S, Alamudun F, Hammond T (2016) R\u00e9suMatcher: a personalized r\u00e9sum\u00e9-job matching system. Expert Syst Appl 60:169\u2013182","journal-title":"Expert Syst Appl"},{"issue":"1","key":"1574_CR18","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1037\/a0036734","volume":"100","author":"AJ Koch","year":"2015","unstructured":"Koch AJ, D\u2019Mello SD, Sackett PR (2015) A meta-analysis of gender stereotypes and bias in experimental simulations of employment decision making. J Appl Psychol 100(1):128\u2013161","journal-title":"J Appl Psychol"},{"key":"1574_CR19","unstructured":"Kodiyan AA (2019) An overview of ethical issues in using AI systems in hiring with a case study of Amazon\u2019s AI based hiring tool. Researchgate Preprint 2019. Retrieved from https:\/\/www.academia.edu\/42965903\/An_overview_of_ethical_issues_in_using_AI_systems_in_hiring"},{"key":"1574_CR20","unstructured":"Le Q, Mikolov T (2014) Distributed Representations of Sentences and Documents. Retrieved from https:\/\/arxiv.org\/abs\/1405.4053. Retrieved 2 Apr 2022"},{"key":"1574_CR21","doi-asserted-by":"crossref","unstructured":"Maheshwary S, Misra H (2018) Matching resumes to jobs via deep Siamese network. In: Companion of the web conference 2018 on the web conference 2018\u2014WWW \u201918","DOI":"10.1145\/3184558.3186942"},{"key":"1574_CR22","unstructured":"Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. Retrieved from https:\/\/arxiv.org\/abs\/1301.3781. Retrieved 2 Apr 2022"},{"key":"1574_CR23","doi-asserted-by":"crossref","unstructured":"Muralidhar S, Nguyen L, Gatica-Perez D (2018) Words worth: verbal content and hirability impressions in YouTube video resumes. In: Proceedings of the 9th workshop on computational approaches to subjectivity, sentiment and social media analysis","DOI":"10.18653\/v1\/W18-6247"},{"issue":"4","key":"1574_CR24","doi-asserted-by":"publisher","first-page":"684","DOI":"10.1016\/j.jacr.2017.12.002","volume":"15","author":"ME Oates","year":"2018","unstructured":"Oates ME (2018) Optimizing recruitment into radiology: some simple approaches to controlling bias. J Am Coll Radiol 15(4):684\u2013686","journal-title":"J Am Coll Radiol"},{"key":"1574_CR25","unstructured":"\u00d6sterlund P (2020) Preventing discrimination in recruiting through unconscious biases. Preventing discrimination in recruiting through unconscious biases, pp 1\u201350. Retrieved from https:\/\/www.theseus.fi\/bitstream\/handle\/10024\/346242\/Osterlund_Pia.pdf"},{"key":"1574_CR26","doi-asserted-by":"crossref","unstructured":"Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)","DOI":"10.3115\/v1\/D14-1162"},{"key":"1574_CR27","doi-asserted-by":"crossref","unstructured":"Peters M, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: Proceedings of the 2018 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies, volume 1 (long papers)","DOI":"10.18653\/v1\/N18-1202"},{"issue":"2","key":"1574_CR28","first-page":"529","volume":"71","author":"M Raub","year":"2018","unstructured":"Raub M (2018) Bots, bias and big data: artificial intelligence, algorithmic bias and disparate impact liability in hiring practices. Arkansas Law Rev 71(2):529\u2013570","journal-title":"Arkansas Law Rev"},{"key":"1574_CR29","unstructured":"Sanyal S, Hazra S, Ghosh N, Adhikary S (2017) Resume parser with natural language processing. Resume parser with natural language processing. Published"},{"key":"1574_CR30","doi-asserted-by":"crossref","unstructured":"Tang S, Zhang X, Cryan J, Metzger MJ, Zheng H, Zhao BY (2017) Gender bias in the job market. In: Proceedings of the ACM on human\u2013computer interaction, 1 (CSCW), pp 1\u201319","DOI":"10.1145\/3134734"},{"issue":"1","key":"1574_CR31","doi-asserted-by":"publisher","first-page":"6","DOI":"10.3390\/fi9010006","volume":"9","author":"S Vidros","year":"2017","unstructured":"Vidros S, Kolias C, Kambourakis G, Akoglu L (2017) Automatic detection of online recruitment frauds: characteristics, methods, and a public dataset. Future Internet 9(1):6","journal-title":"Future Internet"},{"key":"1574_CR33","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1016\/j.procs.2021.10.016","volume":"193","author":"I Wings","year":"2021","unstructured":"Wings I, Nanda R, Adebayo KJ (2021) A context-aware approach for extracting hard and soft skills. Procedia Comput Sci 193:163\u2013172","journal-title":"Procedia Comput Sci"}],"container-title":["AI &amp; SOCIETY"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-022-01574-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00146-022-01574-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-022-01574-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,30]],"date-time":"2023-06-30T06:17:06Z","timestamp":1688105826000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00146-022-01574-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,18]]},"references-count":32,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,4]]}},"alternative-id":["1574"],"URL":"https:\/\/doi.org\/10.1007\/s00146-022-01574-0","relation":{},"ISSN":["0951-5666","1435-5655"],"issn-type":[{"value":"0951-5666","type":"print"},{"value":"1435-5655","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,18]]},"assertion":[{"value":"30 July 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 July 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 October 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that there is no confict of interest in this research.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"All authors listed have consented to participate in the study.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"All authors declare that they have authorized the publication of the study.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Employment Scam Aegean Dataset (EMSCAD) is a publicly available dataset containing 17,880 real-life job ads. The dataset is available online at .","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Availability of data and material"}},{"value":".","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code availability"}}]}}