{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T01:13:45Z","timestamp":1783386825668,"version":"3.54.6"},"reference-count":48,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2021,10,12]],"date-time":"2021-10-12T00:00:00Z","timestamp":1633996800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>In recent years, we have seen a wide use of Artificial Intelligence (AI) applications in the Internet and everywhere. Natural Language Processing and Machine Learning are important sub-fields of AI that have made Chatbots and Conversational AI applications possible. Those algorithms are built based on historical data in order to create language models, however historical data could be intrinsically discriminatory. This article investigates whether a Conversational AI could identify offensive language and it will show how large language models often produce quite a bit of unethical behavior because of bias in the historical data. Our low-level proof-of-concept will present the challenges to detect offensive language in social media and it will discuss some steps to propitiate strong results in the detection of offensive language and unethical behavior using a Conversational AI.<\/jats:p>","DOI":"10.3390\/info12100418","type":"journal-article","created":{"date-parts":[[2021,10,13]],"date-time":"2021-10-13T06:38:41Z","timestamp":1634107121000},"page":"418","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Could a Conversational AI Identify Offensive Language?"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1242-4834","authenticated-orcid":false,"given":"Daniela America","family":"da Silva","sequence":"first","affiliation":[{"name":"Electronic and Computer Engineering Program, Informatics, Brazilian Aeronautics Institute of Technology, ITA, Sao Jose dos Campos 12228-900, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4785-2804","authenticated-orcid":false,"given":"Henrique Duarte Borges","family":"Louro","sequence":"additional","affiliation":[{"name":"Electronic and Computer Engineering Program, Informatics, Brazilian Aeronautics Institute of Technology, ITA, Sao Jose dos Campos 12228-900, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0932-9171","authenticated-orcid":false,"given":"Gildarcio Sousa","family":"Goncalves","sequence":"additional","affiliation":[{"name":"Electronic and Computer Engineering Program, Informatics, Brazilian Aeronautics Institute of Technology, ITA, Sao Jose dos Campos 12228-900, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1551-435X","authenticated-orcid":false,"given":"Johnny Cardoso","family":"Marques","sequence":"additional","affiliation":[{"name":"Electronic and Computer Engineering Program, Informatics, Brazilian Aeronautics Institute of Technology, ITA, Sao Jose dos Campos 12228-900, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5958-8011","authenticated-orcid":false,"given":"Luiz Alberto Vieira","family":"Dias","sequence":"additional","affiliation":[{"name":"Electronic and Computer Engineering Program, Informatics, Brazilian Aeronautics Institute of Technology, ITA, Sao Jose dos Campos 12228-900, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2399-5066","authenticated-orcid":false,"given":"Adilson Marques","family":"da Cunha","sequence":"additional","affiliation":[{"name":"Electronic and Computer Engineering Program, Informatics, Brazilian Aeronautics Institute of Technology, ITA, Sao Jose dos Campos 12228-900, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7069-8430","authenticated-orcid":false,"given":"Paulo Marcelo","family":"Tasinaffo","sequence":"additional","affiliation":[{"name":"Electronic and Computer Engineering Program, Informatics, Brazilian Aeronautics Institute of Technology, ITA, Sao Jose dos Campos 12228-900, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ertel, W. (2017). Introduction to Artificial Intelligence. Introduction to Artificial Intelligence, Springer International Publishing.","DOI":"10.1007\/978-3-319-58487-4"},{"key":"ref_2","unstructured":"Jurafsky, D., and Martin, J. (2017). Dialog systems and chatbots. Speech Lang. Proc., 3, Available online: http:\/\/www.cs.columbia.edu\/~julia\/courses\/CS6998-2019\/25.pdf."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"McKenna, J.P., Choudhary, S., Saxon, M., Strimel, G.P., and Mouchtaris, A. (2020). Semantic complexity in end-to-end spoken language understanding. arXiv.","DOI":"10.21437\/Interspeech.2020-2929"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"da Silva, D.A., Louro, H.D.B., Goncalves, G.S., Marques, J.C., Dias, L.A.V., da Cunha, A.M., and Tasinaffo, P.M. (2020). A Hybrid Dictionary Model for Ethical Analysis. Advances in Intelligent Systems and Computing, Springer International Publishing.","DOI":"10.1007\/978-3-030-43020-7_83"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Petersen, K., Feldt, R., Mujtaba, S., and Mattsson, M. (2008, January 26\u201327). Systematic mapping studies in software engineering. Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering (EASE), Bari, Italy.","DOI":"10.14236\/ewic\/EASE2008.8"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e14394","DOI":"10.2196\/14394","article-title":"The extent and coverage of current knowledge of connected health: Systematic mapping study","volume":"21","author":"Karampela","year":"2019","journal-title":"J. Med. Internet Res."},{"key":"ref_7","unstructured":"Saba, T. (2021, October 03). Module 1\u2014The Concepts of Bias and Fairness in the AI Paradigm \/ The Notion of Diversity (MOOC Lecture). In UMontrealX and IVADO, Bias and Discrimination in AI. edX. Available online: https:\/\/learning.edx.org\/course\/course-v1:UMontrealX+IVADO-BIAS-220+3T2021\/block-v1:UMontrealX+IVADO-BIAS-220+3T2021+type@sequential+block@4c92c4a7912e437cb114995fd817ef2e."},{"key":"ref_8","unstructured":"Farnadi, G. (2021, October 03). Module 1\u2014The Concepts of Bias and Fairness in the AI Paradigm\/Fairness (MOOC Lecture). In UMontrealX and IVADO, Bias and Discrimination in AI. edX. Available online: https:\/\/learning.edx.org\/course\/course-v1:UMontrealX+IVADO-BIAS-220+3T2021\/block-v1:UMontrealX+IVADO-BIAS-220+3T2021+type@sequential+block@bd20a537e32e43b8a1f694f17a9f7b44."},{"key":"ref_9","unstructured":"IEEE (2020, November 18). IEEE Ethically Aligned Design. Available online: https:\/\/ethicsinaction.ieee.org\/."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1038\/s42256-019-0088-2","article-title":"The global landscape of AI ethics guidelines","volume":"1","author":"Jobin","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1038\/d41586-021-00530-0","article-title":"Robo-writers: The rise and risks of language-generating AI","volume":"591","author":"Hutson","year":"2021","journal-title":"Nature"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1038\/s41586-018-0637-6","article-title":"The moral machine experiment","volume":"563","author":"Awad","year":"2018","journal-title":"Nature"},{"key":"ref_13","unstructured":"Suresh, H., and Guttag, J.V. (2019). A framework for understanding sources of harm throughout the machine learning life cycle. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tufekci, Z. (2014, January 1\u20134). Big questions for social media big data: Representativeness, validity and other methodological pitfalls. Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media, Ann Arbor, MI, USA.","DOI":"10.1609\/icwsm.v8i1.14517"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"13","DOI":"10.3389\/fdata.2019.00013","article-title":"Social data: Biases, methodological pitfalls, and ethical boundaries","volume":"2","author":"Olteanu","year":"2019","journal-title":"Front. Big Data"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1126\/science.aal4230","article-title":"Semantics derived automatically from language corpora contain human-like biases","volume":"356","author":"Caliskan","year":"2017","journal-title":"Science"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hutchinson, B., and Mitchell, M. (2019, January 29\u201331). 50 years of test (un) fairness: Lessons for machine learning. Proceedings of the Conference on Fairness, Accountability, and Transparency, Atlanta, GA, USA.","DOI":"10.1145\/3287560.3287600"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Verma, S., and Rubin, J. (2018, January 29). Fairness definitions explained. Proceedings of the 2018 IEEE\/ACM International Workshop on Software Fairness (FairWare), Gothenburg, Sweden.","DOI":"10.1145\/3194770.3194776"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3457607","article-title":"A survey on bias and fairness in machine learning","volume":"54","author":"Mehrabi","year":"2021","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liu, L.T., Dean, S., Rolf, E., Simchowitz, M., and Hardt, M. (2018, January 10\u201315). Delayed impact of fair machine learning. Proceedings of the International Conference on Machine Learning, Stockholmsm\u00e4ssan, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2019\/862"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, W., and Ntoutsi, E. (2019). Faht: An adaptive fairness-aware decision tree classifier. arXiv.","DOI":"10.24963\/ijcai.2019\/205"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhang, W., Bifet, A., Zhang, X., Weiss, J.C., and Nejdl, W. (2021). FARF: A Fair and Adaptive Random Forests Classifier. Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer.","DOI":"10.1007\/978-3-030-75765-6_20"},{"key":"ref_23","unstructured":"Bechavod, Y., Jung, C., and Wu, Z.S. (2020). Metric-free individual fairness in online learning. arXiv."},{"key":"ref_24","unstructured":"Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., and Brunskill, E. (2021). On the Opportunities and Risks of Foundation Models. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Abid, A., Farooqi, M., and Zou, J. (2021). Persistent anti-muslim bias in large language models. arXiv.","DOI":"10.1145\/3461702.3462624"},{"key":"ref_26","unstructured":"Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J.W., Wallach, H., Daum\u00e9, H., and Crawford, K. (2018). Datasheets for datasets. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1162\/tacl_a_00041","article-title":"Data statements for natural language processing: Toward mitigating system bias and enabling better science","volume":"6","author":"Bender","year":"2018","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1177\/0261927X09351676","article-title":"The psychological meaning of words: LIWC and computerized text analysis methods","volume":"29","author":"Tausczik","year":"2010","journal-title":"J. Lang. Soc. Psychol."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Mondal, M., Silva, L.A., and Benevenuto, F. (2017, January 4\u20137). A measurement study of hate speech in social media. Proceedings of the 28th ACM Conference on Hypertext and Social Media, Prague, Czech Republic.","DOI":"10.1145\/3078714.3078723"},{"key":"ref_30","unstructured":"Chiu, K.L., and Alexander, R. (2021). Detecting Hate Speech with GPT-3. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gordon, M.L., Zhou, K., Patel, K., Hashimoto, T., and Bernstein, M.S. (2021, January 8\u201313). The disagreement deconvolution: Bringing machine learning performance metrics in line with reality. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan.","DOI":"10.1145\/3411764.3445423"},{"key":"ref_32","unstructured":"Sap, M., Card, D., Gabriel, S., Choi, Y., and Smith, N.A. (August, January 28). The risk of racial bias in hate speech detection. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Davidson, T., Warmsley, D., Macy, M., and Weber, I. (2017, January 15\u201318). Automated hate speech detection and the problem of offensive language. Proceedings of the International AAAI Conference on Web and Social Media, Montr\u00e9al, QC, Canada.","DOI":"10.1609\/icwsm.v11i1.14955"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Davidson, T., Bhattacharya, D., and Weber, I. (2019). Racial bias in hate speech and abusive language detection datasets. arXiv.","DOI":"10.18653\/v1\/W19-3504"},{"key":"ref_35","unstructured":"JAIC (2021, January 05). Explaining Artificial Intelligence and Machine Learning. Available online: https:\/\/www.youtube.com\/watch?v=y_rY0ZIn5L4."},{"key":"ref_36","unstructured":"The World Bank (2020, December 02). Be Data-Driven: Reimagining Human Connections Technology and Innovation in Education at the World Bank. Available online: https:\/\/www.worldbank.org\/en\/topic\/edutech\/brief\/be-data-driven-reimagining-human-connections-technology-and-innovation-in-education-at-the-world-bank."},{"key":"ref_37","unstructured":"Berkman Klein Center (2020, November 18). Principled AI. Available online: https:\/\/cyber.harvard.edu\/publication\/2020\/principled-ai."},{"key":"ref_38","unstructured":"Executive Office of the President, Munoz, C., Director, D.P., and Megan, D.J. (2016). Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights."},{"key":"ref_39","unstructured":"Swiss Cognitive (2021, June 22). Distinguishing between Chatbots and Conversational AI. Available online: https:\/\/swisscognitive.ch\/2021\/06\/11\/chatbots-and-conversational-ai-3\/."},{"key":"ref_40","first-page":"2001","article-title":"Linguistic inquiry and word count: LIWC 2001","volume":"71","author":"Pennebaker","year":"2001","journal-title":"Mahway Lawrence Erlbaum Assoc."},{"key":"ref_41","unstructured":"Gon\u00e7alves, P., Benevenuto, F., and Cha, M. (2013). Panas-t: A psychometric scale for measuring sentiments on twitter. arXiv."},{"key":"ref_42","unstructured":"Bollen, J., Mao, H., and Pepe, A. (2011, January 17\u201321). Modeling public mood and emotion: Twitter sentiment and socio-economic phenomena. Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media, Barcelona, Spain."},{"key":"ref_43","unstructured":"HAI (2021, July 16). Why AI Struggles To Recognize Toxic Speech on Social Media. Available online: https:\/\/hai.stanford.edu\/news\/why-ai-struggles-recognize-toxic-speech-social-media."},{"key":"ref_44","unstructured":"Facebook (2021, July 16). Update on Our Progress on AI and Hate Speech Detection. Available online: https:\/\/about.fb.com\/news\/2021\/02\/update-on-our-progress-on-ai-and-hate-speech-detection\/."},{"key":"ref_45","unstructured":"WashingtonPost (2021, July 16). YouTube Says It Is Getting Better at Taking Down Videos That Break Its Rules. They Still Number in the Millions. Available online: https:\/\/www.washingtonpost.com\/technology\/2021\/04\/06\/youtube-video-ban-metric\/."},{"key":"ref_46","unstructured":"Time (2021, July 16). Twitter Penalizes Record Number of Accounts for Posting Hate Speech. Available online: https:\/\/time.com\/6080324\/twitter-hate-speech-penalties\/."},{"key":"ref_47","unstructured":"SaferNet Seguran\u00e7a Digital (2021, July 02). SaferNet Seguran\u00e7a Digital. Available online: https:\/\/new.safernet.org.br\/."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Bosselut, A., Rashkin, H., Sap, M., Malaviya, C., Celikyilmaz, A., and Choi, Y. (2019). Comet: Commonsense transformers for automatic knowledge graph construction. arXiv.","DOI":"10.18653\/v1\/P19-1470"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/10\/418\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:12:19Z","timestamp":1760166739000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/10\/418"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,12]]},"references-count":48,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["info12100418"],"URL":"https:\/\/doi.org\/10.3390\/info12100418","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,12]]}}}