{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,9]],"date-time":"2025-06-09T12:48:12Z","timestamp":1749473292949},"reference-count":26,"publisher":"World Scientific Pub Co Pte Ltd","issue":"04","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Semantic Computing"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:p> Currently, Natural Language Processing (NLP) applications like chatbots are very close to mimick human responses. This has been achieved via powerful and sophisticated models like Bidirectional Encoder Representations from Transformers (BERT). Although, the capabilities that such models offer are superior to the technologies that preceded it, these models still possess bias. BERT or similar models are mostly trained on text corpora that deviate in important ways from the text encountered by a chatbot in a problem-specific context. Past research on NLP bias has heavily focused on measuring and mitigating bias with respect to protected attributes (stereotyping like gender, race, ethnicity, etc.), but the exploration of model bias with respect to classification labels remained yet to be explored. We investigate how a classification model hugely favors one class with respect to another. In this paper, we propose a bias evaluation technique called directional pairwise class confusion bias that highlights our chatbot intent classification models bias on pairs of classes. Lastly, we also demonstrate two bias mitigation strategies on a few example-biased pairs. <\/jats:p>","DOI":"10.1142\/s1793351x22500040","type":"journal-article","created":{"date-parts":[[2022,7,9]],"date-time":"2022-07-09T03:28:15Z","timestamp":1657337295000},"page":"497-520","source":"Crossref","is-referenced-by-count":4,"title":["Quantification and Mitigation of Directional Pairwise Class Confusion Bias in a Chatbot Intent Classification Model"],"prefix":"10.1142","volume":"16","author":[{"given":"Sudhashree","family":"Sayenju","sequence":"first","affiliation":[{"name":"School of Data Science and Analytics, Kennesaw State University, 3391 Town Point Dr. NW Suite 2400, MD 9104, Kennesaw, Georgia 30144, USA"}]},{"given":"Ramazan","family":"Aygun","sequence":"additional","affiliation":[{"name":"School of Data Science and Analytics, Kennesaw State University, 680 Arntson Drive, Office J 305A Marietta, Georgia 30060, USA"}]},{"given":"Jonathan","family":"Boardman","sequence":"additional","affiliation":[{"name":"School of Data Science and Analytics, Kennesaw State University, 3391 Town Point Dr. NW Suite 2400, MD 9104, Kennesaw, Georgia 30144, USA"}]},{"given":"Duleep Prasanna Rathgamage","family":"Don","sequence":"additional","affiliation":[{"name":"School of Data Science and Analytics, Kennesaw State University, 3391 Town Point Dr. NW Suite 2400, MD 9104, Kennesaw, Georgia 30144, USA"}]},{"given":"Yifan","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Data Science and Analytics, Kennesaw State University, 3391 Town Point Dr. NW Suite 2400, MD 9104, Kennesaw, Georgia 30144, USA"}]},{"given":"Bill","family":"Franks","sequence":"additional","affiliation":[{"name":"School of Data Science and Analytics, Kennesaw State University, 3391 Town Point Dr. NW Suite 2400, MD 9104, Kennesaw, Georgia 30144, USA"}]},{"given":"Sereres","family":"Johnston","sequence":"additional","affiliation":[{"name":"Enterprise Business Intelligence and Analytics, The Travelers Companies, Inc., 1 Tower Square, Hartford, Connecticut 06183, USA"}]},{"given":"George","family":"Lee","sequence":"additional","affiliation":[{"name":"Enterprise Business Intelligence and Analytics, The Travelers Companies, Inc., 1 Tower Square, Hartford, Connecticut 06183, USA"}]},{"given":"Dan","family":"Sullivan","sequence":"additional","affiliation":[{"name":"Enterprise Business Intelligence and Analytics, The Travelers Companies, Inc., 1 Tower Square, Hartford, Connecticut 06183, USA"}]},{"given":"Girish","family":"Modgil","sequence":"additional","affiliation":[{"name":"Enterprise Business Intelligence and Analytics, The Travelers Companies, Inc., 1 Tower Square, Hartford, Connecticut 06183, USA"}]}],"member":"219","published-online":{"date-parts":[[2022,8,8]]},"reference":[{"key":"S1793351X22500040BIB001","doi-asserted-by":"publisher","DOI":"10.1109\/ICEBE.2018.00019"},{"key":"S1793351X22500040BIB002","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3567047"},{"key":"S1793351X22500040BIB003","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01437-7_16"},{"key":"S1793351X22500040BIB004","doi-asserted-by":"publisher","DOI":"10.1145\/3457607"},{"issue":"7","key":"S1793351X22500040BIB006","first-page":"1","volume":"11","author":"Garrido-Muoz I.","year":"2021","journal-title":"Appl. Sci."},{"key":"S1793351X22500040BIB007","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.485"},{"key":"S1793351X22500040BIB008","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.149"},{"key":"S1793351X22500040BIB012","doi-asserted-by":"publisher","DOI":"10.1002\/9781118646106"},{"key":"S1793351X22500040BIB013","doi-asserted-by":"publisher","DOI":"10.3233\/IDA-2002-6504"},{"key":"S1793351X22500040BIB014","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-019-0192-5"},{"key":"S1793351X22500040BIB015","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2018.07.011"},{"key":"S1793351X22500040BIB016","doi-asserted-by":"publisher","DOI":"10.1109\/ICSC52841.2022.00017"},{"key":"S1793351X22500040BIB017","first-page":"4171","volume-title":"Proc. 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin J.","year":"2019"},{"key":"S1793351X22500040BIB018","doi-asserted-by":"publisher","DOI":"10.1002\/9781119125563.evpsych241"},{"key":"S1793351X22500040BIB019","first-page":"1","volume-title":"1st Int. Conf. Learning Representations, Workshop Track Proc.","author":"Mikolov T.","year":"2013"},{"key":"S1793351X22500040BIB020","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"S1793351X22500040BIB021","first-page":"4349","volume-title":"Advances in Neural Information Processing Systems 29: Annual Conf. Neural Information Processing Systems","author":"Bolukbasi T.","year":"2016"},{"key":"S1793351X22500040BIB022","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-2003"},{"key":"S1793351X22500040BIB023","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1521"},{"key":"S1793351X22500040BIB024","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1062"},{"key":"S1793351X22500040BIB025","doi-asserted-by":"publisher","DOI":"10.1145\/3366424.3383559"},{"key":"S1793351X22500040BIB027","doi-asserted-by":"publisher","DOI":"10.1007\/s12559-021-09881-2"},{"key":"S1793351X22500040BIB028","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-3007"},{"key":"S1793351X22500040BIB029","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-020-09548-1"},{"key":"S1793351X22500040BIB031","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.468"},{"key":"S1793351X22500040BIB032","doi-asserted-by":"publisher","DOI":"10.1145\/3278721.3278729"}],"container-title":["International Journal of Semantic Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S1793351X22500040","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,10]],"date-time":"2022-10-10T09:30:08Z","timestamp":1665394208000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S1793351X22500040"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,8]]},"references-count":26,"journal-issue":{"issue":"04","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["10.1142\/S1793351X22500040"],"URL":"https:\/\/doi.org\/10.1142\/s1793351x22500040","relation":{},"ISSN":["1793-351X","1793-7108"],"issn-type":[{"value":"1793-351X","type":"print"},{"value":"1793-7108","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,8]]}}}