{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T03:13:48Z","timestamp":1783394028361,"version":"3.54.6"},"publisher-location":"Cham","reference-count":40,"publisher":"Springer Nature Switzerland","isbn-type":[{"value":"9783032295316","type":"print"},{"value":"9783032295323","type":"electronic"}],"license":[{"start":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T00:00:00Z","timestamp":1783123200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T00:00:00Z","timestamp":1783123200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2027]]},"DOI":"10.1007\/978-3-032-29532-3_10","type":"book-chapter","created":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T08:55:36Z","timestamp":1783068936000},"page":"123-138","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["BEADS: Bias Evaluation Across Domains"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1061-5845","authenticated-orcid":false,"given":"Shaina","family":"Raza","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mizanur","family":"Rahman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael R.","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,7,4]]},"reference":[{"key":"10_CR1","unstructured":"Adams, C., Borkan, D., Sorensen, J., Dixon, L., Vasserman, L., Thain, N.: Jigsaw unintended bias in toxicity classification (2019). https:\/\/kaggle.com\/competitions\/jigsaw-unintended-bias-in-toxicity-classification"},{"key":"10_CR2","unstructured":"Adams, C., et al.: Toxic comment classification challenge (2017). https:\/\/kaggle.com\/competitions\/jigsaw-toxic-comment-classification-challenge"},{"key":"10_CR3","doi-asserted-by":"publisher","unstructured":"Barikeri, S., Lauscher, A., Vuli\u0107, I., Glava\u0161, G.: RedditBias: a real-world resource for bias evaluation and debiasing of conversational language models. In: Zong, C., Xia, F., Li, W., Navigli, R. (eds.) Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 1941\u20131955. Association for Computational Linguistics, Online (2021). https:\/\/doi.org\/10.18653\/v1\/2021.acl-long.151, https:\/\/aclanthology.org\/2021.acl-long.151","DOI":"10.18653\/v1\/2021.acl-long.151"},{"key":"10_CR4","unstructured":"Confident AI: Deepeval. GitHub repository (2024). https:\/\/github.com\/confident-ai\/deepeval"},{"key":"10_CR5","doi-asserted-by":"crossref","unstructured":"Deshpande, A., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K.: Toxicity in ChatGPT: analyzing persona-assigned language models. arXiv preprint arXiv:2304.05335 (2023)","DOI":"10.18653\/v1\/2023.findings-emnlp.88"},{"key":"10_CR6","doi-asserted-by":"publisher","unstructured":"Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L.: QLoRA: Efficient Finetuning of Quantized LLMs (2023). https:\/\/doi.org\/10.48550\/arXiv.2305.14314, http:\/\/arxiv.org\/abs\/2305.14314, arXiv:2305.14314 [cs]","DOI":"10.48550\/arXiv.2305.14314"},{"key":"10_CR7","doi-asserted-by":"crossref","unstructured":"Dhamala, J., et al.: Bold: dataset and metrics for measuring biases in open-ended language generation. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 862\u2013872 (2021)","DOI":"10.1145\/3442188.3445924"},{"key":"10_CR8","doi-asserted-by":"crossref","unstructured":"D\u00edaz, M., Johnson, I., Lazar, A., Piper, A.M., Gergle, D.: Addressing age-related bias in sentiment analysis. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1\u201314 (2018)","DOI":"10.1145\/3173574.3173986"},{"key":"10_CR9","doi-asserted-by":"crossref","unstructured":"Esiobu, D., et al.: Robbie: robust bias evaluation of large generative language models. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 3764\u20133814 (2023)","DOI":"10.18653\/v1\/2023.emnlp-main.230"},{"key":"10_CR10","doi-asserted-by":"crossref","unstructured":"F\u00e4rber, M., Burkard, V., Jatowt, A., Lim, S.: A multidimensional dataset based on crowdsourcing for analyzing and detecting news bias. In: Proceedings of the 29th ACM International Conference on Information and Knowledge Management, pp. 3007\u20133014 (2020)","DOI":"10.1145\/3340531.3412876"},{"key":"10_CR11","doi-asserted-by":"crossref","unstructured":"Gehman, S., Gururangan, S., Sap, M., Choi, Y., Smith, N.A.: Realtoxicityprompts: evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462 (2020)","DOI":"10.18653\/v1\/2020.findings-emnlp.301"},{"key":"10_CR12","doi-asserted-by":"publisher","unstructured":"Hartvigsen, T., Gabriel, S., Palangi, H., Sap, M., Ray, D., Kamar, E.: ToxiGen: a large-scale machine-generated dataset for adversarial and implicit hate speech detection. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3309\u20133326. Association for Computational Linguistics, Dublin, Ireland (2022). https:\/\/doi.org\/10.18653\/v1\/2022.acl-long.234, https:\/\/aclanthology.org\/2022.acl-long.234","DOI":"10.18653\/v1\/2022.acl-long.234"},{"key":"10_CR13","unstructured":"Inan, H., et al.: Llama guard: LLM-based input-output safeguard for human-AI conversations, arXiv preprint arXiv:2312.06674 (2023)"},{"key":"10_CR14","doi-asserted-by":"publisher","unstructured":"Kiesel, J., et al.: SemEval-2019 task 4: hyperpartisan news detection. In: Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 829\u2013839. Association for Computational Linguistics, Minneapolis, Minnesota, USA (2019). https:\/\/doi.org\/10.18653\/v1\/S19-2145, https:\/\/aclanthology.org\/S19-2145","DOI":"10.18653\/v1\/S19-2145"},{"key":"10_CR15","doi-asserted-by":"crossref","unstructured":"Kim, H., Mitra, K., Chen, R.L., Rahman, S., Zhang, D.: Meganno+: a human-LLM collaborative annotation system. arXiv preprint arXiv:2402.18050 (2024)","DOI":"10.18653\/v1\/2024.eacl-demo.18"},{"key":"10_CR16","doi-asserted-by":"crossref","unstructured":"Liang, P.P., Li, I.M., Zheng, E., Lim, Y.C., Salakhutdinov, R., Morency, L.P.: Towards debiasing sentence representations. arXiv preprint arXiv:2007.08100 (2020)","DOI":"10.18653\/v1\/2020.acl-main.488"},{"issue":"8","key":"10_CR17","doi-asserted-by":"publisher","first-page":"1381","DOI":"10.1093\/bioinformatics\/btx761","volume":"34","author":"L Luo","year":"2018","unstructured":"Luo, L., et al.: An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition. Bioinformatics 34(8), 1381\u20131388 (2018)","journal-title":"Bioinformatics"},{"key":"10_CR18","unstructured":"Maudslay, R.H., Gonen, H., Cotterell, R., Teufel, S.: It\u2019s all in the name: mitigating gender bias with name-based counterfactual data substitution. arXiv preprint arXiv:1909.00871 (2019)"},{"key":"10_CR19","doi-asserted-by":"crossref","unstructured":"McAuley, J.J., Leskovec, J.: From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 897\u2013908 (2013)","DOI":"10.1145\/2488388.2488466"},{"key":"10_CR20","unstructured":"Mishra, S., He, S., Belli, L.: Assessing demographic bias in named entity recognition. arXiv preprint arXiv:2008.03415 (2020)"},{"key":"10_CR21","doi-asserted-by":"publisher","unstructured":"Nadeem, M., Bethke, A., Reddy, S.: StereoSet: measuring stereotypical bias in pretrained language models. In: Zong, C., Xia, F., Li, W., Navigli, R. (eds.) Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 5356\u20135371. Association for Computational Linguistics (2021). https:\/\/doi.org\/10.18653\/v1\/2021.acl-long.416, https:\/\/aclanthology.org\/2021.acl-long.416","DOI":"10.18653\/v1\/2021.acl-long.416"},{"key":"10_CR22","doi-asserted-by":"crossref","unstructured":"Parrish, A., et al.: BBQ: a hand-built bias benchmark for question answering. arXiv preprint arXiv:2110.08193 (2021)","DOI":"10.18653\/v1\/2022.findings-acl.165"},{"key":"10_CR23","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.121542","volume":"237","author":"S Raza","year":"2024","unstructured":"Raza, S., Garg, M., Reji, D.J., Bashir, S.R., Ding, C.: NBias: a natural language processing framework for bias identification in text. Expert Syst. Appl. 237, 121542 (2024)","journal-title":"Expert Syst. Appl."},{"key":"10_CR24","unstructured":"Raza, S., et al.: Vldbench: vision language models disinformation detection benchmark. arXiv preprint arXiv:2502.11361 (2025)"},{"key":"10_CR25","unstructured":"Recasens, M., Danescu-Niculescu-Mizil, C., Jurafsky, D.: Linguistic models for analyzing and detecting biased language. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1650\u20131659 (2013)"},{"key":"10_CR26","unstructured":"Rizzo, G., Troncy, R.: Nerd: evaluating named entity recognition tools in the web of data. In: Workshop on Web Scale Knowledge Extraction (WEKEX\u201911), vol. 21 (2011)"},{"key":"10_CR27","doi-asserted-by":"crossref","unstructured":"Rudinger, R., Naradowsky, J., Leonard, B., Van Durme, B.: Gender bias in coreference resolution. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, New Orleans, Louisiana (2018)","DOI":"10.18653\/v1\/N18-2002"},{"key":"10_CR28","unstructured":"Sang, E.F., De Meulder, F.: Introduction to the conll-2003 shared task: Language-independent named entity recognition. arXiv preprint cs\/0306050 (2003)"},{"key":"10_CR29","doi-asserted-by":"publisher","unstructured":"Sap, M., Gabriel, S., Qin, L., Jurafsky, D., Smith, N.A., Choi, Y.: Social bias frames: reasoning about social and power implications of language. In: Jurafsky, D., Chai, J., Schluter, N., Tetreault, J. (eds.) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5477\u20135490. Association for Computational Linguistics (2020). https:\/\/doi.org\/10.18653\/v1\/2020.acl-main.486, https:\/\/aclanthology.org\/2020.acl-main.486","DOI":"10.18653\/v1\/2020.acl-main.486"},{"key":"10_CR30","doi-asserted-by":"publisher","unstructured":"Schick, T., Udupa, S., Sch\u00fctze, H.: Self-diagnosis and self-debiasing: a proposal for reducing corpus-based bias in NLP. Trans. Assoc. Comput. Linguist. 9, 1408\u20131424 (2021). https:\/\/doi.org\/10.1162\/tacl_a_00434","DOI":"10.1162\/tacl_a_00434"},{"key":"10_CR31","doi-asserted-by":"crossref","unstructured":"Smith, E.M., Hall, M., Kambadur, M., Presani, E., Williams, A.: \u201ci\u2019m sorry to hear that\u201d: Finding new biases in language models with a holistic descriptor dataset. arXiv preprint arXiv:2205.09209 (2022)","DOI":"10.18653\/v1\/2022.emnlp-main.625"},{"key":"10_CR32","unstructured":"Spinde, T., Rudnitckaia, L., Sinha, K., Hamborg, F., Gipp, B., Donnay, K.: Mbic - a media bias annotation dataset including annotator characteristics. arXiv preprint arXiv:2105.11910 (2021)"},{"key":"10_CR33","unstructured":"Wang, B., et al.: Decodingtrust: a comprehensive assessment of trustworthiness in GPT models. arXiv preprint arXiv:2306.11698 (2023)"},{"key":"10_CR34","unstructured":"Wang, Z., Xie, Q., Feng, Y., Ding, Z., Yang, Z., Xia, R.: Is ChatGPT a good sentiment analyzer? A preliminary study. arXiv preprint arXiv:2304.04339 (2023)"},{"key":"10_CR35","doi-asserted-by":"crossref","unstructured":"Wei, A., Haghtalab, N., Steinhardt, J.: Jailbroken: how does LLM safety training fail? In: Advances in Neural Information Processing Systems, vol. 36 (2024)","DOI":"10.52202\/075280-3508"},{"key":"10_CR36","doi-asserted-by":"crossref","unstructured":"Wessel, M., Horych, T., Ruas, T., Aizawa, A., Gipp, B., Spinde, T.: Introducing mbib-the first media bias identification benchmark task and dataset collection. In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2765\u20132774 (2023)","DOI":"10.1145\/3539618.3591882"},{"key":"10_CR37","doi-asserted-by":"publisher","unstructured":"Zhao, J., Wang, T., Yatskar, M., Ordonez, V., Chang, K.W.: Gender bias in coreference resolution: Evaluation and debiasing methods. In: Walker, M., Ji, H., Stent, A. (eds.) Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 15\u201320. Association for Computational Linguistics, New Orleans, Louisiana (2018). https:\/\/doi.org\/10.18653\/v1\/N18-2003, https:\/\/aclanthology.org\/N18-2003","DOI":"10.18653\/v1\/N18-2003"},{"key":"10_CR38","unstructured":"Zhao, W.X., et\u00a0al.: A survey of large language models. arXiv preprint arXiv:2303.18223 (2023)"},{"key":"10_CR39","doi-asserted-by":"crossref","unstructured":"Zhou, C., et\u00a0al.: Lima: less is more for alignment. In: Advances in Neural Information Processing Systems, vol. 36 (2024)","DOI":"10.52202\/075280-2400"},{"key":"10_CR40","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Sheng, Q., Cao, J., Li, S., Wang, D., Zhuang, F.: Generalizing to the future: mitigating entity bias in fake news detection. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2120\u20132125 (2022)","DOI":"10.1145\/3477495.3531816"}],"container-title":["Lecture Notes in Computer Science","Natural Language Processing and Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-032-29532-3_10","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T02:17:24Z","timestamp":1783390644000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-032-29532-3_10"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,4]]},"ISBN":["9783032295316","9783032295323"],"references-count":40,"URL":"https:\/\/doi.org\/10.1007\/978-3-032-29532-3_10","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"value":"0302-9743","type":"print"},{"value":"1611-3349","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,4]]},"assertion":[{"value":"4 July 2026","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"\u2013 Funding: Resources used in this study are supported by Vector Institute.\u2013 Conflict of interest\/Competing interests: The authors declare no conflicting interests. The authors have no relevant financial or non-financial competing interests to disclose.\u2013 Ethics approval and consent to participate: This research did not involve human participants, animal subjects, or personally identifiable data, and therefore did not require formal ethical approval. All data used in this study were publicly available, ensuring compliance with ethical guidelines and data privacy standards.\u2013 Consent for publication: Yes\u2013 Data availability: The data is available at\n                      \n                      .","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"NLDB","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Conference on Applications of Natural Language to Information Systems","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Trondheim","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Norway","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2026","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"17 June 2026","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"19 June 2026","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"31","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"nldb2026","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/www.ntnu.edu\/nldb2026","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}}]}}