{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T16:39:44Z","timestamp":1776443984651,"version":"3.51.2"},"reference-count":87,"publisher":"Springer Science and Business Media LLC","license":[{"start":{"date-parts":[[2022,3,2]],"date-time":"2022-03-02T00:00:00Z","timestamp":1646179200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,2]],"date-time":"2022-03-02T00:00:00Z","timestamp":1646179200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100006254","name":"Ruhr-Universit\u00e4t Bochum","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100006254","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Inf Syst Front"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Hate speech in social media is an increasing problem that can negatively affect individuals and society as a whole. Moderators on social media platforms need to be technologically supported to detect problematic content and react accordingly. In this article, we develop and discuss the design principles that are best suited for creating efficient user interfaces for decision support systems that use artificial intelligence (AI) to assist human moderators. We qualitatively and quantitatively evaluated various design options over three design cycles with a total of 641 participants. Besides measuring perceived ease of use, perceived usefulness, and intention to use, we also conducted an experiment to prove the significant influence of AI explainability on end users\u2019 perceived cognitive efforts, perceived informativeness, mental model, and trustworthiness in AI. Finally, we tested the acquired design knowledge with software developers, who rated the reusability of the proposed design principles as high.<\/jats:p>","DOI":"10.1007\/s10796-021-10234-5","type":"journal-article","created":{"date-parts":[[2022,3,2]],"date-time":"2022-03-02T12:02:31Z","timestamp":1646222551000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["Design Principles for User Interfaces in AI-Based Decision Support Systems: The Case of Explainable Hate Speech Detection"],"prefix":"10.1007","author":[{"given":"Christian","family":"Meske","sequence":"first","affiliation":[]},{"given":"Enrico","family":"Bunde","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,3,2]]},"reference":[{"key":"10234_CR1","doi-asserted-by":"publisher","first-page":"52138","DOI":"10.1109\/ACCESS.2018.2870052","volume":"6","author":"A Adadi","year":"2018","unstructured":"Adadi, A., & Berrada, M. (2018). Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access, 6, 52138\u201352160. https:\/\/doi.org\/10.1109\/ACCESS.2018.2870052","journal-title":"IEEE Access"},{"key":"10234_CR2","doi-asserted-by":"publisher","DOI":"10.1007\/s10796-021-10105-z","author":"SG Arapostathis","year":"2021","unstructured":"Arapostathis, S. G. (2021). A Methodology for Automatix Acquisition of Flood-event Management Information From Social Media: The Flood in Messinia, South Greece, 2016. Information Systems Frontiers. https:\/\/doi.org\/10.1007\/s10796-021-10105-z","journal-title":"Information Systems Frontiers"},{"key":"10234_CR3","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","volume":"58","author":"AB Arrieta","year":"2020","unstructured":"Arrieta, A. B., D\u00edaz-Rodr\u00edguez, N., Ser, J. D., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R., & Herrera, F. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82\u2013115. https:\/\/doi.org\/10.1016\/j.inffus.2019.12.012","journal-title":"Information Fusion"},{"issue":"8","key":"10234_CR4","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/MC\/.2020.2996587","volume":"53","author":"Z Akata","year":"2020","unstructured":"Akata, Z., Balliet, D., Rijke, D., Dignum, F., Dignum, V., Fokkens, G. E., Fokkens, A., Grossi, D., Hindriks, K., Hoos, H., Jonker, H. H., Jonker, C., Monz, C., Oliehoek, M. N., Oliehoek, F., Pakken, H., Schlbach, S., van der Gaag, L., van Harmelen, F., \u2026 Wlling, M. (2020). A Research Agenda for Hybrid Intelligence: Augmenting Human Intellect With Collaborative, Adaptive, Responsible, and Explainable Artificial Intelligence. Computer, 53(8), 18\u201328. https:\/\/doi.org\/10.1109\/MC\/.2020.2996587","journal-title":"Computer"},{"key":"10234_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.cosrev.2020.100311","volume":"38","author":"FE Ayo","year":"2020","unstructured":"Ayo, F. E., Folorunso, O., Ibharalu, F. T., & Osinuga, I. A. (2020). Machine learning techniques for hate speech classification of twitter data: State-of-the-art, future challenges and research directions. Computer Science Review, 38, 1\u201334. https:\/\/doi.org\/10.1016\/j.cosrev.2020.100311","journal-title":"Computer Science Review"},{"key":"10234_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.cose.2019.101710","volume":"90","author":"V Balakrishnan","year":"2020","unstructured":"Balakrishnan, V., Khan, S., & Arabnia, H. R. (2020). Improving cyberbullying detection using Twitter users\u2019 psychological features and machine learning. Computers & Security, 90, 1\u201311. https:\/\/doi.org\/10.1016\/j.cose.2019.101710","journal-title":"Computers & Security"},{"issue":"5","key":"10234_CR7","doi-asserted-by":"publisher","first-page":"358","DOI":"10.17705\/1jais.00495","volume":"19","author":"R Baskerville","year":"2018","unstructured":"Baskerville, R., Baiyere, A., Gregor, S., Hevner, A., & Rossi, M. (2018). Design Science Research Contributions: Finding a Balance between Artifact and Theory. Journal of the Association for Information Systems, 19(5), 358\u2013376. https:\/\/doi.org\/10.17705\/1jais.00495","journal-title":"Journal of the Association for Information Systems"},{"key":"10234_CR8","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1007\/s12599-010-0118-4","volume":"2","author":"R Baskerville","year":"2010","unstructured":"Baskerville, R., & Pries-Heje, J. (2010). Explanatory Design Theory. Business & Information Systems Engineering, 2, 271\u2013282. https:\/\/doi.org\/10.1007\/s12599-010-0118-4","journal-title":"Business & Information Systems Engineering"},{"key":"10234_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/10.1186\/s12911-020-01276-x","volume":"20","author":"AJ Barda","year":"2020","unstructured":"Barda, A. J., Horvat, C. M., & Hochheiser, H. (2020). A qualitative research framework for the design of user-centered displays of explanations for machine learning model predictions in healthcare. BMC Medical Informatics and Decision Making, 20, 1\u201316. https:\/\/doi.org\/10.1186\/10.1186\/s12911-020-01276-x","journal-title":"BMC Medical Informatics and Decision Making"},{"issue":"1","key":"10234_CR10","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1111\/pops.12670","volume":"41","author":"M Bilewicz","year":"2020","unstructured":"Bilewicz, M., & Soral, W. (2020). Hate Speech Epidemic. The Dynamic Effects of Derogatory Language on Intergroup Relations and Political Radicalization. Political Psychology, 41(1), 3\u201333. https:\/\/doi.org\/10.1111\/pops.12670","journal-title":"Political Psychology"},{"key":"10234_CR11","doi-asserted-by":"publisher","DOI":"10.4135\/9781849208604","author":"N Blaikie","year":"2003","unstructured":"Blaikie, N. (2003). Analyzing Quantitative Data. Sage Publications Ltd. https:\/\/doi.org\/10.4135\/9781849208604","journal-title":"Sage Publications Ltd"},{"issue":"2","key":"10234_CR12","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1191\/1478088706qp063oa","volume":"3","author":"V Braun","year":"2006","unstructured":"Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77\u2013101. https:\/\/doi.org\/10.1191\/1478088706qp063oa","journal-title":"Qualitative Research in Psychology"},{"key":"10234_CR13","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1007\/s12525-019-00358-7","volume":"29","author":"J vom Brocke","year":"2019","unstructured":"vom Brocke, J., & Maedche, A. (2019). The DSR grid: Six core dimensions for effectively planning and communicating design science research projects. Electronic Markets, 29, 379\u2013385. https:\/\/doi.org\/10.1007\/s12525-019-00358-7","journal-title":"Electronic Markets"},{"issue":"3","key":"10234_CR14","doi-asserted-by":"publisher","first-page":"520","DOI":"10.17705\/1jais.00611","volume":"21","author":"J vom Brocke","year":"2020","unstructured":"vom Brocke, J., Winter, R., Henver, A., & Maedche, A. (2020). Special Issue Editorial Accumulation and Evolution of Design Knowledge in Design Science Research: A Journey Through Time and Space. Journal of the Association for Information Systems, 21(3), 520\u2013544. https:\/\/doi.org\/10.17705\/1jais.00611","journal-title":"Journal of the Association for Information Systems"},{"key":"10234_CR15","doi-asserted-by":"publisher","first-page":"1531","DOI":"10.1007\/s10845-019-01531-7","volume":"31","author":"JPU Cadavid","year":"2020","unstructured":"Cadavid, J. P. U., Lamouri, S., Grabot, B., Pellerin, R., & Fortin, A. (2020). Machine learning applied in production planning and control: A state-of-the-art in the era of industry 4.0. Journal of Intelligent Manufacturing, 31, 1531\u20131558. https:\/\/doi.org\/10.1007\/s10845-019-01531-7","journal-title":"Journal of Intelligent Manufacturing"},{"issue":"6334","key":"10234_CR16","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1126\/science.aal4230","volume":"356","author":"A Caliskan","year":"2017","unstructured":"Caliskan, A., Bryson, J. J., & Narayanan, A. (2017). Semantics derived automatically from language corpora contain human-like biases. Nature, 356(6334), 183\u2013186. https:\/\/doi.org\/10.1126\/science.aal4230","journal-title":"Nature"},{"issue":"1","key":"10234_CR17","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1111\/j.1365-2575.2005.00183.x","volume":"15","author":"L Carter","year":"2005","unstructured":"Carter, L., & B\u00e9langer, F. (2005). The utilization of e-government services: Citizen trust, innovation and acceptance factors. Information Systems Journal, 15(1), 5\u201325. https:\/\/doi.org\/10.1111\/j.1365-2575.2005.00183.x","journal-title":"Information Systems Journal"},{"issue":"6","key":"10234_CR18","doi-asserted-by":"publisher","first-page":"1446","DOI":"10.1108\/ITP-01-2018-0009","volume":"32","author":"S Celik","year":"2019","unstructured":"Celik, S. (2019). Experiences of internet users regarding cyberhate. Information Technology & People, 32(6), 1446\u20131471. https:\/\/doi.org\/10.1108\/ITP-01-2018-0009","journal-title":"Information Technology & People"},{"key":"10234_CR19","doi-asserted-by":"publisher","unstructured":"Cheng, H.-F., Wang, R., Zhang, Z., O\u2019Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019). Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Paper 559, 1\u201312. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3290605.3300789","DOI":"10.1145\/3290605.3300789"},{"issue":"3","key":"10234_CR20","doi-asserted-by":"publisher","first-page":"319","DOI":"10.2307\/249008","volume":"13","author":"FD Davis","year":"1989","unstructured":"Davis, F. D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13(3), 319\u2013340. https:\/\/doi.org\/10.2307\/249008","journal-title":"MIS Quarterly"},{"key":"10234_CR21","unstructured":"FastAI (2021, March 7). Transfer learning in text. fastai. Retrieved from https:\/\/docs.fast.ai\/. Accessed 12 Jan 2022."},{"issue":"4","key":"10234_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3232676","volume":"51","author":"P Fortuna","year":"2018","unstructured":"Fortuna, P., & Nunes, S. (2018). A Survey on Automatic Detection of Hate Speech in Text. ACM Computing Survey, 51(4), 1\u201330. https:\/\/doi.org\/10.1145\/3232676","journal-title":"ACM Computing Survey"},{"issue":"2","key":"10234_CR23","doi-asserted-by":"publisher","first-page":"337","DOI":"10.25300\/MISQ\/2013\/37.2.01","volume":"37","author":"S Gregor","year":"2013","unstructured":"Gregor, S., & Hevner, A. R. (2013). Positioning and Presenting Design Science Research for Maximum Impact. MIS Quarterly, 37(2), 337\u2013355. https:\/\/doi.org\/10.25300\/MISQ\/2013\/37.2.01","journal-title":"MIS Quarterly"},{"issue":"6","key":"10234_CR24","doi-asserted-by":"publisher","first-page":"1622","DOI":"10.17705\/1jais.00649","volume":"21","author":"S Gregor","year":"2020","unstructured":"Gregor, S., Kruse, L. C., & Seidel, S. (2020). Research Perspectives: The Anatomy of a Design Principle. Journal of the Association for Information Systems, 21(6), 1622\u20131652. https:\/\/doi.org\/10.17705\/1jais.00649","journal-title":"Journal of the Association for Information Systems"},{"issue":"1","key":"10234_CR25","doi-asserted-by":"publisher","first-page":"51","DOI":"10.2307\/30036519","volume":"27","author":"D Greven","year":"2003","unstructured":"Greven, D., Karahanna, E., & Straub, D. W. (2003). Trust and TAM in Online Shopping: An Integrated Model. MIS Quarterly, 27(1), 51\u201390. https:\/\/doi.org\/10.2307\/30036519","journal-title":"MIS Quarterly"},{"issue":"2","key":"10234_CR26","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1609\/aimag.v40i2.2850","volume":"40","author":"D Gunning","year":"2019","unstructured":"Gunning, D., & Aha, D. W. (2019). DARPA\u2019s Explainable Artificial Intelligence (XAI). AI Magazine, 40(2), 44\u201358. https:\/\/doi.org\/10.1609\/aimag.v40i2.2850","journal-title":"AI Magazine"},{"key":"10234_CR27","doi-asserted-by":"publisher","DOI":"10.1007\/s10796-021-10156-2","author":"M Gupta","year":"2021","unstructured":"Gupta, M., Parra, C. M., & Dennehy, D. (2021). Questioning Racial and Gender Bias in AI-based Recommendations: Do Espoused National Cultural Values Matter? Information Systems Frontiers. https:\/\/doi.org\/10.1007\/s10796-021-10156-2","journal-title":"Information Systems Frontiers"},{"key":"10234_CR28","doi-asserted-by":"publisher","first-page":"1481","DOI":"10.1016\/j.dss.2005.12.003","volume":"42","author":"MS G\u00f6n\u00fcl","year":"2006","unstructured":"G\u00f6n\u00fcl, M. S., \u00d6nkal, D., & Lawrence, M. (2006). The effects of structural characteristics of explanations on use of a DSS. Decision Support Systems, 42, 1481\u20131493. https:\/\/doi.org\/10.1016\/j.dss.2005.12.003","journal-title":"Decision Support Systems"},{"key":"10234_CR29","doi-asserted-by":"publisher","unstructured":"Hevner, A. R. (2020). The duality of science: Knowledge in information systems research. Journal of Information Technology, 1-5. https:\/\/doi.org\/10.1177\/0268396220945714","DOI":"10.1177\/0268396220945714"},{"key":"10234_CR30","unstructured":"Hinduja, S., & Patchin, J. W. (2019). Cyberbullying Identification, Prevention, and Response. Cyberbullying Research Center. Retrieved from https:\/\/cyberbullying.org\/Cyberbullying-Identification-Prevention-Response-2019.pdf. Accessed 12 Jan 2022."},{"key":"10234_CR31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/info11020108","volume":"11","author":"J Howard","year":"2020","unstructured":"Howard, J., & Gugger, S. (2020). Fastai: A Layered API for Deep Learning. Information, 11, 1\u201326. https:\/\/doi.org\/10.3390\/info11020108","journal-title":"Information"},{"key":"10234_CR32","doi-asserted-by":"publisher","unstructured":"Howard, J., & Ruder, S. (2018). Universal Language Model Fine-Tuning for Text Classification. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Long Papers), Australia, 328\u2013339. https:\/\/doi.org\/10.18653\/v1\/P18-1031","DOI":"10.18653\/v1\/P18-1031"},{"issue":"3","key":"10234_CR33","doi-asserted-by":"publisher","first-page":"893","DOI":"10.1080\/07421222.2019.1628908","volume":"36","author":"Y Hu","year":"2019","unstructured":"Hu, Y., Xu, A., Hong, Y., Gal, D., Sinha, V., & Akkiraju, R. (2019). Generating Business Intelligence Through Social Media Analytics: Measuring Brand Personality with Consumer-, Employee-, and Firm-Generated Content. Journal of Management Information Systems, 36(3), 893\u2013930. https:\/\/doi.org\/10.1080\/07421222.2019.1628908","journal-title":"Journal of Management Information Systems"},{"issue":"2","key":"10234_CR34","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1016\/S0378-7206(03)00047-8","volume":"41","author":"HH Huang","year":"2003","unstructured":"Huang, H. H. (2003). Effects of multimedia on document browsing and navigation: An exploratory empirical investigation. Information & Management, 41(2), 189\u2013198. https:\/\/doi.org\/10.1016\/S0378-7206(03)00047-8","journal-title":"Information & Management"},{"key":"10234_CR35","unstructured":"Intel (2021). Bleep. Intel Corporations. Retrieved January from https:\/\/devmesh.intel.com\/projects\/bleep#about-section. Accessed 12 Jan 2022."},{"key":"10234_CR36","doi-asserted-by":"publisher","unstructured":"Ivari, J., Hansen, M. R. P., & Haj-Bolouri, A. (2018). A Framework for Light Resuability Evaluation of Design Principles in Design Science Research. 13th International Conference on Design Science Research and Information Systems and Technology: Designing for a Digital and Globalized World (DESRIST 2018), India. https:\/\/doi.org\/10.1007\/978-3-319-91800-6","DOI":"10.1007\/978-3-319-91800-6"},{"issue":"3","key":"10234_CR37","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1080\/0960085X.2020.1793697","volume":"30","author":"J Ivari","year":"2021","unstructured":"Ivari, J., Hansen, M. R. P., & Haj-Bolouri, A. (2021). A proposal for minimum reusability evaluation of design principles. European Journal of Information Systems, 30(3), 286\u2013303. https:\/\/doi.org\/10.1080\/0960085X.2020.1793697","journal-title":"European Journal of Information Systems"},{"key":"10234_CR38","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ijinfomgt.2018.09.003","volume":"44","author":"JL Jimenez-Marquez","year":"2019","unstructured":"Jimenez-Marquez, J. L., Gonzalez-Carrasco, I., Lopez-Cuadrado, J. L., & Ruiz-Mezcua, B. (2019). Towards a big data framework for analyzing social media content. International Journal of Information Management, 44, 1\u201312. https:\/\/doi.org\/10.1016\/j.ijinfomgt.2018.09.003","journal-title":"International Journal of Information Management"},{"key":"10234_CR39","unstructured":"Kaggle (2012). Detecting Insults in Social Commentary. Retrieved from https:\/\/www.kaggle.com\/c\/detectinginsults-in-social-commentary\/overview. Accessed 12 Jan 2022."},{"key":"10234_CR40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.knosys.2020.106458","volume":"210","author":"P Kapil","year":"2020","unstructured":"Kapil, P., & Ekbal, A. (2020). A deep neural network based multi-task learning approach to hate speech detection. Knowledge-Based Systems, 210, 1\u201321. https:\/\/doi.org\/10.1016\/j.knosys.2020.106458","journal-title":"Knowledge-Based Systems"},{"issue":"1","key":"10234_CR41","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/j.bushor.2018.08.004","volume":"62","author":"A Kaplan","year":"2019","unstructured":"Kaplan, A., & Haenlein, M. (2019). Siri, Siri, in my hand: Who\u2019s the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence. Business Horizons, 62(1), 15\u201325. https:\/\/doi.org\/10.1016\/j.bushor.2018.08.004","journal-title":"Business Horizons"},{"issue":"1","key":"10234_CR42","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1016\/j.bushor.2019.09.003","volume":"63","author":"A Kaplan","year":"2020","unstructured":"Kaplan, A., & Haenlein, M. (2020). Rulers of the world, unite! The challenges and opportunities of artificial intelligence. Business Horizons, 63(1), 37\u201350. https:\/\/doi.org\/10.1016\/j.bushor.2019.09.003","journal-title":"Business Horizons"},{"key":"10234_CR43","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.dss.2020.113302","volume":"134","author":"B Kim","year":"2020","unstructured":"Kim, B., Park, J., & Suh, J. (2020). Transparency and accountability in AI decision support: Explaining and visualizing convolutional neural networks for text information. Decision Support Systems, 134, 1\u201311. https:\/\/doi.org\/10.1016\/j.dss.2020.113302","journal-title":"Decision Support Systems"},{"issue":"3","key":"10234_CR44","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1080\/19331681.2020.1871149","volume":"18","author":"M Kunst","year":"2021","unstructured":"Kunst, M., Porten-Chee, P., Emmer, M., & Eilders, C. (2021). Do \u201cGood Citizens\u201d fight hate speech online? Effects of solidarity citizenship norms on user responses to hate comments. Journal of Technology & Politics, 18(3), 258\u2013273. https:\/\/doi.org\/10.1080\/19331681.2020.1871149","journal-title":"Journal of Technology & Politics"},{"key":"10234_CR45","unstructured":"K\u00fchl, N., Lobana, J., & Meske, C. (2019). Do you comply with AI? \u2013 Personalized explanations of learning algorithms and their impact on employees\u2019 compliance behavior. Fortieth International Conference on Information Systems 2019, 1\u20136. Retrieved from https:\/\/aisel.aisnet.org\/icis2019\/paperathon\/paperathon\/1\/. Accessed 12 Jan 2022."},{"key":"10234_CR46","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.artmed.2019.01.001","volume":"94","author":"J-B Lamy","year":"2019","unstructured":"Lamy, J.-B., Sekar, B., Guezennec, G., Bouaud, J., & Seroussi, B. (2019). Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach. Artificial Intelligence in Medicine, 94, 42\u201353. https:\/\/doi.org\/10.1016\/j.artmed.2019.01.001","journal-title":"Artificial Intelligence in Medicine"},{"issue":"1","key":"10234_CR47","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/j.dss.2011.06.001","volume":"52","author":"M Li","year":"2011","unstructured":"Li, M., & Gregor, S. (2011). Outcomes of effective explanations: Empowering citizens through online advice. Decision Support Systems, 52(1), 119\u2013132. https:\/\/doi.org\/10.1016\/j.dss.2011.06.001","journal-title":"Decision Support Systems"},{"key":"10234_CR48","doi-asserted-by":"publisher","DOI":"10.1007\/s10796-021-10205-w","author":"Y Li","year":"2021","unstructured":"Li, Y., & Kettinger, W. J. (2021). Testing the Relationship Between Information and Knowledge in Computer-Aided Decision-Making. Information Systems Frontiers. https:\/\/doi.org\/10.1007\/s10796-021-10205-w","journal-title":"Information Systems Frontiers"},{"key":"10234_CR49","doi-asserted-by":"publisher","first-page":"433","DOI":"10.3758\/s13428-016-0727-z","volume":"49","author":"L Litman","year":"2017","unstructured":"Litman, L., Robinson, J., & Abberbock, T. (2017). TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences. Behavior Research Methods, 49, 433\u2013442. https:\/\/doi.org\/10.3758\/s13428-016-0727-z","journal-title":"Behavior Research Methods"},{"issue":"8","key":"10234_CR50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0221152","volume":"14","author":"S MacAvaney","year":"2019","unstructured":"MacAvaney, S., Yao, H.-R., Yang, E., Russell, K., Goharian, N., & Frieder, O. (2019). Hate speech detection: Challenges and Solutions. PLoS ONE, 14(8), 1\u201316. https:\/\/doi.org\/10.1371\/journal.pone.0221152","journal-title":"PLoS ONE"},{"issue":"1","key":"10234_CR51","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1214\/aoms\/1177730491","volume":"18","author":"HB Mann","year":"1947","unstructured":"Mann, H. B., & Whitney, D. R. (1947). On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. The Annals of Mathematical Statistics, 18(1), 50\u201360. https:\/\/doi.org\/10.1214\/aoms\/1177730491","journal-title":"The Annals of Mathematical Statistics"},{"issue":"1","key":"10234_CR52","doi-asserted-by":"publisher","first-page":"73","DOI":"10.25300\/MISQ\/2014\/38.1.04","volume":"38","author":"D Martens","year":"2014","unstructured":"Martens, D., & Provost, F. (2014). Explaining Data-Driven Document Classifications. MIS Quarterly, 38(1), 73\u201399. https:\/\/doi.org\/10.25300\/MISQ\/2014\/38.1.04","journal-title":"MIS Quarterly"},{"issue":"6","key":"10234_CR53","doi-asserted-by":"publisher","first-page":"930","DOI":"10.1080\/1369118X.2017.1293130","volume":"20","author":"A Matamoros-Fern\u00e1ndez","year":"2017","unstructured":"Matamoros-Fern\u00e1ndez, A. (2017). Platformed racism: The mediation and circulation of an Australian race-based controversy on Twitter, Facebook and YouTube. Information, Communication & Society, 20(6), 930\u2013946. https:\/\/doi.org\/10.1080\/1369118X.2017.1293130","journal-title":"Information, Communication & Society"},{"key":"10234_CR54","doi-asserted-by":"publisher","unstructured":"Meske, C., & Bunde, E. (2020). Transparency and Trust in Human-AI-Interaction: The Role of Model-Agnostic Explanations in Computer Vision-Based Decision Support. In Degen H., & Reinerman-Jones L., (Eds.), Artificial Intelligence in HCI. HCII 2020. Lecture Notes in Computer Science, 12217, 54\u201369. Springer, Cham. https:\/\/doi.org\/10.1007\/978-3-030-50334-5_4","DOI":"10.1007\/978-3-030-50334-5_4"},{"key":"10234_CR55","doi-asserted-by":"publisher","unstructured":"Meske, C., Bunde, E., Schneider, J., & Gersch, M. (2020). Explainable Artificial Intelligence: Objectives, Stakeholders, and Future Research Opportunities. Information Systems Management, 1-11. https:\/\/doi.org\/10.1080\/10580530.2020.1849465","DOI":"10.1080\/10580530.2020.1849465"},{"key":"10234_CR56","unstructured":"Meske, C., & Amojo, I. J. (2020). Enterprise Social Bots as Perception-Benefactors of Social Network Affordances. Forty-First International Conference on Information Systems 2020, 1\u201317. Retrieved from https:\/\/aisel.aisnet.org\/icis2020\/social_media\/social_media\/5\/. Accessed 12 Jan 2022."},{"issue":"9","key":"10234_CR57","doi-asserted-by":"publisher","first-page":"779","DOI":"10.17705\/1jais.00408","volume":"16","author":"H Meth","year":"2015","unstructured":"Meth, H., Mueller, B., & Maedche, A. (2015). Designing a Requirement Mining System. Journal of the Association for Information Systems, 16(9), 779\u2013837. https:\/\/doi.org\/10.17705\/1jais.00408","journal-title":"Journal of the Association for Information Systems"},{"key":"10234_CR58","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.eswa.2020.113725","volume":"161","author":"S Modha","year":"2020","unstructured":"Modha, S., Majumder, P., Mandl, T., & Mandalia, C. (2020). Detecting and visualizing hate speech in social media: A cyber Watchdog for surveillance. Expert Systems with Applications, 161, 1\u201311. https:\/\/doi.org\/10.1016\/j.eswa.2020.113725","journal-title":"Expert Systems with Applications"},{"key":"10234_CR59","doi-asserted-by":"publisher","DOI":"10.1007\/s10796-021-10125-9","author":"S Motorny","year":"2021","unstructured":"Motorny, S., Sarnikar, S., & Noteboom, C. (2021). Design of an Intelligent Patient Decision aid Based on Individual Decision-Making Styles and Information Need Preferences. Information Systems Frontiers. https:\/\/doi.org\/10.1007\/s10796-021-10125-9","journal-title":"Information Systems Frontiers"},{"issue":"9","key":"10234_CR60","doi-asserted-by":"publisher","first-page":"1229","DOI":"10.1080\/1369118X.2019.1697339","volume":"24","author":"A Nienierza","year":"2019","unstructured":"Nienierza, A., Reinemann, C., Fawzi, N., Riesmeyer, C., & Neumann, K. (2019). Too dark to see? Explaining adolescents\u2018 contact with online extremism and their ability to recognize it. Information, Communication & Society, 24(9), 1229\u20131246. https:\/\/doi.org\/10.1080\/1369118X.2019.1697339","journal-title":"Information, Communication & Society"},{"key":"10234_CR61","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1016\/j.dss.2016.08.003","volume":"91","author":"H Park","year":"2016","unstructured":"Park, H., Bellamy, M. A., & Basole, R. C. (2016). Visual analytics for supply network management: System design and evaluation. Decision Support Systems, 91, 89\u2013102. https:\/\/doi.org\/10.1016\/j.dss.2016.08.003","journal-title":"Decision Support Systems"},{"issue":"2","key":"10234_CR62","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3371276","volume":"20","author":"D Paschalides","year":"2020","unstructured":"Paschalides, D., Stephanidis, D., Andreou, A., Orphanou, K., Pallis, G., Dikaiakos, M. D., & Markatos, E. (2020). MANDOLA: A Big-Data Processing and Visualization Platform for Monitoring and Detecting Hate Speech. ACM Transactions on Internet Technology, 20(2), 1\u201321. https:\/\/doi.org\/10.1145\/3371276","journal-title":"ACM Transactions on Internet Technology"},{"key":"10234_CR63","volume-title":"Qualitative Research & Evaluation Methods","author":"MQ Patton","year":"2014","unstructured":"Patton, M. Q. (2014). Qualitative Research & Evaluation Methods (4th ed.). Sage Publications Inc.","edition":"4"},{"key":"10234_CR64","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825\u20132830.","journal-title":"Journal of Machine Learning Research"},{"issue":"3","key":"10234_CR65","doi-asserted-by":"publisher","first-page":"45","DOI":"10.2753\/MIS0742-1222240302","volume":"24","author":"K Peffers","year":"2007","unstructured":"Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A Design Science Research Methodology for Information Systems Research. Journal of Management Information Systems, 24(3), 45\u201377. https:\/\/doi.org\/10.2753\/MIS0742-1222240302","journal-title":"Journal of Management Information Systems"},{"key":"10234_CR66","doi-asserted-by":"publisher","first-page":"1291","DOI":"10.10007\/s10796-019-09932-y","volume":"22","author":"S Peng","year":"2020","unstructured":"Peng, S., Wang, Y., Liu, C., & Chen, Z. (2020). TL-NER: S Transfer Learning Model for Chinese Named Entity Recognition. Information Systems Frontiers, 22, 1291\u20131304. https:\/\/doi.org\/10.10007\/s10796-019-09932-y","journal-title":"Information Systems Frontiers"},{"issue":"21","key":"10234_CR67","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/s19214654","volume":"19","author":"JC Pereira-Kohatsu","year":"2019","unstructured":"Pereira-Kohatsu, J. C., Quijano-Sanchez, L., Liberatore, F., & Camacho-Collados, M. (2019). Detecting and Monitoring Hate Speech in Twitter. Sensors, 19(21), 1\u201337. https:\/\/doi.org\/10.3390\/s19214654","journal-title":"Sensors"},{"key":"10234_CR68","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.eswa.2020.114120","volume":"166","author":"F Plaza-del-Arco","year":"2021","unstructured":"Plaza-del-Arco, F., Molina-Gonzalez, M., Urena-Lopez, L., & Martin-Valdivia, M. (2021). Comparing pre-trained language models for Spanish hate speech detection. Expert Systems with Applications, 166, 1\u201310. https:\/\/doi.org\/10.1016\/j.eswa.2020.114120","journal-title":"Expert Systems with Applications"},{"issue":"5\u20136","key":"10234_CR69","doi-asserted-by":"publisher","first-page":"413","DOI":"10.1080\/07370024.2020.1734931","volume":"35","author":"G Ramos","year":"2020","unstructured":"Ramos, G., Meek, C., Simard, P., Suh, J., & Ghorashi, S. (2020). Interactive machine teaching: A human-centered approach to building machine-learned models. Human-Computer Interaction, 35(5\u20136), 413\u2013451. https:\/\/doi.org\/10.1080\/07370024.2020.1734931","journal-title":"Human-Computer Interaction"},{"key":"10234_CR70","unstructured":"Schneider, J., Handali, J., Vlachos, M., & Meske, C. (2020). Deceptive AI Explanations: Creation and Detection. arXiv, 1\u20139. Retrieved from https:\/\/arxiv.org\/abs\/2001.07641. Accessed 12 Jan 2022."},{"issue":"2","key":"10234_CR71","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1057\/s41303-017-0039-0","volume":"27","author":"S Seidel","year":"2018","unstructured":"Seidel, S., Kruse, L. C., Sz\u00e9kely, N., Gau, M., & Stieger, D. (2018). Design principles for sensemaking support systems in environmental sustainability transformations. European Journal of Information Systems, 27(2), 221\u2013247. https:\/\/doi.org\/10.1057\/s41303-017-0039-0","journal-title":"European Journal of Information Systems"},{"issue":"4","key":"10234_CR72","doi-asserted-by":"publisher","first-page":"1459","DOI":"10.25300\/MISQ\/2020\/14870","volume":"44","author":"D Shin","year":"2020","unstructured":"Shin, D., He, S., Lee, G. M., Whinston, A. B., Cetintas, S., & Lee, K.-C. (2020). Enhancing Social Media Analysis with Visual Data Analytics: A Deep Learning Approach. MIS Quarterly, 44(4), 1459\u20131492. https:\/\/doi.org\/10.25300\/MISQ\/2020\/14870","journal-title":"MIS Quarterly"},{"issue":"4","key":"10234_CR73","doi-asserted-by":"publisher","first-page":"376","DOI":"10.1111\/j.1469-7610.2001.01846.x","volume":"49","author":"PK Smith","year":"2008","unstructured":"Smith, P. K., Mahdavi, J., Carvalho, M., Fisher, S., Russel, S., & Tippett, N. (2008). Cyberbullying: Its nature and impact in secondary school pupils. Journal of Child Psychology and Psychiatry, 49(4), 376\u2013385. https:\/\/doi.org\/10.1111\/j.1469-7610.2001.01846.x","journal-title":"Journal of Child Psychology and Psychiatry"},{"key":"10234_CR74","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1016\/j.ijinfomgt.2017.12.002","volume":"39","author":"S Stieglitz","year":"2018","unstructured":"Stieglitz, S., Mirbabaie, M., Ross, B., & Neuberger, C. (2018). Social media analytics \u2013 Challenges in topic discovery, data collection, and data preparation. International Journal of Information Management, 39, 156\u2013168. https:\/\/doi.org\/10.1016\/j.ijinfomgt.2017.12.002","journal-title":"International Journal of Information Management"},{"issue":"7","key":"10234_CR75","doi-asserted-by":"publisher","first-page":"938","DOI":"10.1016\/S1470-2045(19)30333-X","volume":"20","author":"P Tschandl","year":"2019","unstructured":"Tschandl, P., Codella, N., Akay, B. N., Argenziano, G., Braun, R., Cabo, H., Gutman, D., Halpern, A., Helba, B., Hofmann-Wellenhof, R., Lallas, A., Lapins, J., Longo, C., Malvehy, J., Marchetti, M. A., Marghoob, A., Menzies, S., Oakley, A., Paoli, J., \u2026 Kittler, H. (2019). Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: An open, web-based, international, diagnostic study. The Lancet Oncology, 20(7), 938\u2013947. https:\/\/doi.org\/10.1016\/S1470-2045(19)30333-X","journal-title":"The Lancet Oncology"},{"key":"10234_CR76","doi-asserted-by":"publisher","first-page":"1229","DOI":"10.1038\/s41591\/s41591-020-0942-0","volume":"26","author":"P Tschandl","year":"2020","unstructured":"Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., Paoli, J., Puig, S., Rosendahl, C., Soyer, H. P., Zalaudek, I., & Kittler, H. (2020). Human-computer collaboration for skin cancer recognition. Nature Medicine, 26, 1229\u20131234. https:\/\/doi.org\/10.1038\/s41591\/s41591-020-0942-0","journal-title":"Nature Medicine"},{"key":"10234_CR77","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1007\/s10676-019-09516-z","volume":"22","author":"S Ullmann","year":"2020","unstructured":"Ullmann, S., & Tomalin, M. (2020). Quarantining online hate speech: Technical and ethical perspectives. Ethics and Information Technology, 22, 59\u201380. https:\/\/doi.org\/10.1007\/s10676-019-09516-z","journal-title":"Ethics and Information Technology"},{"key":"10234_CR78","unstructured":"United Nations (2019). United Nations Strategy and Plan of Action on Hate Speech. Retrieved from https:\/\/www.un.org\/en\/genocideprevention\/hate-speech-strategy.shtml. Accessed 12 Jan 2022."},{"issue":"1","key":"10234_CR79","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1007\/s10796-020-09994-3","volume":"23","author":"S Vallejos","year":"2021","unstructured":"Vallejos, S., Alonso, D. G., Caimmi, B., Berdun, L., Armentano, M. G., & Soria, A. (2021). Mining Social Networks to Detect Traffict Incidents. Information Systems Frontiers, 23(1), 115\u2013134. https:\/\/doi.org\/10.1007\/s10796-020-09994-3","journal-title":"Information Systems Frontiers"},{"key":"10234_CR80","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1057\/ejis.2014.36","volume":"25","author":"J Venable","year":"2016","unstructured":"Venable, J., Pries-Heje, J., & Baskerville, R. (2016). FEDS: A Framework for Evaluation in Design Science Research. European Journal of Information Systems, 25, 77\u201389. https:\/\/doi.org\/10.1057\/ejis.2014.36","journal-title":"European Journal of Information Systems"},{"issue":"3","key":"10234_CR81","doi-asserted-by":"publisher","first-page":"425","DOI":"10.2307\/30036540","volume":"27","author":"V Venkatesh","year":"2003","unstructured":"Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425\u2013478. https:\/\/doi.org\/10.2307\/30036540","journal-title":"MIS Quarterly"},{"issue":"12","key":"10234_CR82","doi-asserted-by":"publisher","first-page":"804","DOI":"10.17705\/1jais.00444","volume":"17","author":"P Vitharana","year":"2016","unstructured":"Vitharana, P., Zahedi, F. M., & Hemant, J. K. (2016). Enhancing Analysts\u00b4 Mental Model for Improving Requirements Elicitation: A Two-stage Theoretical Framework and Empirical Results. Journal of the Association for Information Systems, 17(12), 804\u2013840. https:\/\/doi.org\/10.17705\/1jais.00444","journal-title":"Journal of the Association for Information Systems"},{"key":"10234_CR83","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.artint.2020.103404","volume":"291","author":"J van der Waa","year":"2021","unstructured":"van der Waa, J., Nieuwburg, E., Cremers, A., & Neerincx, M. (2021). Evaluating XAI: A comparison of rule-based and example-based explanations. Artificial Intelligence, 291, 1\u201319. https:\/\/doi.org\/10.1016\/j.artint.2020.103404","journal-title":"Artificial Intelligence"},{"issue":"2","key":"10234_CR84","doi-asserted-by":"publisher","first-page":"293","DOI":"10.2307\/20650293","volume":"33","author":"W Wang","year":"2009","unstructured":"Wang, W., & Benbasat, I. (2009). Interactive Decision Aids for Consumer Decision Making in E-Commerce: The Influence of Perceived Strategy Restrictiveness. MIS Quarterly, 33(2), 293\u2013320. https:\/\/doi.org\/10.2307\/20650293","journal-title":"MIS Quarterly"},{"issue":"6","key":"10234_CR85","doi-asserted-by":"publisher","first-page":"921","DOI":"10.1177\/0093650219855330","volume":"47","author":"C Wilhelm","year":"2020","unstructured":"Wilhelm, C., Joeckel, S., & Ziegler, I. (2020). Reporting Hate Comments: Investigating the Effects of Deviance Characteristics, Neutralization Strategies, and Users\u00b4 Moral Orientation. Communication Research, 47(6), 921\u2013944. https:\/\/doi.org\/10.1177\/0093650219855330","journal-title":"Communication Research"},{"key":"10234_CR86","doi-asserted-by":"publisher","first-page":"1664","DOI":"10.1016\/j.dss.2006.09.003","volume":"43","author":"MH Zack","year":"2007","unstructured":"Zack, M. H. (2007). The role of decision support systems in an indeterminate world. Decision Support Systems, 43, 1664\u20131674. https:\/\/doi.org\/10.1016\/j.dss.2006.09.003","journal-title":"Decision Support Systems"},{"key":"10234_CR87","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1016\/j.dss.2014.08.005","volume":"67","author":"KZK Zhang","year":"2014","unstructured":"Zhang, K. Z. K., Zhao, S. J., Cheung, C. M. K., & Lee, M. K. O. (2014). Examining the influence of online reviews on consumers\u00b4 decision-making: A heuristic-systematic model. Decision Support Systems, 67, 78\u201389. https:\/\/doi.org\/10.1016\/j.dss.2014.08.005","journal-title":"Decision Support Systems"}],"container-title":["Information Systems Frontiers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10796-021-10234-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10796-021-10234-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10796-021-10234-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,3,2]],"date-time":"2022-03-02T12:13:50Z","timestamp":1646223230000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10796-021-10234-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,2]]},"references-count":87,"alternative-id":["10234"],"URL":"https:\/\/doi.org\/10.1007\/s10796-021-10234-5","relation":{},"ISSN":["1387-3326","1572-9419"],"issn-type":[{"value":"1387-3326","type":"print"},{"value":"1572-9419","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,2]]},"assertion":[{"value":"13 December 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 March 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}