{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T21:43:20Z","timestamp":1780695800675,"version":"3.54.1"},"reference-count":189,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,8,24]],"date-time":"2023-08-24T00:00:00Z","timestamp":1692835200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,24]],"date-time":"2023-08-24T00:00:00Z","timestamp":1692835200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Cogn Comput"],"published-print":{"date-parts":[[2024,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Recent years have seen a tremendous growth in Artificial Intelligence (AI)-based methodological development in a broad range of\u00a0domains. In this\u00a0rapidly evolving field, large number of methods are being reported using machine learning (ML) and Deep Learning (DL) models. Majority of these models are\u00a0inherently complex and lacks explanations\u00a0of the decision making process\u00a0causing these models to be termed as 'Black-Box'. One of the major bottlenecks to adopt such models in mission-critical application domains, such as\u00a0banking, e-commerce, healthcare, and public services and\u00a0safety, is the difficulty in\u00a0interpreting them. Due to the rapid proleferation of these AI models, explaining their learning and decision making process are\u00a0getting harder which require\u00a0transparency\u00a0and easy predictability.\u00a0Aiming to collate the current state-of-the-art in interpreting the black-box models,\u00a0this study provides a comprehensive analysis of the\u00a0explainable AI (XAI)\u00a0models. To reduce\u00a0false negative and false positive outcomes of these back-box models,\u00a0finding flaws in them\u00a0is still difficult and inefficient. In this paper, the development of XAI is reviewed meticulously\u00a0through careful selection and analysis of the current state-of-the-art of XAI research. It\u00a0also provides a comprehensive and in-depth evaluation of the XAI frameworks and their efficacy to serve as a starting point of XAI for applied and theoretical researchers. Towards the end, it\u00a0highlights emerging\u00a0and\u00a0critical issues pertaining to XAI research to\u00a0showcase major, model-specific trends\u00a0for better explanation, enhanced\u00a0transparency, and improved\u00a0prediction\u00a0accuracy.<\/jats:p>","DOI":"10.1007\/s12559-023-10179-8","type":"journal-article","created":{"date-parts":[[2023,8,24]],"date-time":"2023-08-24T09:02:43Z","timestamp":1692867763000},"page":"45-74","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1639,"title":["Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence"],"prefix":"10.1007","volume":"16","author":[{"given":"Vikas","family":"Hassija","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6730-3060","authenticated-orcid":false,"given":"Vinay","family":"Chamola","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Atmesh","family":"Mahapatra","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abhinandan","family":"Singal","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Divyansh","family":"Goel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kaizhu","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Simone","family":"Scardapane","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Indro","family":"Spinelli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2037-8348","authenticated-orcid":false,"given":"Mufti","family":"Mahmud","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amir","family":"Hussain","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,8,24]]},"reference":[{"key":"10179_CR1","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1007\/s12559-017-9519-8","volume":"10","author":"M Anbar","year":"2018","unstructured":"Anbar M, Abdullah R, Al-Tamimi BN, Hussain A. A machine learning approach to detect router advertisement flooding attacks in next-generation ipv6 networks. Cogn Comput. 2018;10:201\u201314.","journal-title":"Cogn Comput"},{"issue":"3","key":"10179_CR2","doi-asserted-by":"publisher","first-page":"927","DOI":"10.1007\/s12559-022-10012-8","volume":"14","author":"E Osaba","year":"2022","unstructured":"Osaba E, Del Ser J, Martinez AD, Hussain A. Evolutionary multitask optimization: A methodological overview, challenges, and future research directions. Cogn Comput. 2022;14(3):927\u201354.","journal-title":"Cogn Comput"},{"issue":"1","key":"10179_CR3","first-page":"29","volume":"34","author":"XH Li","year":"2022","unstructured":"Li XH, Cao CC, Shi Y, Bai W, Gao H, Qiu L, Wang C, Gao Y, Zhang S, Xue X, Chen L. A survey of data-driven and knowledge-aware explainable ai. IEEE Trans Knowl Data Eng. 2022;34(1):29\u201349.","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"3","key":"10179_CR4","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L. Imagenet large scale visual recognition challenge. Int J Comput Vision. 2015;115(3):211\u201352.","journal-title":"Int J Comput Vision"},{"key":"10179_CR5","doi-asserted-by":"crossref","unstructured":"Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Doll\u00e1r P, Zitnick CL. Microsoft coco: Common objects in context. 2014. p. 740\u201355.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"10179_CR6","unstructured":"Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ, editor. Advances in Neural Information Processing Systems 25. Curran Associates, Inc.; 2012. p. 1097\u2013105. http:\/\/papers.nips.cc\/paper\/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf."},{"key":"10179_CR7","unstructured":"Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv. 2014."},{"key":"10179_CR8","doi-asserted-by":"crossref","unstructured":"Antol S, Agrawal A, Lu J, Mitchell M, Batra D, Zitnick CL, Parikh D. VQA: Visual question answering. CoRR. 2015;abs\/1505.00468.\u00a0http:\/\/arxiv.org\/abs\/1505.00468.","DOI":"10.1109\/ICCV.2015.279"},{"issue":"7587","key":"10179_CR9","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","volume":"529","author":"D Silver","year":"2016","unstructured":"Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D. Mastering the game of Go with deep neural networks and tree search. Nature. 2016;529(7587):484\u20139.","journal-title":"Nature"},{"key":"10179_CR10","doi-asserted-by":"crossref","unstructured":"Sharma P, Jain S, Gupta S, Chamola V. Role of machine learning and deep learning in securing 5g-driven industrial iot applications. Ad Hoc Netw. 2021;123:102685.","DOI":"10.1016\/j.adhoc.2021.102685"},{"key":"10179_CR11","doi-asserted-by":"crossref","unstructured":"Brown N, Sandholm T. Superhuman ai for multiplayer poker.\u00a0Science. 2019;365:eaay2400.","DOI":"10.1126\/science.aay2400"},{"key":"10179_CR12","unstructured":"Berner C, Brockman G, Chan B, Cheung V, Debiak P, Dennison C, Farhi D, Fischer Q, Hashme S, Hesse C, J\u00f3zefowicz R, Gray S, Olsson C, Pachocki J, Petrov M, de Oliveira Pinto HP, Raiman J, Salimans T, Schlatter J, Schneider J, Sidor S, Sutskever I, Tang J, Wolski F, Zhang S. Dota 2 with large scale deep reinforcement learning. CoRR. 2019;abs\/1912.06680.\u00a0http:\/\/arxiv.org\/abs\/1912.06680."},{"key":"10179_CR13","unstructured":"Todorov G. 65 artificial intelligence statistics for 2021 and beyond. 2021.\u00a0https:\/\/www.semrush.com\/blog\/artificial-intelligence-stats\/."},{"key":"10179_CR14","doi-asserted-by":"publisher","first-page":"698","DOI":"10.1007\/s12559-021-09851-8","volume":"13","author":"A Roy","year":"2021","unstructured":"Roy A, Banerjee B, Hussain A, Poria S. Discriminative dictionary design for action classification in still images and videos. Cogn Comput. 2021;13:698\u2013708.","journal-title":"Cogn Comput"},{"issue":"7","key":"10179_CR15","doi-asserted-by":"publisher","first-page":"1240","DOI":"10.1049\/iet-ipr.2019.1164","volume":"14","author":"G Bansal","year":"2020","unstructured":"Bansal G, Chamola V, Narang P, Kumar S, Raman S. Deep3dscan: Deep residual network and morphological descriptor based framework forlung cancer classification and 3d segmentation. IET Image Proc. 2020;14(7):1240\u20137.","journal-title":"IET Image Proc"},{"issue":"3","key":"10179_CR16","doi-asserted-by":"publisher","first-page":"989","DOI":"10.1007\/s12559-022-09993-3","volume":"14","author":"B Li","year":"2022","unstructured":"Li B, Xu Z, Hong N, Hussain A. A bibliometric study and science mapping research of intelligent decision. Cogn Comput. 2022;14(3):989\u20131008.","journal-title":"Cogn Comput"},{"key":"10179_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s12559-020-09773-x","volume":"13","author":"M Mahmud","year":"2021","unstructured":"Mahmud M, Kaiser MS, McGinnity TM, Hussain A. Deep learning in mining biological data. Cogn Comput. 2021;13:1\u201333.","journal-title":"Cogn Comput"},{"key":"10179_CR18","doi-asserted-by":"publisher","DOI":"10.1016\/j.scs.2020.102552","volume":"66","author":"V Hassija","year":"2021","unstructured":"Hassija V, Chamola V, Bajpai BC, Zeadally S, et al. Security issues in implantable medical devices: Fact or fiction? Sustain Cities Soc. 2021;66: 102552.","journal-title":"Sustain Cities Soc"},{"key":"10179_CR19","doi-asserted-by":"crossref","unstructured":"Rohmetra H, Raghunath N, Narang P, Chamola V, Guizani M, Lakkaniga NR. Ai-enabled remote monitoring of vital signs for covid-19: Methods, prospects and challenges. Computing. 2021;1\u201327.","DOI":"10.1007\/s00607-021-00937-7"},{"issue":"3","key":"10179_CR20","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1109\/MWC.001.2000428","volume":"28","author":"T Alladi","year":"2021","unstructured":"Alladi T, Kohli V, Chamola V, Yu FR, Guizani M. Artificial intelligence (ai)-empowered intrusion detection architecture for the internet of vehicles. IEEE Wirel Commun. 2021;28(3):144\u20139.","journal-title":"IEEE Wirel Commun"},{"key":"10179_CR21","doi-asserted-by":"crossref","unstructured":"Barredo Arrieta A, D\u00edaz-Rodr\u00edguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, Garcia S, Gil-Lopez S, Molina D, Benjamins R, Chatila R, Herrera F. Explainable artificial intelligence (xai): concepts, taxonomies, opportunities and challenges toward responsible ai. Inf Fusion. 2020;58:82\u2013115.","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"10179_CR22","unstructured":"Das A, Rad P. Opportunities and challenges in explainable artificial intelligence (XAI): A survey. CoRR. 2020;abs\/2006.11371.\u00a0https:\/\/arxiv.org\/abs\/2006.11371."},{"key":"10179_CR23","unstructured":"Khaleghi B. An Explanation of What, Why, and How of eXplainable AI (XAI). 2020.\u00a0https:\/\/towardsdatascience.com\/an-explanation-of-what-why-and-how-of-explainable-ai-xai-117d9c441265."},{"issue":"16","key":"10179_CR24","doi-asserted-by":"publisher","first-page":"17581","DOI":"10.1109\/JSEN.2021.3071290","volume":"21","author":"T Anand","year":"2021","unstructured":"Anand T, Sinha S, Mandal M, Chamola V, Yu FR. Agrisegnet: Deep aerial semantic segmentation framework for IoT-assisted precision agriculture. IEEE Sens J. 2021;21(16):17581\u201390.","journal-title":"IEEE Sens J"},{"issue":"6","key":"10179_CR25","doi-asserted-by":"publisher","first-page":"4448","DOI":"10.1109\/JIOT.2020.3027095","volume":"8","author":"P Chhikara","year":"2020","unstructured":"Chhikara P, Tekchandani R, Kumar N, Chamola V, Guizani M. Dcnn-ga: A deep neural net architecture for navigation of uav in indoor environment. IEEE Internet Things J. 2020;8(6):4448\u201360.","journal-title":"IEEE Internet Things J"},{"key":"10179_CR26","doi-asserted-by":"crossref","unstructured":"Chamola V, Goyal A, Sharma P, Hassija V, Binh HTT, Saxena V. Artificial intelligence-assisted blockchain-based framework for smart and secure EMR management. Neural Comput Appl. 2022;1\u201311.","DOI":"10.1007\/s00521-022-07087-7"},{"key":"10179_CR27","doi-asserted-by":"crossref","unstructured":"Shen Y, Ding N, Zheng HT, Li Y, Yang M. Modeling relation paths for knowledge graph completion. IEEE Trans Knowl Data Eng. 2021;33(11):3607\u201317.","DOI":"10.1109\/TKDE.2020.2970044"},{"key":"10179_CR28","doi-asserted-by":"crossref","unstructured":"Lu S, Liu M, Yin L, Yin Z, Liu X, Zheng W, Kong X. The multi-modal fusion in visual question answering: A review of attention mechanisms. PeerJ Comput Sci. 2023;9.","DOI":"10.7717\/peerj-cs.1400"},{"issue":"3","key":"10179_CR29","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1016\/j.icte.2022.04.007","volume":"8","author":"M Wazid","year":"2022","unstructured":"Wazid M, Das AK, Chamola V, Park Y. Uniting cyber security and machine learning: Advantages, challenges and future research. ICT Express. 2022;8(3):313\u201321.","journal-title":"ICT Express"},{"key":"10179_CR30","doi-asserted-by":"publisher","DOI":"10.1016\/j.adhoc.2021.102537","volume":"119","author":"V Hassija","year":"2021","unstructured":"Hassija V, Batra S, Chamola V, Anand T, Goyal P, Goyal N, Guizani M. A blockchain and deep neural networks-based secure framework for enhanced crop protection. Ad Hoc Netw. 2021;119: 102537.","journal-title":"Ad Hoc Netw"},{"issue":"3","key":"10179_CR31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3418205","volume":"21","author":"P Garg","year":"2021","unstructured":"Garg P, Chakravarthy AS, Mandal M, Narang P, Chamola V, Guizani M. Isdnet: Ai-enabled instance segmentation of aerial scenes for smart cities. ACM Trans Internet Technol (TOIT). 2021;21(3):1\u201318.","journal-title":"ACM Trans Internet Technol (TOIT)"},{"key":"10179_CR32","doi-asserted-by":"crossref","unstructured":"Ahmed F, Sultana S, Reza MT, Joy SKS, Golam M. Interpretable movie review analysis using machine learning and transformer models leveraging xai. 2023.","DOI":"10.1109\/CSDE56538.2022.10089294"},{"issue":"1","key":"10179_CR33","doi-asserted-by":"publisher","first-page":"236","DOI":"10.1109\/JIOT.2021.3098051","volume":"9","author":"S Singh","year":"2021","unstructured":"Singh S, Sulthana R, Shewale T, Chamola V, Benslimane A, Sikdar B. Machine-learning-assisted security and privacy provisioning for edge computing: A survey. IEEE Internet Things J. 2021;9(1):236\u201360.","journal-title":"IEEE Internet Things J"},{"key":"10179_CR34","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C. \u201cWhy should I trust you?\u201d: Explaining the predictions of any classifier. CoRR. 2016;abs\/1602.04938.\u00a0http:\/\/arxiv.org\/abs\/1602.04938.","DOI":"10.1145\/2939672.2939778"},{"issue":"5","key":"10179_CR35","first-page":"417","volume":"5","author":"AM Chekroud","year":"2018","unstructured":"Chekroud AM, Zotti RJ, Shehzad Z, Gueorguieva R, Johnson MK, Trivedi MH, Cannon TD. Cross-trial prediction of treatment outcome in depression: A machine learning approach. Lancet Psychiat. 2018;5(5):417\u201325.","journal-title":"Lancet Psychiat"},{"key":"10179_CR36","unstructured":"Doshi-Velez F, Kim B. Towards a rigorous science of interpretable machine learning. 2017.\u00a0https:\/\/arxiv.org\/abs\/1702.08608."},{"key":"10179_CR37","doi-asserted-by":"crossref","unstructured":"Wang D, Yang Q, Abdul A, Lim B. Designing theory-driven user-centric explainable ai. 2019.","DOI":"10.1145\/3290605.3300831"},{"key":"10179_CR38","doi-asserted-by":"crossref","unstructured":"Lapuschkin S, W\u00e4ldchen S, Binder A, Montavon G, Samek W, M\u00fcller K. Unmasking clever hans predictors and assessing what machines really learn. CoRR. 2019;abs\/1902.10178. Available:\u00a0http:\/\/arxiv.org\/abs\/1902.10178.","DOI":"10.1038\/s41467-019-08987-4"},{"issue":"11","key":"10179_CR39","doi-asserted-by":"publisher","first-page":"4793","DOI":"10.1109\/TNNLS.2020.3027314","volume":"32","author":"E Tjoa","year":"2021","unstructured":"Tjoa E, Guan C. A survey on explainable artificial intelligence (xai): Toward medical xai. IEEE Trans Neural Netw Learn Sys. 2021;32(11):4793\u2013813.","journal-title":"IEEE Trans Neural Netw Learn Sys"},{"key":"10179_CR40","unstructured":"Ghorbani A, Wexler J, Zou J, Kim B. Towards automatic concept-based explanations. 2019.\u00a0https:\/\/arxiv.org\/abs\/1902.03129."},{"key":"10179_CR41","doi-asserted-by":"crossref","unstructured":"Selvaraju RR, Das A, Vedantam R, Cogswell M, Parikh D, Batra D. Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization. CoRR. 2016;abs\/1610.02391. Available:\u00a0http:\/\/arxiv.org\/abs\/1610.02391.","DOI":"10.1109\/ICCV.2017.74"},{"key":"10179_CR42","unstructured":"Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A. Learning deep features for discriminative localization. CoRR. 2015;abs\/1512.04150.\u00a0http:\/\/arxiv.org\/abs\/1512.04150."},{"key":"10179_CR43","unstructured":"Samek W, Binder A, Montavon G, Bach S, M\u00fcller K. Evaluating the visualization of what a deep neural network has learned. CoRR. 2015;abs\/1509.06321.\u00a0http:\/\/arxiv.org\/abs\/1509.06321."},{"key":"10179_CR44","unstructured":"Becker S, Ackermann M, Lapuschkin S, M\u00fcller K, Samek W. Interpreting and explaining deep neural networks for classification of audio signals. CoRR. 2018;abs\/1807.03418.\u00a0http:\/\/arxiv.org\/abs\/1807.03418."},{"key":"10179_CR45","doi-asserted-by":"crossref","unstructured":"Arras L, Horn F, Montavon G, M\u00fcller KR, Samek W. \u201cWhat is relevant in a text document?\u201d: An interpretable machine learning approach. PLoS ONE. 2017;12:E0181142.","DOI":"10.1371\/journal.pone.0181142"},{"key":"10179_CR46","unstructured":"Zeiler MD, Fergus R. Visualizing and understanding convolutional networks. CoRR. 2013;abs\/1311.2901.\u00a0http:\/\/arxiv.org\/abs\/1311.2901."},{"key":"10179_CR47","unstructured":"Kim B, Wattenberg M, Gilmer J, Cai C, Wexler J, Viegas F, Sayres R. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). 2017. Available:\u00a0https:\/\/arxiv.org\/abs\/1711.11279."},{"key":"10179_CR48","unstructured":"Raghu M, Gilmer J, Yosinski J, Sohl-Dickstein J. Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. 2017.\u00a0https:\/\/arxiv.org\/abs\/1706.05806."},{"key":"10179_CR49","doi-asserted-by":"crossref","unstructured":"Silva A, Schrum M, Hedlund-Botti E, Gopalan N, Gombolay M. Explainable artificial intelligence: Evaluating the objective and subjective impacts of xai on human-agent interaction. Int J Hum-Comput Interact. 2022;1\u201315.","DOI":"10.1080\/10447318.2022.2101698"},{"key":"10179_CR50","doi-asserted-by":"publisher","unstructured":"Mohseni S, Zarei N, Ragan ED. A multidisciplinary survey and framework for design and evaluation of explainable ai systems. ACM Trans Interact Intell Syst. 2021;11(3\u20134). https:\/\/doi.org\/10.1145\/3387166.","DOI":"10.1145\/3387166"},{"key":"10179_CR51","doi-asserted-by":"crossref","unstructured":"Liu D, Cao Z, Jiang H, Zhou S, Xiao Z, Zeng F. Concurrent low-power listening: A new design paradigm for duty-cycling communication. ACM Trans Sen Netw. 2022;19(1).","DOI":"10.1145\/3517013"},{"issue":"17","key":"10179_CR52","doi-asserted-by":"publisher","first-page":"15538","DOI":"10.1109\/JIOT.2022.3181607","volume":"9","author":"X Shen","year":"2022","unstructured":"Shen X, Jiang H, Liu D, Yang K, Deng F, Lui JCS, Luo J. Pupilrec: leveraging pupil morphology for recommending on smartphones. IEEE Internet Things J. 2022;9(17):15538\u201353.","journal-title":"IEEE Internet Things J"},{"key":"10179_CR53","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2022.108120","volume":"240","author":"Y Ren","year":"2022","unstructured":"Ren Y, Jiang H, Ji N, Yu H. Tbsm: A traffic burst-sensitive model for short-term prediction under special events. Knowl-Based Syst. 2022;240: 108120.","journal-title":"Knowl-Based Syst"},{"issue":"2","key":"10179_CR54","doi-asserted-by":"publisher","first-page":"1536","DOI":"10.1109\/TIV.2022.3221927","volume":"8","author":"Y Ren","year":"2022","unstructured":"Ren Y, Jiang H, Feng X, Zhao Y, Liu R, Yu H. Acp-based modeling of the parallel vehicular crowd sensing system: Framework, components and an application example. IEEE Trans Intell Veh. 2022;8(2):1536\u201348.","journal-title":"IEEE Trans Intell Veh"},{"key":"10179_CR55","volume-title":"Robust intelligence and trust in autonomous systems","author":"Mittu R, Sofge D, Wagner A, Lawless W","year":"2016","unstructured":"Mittu R, Sofge D, Wagner A, Lawless W. Robust intelligence and trust in autonomous systems. 2016."},{"key":"10179_CR56","volume-title":"Effects of augmented situational awareness on driver trust in semi-autonomous vehicle operation","author":"Petersen L, Tilbury DM, Yang XY, Robert LP","year":"2017","unstructured":"Petersen L, Tilbury DM, Yang XY, Robert LP. Effects of augmented situational awareness on driver trust in semi-autonomous vehicle operation. 2017."},{"key":"10179_CR57","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1145\/3173386.3177057","volume-title":"Companion of the 2018 ACM\/IEEE International Conference on Human-Robot Interaction, ser. HRI \u201918","author":"Haspiel J, Du N, Meyerson J, Robert LP, Tilbury D, Yang XJ, Pradhan AK","year":"2018","unstructured":"Haspiel J, Du N, Meyerson J, Robert LP, Tilbury D, Yang XJ, Pradhan AK. Explanations and expectations: Trust building in automated vehicles. In: Companion of the 2018 ACM\/IEEE International Conference on Human-Robot Interaction, ser. HRI \u201918. New York, NY, USA: Association for Computing Machinery; 2018. p. 119\u201320. https:\/\/doi.org\/10.1145\/3173386.3177057."},{"issue":"3","key":"10179_CR58","doi-asserted-by":"publisher","first-page":"3307","DOI":"10.1007\/s11069-023-05988-x","volume":"117","author":"X Xie","year":"2023","unstructured":"Xie X, Huang L, Marson SM, Wei G. Emergency response process for sudden rainstorm and flooding: Scenario deduction and Bayesian network analysis using evidence theory and knowledge meta-theory. Nat Hazards. 2023;117(3):3307\u201329.","journal-title":"Nat Hazards"},{"issue":"11","key":"10179_CR59","doi-asserted-by":"publisher","first-page":"2909","DOI":"10.1093\/comjnl\/bxac085","volume":"65","author":"P Chen","year":"2022","unstructured":"Chen P, Liu H, Xin R, Carval T, Zhao J, Xia Y, Zhao Z. Effectively detecting operational anomalies in large-scale IoT data infrastructures by using a gan-based predictive model. Comput J. 2022;65(11):2909\u201325.","journal-title":"Comput J"},{"issue":"3","key":"10179_CR60","doi-asserted-by":"publisher","first-page":"2138","DOI":"10.1177\/1460458219900452","volume":"26","author":"K Cresswell","year":"2020","unstructured":"Cresswell K, Callaghan M, Khan S, Sheikh Z, Mozaffar H, Sheikh A. Investigating the use of data-driven artificial intelligence in computerised decision support systems for health and social care: A systematic review. Health Inform J. 2020;26(3):2138\u201347.","journal-title":"Health Inform J"},{"issue":"11","key":"10179_CR61","doi-asserted-by":"publisher","first-page":"5762","DOI":"10.1109\/TAC.2021.3124750","volume":"67","author":"B Li","year":"2021","unstructured":"Li B, Tan Y, Wu A, Duan G. A distributionally robust optimization based method for stochastic model predictive control. IEEE Trans Autom Control. 2021;67(11):5762\u201376.","journal-title":"IEEE Trans Autom Control"},{"key":"10179_CR62","doi-asserted-by":"crossref","unstructured":"Qu Z, Liu X, Zheng M. Temporal-spatial quantum graph convolutional neural network based on schr\u00f6dinger approach for traffic congestion prediction. IEEE Trans Intell Transp Syst. 2022.","DOI":"10.1109\/TITS.2022.3203791"},{"key":"10179_CR63","unstructured":"Leodolter W. Ai-based prediction in clinical settings: Can we trust it? 2019.\u00a0https:\/\/healthmanagement.org\/c\/hospital\/issuearticle\/ai-based-prediction-in-clinical-settings-can-we-trust-it."},{"key":"10179_CR64","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2023.105860","volume":"120","author":"K Zhao","year":"2023","unstructured":"Zhao K, Jia Z, Jia F, Shao H. Multi-scale integrated deep self-attention network for predicting remaining useful life of aero-engine. Eng Appl Artif Intell. 2023;120: 105860.","journal-title":"Eng Appl Artif Intell"},{"key":"10179_CR65","doi-asserted-by":"crossref","unstructured":"Lecue F, Wu J. Explaining and predicting abnormal expenses at large scale using knowledge graph based reasoning. J Web Semant. 2017;44:89\u2013103.\u00a0https:\/\/www.sciencedirect.com\/science\/article\/pii\/S1570826817300252.","DOI":"10.1016\/j.websem.2017.05.003"},{"key":"10179_CR66","unstructured":"Akur8. 2021.\u00a0https:\/\/akur8-tech.com\/. Accessed 31 July 2023."},{"key":"10179_CR67","unstructured":"F. of Privacy Forum. Unfairness by algorithm: Distilling the harms of automated decision-making. 2017.\u00a0https:\/\/fpf.org\/wp-content\/uploads\/2017\/12\/FPF-AutomatedDecision-Making-Harms-and-Mitigation-Charts.pdf."},{"key":"10179_CR68","doi-asserted-by":"crossref","unstructured":"Angelov P, Soares E, Jiang R, Arnold N, Atkinson P. Explainable artificial intelligence: An analytical review. Wiley Interdiscip Rev: Data Min Knowl Discov. 2021;11.","DOI":"10.1002\/widm.1424"},{"key":"10179_CR69","unstructured":"Guidotti R, Monreale A, Turini F, Pedreschi D, Giannotti F. A survey of methods for explaining black box models. CoRR. 2018;abs\/1802.01933.\u00a0http:\/\/arxiv.org\/abs\/1802.01933."},{"key":"10179_CR70","doi-asserted-by":"crossref","unstructured":"Dosilovic FK, Bri M, Hlupic N. Explainable artificial intelligence: A survey. 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO). 2018. p. 210\u20135.","DOI":"10.23919\/MIPRO.2018.8400040"},{"key":"10179_CR71","doi-asserted-by":"crossref","unstructured":"Zhong H, Wang Y, Tu C, Zhang T, Liu Z, Sun M. Iteratively questioning and answering for interpretable legal judgment prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34. 01 ed. 2020. p. 1250\u20137.\u00a0https:\/\/ojs.aaai.org\/index.php\/AAAI\/article\/view\/5479.","DOI":"10.1609\/aaai.v34i01.5479"},{"key":"10179_CR72","unstructured":"European union general data protection regulation (gdpr). 2016.\u00a0https:\/\/gdpr.eu\/. Accessed 31 July 2023."},{"key":"10179_CR73","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2022.102470","volume":"79","author":"BH van der Velden","year":"2022","unstructured":"van der Velden BH, Kuijf HJ, Gilhuijs KG, Viergever MA. Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Med Image Anal. 2022;79: 102470.","journal-title":"Med Image Anal"},{"key":"10179_CR74","doi-asserted-by":"crossref","unstructured":"Rudin C, Chen C, Chen Z, Huang H, Semenova L, Zhong C. Interpretable machine learning: Fundamental principles and 10 grand challenges. CoRR. 2021;abs\/2103.11251.\u00a0https:\/\/arxiv.org\/abs\/2103.11251.","DOI":"10.1214\/21-SS133"},{"key":"10179_CR75","doi-asserted-by":"crossref","unstructured":"Abdul A, Vermeulen J, Wang D, Lim BY, Kankanhalli M. Trends and trajectories for explainable, accountable and intelligible systems: An hci research agenda. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 2018.","DOI":"10.1145\/3173574.3174156"},{"key":"10179_CR76","doi-asserted-by":"crossref","unstructured":"Machlev R, Heistrene L, Perl M, Levy K, Belikov J, Mannor S, Levron Y. Explainable artificial intelligence (XAI) techniques for energy and power systems: Review, challenges and opportunities. Energy AI. 2022;9.","DOI":"10.1016\/j.egyai.2022.100169"},{"key":"10179_CR77","doi-asserted-by":"publisher","first-page":"52138","DOI":"10.1109\/ACCESS.2018.2870052","volume":"6","author":"A Adadi","year":"2018","unstructured":"Adadi A, Berrada M. Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access. 2018;6:52138\u201360.","journal-title":"IEEE Access"},{"key":"10179_CR78","doi-asserted-by":"crossref","unstructured":"Gabbay F, Bar-lev S, Montano O, Hadad N. A lime-based explainable machine learning model for predicting the severity level of covid-19 diagnosed patients. Appl Sci. 2021;11:10417.","DOI":"10.3390\/app112110417"},{"key":"10179_CR79","doi-asserted-by":"crossref","first-page":"9613936","DOI":"10.1155\/2022\/7325064","volume":"2022","author":"AM Ahmed","year":"2022","unstructured":"Ahmed AM, Kun Y, Chunqing G, Yuehui G. An optimized lime scheme for medical low light level image enhancement. Comput Intell Neurosci. 2022;2022:9613936.","journal-title":"Comput Intell Neurosci"},{"key":"10179_CR80","doi-asserted-by":"publisher","first-page":"1437","DOI":"10.1109\/LSP.2022.3178656","volume":"29","author":"H Zhu","year":"2022","unstructured":"Zhu H, Xue M, Wang Y, Yuan G, Li X. Fast visual tracking with siamese oriented region proposal network. IEEE Signal Process Lett. 2022;29:1437.","journal-title":"IEEE Signal Process Lett"},{"key":"10179_CR81","unstructured":"Goodfellow IJ, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. 2014.\u00a0https:\/\/arxiv.org\/abs\/1412.6572."},{"key":"10179_CR82","doi-asserted-by":"crossref","unstructured":"Lyu C, Huang K, Liang HN. A unified gradient regularization family for adversarial examples. IEEE Int Conf Data Min. 2015;301\u20139.","DOI":"10.1109\/ICDM.2015.84"},{"key":"10179_CR83","unstructured":"Zhang S, Qian Z, Huang K, Wang Q, Zhang R, Yi X. Towards better robust generalization with shift consistency regularization. Intl Conf Mach Learn. 2021;12524\u201334."},{"issue":"9","key":"10179_CR84","doi-asserted-by":"publisher","first-page":"2805","DOI":"10.1109\/TNNLS.2018.2886017","volume":"30","author":"X Yuan","year":"2019","unstructured":"Yuan X, He P, Zhu Q, Li X. Adversarial examples: Attacks and defenses for deep learning. IEEE Trans Neural Netw Learn Syst. 2019;30(9):2805\u201324.","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"10179_CR85","doi-asserted-by":"crossref","unstructured":"Qian Z, Huang K, Wang QF, Zhang XY. A survey of robust adversarial training in pattern recognition: Fundamental, theory, and methodologies. Pattern Recogn. 2023;132.","DOI":"10.1016\/j.patcog.2022.108889"},{"key":"10179_CR86","unstructured":"Dave P. Ai is explaining itself to humans. And it\u2019s paying off. 2022.\u00a0https:\/\/www.reuters.com\/technology\/ai-is-explaining-itself-humans-its-paying-off-2022-04-06\/."},{"key":"10179_CR87","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1038\/s42256-019-0088-2","volume":"1","author":"A Jobin","year":"2019","unstructured":"Jobin A, Ienca M, Vayena E. The global landscape of ai ethics guidelines. Nat Mach Intell. 2019;1:389\u201399.","journal-title":"Nat Mach Intell"},{"key":"10179_CR88","doi-asserted-by":"crossref","unstructured":"Zhu J, Liapis A, Risi S, Bidarra R, Youngblood GM. Explainable AI for designers: A human-centered perspective on mixed-initiative co-creation. In: IEEE Conference on Computational Intelligence and Games (CIG). 2018. p. 1\u20138.","DOI":"10.1109\/CIG.2018.8490433"},{"key":"10179_CR89","doi-asserted-by":"crossref","unstructured":"Miller T. Explanation in artificial intelligence: Insights from the social sciences. Artif Intell. 2019;267:1\u201338.\u00a0https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0004370218305988.","DOI":"10.1016\/j.artint.2018.07.007"},{"key":"10179_CR90","doi-asserted-by":"crossref","unstructured":"Kaur S, Singla J, Nkenyereye L, Jha S, Prashar D, Joshi GP, El-Sappagh S, Islam MS, Islam SMR. Medical diagnostic systems using artificial intelligence (AI) algorithms: principles and perspectives. IEEE Access. 2020;8:228049\u201369.","DOI":"10.1109\/ACCESS.2020.3042273"},{"key":"10179_CR91","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1016\/j.inffus.2021.11.003","volume":"81","author":"YL Chou","year":"2022","unstructured":"Chou YL, Moreira C, Bruza P, Ouyang C, Jorge J. Counterfactuals and causability in explainable artificial intelligence: Theory, algorithms, and applications. Inf Fusion. 2022;81:59\u201383.","journal-title":"Inf Fusion"},{"key":"10179_CR92","doi-asserted-by":"publisher","unstructured":"Bunt A, Lount M, Lauzon C. Are explanations always important? A study of deployed, low-cost intelligent interactive systems. In: Proceedings of the 2012 ACM International Conference on Intelligent User Interfaces, ser. IUI \u201912. New York, NY, USA: Association for Computing Machinery; 2012. p. 169\u201378.\u00a0https:\/\/doi.org\/10.1145\/2166966.2166996.","DOI":"10.1145\/2166966.2166996"},{"key":"10179_CR93","doi-asserted-by":"crossref","unstructured":"Palacio S, Lucieri A, Munir M, Hees J, Ahmed S, Dengel A. XAI handbook: Towards a unified framework for explainable AI. CoRR. 2021;abs\/2105.06677.\u00a0https:\/\/arxiv.org\/abs\/2105.06677.","DOI":"10.1109\/ICCVW54120.2021.00420"},{"issue":"5","key":"10179_CR94","doi-asserted-by":"publisher","first-page":"2228","DOI":"10.1109\/TNET.2021.3084251","volume":"29","author":"H Jiang","year":"2021","unstructured":"Jiang H, Wang M, Zhao P, Xiao Z, Dustdar S. A utility-aware general framework with quantifiable privacy preservation for destination prediction in lbss. IEEE\/ACM Trans Netw. 2021;29(5):2228\u201341.","journal-title":"IEEE\/ACM Trans Netw"},{"key":"10179_CR95","doi-asserted-by":"crossref","unstructured":"Han S, Ding H, Zhao S, Ren S, Wang Z, Lin J, Zhou S. Practical and robust federated learning with highly scalable regression training. IEEE Trans Neural Netw Learn Syst. 2023.","DOI":"10.1109\/TNNLS.2023.3271859"},{"key":"10179_CR96","doi-asserted-by":"crossref","unstructured":"Craven MW, Shavlik JW. Using sampling and queries to extract rules from trained neural networks. In: Cohen WW, Hirsch H, editors. Machine Learning Proceedings 1994. San Francisco (CA): Morgan Kaufmann; 1994. p. 37\u201345.\u00a0https:\/\/www.sciencedirect.com\/science\/article\/pii\/B9781558603356500131.","DOI":"10.1016\/B978-1-55860-335-6.50013-1"},{"key":"10179_CR97","unstructured":"Ras G, van Gerven M, Haselager P. Explanation methods in deep learning: Users, values, concerns and challenges. CoRR. 2018;abs\/1803.07517.\u00a0http:\/\/arxiv.org\/abs\/1803.07517."},{"key":"10179_CR98","unstructured":"Johansson U, K\u00f6nig R, Niklasson L. Rule extraction from trained neural networks using genetic programming. In: 13th International Conference on Artificial Neural Networks. 2003. p. 13\u20136."},{"key":"10179_CR99","unstructured":"Johansson U, K\u00f6nig R, Niklasson L. The truth is in there - rule extraction from opaque models using genetic programming. 2004."},{"issue":"1","key":"10179_CR100","first-page":"3","volume":"16","author":"ZH Zhou","year":"2003","unstructured":"Zhou ZH, Jiang Y, Chen SF. Extracting symbolic rules from trained neural network ensembles. AI Commun. 2003;16(1):3\u201315.","journal-title":"AI Commun"},{"key":"10179_CR101","doi-asserted-by":"publisher","unstructured":"Biswas SK, Chakraborty M, Purkayastha B, Roy P, Thounaojam DM. Rule extraction from training data using neural network. Int J Artif Intell Tools. 2017;26(3):1750006. https:\/\/doi.org\/10.1142\/S0218213017500063.","DOI":"10.1142\/S0218213017500063"},{"key":"10179_CR102","unstructured":"Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. 2015. https:\/\/arxiv.org\/abs\/1503.02531."},{"key":"10179_CR103","doi-asserted-by":"publisher","unstructured":"Tan S, Caruana R, Hooker G, Lou Y. Distill-and-compare. In: Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society. 2018. https:\/\/doi.org\/10.1145\/3278721.3278725.","DOI":"10.1145\/3278721.3278725"},{"key":"10179_CR104","unstructured":"Che Z, Purushotham S, Khemani R, Liu Y. Distilling knowledge from deep networks with applications to healthcare domain. 2015. https:\/\/arxiv.org\/abs\/1512.03542."},{"key":"10179_CR105","unstructured":"Xu K, Park DH, Yi C, Sutton C. Interpreting deep classifier by visual distillation of dark knowledge. 2018. https:\/\/arxiv.org\/abs\/1803.04042."},{"key":"10179_CR106","doi-asserted-by":"publisher","unstructured":"Friedman JH, Popescu BE. Predictive learning via rule ensembles. Ann Appl Stat. 2008;2(3). https:\/\/doi.org\/10.1214\/07-AOAS148.","DOI":"10.1214\/07-AOAS148"},{"key":"10179_CR107","unstructured":"Molnar C. Interpretable Machine Learning. 2nd ed. 2022. https:\/\/christophm.github.io\/interpretable-ml-book."},{"key":"10179_CR108","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L. Random forests. Mach Learn. 2001;45:5\u201332.","journal-title":"Mach Learn"},{"key":"10179_CR109","unstructured":"Fisher A, Rudin C, Dominici F. All models are wrong, but many are useful: Learning a variable\u2019s importance by studying an entire class of prediction models simultaneously. 2018. https:\/\/arxiv.org\/abs\/1801.01489."},{"key":"10179_CR110","doi-asserted-by":"crossref","unstructured":"Adhikari A, Tax DMJ, Satta R, Faeth M. Leafage: Example-based and feature importance-based explanations for black-box ml models. In: 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE). 2019. p. 1\u20137.","DOI":"10.1109\/FUZZ-IEEE.2019.8858846"},{"key":"10179_CR111","doi-asserted-by":"publisher","first-page":"02","DOI":"10.1007\/s42452-021-04148-9","volume":"3","author":"M Saarela","year":"2021","unstructured":"Saarela M, Jauhiainen S. Comparison of feature importance measures as explanations for classification models. SN Appl Sci. 2021;3:02.","journal-title":"SN Appl Sci"},{"key":"10179_CR112","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C. Anchors: High-precision model-agnostic explanations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32. 1st ed. 2018. https:\/\/ojs.aaai.org\/index.php\/AAAI\/article\/view\/11491.","DOI":"10.1609\/aaai.v32i1.11491"},{"key":"10179_CR113","unstructured":"Ying R, Bourgeois D, You J, Zitnik M, Leskovec J. Gnnexplainer: Generating explanations for graph neural networks. 2019. https:\/\/arxiv.org\/abs\/1903.03894."},{"key":"10179_CR114","doi-asserted-by":"crossref","unstructured":"Sato R, Yamada M, Kashima H. Random features strengthen graph neural networks. 2020.","DOI":"10.1137\/1.9781611976700.38"},{"key":"10179_CR115","doi-asserted-by":"crossref","unstructured":"Kadir M, Mosavi A, Sonntag D. Assessing xai: Unveiling evaluation metrics for local explanation, taxonomies, key concepts, and practical applications. 2023.","DOI":"10.31224\/2989"},{"key":"10179_CR116","unstructured":"Lundberg SM, Lee S. A unified approach to interpreting model predictions. CoRR. 2017;abs\/1705.07874. http:\/\/arxiv.org\/abs\/1705.07874."},{"key":"10179_CR117","doi-asserted-by":"publisher","unstructured":"\u0160trumbelj E, Kononenko I. Explaining prediction models and individual predictions with feature contributions. In: Knowledge and information systems. 2014;41(3):647\u2013665.\u00a0https:\/\/doi.org\/10.1007\/s10115-013-0679-x.","DOI":"10.1007\/s10115-013-0679-x"},{"key":"10179_CR118","unstructured":"Shrikumar A, Greenside P, Shcherbina A, Kundaje A. Not just a black box: Learning important features through propagating activation differences. CoRR. 2016;abs\/1605.01713. http:\/\/arxiv.org\/abs\/1605.01713."},{"key":"10179_CR119","unstructured":"Islam SR, Eberle W, Ghafoor SK, Ahmed M. Explainable artificial intelligence approaches: A survey. 2021. https:\/\/arxiv.org\/abs\/2101.09429."},{"key":"10179_CR120","unstructured":"Nagpal A. L1 and l2 regularization methods, explained. 2022. https:\/\/builtin.com\/data-science\/l2-regularization."},{"key":"10179_CR121","doi-asserted-by":"crossref","unstructured":"Demir-Kavuk O, Kamada M, Akutsu T, Knapp EW. Prediction using step-wise l1, l2 regularization and feature selection for small data sets with large number of features. BMC Bioinform. 2011;12:412.","DOI":"10.1186\/1471-2105-12-412"},{"key":"10179_CR122","doi-asserted-by":"crossref","unstructured":"Huynh-Cam TT, Chen LS, Le H. Using decision trees and random forest algorithms to predict and determine factors contributing to first-year university students\u2019 learning performance. Algorithms. 2021;14(11). https:\/\/www.mdpi.com\/1999-4893\/14\/11\/318.","DOI":"10.3390\/a14110318"},{"key":"10179_CR123","unstructured":"Sanjeevi M. Chapter 4: Decision trees algorithms. 2017. https:\/\/medium.com\/deep-math-machine-learning-ai\/chapter-4-decision-trees-algorithms-b93975f7a1f1."},{"key":"10179_CR124","doi-asserted-by":"publisher","first-page":"04020064","DOI":"10.1061\/(ASCE)CO.1943-7862.0001854","volume":"146","author":"A Fayek","year":"2020","unstructured":"Fayek A. Fuzzy logic and fuzzy hybrid techniques for construction engineering and management. J Constr Eng Manag. 2020;146:04020064.","journal-title":"J Constr Eng Manag"},{"key":"10179_CR125","doi-asserted-by":"crossref","unstructured":"Guo G, Wang H, Bell D, Bi Y. Knn model-based approach in classification. 2004.","DOI":"10.1007\/978-3-540-39964-3_62"},{"key":"10179_CR126","doi-asserted-by":"publisher","unstructured":"Letham B, Rudin C, McCormick TH, Madigan D. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. Ann Appl Stat. 2015;9(3). https:\/\/doi.org\/10.1214\/15-AOAS848.","DOI":"10.1214\/15-AOAS848"},{"key":"10179_CR127","doi-asserted-by":"crossref","unstructured":"Cheng L, Yin F, Theodoridis S, Chatzis S, Chang T. Rethinking bayesian learning for data analysis: The art of prior and inference in sparsity-aware modeling. IEEE Signal Process Mag. 2022;39(6).","DOI":"10.1109\/MSP.2022.3198201"},{"issue":"3","key":"10179_CR128","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1214\/ss\/1009213726","volume":"16","author":"L Breiman","year":"2001","unstructured":"Breiman L. Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author). Stat Sci. 2001;16(3):199\u2013231. https:\/\/doi.org\/10.1214\/ss\/1009213726.","journal-title":"Stat Sci"},{"key":"10179_CR129","unstructured":"Sarkar S, Weyde T, d\u2019Avila Garcez AS, Slabaugh GG, Dragicevic S, Percy C. Accuracy and interpretability trade-offs in machine learning applied to safer gambling. In: CoCo@NIPS. 2016."},{"issue":"5","key":"10179_CR130","doi-asserted-by":"publisher","first-page":"1189","DOI":"10.1214\/aos\/1013203451","volume":"29","author":"JH Friedman","year":"2001","unstructured":"Friedman JH. Greedy function approximation: A gradient boosting machine. Ann Stat. 2001;29(5):1189\u2013232. https:\/\/doi.org\/10.1214\/aos\/1013203451.","journal-title":"Ann Stat"},{"key":"10179_CR131","volume-title":"Modeling heterogeneous treatment effects in large-scale experiments using Bayesian additive regression trees","author":"D Green","year":"2010","unstructured":"Green D, Kern H. Modeling heterogeneous treatment effects in large-scale experiments using Bayesian additive regression trees. Iowa City: The Annual Summer Meeting of the Society of Political Methodology; 2010."},{"key":"10179_CR132","doi-asserted-by":"crossref","unstructured":"Elith J, Leathwick JR, Hastie T. A working guide to boosted regression trees. J Anim Ecol. 2008;77(4):802\u201313. https:\/\/besjournals.onlinelibrary.wiley.com.","DOI":"10.1111\/j.1365-2656.2008.01390.x"},{"key":"10179_CR133","doi-asserted-by":"crossref","unstructured":"Goldstein A, Kapelner A, Bleich J, Pitkin E. Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. J Comput Graph Stat. 2013;24.","DOI":"10.1080\/10618600.2014.907095"},{"key":"10179_CR134","doi-asserted-by":"publisher","unstructured":"Casalicchio G, Molnar C, Bischl B. Visualizing the feature importance for black box models. In: Machine Learning and Knowledge Discovery in Databases. 2019. p. 655\u201370. https:\/\/doi.org\/10.1007\/978-3-030-10925-7_40.","DOI":"10.1007\/978-3-030-10925-7_40"},{"key":"10179_CR135","doi-asserted-by":"crossref","unstructured":"Han H, Li W, Wang J, Qin G, Qin X. Enhance explainability of manifold learning. Neurocomputing. 2022;500:877\u201395. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0925231222007044.","DOI":"10.1016\/j.neucom.2022.05.119"},{"key":"10179_CR136","doi-asserted-by":"crossref","unstructured":"Liu S, Wang X, Liu M, Zhu J. Towards better analysis of machine learning models: A visual analytics perspective. Vis Inform. 2017;1(1):48\u201356. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S2468502X17300086.","DOI":"10.1016\/j.visinf.2017.01.006"},{"key":"10179_CR137","doi-asserted-by":"crossref","unstructured":"Apley DW, Zhu J. Visualizing the effects of predictor variables in black box supervised learning models. 2019.","DOI":"10.1111\/rssb.12377"},{"key":"10179_CR138","doi-asserted-by":"crossref","unstructured":"Moustafa N, Koroniotis N, Keshk M, Zomaya AY, Tari Z. Explainable intrusion detection for cyber defences in the internet of things: Opportunities and solutions. IEEE Commun Surv Tutor. 2023;1\u20131.","DOI":"10.1109\/COMST.2023.3280465"},{"key":"10179_CR139","doi-asserted-by":"crossref","unstructured":"Clement T, Kemmerzell N, Abdelaal M, Amberg M. Xair: A systematic metareview of explainable ai (xai) aligned to the software development process. Mach Learn Knowl Extr. 2023;5(1):78\u2013108. https:\/\/www.mdpi.com\/2504-4990\/5\/1\/6.","DOI":"10.3390\/make5010006"},{"key":"10179_CR140","unstructured":"Gurumoorthy KS, Dhurandhar A, Cecchi G, Aggarwal C. Efficient data representation by selecting prototypes with importance weights. 2017. https:\/\/arxiv.org\/abs\/1707.01212."},{"key":"10179_CR141","unstructured":"Kim B, Rudin C, Shah J. The Bayesian case model: A generative approach for case-based reasoning and prototype classification. 2015. https:\/\/arxiv.org\/abs\/1503.01161."},{"key":"10179_CR142","doi-asserted-by":"publisher","unstructured":"Bien J, Tibshirani R. Prototype selection for interpretable classification. Ann Appl Stat. 2011;5(4). https:\/\/doi.org\/10.1214\/11-AOAS495.","DOI":"10.1214\/11-AOAS495"},{"key":"10179_CR143","unstructured":"Olsson C. How to make your data and models interpretable by learning from cognitive science. 2017. https:\/\/medium.com\/south-park-commons\/how-to-make-your-data-and-models-interpretable-by-learning-from-cognitive-science-a6a29867790."},{"key":"10179_CR144","unstructured":"Kim B, Khanna R, Koyejo OO. Examples are not enough, learn to criticize! criticism for interpretability. In: Lee D, Sugiyama M, Luxburg U, Guyon I, Garnett R, editors. Advances in Neural Information Processing Systems. 29th ed. Curran Associates, Inc.; 2016. https:\/\/proceedings.neurips.cc\/paper\/2016\/file\/5680522b8e2bb01943234bce7bf84534-Paper.pdf."},{"key":"10179_CR145","doi-asserted-by":"crossref","unstructured":"Wachter S, Mittelstadt B, Russell C. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harv J Law Technol. 2018;31:841\u201387.","DOI":"10.2139\/ssrn.3063289"},{"key":"10179_CR146","doi-asserted-by":"crossref","unstructured":"Mehedi Hasan MGM, Talbert D. Mitigating the rashomon effect in counterfactual explanation: A game-theoretic approach. In: The International FLAIRS Conference Proceedings, vol. 35. 2022.","DOI":"10.32473\/flairs.v35i.130711"},{"key":"10179_CR147","unstructured":"Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R. Intriguing properties of neural networks. 2013. https:\/\/arxiv.org\/abs\/1312.6199."},{"key":"10179_CR148","unstructured":"Su J, Vargas D, Sakurai K. One pixel attack for fooling deep neural networks. IEEE Trans Evolut Comput. 2017."},{"key":"10179_CR149","unstructured":"Athalye A, Engstrom L, Ilyas A, Kwok K. Synthesizing robust adversarial examples. 2017. https:\/\/arxiv.org\/abs\/1707.07397."},{"key":"10179_CR150","unstructured":"Leino K. Ai explainabilityrequires robustness. 2021. https:\/\/towardsdatascience.com\/ai-explainability-requires-robustness."},{"key":"10179_CR151","unstructured":"Ilyas A, Santurkar S, Tsipras D, Engstrom L, Tran B, Madry A. Adversarial examples are not bugs, they are features. 2019. https:\/\/arxiv.org\/abs\/1905.02175."},{"key":"10179_CR152","unstructured":"Sadiku M, Shadare A, Musa S, Akujuobi C, Perry R. Data visualization. Int J Eng Res Adv Technol (IJERAT). 2016;12:2454\u20136135."},{"key":"10179_CR153","doi-asserted-by":"crossref","unstructured":"Lu Z, Cheng R, Jin Y, Tan KC, Deb K. Neural architecture search as multiobjective optimization benchmarks: Problem formulation and performance assessment. IEEE Trans Evol Comput. 2022.","DOI":"10.1109\/TEVC.2022.3233364"},{"issue":"11","key":"10179_CR154","doi-asserted-by":"publisher","first-page":"8037","DOI":"10.1109\/TCSVT.2022.3182426","volume":"32","author":"S Yang","year":"2022","unstructured":"Yang S, Li Q, Li W, Li X, Liu A. Dual-level representation enhancement on characteristic and context for image-text retrieval. IEEE Trans Circuits Syst Video Technol. 2022;32(11):8037\u201350.","journal-title":"IEEE Trans Circuits Syst Video Technol"},{"key":"10179_CR155","unstructured":"Wu A, Wang Y, Shu X, Moritz D, Cui W, Zhang H, Zhang D, Qu H. Survey on artificial intelligence approaches for visualization data. 2021."},{"key":"10179_CR156","doi-asserted-by":"publisher","first-page":"277","DOI":"10.14257\/ijunesst.2015.8.7.28","volume":"8","author":"A Khanna","year":"2015","unstructured":"Khanna A, Pandey B, Vashishta K, Kalia K, Bhale P, Das T. A study of today\u2019s AI through chatbots and rediscovery of machine intelligence. Int J of u- and e-Serv, Sci Technol. 2015;8:277\u201384.","journal-title":"Int J of u- and e-Serv, Sci Technol"},{"key":"10179_CR157","unstructured":"Yelekeri Jagadeesha RG. Artificial intelligence for data analysis and management. 2020."},{"key":"10179_CR158","doi-asserted-by":"crossref","unstructured":"Zhang J, Peng S, Gao Y, Zhang Z, Hong Q. Apmsa: Adversarial perturbation against model stealing attacks. IEEE Trans Inf Forensics Secur. 2023.","DOI":"10.1109\/TIFS.2023.3246766"},{"key":"10179_CR159","doi-asserted-by":"publisher","unstructured":"Acharjya DP, Ahmed KA. A survey on big data analytics: Challenges, open research issues and tools. Int J Adv Comput Sci Appl. 2016;7(2). https:\/\/doi.org\/10.14569\/IJACSA.2016.070267.","DOI":"10.14569\/IJACSA.2016.070267"},{"key":"10179_CR160","doi-asserted-by":"publisher","unstructured":"Jesus S, Bel\u00e9m C, Balayan V, Bento JA, Saleiro P, Bizarro P, Gama JA. How can i choose an explainer? An application-grounded evaluation of post-hoc explanations. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, ser. FAccT \u201921. New York, NY, USA: Association for Computing Machinery; 2021. p. 805\u201315. https:\/\/doi.org\/10.1145\/3442188.3445941.","DOI":"10.1145\/3442188.3445941"},{"key":"10179_CR161","unstructured":"Chang J, Boyd-Graber J, Gerrish S, Wang C, Blei D. Reading tea leaves: how humans interpret topic models. vol 32. 2009. p. 288\u201396."},{"key":"10179_CR162","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2594473.2594475","volume":"15","author":"A Freitas","year":"2014","unstructured":"Freitas A. Comprehensible classification models: A position paper. ACM SIGKDD Explorations Newsl. 2014;15:1\u201310.","journal-title":"ACM SIGKDD Explorations Newsl"},{"key":"10179_CR163","unstructured":"Holzinger A, Carrington AM, M\u00fcller H. Measuring the quality of explanations: The system causability scale (SCS). Comparing human and machine explanations. CoRR. 2019;abs\/1912.09024. http:\/\/arxiv.org\/abs\/1912.09024."},{"key":"10179_CR164","doi-asserted-by":"crossref","unstructured":"Grundy SM, Pasternak R, Greenland P, Smith S, Fuster V. Assessment of cardiovascular risk by use of multiple-risk-factor assessment equations. Circulation. 1999;100(13):1481\u201392. https:\/\/www.ahajournals.org.","DOI":"10.1161\/01.CIR.100.13.1481"},{"key":"10179_CR165","unstructured":"Ismail AA, Bravo HC, Feizi S. Improving deep learning interpretability by saliency guided training. CoRR. 2021;abs\/2111.14338. https:\/\/arxiv.org\/abs\/2111.14338."},{"key":"10179_CR166","doi-asserted-by":"crossref","unstructured":"Amann J, Blasimme A, Vayena E, Frey D, Madai VI, Precise4Q consortium. Explainability for artificial intelligence in healthcare: A multidisciplinary perspective. BMC Med Inform Decis Mak. 2020;20(1):310.","DOI":"10.1186\/s12911-020-01332-6"},{"issue":"3","key":"10179_CR167","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1145\/3236386.3241340","volume":"16","author":"ZC Lipton","year":"2018","unstructured":"Lipton ZC. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue. 2018;16(3):31\u201357. https:\/\/doi.org\/10.1145\/3236386.3241340.","journal-title":"Queue"},{"key":"10179_CR168","doi-asserted-by":"crossref","unstructured":"Xiong W, Fan H, Ma L, Wang CM. Challenges of human\u2014machine collaboration in risky decision-making. Front Eng Manag. 2022;9.","DOI":"10.1007\/s42524-021-0182-0"},{"key":"10179_CR169","unstructured":"Damacharla P, Javaid AY, Gallimore JJ, Devabhaktuni VK. Common metrics to benchmark human-machine teams (HMT): A review. CoRR. 2020;abs\/2008.04855. https:\/\/arxiv.org\/abs\/2008.04855."},{"key":"10179_CR170","doi-asserted-by":"crossref","unstructured":"Perelman BS, Mueller ST, Schaefer KE. Evaluating path planning in human-robot teams: Quantifying path agreement and mental model congruency. In: IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA). 2017. p. 1\u20137","DOI":"10.1109\/COGSIMA.2017.7929595"},{"key":"10179_CR171","doi-asserted-by":"crossref","unstructured":"Martin L, Gonz\u00e1lez-Romo M, Sahnoun M, Bettayeb B, He N, Gao J. Effect of human-robot interaction on the fleet size of AIV transporters in FMS. In: 2021 1st International Conference On Cyber Management And Engineering (CyMaEn). 2021. p. 1\u20135.","DOI":"10.1109\/CyMaEn50288.2021.9497273"},{"key":"10179_CR172","first-page":"138","volume":"4","author":"A Ballav","year":"2017","unstructured":"Ballav A, Ghosh M. Human factors of human machine interaction: Analyzing future trends through the past and the present. Int J Res. 2017;4:138\u201344.","journal-title":"Int J Res"},{"key":"10179_CR173","unstructured":"Han K, Cook K, Shih P. Exploring effective decision making through human-centered and computational intelligence methods. 2016."},{"key":"10179_CR174","doi-asserted-by":"crossref","unstructured":"Lyons JB, Havig PR. Transparency in a human-machine context: Approaches for fostering shared awareness\/intent. In: Shumaker R, Lackey S, editors. Virtual, Augmented and Mixed Reality. Designing and Developing Virtual and Augmented Environments. Cham: Springer International Publishing; 2014. p. 181\u201390.","DOI":"10.1007\/978-3-319-07458-0_18"},{"key":"10179_CR175","doi-asserted-by":"crossref","unstructured":"Raheem F, Iqbal N. Artificial Intelligence and Machine Learning for the Industrial Internet of Things (IIoT). 2022. p. 1\u201320.","DOI":"10.1201\/9781003145004-1"},{"key":"10179_CR176","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1007\/978-3-030-50334-5_5","volume-title":"Artificial Intelligence in HCI","author":"M Qian","year":"2020","unstructured":"Qian M, Qian D. Defining a human-machine teaming model for ai-powered human-centered machine translation agent by learning from human-human group discussion: dialog categories and dialog moves. In: Degen H, Reinerman-Jones L, editors. Artificial Intelligence in HCI. Cham: Springer International Publishing; 2020. p. 70\u201381."},{"issue":"1","key":"10179_CR177","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1007\/s42524-021-0182-0","volume":"9","author":"W Xiong","year":"2022","unstructured":"Xiong W, Fan H, Ma L, Wang C. Challenges of human\u2014machine collaboration in risky decision-making. Front Eng Manag. 2022;9(1):89\u2013103.","journal-title":"Front Eng Manag"},{"issue":"4","key":"10179_CR178","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1016\/j.bushor.2018.03.007","volume":"61","author":"MH Jarrahi","year":"2018","unstructured":"Jarrahi MH. Artificial intelligence and the future of work: human-AI symbiosis in organizational decision making. Bus Horiz. 2018;61(4):577\u201386.","journal-title":"Bus Horiz"},{"issue":"4","key":"10179_CR179","doi-asserted-by":"publisher","first-page":"541","DOI":"10.1080\/08838151.2020.1843357","volume":"64","author":"D Shin","year":"2020","unstructured":"Shin D. User perceptions of algorithmic decisions in the personalized ai system: perceptual evaluation of fairness, accountability, transparency, and explainability. J Broadcast Electron Media. 2020;64(4):541\u201365.","journal-title":"J Broadcast Electron Media"},{"key":"10179_CR180","unstructured":"IBM. Building trust in ai. 2018. https:\/\/www.ibm.com\/watson\/advantage-reports\/future-of-artificial-intelligence\/building-trust-in-ai.html."},{"key":"10179_CR181","doi-asserted-by":"crossref","unstructured":"Ma Q, Liu L. The Technology Acceptance Model. 2005.","DOI":"10.4018\/978-1-59140-474-3.ch006"},{"key":"10179_CR182","doi-asserted-by":"crossref","unstructured":"Kulesza T, Stumpf S, Burnett M, Yang S, Kwan I, Wong WK. Too much, too little, or just right? ways explanations impact end users\u2019 mental models. 2013.","DOI":"10.1109\/VLHCC.2013.6645235"},{"key":"10179_CR183","unstructured":"Akyol E, Langbort C, Basar T. Price of transparency in strategic machine learning. 2016. https:\/\/arxiv.org\/abs\/1610.08210."},{"key":"10179_CR184","doi-asserted-by":"crossref","unstructured":"Igami M. Artificial intelligence as structural estimation: deep blue, bonanza, and alphago. J Econom. 2020;23.","DOI":"10.1093\/ectj\/utaa005"},{"key":"10179_CR185","unstructured":"Dignum V. Responsible artificial intelligence: Designing ai for human values. 2017."},{"key":"10179_CR186","doi-asserted-by":"crossref","unstructured":"Antoniadi AM, Du Y, Guendouz Y, Wei L, Mazo C, Becker BA, Mooney C. Current challenges and future opportunities for XAI in machine learning-based clinical decision support systems: A systematic review. Appl Sci. 2021;11(11).","DOI":"10.3390\/app11115088"},{"key":"10179_CR187","doi-asserted-by":"crossref","unstructured":"Nie W, Bao Y, Zhao Y, Liu A. Long dialogue emotion detection based on commonsense knowledge graph guidance. IEEE Trans Multimed. 2023.","DOI":"10.1109\/TMM.2023.3267295"},{"issue":"10","key":"10179_CR188","doi-asserted-by":"publisher","first-page":"2320","DOI":"10.3390\/electronics12102320","volume":"12","author":"X Liu","year":"2023","unstructured":"Liu X, He J, Liu M, Yin Z, Yin L, Zheng W. A scenario-generic neural machine translation data augmentation method. Electronics. 2023;12(10):2320.","journal-title":"Electronics"},{"issue":"1","key":"10179_CR189","doi-asserted-by":"publisher","first-page":"306","DOI":"10.1057\/s41599-023-01816-6","volume":"10","author":"X Liu","year":"2023","unstructured":"Liu X, Shi T, Zhou G, Liu M, Yin Z, Yin L, Zheng W. Emotion classification for short texts: An improved multi-label method. Humanit Soc Sci Commun. 2023;10(1):306.","journal-title":"Humanit Soc Sci Commun"}],"container-title":["Cognitive Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12559-023-10179-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12559-023-10179-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12559-023-10179-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T07:04:33Z","timestamp":1705475073000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12559-023-10179-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,24]]},"references-count":189,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1]]}},"alternative-id":["10179"],"URL":"https:\/\/doi.org\/10.1007\/s12559-023-10179-8","relation":{},"ISSN":["1866-9956","1866-9964"],"issn-type":[{"value":"1866-9956","type":"print"},{"value":"1866-9964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,24]]},"assertion":[{"value":"16 March 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 July 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 August 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This article does not contain any studies with human participants or animals performed by any of the authors.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}]}}