{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T18:43:47Z","timestamp":1780512227655,"version":"3.54.1"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"17","license":[{"start":{"date-parts":[[2024,3,10]],"date-time":"2024-03-10T00:00:00Z","timestamp":1710028800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,10]],"date-time":"2024-03-10T00:00:00Z","timestamp":1710028800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004175","name":"Middle East Technical University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004175","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Deep learning models are extensively used but often lack transparency due to their complex internal mechanics. To bridge this gap, the field of explainable AI (XAI) strives to make these models more interpretable. However, a significant obstacle in XAI is the absence of quantifiable metrics for evaluating explanation quality. Existing techniques, reliant on manual assessment or inadequate metrics, face limitations in scalability, reproducibility, and trustworthiness. Recognizing these issues, the current study specifically addresses the quality assessment of visual explanations in medical imaging, where interpretability profoundly influences diagnostic accuracy and trust in AI-assisted decisions. Introducing novel criteria such as informativeness, localization, coverage, multi-target capturing, and proportionality, this work presents a comprehensive method for the objective assessment of various explainability algorithms. These newly introduced criteria aid in identifying optimal evaluation metrics. The study expands the domain\u2019s analytical toolkit by examining existing metrics, which have been prevalent in recent works for similar applications, and proposing new ones. Rigorous analysis led to selecting Jensen\u2013Shannon divergence (JS_DIV) as the most effective metric for visual explanation quality. Applied to the multi-label, multi-class diagnosis of thoracic diseases using a trained classifier on the CheXpert dataset, local interpretable model-agnostic explanations (LIME) with diverse segmentation strategies interpret the classifier\u2019s decisions. A qualitative analysis on an unseen subset of the VinDr-CXR dataset evaluates these metrics, confirming JS_DIV\u2019s superiority. The subsequent quantitative analysis optimizes LIME\u2019s hyper-parameters and benchmarks its performance across various segmentation algorithms, underscoring the utility of an objective assessment metric in practical applications.<\/jats:p>","DOI":"10.1007\/s00521-024-09587-0","type":"journal-article","created":{"date-parts":[[2024,3,10]],"date-time":"2024-03-10T09:01:23Z","timestamp":1710061283000},"page":"10239-10255","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Evaluating the quality of visual explanations on chest X-ray images for thorax diseases classification"],"prefix":"10.1007","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6081-1727","authenticated-orcid":false,"given":"Shakiba","family":"Rahimiaghdam","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hande","family":"Alemdar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,3,10]]},"reference":[{"key":"9587_CR1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cosrev.2021.100379","volume":"40","author":"S Dong","year":"2021","unstructured":"Dong S, Wang P, Abbas K (2021) A survey on deep learning and its applications. Comput Sci Rev 40:100379. https:\/\/doi.org\/10.1016\/j.cosrev.2021.100379","journal-title":"Comput Sci Rev"},{"key":"9587_CR2","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1002\/9781119861850.ch7","volume-title":"Machine learning algorithms for signal and image processing","author":"DL Prasanna","year":"2023","unstructured":"Prasanna DL, Tripathi SL (2023) Machine and deep learning techniques for text and speech processing. In: Ghai D, Tripathi SL, Saxena S, Chanda M, Alazab M (eds) Machine learning algorithms for signal and image processing. Wiley, New York, pp 115\u2013128. https:\/\/doi.org\/10.1002\/9781119861850.ch7"},{"key":"9587_CR3","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2023.103861","volume":"317","author":"J Collenette","year":"2023","unstructured":"Collenette J, Atkinson K, Bench-Capon T (2023) Explainable AI tools for legal reasoning about cases: a study on the European Court of Human Rights. Artif Intell 317:103861. https:\/\/doi.org\/10.1016\/j.artint.2023.103861","journal-title":"Artif Intell"},{"issue":"3","key":"9587_CR4","doi-asserted-by":"publisher","first-page":"1318","DOI":"10.1002\/qre.2939","volume":"38","author":"P Giudici","year":"2022","unstructured":"Giudici P, Raffinetti E (2022) Explainable AI methods in cyber risk management. Qual Reliab Eng Int 38(3):1318\u20131326. https:\/\/doi.org\/10.1002\/qre.2939","journal-title":"Qual Reliab Eng Int"},{"key":"9587_CR5","doi-asserted-by":"publisher","DOI":"10.1002\/wsbm.1548","author":"D Jin","year":"2022","unstructured":"Jin D, Sergeeva E, Weng W-H, Chauhan G, Szolovits P (2022) Explainable deep learning in healthcare: a methodological survey from an attribution view. WIREs Mech Dis. https:\/\/doi.org\/10.1002\/wsbm.1548","journal-title":"WIREs Mech Dis"},{"issue":"4","key":"9587_CR6","doi-asserted-by":"publisher","first-page":"1607","DOI":"10.1007\/s13347-021-00477-0","volume":"34","author":"WJ Eschenbach","year":"2021","unstructured":"Eschenbach WJ (2021) Transparency and the black box problem: why we do not trust AI. Philos Technol 34(4):1607\u20131622. https:\/\/doi.org\/10.1007\/s13347-021-00477-0","journal-title":"Philos Technol"},{"issue":"4","key":"9587_CR7","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1016\/j.jormas.2019.06.002","volume":"120","author":"A Fourcade","year":"2019","unstructured":"Fourcade A, Khonsari RH (2019) Deep learning in medical image analysis: a third eye for doctors. J Stomatol Oral Maxillofac Surg 120(4):279\u2013288. https:\/\/doi.org\/10.1016\/j.jormas.2019.06.002","journal-title":"J Stomatol Oral Maxillofac Surg"},{"key":"9587_CR8","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2022.102470","volume":"79","author":"BHM Velden","year":"2022","unstructured":"Velden BHM, Kuijf HJ, Gilhuijs KGA, Viergever MA (2022) Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Med Image Anal 79:102470. https:\/\/doi.org\/10.1016\/j.media.2022.102470","journal-title":"Med Image Anal"},{"key":"9587_CR9","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","volume":"58","author":"A Barredo Arrieta","year":"2020","unstructured":"Barredo Arrieta A, D\u00edaz-Rodr\u00edguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, Garcia S, Gil-Lopez S, Molina D, Benjamins R, Chatila R, Herrera F (2020) Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf Fusion 58:82\u2013115. https:\/\/doi.org\/10.1016\/j.inffus.2019.12.012","journal-title":"Inf Fusion"},{"issue":"1","key":"9587_CR10","doi-asserted-by":"publisher","first-page":"15364","DOI":"10.1038\/s41598-020-71294-2","volume":"10","author":"AT Sahlol","year":"2020","unstructured":"Sahlol AT, Yousri D, Ewees AA, Al-Qaness MAA, Damasevicius R, Elaziz MA (2020) Covid-19 image classification using deep features and fractional-order marine predators algorithm. Sci Rep 10(1):15364. https:\/\/doi.org\/10.1038\/s41598-020-71294-2","journal-title":"Sci Rep"},{"key":"9587_CR11","doi-asserted-by":"publisher","first-page":"107052","DOI":"10.1016\/j.asoc.2020.107052","volume":"101","author":"D Yousri","year":"2021","unstructured":"Yousri D, Abd Elaziz M, Abualigah L, Oliva D, Al-qaness MAA, Ewees AA (2021) Covid-19 x-ray images classification based on enhanced fractional-order cuckoo search optimizer using heavy-tailed distributions. Appl Soft Comput 101:107052. https:\/\/doi.org\/10.1016\/j.asoc.2020.107052","journal-title":"Appl Soft Comput"},{"key":"9587_CR12","doi-asserted-by":"publisher","first-page":"125306","DOI":"10.1109\/ACCESS.2020.3007928","volume":"8","author":"MA Elaziz","year":"2020","unstructured":"Elaziz MA, Ewees AA, Yousri D, Alwerfali HSN, Awad QA, Lu S, Al-Qaness MAA (2020) An improved marine predators algorithm with fuzzy entropy for multi-level thresholding: real world example of covid-19 CT image segmentation. IEEE Access 8:125306\u2013125330. https:\/\/doi.org\/10.1109\/ACCESS.2020.3007928","journal-title":"IEEE Access"},{"key":"9587_CR13","doi-asserted-by":"publisher","unstructured":"Ribeiro MT, Singh S, Guestrin C (2016) Why should I trust you? Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135\u20131144. https:\/\/doi.org\/10.1145\/2939672.2939778","DOI":"10.1145\/2939672.2939778"},{"key":"9587_CR14","doi-asserted-by":"publisher","unstructured":"Vedaldi A, Soatto S (2008) Quick shift and kernel methods for mode seeking. In: European conference on computer vision, pp 705\u2013718. https:\/\/doi.org\/10.1007\/978-3-540-88693-8_52. Springer","DOI":"10.1007\/978-3-540-88693-8_52"},{"key":"9587_CR15","first-page":"1246","volume":"2019","author":"A Xiang","year":"2019","unstructured":"Xiang A, Wang F (2019) Towards interpretable skin lesion classification with deep learning models. AMIA Annu Symp Proc 2019:1246\u20131255","journal-title":"AMIA Annu Symp Proc"},{"key":"9587_CR16","doi-asserted-by":"publisher","DOI":"10.3390\/app8101715","author":"S Rajaraman","year":"2018","unstructured":"Rajaraman S, Candemir S, Kim I, Thoma G, Antani S (2018) Visualization and interpretation of convolutional neural network predictions in detecting pneumonia in pediatric chest radiographs. Appl Sci (Switz). https:\/\/doi.org\/10.3390\/app8101715","journal-title":"Appl Sci (Switz)"},{"key":"9587_CR17","doi-asserted-by":"publisher","unstructured":"Ahsan MM, Gupta KD, Islam MM, Sen S, Rahman ML, Hossain MS (2020) Study of different deep learning approach with explainable AI for screening patients with COVID-19 symptoms: using CT scan and chest X-ray image dataset https:\/\/doi.org\/10.3390\/make2040027","DOI":"10.3390\/make2040027"},{"key":"9587_CR18","doi-asserted-by":"publisher","DOI":"10.3390\/s21217116","author":"LO Teixeira","year":"2021","unstructured":"Teixeira LO, Pereira RM, Bertolini D, Oliveira LS, Nanni L, Cavalcanti GDC, Costa YMG (2021) Impact of lung segmentation on the diagnosis and explanation of COVID-19 in chest X-ray images. Sensors. https:\/\/doi.org\/10.3390\/s21217116","journal-title":"Sensors"},{"key":"9587_CR19","doi-asserted-by":"publisher","unstructured":"Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention\u2014MICCAI 2015. https:\/\/doi.org\/10.1007\/978-3-319-24574-4_28","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"9587_CR20","doi-asserted-by":"crossref","unstructured":"Sattarzadeh S, Sudhakar M, Lem A, Mehryar S, Plataniotis KN, Jang J, Kim H, Jeong Y, Lee S, Bae K (2021) Explaining convolutional neural networks through attribution-based input sampling and block-wise feature aggregation. In: 34th AAAI Conference on artificial intelligence","DOI":"10.1609\/aaai.v35i13.17384"},{"key":"9587_CR21","doi-asserted-by":"publisher","unstructured":"DoshiVelez F, Kim B (2018) Considerations for evaluation and generalization in interpretable machine learning. In: Explainable and interpretable models in computer vision and machine learning, pp 3\u201317. https:\/\/doi.org\/10.1007\/978-3-319-98131-4_1","DOI":"10.1007\/978-3-319-98131-4_1"},{"key":"9587_CR22","doi-asserted-by":"publisher","unstructured":"Li X-H, Shi Y, Li H, Bai W, Cao CC, Chen L (2021) An experimental study of quantitative evaluations on saliency methods. In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery and data mining, pp 3200\u20133208. https:\/\/doi.org\/10.1145\/3447548.3467148","DOI":"10.1145\/3447548.3467148"},{"key":"9587_CR23","unstructured":"Alvarez\u00a0Melis D, Jaakkola T (2018) Towards robust interpretability with self-explaining neural networks. In: Advances in neural information processing systems, pp 31"},{"key":"9587_CR24","doi-asserted-by":"publisher","unstructured":"Chattopadhay A, Sarkar A, Howlader P, Balasubramanian VN (2018) Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE Winter conference on applications of computer vision (WACV). IEEE, pp 839\u2013847. https:\/\/doi.org\/10.1109\/WACV.2018.00097","DOI":"10.1109\/WACV.2018.00097"},{"key":"9587_CR25","doi-asserted-by":"publisher","unstructured":"Ramaswamy HG, Desai S (2020) Ablation-CAM: visual explanations for deep convolutional network via gradient-free localization. In: 2020 IEEE winter conference on applications of computer vision (WACV), pp 972\u2013980. https:\/\/doi.org\/10.1109\/WACV45572.2020.9093360","DOI":"10.1109\/WACV45572.2020.9093360"},{"key":"9587_CR26","unstructured":"Petsiuk V, Das A, Saenko K (2018) RISE: randomized input sampling for explanation of black-box models"},{"key":"9587_CR27","doi-asserted-by":"publisher","unstructured":"Sokol K, Flach P (2020) Explainability fact sheets: a framework for systematic assessment of explainable approaches. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp 56\u201367. https:\/\/doi.org\/10.1145\/3351095.3372870","DOI":"10.1145\/3351095.3372870"},{"key":"9587_CR28","doi-asserted-by":"publisher","unstructured":"Hailemariam Y, Yazdinejad A, Parizi RM, Srivastava G, Dehghantanha A (2020) An empirical evaluation of AI deep explainable tools. In: 2020 IEEE Globecom workshops (GC Wkshps). IEEE, pp 1\u20136.https:\/\/doi.org\/10.1109\/GCWkshps50303.2020.9367541","DOI":"10.1109\/GCWkshps50303.2020.9367541"},{"key":"9587_CR29","unstructured":"Graziani M, Lompech T, M\u00fcller H, Andrearczyk V (2020) Evaluation and comparison of CNN visual explanations for histopathology. In: Explainable agency in artificial intelligence at AAAI21, pp 195\u2013201"},{"issue":"10","key":"9587_CR30","doi-asserted-by":"publisher","first-page":"1084","DOI":"10.1007\/s11263-017-1059-x","volume":"126","author":"J Zhang","year":"2018","unstructured":"Zhang J, Bargal SA, Lin Z, Brandt J, Shen X, Sclaroff S (2018) Top-down neural attention by excitation backprop. Int J Comput Vis 126(10):1084\u20131102. https:\/\/doi.org\/10.1007\/s11263-017-1059-x","journal-title":"Int J Comput Vis"},{"key":"9587_CR31","unstructured":"Schulz K, Sixt L, Tombari F, Landgraf T (2020) Restricting the flow: information bottlenecks for attribution. In: International conference on learning representations"},{"key":"9587_CR32","doi-asserted-by":"publisher","unstructured":"Wang H, Wang Z, Du M, Yang F, Zhang Z, Ding S, Mardziel P, Hu X (2020) Score-CAM: score-weighted visual explanations for convolutional neural networks. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition workshops, pp 111\u2013119. https:\/\/doi.org\/10.1109\/CVPRW50498.2020.00020","DOI":"10.1109\/CVPRW50498.2020.00020"},{"key":"9587_CR33","doi-asserted-by":"crossref","unstructured":"Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer, pp 818\u2013833","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"9587_CR34","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1007\/978-3-030-43823-4_13","volume-title":"Mach Learn Knowl Discov Databases","author":"L Schallner","year":"2020","unstructured":"Schallner L, Rabold J, Scholz O, Schmid U (2020) Effect of superpixel aggregation on explanations in LIME\u2014a case study with biological data. In: Cellier P, Driessens K (eds) Mach Learn Knowl Discov Databases. Springer, Cham, pp 147\u2013158"},{"key":"9587_CR35","doi-asserted-by":"publisher","first-page":"590","DOI":"10.1609\/aaai.v33i01.3301590","volume":"33","author":"J Irvin","year":"2019","unstructured":"Irvin J, Rajpurkar P, Ko M, Yu Y, Ciurea-Ilcus S, Chute C, Marklund H, Haghgoo B, Ball R, Shpanskaya K, Seekins J, Mong D, Halabi S, Sandberg J, Jones R, Larson D, Langlotz C, Patel B, Lungren M, Ng A (2019) CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. Proc AAAI Conf Artif Intell 33:590\u2013597. https:\/\/doi.org\/10.1609\/aaai.v33i01.3301590","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"9587_CR36","unstructured":"Nguyen HQ, Lam K, Le LT, Pham HH, Tran DQ, Nguyen DB, Le DD, Pham CM, Tong HTT, Dinh DH, Do CD, Doan LT, Nguyen CN, Nguyen BT, Nguyen QV, Hoang AD, Phan HN, Nguyen AT, Ho PH, Ngo DT, Nguyen NT, Nguyen NT, Dao M, Vu V (2020) VinDr-CXR: an open dataset of chest X-rays with radiologist\u2019s annotations, 1\u201310"},{"key":"9587_CR37","doi-asserted-by":"crossref","unstructured":"Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818\u20132826","DOI":"10.1109\/CVPR.2016.308"},{"key":"9587_CR38","doi-asserted-by":"publisher","unstructured":"Huang G, Liu Z, Van Der\u00a0Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700\u20134708. https:\/\/doi.org\/10.1109\/CVPR.2017.243","DOI":"10.1109\/CVPR.2017.243"},{"key":"9587_CR39","unstructured":"Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd International conference on learning representations, ICLR"},{"key":"9587_CR40","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778. https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"9587_CR41","doi-asserted-by":"publisher","unstructured":"Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L-C (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510\u20134520. https:\/\/doi.org\/10.1109\/CVPR.2018.00474","DOI":"10.1109\/CVPR.2018.00474"},{"key":"9587_CR42","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.patrec.2014.01.008","volume":"42","author":"M L\u00e4ngkvist","year":"2014","unstructured":"L\u00e4ngkvist M, Karlsson L, Loutfi A (2014) A review of unsupervised feature learning and deep learning for time-series modeling. Pattern Recognit Lett 42:11\u201324. https:\/\/doi.org\/10.1016\/j.patrec.2014.01.008","journal-title":"Pattern Recognit Lett"},{"issue":"2","key":"9587_CR43","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1023\/B:VISI.0000022288.19776.77","volume":"59","author":"PF Felzenszwalb","year":"2004","unstructured":"Felzenszwalb PF, Huttenlocher DP (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2):167\u2013181. https:\/\/doi.org\/10.1023\/B:VISI.0000022288.19776.77","journal-title":"Int J Comput Vis"},{"issue":"11","key":"9587_CR44","doi-asserted-by":"publisher","first-page":"2274","DOI":"10.1109\/TPAMI.2012.120","volume":"34","author":"R Achanta","year":"2012","unstructured":"Achanta R, Shaji A, Smith K, Lucchi A, Fua P, S\u00fcsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274\u20132282. https:\/\/doi.org\/10.1109\/TPAMI.2012.120","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"9587_CR45","doi-asserted-by":"publisher","unstructured":"Mohseni S, Block JE, Ragan E (2021) Quantitative evaluation of machine learning explanations: a human-grounded benchmark. In: 26th International conference on intelligent user interfaces, pp 22\u201331. https:\/\/doi.org\/10.1145\/3397481.3450689","DOI":"10.1145\/3397481.3450689"},{"key":"9587_CR46","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-28954-6","author":"W Samek","year":"2019","unstructured":"Samek W, Montavon G, Vedaldi A, Hansen LK, Muller K-R (2019) Explainable AI: interpreting, explaining and visualizing deep. Learning. https:\/\/doi.org\/10.1007\/978-3-030-28954-6","journal-title":"Learning"},{"issue":"3","key":"9587_CR47","doi-asserted-by":"publisher","first-page":"740","DOI":"10.1109\/TPAMI.2018.2815601","volume":"41","author":"Z Bylinskii","year":"2019","unstructured":"Bylinskii Z, Judd T, Oliva A, Torralba A, Durand F (2019) What do different evaluation metrics tell us about saliency models? IEEE Trans Pattern Anal Mach Intell 41(3):740\u2013757. https:\/\/doi.org\/10.1109\/TPAMI.2018.2815601","journal-title":"IEEE Trans Pattern Anal Mach Intell"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-09587-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-024-09587-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-09587-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,27]],"date-time":"2024-05-27T08:13:15Z","timestamp":1716797595000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-024-09587-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,10]]},"references-count":47,"journal-issue":{"issue":"17","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["9587"],"URL":"https:\/\/doi.org\/10.1007\/s00521-024-09587-0","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,10]]},"assertion":[{"value":"23 March 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All authors certify that they have no affiliations with or involvement in any organization or entity with any financial or non-financial interest in the subject matter or materials discussed in this manuscript. This research received no specific grant from funding agencies in the public, commercial, or not-for-profit sectors.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}