{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,6]],"date-time":"2026-11-06T04:24:22Z","timestamp":1793939062084,"version":"4.8.0"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,10,3]],"date-time":"2025-10-03T00:00:00Z","timestamp":1759449600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,10,3]],"date-time":"2025-10-03T00:00:00Z","timestamp":1759449600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005713","name":"Technische Universit\u00e4t M\u00fcnchen","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100005713","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Artificial Intelligence (AI) holds promise for advancing the field of neurotechnology and accelerating its clinical translation. AI-driven clinical neurotechnologies leverage the power of non-linear algorithms to analyze complex brain data and enable adaptive, closed-loop neurostimulation. Despite these promises, the integration of AI into clinical practice remains limited, with lack of explainability being commonly cited as one main obstacle. This raises the question of whether opacity and lack of explainability also hinder the adoption of AI in closed-loop medical neurotechnologies. We investigated the attitudes, informational needs and preferences of clinicians regarding AI-driven closed-loop neurotechnologies and explored what forms of explanation they consider necessary for clinical use. We conducted semi-structured expert interviews with twenty clinicians (including neurologists, neurosurgeons, and psychiatrists) from Germany and Switzerland. Using reflexive thematic analysis, we explored their understanding of and expectations for explainability in the context of AI-driven closed-loop neurotechnology systems. Clinicians consistently emphasized the importance of context-sensitive, clinically meaningful forms of explainability such as understanding what input data were used to train the system and how the output relates to clinically relevant outcomes. By contrast, detailed knowledge of the model\u2019s inner architecture or technical mechanics were of limited interest. Several participants specifically called for Explainable AI (XAI) techniques, particularly feature importance and relevance measures, to support their interpretation of system outputs. Our findings suggest that the clinical utility of AI-driven neurotechnologies can be improved by focusing on intuitive, user-centered and clinically meaningful forms of explainability rather than full algorithmic transparency. Designing systems that meet these pragmatic needs may help bridge the translational gap between AI development and clinical implementation.<\/jats:p>","DOI":"10.1038\/s41598-025-19510-9","type":"journal-article","created":{"date-parts":[[2025,10,3]],"date-time":"2025-10-03T13:45:15Z","timestamp":1759499115000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Clinician perspectives on explainability in AI-driven closed-loop neurotechnology"],"prefix":"10.1038","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-1168-4896","authenticated-orcid":false,"given":"Laura","family":"Schopp","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7428-2619","authenticated-orcid":false,"given":"Georg","family":"Starke","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8835-5444","authenticated-orcid":false,"given":"Marcello","family":"Ienca","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,10,3]]},"reference":[{"key":"19510_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-024-01253-y","volume":"7","author":"E Ferrea","year":"2024","unstructured":"Ferrea, E., Negahbani, F., Cebi, I., Weiss, D. & Gharabaghi, A. Machine learning explains response variability of deep brain stimulation on Parkinson\u2019s disease quality of life. npj Digit Med. 7, 1\u201311 (2024).","journal-title":"npj Digit Med."},{"key":"19510_CR2","doi-asserted-by":"publisher","first-page":"3043","DOI":"10.1038\/s41467-021-23311-9","volume":"12","author":"A Boutet","year":"2021","unstructured":"Boutet, A. et al. Predicting optimal deep brain stimulation parameters for Parkinson\u2019s disease using functional MRI and machine learning. Nat. Commun. 12, 3043 (2021).","journal-title":"Nat. Commun."},{"key":"19510_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-023-00779-x","volume":"6","author":"AS Chandrabhatla","year":"2023","unstructured":"Chandrabhatla, A. S., Pomeraniec, I. J., Horgan, T. M., Wat, E. K. & Ksendzovsky, A. Landscape and future directions of machine learning applications in closed-loop brain stimulation. npj Digit. Med. 6, 1\u201313 (2023).","journal-title":"npj Digit. Med."},{"key":"19510_CR4","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1038\/s41582-024-00965-9","volume":"20","author":"A Lucas","year":"2024","unstructured":"Lucas, A., Revell, A. & Davis, K. A. Artificial intelligence in epilepsy \u2014 applications and pathways to the clinic. Nat. Rev. Neurol. 20, 319\u2013336 (2024).","journal-title":"Nat. Rev. Neurol."},{"key":"19510_CR5","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa2027540","author":"DA Moses","year":"2021","unstructured":"Moses, D. A. et al. Neuroprosthesis for decoding speech in a paralyzed person with anarthria. N. Engl. J. Med. https:\/\/doi.org\/10.1056\/NEJMoa2027540 (2021).","journal-title":"N. Engl. J. Med."},{"key":"19510_CR6","doi-asserted-by":"publisher","first-page":"1021","DOI":"10.1093\/schbul\/sbx114","volume":"44","author":"N Koutsouleris","year":"2018","unstructured":"Koutsouleris, N. et al. Predicting response to repetitive transcranial magnetic stimulation in patients with schizophrenia using structural magnetic resonance imaging: A multisite machine learning analysis. Schizophr Bull. 44, 1021\u20131034 (2018).","journal-title":"Schizophr Bull."},{"key":"19510_CR7","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1038\/s41582-023-00836-9","volume":"19","author":"P Limousin","year":"2023","unstructured":"Limousin, P. & Akram, H. AI and deep brain stimulation: what have we learned?. Nat. Rev. Neurol. 19, 451\u2013452 (2023).","journal-title":"Nat. Rev. Neurol."},{"key":"19510_CR8","first-page":"e46454","volume":"15","author":"MI Ahmed","year":"2023","unstructured":"Ahmed, M. I. et al. A systematic review of the barriers to the implementation of artificial intelligence in healthcare. Cureus 15, e46454 (2023).","journal-title":"Cureus"},{"key":"19510_CR9","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1038\/s41746-023-00837-4","volume":"6","author":"N Bienefeld","year":"2023","unstructured":"Bienefeld, N. et al. Solving the explainable AI conundrum by bridging clinicians\u2019 needs and developers\u2019 goals. NPJ Digit. Med. 6, 94 (2023).","journal-title":"NPJ Digit. Med."},{"key":"19510_CR10","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1186\/s12911-020-01332-6","volume":"20","author":"J Amann","year":"2020","unstructured":"Amann, J. et al. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med. Inform. Decis. Mak. 20, 310 (2020).","journal-title":"BMC Med. Inform. Decis. Mak."},{"key":"19510_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2023.107555","volume":"166","author":"S Ali","year":"2023","unstructured":"Ali, S. et al. The enlightening role of explainable artificial intelligence in medical and healthcare domains: A systematic literature review. Comput. Biol. Med. 166, 107555 (2023).","journal-title":"Comput. Biol. Med."},{"key":"19510_CR12","doi-asserted-by":"publisher","first-page":"1328","DOI":"10.1038\/s41591-021-01461-z","volume":"27","author":"S Kundu","year":"2021","unstructured":"Kundu, S. AI in medicine must be explainable. Nat Med 27, 1328\u20131328 (2021).","journal-title":"Nat Med"},{"key":"19510_CR13","doi-asserted-by":"publisher","DOI":"10.1016\/j.compeleceng.2024.109370","volume":"118","author":"Z Sadeghi","year":"2024","unstructured":"Sadeghi, Z. et al. A review of Explainable Artificial Intelligence in healthcare. Comput. Electr. Eng. 118, 109370 (2024).","journal-title":"Comput. Electr. Eng."},{"key":"19510_CR14","doi-asserted-by":"publisher","first-page":"7120","DOI":"10.1126\/scirobotics.aay7120","volume":"4","author":"D Gunning","year":"2019","unstructured":"Gunning, D. et al. XAI-Explainable artificial intelligence. Sci. Robot 4, 7120 (2019).","journal-title":"Sci. Robot"},{"key":"19510_CR15","doi-asserted-by":"publisher","first-page":"25240","DOI":"10.1038\/s41598-025-10906-1","volume":"15","author":"Y Li","year":"2025","unstructured":"Li, Y., Wang, X. & Liu, X. Prediction of gastrointestinal hemorrhage in cardiology inpatients using an interpretable XGBoost model. Sci. Rep. 15, 25240 (2025).","journal-title":"Sci. Rep."},{"key":"19510_CR16","doi-asserted-by":"publisher","first-page":"18336","DOI":"10.1038\/s41598-025-01874-7","volume":"15","author":"A Li","year":"2025","unstructured":"Li, A. et al. Machine learning based predictive model of the risk of Tourette syndrome with SHAP value interpretation: A retrospective observational study. Sci. Rep. 15, 18336 (2025).","journal-title":"Sci. Rep."},{"key":"19510_CR17","doi-asserted-by":"publisher","first-page":"1180773","DOI":"10.3389\/fmed.2023.1180773","volume":"10","author":"BM de Vries","year":"2023","unstructured":"de Vries, B. M. et al. Explainable artificial intelligence (XAI) in radiology and nuclear medicine: A literature review. Front. Med. (Lausanne) 10, 1180773 (2023).","journal-title":"Front. Med. (Lausanne)"},{"key":"19510_CR18","doi-asserted-by":"publisher","DOI":"10.1002\/cai2.136","volume":"3","author":"A Ghasemi","year":"2024","unstructured":"Ghasemi, A., Hashtarkhani, S., Schwartz, D. L. & Shaban-Nejad, A. Explainable artificial intelligence in breast cancer detection and risk prediction: A systematic scoping review. Cancer Innov. 3, e136 (2024).","journal-title":"Cancer Innov."},{"key":"19510_CR19","doi-asserted-by":"publisher","first-page":"524","DOI":"10.1038\/s41467-023-43095-4","volume":"15","author":"T Chanda","year":"2024","unstructured":"Chanda, T. et al. Dermatologist-like explainable AI enhances trust and confidence in diagnosing melanoma. Nat Commun. 15, 524 (2024).","journal-title":"Nat Commun."},{"key":"19510_CR20","doi-asserted-by":"crossref","unstructured":"E. Ihongbe, I., Fouad, S., F. Mahmoud, T., Rajasekaran, A. & Bhatia, B. Evaluating Explainable Artificial Intelligence (XAI) techniques in chest radiology imaging through a human-centered Lens. PLoS One 19, e0308758 (2024).","DOI":"10.1371\/journal.pone.0308758"},{"key":"19510_CR21","unstructured":"Health, C. for D. and R. Artificial Intelligence and Machine Learning (AI\/ML)-Enabled Medical Devices. FDA (2024)."},{"key":"19510_CR22","doi-asserted-by":"publisher","unstructured":"Rajpura, P., Cecotti, H. & Meena, Y. K. Explainable artificial intelligence approaches for brain-computer interfaces: a review and design space. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2312.13033 (2023).","DOI":"10.48550\/arXiv.2312.13033"},{"key":"19510_CR23","doi-asserted-by":"publisher","first-page":"1346","DOI":"10.3389\/fnins.2019.01346","volume":"13","author":"JM Fellous","year":"2019","unstructured":"Fellous, J. M., Sapiro, G., Rossi, A., Mayberg, H. & Ferrante, M. Explainable artificial intelligence for neuroscience: Behavioral neurostimulation. Front. Neurosci. 13, 1346 (2019).","journal-title":"Front. Neurosci."},{"key":"19510_CR24","doi-asserted-by":"publisher","first-page":"5261","DOI":"10.1007\/s10462-022-10304-3","volume":"56","author":"F Di Martino","year":"2023","unstructured":"Di Martino, F. & Delmastro, F. Explainable AI for clinical and remote health applications: A survey on tabular and time series data. Artif. Intell. Rev. 56, 5261\u20135315 (2023).","journal-title":"Artif. Intell. Rev."},{"key":"19510_CR25","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1038\/s41746-024-01190-w","volume":"7","author":"A Pahud de Mortanges","year":"2024","unstructured":"Pahud de Mortanges, A. et al. Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging. npj Digit. Med. 7, 195 (2024).","journal-title":"npj Digit. Med."},{"key":"19510_CR26","unstructured":"Article 86: Right to Explanation of Individual Decision-Making | EU Artificial Intelligence Act. https:\/\/artificialintelligenceact.eu\/article\/86\/."},{"key":"19510_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3712296","volume":"6","author":"Q Sun","year":"2025","unstructured":"Sun, Q., Akman, A. & Schuller, B. W. Explainable artificial intelligence for medical applications: A review. ACM Trans. Comput. Healthcare 6, 1\u201317 (2025).","journal-title":"ACM Trans. Comput. Healthcare"},{"key":"19510_CR28","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1002\/hast.973","volume":"49","author":"AJ London","year":"2019","unstructured":"London, A. J. Artificial intelligence and black-box medical decisions: Accuracy versus explainability. Hastings Cent. Rep. 49, 15\u201321 (2019).","journal-title":"Hastings Cent. Rep."},{"key":"19510_CR29","doi-asserted-by":"publisher","unstructured":"Kasirajan, M., Bin Azhar, M. A. H. & Turner, S. Trustworthy Insights: A Novel Multi-Tier Explainable Framework for Ambient Assisted Living. in 2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) 2554\u20132561 (2023). https:\/\/doi.org\/10.1109\/TrustCom60117.2023.00357.","DOI":"10.1109\/TrustCom60117.2023.00357"},{"key":"19510_CR30","doi-asserted-by":"publisher","first-page":"1893","DOI":"10.1007\/s11135-017-0574-8","volume":"52","author":"B Saunders","year":"2018","unstructured":"Saunders, B. et al. Saturation in qualitative research: Exploring its conceptualization and operationalization. Qua.l Quant. 52, 1893\u20131907 (2018).","journal-title":"Qua.l Quant."},{"key":"19510_CR31","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/s43681-022-00177-1","volume":"3","author":"G Starke","year":"2023","unstructured":"Starke, G., Schmidt, B., De Clercq, E. & Elger, B. S. Explainability as fig leaf? An exploration of experts\u2019 ethical expectations towards machine learning in psychiatry. AI Ethics 3, 303\u2013314 (2023).","journal-title":"AI Ethics"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-19510-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-19510-9","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-19510-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,3]],"date-time":"2025-10-03T13:45:17Z","timestamp":1759499117000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-19510-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,3]]},"references-count":31,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["19510"],"URL":"https:\/\/doi.org\/10.1038\/s41598-025-19510-9","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-6704151\/v1","asserted-by":"object"}]},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,3]]},"assertion":[{"value":"20 May 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 September 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 October 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"LS is an employee at Boston Consulting Group GmbH. MI has been a policy advisor on neurotechnology to the OECD, the Council of Europe, the UN, and the EU Parliament. He is a member of the ethics board of IDUN Technologies, a company producing EEG-earbuds.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"This study received an ethics waiver granted by the TUM Ethics Committee (2024-69-W-NP). The ethics committee judged that this study does not fall under restrictions imposed on research with human subjects. This study adheres to the Declaration of Helsinki.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"All authors contributed to this article and approved the submitted version.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}],"article-number":"34638"}}