{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T20:20:34Z","timestamp":1778185234795,"version":"3.51.4"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2025,4,3]],"date-time":"2025-04-03T00:00:00Z","timestamp":1743638400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,4,3]],"date-time":"2025-04-03T00:00:00Z","timestamp":1743638400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010269","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["203148\/A\/16\/Z"],"award-info":[{"award-number":["203148\/A\/16\/Z"]}],"id":[{"id":"10.13039\/100010269","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/R513064\/1"],"award-info":[{"award-number":["EP\/R513064\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Purpose<\/jats:title>\n            <jats:p>Autonomous systems in mechanical thrombectomy (MT) hold promise for reducing procedure times, minimizing radiation exposure, and enhancing patient safety. However, current reinforcement learning (RL) methods only reach the carotid arteries, are not generalizable to other patient vasculatures, and do not consider safety. We propose a safe dual-device RL algorithm that can navigate beyond the carotid arteries to cerebral vessels.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>We used the Simulation Open Framework Architecture to represent the intricacies of cerebral vessels, and a modified Soft Actor-Critic RL algorithm to learn, for the first time, the navigation of micro-catheters and micro-guidewires. We incorporate patient safety metrics into our reward function by integrating guidewire tip forces. Inverse RL is used with demonstrator data on 12 patient-specific vascular cases.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>Our simulation demonstrates successful autonomous navigation within unseen cerebral vessels, achieving a 96% success rate, 7.0\u00a0s procedure time, and 0.24\u00a0N mean forces, well below the proposed 1.5\u00a0N vessel rupture threshold.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion<\/jats:title>\n            <jats:p>To the best of our knowledge, our proposed autonomous system for MT two-device navigation reaches cerebral vessels, considers safety, and is generalizable to unseen patient-specific cases for the first time. We envisage future work will extend the validation to vasculatures of different complexity and on in vitro models. While our contributions pave the way toward deploying agents in clinical settings, safety and trustworthiness will be crucial elements to consider when proposing new methodology.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1007\/s11548-025-03339-8","type":"journal-article","created":{"date-parts":[[2025,4,5]],"date-time":"2025-04-05T06:53:18Z","timestamp":1743835998000},"page":"1077-1086","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Reinforcement learning for safe autonomous two-device navigation of cerebral vessels in mechanical thrombectomy"],"prefix":"10.1007","volume":"20","author":[{"given":"Harry","family":"Robertshaw","sequence":"first","affiliation":[]},{"given":"Benjamin","family":"Jackson","sequence":"additional","affiliation":[]},{"given":"Jiaheng","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Hadi","family":"Sadati","sequence":"additional","affiliation":[]},{"given":"Lennart","family":"Karstensen","sequence":"additional","affiliation":[]},{"given":"Alejandro","family":"Granados","sequence":"additional","affiliation":[]},{"given":"Thomas C.","family":"Booth","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,4,3]]},"reference":[{"key":"3339_CR1","doi-asserted-by":"crossref","unstructured":"Vos T, Lim SS, Abbafati C, Abbas KM, Abbasi M, Abbasifard M, Abbasi-Kangevari M, Abbastabar H, Abd-Allah F, Abdelalim A, Abdollahi M, Abdollahpour I, Abolhassani H, Aboyans V, Abrams EM, Abreu LG, Abrigo MRM, Abu-Raddad LJ, Abushouk AI, Acebedo A et al (2020) Global burden of 369 diseases and injuries in 204 countries and territories, 1990\u20132019: a systematic analysis for the global burden of disease study 2019. The Lancet vol 396, pp 1204\u20131222, 10 [Online]. Available: https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0140673620309259","DOI":"10.1016\/S0140-6736(20)30925-9"},{"key":"3339_CR2","doi-asserted-by":"crossref","unstructured":"Tsao CW, Aday AW, Almarzooq ZI, Anderson CA, Arora P, Avery CL, Baker-Smith CM, Beaton AZ, Boehme AK, Buxton AE, Commodore-Mensah Y, Elkind MS, Evenson KR, Eze-Nliam C, Fugar S, Generoso G, Heard DG, Hiremath S, Ho JE, Kalani R et al (2023) \u201cHeart disease and stroke statistics - 2023 update: a report from the American heart association. E93\u2013E621, 2","DOI":"10.1161\/CIR.0000000000001123"},{"key":"3339_CR3","doi-asserted-by":"publisher","unstructured":"Bendszus M, Fiehler J, Subtil F, Bonekamp S, Aamodt AH, Fuentes B, Gizewski ER, Hill MD, Krajina A, Pierot L, Simonsen CZ, Zele\u0148\u00e1k K, Blauenfeldt RA, Cheng B, Denis A, Deutschmann H, Dorn F, Flottmann F, Gelli\u00dfen S, Gerber JC, Goyal M, Haring J, Herweh C, Hopf-Jensen S, Hua VT, Jensen M, Kastrup A, Keil CF, Klepanec A, Kur\u010da E, Mikkelsen R, M\u00f6hlenbruch M, M\u00fcller-H\u00fclsbeck S, M\u00fcnnich N, Pagano P, Papanagiotou P, Petzold GC, Pham M, Puetz V, Raupach J, Reimann G, Ringleb PA, Schell M, Schlemm E, Sch\u00f6nenberger S, Tenn\u00f8e B, Ulfert C, Vali\u0161 K, V\u00edtkov\u00e1 E, Vollherbst DF, Wick W, Thomalla G; TENSION Investigators (2023) Endovascular thrombectomy for acute ischaemic stroke with established large infarct: multicentre, open-label, randomised trial. Lancet 402(10414):1753\u20131763. https:\/\/doi.org\/10.1016\/S0140-6736(23)02032-9","DOI":"10.1016\/S0140-6736(23)02032-9"},{"key":"3339_CR4","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1056\/NEJMoa1706442","volume":"378","author":"RG Nogueira","year":"2018","unstructured":"Nogueira RG, Jadhav AP, Haussen DC, Bonafe A, Budzik RF, Bhuva P, Yavagal DR, Ribo M, Cognard C, Hanel RA, Sila CA, Hassan AE, Millan M, Levy EI, Mitchell P, Chen M, English JD, Shah QA, Silver FL, Pereira VM et al (2018) Thrombectomy 6 to 24 hours after stroke with a mismatch between deficit and infarct. N Engl J Med 378:11\u201321","journal-title":"N Engl J Med"},{"key":"3339_CR5","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1177\/2396987317733343","volume":"2","author":"P McMeekin","year":"2017","unstructured":"McMeekin P, White P, James MA, Price CI, Flynn D, Ford GA (2017) Estimating the number of UK stroke patients eligible for endovascular thrombectomy. Eur Stroke J 2:319\u2013326","journal-title":"Eur Stroke J"},{"key":"3339_CR6","unstructured":"(2023) Sentinel Stroke National Audit Programme, Ssnap annual report 2023,\u201d Healthcare Quality Improvement Partnership, 11 accessed 23 October 2024. (online). Available: https:\/\/www.hqip.org.uk\/resource\/ssnap-nov-2023\/"},{"issue":"1","key":"3339_CR7","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1016\/S0720-048X(01)00339-4","volume":"39","author":"KA Hausegger","year":"2001","unstructured":"Hausegger KA, Schedlbauer P, Deutschmann HA, Tiesenhausen K (2001) Complications in endoluminal repair of abdominal aortic aneurysms. Eur J Radiol 39(1):22\u201333","journal-title":"Eur J Radiol"},{"key":"3339_CR8","first-page":"538","volume":"250","author":"LW Klein","year":"2009","unstructured":"Klein LW, Miller DL, Balter S, Laskey W, Haines D, Norbash A, Mauro MA, Goldstein JA (2009) Occupational health hazards in the interventional laboratory: time for a safer environment. Soc Interv Radiol 250:538\u2013544","journal-title":"Soc Interv Radiol"},{"key":"3339_CR9","doi-asserted-by":"publisher","first-page":"539","DOI":"10.1136\/neurintsurg-2021-018096","volume":"14","author":"W Crinnion","year":"2022","unstructured":"Crinnion W, Jackson B, Sood A, Lynch J, Bergeles C, Liu H, Rhode K, Pereira VM, Booth TC (2022) Robotics in neurointerventional surgery: a systematic review of the literature. J Neurointerv Surg 14:539\u2013545","journal-title":"J Neurointerv Surg"},{"key":"3339_CR10","doi-asserted-by":"publisher","unstructured":"Jackson B, Crinnion W, De Iturrate Reyzabal M, Robertshaw H, Bergeles C, Rhode K, Booth T (2023) Comparative verification of control methodology for robotic interventional neuroradiology procedures. Int J Comput Assist Radiol Surg 18(11):1977\u20131986. https:\/\/doi.org\/10.1007\/s11548-023-02991-2","DOI":"10.1007\/s11548-023-02991-2"},{"key":"3339_CR11","doi-asserted-by":"crossref","unstructured":"Robertshaw H, Karstensen L, Jackson B, Sadati H, Rhode K, Ourselin S, Granados A, Booth TC (2023) Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review. Front Hum Neurosci 17 (online). Available: https:\/\/www.frontiersin.org\/articles\/10.3389\/fnhum.2023.1239374\/full","DOI":"10.3389\/fnhum.2023.1239374"},{"key":"3339_CR12","doi-asserted-by":"crossref","unstructured":"Robertshaw H, Karstensen L, Jackson B, Granados A, Booth TC (2024) Autonomous navigation of catheters and guidewires in mechanical thrombectomy using inverse reinforcement learning. Int J Comput Assisted Radiol Surg 6 (online). Available: https:\/\/link.springer.com\/10.1007\/s11548-024-03208-w","DOI":"10.1007\/s11548-024-03208-w"},{"key":"3339_CR13","doi-asserted-by":"publisher","unstructured":"Karstensen L, Ritter J, Hatzl J, Ernst F, Langej\u00fcrgen J, Uhl C, Mathis-Ullrich F (2023) Recurrent neural networks for generalization towards the vessel geometry in autonomous endovascular guidewire navigation in the aortic arch. Int J Comput Assist Radiol Surg 18(9):1735\u20131744. https:\/\/doi.org\/10.1007\/s11548-023-02938-7","DOI":"10.1007\/s11548-023-02938-7"},{"key":"3339_CR14","doi-asserted-by":"publisher","unstructured":"Scarponi V, Duprez M, Nageotte F et al (2024) A zero-shot reinforcement learning strategy for autonomous guidewire navigation. Int J CARS 19:1185\u20131192. https:\/\/doi.org\/10.1007\/s11548-024-03092-4","DOI":"10.1007\/s11548-024-03092-4"},{"key":"3339_CR15","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1016\/j.triboint.2005.10.010","volume":"40","author":"K Takashima","year":"2007","unstructured":"Takashima K, Shimomura R, Kitou T, Terada H, Yoshinaka K, Ikeuchi K (2007) Contact and friction between catheter and blood vessel. Tribol Int 40:319\u2013328","journal-title":"Tribol Int"},{"key":"3339_CR16","doi-asserted-by":"publisher","first-page":"1422","DOI":"10.1016\/j.jvs.2015.06.215","volume":"11","author":"H Rafii-Tari","year":"2016","unstructured":"Rafii-Tari H, Riga CV, Payne CJ, Hamady MS, Cheshire NJ, Bicknell CD, Yang GZ (2016) Reducing contact forces in the arch and supra-aortic vessels using the Magellan robot. J Vasc Surg 11:1422\u20131432","journal-title":"J Vasc Surg"},{"key":"3339_CR17","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1515\/cdbme-2022-0023","volume":"7","author":"C Eyberg","year":"2022","unstructured":"Eyberg C, Karstensen L, Pusch T, Horsch J, Langej\u00fcrgen J (2022) A ros2-based testbed environment for endovascular robotic systems. Curr Direct Biomed Eng 7:89\u201392","journal-title":"Curr Direct Biomed Eng"},{"key":"3339_CR18","doi-asserted-by":"publisher","first-page":"230","DOI":"10.3174\/ajnr.A3666","volume":"35","author":"M Shapiro","year":"2014","unstructured":"Shapiro M, Becske T, Riina HA, Raz E, Zumofen D, Jafar JJ, Huang PP, Nelson PK (2014) Toward an endovascular internal carotid artery classification system. Am J Neuroradiol 35:230\u2013236","journal-title":"Am J Neuroradiol"},{"key":"3339_CR19","unstructured":"Karstensen L, Robertshaw H, Hatzl J, Jackson B, Langej\u00fcrgen J, Breininger K, Uhl C, Sadati SMH, Booth T, Bergeles C, Mathis-Ullrich F (2024) Learning-based autonomous navigation, benchmark environments and simulation framework for endovascular interventions. arXiv preprint, 10 (online). Available: http:\/\/arxiv.org\/abs\/2410.01956"},{"key":"3339_CR20","doi-asserted-by":"publisher","first-page":"300","DOI":"10.3109\/10929080601090623","volume":"11","author":"C Duriez","year":"2006","unstructured":"Duriez C, Cotin S, Lenoir J, Neumann P (2006) New approaches to catheter navigation for interventional radiology simulation. Comput Aided Surg 11:300\u2013308","journal-title":"Comput Aided Surg"},{"key":"3339_CR21","doi-asserted-by":"crossref","unstructured":"Faure F, Duriez C, Delingette H, Allard J, Gilles B, Marchesseau S, Talbot H, Courtecuisse H, Bousquet G, Peterlik I, Cotin S (2012) SOFA, a multi-model framework for interactive physical simulation. Springer, Berlin (online). Available: https:\/\/doi.org\/10.1007\/8415_2012_125","DOI":"10.1007\/8415_2012_125"},{"key":"3339_CR22","doi-asserted-by":"publisher","first-page":"2033","DOI":"10.1007\/s11548-022-02646-8","volume":"17","author":"L Karstensen","year":"2022","unstructured":"Karstensen L, Ritter J, Hatzl J, P\u00e4tz T, Langej\u00fcrgen J, Uhl C, Mathis-Ullrich F (2022) Learning-based autonomous vascular guidewire navigation without human demonstration in the venous system of a porcine liver. Int J Comput Assist Radiol Surg 17:2033\u20132040","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3339_CR23","doi-asserted-by":"publisher","first-page":"1323","DOI":"10.1016\/j.mri.2012.05.001","volume":"30","author":"A Fedorov","year":"2012","unstructured":"Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, Fillion-Robin JC, Pujol S, Bauer C, Jennings D, Fennessy F, Sonka M, Buatti J, Aylward S, Miller JV, Pieper S, Kikinis R (2012) 3d slicer as an image computing platform for the quantitative imaging network. Magn Reson Imaging 30:1323\u20131341","journal-title":"Magn Reson Imaging"},{"key":"3339_CR24","doi-asserted-by":"publisher","first-page":"1141","DOI":"10.1109\/TMI.2009.2021652","volume":"28","author":"M Piccinelli","year":"2009","unstructured":"Piccinelli M, Veneziani A, Steinman DA, Remuzzi A, Antiga L (2009) A framework for geometric analysis of vascular structures: application to cerebral aneurysms. IEEE Trans Med Imaging 28:1141\u20131155","journal-title":"IEEE Trans Med Imaging"},{"key":"3339_CR25","first-page":"8024","volume":"32","author":"A Paszke","year":"2019","unstructured":"Paszke A, Gross S, Massa F, Lerer A, Google JB, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, Xamla AK, Yang E, Devito Z, Nabla MR, Tejani A, Chilamkurthy S, Ai Q, Steiner B, Facebook LF, Facebook JB, Chintala S (2019) Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32:8024\u20138035","journal-title":"Adv Neural Inf Process Syst"},{"key":"3339_CR26","unstructured":"Ziebart BD, Maas A, Bagnell JA, Dey AK (2008) Maximum entropy inverse reinforcement learning. In: Proceedings of the twenty-third AAAI conference on artificial intelligence, pp 1433\u20131438"},{"key":"3339_CR27","unstructured":"Gholamy A, Kreinovich V, Kosheleva O (2018) A pedagogical explanation a pedagogical explanation part of the computer sciences commons. Departmental Technical Reports (CS), 1209"},{"key":"3339_CR28","unstructured":"Mankins J (1995) Technology readiness level\u2014a white paper. NASA, Office of Space Access and Technology, Advanced Concepts Office 4 [Online]. Available: https:\/\/www.researchgate.net\/publication\/247705707"},{"key":"3339_CR29","doi-asserted-by":"publisher","first-page":"3213","DOI":"10.1007\/s10462-021-10085-1","volume":"55","author":"J Ram\u00edrez","year":"2022","unstructured":"Ram\u00edrez J, Yu W, Perrusqu\u00eda A (2022) Model-free reinforcement learning from expert demonstrations: a survey. Artif Intell Rev 55:3213\u20133241","journal-title":"Artif Intell Rev"},{"key":"3339_CR30","doi-asserted-by":"crossref","unstructured":"Kweon J, Kim K, Lee C, Kwon H, Park J, Song K, Kim YI, Park J, Back I, Roh JH, Moon Y, Choi J, Kim YH (2021) Deep reinforcement learning for guidewire navigation in coronary artery phantom. IEEE Access 9 166 409\u2013166, 422","DOI":"10.1109\/ACCESS.2021.3135277"},{"key":"3339_CR31","doi-asserted-by":"crossref","unstructured":"Chi W, Dagnino G, Kwok TMY, Nguyen A, Kundrat D, Abdelaziz MEMK, Riga C, Bicknell C, Yang G-Z (2020) Collaborative robot-assisted endovascular catheterization with generative adversarial imitation learning. IEEE International Conference on Robotics and Automation (ICRA) 2414\u20132420","DOI":"10.1109\/ICRA40945.2020.9196912"},{"key":"3339_CR32","doi-asserted-by":"crossref","unstructured":"Beaman C, Gautam A, Peterson C, Kaneko N, Ponce L, Saber H, Khatibi K, Morales J, Kimball D, Lipovac JR, Narsinh KH, Baker A, Caton MT, Smith ER, Nour M, Szeder V, Jahan R, Colby GP, Cord BJ, Cooke DL, Tateshima S, Duckwiler G, Waldau B (2023) Robotic diagnostic cerebral angiography: a multicenter experience of 113 patients. J NeuroInterv Surg","DOI":"10.1136\/jnis-2023-SNIS.46"},{"key":"3339_CR33","doi-asserted-by":"crossref","unstructured":"Lahlouh M, Chenoune Y, Blanc R, Piotin M, Escalard S, Fahed R, Szewczyk J, Passat N (2023) Automated aortic anatomy analysis: from image to clinical indicators. In: Proceedings of the annual international conference of the IEEE engineering in medicine and biology society, EMBS. Institute of Electrical and Electronics Engineers Inc,","DOI":"10.1109\/EMBC40787.2023.10340921"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-025-03339-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-025-03339-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-025-03339-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,14]],"date-time":"2025-06-14T11:34:23Z","timestamp":1749900863000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-025-03339-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,3]]},"references-count":33,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["3339"],"URL":"https:\/\/doi.org\/10.1007\/s11548-025-03339-8","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,3]]},"assertion":[{"value":"22 January 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 March 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 April 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflict of interest to declare.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}