{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T17:14:56Z","timestamp":1772644496641,"version":"3.50.1"},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T00:00:00Z","timestamp":1745884800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T00:00:00Z","timestamp":1745884800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010269","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["203148\/A\/16\/Z"],"award-info":[{"award-number":["203148\/A\/16\/Z"]}],"id":[{"id":"10.13039\/100010269","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/R513064\/1"],"award-info":[{"award-number":["EP\/R513064\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Purpose:<\/jats:title>\n            <jats:p>Mechanical thrombectomy (MT) is the gold standard for treating acute ischemic stroke. However, challenges such as operator radiation exposure, reliance on operator experience, and limited treatment access remain. Although autonomous robotics could mitigate some of these limitations, current research lacks benchmarking of reinforcement learning (RL) algorithms for MT. This study aims to evaluate the performance of Deep Deterministic Policy Gradient, Twin Delayed Deep Deterministic Policy Gradient, Soft Actor-Critic, and Proximal Policy Optimization for MT.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods:<\/jats:title>\n            <jats:p>Simulated endovascular interventions based on the open-source stEVE platform were employed to train and evaluate RL algorithms. We simulated navigation of a guidewire from the descending aorta to the supra-aortic arteries, a key phase in MT. The impact of tuning hyperparameters, such as learning rate and network size, was explored. Optimized hyperparameters were used for assessment on an MT benchmark.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results:<\/jats:title>\n            <jats:p>Before tuning, Deep Deterministic Policy Gradient had the highest success rate at 80% with a procedure time of 6.87\u00a0s when navigating to the supra-aortic arteries. After tuning, Proximal Policy Optimization achieved the highest success rate at 84% with a procedure time of 5.08\u00a0s. On the MT benchmark, Twin Delayed Deep Deterministic Policy Gradient recorded the highest success rate at 68% with a procedure time of 214.05\u00a0s.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion:<\/jats:title>\n            <jats:p>This work advances autonomous endovascular navigation by establishing a benchmark for MT. The results emphasize the importance of hyperparameter tuning on the performance of RL algorithms. Future research should extend this benchmark to identify the most effective RL algorithm. \n<\/jats:p>\n          <\/jats:sec>","DOI":"10.1007\/s11548-025-03360-x","type":"journal-article","created":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T11:45:41Z","timestamp":1745927141000},"page":"1231-1238","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Benchmarking reinforcement learning algorithms for autonomous mechanical thrombectomy"],"prefix":"10.1007","volume":"20","author":[{"given":"Farhana","family":"Moosa","sequence":"first","affiliation":[]},{"given":"Harry","family":"Robertshaw","sequence":"additional","affiliation":[]},{"given":"Lennart","family":"Karstensen","sequence":"additional","affiliation":[]},{"given":"Thomas C.","family":"Booth","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2866-1324","authenticated-orcid":false,"given":"Alejandro","family":"Granados","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,4,29]]},"reference":[{"issue":"1","key":"3360_CR1","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1515\/cdbme-2019-0002","volume":"5","author":"T Behr","year":"2019","unstructured":"Behr T, Pusch TP, Siegfarth M, H\u00fcsener D, M\u00f6rschel T, Karstensen L (2019) Deep reinforcement learning for the navigation of neurovascular catheters. Curr Dir Biomed Eng 5(1):5\u20138. https:\/\/doi.org\/10.1515\/cdbme-2019-0002","journal-title":"Curr Dir Biomed Eng"},{"key":"3360_CR2","doi-asserted-by":"crossref","unstructured":"Cho Y, Park JH, Choi J, Chang DE (2022) Sim-to-real transfer of image-based autonomous guidewire navigation trained by deep deterministic policy gradient with behavior cloning for fast learning. In: 2022 IEEE\/RSJ international conference on intelligent robots and systems, pp 3468\u20133475","DOI":"10.1109\/IROS47612.2022.9982168"},{"issue":"1","key":"3360_CR3","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1177\/17474930211065917","volume":"17","author":"VL Feigin","year":"2022","unstructured":"Feigin VL, Brainin M, Norrving B, Martins S, Sacco RL, Hacke W, Fisher M, Pandian JD, Lindsay P (2022) World Stroke Organization (WSO): global stroke fact sheet 2022. Int J Stroke 17(1):18\u201329. https:\/\/doi.org\/10.1177\/17474930211065917","journal-title":"Int J Stroke"},{"key":"3360_CR4","unstructured":"Fujimoto S, van Hoof H, Meger D (2018) Addressing function approximation error in actor-critic methods. In: International conference on machine learning, pp 1587\u20131596"},{"key":"3360_CR5","unstructured":"Haarnoja T, Zhou A, Abbeel P, Levine S (2018) Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: International conference on machine learning, pp 1861\u20131870"},{"key":"3360_CR6","unstructured":"Helfenstein F (2021) Benchmarking deep reinforcement learning algorithms. Bachelor\u2019s thesis, Technische Universit\u00e4t Darmstadt"},{"issue":"11","key":"3360_CR7","doi-asserted-by":"publisher","first-page":"1977","DOI":"10.1007\/s11548-023-02991-2","volume":"18","author":"B Jackson","year":"2023","unstructured":"Jackson B, Crinnion W, De Iturrate Reyzabal M, Robertshaw H, Bergeles C, Rhode K, Booth T (2023) Comparative verification of control methodology for robotic interventional neuroradiology procedures. Int J Comput Assist Radiol Surg 18(11):1977\u20131986. https:\/\/doi.org\/10.1007\/s11548-023-02991-2","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3360_CR8","doi-asserted-by":"publisher","unstructured":"Jianu T, Huang B, Vu MN, Abdelaziz ME, Fichera S, Lee CY, Berthet-Rayne P, y Baena FR, Nguyen A, (2024) Cathsim: an open-source simulator for endovascular intervention. IEEE Trans Med Robot Bion 6(3):971\u2013979. https:\/\/doi.org\/10.1109\/TMRB.2024.3421256","DOI":"10.1109\/TMRB.2024.3421256"},{"issue":"9","key":"3360_CR9","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1007\/s11548-023-02938-7","volume":"18","author":"L Karstensen","year":"2023","unstructured":"Karstensen L, Ritter J, Hatzl J, Ernst F, Langej\u00fcrgen J, Uhl C, Mathis-Ullrich F (2023) Recurrent neural networks for generalization towards the vessel geometry in autonomous endovascular guidewire navigation in the aortic arch. Int J Comput Assist Radiol Surg 18(9):1735\u20131744. https:\/\/doi.org\/10.1007\/s11548-023-02938-7","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3360_CR10","unstructured":"Karstensen L, Robertshaw H, Hatzl J, Jackson B, Langej\u00fcrgen J, Breininger K, Uhl C, Sadati SMH, Booth T, Bergeles C, Mathis-Ullrich F (2024) Learning-based autonomous navigation, benchmark environments and simulation framework for endovascular interventions. Preprint at https:\/\/arxiv.org\/abs\/2410.01956"},{"key":"3360_CR11","unstructured":"Lillicrap T (2015) Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971"},{"issue":"4","key":"3360_CR12","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1177\/2396987317733343","volume":"2","author":"P McMeekin","year":"2017","unstructured":"McMeekin P, White P, James MA, Price CI, Flynn D, Ford GA (2017) Estimating the number of uk stroke patients eligible for endovascular thrombectomy. Eur Stroke J 2(4):319\u2013326. https:\/\/doi.org\/10.1177\/2396987317733343","journal-title":"Eur Stroke J"},{"key":"3360_CR13","doi-asserted-by":"crossref","unstructured":"Meng F, Guo S, Zhou W, Chen Z (2022) Evaluation of an autonomous navigation method for vascular interventional surgery in virtual environment. In: 2022 IEEE international conference on mechatronics and automation, pp 1599\u20131604","DOI":"10.1109\/ICMA54519.2022.9856107"},{"issue":"1","key":"3360_CR14","first-page":"36","volume":"15","author":"JW Mock","year":"2023","unstructured":"Mock JW, Muknahallipatna SS (2023) A comparison of PPO, TD3 and SAC reinforcement algorithms for quadruped walking gait generation. J Intell Learn Syst Appl 15(1):36\u201356","journal-title":"J Intell Learn Syst Appl"},{"key":"3360_CR15","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1007\/s00276-008-0442-2","volume":"31","author":"KI Natsis","year":"2009","unstructured":"Natsis KI, Tsitouridis IA, Didagelos MV, Fillipidis AA, Vlasis KG, Tsikaras PD (2009) Anatomical variations in the branches of the human aortic arch in 633 angiographies: clinical significance and literature review. Surg Radiol Anat 31:319\u2013323. https:\/\/doi.org\/10.1007\/s00276-008-0442-2","journal-title":"Surg Radiol Anat"},{"issue":"1\u20132","key":"3360_CR16","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1159\/000438774","volume":"4","author":"M Palaniswami","year":"2015","unstructured":"Palaniswami M, Yan B (2015) Mechanical thrombectomy is now the gold standard for acute ischemic stroke: implications for routine clinical practice. Interv Neurol 4(1\u20132):18\u201329. https:\/\/doi.org\/10.1159\/000438774","journal-title":"Interv Neurol"},{"key":"3360_CR17","unstructured":"Sentinel Stroke National\u00a0Audit Programme (2023) SSNAP annual report 2023"},{"issue":"268","key":"3360_CR18","first-page":"1","volume":"22","author":"A Raffin","year":"2021","unstructured":"Raffin A, Hill A, Gleave A, Kanervisto A, Ernestus M, Dormann N (2021) Stable-baselines3: reliable reinforcement learning implementations. J Mach Learn Res 22(268):1\u20138","journal-title":"J Mach Learn Res"},{"key":"3360_CR19","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1515\/cdbme-2022-0006","volume":"8","author":"J Ritter","year":"2022","unstructured":"Ritter J, Karstensen L, Langej\u00fcrgen J, Hatzl J, Mathis-Ullrich F, Uhl C (2022) Quality-dependent deep learning for safe autonomous guidewire navigation. Curr Dir Biomed Eng 8:21\u201324. https:\/\/doi.org\/10.1515\/cdbme-2022-0006","journal-title":"Curr Dir Biomed Eng"},{"key":"3360_CR20","doi-asserted-by":"publisher","unstructured":"Robertshaw H, Karstensen L, Jackson B, Sadati H, Rhode K, Ourselin S, Granados A, Booth TC (2023) Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review. Front Hum Neurosci. https:\/\/doi.org\/10.3389\/fnhum.2023.1239374","DOI":"10.3389\/fnhum.2023.1239374"},{"issue":"8","key":"3360_CR21","doi-asserted-by":"publisher","first-page":"1569","DOI":"10.1007\/s11548-024-03208-w","volume":"19","author":"H Robertshaw","year":"2024","unstructured":"Robertshaw H, Karstensen L, Jackson B, Granados A, Booth TC (2024) Autonomous navigation of catheters and guidewires in mechanical thrombectomy using inverse reinforcement learning. Int J Comput Assist Radiol Surg 19(8):1569\u20131578. https:\/\/doi.org\/10.1007\/s11548-024-03208-w","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3360_CR22","doi-asserted-by":"publisher","unstructured":"Rodrigues FB, Neves JB, Caldeira D, Ferro JM, Ferreira JJ, Costa J (2016) Endovascular treatment versus medical care alone for ischaemic stroke: systematic review and meta-analysis. BMJ. https:\/\/doi.org\/10.1136\/bmj.i1754","DOI":"10.1136\/bmj.i1754"},{"key":"3360_CR23","doi-asserted-by":"crossref","unstructured":"Saeidi H, Le HN, Opfermann JD, L\u00e9onard S, Kim A, Hsieh MH, Kang JU, Krieger A (2019) Autonomous laparoscopic robotic suturing with a novel actuated suturing tool and 3D endoscope. In: 2019 international conference on robotics and automation, pp 1541\u20131547","DOI":"10.1109\/ICRA.2019.8794306"},{"key":"3360_CR24","doi-asserted-by":"publisher","unstructured":"Saver JL, Goyal M, van der Lugt A, Menon BK, Majoie CBLM, Dippel DW, Campbell BC, Nogueira RG, Demchuk AM, Tomasello A, Cardona P, Devlin TG, Frei DF, du Mesnil de Rochemont R, Berkhemer OA, Jovin TG, Siddiqui AH, van Zwam WH, Davis SM, Casta\u00f1o C, Sapkota BL, Fransen PS, Molina C, van Oostenbrugge RJ, Chamorro A, Lingsma H, Silver FL, Donnan GA, Shuaib A, Brown S, Stouch B, Mitchell PJ, Davalos A, Roos YBWEM, Hill MD, for the HERMES Collaborators, (2016) Time to treatment with endovascular thrombectomy and outcomes from ischemic stroke: a meta-analysis. JAMA 316(12):1279\u20131289. https:\/\/doi.org\/10.1001\/jama.2016.13647","DOI":"10.1001\/jama.2016.13647"},{"key":"3360_CR25","unstructured":"Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347"},{"issue":"9","key":"3360_CR26","doi-asserted-by":"publisher","first-page":"2736","DOI":"10.1161\/STROKEAHA.120.031940","volume":"52","author":"Zhu F, Hassen WB, Bricout N, Kerleroux B, Janot K, Gory B, Anxionnat R, Richard S, Marchal A, Blanc R, Piotin M, Consoli A, Trystram D, Regent CR, Desilles JP, Weisenburger-Lile D, Escalard S, Herbreteau D, Ifergan H, Maldonado IL, Labreuche J, Henon H, Naggara O, Lapergue B, Boulouis G, on behalf of the ETIS Registry and Study Collaborators","year":"2021","unstructured":"Zhu F, Hassen WB, Bricout N, Kerleroux B, Janot K, Gory B, Anxionnat R, Richard S, Marchal A, Blanc R, Piotin M, Consoli A, Trystram D, Regent CR, Desilles JP, Weisenburger-Lile D, Escalard S, Herbreteau D, Ifergan H, Maldonado IL, Labreuche J, Henon H, Naggara O, Lapergue B, Boulouis G, on behalf of the ETIS Registry and Study Collaborators (2021) Effect of operator\u2019s experience on proficiency in mechanical thrombectomy: a multicenter study. Stroke 52(9):2736\u20132742. https:\/\/doi.org\/10.1161\/STROKEAHA.120.031940","journal-title":"Stroke"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-025-03360-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-025-03360-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-025-03360-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,14]],"date-time":"2025-06-14T11:34:46Z","timestamp":1749900886000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-025-03360-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,29]]},"references-count":26,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["3360"],"URL":"https:\/\/doi.org\/10.1007\/s11548-025-03360-x","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,29]]},"assertion":[{"value":"28 February 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 March 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 April 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"We obtained ethics approval from the Health Research Authority (HRA) to use anonymised computed tomography angiography (CTA) scans which encompassed the aortic arch and extended to the cerebral vessels (UK Research Ethics Committee 20\/ES\/0005).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics statement"}},{"value":"The authors have no conflict of interest to declare that are relevant to the content of this article.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The code is available at https:\/\/github.com\/Farhana-M\/RL-ENDOVASCULAR","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code availability"}}]}}