{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T20:20:34Z","timestamp":1778185234014,"version":"3.51.4"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"8","license":[{"start":{"date-parts":[[2024,6,17]],"date-time":"2024-06-17T00:00:00Z","timestamp":1718582400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,6,17]],"date-time":"2024-06-17T00:00:00Z","timestamp":1718582400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/R513064\/1"],"award-info":[{"award-number":["EP\/R513064\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010269","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["203148\/A\/16\/Z"],"award-info":[{"award-number":["203148\/A\/16\/Z"]}],"id":[{"id":"10.13039\/100010269","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Purpose<\/jats:title>\n                <jats:p>Autonomous navigation of catheters and guidewires can enhance endovascular surgery safety and efficacy, reducing procedure times and operator radiation exposure. Integrating tele-operated robotics could widen access to time-sensitive emergency procedures like mechanical thrombectomy (MT). Reinforcement learning (RL) shows potential in endovascular navigation, yet its application encounters challenges without a reward signal. This study explores the viability of autonomous guidewire navigation in MT vasculature using inverse reinforcement learning (IRL) to leverage expert demonstrations.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>Employing the Simulation Open Framework Architecture (SOFA), this study established a simulation-based training and evaluation environment for MT navigation. We used IRL to infer reward functions from expert behaviour when navigating a guidewire and catheter. We utilized the soft actor-critic algorithm to train models with various reward functions and compared their performance in silico.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>We demonstrated feasibility of navigation using IRL. When evaluating single- versus dual-device (i.e. guidewire versus catheter and guidewire) tracking, both methods achieved high success rates of 95% and 96%, respectively. Dual tracking, however, utilized both devices mimicking an expert. A success rate of 100% and procedure time of 22.6\u00a0s were obtained when training with a reward function obtained through \u2018reward shaping\u2019. This outperformed a dense reward function (96%, 24.9\u00a0s) and an IRL-derived reward function (48%, 59.2\u00a0s).<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>We have contributed to the advancement of autonomous endovascular intervention navigation, particularly MT, by effectively employing IRL based on demonstrator expertise. The results underscore the potential of using reward shaping to efficiently train models, offering a promising avenue for enhancing the accessibility and precision of MT procedures. We envisage that future research can extend our methodology to diverse anatomical structures to enhance generalizability.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s11548-024-03208-w","type":"journal-article","created":{"date-parts":[[2024,6,17]],"date-time":"2024-06-17T13:02:07Z","timestamp":1718629327000},"page":"1569-1578","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Autonomous navigation of catheters and guidewires in mechanical thrombectomy using inverse reinforcement learning"],"prefix":"10.1007","volume":"19","author":[{"given":"Harry","family":"Robertshaw","sequence":"first","affiliation":[]},{"given":"Lennart","family":"Karstensen","sequence":"additional","affiliation":[]},{"given":"Benjamin","family":"Jackson","sequence":"additional","affiliation":[]},{"given":"Alejandro","family":"Granados","sequence":"additional","affiliation":[]},{"given":"Thomas C.","family":"Booth","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,6,17]]},"reference":[{"issue":"3232\u20133245","key":"3208_CR1","first-page":"11","volume":"37","author":"N Townsend","year":"2016","unstructured":"Townsend N, Wilson L, Bhatnagar P, Wickramasinghe K, Rayner M, Nichols M (2016) Cardiovascular disease in europe: epidemiological update 2016. Eur Heart J 37(3232\u20133245):11","journal-title":"Eur Heart J"},{"key":"3208_CR2","doi-asserted-by":"publisher","first-page":"1723","DOI":"10.1016\/S0140-6736(16)00163-X","volume":"387","author":"M Goyal","year":"2016","unstructured":"Goyal M, Menon BK, Zwam WHV, Dippel DW, Mitchell PJ, Demchuk AM, D\u00e1valos A, Majoie CB, Lugt AVD, Miquel MAD, Donnan GA, Roos YB, Bonafe A, Jahan R, Diener HC, Berg LAVD, Levy EI, Berkhemer OA, Pereira VM, Rempel J, Mill\u00e1n M, Davis SM, Roy D, Thornton J, Rom\u00e1n LS, Rib\u00f3 M, Beumer D, Stouch B, Brown S, Campbell BC, Oostenbrugge RJV, Saver JL, Hill MD, Jovin TG (2016) Endovascular thrombectomy after large-vessel ischaemic stroke: a meta-analysis of individual patient data from five randomised trials. The Lancet 387:1723\u20131731","journal-title":"The Lancet"},{"key":"3208_CR3","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1177\/1538574417698905","volume":"51","author":"S Vidale","year":"2017","unstructured":"Vidale S, Agostoni E (2017) Endovascular treatment of ischemic stroke: an updated meta-analysis of efficacy and safety. Vasc Endovasc Surg 51:215\u2013219","journal-title":"Vasc Endovasc Surg"},{"key":"3208_CR4","doi-asserted-by":"publisher","first-page":"967","DOI":"10.1161\/01.STR.0000258112.14918.24","volume":"38","author":"JH Rha","year":"2007","unstructured":"Rha JH, Saver JL (2007) The impact of recanalization on ischemic stroke outcome: a meta-analysis. Stroke 38:967\u2013973","journal-title":"Stroke"},{"issue":"11\u201320","key":"3208_CR5","first-page":"1","volume":"372","author":"OA Berkhemer","year":"2015","unstructured":"Berkhemer OA, Fransen PS, Beumer D, van den Berg LA, Lingsma HF, Yoo AJ, Schonewille WJ, Vos JA, Nederkoorn PJ, Wermer MJ, van Walderveen MA, Staals J, Hofmeijer J, van Oostayen JA, Nijeholt GJL, Boiten J, Brouwer PA, Emmer BJ, de Bruijn SF, van Dijk LC, Kappelle LJ, Lo RH, van Dijk EJ, de Vries J, de Kort PL, van Rooij WJJ, van den Berg JS, van Hasselt BA, Aerden LA, Dallinga RJ, Visser MC, Bot JC, Vroomen PC, Eshghi O, Schreuder TH, Heijboer RJ, Keizer K, Tielbeek AV, den Hertog HM, Gerrits DG, van den Berg-Vos RM, Karas GB, Steyerberg EW, Flach HZ, Marquering HA, Sprengers ME, Jenniskens SF, Beenen LF, van den Berg R, Koudstaal PJ, van Zwam WH, Roos YB, van der Lugt A, van Oostenbrugge RJ, Majoie CB, Dippel DW (2015) A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med 372(11\u201320):1","journal-title":"N Engl J Med"},{"key":"3208_CR6","doi-asserted-by":"publisher","first-page":"1279","DOI":"10.1001\/jama.2016.13647","volume":"316","author":"JL Saver","year":"2016","unstructured":"Saver JL, Goyal Lugt AVD, Menon BK, Majoie CB, Dippel DW, Campbell BC, Nogueira RG, Demchuk AM, Tomasello A, Cardona P, Devlin TG, Frei DF, Rochemont RDMD, Berkhemer OA, Jovin TG, Siddiqui AH, Zwam WHV, Davis SM, Casta\u00f1o C, Sapkota BL, Fransen PS, Molina C, Oostenbrugge RJV, \u00c1ngel Chamorro, Lingsma H, Silver FL, Donnan GA, Shuaib A, Brown S, Stouch B, Mitchell PJ, Davalos A, Roos YB, Hill MD (2016) Time to treatment with endovascular thrombectomy and outcomes from ischemic stroke: a meta-analysis. JAMA J Am Med Assoc 316:1279\u20131288","journal-title":"JAMA J Am Med Assoc"},{"key":"3208_CR7","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1177\/2396987317733343","volume":"2","author":"P McMeekin","year":"2017","unstructured":"McMeekin P, White P, James MA, Price CI, Flynn D, Ford GA (2017) Estimating the number of UK stroke patients eligible for endovascular thrombectomy. Eur Stroke J 2:319\u2013326","journal-title":"Eur Stroke J"},{"key":"3208_CR8","unstructured":"Programme SSNA (2023) Ssnap annual report 2023 [Online]. Available: www.hqip.org.uk\/national-programmes"},{"key":"3208_CR9","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1016\/S0720-048X(01)00339-4","volume":"39","author":"KA Hausegger","year":"2001","unstructured":"Hausegger KA, Schedlbauer P, Deutschmann HA, Tiesenhausen K (2001) Complications in endoluminal repair of abdominal aortic aneurysms. Eur J Radiol 39:22\u201333","journal-title":"Eur J Radiol"},{"key":"3208_CR10","doi-asserted-by":"publisher","first-page":"254","DOI":"10.1038\/ki.1995.32","volume":"47","author":"MR Rudnick","year":"1995","unstructured":"Rudnick MR, Goldfarb S, Wexler L, Ludbrook PA, Murphy MJ, Halpern EF, Hill JA, Winniford M, Cohen MB, VanFossen DB (1995) Nephrotoxicity of ionic and nonionic contrast media in 1196 patients: a randomized trial. Kidney Int 47:254\u2013261","journal-title":"Kidney Int"},{"key":"3208_CR11","first-page":"538","volume":"250","author":"LW Klein","year":"2009","unstructured":"Klein LW, Miller DL, Balter S, Laskey W, Haines D, Norbash A, Mauro MA, Goldstein JA (2009) Occupational health hazards in the interventional laboratory: time for a safer environment. Soc Interv Radiol 250:538\u2013544","journal-title":"Soc Interv Radiol"},{"key":"3208_CR12","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1016\/j.jvs.2007.04.034","volume":"46","author":"P Ho","year":"2007","unstructured":"Ho P, Cheng SW, Wu PM, Ting AC, Poon JT, Cheng CK, Mok JH, Tsang MS (2007) Ionizing radiation absorption of vascular surgeons during endovascular procedures. J Vasc Surg 46:455\u2013459","journal-title":"J Vasc Surg"},{"key":"3208_CR13","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1016\/j.carrev.2016.12.011","volume":"18","author":"RD Madder","year":"2017","unstructured":"Madder RD, VanOosterhout S, Mulder A, Elmore M, Campbell J, Borgman A, Parker J, Wohns D (2017) Impact of robotics and a suspended lead suit on physician radiation exposure during percutaneous coronary intervention. Cardiovasc Revascularization Med 18:190\u2013196","journal-title":"Cardiovasc Revascularization Med"},{"key":"3208_CR14","doi-asserted-by":"publisher","first-page":"539","DOI":"10.1136\/neurintsurg-2021-018096","volume":"14","author":"W Crinnion","year":"2022","unstructured":"Crinnion W, Jackson B, Sood A, Lynch J, Bergeles C, Liu H, Rhode K, Pereira VM, Booth TC (2022) Robotics in neurointerventional surgery: a systematic review of the literature. J Neurointerventional Surg 14:539\u2013545","journal-title":"J Neurointerventional Surg"},{"key":"3208_CR15","doi-asserted-by":"publisher","first-page":"810","DOI":"10.1016\/j.jvs.2009.08.101","volume":"51","author":"CV Riga","year":"2010","unstructured":"Riga CV, Cheshire NJ, Hamady MS, Bicknell CD (2010) The role of robotic endovascular catheters in fenestrated stent grafting. J Vasc Surg 51:810\u2013820","journal-title":"J Vasc Surg"},{"key":"3208_CR16","doi-asserted-by":"publisher","first-page":"477","DOI":"10.3934\/Neuroscience.2021025","volume":"8","author":"M Mofatteh","year":"2021","unstructured":"Mofatteh M (2021) Neurosurgery and artificial intelligence. AIMS Neurosci 8:477\u2013495","journal-title":"AIMS Neurosci"},{"key":"3208_CR17","doi-asserted-by":"crossref","unstructured":"Jackson B, Crinnion W, Reyzabal MDI, Robertshaw H, Bergeles C, Rhode K, Booth T (2023) Comparative verification of control methodology for robotic interventional neuroradiology procedures, International Journal of Computer Assisted Radiology and Surgery, 7. [Online]. Available: https:\/\/link.springer.com\/10.1007\/s11548-023-02991-2","DOI":"10.1007\/s11548-023-02991-2"},{"key":"3208_CR18","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s42979-021-00592-x","volume":"2","author":"IH Sarker","year":"2021","unstructured":"Sarker IH (2021) Machine learning: algorithms, real-world applications and research directions. SN Comput Sci 2:5","journal-title":"SN Comput Sci"},{"key":"3208_CR19","first-page":"1","volume":"09","author":"M Fatima","year":"2017","unstructured":"Fatima M, Pasha M (2017) Survey of machine learning algorithms for disease diagnostic. J Intell Learn Syst Appl 09:1\u201316","journal-title":"J Intell Learn Syst Appl"},{"key":"3208_CR20","doi-asserted-by":"publisher","first-page":"1547","DOI":"10.1080\/03610926.2019.1622728","volume":"50","author":"G Silahtaro\u011flu","year":"2021","unstructured":"Silahtaro\u011flu G, Y\u0131lmazt\u00fcrk N (2021) Data analysis in health and big data: a machine learning medical diagnosis model based on patients\u2019 complaints. Commun Stat Theory Methods 50:1547\u20131556","journal-title":"Commun Stat Theory Methods"},{"key":"3208_CR21","doi-asserted-by":"publisher","first-page":"1239374","DOI":"10.3389\/fnhum.2023.1239374","volume":"17","author":"H Robertshaw","year":"2023","unstructured":"Robertshaw H, Karstensen L, Jackson B, Sadati H, Rhode K, Ourselin S, Granados A, Booth TC (2023) Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review. Front Hum Neurosci 17:1239374","journal-title":"Front Hum Neurosci"},{"key":"3208_CR22","doi-asserted-by":"crossref","unstructured":"Sutton RS, Barto AG (1998) Introduction to reinforcement learning, 1st\u00a0ed. MIT Press","DOI":"10.1007\/978-1-4615-3618-5_1"},{"key":"3208_CR23","doi-asserted-by":"publisher","first-page":"4307","DOI":"10.1007\/s10462-021-10108-x","volume":"55","author":"S Adams","year":"2022","unstructured":"Adams S, Cody T, Beling PA (2022) A survey of inverse reinforcement learning. Artif Intelli Rev 55:4307\u20134346","journal-title":"Artif Intelli Rev"},{"key":"3208_CR24","doi-asserted-by":"crossref","unstructured":"Chi W, Liu J, Abdelaziz MEMK, Dagnino G, Riga C, Bicknell C, Yang GZ (2018) Trajectory optimization of robot-assisted endovascularcatheterization with reinforcement learning. In: 2018 IEEE\/RSJ International conference on intelligent robots and systems (IROS). IEEE, vol 8, pp 3875\u20133881","DOI":"10.1109\/IROS.2018.8593421"},{"key":"3208_CR25","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1515\/cdbme-2019-0002","volume":"5","author":"T Behr","year":"2019","unstructured":"Behr T, Pusch TP, Siegfarth M, H\u00fcsener D, M\u00f6rschel T, Karstensen L (2019) Deep reinforcement learning for the navigation of neurovascular catheters. Curr Dir Biomed Eng 5:5\u20138","journal-title":"Curr Dir Biomed Eng"},{"key":"3208_CR26","doi-asserted-by":"publisher","first-page":"166409","DOI":"10.1109\/ACCESS.2021.3135277","volume":"9","author":"J Kweon","year":"2021","unstructured":"Kweon J, Kim K, Lee C, Kwon H, Park J, Song K, Kim YI, Park J, Back I, Roh JH, Moon Y, Choi J, Kim YH (2021) Deep reinforcement learning for guidewire navigation in coronary artery phantom. IEEE Access 9:166409\u2013166422","journal-title":"IEEE Access"},{"key":"3208_CR27","unstructured":"Ng AY, Russel S (2000) Algorithms for inverse reinforcement learning. Morgan Kaufmann Publishers Inc., pp 663\u2013670"},{"key":"3208_CR28","unstructured":"Haarnoja T, Zhou A, Abbeel P, Levine S (2018) Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: Dy J, Krause A (Eds.), Proceedings of the 35th international conference on machine learning, ser. Proceedings of machine learning research, vol.\u00a080. PMLR, pp 1861\u20131870. [Online]. Available: https:\/\/proceedings.mlr.press\/v80\/haarnoja18b.html"},{"key":"3208_CR29","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1007\/s11548-023-02938-7","volume":"18","author":"L Karstensen","year":"2023","unstructured":"Karstensen L, Ritter J, Hatzl J, Ernst F, Langej\u00fcrgen J, Uhl C, Mathis-Ullrich F (2023) Recurrent neural networks for generalization towards the vessel geometry in autonomous endovascular guidewire navigation in the aortic arch. Int J Comput Assist Radiol Surg 18:1735\u20131744","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3208_CR30","doi-asserted-by":"publisher","unstructured":"Faure F, Duriez C, Delingette H, Allard J, Gilles B, Marchesseau S, Talbot H, Courtecuisse H, Bousquet G, Peterlik I, Cotin S (2012) SOFA, a multi-model framework for interactive physical simulation. Springer Berlin Heidelberg, 2012. [Online]. Available: https:\/\/doi.org\/10.1007\/8415_2012_125","DOI":"10.1007\/8415_2012_125"},{"key":"3208_CR31","doi-asserted-by":"publisher","first-page":"300","DOI":"10.3109\/10929080601090623","volume":"11","author":"C Duriez","year":"2006","unstructured":"Duriez C, Cotin S, Lenoir J, Neumann P (2006) New approaches to catheter navigation for interventional radiology simulation. Comput Aided Surg 11:300\u2013308","journal-title":"Comput Aided Surg"},{"key":"3208_CR32","unstructured":"Community BO (2018) Blender\u2014a 3D modelling and rendering package, Blender Foundation, Stichting Blender Foundation, Amsterdam. [Online]. Available: http:\/\/www.blender.org"},{"key":"3208_CR33","doi-asserted-by":"publisher","first-page":"2033","DOI":"10.1007\/s11548-022-02646-8","volume":"17","author":"L Karstensen","year":"2022","unstructured":"Karstensen L, Ritter J, Hatzl J, P\u00e4tz T, Langej\u00fcrgen J, Uhl C, Mathis-Ullrich F (2022) Learning-based autonomous vascular guidewire navigation without human demonstration in the venous system of a porcine liver. Int J Comput Assist Radiol Surg 17:2033\u20132040","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3208_CR34","unstructured":"Ziebart BD, Maas A, Bagnell JA, Dey AK (2008) Maximum entropy inverse reinforcement learning. AAAI Press, pp 1433\u20131438. [Online]. Available: www.aaai.org"},{"key":"3208_CR35","unstructured":"Mankins J (1995) Technology readiness level\u2014a white paper, 4. [Online]. Available: https:\/\/www.researchgate.net\/publication\/247705707"},{"key":"3208_CR36","doi-asserted-by":"crossref","unstructured":"Beaman A Gautam, Peterson C, Kaneko N, Ponce L, Saber H, Khatibi K, Morales J, Kimball D, Lipovac JR, Narsinh KH, Baker A, Caton MT, Smith ER, Nour M, Szeder V, Jahan R, Colby GP, Cord BJ, Cooke DL, Tateshima S, Duckwiler G, Waldau B (2023) Robotic diagnostic cerebral angiography: A multicenter experience of 113 patients. J NeuroInterventional Surg","DOI":"10.1136\/jnis-2023-SNIS.46"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03208-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-024-03208-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03208-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,25]],"date-time":"2024-11-25T12:12:21Z","timestamp":1732536741000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-024-03208-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,17]]},"references-count":36,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["3208"],"URL":"https:\/\/doi.org\/10.1007\/s11548-024-03208-w","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,17]]},"assertion":[{"value":"11 January 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 May 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 June 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflict of interest to declare that is relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}