{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,10]],"date-time":"2026-05-10T10:23:34Z","timestamp":1778408614654,"version":"3.51.4"},"reference-count":59,"publisher":"American Association for the Advancement of Science (AAAS)","issue":"109","funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["2R01EB023943-04A1"],"award-info":[{"award-number":["2R01EB023943-04A1"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["1R01EB025883-01A1"],"award-info":[{"award-number":["1R01EB025883-01A1"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007880","name":"Johns Hopkins University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007880","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.science.org"],"crossmark-restriction":true},"short-container-title":["Sci. Robot."],"published-print":{"date-parts":[[2025,12,17]]},"abstract":"<jats:p>Retinal vein cannulation (RVC) is an emerging method for treating retinal vein occlusion (RVO). The success of this procedure depends on surgeon expertise and, recently, robotic assistance. This paper proposes an autonomous RVC workflow leveraging deep learning and computer vision. Two Steady-Hand Eye Robots (SHERs) controlled a 100-micrometer metal needle and a medical spatula to execute precise tasks. Three convolutional neural networks were trained to predict needle movement direction and identify contact and puncture events. A surgical microscope with an intraoperative optical coherence tomography (iOCT) system captured the surgical field through a microscope and cross-sectional images (B-scans). The goal was to enable the robot to autonomously carry out the critical steps of the RVC procedure, especially those that are challenging and require expert knowledge. The less technically demanding tasks were assigned to the user, who also supervised the robot during these steps. Our method was tested on 20 ex vivo porcine eyes, achieving a success rate of 90%. In addition, we simulated eye movements caused by breathing on six other ex vivo porcine eyes. With the eyes moving in a sinusoidal pattern, we achieved a success rate of 83%, demonstrating the robustness and stability of the proposed workflow. Our results demonstrate that the autonomous RVC workflow, incorporating deep learning and robotic assistance, achieves high success rates in both static and dynamic conditions, indicating its potential to enhance the precision and reliability of RVO treatment.<\/jats:p>","DOI":"10.1126\/scirobotics.adw2969","type":"journal-article","created":{"date-parts":[[2025,12,17]],"date-time":"2025-12-17T18:58:21Z","timestamp":1765997901000},"update-policy":"https:\/\/doi.org\/10.34133\/aaas_crossmark","source":"Crossref","is-referenced-by-count":5,"title":["Deep learning\u2013based autonomous retinal vein cannulation in ex vivo porcine eyes"],"prefix":"10.1126","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3717-7945","authenticated-orcid":true,"given":"Peiyao","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD 21211, USA."},{"name":"Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD 21211, USA."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9608-9171","authenticated-orcid":true,"given":"Peter","family":"Gehlbach","sequence":"additional","affiliation":[{"name":"Wilmer Eye Institute, Johns Hopkins University, Baltimore, MD 21211, USA."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6272-1100","authenticated-orcid":true,"given":"Russell H.","family":"Taylor","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD 21211, USA."},{"name":"Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD 21211, USA."},{"name":"Department of Computer Science, Johns Hopkins University, Baltimore, MD 21211, USA."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2510-9008","authenticated-orcid":true,"given":"Iulian","family":"Iordachita","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD 21211, USA."},{"name":"Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD 21211, USA."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-4982-5990","authenticated-orcid":true,"given":"Marin","family":"Kobilarov","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD 21211, USA."},{"name":"Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, MD 21211, USA."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"221","reference":[{"key":"e_1_3_2_2_2","first-page":"40","article-title":"Retinal vein occlusion review","volume":"7","author":"Ip M.","year":"2018","unstructured":"M. Ip, A. Hendrick, Retinal vein occlusion review. Asia Pac. J. Ophthalmol. 7, 40\u201345 (2018).","journal-title":"Asia Pac. J. Ophthalmol."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ophtha.2009.07.017"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ophtha.2012.01.022"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ophtha.2010.02.021"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1136\/bjophthalmol-2012-301504"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1001\/archophthalmol.2009.234"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ophtha.2010.03.032"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1159\/000381357"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1111\/aos.12830"},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","unstructured":"C. N. Riviere P. S. Jensen \u201cA study of instrument motion in retinal microsurgery \u201d in Proceedings of the 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE 2000) vol. 1 pp. 59\u201360.","DOI":"10.1109\/IEMBS.2000.900667"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1097\/IAE.0b013e31829477f2"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2022.3169466"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","unstructured":"E. Vander Poorten C. N. Riviere J. J. Abbott C. Bergeles M. A. Nasseri J. U. Kang R. Sznitman K. Faridpooya I. Iordachita \u201cRobotic retinal surgery \u201d in Handbook of Robotic and Image-Guided Surgery (Elsevier 2020) pp. 627\u2013672.","DOI":"10.1016\/B978-0-12-814245-5.00036-0"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2014.2320858"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajo.2021.11.011"},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","unstructured":"A. \u00dcneri M. A. Balicki J. Handa P. Gehlbach R. H. Taylor I. Iordachita \u201cNew steady-hand eye robot with micro-force sensing for vitreoretinal surgery \u201d in 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (IEEE 2010) pp. 814\u2013819.","DOI":"10.1109\/BIOROB.2010.5625991"},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","unstructured":"M. Esfandiari J. W. Kim B. Zhao G. Amirkhani M. Hadi P. Gehlbach R. H. Taylor I. Iordachita \u201cCooperative vs. teleoperation control of the steady hand eye robot with adaptive sclera force control: A comparative study \u201d in 2024 IEEE International Conference on Robotics and Automation (ICRA) (IEEE 2024) pp. 8209\u20138215.","DOI":"10.1109\/ICRA57147.2024.10611084"},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","unstructured":"J. Yang Z. Zhao M. Maier K. Huang N. Navab M. A. Nasseri \u201cShadow-based 3D pose estimation of intraocular instrument using only 2D images \u201d in 2024 IEEE International Conference on Robotics and Automation (ICRA) (IEEE 2024) pp. 1323\u20131329.","DOI":"10.1109\/ICRA57147.2024.10611011"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2024.3368192"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2024.3370000"},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","unstructured":"P. Zhang J. W. Kim M. Kobilarov \u201cTowards safer retinal surgery through chance constraint optimization and real-time geometry estimation \u201d in 2021 60th IEEE Conference on Decision and Control (CDC) (IEEE 2021) pp. 5175\u20135180.","DOI":"10.1109\/CDC45484.2021.9683329"},{"key":"e_1_3_2_23_2","doi-asserted-by":"crossref","unstructured":"P. Zhang J. W. Kim P. Gehlbach I. Iordachita M. Kobilarov Autonomous needle navigation in retinal microsurgery: Evaluation in ex vivo porcine eyes. arXiv:2301.11839 [cs.RO] (2023).","DOI":"10.1109\/ICRA48891.2023.10161151"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.3390\/app8081287"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1364\/BOE.415477"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2024.3375710"},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","unstructured":"B. Gonenc N. Tran P. Gehlbach R. H. Taylor I. Iordachita \u201cRobot-assisted retinal vein cannulation with force-based puncture detection: Micron vs. the steady-hand eye robot \u201d in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (IEEE 2016) pp. 5107\u20135111.","DOI":"10.1109\/EMBC.2016.7591876"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1167\/iovs.11-7327"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1111\/aos.13358"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-018-2053-3"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1136\/bjophthalmol-2016-309190"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1111\/aos.14480"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2021.3114638"},{"key":"e_1_3_2_34_2","doi-asserted-by":"crossref","unstructured":"A. Gijbels K. Willekens L. Esteveny P. Stalmans D. Reynaerts E. B. Vander Poorten \u201cTowards a clinically applicable robotic assistance system for retinal vein cannulation \u201d in 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob) (IEEE 2016) pp. 284\u2013291.","DOI":"10.1109\/BIOROB.2016.7523639"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1167\/iovs.11-8486"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1136\/bjo.2010.193391"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0162037"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2020.3045875"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2021.3071135"},{"key":"e_1_3_2_40_2","doi-asserted-by":"crossref","unstructured":"P. Zhang P. Gehlbach M. Kobilarov I. Iordachita \u201cA feasible workflow for retinal vein cannulation in ex vivo porcine eyes with robotic assistance \u201d in 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (IEEE 2024) pp. 1\u20135.","DOI":"10.1109\/EMBC53108.2024.10781906"},{"key":"e_1_3_2_41_2","doi-asserted-by":"crossref","unstructured":"K. He X. Zhang S. Ren J. Sun \u201cDeep residual learning for image recognition \u201d in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE 2016) pp. 770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_42_2","unstructured":"J. W. Kim P. Zhang P. Gehlbach I. Iordachita M. Kobilarov \u201cTowards autonomous eye surgery by combining deep imitation learning with optimal control \u201d in Conference on Robot Learning (PMLR 2021) pp. 2347\u20132358."},{"key":"e_1_3_2_43_2","unstructured":"G. Jocher A. Chaurasia J. Qiu YOLO by Ultralytics (2023); https:\/\/github.com\/ultralytics\/ultralytics."},{"key":"e_1_3_2_44_2","unstructured":"B. Dwyer J. Nelson T. Hansen Roboflow (Version 1.0) [Software] (2024); https:\/\/roboflow.com."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-018-1829-0"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1167\/tvst.9.5.3"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1167\/iovs.10-6738"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1142\/S2424905X2350006X"},{"key":"e_1_3_2_49_2","doi-asserted-by":"crossref","unstructured":"J. Shi \u201cGood features to track \u201d in 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE 1994) pp. 593\u2013600.","DOI":"10.1109\/CVPR.1994.323794"},{"key":"e_1_3_2_50_2","unstructured":"B. D. Lucas T. Kanade \u201cAn iterative image registration technique with an application to stereo vision \u201d in IJCAI\u201981: 7th International Joint Conference on Artificial Intelligence (ACM 1981) vol. 2 pp. 674\u2013679."},{"key":"e_1_3_2_51_2","unstructured":"A. Dosovitskiy L. Beyer A. Kolesnikov D. Weissenborn X. Zhai T. Unterthiner M. Dehghani M. Minderer G. Heigold S. Gelly J. Uszkoreit N. Houlsby An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929 [cs.CV] (2020)."},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","unstructured":"S. Woo S. Debnath R. Hu X. Chen Z. Liu I. S. Kweon S. Xie \u201cConvnext v2: Co-designing and scaling convnets with masked autoencoders \u201d in Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (IEEE 2023) pp. 16133\u201316142.","DOI":"10.1109\/CVPR52729.2023.01548"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.adp7700"},{"key":"e_1_3_2_54_2","doi-asserted-by":"crossref","unstructured":"X. He D. Roppenecker D. Gierlach M. Balicki K. Olds P. Gehlbach J. Handa R. Taylor I. Iordachita \u201cToward clinically applicable steady-hand eye robot for vitreoretinal surgery \u201d in ASME International Mechanical Engineering Congress and Exposition (American Society of Mechanical Engineers 2012) vol. 45189 pp. 145\u2013153.","DOI":"10.1115\/IMECE2012-88384"},{"key":"e_1_3_2_55_2","unstructured":"Supervisely \u201cAll computer vision in one platform\u201d; https:\/\/supervisely.com."},{"key":"e_1_3_2_56_2","unstructured":"D. P. Kingma J. Ba Adam: A method for stochastic optimization. arXiv:1412.6980 [cs.LG] (2014)."},{"key":"e_1_3_2_57_2","doi-asserted-by":"crossref","unstructured":"R. M. Murray Z. Li S. S. Sastry A Mathematical Introduction to Robotic Manipulation (CRC Press 2017).","DOI":"10.1201\/9781315136370"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1177\/027836499101000409"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0161-6420(01)00875-2"},{"key":"e_1_3_2_60_2","doi-asserted-by":"crossref","unstructured":"M. Sohan T. Sai Ram R. Reddy C. Venkata \u201cA review on yolov8 and its advancements \u201d in International Conference on Data Intelligence and Cognitive Informatics (Springer 2024) pp. 529\u2013545.","DOI":"10.1007\/978-981-99-7962-2_39"}],"container-title":["Science Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.science.org\/doi\/pdf\/10.1126\/scirobotics.adw2969","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,17]],"date-time":"2025-12-17T18:58:27Z","timestamp":1765997907000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.science.org\/doi\/10.1126\/scirobotics.adw2969"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,17]]},"references-count":59,"journal-issue":{"issue":"109","published-print":{"date-parts":[[2025,12,17]]}},"alternative-id":["10.1126\/scirobotics.adw2969"],"URL":"https:\/\/doi.org\/10.1126\/scirobotics.adw2969","relation":{},"ISSN":["2470-9476"],"issn-type":[{"value":"2470-9476","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,17]]},"assertion":[{"value":"2025-01-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-18","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"eadw2969"}}