{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T16:55:25Z","timestamp":1774716925904,"version":"3.50.1"},"reference-count":36,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2023,11,8]],"date-time":"2023-11-08T00:00:00Z","timestamp":1699401600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62373248"],"award-info":[{"award-number":["62373248"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62003209"],"award-info":[{"award-number":["62003209"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100013105","name":"Shanghai Rising-Star Program","doi-asserted-by":"publisher","award":["22QC1401400"],"award-info":[{"award-number":["22QC1401400"]}],"id":[{"id":"10.13039\/501100013105","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003399","name":"Science and Technology Commission of Shanghai Municipality","doi-asserted-by":"publisher","award":["21511102100"],"award-info":[{"award-number":["21511102100"]}],"id":[{"id":"10.13039\/501100003399","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003399","name":"Science and Technology Commission of Shanghai Municipality","doi-asserted-by":"publisher","award":["21DZ1100300"],"award-info":[{"award-number":["21DZ1100300"]}],"id":[{"id":"10.13039\/501100003399","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003399","name":"Science and Technology Commission of Shanghai Municipality","doi-asserted-by":"publisher","award":["20DZ2220400"],"award-info":[{"award-number":["20DZ2220400"]}],"id":[{"id":"10.13039\/501100003399","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["advanced.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Advanced Intelligent Systems"],"published-print":{"date-parts":[[2024,1]]},"abstract":"<jats:p>Robotic telemedicine can provide timely treatment to critical patients in geographically remote locations. However, owing to the lack of depth information, occlusion of instrument, and view direction limitations, visual feedback from the patient to the clinician is typically unintuitive, which affects surgical safety. Herein, an omnidirectional augmented reality (AR)\u2010assisted robotic telepresence for interventional medicine is developed. A monocular camera is used as an AR device with a manipulator, where virtual key anatomies and instruments are superimposed on video images using multiobject hand\u2013eye calibration, iterative closest point, and images superposition algorithms. The view direction of the camera can be changed via the manipulator, which allows the key anatomies and instruments to be viewed from different directions. Structure\u2010from\u2010motion and multi\u2010view stereo algorithms are used to reconstruct the scene on the patient's side, thus providing a virtual reality (VR) interactive interface for the manipulator's safe teleoperation. Two different phantom experiments are conducted to validate the effectiveness of the proposed method. Finally, an ex\u2009vivo experiment involving a porcine lumen is performed, in which the operator can view the interventional flexible robot. The proposed method integrates omnidirectional AR and VR with robotic telepresence, which can provide intuitive visual feedback to clinicians.<\/jats:p>","DOI":"10.1002\/aisy.202300373","type":"journal-article","created":{"date-parts":[[2023,11,8]],"date-time":"2023-11-08T15:44:41Z","timestamp":1699458281000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Robotic Manipulator\u2010Assisted Omnidirectional Augmented Reality for Endoluminal Intervention Telepresence"],"prefix":"10.1002","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6043-2903","authenticated-orcid":false,"given":"Zecai","family":"Lin","sequence":"first","affiliation":[{"name":"The Institute of Medical Robotics and Department of Automation and the Key Laboratory of System Control and Information Processing of Ministry of Education Shanghai Jiao Tong University  Shanghai 200240 P. R. China"}]},{"given":"Zheng","family":"Xu","sequence":"additional","affiliation":[{"name":"The Institute of Medical Robotics and Department of Automation and the Key Laboratory of System Control and Information Processing of Ministry of Education Shanghai Jiao Tong University  Shanghai 200240 P. R. China"}]},{"given":"Huanghua","family":"Liu","sequence":"additional","affiliation":[{"name":"The Institute of Medical Robotics and Department of Automation and the Key Laboratory of System Control and Information Processing of Ministry of Education Shanghai Jiao Tong University  Shanghai 200240 P. R. China"}]},{"given":"Xuyang","family":"Wang","sequence":"additional","affiliation":[{"name":"The Institute of Medical Robotics and Department of Automation and the Key Laboratory of System Control and Information Processing of Ministry of Education Shanghai Jiao Tong University  Shanghai 200240 P. R. China"}]},{"given":"Xiaojie","family":"Ai","sequence":"additional","affiliation":[{"name":"The Institute of Medical Robotics and Department of Automation and the Key Laboratory of System Control and Information Processing of Ministry of Education Shanghai Jiao Tong University  Shanghai 200240 P. R. China"}]},{"given":"Cheng","family":"Zhou","sequence":"additional","affiliation":[{"name":"The Institute of Medical Robotics and Department of Automation and the Key Laboratory of System Control and Information Processing of Ministry of Education Shanghai Jiao Tong University  Shanghai 200240 P. R. China"}]},{"given":"Bidan","family":"Huang","sequence":"additional","affiliation":[{"name":"Tencent Robotics X Tencent  Shenzhen 518000 P. R. China"}]},{"given":"Weidong","family":"Chen","sequence":"additional","affiliation":[{"name":"The Institute of Medical Robotics and Department of Automation and the Key Laboratory of System Control and Information Processing of Ministry of Education Shanghai Jiao Tong University  Shanghai 200240 P. R. China"}]},{"given":"Guang-Zhong","family":"Yang","sequence":"additional","affiliation":[{"name":"The Institute of Medical Robotics Shanghai Jiao Tong University  Shanghai 200240 P. R. China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8307-4416","authenticated-orcid":false,"given":"Anzhu","family":"Gao","sequence":"additional","affiliation":[{"name":"The Institute of Medical Robotics and Department of Automation and the Key Laboratory of System Control and Information Processing of Ministry of Education Shanghai Jiao Tong University  Shanghai 200240 P. R. China"}]}],"member":"311","published-online":{"date-parts":[[2023,11,8]]},"reference":[{"key":"e_1_2_9_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amjsurg.2019.07.018"},{"key":"e_1_2_9_3_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-021-00324-z"},{"key":"e_1_2_9_4_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abf1462"},{"key":"e_1_2_9_5_1","unstructured":"J. P.Whitney T.Chen J.Mars J. K.Hodgins inProc. IEEE Int. Conf. Robotic Automation IEEE Piscataway NJ2016 pp.690\u2013695."},{"key":"e_1_2_9_6_1","doi-asserted-by":"publisher","DOI":"10.3390\/bios12110943"},{"key":"e_1_2_9_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2017.2676351"},{"key":"e_1_2_9_8_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.201900138"},{"key":"e_1_2_9_9_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364909101795"},{"key":"e_1_2_9_10_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202000071"},{"key":"e_1_2_9_11_1","doi-asserted-by":"publisher","DOI":"10.1097\/00000658-200204000-00005"},{"key":"e_1_2_9_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMRB.2020.3033007"},{"key":"e_1_2_9_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-015-1289-8"},{"key":"e_1_2_9_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2021.3105536"},{"key":"e_1_2_9_15_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202100228"},{"key":"e_1_2_9_16_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-31893-1"},{"key":"e_1_2_9_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-018-1814-7"},{"key":"e_1_2_9_18_1","doi-asserted-by":"publisher","DOI":"10.1093\/ons\/opz236"},{"key":"e_1_2_9_19_1","doi-asserted-by":"publisher","DOI":"10.1177\/1553350612446353"},{"key":"e_1_2_9_20_1","unstructured":"F.Richter Y.Zhang Y.Zhi R. K.Orosco M. C.Yip inProc. IEEE Int. Conf. Robotics Automation IEEE Piscataway NJ2019 pp.444\u2013450."},{"key":"e_1_2_9_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11548-021-02463-5"},{"key":"e_1_2_9_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMRB.2022.3201652"},{"key":"e_1_2_9_23_1","unstructured":"S.Rusinkiewicz M.Levoy inProc. Int. Conf. 3D Digital Imaging Modeling IEEE Piscataway NJ2001 pp.145\u2013152."},{"key":"e_1_2_9_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/70.34770"},{"key":"e_1_2_9_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.41"},{"key":"e_1_2_9_26_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJMA.2011.045271"},{"key":"e_1_2_9_27_1","unstructured":"J. L.Sch\u00f6nberger J.-M.Frahm inProc. IEEE Conf. Computer Vision Pattern Recognition IEEE Piscataway NJ2016 pp.4104\u20134113."},{"key":"e_1_2_9_28_1","doi-asserted-by":"crossref","unstructured":"J. L.Sch\u00f6nberger E.Zheng M.Pollefeys J.-M.Frahm inProc. European Conf. Computer Vision Springer Cham2016 pp.501\u2013518.","DOI":"10.1007\/978-3-319-46487-9_31"},{"key":"e_1_2_9_29_1","doi-asserted-by":"publisher","DOI":"10.1155\/2014\/798595"},{"key":"e_1_2_9_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2011.172"},{"key":"e_1_2_9_31_1","doi-asserted-by":"crossref","unstructured":"M.Waechter N.Moehrle M.Goesele inProc. European Conf. Computer Vision Springer Cham2014 pp.836\u2013850.","DOI":"10.1007\/978-3-319-10602-1_54"},{"key":"e_1_2_9_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.88573"},{"key":"e_1_2_9_33_1","unstructured":"D.Zhang Z.Wu J.Chen R.Zhu A.Munawar B.Xiao Y.Guan H.Su W.Hong Y.Guo G. S.Fischer B.Lo G.-Z.Yang inProc. IEEE Int. Conf. on Robotics and Automation IEEE Piscataway NJ2022 pp.7694\u20137700."},{"key":"e_1_2_9_34_1","doi-asserted-by":"publisher","DOI":"10.1615\/CritRevBiomedEng.v40.i2.40"},{"key":"e_1_2_9_35_1","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2020.594673"},{"key":"e_1_2_9_36_1","doi-asserted-by":"crossref","unstructured":"Z.Lin H.Wu H.Jia H.Liu X.Ai Y.Zou Z.Sun W.Chen G.-Z.Yang A.Gao inProc. IEEE Int. Conf. on Robotics and Automation IEEE Piscataway NJ2022 pp.9593\u20139599.","DOI":"10.1109\/ICRA46639.2022.9811947"},{"key":"e_1_2_9_37_1","doi-asserted-by":"publisher","DOI":"10.1089\/soro.2019.0051"}],"container-title":["Advanced Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aisy.202300373","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,7]],"date-time":"2025-10-07T23:12:52Z","timestamp":1759878772000},"score":1,"resource":{"primary":{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202300373"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,8]]},"references-count":36,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1]]}},"alternative-id":["10.1002\/aisy.202300373"],"URL":"https:\/\/doi.org\/10.1002\/aisy.202300373","archive":["Portico"],"relation":{},"ISSN":["2640-4567","2640-4567"],"issn-type":[{"value":"2640-4567","type":"print"},{"value":"2640-4567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,8]]},"assertion":[{"value":"2023-07-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"2300373"}}