{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T17:57:53Z","timestamp":1791914273799,"version":"4.3.4"},"reference-count":24,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,6,5]],"date-time":"2023-06-05T00:00:00Z","timestamp":1685923200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62076070"],"award-info":[{"award-number":["62076070"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurosci."],"abstract":"<jats:p>Robot-assisted minimally invasive surgery (RAMIS) has gained significant traction in clinical practice in recent years. However, most surgical robots rely on touch-based human-robot interaction (HRI), which increases the risk of bacterial diffusion. This risk is particularly concerning when surgeons must operate various equipment with their bare hands, necessitating repeated sterilization. Thus, achieving touch-free and precise manipulation with a surgical robot is challenging. To address this challenge, we propose a novel HRI interface based on gesture recognition, leveraging hand-keypoint regression and hand-shape reconstruction methods. By encoding the 21 keypoints from the recognized hand gesture, the robot can successfully perform the corresponding action according to predefined rules, which enables the robot to perform fine-tuning of surgical instruments without the need for physical contact with the surgeon. We evaluated the surgical applicability of the proposed system through both phantom and cadaver studies. In the phantom experiment, the average needle tip location error was 0.51\u2009 mm, and the mean angle error was 0.34 degrees. In the simulated nasopharyngeal carcinoma biopsy experiment, the needle insertion error was 0.16\u2009 mm, and the angle error was 0.10 degrees. These results indicate that the proposed system achieves clinically acceptable accuracy and can assist surgeons in performing contactless surgery with hand gesture interaction.<\/jats:p>","DOI":"10.3389\/fnins.2023.1200576","type":"journal-article","created":{"date-parts":[[2023,6,5]],"date-time":"2023-06-05T04:28:34Z","timestamp":1685939314000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":14,"title":["A touch-free human-robot collaborative surgical navigation robotic system based on hand gesture recognition"],"prefix":"10.3389","volume":"17","author":[{"given":"Jie","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinkang","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinrong","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhijian","family":"Song","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,6,5]]},"reference":[{"key":"ref1","first-page":"1","article-title":"3D gesture-based control system using processing open source software","author":"Ahmad","year":"2016"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"2025","DOI":"10.1109\/TBME.2016.2647680","article-title":"A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery","volume":"64","author":"Ahmidi","year":"2017","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref01","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TMECH.2022.3224314","article-title":"Efficient Grasp Detection Network With Gaussian-Based Grasp Representation for Robotic Manipulation","author":"Cao","year":"2022","journal-title":"IEEE\/ASME Trans. Mechatron.: IEEE"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"2140009","DOI":"10.1142\/S0219519421400091","article-title":"Robust surface-matching registration based on the structure information for image-guided neurosurgery system","volume":"21","author":"Chen","year":"2021","journal-title":"J. Mech. Med. Biol."},{"key":"ref4","doi-asserted-by":"publisher","first-page":"48","DOI":"10.5772\/60044","article-title":"Hand gesture modeling and recognition for human and robot interactive assembly using hidden markov models","volume":"12","author":"Chen","year":"2015","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref5","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1016\/j.cmpb.2018.04.003","article-title":"Enhancement of gesture recognition for contactless interface using a personalized classifier in the operating room","volume":"161","author":"Cho","year":"2018","journal-title":"Comput. Methods Prog. Biomed."},{"key":"ref6","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1007\/s11548-017-1666-6","article-title":"Evaluation of contactless human\u2013machine interface for robotic surgical training","volume":"13","author":"Despinoy","year":"2018","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1016\/j.cmpb.2013.10.010","article-title":"A haptic pedal for surgery assistance","volume":"116","author":"D\u00edaz","year":"2014","journal-title":"Comput. Methods Prog. Biomed."},{"key":"ref8","first-page":"66","article-title":"Arabic sign language recognition with 3D convolutional neural networks","author":"Elbadawy","year":"2017"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"28230","DOI":"10.1109\/ACCESS.2019.2901930","article-title":"Gesture recognition based on CNN and DCGAN for calculation and text output","volume":"7","author":"Fang","year":"2019","journal-title":"IEEE Access"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"835","DOI":"10.1109\/JPROC.2022.3180350","article-title":"Robot-assisted minimally invasive surgery\u2014surgical robotics in the data age","volume":"110","author":"Haidegger","year":"2022","journal-title":"Proc. IEEE"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"6031","DOI":"10.1016\/j.eswa.2010.11.016","article-title":"Gabor filter-based hand-pose angle estimation for hand gesture recognition under varying illumination","volume":"38","author":"Huang","year":"2011","journal-title":"Expert Syst. Appl."},{"key":"ref13","first-page":"1","article-title":"Toward gesture recognition in robot-assisted surgical procedures","author":"Huynhnguyen","year":"2021"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1016\/j.trit.2016.03.010","article-title":"Multi-LeapMotion sensor based demonstration for robotic refine tabletop object manipulation task","volume":"1","author":"Jin","year":"2016","journal-title":"CAAI Trans. Intell. Technol."},{"key":"ref15","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1080\/24699322.2017.1389411","article-title":"A new robust markerless method for automatic image-to-patient registration in image-guided neurosurgery system","volume":"22","author":"Liu","year":"2017","journal-title":"Comput. Assist. Surg."},{"key":"ref16","first-page":"1558","article-title":"End-to-end fine-grained action segmentation and recognition using conditional random field models and discriminative sparse coding","author":"Mavroudi","year":"2018"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"141","DOI":"10.12700\/APH.16.8.2019.8.9","article-title":"Robot-assisted minimally invasive surgical skill assessment\u2014manual and automated platforms","volume":"16","author":"Nagyn\u00e9 Elek","year":"2019","journal-title":"Acta Polytech. Hungarica"},{"key":"ref18","first-page":"229","article-title":"Application of natural user interface devices for touch-free control of radiological images during surgery","author":"Nestorov","year":"2016"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1159\/000485524","article-title":"Feasibility and usefulness of a joystick-guided robotic scope holder (Soloassist) in laparoscopic surgery","volume":"34","author":"Ohmura","year":"2018","journal-title":"Visc. Med."},{"key":"ref20","doi-asserted-by":"publisher","first-page":"3941","DOI":"10.1007\/s00521-016-2294-8","article-title":"Deep learning in vision-based static hand gesture recognition","volume":"28","author":"Oyedotun","year":"2017","journal-title":"Neural Comput. Applic."},{"key":"ref21","first-page":"396","article-title":"Automatic recognition of surgical motions using statistical modeling for capturing variability","volume":"132","author":"Reiley","year":"2008","journal-title":"Stud. Health Technol. Inform."},{"key":"ref22","first-page":"9453","article-title":"Domain adaptive robotic gesture recognition with unsupervised kinematic-visual data alignment","author":"Shi","year":"2021"},{"key":"ref23","first-page":"1929","article-title":"Hand gesture recognition system for touch-less car interface using multiclass support vector machine","author":"Tarvekar","year":"2018"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"2021","DOI":"10.1109\/TBME.2021.3054828","article-title":"Gesture recognition in robotic surgery: a review","volume":"68","author":"Van Amsterdam","year":"2021","journal-title":"IEEE Trans. Biomed. Eng."}],"container-title":["Frontiers in Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2023.1200576\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,5]],"date-time":"2023-06-05T04:28:36Z","timestamp":1685939316000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2023.1200576\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,5]]},"references-count":24,"alternative-id":["10.3389\/fnins.2023.1200576"],"URL":"https:\/\/doi.org\/10.3389\/fnins.2023.1200576","relation":{},"ISSN":["1662-453X"],"issn-type":[{"value":"1662-453X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,5]]},"article-number":"1200576"}}