{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T22:10:46Z","timestamp":1708726246866},"reference-count":31,"publisher":"MIT Press","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Traditionally, teleoperation means that the system sends a series of signal commands from the master while the slave manipulator receives and realizes the desired control operations. For the purpose of implementing more dexterous and complex tasks, we propose a novel framework with dual-hand master teleoperation systems under time-varied delays. In this paper, we emphasize studying the bilateral grasping teleoperation control, as the time delay causes a communication outage. Combining a wave-variable structure with a four-channel framework, an event-trigger-based bilateral sliding mode teleoperation control and an adaptive neural network are designed to effectively achieve master-slave trajectory tracking. In the virtual 3D environment, we created a mixed-reality interface based on dual-hand master teleoperation control that effectively responded to the two Omni manipulators' position transformation of the virtual manipulator. The time delay between the real slave force feedback and the virtual interface is addressed by designed event-trigger-based control in order to efficiently reduce the impact of time communication outage. The system's stability is analyzed and robot experiments are performed. From the experimental results, the telepresence platform innovatively applied virtual force feedback to reveal the soft target grasping and to accurately estimate the interactive force, enabling sensorless force feedback control.<\/jats:p>","DOI":"10.1162\/pres_a_00402","type":"journal-article","created":{"date-parts":[[2024,1,4]],"date-time":"2024-01-04T19:45:36Z","timestamp":1704397536000},"page":"81-98","update-policy":"http:\/\/dx.doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":0,"title":["Mixed-Reality-Based Teleoperation Grasping Control"],"prefix":"10.1162","volume":"32","author":[{"given":"Dekun","family":"Zheng","sequence":"first","affiliation":[{"name":"College of Electrical Engineering and Control Science Nanjing Tech University Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ting","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Electrical Engineering and Control Science Nanjing Tech University 30, Puzhu South Road 211816, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gao","family":"Jian","sequence":"additional","affiliation":[{"name":"School of Marine Science and Technology Northwestern Polytechnical University 127, Youyi West Road 710072, Xian, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liang","family":"Li","sequence":"additional","affiliation":[{"name":"College of Electrical Engineering and Control Science Nanjing Tech University Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiangjun","family":"Ji","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery Jinling Hospital, 305, East Zhongshan Road Nanjing, 210002, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kurosh","family":"Madani","sequence":"additional","affiliation":[{"name":"Signals, Images, and Intelligent Systems Laboratory (LISSI\/EA 3956) Paris Est University Lieusaint, 77127, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2023,12,1]]},"reference":[{"key":"2024022321251224300_B1","doi-asserted-by":"publisher","first-page":"5757","DOI":"10.1016\/j.amc.2011.11.080","article-title":"A chattering-free robust adaptive sliding mode controller for synchronization of two different chaotic systems with unknown uncertainties and external disturbances","volume":"218","author":"Aghababa","year":"2012","journal-title":"Applied Mathematics and Computation"},{"key":"2024022321251224300_B2","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1109\/TCYB.2018.2876520","article-title":"Bilateral teleoperation of single-master multislave systems with semi-Markovian jump stochastic interval time-varying delayed communication channels","volume":"51","author":"Baranitha","year":"2019","journal-title":"IEEE Transactions on Cybernetics"},{"key":"2024022321251224300_B3","doi-asserted-by":"publisher","first-page":"2501","DOI":"10.1109\/LRA.2018.2812907","article-title":"Inverse kinematics control methods for redundant snakelike robot teleoperation during minimally invasive surgery","volume":"3","author":"Berthet-Rayne","year":"2018","journal-title":"IEEE Robotics and Automation Letters"},{"key":"2024022321251224300_B4","doi-asserted-by":"publisher","first-page":"863","DOI":"10.1109\/TLA.2016.7437233","article-title":"An augmented reality application to assist teleoperation of underwater manipulators","volume":"14","author":"C\u00e1rdenas","year":"2016","journal-title":"IEEE Latin America Transactions"},{"key":"2024022321251224300_B5","doi-asserted-by":"publisher","first-page":"12848","DOI":"10.1109\/ACCESS.2018.2805782","article-title":"An improved wave-variable based four-channel control design in bilateral teleoperation system for time-delay compensation","volume":"6","author":"Chen","year":"2018","journal-title":"IEEE Access"},{"key":"2024022321251224300_B6","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1109\/TRO.2017.2769123","article-title":"Direct force-reflecting two-layer approach for passive bilateral teleoperation with time delays","volume":"34","author":"Heck","year":"2018","journal-title":"IEEE Transactions on Robotics"},{"key":"2024022321251224300_B7","doi-asserted-by":"publisher","first-page":"2923","DOI":"10.1109\/TAC.2016.2603780","article-title":"On exploring the domain of attraction for bilateral teleoperator subject to interval delay and saturated P + d control scheme","volume":"62","author":"Hua","year":"2017","journal-title":"IEEE Transactions on Automatic Control"},{"key":"2024022321251224300_B8","doi-asserted-by":"publisher","first-page":"3052","DOI":"10.1109\/TCYB.2018.2838573","article-title":"Adaptive fuzzy control for coordinated multiple robots with constraint using impedance learning","volume":"49","author":"Kong","year":"2019","journal-title":"IEEE Transactions on Cybernetics"},{"key":"2024022321251224300_B9","doi-asserted-by":"publisher","first-page":"978","DOI":"10.1109\/TRO.2010.2082430","article-title":"Passive decomposition and control of nonholonomic mechanical systems","volume":"26","author":"Lee","year":"2010","journal-title":"IEEE Transactions on Robotics"},{"key":"2024022321251224300_B10","doi-asserted-by":"publisher","first-page":"2315","DOI":"10.1109\/JIOT.2017.2737479","article-title":"Motor anomaly detection for unmanned aerial vehicles using reinforcement learning","volume":"5","author":"Lu","year":"2018","journal-title":"IEEE Internet of Things Journal"},{"key":"2024022321251224300_B11","doi-asserted-by":"publisher","first-page":"2439","DOI":"10.1109\/TIE.2017.2745445","article-title":"Predictive approach for sensorless bimanual teleoperation under random time delays with adaptive fuzzy control","volume":"65","author":"Lu","year":"2018","journal-title":"IEEE Transactions on Industrial Electronics"},{"key":"2024022321251224300_B12","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1109\/tmrb.2022.3146621","article-title":"GuLiM: A hybrid motion mapping technique for teleoperation of medical assistive robot in combating the COVID-19 pandemic","volume":"4","author":"Lv","year":"2022","journal-title":"IEEE Transactions on Medical Robotics and Bionics"},{"key":"2024022321251224300_B13","doi-asserted-by":"publisher","first-page":"60559","DOI":"10.1109\/ACCESS.2018.2876040","article-title":"Robotics for seabed teleoperation: Part-1, conception and practical implementation of a hybrid seabed robot","volume":"6","author":"Saltar\u00e9n","year":"2018","journal-title":"IEEE Access"},{"key":"2024022321251224300_B14","doi-asserted-by":"publisher","DOI":"10.1016\/j.asr.2022.07.013","article-title":"Challenges in threat modelling of new space systems: A teleoperation use-case","author":"Sheik","year":"2022","journal-title":"Advances in Space Research"},{"key":"2024022321251224300_B15","doi-asserted-by":"publisher","first-page":"1777","DOI":"10.1109\/TLA.2020.9387669","article-title":"Dual coordination for bilateral teleoperation of a mobile robot with time varying delay","volume":"18","author":"Slawi\u00f1ski","year":"2020","journal-title":"IEEE Latin America Transactions"},{"key":"2024022321251224300_B16","doi-asserted-by":"publisher","first-page":"1666","DOI":"10.1109\/TCYB.2016.2554630","article-title":"Neural network-based passivity control of teleoperation system under time-varying delays","volume":"47","author":"Sun","year":"2017","journal-title":"IEEE Transactions on Cybernetics"},{"key":"2024022321251224300_B17","doi-asserted-by":"publisher","first-page":"2785","DOI":"10.1109\/tfuzz.2020.3007438","article-title":"Event-triggered prescribed-time fuzzy control for space teleoperation systems subject to multiple constraints and uncertainties","volume":"29","author":"Wang","year":"2021","journal-title":"IEEE Transactions on Fuzzy Systems"},{"key":"2024022321251224300_B18","doi-asserted-by":"publisher","first-page":"1714","DOI":"10.1109\/TBME.2020.3046357","article-title":"Teleoperation of an ankle-foot prosthesis with a wrist exoskeleton","volume":"68","author":"Welker","year":"2020","journal-title":"IEEE Transactions on Biomedical Engineering"},{"key":"2024022321251224300_B19","doi-asserted-by":"publisher","DOI":"10.1177\/1729881417746024","article-title":"Kalman smoothing with soft inequality constraints for space robot teleoperation","volume":"15","author":"Wu","year":"2018","journal-title":"International Journal of Advanced Robotic Systems"},{"key":"2024022321251224300_B20","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1109\/TFUZZ.2018.2864940","article-title":"Finite-time convergence adaptive fuzzy control for dual-arm robot with unknown kinematics and dynamics","volume":"27","author":"Yang","year":"2019","journal-title":"IEEE Transactions on Fuzzy Systems"},{"key":"2024022321251224300_B21","doi-asserted-by":"publisher","first-page":"1759","DOI":"10.1109\/TSMC.2017.2694020","article-title":"Personalized variable gain control with tremor attenuation for robot teleoperation","volume":"48","author":"Yang","year":"2018","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics: Systems"},{"key":"2024022321251224300_B22","doi-asserted-by":"publisher","first-page":"2514","DOI":"10.1109\/TCYB.2018.2826016","article-title":"Adaptive formation control of cooperative teleoperators with intermittent communications","volume":"49","author":"Yang","year":"2019","journal-title":"IEEE Transactions on Cybernetics"},{"key":"2024022321251224300_B23","doi-asserted-by":"publisher","first-page":"2814","DOI":"10.1016\/j.jfranklin.2016.04.009","article-title":"Adaptive fuzzy synchronization control for networked teleoperation system with input and multi-state constraints","volume":"353","author":"Yang","year":"2016","journal-title":"Journal of the Franklin Institute"},{"key":"2024022321251224300_B24","doi-asserted-by":"publisher","first-page":"324","DOI":"10.1109\/AMC.2008.4516087","article-title":"L2 stable four-channel control architecture for bilateral teleoperation with time delay","author":"Yashiro","year":"2008","journal-title":"10th IEEE International Workshop on Advanced Motion Control"},{"key":"2024022321251224300_B25","doi-asserted-by":"publisher","first-page":"1717","DOI":"10.1016\/J.NET.2017.09.006","article-title":"Design considerations for teleoperation systems operating in gas-tight argon cells","volume":"49","author":"Yu","year":"2017","journal-title":"Nuclear Engineering and Technology"},{"key":"2024022321251224300_B26","doi-asserted-by":"publisher","first-page":"1162","DOI":"10.1109\/TII.2018.2822670","article-title":"Force reflecting control for bilateral teleoperation system under time-varying delays","volume":"15","author":"Yuan","year":"2019","journal-title":"IEEE Transactions on Industrial Informatics"},{"key":"2024022321251224300_B27","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1109\/TCYB.2018.2863689","article-title":"Resilient control of wireless networked control system under denial-of-service attacks: A cross-layer design approach","volume":"50","author":"Yuan","year":"2020","journal-title":"IEEE Transactions on Cybernetics"},{"key":"2024022321251224300_B28","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1109\/TFUZZ.2015.2426215","article-title":"Adaptive fuzzy control of multilateral asymmetric teleoperation for coordinated multiple mobile manipulators","volume":"24","author":"Zhai","year":"2016","journal-title":"IEEE Transactions on Fuzzy Systems"},{"key":"2024022321251224300_B29","doi-asserted-by":"publisher","first-page":"3724","DOI":"10.1109\/TCYB.2019.2924446","article-title":"Novel adaptive finite-time control of teleoperation system with time-varying delays and input saturation","volume":"51","author":"Zhang","year":"2019","journal-title":"IEEE Transactions on Cybernetics"},{"key":"2024022321251224300_B30","doi-asserted-by":"publisher","first-page":"1716","DOI":"10.1109\/TIE.2016.2557309","article-title":"Development of a virtual platform for telepresence control of an underwater manipulator mounted on a submersible vehicle","volume":"64","author":"Zhang","year":"2017","journal-title":"IEEE Transactions on Industrial Electronics"},{"key":"2024022321251224300_B31","doi-asserted-by":"publisher","first-page":"3088","DOI":"10.1109\/TPAMI.2019.2920899","article-title":"Reconstruct and represent video contents for captioning via reinforcement learning","volume":"42","author":"Zhang","year":"2019","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"}],"container-title":["PRESENCE: Virtual and Augmented Reality"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/pvar\/article-pdf\/doi\/10.1162\/pres_a_00402\/2339579\/pres_a_00402.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/pvar\/article-pdf\/doi\/10.1162\/pres_a_00402\/2339579\/pres_a_00402.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T21:25:25Z","timestamp":1708723525000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/pvar\/article\/doi\/10.1162\/pres_a_00402\/118941\/Mixed-Reality-Based-Teleoperation-Grasping-Control"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":31,"URL":"https:\/\/doi.org\/10.1162\/pres_a_00402","relation":{},"ISSN":["1531-3263"],"issn-type":[{"value":"1531-3263","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}