{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T23:15:15Z","timestamp":1782774915240,"version":"3.54.5"},"reference-count":30,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,2,14]],"date-time":"2025-02-14T00:00:00Z","timestamp":1739491200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Research Foundation of Korea (NRF) funded by the Ministry of Education","award":["RS-2022-NR073933"],"award-info":[{"award-number":["RS-2022-NR073933"]}]},{"name":"National Research Foundation of Korea (NRF) funded by the Ministry of Education","award":["2022-0-00218"],"award-info":[{"award-number":["2022-0-00218"]}]},{"name":"Institute of Information &amp; Communications Technology Planning &amp; Evaluation (IITP) grant funded by the Korean government (MSIT)","award":["RS-2022-NR073933"],"award-info":[{"award-number":["RS-2022-NR073933"]}]},{"name":"Institute of Information &amp; Communications Technology Planning &amp; Evaluation (IITP) grant funded by the Korean government (MSIT)","award":["2022-0-00218"],"award-info":[{"award-number":["2022-0-00218"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Telemanipulation-based object-side picking with a suction gripper often faces challenges such as occlusion of the target object or the gripper and the need for precise alignment between the suction cup and the object\u2019s surface. These issues can significantly affect task success rates in logistics environments. To address these problems, this study proposes a multi-mode hand gesture-based virtual reality (VR) locomotion method to enable intuitive and precise viewpoint control. The system utilizes a head-mounted display (HMD) camera to capture hand skeleton data, which a multi-layer perceptron (MLP) model processes. The model classifies gestures into three modes: translation, rotation, and fixed, corresponding to fist, pointing, and unknown gestures, respectively. Translation mode moves the viewpoint based on the wrist\u2019s displacement, rotation mode adjusts the viewpoint\u2019s angle based on the wrist\u2019s angular displacement, and fixed mode stabilizes the viewpoint when gestures are ambiguous. A dataset of 4312 frames was used for training and validation, with 666 frames for testing. The MLP model achieved a classification accuracy of 98.4%, with precision, recall, and F1-score exceeding 0.98. These results demonstrate the system\u2019s ability to address the challenges of telemanipulation tasks by enabling accurate gesture recognition and seamless mode transitions.<\/jats:p>","DOI":"10.3390\/s25041181","type":"journal-article","created":{"date-parts":[[2025,2,14]],"date-time":"2025-02-14T11:14:41Z","timestamp":1739531681000},"page":"1181","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Multi-Mode Hand Gesture-Based VR Locomotion Technique for Intuitive Telemanipulation Viewpoint Control in Tightly Arranged Logistic Environments"],"prefix":"10.3390","volume":"25","author":[{"given":"Jaehoon","family":"Jeong","sequence":"first","affiliation":[{"name":"Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haegyeom","family":"Choi","sequence":"additional","affiliation":[{"name":"Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1814-2683","authenticated-orcid":false,"given":"Donghun","family":"Lee","sequence":"additional","affiliation":[{"name":"Mechanical Engineering Department, Soongsil University, Seoul 06978, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,2,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Iriondo, A., Lazkano, E., Susperregi, L., Urain, J., Fernandez, A., and Molina, J. (2019). Pick and Place Operations in Logistics Using a Mobile Manipulator Controlled with Deep Reinforcement Learning. Appl. Sci., 9.","DOI":"10.3390\/app9020348"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"690","DOI":"10.1177\/0278364919868017","article-title":"Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching","volume":"41","author":"Zeng","year":"2022","journal-title":"Int. J. Robot. Res."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Galarza, B.R., Ayala, P., Manzano, S., and Garcia, M.V. (2023). Virtual reality teleoperation system for mobile robot manipulation. Robotics, 12.","DOI":"10.3390\/robotics12060163"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Krupke, D., Einig, L., Langbehn, E., Zhang, J., and Steinicke, F. (2016, January 2\u20134). Immersive remote grasping: Realtime gripper control by a heterogenous robot control system. Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology, Munich, Germany.","DOI":"10.1145\/2993369.2996345"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Krupke, D., Steinicke, F., Lubos, P., Jonetzko, Y., Gorner, M., and Zhang, J. (2018, January 1\u20135). Comparison of multimodal heading and pointing gestures for co-located mixed reality human-robot interaction. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594043"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Boletsis, C. (2017). The new era of virtual reality locomotion: A systematic literature review of techniques and a proposed typology. Multimodal Technol. Interact., 1.","DOI":"10.3390\/mti1040024"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1016\/j.ijhcs.2018.08.002","article-title":"Locomotion in virtual reality for room scale tracked areas","volume":"122","author":"Bozgeyikli","year":"2019","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Bozgeyikli, E., Raij, A., Katkoori, S., and Dubey, R. (2016, January 16\u201319). Point & teleport locomotion technique for virtual reality. Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play, Austin, TX, USA.","DOI":"10.1145\/2967934.2968105"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Von Willich, J., Schmitz, M., M\u00fcller, F., Schmitt, D., and M\u00fchlh\u00e4user, M. (2020, January 25\u201330). Podoportation: Foot-based locomotion in virtual reality. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3313831.3376626"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Sch\u00e4fer, A., Reis, G., and Stricker, D. (2021). Controlling teleportation-based locomotion in virtual reality with hand gestures: A comparative evaluation of two-handed and one-handed techniques. Electronics, 10.","DOI":"10.3390\/electronics10060715"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Englmeier, D., Fan, F., and Butz, A. (2020, January 9\u201313). Rock or roll\u2013locomotion techniques with a handheld spherical device in virtual reality. Proceedings of the 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Virtual.","DOI":"10.1109\/ISMAR50242.2020.00089"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lee, J., Ahn, S.C., and Hwang, J.-I. (2018). A walking-in-place method for virtual reality using position and orientation tracking. Sensors, 18.","DOI":"10.3390\/s18092832"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chan, L., Mi, T.-W., Hsueh, Z.H., Huang, Y.-C., and Hsu, M.Y. (2024, January 11\u201316). Seated-WIP: Enabling walking-in-place locomotion for stationary chairs in confined spaces. Proceedings of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3613904.3642395"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Wilson, P.T., Kalescky, W., MacLaughlin, A., and Williams, B. (2016, January 3\u20134). VR locomotion: Walking> walking in place> arm swinging. Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry-Volume 1, Zhuhai, China.","DOI":"10.1145\/3013971.3014010"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3724\/SP.J.2096-5796.2018.0007","article-title":"Design of finger gestures for locomotion in virtual reality","volume":"1","author":"Huang","year":"2019","journal-title":"Virtual Real. Intell. Hardw."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kronemberger, V., Cherullo, R., Azevedo, T., Porcino, T., and Raposo, A. (2023, January 6\u20139). Hand gestures for continuous locomotion and snap turn for vr experiences. Proceedings of the 25th Symposium on Virtual and Augmented Reality, Rio Grande, Brazil.","DOI":"10.1145\/3625008.3625029"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhang, F., Chu, S., Pan, R., Ji, N., and Xi, N. (2017, January 24\u201326). Double hand-gesture interaction for walk-through in VR environment. Proceedings of the 2017 IEEE\/ACIS 16th International Conference on Computer and Information Science (ICIS), Wuhan, China.","DOI":"10.1109\/ICIS.2017.7960051"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kirihata, H., and Ishikawa, T. (2024, January 27\u201329). Handwindowteleportation: Locomotion with hand gestures for virtual reality games. Proceedings of the VISIGRAPP (1): GRAPP, HUCAPP, IVAPP, Rome, Italy.","DOI":"10.5220\/0012472900003660"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Makaussov, O., Krassavin, M., Zhabinets, M., and Fazli, S. (2020, January 11\u201314). A low-cost, imu-based real-time on device gesture recognition glove. Proceedings of the 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Toronto, ON, Canada.","DOI":"10.1109\/SMC42975.2020.9283231"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kim, M., Cho, J., Lee, S., and Jung, Y. (2019). Imu sensor-based hand gesture recognition for human-machine interfaces. Sensors, 19.","DOI":"10.3390\/s19183827"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Mummadi, C.K., Leo, F.P.P., Verma, K.D., Kasireddy, S., Scholl, P.M., Kempfle, J., and Laerhoven, K.V. (2018). Real-Time and Embedded Detection of Hand Gestures with an IMU-Based Glove. Informatics, 5.","DOI":"10.3390\/informatics5020028"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jaramillo, A.G., and Benalcazar, M.E. (2017, January 16\u201320). Real-time hand gesture recognition with EMG using machine learning. Proceedings of the 2017 IEEE second Ecuador technical chapters meeting (ETCM), Salinas, Ecuador.","DOI":"10.1109\/ETCM.2017.8247487"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6343","DOI":"10.1007\/s00521-019-04142-8","article-title":"Surface EMG hand gesture recognition system based on PCA and GRNN","volume":"32","author":"Qi","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Panda, A.K., Chakravarty, R., and Moulik, S. (2021, January 1\u20133). Hand gesture recognition using flex sensor and machine learning algorithms. Proceedings of the 2020 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES), Langkawi, Malaysia.","DOI":"10.1109\/IECBES48179.2021.9398789"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Oudah, M., Al-Naji, A., and Chahl, J. (2020). Hand gesture recognition based on computer vision: A review of techniques. J. Imaging, 6.","DOI":"10.3390\/jimaging6080073"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Nguyen, X.S., Brun, L., L\u00e9zoray, O., and Bougleux, S. (2019, January 15\u201320). A neural network based on SPD manifold learning for skeleton-based hand gesture recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01231"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Devineau, G., Moutarde, F., Xi, W., and Yang, J. (2018, January 15\u201319). Deep learning for hand gesture recognition on skeletal data. Presented at 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00025"},{"key":"ref_28","unstructured":"Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.L., Yong, M.G., and Lee, J. (2019). Mediapipe: A framework for building perception pipelines. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1002\/rob.21831","article-title":"Rtab-map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation","volume":"36","author":"Michaud","year":"2019","journal-title":"J. Field Robot."},{"key":"ref_30","unstructured":"(2024, December 31). Oculus Integration SDK. Available online: https:\/\/developers.meta.com\/horizon\/downloads\/package\/unity-integration\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/4\/1181\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:34:47Z","timestamp":1760027687000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/4\/1181"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,14]]},"references-count":30,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["s25041181"],"URL":"https:\/\/doi.org\/10.3390\/s25041181","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,14]]}}}