{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T01:16:23Z","timestamp":1770513383584,"version":"3.49.0"},"reference-count":34,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,1,14]],"date-time":"2019-01-14T00:00:00Z","timestamp":1547424000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>As the aging of the population becomes more severe, wheelchair-mounted robotic arms (WMRAs) are gaining an increased amount of attention. Laser pointer interactions are an attractive method enabling humans to unambiguously point out objects and pick them up. In addition, they bring about a greater sense of participation in the interaction process as an intuitive interaction mode. However, the issue of human\u2013robot interactions remains to be properly tackled, and traditional laser point interactions still suffer from poor real-time performance and low accuracy amid dynamic backgrounds. In this study, combined with an advanced laser point detection method and an improved pose estimation algorithm, a laser pointer is used to facilitate the interactions between humans and a WMRA in an indoor environment. Assistive grasping using a laser selection consists of two key steps. In the first step, the images captured using an RGB-D camera are pre-processed, and then fed to a convolutional neural network (CNN) to determine the 2D coordinates of the laser point and objects within the image. Meanwhile, the centroid coordinates of the selected object are also obtained using the depth information. In this way, the object to be picked up and its location are determined. The experimental results show that the laser point can be detected with almost 100% accuracy in a complex environment. In the second step, a compound pose-estimation algorithm aiming at a sparse use of multi-view templates is applied, which consists of both coarse- and precise-matching of the target to the template objects, greatly improving the grasping performance. The proposed algorithms were implemented on a Kinova Jaco robotic arm, and the experimental results demonstrate their effectiveness. Compared with commonly accepted methods, the time consumption of the pose generation can be reduced from 5.36 to 4.43 s, and synchronously, the pose estimation error is significantly improved from 21.31% to 3.91%.<\/jats:p>","DOI":"10.3390\/s19020303","type":"journal-article","created":{"date-parts":[[2019,1,14]],"date-time":"2019-01-14T12:20:07Z","timestamp":1547468407000},"page":"303","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1612-0459","authenticated-orcid":false,"given":"Ming","family":"Zhong","sequence":"first","affiliation":[{"name":"Industrial Research Institute of Robotics and Intelligent Equipment, Harbin Institute of Technology, Weihai 264209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanqiang","family":"Zhang","sequence":"additional","affiliation":[{"name":"Industrial Research Institute of Robotics and Intelligent Equipment, Harbin Institute of Technology, Weihai 264209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xi","family":"Yang","sequence":"additional","affiliation":[{"name":"Industrial Research Institute of Robotics and Intelligent Equipment, Harbin Institute of Technology, Weihai 264209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yufeng","family":"Yao","sequence":"additional","affiliation":[{"name":"Industrial Research Institute of Robotics and Intelligent Equipment, Harbin Institute of Technology, Weihai 264209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Junlong","family":"Guo","sequence":"additional","affiliation":[{"name":"Industrial Research Institute of Robotics and Intelligent Equipment, Harbin Institute of Technology, Weihai 264209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0903-9759","authenticated-orcid":false,"given":"Yaping","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering, University of Houston, Houston, TX 77004, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yaxin","family":"Liu","sequence":"additional","affiliation":[{"name":"Industrial Research Institute of Robotics and Intelligent Equipment, Harbin Institute of Technology, Weihai 264209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,1,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.5391\/IJFIS.2015.15.1.1","article-title":"Deep Level Situation Understanding for Casual Communication in Humans-Robots Interaction","volume":"15","author":"Tang","year":"2015","journal-title":"Int. J. Fuzzy Log. Intell. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Wu, Q., and Wu, H. (2018). Development, Dynamic Modeling, and Multi-Modal Control of a Therapeutic Exoskeleton for Upper Limb Rehabilitation Training. Sensors, 18.","DOI":"10.3390\/s18113611"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"961","DOI":"10.1109\/34.799904","article-title":"An HMM-based threshold model approach for gesture recognition","volume":"21","author":"Lee","year":"1999","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Tanaka, H., Sumi, Y., and Matsumoto, Y. (2010, January 26\u201328). Assistive robotic arm autonomously bringing a cup to the mouth by face recognition. Proceedings of the 2010 IEEE Advanced Robotics and ITS Social Impacts, Seoul, Korea.","DOI":"10.1109\/ARSO.2010.5679633"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1017\/S0263574798000666","article-title":"Knowledge driven planning and multimodal control of a telerobot","volume":"16","author":"Kazi","year":"1998","journal-title":"Robotica"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1109\/TRO.2012.2228134","article-title":"The impact of human\u2013robot interfaces on the learning of visual objects","volume":"29","author":"Rouanet","year":"2013","journal-title":"IEEE Trans. Robot."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Choi, K., and Min, B.K. (2015). Future directions for brain-machine interfacing technology. Recent Progress in Brain and Cognitive Engineering, Springer.","DOI":"10.1007\/978-94-017-7239-6_1"},{"key":"ref_8","first-page":"2238","article-title":"Laser pointer detection based on intensity profile analysis for application in teleconsultation","volume":"12","author":"Imtiaz","year":"2017","journal-title":"J. Eng. Sci. Technol."},{"key":"ref_9","unstructured":"Kang, S.H., and Yang, C.K. (July, January 29). Laser-pointer human computer interaction system. Proceedings of the IEEE International Conference on Multimedia & Expo Workshops, Turin, Italy."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Karvelis, P., Roijezon, U., Faleij, R., Georgoulas, G., Mansouri, S.S., and Nikolakopoulos, G. (2017, January 3\u20136). A laser dot tracking method for the assessment of sensorimotor function of the hand. Proceedings of the Mediterranean Conference on Control and Automation, Valletta, Malta.","DOI":"10.1109\/MED.2017.7984121"},{"key":"ref_11","unstructured":"Fukuda, Y., Kurihara, Y., Kobayashi, K., and Watanabe, K. (2009, January 18\u201321). Development of electric wheelchair interface based on laser pointer. Proceedings of the ICCAS-SICE, Fukuoka, Japan."},{"key":"ref_12","unstructured":"Gualtieri, M., Kuczynski, J., Shultz, A.M., Pas, A.T., Platt, R., and Yanco, H. (June, January 29). Open world assistive grasping using laser selection. Proceedings of the IEEE International Conference on Robotics and Automation, Singapore."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kemp, C.C., Anderson, C.D., Hai, N., Trevor, A.J., and Xu, Z. (2008, January 12\u201315). A point-and-click interface for the real world: Laser designation of objects for mobile manipulation. Proceedings of the 2008 3rd ACM\/IEEE International Conference on Human-Robot Interaction, Amsterdam, The Netherlands.","DOI":"10.1145\/1349822.1349854"},{"key":"ref_14","unstructured":"Hai, N., Anderson, C., Trevor, A., Jain, A., Xu, Z., and Kemp, C.C. (2008, January 12). EL-E: An assistive robot that fetches objects from flat surfaces. Proceedings of the Robotic Helpers Workshop at HRI\u201908, Amsterdam, The Netherlands."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1007\/s10514-009-9148-5","article-title":"EL-E: An assistive mobile manipulator that autonomously fetches objects from flat surfaces","volume":"28","author":"Jain","year":"2010","journal-title":"Auton. Robot."},{"key":"ref_16","unstructured":"Lapointe, J.F., and Godin, G. (2005, January 1). On-screen laser spot detection for large display interaction. Proceedings of the IEEE International Workshop on Haptic Audio Visual Environments & Their Applications, Ottawa, ON, Canada."},{"key":"ref_17","unstructured":"Nguyen, H., Jain, A., Anderson, C., and Kemp, C.C. (2008, January 22\u201326). A clickable world: behavior selection through pointing and context for mobile manipulation. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Nice, France."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhou, P., Wang, X., Huang, Q., and Ma, C. (2016, January 25\u201327). Laser spot center detection based on improved circle fitting algorithm. Proceedings of the 2018 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), Xi\u2019an, China.","DOI":"10.1109\/IMCEC.2018.8469554"},{"key":"ref_19","unstructured":"Stauffer, C., and Grimson, W.E.L. (1999, January 23\u201325). Adaptive background mixture models for real-time tracking. Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), Fort Collins, CO, USA."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Geng, L., and Xiao, Z. (2011, January 30\u201331). Real time foreground-background segmentation using two-layer codebook model. Proceedings of the 2011 International Conference on Control, Automation and Systems Engineering, Singapore.","DOI":"10.1109\/ICCASE.2011.5997546"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1016\/j.isprsjprs.2018.01.003","article-title":"One-two-one networks for compression artifacts reduction in remote sensing","volume":"145","author":"Zhang","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"693","DOI":"10.1109\/TSMC.2016.2629509","article-title":"Output Constraint Transfer for Kernelized Correlation Filter in Tracking","volume":"47","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"26","DOI":"10.5391\/IJFIS.2017.17.1.26","article-title":"Plant Leaf Recognition Using a Convolution Neural Network","volume":"17","author":"Jeon","year":"2017","journal-title":"Int. J. Fuzzy Log. Intell. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"208","DOI":"10.5391\/IJFIS.2016.16.3.208","article-title":"CNN Based Lithography Hotspot Detection","volume":"16","author":"Shin","year":"2016","journal-title":"Int. J. Fuzzy Log. Intell. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"19959","DOI":"10.1109\/ACCESS.2018.2815149","article-title":"Object Detection Based on Multi-Layer Convolution Feature Fusion and Online Hard Example Mining","volume":"6","author":"Chu","year":"2018","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jiang, S., Yao, W., Hong, Z., Li, L., Su, C., and Kuc, T.-Y. (2018). A Classification-Lock Tracking Strategy Allowing a Person-Following Robot to Operate in a Complicated Indoor Environment. Sensors, 18.","DOI":"10.3390\/s18113903"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"98","DOI":"10.5391\/IJFIS.2017.17.2.98","article-title":"CNN Output Optimization for More Balanced Classification","volume":"17","author":"Choi","year":"2017","journal-title":"Int. J. Fuzzy Log. Intell. Syst."},{"key":"ref_28","unstructured":"Redmon, J., and Farhadi, A. (arXiv, 2018). YOLOv3: An Incremental Improvement, arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"4357","DOI":"10.1109\/TIP.2018.2835143","article-title":"Gabor Convolutional Networks","volume":"27","author":"Luan","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1007\/s11263-016-0880-y","article-title":"Bounding Multiple Gaussians Uncertainty with Application to Object Tracking","volume":"118","author":"Zhang","year":"2016","journal-title":"Int. J. Comput. Vis."},{"key":"ref_31","unstructured":"Rusu, R.B., Bradski, G., Thibaux, R., and Hsu, J. (2014, January 18\u201322). Fast 3D recognition and pose using the Viewpoint Feature Histogram. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Aldoma, A., Vincze, M., Blodow, N., Gossow, D., Gedikli, S., Rusu, R.B., and Bradski, G. (2011, January 6\u201313). CAD-model recognition and 6DOF pose estimation using 3D cues. Proceedings of the IEEE International Conference on Computer Vision Workshops, Barcelona, Spain.","DOI":"10.1109\/ICCVW.2011.6130296"},{"key":"ref_33","unstructured":"Filipe, S., and Alexandre, L.A. (2014, January 5\u20138). A comparative evaluation of 3D keypoint detectors in a RGB-D object dataset. Proceedings of the International Conference on Computer Vision Theory and Applications, Lisbon, Portugal."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Tombari, F., and Stefano, L.D. (2010, January 14\u201317). Object recognition in 3D scenes with occlusions and clutter by Hough Voting. Proceedings of the 2010 Fourth Pacific-Rim Symposium on Image and Video Technology, Singapore.","DOI":"10.1109\/PSIVT.2010.65"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/2\/303\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:25:42Z","timestamp":1760185542000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/2\/303"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,14]]},"references-count":34,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,1]]}},"alternative-id":["s19020303"],"URL":"https:\/\/doi.org\/10.3390\/s19020303","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,14]]}}}