{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,27]],"date-time":"2026-07-27T14:50:25Z","timestamp":1785163825071,"version":"3.55.0"},"reference-count":54,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2023,5,4]],"date-time":"2023-05-04T00:00:00Z","timestamp":1683158400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2018YFB1309400"],"award-info":[{"award-number":["2018YFB1309400"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the growth of the world\u2019s population, limited healthcare resources cannot provide adequate nursing services for all people in need. The wheelchair-mounted robotic arm (WMRA) with interactive technology could help to improve users\u2019 self-care ability and relieve nursing stress. However, the users struggle to control the WMRA due to complex operations. To use the WMRA with less burden, this paper proposes an object affordance-based implicit interaction technology using a laser pointer. Firstly, a laser semantic identification algorithm combined with the YOLOv4 and the support vector machine (SVM) is designed to identify laser semantics. Then, an implicit action intention reasoning algorithm, based on the concept of object affordance, is explored to infer users\u2019 intentions and learn their preferences. For the purpose of performing the actions about task intention in the scene, the dynamic movement primitives (DMP) and the finite state mechanism (FSM) are respectively used to generalize the trajectories of actions and reorder the sequence of actions in the template library. In the end, we verified the feasibility of the proposed technology on a WMRA platform. Compared with the previous method, the proposed technology can output the desired intention faster and significantly reduce the user\u2019s limb involvement time (about 85%) in operating the WMRA under the same task.<\/jats:p>","DOI":"10.3390\/s23094477","type":"journal-article","created":{"date-parts":[[2023,5,5]],"date-time":"2023-05-05T02:56:51Z","timestamp":1683255411000},"page":"4477","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Object Affordance-Based Implicit Interaction for Wheelchair-Mounted Robotic Arm Using a Laser Pointer"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3694-9800","authenticated-orcid":false,"given":"Yaxin","family":"Liu","sequence":"first","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8655-5335","authenticated-orcid":false,"given":"Yan","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9595-4322","authenticated-orcid":false,"given":"Yufeng","family":"Yao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1612-0459","authenticated-orcid":false,"given":"Ming","family":"Zhong","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,4]]},"reference":[{"key":"ref_1","unstructured":"United Nations (2022). World Population Prospects 2022: Summary of Results, United Nations."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"27","DOI":"10.3928\/00989134-20221206-01","article-title":"Tiredness Takes Its Toll: An Integrative Review on Sleep and Occupational Outcomes for Long-Term Care Workers","volume":"49","author":"Keller","year":"2023","journal-title":"J. Gerontol. Nurs."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Argall, B.D. (2015, January 8\u201312). Turning Assistive Machines into Assistive Robots. Proceedings of the Conference on Quantum Sensing and Nanophotonic Devices XII, San Francisco, CA, USA.","DOI":"10.1117\/12.2085352"},{"key":"ref_4","unstructured":"Kim, D.J., Hazlett, R., Godfrey, H., Rucks, G., Portee, D., Bricout, J., Cunningham, T., and Behal, A. (2010, January 3\u20138). On the Relationship between Autonomy, Performance, and Satisfaction: Lessons from a Three-Week User Study with post-SCI Patients using a Smart 6DOF Assistive Robotic Manipulator. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Anchorage, AK, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"892","DOI":"10.1177\/1460458217729729","article-title":"The effectiveness of various robotic technologies in assisting older adults","volume":"25","author":"Shishehgar","year":"2019","journal-title":"Health Inform. J"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1109\/MRA.2013.2275695","article-title":"A Supportive FRIEND at Work Robotic Workplace Assistance for the Disabled","volume":"20","author":"Graser","year":"2013","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1023\/B:AURO.0000016864.12513.77","article-title":"Integration of a rehabilitation robotic system (KARES II) with human-friendly man-machine interaction units","volume":"16","author":"Bien","year":"2004","journal-title":"Auton. Robot."},{"key":"ref_8","first-page":"6650447","article-title":"Integrated vision-based robotic arm interface for operators with upper limb mobility impairments","volume":"2013","author":"Jiang","year":"2013","journal-title":"IEEE Int. Conf. Rehabil. Robot."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Rubies, E., Palacin, J., and Clotet, E. (2022). Enhancing the Sense of Attention from an Assistance Mobile Robot by Improving Eye-Gaze Contact from Its Iconic Face Displayed on a Flat Screen. Sensors, 22.","DOI":"10.3390\/s22114282"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Perera, C.J., Lalitharatne, T.D., and Kiguchi, K. (June, January 29). EEG-controlled meal assistance robot with camera-based automatic mouth position tracking and mouth open detection. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989208"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Quiles, E., Dadone, J., Chio, N., and Garcia, E. (2022). Cross-Platform Implementation of an SSVEP-Based BCI for the Control of a 6-DOF Robotic Arm. Sensors, 22.","DOI":"10.3390\/s22135000"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"20","DOI":"10.3389\/fnsys.2021.578875","article-title":"Progress in Brain Computer Interface: Challenges and Opportunities","volume":"15","author":"Saha","year":"2021","journal-title":"Front. Syst. Neurosci."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Belkhiria, C., Boudir, A., Hurter, C., and Peysakhovich, V. (2022). EOG-Based Human-Computer Interface: 2000\u20132020 Review. Sensors, 22.","DOI":"10.3390\/s22134914"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1177\/1745691615589104","article-title":"Shared Attention","volume":"10","author":"Shteynberg","year":"2015","journal-title":"Perspect. Psychol. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Quintero, C.P., Ramirez, O., and Jagersand, M. (2015, January 26\u201330). VIBI: Assistive Vision-Based Interface for Robot Manipulation. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139816"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"17","DOI":"10.3389\/fnbot.2021.647930","article-title":"Gaze-Based Intention Estimation for Shared Autonomy in Pick-and-Place Tasks","volume":"15","author":"Fuchs","year":"2021","journal-title":"Front. Neurorobotics"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kemp, C.C., Anderson, C.D., Nguyen, H., Trevor, A.J., and Xu, Z. (2008, January 12\u201315). A point-and-click interface for the real world: Laser designation of objects for mobile manipulation. Proceedings of the 2008 3rd ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Amsterdam, The Netherlands.","DOI":"10.1145\/1349822.1349854"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Gualtieri, M., Kuczynski, J., Shultz, A.M., Pas, A.T., Platt, R., and Yanco, H. (June, January 29). Open world assistive grasping using laser selection. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989465"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Padfield, N., Camilleri, K., Camilleri, T., Fabri, S., and Bugeja, M. (2022). A Comprehensive Review of Endogenous EEG-Based BCIs for Dynamic Device Control. Sensors, 22.","DOI":"10.3390\/s22155802"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1145\/3446370","article-title":"Visual Affordance and Function Understanding: A Survey","volume":"54","author":"Hassanin","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Sprute, D., Tonnies, K., Konig, M., and IEEE (2019, January 25\u201327). This Far, No Further: Introducing Virtual Borders to Mobile Robots Using a Laser Pointer. In Proceedings of the 3rd IEEE International Conference on Robotic Computing (IRC), Naples, Italy.","DOI":"10.1109\/IRC.2019.00074"},{"key":"ref_22","unstructured":"Yoshihisa, F., Yosuke, K., Kazuyuki, K., and Kajiro, W. (2009, January 18\u201321). Development of electric wheelchair interface based on laser pointer. Proceedings of the 2009 ICCAS-SICE, Fukuoka, Japan."},{"key":"ref_23","unstructured":"Minato, Y., Tsujimura, T., and Izumi, K. (2011, January 13\u201318). Sign-at-ease: Robot navigation system operated by connoted shapes drawn with laser beam. Proceedings of the SICE Annual Conference 2011, Tokyo, Japan."},{"key":"ref_24","unstructured":"Widodo, R.B., Chen, W.J., Matsumaru, T., and SICE (2012, January 20\u201323). Interaction Using the Projector Screen and Spot-light from a Laser Pointer: Handling Some Fundamentals Requirements. In Proceedings of the Annual Conference of the Society-of-Instrument-and-Control-Engineers (SICE), Akita University, Akita, Japan."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1007\/s10514-009-9148-5","article-title":"EL-E: An assistive mobile manipulator that autonomously fetches objects from flat surfaces","volume":"28","author":"Jain","year":"2010","journal-title":"Auton. Robot."},{"key":"ref_26","unstructured":"Nguyen, H., Jain, A., Anderson, C., and Kemp, C.C. (2008, January 22\u201326). A Clickable World: Behavior Selection Through Pointing and Context for Mobile Manipulation. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Nice, France."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chavez, F., Fernandez, F., Alcala, R., Alcala-Fdez, J., and Herrera, F. (2011, January 27\u201330). Evolutionary Learning of a Laser Pointer Detection Fuzzy System for an Environment Control System. Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ 2011), Taipei, Taiwan.","DOI":"10.1109\/FUZZY.2011.6007528"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1207\/s15326969eco1504_2","article-title":"An interview with Eleanor Gibson","volume":"15","author":"Szokolszky","year":"2003","journal-title":"Ecol. Psychol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1016\/S0028-3932(01)00089-6","article-title":"Does visual perception of object afford action? Evidence from a neuroimaging study","volume":"40","author":"Grezes","year":"2002","journal-title":"Neuropsychologia"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.actpsy.2003.11.004","article-title":"Object concepts and action: Extracting affordances from objects parts","volume":"115","author":"Borghi","year":"2004","journal-title":"Acta Psychol."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cramer, M., Cramer, J., Kellens, K., and Demeester, E. (2018, January 23\u201325). Towards robust intention estimation based on object affordance enabling natural human-robot collaboration in assembly tasks. Proceedings of the 6th CIRP Global Web Conference on Envisaging the Future Manufacturing, Design, Technologies and Systems in Innovation Era (CIRPe), Shantou, China.","DOI":"10.1016\/j.procir.2018.09.069"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.cogsys.2018.12.010","article-title":"Object affordance based multimodal fusion for natural Human-Robot interaction","volume":"54","author":"Mi","year":"2019","journal-title":"Cogn. Syst. Res."},{"key":"ref_33","unstructured":"Mo, K., Qin, Y., Xiang, F., Su, H., and Guibas, L. (2021, January 8\u201311). O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning. Proceedings of the 5th Conference on Robot Learning, London, UK."},{"key":"ref_34","unstructured":"Mandikal, P., and Grauman, K. (June, January 30). Learning Dexterous Grasping with Object-Centric Visual Affordances. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Deng, S.H., Xu, X., Wu, C.Z., Chen, K., and Jia, K. (2021, January 19\u201325). 3D AffordanceNet: A Benchmark for Visual Object Affordance Understanding. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), ELECTR NETWORK, Virtual.","DOI":"10.1109\/CVPR46437.2021.00182"},{"key":"ref_36","unstructured":"Xu, D.F., Mandlekar, A., Martin-Martin, R., Zhu, Y.K., Savarese, S., Li, F.F., and IEEE (June, January 30). Deep Affordance Foresight: Planning Through What Can Be Done in the Future. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Muller, S., Wengefeld, T., Trinh, T.Q., Aganian, D., Eisenbach, M., and Gross, H.M. (2020). A Multi-Modal Person Perception Framework for Socially Interactive Mobile Service Robots. Sensors, 20.","DOI":"10.3390\/s20030722"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Jain, A., Zamir, A.R., Savarese, S., and Saxena, A. (2016, January 27\u201330). Structural-RNN: Deep Learning on Spatio-Temporal Graphs. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.573"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"17","DOI":"10.3389\/fnbot.2020.567571","article-title":"Toward Shared Autonomy Control Schemes for Human-Robot Systems: Action Primitive Recognition Using Eye Gaze Features","volume":"14","author":"Wang","year":"2020","journal-title":"Front. Neurorobotics"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kim, S., Jung, J., Kavuri, S., and Lee, M. (2013, January 3\u20137). Intention Estimation and Recommendation System Based on Attention Sharing. Proceedings of the Neural Information Processing: 20th International Conference, Daegu, Republic of Korea.","DOI":"10.1007\/978-3-642-42054-2_49"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Duncan, K., Sarkar, S., Alqasemi, R., and Dubey, R. (2014, January 6\u201312). Scene-Dependent Intention Recognition for Task Communication with Reduced Human-Robot Interaction. Proceedings of the 13th European Conference on Computer Vision (ECCV), Zurich, Switzerland.","DOI":"10.1007\/978-3-319-16199-0_51"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2824","DOI":"10.1109\/TBME.2017.2677902","article-title":"3-D-Gaze-Based Robotic Grasping Through Mimicking Human Visuomotor Function for People with Motion Impairments","volume":"64","author":"Li","year":"2017","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_43","unstructured":"Ashish, D. (2012). Robotic Systems, IntechOpen."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Zhong, M., Zhang, Y.Q., Yang, X., Yao, Y.F., Guo, J.L., Wang, Y.P., and Liu, Y.X. (2019). Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms. Sensors, 19.","DOI":"10.3390\/s19020303"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M., and IEEE (2021, January 19\u201325). Scaled-YOLOv4: Scaling Cross Stage Partial Network. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), ELECTR NETWORK, Virtual.","DOI":"10.1109\/CVPR46437.2021.01283"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1565","DOI":"10.1038\/nbt1206-1565","article-title":"What is a support vector machine?","volume":"24","author":"Noble","year":"2006","journal-title":"Nat. Biotechnol."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1023\/A:1018628609742","article-title":"Least squares support vector machine classifiers","volume":"9","author":"Suykens","year":"1999","journal-title":"Neural Process. Lett."},{"key":"ref_48","unstructured":"Ijspeert, A.J., Nakanishi, J., and Schaal, S. (2002, January 9\u201314). Learning attractor landscapes for learning motor primitives. Proceedings of the 15th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_49","unstructured":"Schaal, S. (2003, January 4\u20138). Dynamic movement primitives\u2014A framework for motor control in humans and humanoid robotics. Proceedings of the International Symposium on Adaptive Motion of Animals and Machines (AMAM), Kyoto, Japan."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Chi, M.S., Yao, Y.F., Liu, Y.X., and Zhong, M. (2019). Learning, Generalization, and Obstacle Avoidance with Dynamic Movement Primitives and Dynamic Potential Fields. Appl. Sci. Basel, 9.","DOI":"10.3390\/app9081535"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Fan, H.Q., Xiong, B., Mangalam, K., Li, Y.H., Yan, Z.C., Malik, J., Feichtenhofer, C., and IEEE (2021, January 11\u201317). Multiscale Vision Transformers. Proceedings of the 18th IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada. ELECTR NETWORK.","DOI":"10.1109\/ICCV48922.2021.00675"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Alves, C., Cardoso, A., Colim, A., Bicho, E., Braga, A.C., Cunha, J., Faria, C., and Rocha, L.A. (2022). Human-Robot Interaction in Industrial Settings: Perception of Multiple Participants at a Crossroad Intersection Scenario with Different Courtesy Cues. Robotics, 11.","DOI":"10.3390\/robotics11030059"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Cardoso, A., Colim, A., Bicho, E., Braga, A.C., Menozzi, M., and Arezes, P. (2021). Ergonomics and Human Factors as a Requirement to Implement Safer Collaborative Robotic Workstations: A Literature Review. Safety, 7.","DOI":"10.3390\/safety7040071"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1016\/j.rcim.2020.101998","article-title":"Emerging research fields in safety and ergonomics in industrial collaborative robotics: A systematic literature review","volume":"67","author":"Gualtieri","year":"2021","journal-title":"Robot. Comput. Integr. Manuf."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/9\/4477\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:29:12Z","timestamp":1760124552000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/9\/4477"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,4]]},"references-count":54,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["s23094477"],"URL":"https:\/\/doi.org\/10.3390\/s23094477","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,4]]}}}