{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,12]],"date-time":"2026-10-12T15:53:52Z","timestamp":1791820432199,"version":"4.3.4"},"reference-count":50,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,1,14]],"date-time":"2022-01-14T00:00:00Z","timestamp":1642118400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62073097"],"award-info":[{"award-number":["62073097"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Assistive robotic arms (ARAs) that provide care to the elderly and people with disabilities, are a significant part of Human-Robot Interaction (HRI). Presently available ARAs provide non-intuitive interfaces such as joysticks for control and thus, lacks the autonomy to perform daily activities. This study proposes that, for inducing autonomous behavior in ARAs, visual sensors integration is vital, and visual servoing in the direct Cartesian control mode is the preferred method. Generally, ARAs are designed in a configuration where its end-effector\u2019s position is defined in the fixed base frame while orientation is expressed in the end-effector frame. We denoted this configuration as \u2018mixed frame robotic arms\u2019. Consequently, conventional visual servo controllers which operate in a single frame of reference are incompatible with mixed frame ARAs. Therefore, we propose a mixed-frame visual servo control framework for ARAs. Moreover, we enlightened the task space kinematics of a mixed frame ARAs, which led us to the development of a novel \u201cmixed frame Jacobian matrix\u201d. The proposed framework was validated on a mixed frame JACO-2 7 DoF ARA using an adaptive proportional derivative controller for achieving image-based visual servoing (IBVS), which showed a significant increase of 31% in the convergence rate, outperforming conventional IBVS joint controllers, especially in the outstretched arm positions and near the base frame. Our Results determine the need for the mixed frame controller for deploying visual servo control on modern ARAs, that can inherently cater to the robotic arm\u2019s joint limits, singularities, and self-collision problems.<\/jats:p>","DOI":"10.3390\/s22020642","type":"journal-article","created":{"date-parts":[[2022,1,16]],"date-time":"2022-01-16T20:45:21Z","timestamp":1642365921000},"page":"642","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Mix Frame Visual Servo Control Framework for Autonomous Assistive Robotic Arms"],"prefix":"10.3390","volume":"22","author":[{"given":"Zubair","family":"Arif","sequence":"first","affiliation":[{"name":"State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yili","family":"Fu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kyrarini, M., Lygerakis, F., Rajavenkatanarayanan, A., Sevastopoulos, C., Nambiappan, H.R., Chaitanya, K.K., Babu, A.R., Mathew, J., and Makedon, F. (2021). A Survey of Robots in Healthcare. Technologies, 9.","DOI":"10.3390\/technologies9010008"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1080\/07370024.2017.1312406","article-title":"Why Would I Use This in My Home? A Model of Domestic Social Robot Acceptance","volume":"34","year":"2019","journal-title":"Hum. Comput. Interact"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1007\/s10514-017-9677-2","article-title":"Progress and prospects of the human\u2013robot collaboration","volume":"42","author":"Ajoudani","year":"2018","journal-title":"Auton. Robot."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"24","DOI":"10.3389\/fnbot.2017.00024","article-title":"A human-robot interaction perspective on assistive and rehabilitation robotics","volume":"11","author":"Beckerle","year":"2017","journal-title":"Front. Neurorobotics"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/fnagi.2015.00141","article-title":"\u201cAre we ready for robots that care for us?\u201d Attitudes and opinions of older adults toward socially assistive robots","volume":"7","author":"Pino","year":"2015","journal-title":"Front. Aging Neurosci."},{"key":"ref_6","unstructured":"(2021, November 02). Kinova Robotic Arms. Available online: https:\/\/www.kinovarobotics.com\/en."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Capille, J.W. (2010). Kinematic and Experimental Evaluation of Commercial Wheelchair-Mounted Robotic Arms. [Master\u2019s Thesis, University of South Florida].","DOI":"10.1109\/ICSMC.2011.6083725"},{"key":"ref_8","unstructured":"Sachin, G. (2021, November 02). Assistive Robotics Market, Markets & Markets; Report Codese 7298. Available online: https:\/\/www.marketsandmarkets.com\/Market-Reports\/assistive-robotics-market-37247851.html."},{"key":"ref_9","unstructured":"Pedro, E., and Albert, M.C. (2017). Robotic Assistive Technologies, CRC Press."},{"key":"ref_10","first-page":"509","article-title":"Intuitive Adaptive Orientation Control for Enhanced Human\u2013Robot Interaction","volume":"35","author":"Vu","year":"2018","journal-title":"IEEE Trans. Robot."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1177\/0008417418820878","article-title":"Impacts of robotic arm use on individuals with upper extremity disabilities: A scoping review","volume":"85","author":"Beaudoin","year":"2018","journal-title":"Can. J. Occup. Ther."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1080\/17483107.2017.1299804","article-title":"Performance evaluation of 3D vision-based semi-autonomous control method for assistive robotic manipulator","volume":"13","author":"Ka","year":"2018","journal-title":"Disabil. Rehabil. Assist. Technol."},{"key":"ref_13","first-page":"46","article-title":"Voice control interface prototype for assistive robots for people living with upper limb disabilities","volume":"2019","author":"Poirier","year":"2019","journal-title":"IEEE Int. Conf. Rehabil. Robot."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1016\/j.robot.2019.02.015","article-title":"A service assistant combining autonomous robotics, flexible goal formulation, and deep-learning-based brain\u2013computer interfacing","volume":"116","author":"Kuhner","year":"2019","journal-title":"Rob. Auton. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Aronson, R.M., Santini, T., K\u00fcbler, T.C., Kasneci, E., Srinivasa, S., and Admoni, H. (2018, January 5\u20138). Eye-Hand Behavior in Human-Robot Shared Manipulation. Proceedings of the ACM\/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA.","DOI":"10.1145\/3171221.3171287"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.cviu.2016.03.015","article-title":"Enhanced control of a wheelchair-mounted robotic manipulator using 3-D vision and multimodal interaction","volume":"149","author":"Jiang","year":"2016","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2344","DOI":"10.1109\/TNSRE.2019.2950619","article-title":"Proof of Concept of an Assistive Robotic Arm Control Using Artificial Stereovision and Eye-Tracking","volume":"27","author":"Raison","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_18","unstructured":"Snoswell, A.J., Dewanto, V., Hoerger, M., Song, J., Kurniawati, H., and Singh, S.P.N. (2018, January 4\u20136). A distributed, any-time robot architecture for robust manipulation. Proceedings of the Australasian Conference on Robotics and Automation, Lincoln, New Zealand."},{"key":"ref_19","first-page":"1","article-title":"Three Dimensional Computer Vision-Based Alternative Control Method For Assistive Robotic Manipulator","volume":"1","author":"Ka","year":"2016","journal-title":"Int. J. Adv. Robot. Autom."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Siciliano, B., Sciavicco, L., Villani, L., and Oriolo, G. (2009). Robotics: Modeling, Planning, and Control, Springer.","DOI":"10.1007\/978-1-84628-642-1"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Akalin, N., Kristoffersson, A., and Loutfi, A. (2019). Evaluating the Sense of Safety and Security in Human\u2013Robot Interaction with Older People, Springer.","DOI":"10.1007\/978-3-030-17107-0_12"},{"key":"ref_22","unstructured":"Herlant, L.V. (2018). Algorithms, Implementation, and Studies on Eating with a Shared Control Robot Arm. [Ph.D. Thesis, Carnegie Mellon University]."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Herlant, L.V., Holladay, R.M., and Srinivasa, S.S. (2016, January 7\u201310). Assistive teleoperation of robot arms via automatic time-optimal mode switching. Proceedings of the 2016 11th ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Christchurch, New Zealand.","DOI":"10.1109\/HRI.2016.7451731"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MRA.2006.250573","article-title":"Visual servo control. I. Basic approaches","volume":"13","author":"Chaumette","year":"2006","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_25","unstructured":"Bruno, S., and Khatib, O. (2008). Visual Servoing and Visual tracking. Handbook of Robotics, Springer."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Sun, X., Zhu, X., Wang, P., and Chen, H. (2018, January 19\u201323). A Review of Robot Control with Visual Servoing. Proceedings of the 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems, Tianjin, China.","DOI":"10.1109\/CYBER.2018.8688060"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Kim, D.-J., Lovelett, R., and Behal, A. (2009, January 12\u201317). Eye-in-hand stereo visual servoing of an assistive robot arm in unstructured environments. Proceedings of the IEEE International Conference on Robotics and Automation, Kobe, Japan.","DOI":"10.1109\/ROBOT.2009.5152821"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"912","DOI":"10.1109\/LRA.2017.2654544","article-title":"Image-Based Visual Servoing with Light Field Cameras","volume":"2","author":"Tsai","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1109\/MRA.2005.1577023","article-title":"ViSP: A generic software platform for visual servoing","volume":"12","author":"Marchand","year":"2005","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Campeau-lecours, A. (2017). Kinova Modular Robot Arms for Service Robotics Applications. Int. J. Robot. Appl. Technol., 5.","DOI":"10.4018\/IJRAT.2017070104"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Lebrasseur, A., Josiane Lettre, O.T., and Routhier, F. (2019). Archambault: Assistive robotic arm: Evaluation of the performance of intelligent algorithms. Assist. Technol. J. RESNA, 1\u201312.","DOI":"10.1080\/10400435.2019.1601649"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1097\/PHM.0b013e3181cf569b","article-title":"The role of assistive robotics in the lives of persons with disability","volume":"89","author":"Brose","year":"2010","journal-title":"Am. J. Phys. Med. Rehabil."},{"key":"ref_33","unstructured":"Robotics, K. (2020, March 01). Kinova Assistive Robot Configurator User Guide. Available online: https:\/\/www.kinovarobotics.com\/sites\/default\/files\/UG-015_KINOVA_Assitive_robot_configurator-User_guide_EN_R01.pdf."},{"key":"ref_34","unstructured":"Robotics, K. (2020, March 01). Kinova Software Development Kit-User Guide, s. Available online: https:\/\/www.kinovarobotics.com\/sites\/default\/files\/UG-008_KINOVA_Software_development_kit-User_guide_EN_R02%281%29.pdf."},{"key":"ref_35","unstructured":"(2021, November 02). Kinova Robotics. Kinova Gen2 Ultra-Lightweight Robot. Available online: www.kinovarobotics.com\/product\/gen2-robots."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Khatib, O., and Siciliano, B. (2016). Redundant Robots. Handbook of Robotics, Springer.","DOI":"10.1007\/978-3-319-32552-1"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/j.rcim.2010.08.004","article-title":"The self-adaptation of weights for joint-limits and singularity avoidances of functionally redundant robotic-task","volume":"27","author":"Huo","year":"2011","journal-title":"Robot. Comput. Integr. Manuf."},{"key":"ref_38","unstructured":"Siciliano, B., and Lorenzo, S. (2009). Differential Kinematics and Statics. Robotics: Modelling, Planning and Control, Springer."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Nakanishi, J., Mistry, M.N., Peters, J., and Schaal, S. (November, January 29). Towards compliant humanoids-an experimental assessment of suitable task space position\/orientation controllers. Proceedings of the 2007 IEEE\/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA, USA.","DOI":"10.1109\/IROS.2007.4399562"},{"key":"ref_40","unstructured":"Bruno, S., and Oussama, K. (2017). Velocity relationships. Robotics; Vision and Control, Springer."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Marey, M., and Chaumette, F. (2010, January 18\u201322). New strategies for avoiding robot joint limits: Application to visual servoing using a large projection operator. Proceedings of the 2010 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan.","DOI":"10.1109\/IROS.2010.5653044"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Wang, J., and Olson, E. (2016, January 9\u201314). AprilTag 2: Efficient and robust fiducial detection. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759617"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"844","DOI":"10.1007\/s12555-016-0720-4","article-title":"An Enhanced IBVS Controller of a 6DOF Manipulator Using Hybrid PD- SMC Method","volume":"16","author":"Li","year":"2018","journal-title":"Int. J. Control. Autom. Syst."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Bolton, W. (2015). Process controllers. Instrumentation and Control Systems, Elsevier.","DOI":"10.1016\/B978-0-08-100613-9.00005-5"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1109\/TRO.2013.2281560","article-title":"Dealing with constraints in sensor-based robot control","volume":"30","author":"Kermorgant","year":"2014","journal-title":"IEEE Trans. Robot."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1109\/TRO.2006.889487","article-title":"Task Sequencing for High Level Sensor-Based Control","volume":"23","author":"Mansard","year":"2007","journal-title":"IEEE Trans. Robot."},{"key":"ref_47","unstructured":"(2021, October 30). ViSP-Visual Servoying Paltform. Available online: https:\/\/visp.inria.fr\/."},{"key":"ref_48","unstructured":"(2021, October 30). VISP_Developer Tutorial_ Introduce a New Class with 3rd-Party Dependencies_KINOVA JACOSDK. Available online: https:\/\/visp-doc.inria.fr\/doxygen\/visp-daily\/tutorial-add-new-dependency.html."},{"key":"ref_49","unstructured":"Arif, Z. (2021, October 30). Visp-Jaco-2-Robot API Wrapper (Github Repository). Available online: https:\/\/github.com\/engr-zubair\/visp\/tree\/feat_jaco2_robot."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Claudio, G., Spindler, F., and Chaumette, F. (2016, January 15\u201317). Vision-based manipulation with the humanoid robot Romeo. Proceedings of the IEEE-RAS 16th International Conference on Humanoid Robots, Cancun, Mexico.","DOI":"10.1109\/HUMANOIDS.2016.7803290"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/2\/642\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:02:26Z","timestamp":1760364146000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/2\/642"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,14]]},"references-count":50,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["s22020642"],"URL":"https:\/\/doi.org\/10.3390\/s22020642","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,14]]}}}