{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T11:05:52Z","timestamp":1778497552041,"version":"3.51.4"},"reference-count":36,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T00:00:00Z","timestamp":1699833600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Alleviating the burden on amputees in terms of high-level control of their prosthetic devices is an open research challenge. EMG-based intention detection presents some limitations due to movement artifacts, fatigue, and stability. The integration of exteroceptive sensing can provide a valuable solution to overcome such limitations. In this paper, a novel semiautonomous control system (SCS) for wrist\u2013hand prostheses using a computer vision system (CVS) is proposed and validated. The SCS integrates object detection, grasp selection, and wrist orientation estimation algorithms. By combining CVS with a simulated EMG-based intention detection module, the SCS guarantees reliable prosthesis control. Results show high accuracy in grasping and object classification (\u226597%) at a fast frame analysis frequency (2.07 FPS). The SCS achieves an average angular estimation error \u226418\u00b0 and stability \u22640.8\u00b0 for the proposed application. Operative tests demonstrate the capabilities of the proposed approach to handle complex real-world scenarios and pave the way for future implementation on a real prosthetic device.<\/jats:p>","DOI":"10.3390\/robotics12060152","type":"journal-article","created":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T02:02:42Z","timestamp":1699840962000},"page":"152","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["A Semiautonomous Control Strategy Based on Computer Vision for a Hand\u2013Wrist Prosthesis"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-4409-9167","authenticated-orcid":false,"given":"Gianmarco","family":"Cirelli","sequence":"first","affiliation":[{"name":"Research Unit of Advanced Robotics and Human-Centred Technologies, Universit\u00e1 Campus Bio-Medico di Roma, 00128 Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6238-2241","authenticated-orcid":false,"given":"Christian","family":"Tamantini","sequence":"additional","affiliation":[{"name":"Research Unit of Advanced Robotics and Human-Centred Technologies, Universit\u00e1 Campus Bio-Medico di Roma, 00128 Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luigi Pietro","family":"Cordella","sequence":"additional","affiliation":[{"name":"Universit\u00e1 di Napoli Federico II, 80125 Naples, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6946-0377","authenticated-orcid":false,"given":"Francesca","family":"Cordella","sequence":"additional","affiliation":[{"name":"Research Unit of Advanced Robotics and Human-Centred Technologies, Universit\u00e1 Campus Bio-Medico di Roma, 00128 Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,11,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e2205","DOI":"10.1097\/GOX.0000000000002205","article-title":"Cross-sectional international multicenter study on quality of life and reasons for abandonment of upper limb prostheses","volume":"7","author":"Yamamoto","year":"2019","journal-title":"Plast. Reconstr. Surg. Glob. Open"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Tamantini, C., Cordella, F., Lauretti, C., and Zollo, L. (2021). The WGD\u2014A dataset of assembly line working gestures for ergonomic analysis and work-related injuries prevention. Sensors, 21.","DOI":"10.3390\/s21227600"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"907","DOI":"10.5535\/arm.2011.35.6.907","article-title":"A survey on activities of daily living and occupations of upper extremity amputees","volume":"35","author":"Jang","year":"2011","journal-title":"Ann. Rehabilit. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"821","DOI":"10.1080\/17483107.2020.1738567","article-title":"Comfort and function remain key factors in upper limb prosthetic abandonment: Findings of a scoping review","volume":"16","author":"Smail","year":"2021","journal-title":"Disabil. Rehabilit. Assist. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Igual, C., Pardo, L.A., Hahne, J.M., and Igual, J. (2019). Myoelectric control for upper limb prostheses. Electronics, 8.","DOI":"10.3390\/electronics8111244"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1007\/s40137-013-0044-8","article-title":"Prosthetic myoelectric control strategies: A clinical perspective","volume":"2","author":"Roche","year":"2014","journal-title":"Curr. Surg. Rep."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"9","DOI":"10.3389\/fnbot.2016.00009","article-title":"Deep learning with convolutional neural networks applied to electromyography data: A resource for the classification of movements for prosthetic hands","volume":"10","author":"Atzori","year":"2016","journal-title":"Front. Neurorobot."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"eaat3630","DOI":"10.1126\/scirobotics.aat3630","article-title":"Simultaneous control of multiple functions of bionic hand prostheses: Performance and robustness in end users","volume":"3","author":"Hahne","year":"2018","journal-title":"Sci. Robot."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1186\/s12984-022-00982-z","article-title":"A parallel classification strategy to simultaneous control elbow, wrist, and hand movements","volume":"19","author":"Leone","year":"2022","journal-title":"J. NeuroEng. Rehabilit."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1007\/s13534-023-00281-z","article-title":"Recent trends and challenges of surface electromyography in prosthetic applications","volume":"13","author":"Yadav","year":"2023","journal-title":"Biomed. Eng. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhang, Q., and Zhu, J. (2022, January 21\u201323). The Application of EMG and Machine Learning in Human Machine Interface. Proceedings of the 2nd International Conference on Bioinformatics and Intelligent Computing, Harbin, China.","DOI":"10.1145\/3523286.3524588"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/TAC.1962.1105456","article-title":"An adaptive artificial hand","volume":"7","author":"Tomovic","year":"1962","journal-title":"IRE Trans. Autom. Control"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Stefanelli, E., Cordella, F., Gentile, C., and Zollo, L. (2023). Hand Prosthesis Sensorimotor Control Inspired by the Human Somatosensory System. Robotics, 12.","DOI":"10.3390\/robotics12050136"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1186\/1743-0003-7-42","article-title":"Cognitive vision system for control of dexterous prosthetic hands: Experimental evaluation","volume":"7","author":"Dosen","year":"2010","journal-title":"J. Neuroeng. Rehabilit."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1111\/j.1525-1594.2010.01040.x","article-title":"Transradial prosthesis: Artificial vision for control of prehension","volume":"35","year":"2011","journal-title":"Artif. Organs"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"814973","DOI":"10.3389\/fnbot.2022.814973","article-title":"Continuous Semi-autonomous Prosthesis Control Using a Depth Sensor on the Hand","volume":"16","author":"Castro","year":"2022","journal-title":"Front. Neurorobot."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"036025","DOI":"10.1088\/1741-2552\/aa6802","article-title":"Deep learning-based artificial vision for grasp classification in myoelectric hands","volume":"14","author":"Ghazaei","year":"2017","journal-title":"J. Neural Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1007\/s13748-019-00203-0","article-title":"Convolutional neural network: A review of models, methodologies and applications to object detection","volume":"9","author":"Dhillon","year":"2020","journal-title":"Prog. Artif. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"815716","DOI":"10.3389\/fnbot.2022.815716","article-title":"Designing Prosthetic Hands With Embodied Intelligence: The KIT Prosthetic Hands","volume":"16","author":"Weiner","year":"2022","journal-title":"Front. Neurorobot."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Perera, D.M., and Madusanka, D. (2021, January 27\u201329). Vision-EMG Fusion Method for Real-time Grasping Pattern Classification System. Proceedings of the 2021 Moratuwa Engineering Research Conference, Moratuwa, Sri Lanka.","DOI":"10.1109\/MERCon52712.2021.9525702"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"744476","DOI":"10.3389\/frai.2021.744476","article-title":"Improving robotic hand prosthesis control with eye tracking and computer vision: A multimodal approach based on the visuomotor behavior of grasping","volume":"4","author":"Cognolato","year":"2022","journal-title":"Front. Artif. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"6004804","DOI":"10.1109\/LSENS.2023.3301837","article-title":"Robust Fusion Model for Handling EMG and Computer Vision Data in Prosthetic Hand Control","volume":"7","author":"Deshmukh","year":"2023","journal-title":"IEEE Sens. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Cordella, F., Di Corato, F., Loianno, G., Siciliano, B., and Zollo, L. (2013, January 3\u20137). Robust pose estimation algorithm for wrist motion tracking. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696891"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Boshlyakov, A.A., and Ermakov, A.S. (2020, January 28\u201329). Development of a Vision System for an Intelligent Robotic Hand Prosthesis Using Neural Network Technology. Proceedings of the ITM Web of Conference EDP Sciences, Moscow, Russia.","DOI":"10.1051\/itmconf\/20203504006"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_26","first-page":"b822","article-title":"Comparison between YOLO and SSD mobile net for object detection in a surveillance drone","volume":"5","author":"Phadtare","year":"2021","journal-title":"Int. J. Sci. Res. Eng. Manag."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Fleet, D., Pajdla, T., Schiele, B., and Tuytelaars, T. (2014, January 6\u201312). Microsoft COCO: Common Objects in Context. Proceedings of the Computer Vision\u2014ECCV 2014, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1109\/THMS.2015.2470657","article-title":"The grasp taxonomy of human grasp types","volume":"46","author":"Feix","year":"2015","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1533","DOI":"10.1152\/jn.90518.2008","article-title":"Gaze behavior when reaching to remembered targets","volume":"100","author":"Flanagan","year":"2008","journal-title":"J. Neurophysiol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1109\/TSMC.1979.4310076","article-title":"A Threshold Selection Method from Gray-Level Histograms","volume":"9","author":"Otsu","year":"1979","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Tamantini, C., Lapresa, M., Cordella, F., Scotto di Luzio, F., Lauretti, C., and Zollo, L. (2022, January 13\u201316). A robot-aided rehabilitation platform for occupational therapy with real objects. Proceedings of the Converging Clinical and Engineering Research on Neurorehabilitation IV: 5th ICNR2020, Vigo, Spain.","DOI":"10.1007\/978-3-030-70316-5_136"},{"key":"ref_32","unstructured":"Gardner, M., Woodward, R., Vaidyanathan, R., B\u00fcrdet, E., and Khoo, B.C. (2004, January 6\u20139). An unobtrusive vision system to reduce the cognitive burden of hand prosthesis control. Proceedings of the 13th ICARCV, Kunming, China."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"DeGol, J., Akhtar, A., Manja, B., and Bretl, T. (2016, January 16\u201320). Automatic grasp selection using a camera in a hand prosthesis. Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7590732"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"751282","DOI":"10.3389\/fnbot.2021.751282","article-title":"A Hybrid 3D Printed Hand Prosthesis Prototype Based on sEMG and a Fully Embedded Computer Vision System","volume":"15","author":"Castro","year":"2022","journal-title":"Front. Neurorobot."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1016\/j.robot.2017.11.005","article-title":"A novel underactuated multi-fingered soft robotic hand for prosthetic application","volume":"100","author":"Devi","year":"2018","journal-title":"Robot. Auton. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2026","DOI":"10.1109\/TMECH.2022.3170800","article-title":"Larg: A lightweight robotic gripper with 3-d topology optimized adaptive fingers","volume":"27","author":"Sun","year":"2022","journal-title":"IEEE\/ASME Trans. Mechatronics"}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/12\/6\/152\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:22:01Z","timestamp":1760131321000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/12\/6\/152"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,13]]},"references-count":36,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["robotics12060152"],"URL":"https:\/\/doi.org\/10.3390\/robotics12060152","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,13]]}}}