{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T21:00:47Z","timestamp":1781038847087,"version":"3.54.1"},"reference-count":41,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,1,17]],"date-time":"2023-01-17T00:00:00Z","timestamp":1673913600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and Technology (MOST)","award":["MOST 111-2221-E-239-031"],"award-info":[{"award-number":["MOST 111-2221-E-239-031"]}]},{"name":"Ministry of Science and Technology (MOST)","award":["MOST 109-2221-E-027-044-MY3"],"award-info":[{"award-number":["MOST 109-2221-E-027-044-MY3"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This research focused on using RGB-D images and modifying an existing machine learning network architecture to generate predictions of the location of successfully grasped objects and to optimize the control system for state delays. A five-finger gripper designed to mimic the human palm was tested to demonstrate that it can perform more delicate missions than many two- or three-finger grippers. Experiments were conducted using the 6-DOF robot arm with the five-finger and two-finger grippers to perform at least 100 actual machine grasps, and compared to the results of other studies. Additionally, we investigated state time delays and proposed a control method for a robot manipulator. Many studies on time-delay systems have been conducted, but most focus on input and output delays. One reason for this emphasis is that input and output delays are the most commonly occurring delays in physical or electronic systems. An additional reason is that state delays increase the complexity of the overall control system. Finally, it was demonstrated that our network can perform as well as a deep network architecture with little training data and omitting steps, such as posture evaluation, and when combined with the hardware advantages of the five-finger gripper, it can produce an automated system with a gripping success rate of over 90%. This paper is an extended study of the conference paper.<\/jats:p>","DOI":"10.3390\/s23031076","type":"journal-article","created":{"date-parts":[[2023,1,18]],"date-time":"2023-01-18T01:33:26Z","timestamp":1674005606000},"page":"1076","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Eye-in-Hand Robotic Arm Gripping System Based on Machine Learning and State Delay Optimization"],"prefix":"10.3390","volume":"23","author":[{"given":"Chin-Sheng","family":"Chen","sequence":"first","affiliation":[{"name":"Graduate Institute of Automation Technology, National Taipei University of Technology, Taipei 10608, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8150-0215","authenticated-orcid":false,"given":"Nien-Tsu","family":"Hu","sequence":"additional","affiliation":[{"name":"Graduate Institute of Automation Technology, National Taipei University of Technology, Taipei 10608, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chen, C.S., Li, T.C., and Hu, N.T. (2022, January 24\u201327). The Gripping Posture Prediction of Eye-in-hand Robotic Arm Using Min-Pnet. Proceedings of the 2022 International Conference on Advanced Robotics and Intelligent Systems (ARIS), Taipei, Taiwan.","DOI":"10.1109\/ARIS56205.2022.9910442"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"6320","DOI":"10.1109\/LRA.2022.3168936","article-title":"Advanced 2-DOF Counterbalance Mechanism Based on Gear Units and Springs to Minimize Required Torques of Robot Arm","volume":"7","author":"Kim","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"12046","DOI":"10.1109\/LRA.2022.3211776","article-title":"Using Quadratic Interpolated Beetle Antennae Search to Enhance Robot Arm Calibration Accuracy","volume":"7","author":"Li","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Yun, A., Lee, W., Kim, S., Kim, J.-H., and Yoon, H. (2022). Development of a Robot Arm Link System Embedded with a Three-Axis Sensor with a Simple Structure Capable of Excellent External Collision Detection. Sensors, 22.","DOI":"10.3390\/s22031222"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Righi, M., Magrini, M., Dolciotti, C., and Moroni, D. (2022). A Case Study of Upper Limb Robotic-Assisted Therapy Using the Track-Hold Device. Sensors, 22.","DOI":"10.3390\/s22031009"},{"key":"ref_6","unstructured":"Borst, C., Fischer, M., and Hirzinger, G. (May, January 26). Grasp Planning: How to Choose a Suitable Task Wrench Space. Proceedings of the IEEE International Con-ference on Robotics and Automation (ICRA), New Orleans, LA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Tang, T., Lin, H.C., Zhao, Y., Chen, W., and Tomizuka, M. (2016, January 21\u201325). Autonomous alignment of peg and hole by force\/torque measurement for robotic assembly. Proceedings of the 2016 IEEE International Conference on Automation Science and Engineering (CASE), Fort Worth, TX, USA.","DOI":"10.1109\/COASE.2016.7743375"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Luo, J., Solowjow, E., Wen, C., Ojea, J.A., and Agogino, A.M. (2018, January 1\u20135). Deep Reinforcement Learning for Robotic Assembly of Mixed Deformable and Rigid Objects. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594353"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Klingbeil, E., Rao, D., Carpenter, B., Ganapathi, V., Ng, A.Y., and Khatib, O. (2011, January 9\u201313). Grasping with Application to an autonomous checkout robot. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980287"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1242","DOI":"10.1017\/S0263574719001383","article-title":"Estimating Fingertip Forces, Torques, and Local Curvatures from Fingernail Images","volume":"38","author":"Chen","year":"2020","journal-title":"Robotica"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1299","DOI":"10.1017\/S0263574720001186","article-title":"Motion Adaptation Based on Learning the Manifold of Task and Dynamic Movement Primitive Parameters","volume":"39","author":"Cohen","year":"2021","journal-title":"Robotica"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/j.mechmachtheory.2018.06.019","article-title":"Grasp configuration planning for a low-cost and easy-operation underactuated three-fingered robot hand","volume":"129","author":"Yao","year":"2018","journal-title":"Mech. Mach. Theory"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"4447","DOI":"10.1109\/LRA.2020.3000428","article-title":"Compliant Peg-in-Hole Assembly Using Partial Spiral Force Trajectory With Tilted Peg Posture","volume":"5","author":"Park","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1177\/0278364917710318","article-title":"Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection","volume":"37","author":"Levine","year":"2018","journal-title":"Int. J. Robot. Res."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Calli, B., Wisse, M., and Jonker, P. (2011, January 25\u201330). Grasping of unknown objects via curvature maximization using active vision. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6048739"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yu, S., Zhai, D.-H., Wu, H., Yang, H., and Xia, Y. (2020, January 27\u201329). Object recognition and robot grasping technology based on RGB-D data. Proceedings of the 2020 39th Chinese Control Conference (CCC), Shenyang, China.","DOI":"10.23919\/CCC50068.2020.9189078"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bae, J.-H., Jo, H., Kim, D.-W., and Song, J.-B. (2020, January 13\u201316). Grasping System for Industrial Application Using Point Cloud-Based Clustering. Proceedings of the 2020 20th International Conference on Control, Automation and Systems (ICCAS), Busan, Republic of Korea.","DOI":"10.23919\/ICCAS50221.2020.9268284"},{"key":"ref_18","unstructured":"Jeng, K.Y., Liu, Y.C., Liu, Z.Y., Wang, J.W., Chang, Y.L., Su, H.T., and Hsu, W.H. (2020). GDN: A Coarse-To-Fine (C2F) Representation for End-To-End 6-DoF Grasp Detection. arXiv."},{"key":"ref_19","unstructured":"Pas, A.T., and Platt, R. (2018). Using Geometry to Detect Grasp Poses in 3D Point Clouds. Robotics Research, Springer."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liang, H., Ma, X., Li, S., G\u00f6rner, M., Tang, S., Fang, B., and Zhang, J. (2019, January 20\u201324). PointNetGPD: Detecting Grasp Configurations from Point Sets. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794435"},{"key":"ref_21","unstructured":"Mousavian, A., Eppner, C., and Fox, D. (November, January 27). 6-DOF GraspNet: Variational Grasp Generation for Object Manipulation. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_22","unstructured":"Varadarajan, K.M., Zhou, K., and Vincze, M. (2022, December 01). Holistic Shape Detection and Analysis using RGB-D Range Data for Grasping. Available online: https:\/\/www.semanticscholar.org\/paper\/Holistic-Shape-Detection-and-Analysis-using-RGB-D-Varadarajan-Zhou\/bf74c4e2453608042c23ab94a94edc1e68046e19."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1515\/fcds-2017-0011","article-title":"3D Object Detection and Recognition for Robotic Grasping Based on RGB-D Images and Global Features","volume":"42","author":"Czajewski","year":"2017","journal-title":"Found. Comput. Decis. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Kingry, N., Jung, M., Derse, E., and Dai, R. (2018, January 1\u20135). Vision-Based Terrain Classification and Solar Irradiance Mapping for Solar-Powered Robotics. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593635"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kang, H., Zhou, H., Wang, X., and Chen, C. (2020). Real-Time Fruit Recognition and Grasping Estimation for Robotic Apple Harvesting. Sensors, 20.","DOI":"10.3390\/s20195670"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Vignesh, T., Karthikeyan, P., and Sridevi, S. (2017, January 23\u201325). Modeling and trajectory generation of bionic hand for dexterous task. Proceedings of the 2017 IEEE International Conference on Intelligent Techniques in Control, Optimization and Signal Processing (INCOS), Srivilliputtur, India.","DOI":"10.1109\/ITCOSP.2017.8303072"},{"key":"ref_27","unstructured":"Zhu, Q. (2020). Teleoperated Grasping Using an Upgraded Haptic-Enabled Human-Like Robotic Hand and a Cyber Touch Glove. [Ph.D. Thesis, University of Ottawa]."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1016\/j.compag.2019.01.032","article-title":"Effect of fruit and hand characteristics on thumb\u2013index finger power-grasp stability during manual fruit sorting","volume":"157","author":"Chen","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Anzai, Y., Sagara, Y., Kato, R., and Mukai, M. (2019, January 14\u201318). Development of a foldable five-finger robotic hand for assisting in laparoscopic surgery. Proceedings of the 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), New Delhi, India.","DOI":"10.1109\/RO-MAN46459.2019.8956392"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1049\/iet-cvi.2018.5002","article-title":"Deep learning-based grasp-detection method for a five-fingered industrial robot hand","volume":"13","author":"Chao","year":"2019","journal-title":"IET Comput. Vis."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wang, C., Freer, D., Liu, J., and Yang, G.-Z. (2019, January 3\u20138). Vision-based Automatic Control of a 5-Fingered Assistive Robotic Manipulator for Activities of Daily Living. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.","DOI":"10.1109\/IROS40897.2019.8967863"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ji, S.-Q., Huang, M.-B., and Huang, H.-P. (2019). Robot Intelligent Grasp of Unknown Objects Based on Multi-Sensor Information. Sensors, 19.","DOI":"10.3390\/s19071595"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Xu, Z., Yang, C., Wu, W., and Wei, Q. (2020, January 12\u201315). Design of Underwater Humanoid Flexible Manipulator Motion Control System Based on Data Glove. Proceedings of the 2020 6th International Conference on Mechatronics and Robotics Engineering (ICMRE), Barcelona, Spain.","DOI":"10.1109\/ICMRE49073.2020.9065105"},{"key":"ref_34","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv."},{"key":"ref_35","unstructured":"Bochkovskiy, A., Wang, C., and Liao, H. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zheng, S. (2021, January 12\u201314). Network Intrusion Detection Model Based on Convolutional Neural Network. Proceedings of the 2021 IEEE 5th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, China.","DOI":"10.1109\/IAEAC50856.2021.9390930"},{"key":"ref_37","unstructured":"Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2016, January 24\u201329). Pointnet: Deep learning on point sets for 3d classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Rusu, R.B., and Cousins, S. (2011, January 9\u201313). 3d is here: Point cloud library (pcl). Proceedings of the IEEE International Conference on Robotics and Automation, Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980567"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1016\/j.camwa.2010.05.002","article-title":"Modeling of decentralized linear observer and tracker for a class of unknown interconnected large-scale sampled-data nonlinear systems with closed-loop decoupling property","volume":"60","author":"Tsai","year":"2010","journal-title":"Comput. Math. Appl."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"942","DOI":"10.1109\/TSMCB.2003.818562","article-title":"State Observer Based Robust Adaptive Fuzzy Controller for Nonlinear Uncertain and Perturbed Systems","volume":"34","author":"Hamzaoui","year":"2004","journal-title":"IEEE Trans. Syst. Man Cybern. Part B Cybern."},{"key":"ref_41","unstructured":"Shang, S.-Y. (2022, December 01). Eye-in-hand Robotic Arm Gripping System Based on Two Dimensional Object Recognition Using Machine Learning and Three Dimensional Object Posture Estimation. Available online: https:\/\/hdl.handle.net\/11296\/n9nqt6."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1076\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:08:26Z","timestamp":1760119706000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1076"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,17]]},"references-count":41,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23031076"],"URL":"https:\/\/doi.org\/10.3390\/s23031076","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,17]]}}}