{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T13:23:07Z","timestamp":1777555387693,"version":"3.51.4"},"reference-count":36,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2021,1,19]],"date-time":"2021-01-19T00:00:00Z","timestamp":1611014400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this research, we focused on Human-Robot collaboration. There were two goals: (1) to develop and evaluate a real-time Human-Robot collaborative system, and (2) to achieve concrete tasks such as collaborative peg-in-hole using the developed system. We proposed an algorithm for visual sensing and robot hand control to perform collaborative motion, and we analyzed the stability of the collaborative system and a so-called collaborative error caused by image processing and latency. We achieved collaborative motion using this developed system and evaluated the collaborative error on the basis of the analysis results. Moreover, we aimed to realize a collaborative peg-in-hole task that required a system with high speed and high accuracy. To achieve this goal, we analyzed the conditions required for performing the collaborative peg-in-hole task from the viewpoints of geometric, force and posture conditions. Finally, in this work, we show the experimental results and data of the collaborative peg-in-hole task, and we examine the effectiveness of our collaborative system.<\/jats:p>","DOI":"10.3390\/s21020663","type":"journal-article","created":{"date-parts":[[2021,1,20]],"date-time":"2021-01-20T03:34:25Z","timestamp":1611113665000},"page":"663","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Development of a Real-Time Human-Robot Collaborative System Based on 1 kHz Visual Feedback Control and Its Application to a Peg-in-Hole Task"],"prefix":"10.3390","volume":"21","author":[{"given":"Yuji","family":"Yamakawa","sequence":"first","affiliation":[{"name":"Interfaculty Initiative in Information Studies, The University of Tokyo, Tokyo 153-8505, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yutaro","family":"Matsui","sequence":"additional","affiliation":[{"name":"Graduate School of Information Science and Technology, The University of Tokyo, Tokyo 113-8656, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Masatoshi","family":"Ishikawa","sequence":"additional","affiliation":[{"name":"Information Technology Center, The University of Tokyo, Tokyo 113-8656, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chandrasekaran, B., and Conrad, M.J. (2015, January 9\u201312). Human-robot collaboration: A survey. Proceedings of the IEEE SoutheastCon 2015, Fort Lauderdale, FL, USA.","DOI":"10.1109\/SECON.2015.7132964"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1109\/TMECH.2006.871087","article-title":"Biomechanical design of the berkeley lower extremity exoskeleton (bleex)","volume":"11","author":"Zoss","year":"2006","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"ref_3","unstructured":"Hayashibara, Y., Takubo, T., Sonoda, Y., Arai, H., and Tanie, K. (1999, January 17\u201321). Assist system for carrying a long object with a human-analysis of a human cooperative behavior in the vertical direction. Proceedings of the 1999 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Kyongju, Korea."},{"key":"ref_4","unstructured":"Yokoyama, K., Handa, H., Isozumi, T., Fukase, Y., Kaneko, K., Kanehiro, F., Kawai, Y., Tomita, F., and Hirukawa, H. (2003, January 14\u201319). Cooperative works by a human and a humanoid robot. Proceedings of the 2003 IEEE International Conference on Robotics and Automation, Taipei, Taiwan."},{"key":"ref_5","unstructured":"Kosuge, K., Kakuya, H., and Hirata, Y. (2001, January 7\u201310). Control algorithm of dual arms mobile robot for cooperative works with human. Proceedings of the 2001 IEEE International Conference on Systems, Man, and Cybernetics, Tucson, AZ, USA."},{"key":"ref_6","unstructured":"Suda, R., and Kosuge, K. (October, January 30). Handling of object by mobile robot helper in cooperation with a human using visual information and force information. Proceedings of the 2002 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Lausanne, Switzerland."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"St\u00fcckler, J., and Behnke, S. (2011, January 26\u201328). Following human guidance to cooperatively carry a large object. Proceedings of the 11th IEEE-RAS International Conference on Humanoid Robots, Bled, Slovenia.","DOI":"10.1109\/Humanoids.2011.6100917"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ant\u00e3o, L., Pinto, R., Reis, J., Gon\u00e7alves, G., and Pereira, F.L. (2018, January 4\u20136). Cooperative human-machine interaction in industrial environments. Proceedings of the 2018 13th APCA International Conference on Automatic Control and Soft Computing, Ponta Delgada, Portugal.","DOI":"10.1109\/CONTROLO.2018.8514549"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Teke, B., Lanz, M., K\u00e4m\u00e4r\u00e4inen, J., and Hietanen, A. (2018, January 2\u20134). Real-time and robust collaborative robot motion control with Microsoft Kinect\u00ae v2. Proceedings of the 2018 14th IEEE\/ASME International Conference on Mechatronic and Embedded Systems and Applications, Oulu, Finland.","DOI":"10.1109\/MESA.2018.8449156"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"\u00c7oban, M., and Gelen, G. (2018, January 25\u201327). Realization of human-robot collaboration in hybrid assembly systems by using wearable technology. Proceedings of the 2018 6th International Conference on Control Engineering & Information Technology, Istanbul, Turkey.","DOI":"10.1109\/CEIT.2018.8751814"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1109\/TASE.2018.2840345","article-title":"Facilitating human-robot collaborative tasks by teaching-learning-collaboration from human demonstrations","volume":"16","author":"Wang","year":"2019","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Shayganfar, M., Rich, C., and Sidner, C. (2019, January 25\u201327). Appraisal algorithms for relevance and controllability in human-robot collaboration. Proceedings of the 2019 IEEE International Conference on Humanized Computing and Communication, Laguna Hills, CA, USA.","DOI":"10.1109\/HCC46620.2019.00013"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Scimmi, L.S., Melchiorre, M., Mauro, S., and Pastorelli, S. (2019, January 2\u20134). Experimental real-time setup for vision driven hand-over with a collaborative robot. Proceedings of the 2019 International Conference on Control, Automation and Diagnosis, Grenoble, France.","DOI":"10.1109\/ICCAD46983.2019.9037961"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Galin, R., Meshcheryakov, R., and Samoshina, A. (2020, January 6\u201312). Mathematical modelling and simulation of human-robot collaboration. Proceedings of the 2020 International Russian Automation Conference, Sochi, Russia.","DOI":"10.1109\/RusAutoCon49822.2020.9208040"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Darvish, K., Simetti, E., Mastrogiovanni, F., and Casalino, G. (2020). A Hierarchical architecture for human-robot cooperation processes. IEEE Trans. Robot.","DOI":"10.1109\/TRO.2020.3033715"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Muthugala, M.A.V.J., Srimal, P.H.D.A.S., and Jayasekara, A.G.B.P. (2020). Improving robot\u2019s perception of uncertain spatial descriptors in navigational instructions by evaluating influential gesture notions. J. Multimodal User Interfaces.","DOI":"10.1007\/s12193-020-00328-w"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2030","DOI":"10.1109\/TCYB.2014.2363664","article-title":"An integrated framework for human-robot collaborative manipulation","volume":"45","author":"Sheng","year":"2015","journal-title":"IEEE Trans. Cybern."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Lemmerz, K., Glogowski, P., Kleineberg, P., Hypki, A., and Kuhlenk\u00f6tter, B. (2019, January 25\u201327). A hybrid collaborative operation for human-robot interaction supported by machine learning. Proceedings of the 2019 12th International Conference on Human System Interaction, Richmond, VA, USA.","DOI":"10.1109\/HSI47298.2019.8942606"},{"key":"ref_19","unstructured":"(2020, December 15). Ishikawa Group Laboratory. Available online: http:\/\/www.k2.t.u-tokyo.ac.jp\/fusion\/Janken\/index-e.html."},{"key":"ref_20","unstructured":"(2020, December 15). Janken (rock-paper-scissors) robot with 100% winning rate (human-machine cooperation system) in Yamakawa Laboratory. Available online: http:\/\/www.hfr.iis.u-tokyo.ac.jp\/research\/Janken\/index-e.html."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Yamakawa, Y., Kuno, K., and Ishikawa, M. (2015, January 7\u201311). Human-robot cooperative task realization using high-speed robot hand system. Proceedings of the 2015 IEEE International Conference on Advanced Intelligent Mechatronics, Busan, Korea.","DOI":"10.1109\/AIM.2015.7222545"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Matsui, Y., Yamakawa, Y., and Ishikawa, M. (2017, January 27\u201330). Cooperative operation between a human and a robot based on real-time measurement of location and posture of target object by high-speed vision. Proceedings of the 2017 IEEE International Conference on Control Technology and Applications, Mauna Lani, HI, USA.","DOI":"10.1109\/CCTA.2017.8062504"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yamakawa, Y., Matsui, Y., and Ishikawa, M. (2018, January 25\u201327). Human-robot collaborative manipulation using a high-speed robot hand and a high-speed camera. Proceedings of the 2018 IEEE International Conference on Cyborg and Bionic Systems, Shenzhen, China.","DOI":"10.1109\/CBS.2018.8612172"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yamakawa, Y., Matsui, Y., and Ishikawa, M. (2018, January 12\u201315). Development and analysis of a high-speed human-robot collaborative system and its application. Proceedings of the 2018 IEEE International Conference on Robotics and Biomimetics, Kuala Lumpur, Malaysia.","DOI":"10.1109\/ROBIO.2018.8665156"},{"key":"ref_25","unstructured":"Namiki, A., Imai, Y., Ishikawa, M., and Kaneko, M. (2003, January 27\u201331). Development of a high-speed multifingered hand system and its application to catching. Proceedings of the 2003 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Las Vegas, NV, USA."},{"key":"ref_26","unstructured":"(2020, December 15). MIKROTRON. Available online: http:\/\/www.mikrotron.de\/."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A flexible new technique for camera calibration","volume":"22","author":"Zhang","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_28","unstructured":"Ishii, I., Nakabo, Y., and Ishikawa, M. (1996, January 22\u201328). Target tracking algorithm for 1 ms visual feedback system using massively parallel processing. Proceedings of the 1996 IEEE International Conference on Robotics and Automation, Minneapolis, MN, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Peternel, L., Tsagarakis, N., Caldwell, D., and Ajoudani, A. (2016, January 15\u201317). Adaptation of robot physical behaviour to human fatigue in human-robot co-manipulation. Proceedings of the 2016 IEEE-RAS 16th International Conference on Humanoid Robots, Cancun, Mexico.","DOI":"10.1109\/HUMANOIDS.2016.7803320"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1109\/TNSRE.2017.2694553","article-title":"A human-robot co-manipulation approach based on human sensorimotor information","volume":"25","author":"Peternel","year":"2017","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1177\/1729881418773190","article-title":"Calibrating intuitive and natural human-robot interaction and performance for power-assisted heavy object manipulation using cognition-based intelligent admittance control schemes","volume":"15","author":"Rahman","year":"2018","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_32","unstructured":"(2020, December 15). Dynamic Human-Robot Interactive System in Yamakawa Laboratory. Available online: http:\/\/www.hfr.iis.u-tokyo.ac.jp\/research\/collaboration\/index-e.html."},{"key":"ref_33","unstructured":"(2020, December 15). YouTube. Available online: http:\/\/www.youtube.com\/watch?v=xB9-vEiZwKY."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1115\/1.3149634","article-title":"Quasi-static assembly of compliantly supported rigid parts","volume":"104","author":"Whitney","year":"1982","journal-title":"J. Dyn. Syst. Meas. Control"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1299\/kikaic.64.604","article-title":"Kinematic evaluation of task-performance of a manipulator for a peg-in-hole task: A method of evaluation including a condition for avoidance of jamming on a chamfer)","volume":"64","author":"Hara","year":"1998","journal-title":"Trans. Jpn. Soc. Mech. Eng. Ser. C"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"26435","DOI":"10.1109\/ACCESS.2017.2777823","article-title":"Enhancing user satisfaction by adapting robot\u2019s perception of uncertain information based on environment and user feedback","volume":"5","author":"Muthugala","year":"2017","journal-title":"IEEE Access"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/2\/663\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:12:58Z","timestamp":1760159578000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/2\/663"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,19]]},"references-count":36,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["s21020663"],"URL":"https:\/\/doi.org\/10.3390\/s21020663","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,19]]}}}