{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T22:33:50Z","timestamp":1791326030406,"version":"4.3.1"},"reference-count":41,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2023,5,31]],"date-time":"2023-05-31T00:00:00Z","timestamp":1685491200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Research Foundation of Korea (NRF)","award":["NRF-2019R1A2C1011270"],"award-info":[{"award-number":["NRF-2019R1A2C1011270"]}]},{"name":"National Research Foundation of Korea (NRF)","award":["2023R1A2C1006745"],"award-info":[{"award-number":["2023R1A2C1006745"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Reinforcement learning is one of the artificial intelligence methods that enable robots to judge and operate situations on their own by learning to perform tasks. Previous reinforcement learning research has mainly focused on tasks performed by individual robots; however, everyday tasks, such as balancing tables, often require cooperation between two individuals to avoid injury when moving. In this research, we propose a deep reinforcement learning-based technique for robots to perform a table-balancing task in cooperation with a human. The cooperative robot proposed in this paper recognizes human behavior to balance the table. This recognition is achieved by utilizing the robot\u2019s camera to take an image of the state of the table, then the table-balance action is performed afterward. Deep Q-network (DQN) is a deep reinforcement learning technology applied to cooperative robots. As a result of learning table balancing, on average, the cooperative robot showed a 90% optimal policy convergence rate in 20 runs of training with optimal hyperparameters applied to DQN-based techniques. In the H\/W experiment, the trained DQN-based robot achieved an operation precision of 90%, thus verifying its excellent performance.<\/jats:p>","DOI":"10.3390\/s23115235","type":"journal-article","created":{"date-parts":[[2023,6,1]],"date-time":"2023-06-01T02:39:47Z","timestamp":1685587187000},"page":"5235","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Table-Balancing Cooperative Robot Based on Deep Reinforcement Learning"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0774-7217","authenticated-orcid":false,"given":"Yewon","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Artificial Intelligence, Kyungpook National University, Daegu 41566, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7124-1141","authenticated-orcid":false,"given":"Dae-Won","family":"Kim","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Chung-Ang University, 84 Heukseok-Ro, Seoul 06974, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0546-7457","authenticated-orcid":false,"given":"Bo-Yeong","family":"Kang","sequence":"additional","affiliation":[{"name":"Department of Robot and Smart System Engineering, Kyungpook National University, Daegu 41566, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,31]]},"reference":[{"key":"ref_1","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_3","unstructured":"Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015, January 6\u201311). Trust region policy optimization. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_4","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_5","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2016, January 2\u20134). Continuous control with deep reinforcement learning. Proceedings of the 4th International Conference on Learning Representations, ICLR 2016\u2014Conference Track Proceedings, San Juan, Puerto Rico. Available online: http:\/\/xxx.lanl.gov\/abs\/1509.02971."},{"key":"ref_6","unstructured":"Merel, J., Tassa, Y., Srinivasan, S., Lemmon, J., Wang, Z., Wayne, G., and Heess, N. (2017). Learning human behaviors from motion capture by adversarial imitation. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1145\/2897824.2925881","article-title":"Terrain-adaptive locomotion skills using deep reinforcement learning","volume":"35","author":"Peng","year":"2016","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3247","DOI":"10.1109\/LRA.2018.2851148","article-title":"Visual navigation for biped humanoid robots using deep reinforcement learning","volume":"3","author":"Leiva","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.robot.2015.04.003","article-title":"The Q-learning obstacle avoidance algorithm based on EKF-SLAM for NAO autonomous walking under unknown environments","volume":"72","author":"Wen","year":"2015","journal-title":"Robot. Auton. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Debnath, S., and Nassour, J. (2014, January 26\u201330). Extending cortical-basal inspired reinforcement learning model with success-failure experience. Proceedings of the 4th International Conference on Development and Learning and on Epigenetic Robotics, Valparaiso, Chile.","DOI":"10.1109\/DEVLRN.2014.6982996"},{"key":"ref_12","unstructured":"A\u015f\u0131k, O., G\u00f6rer, B., and Ak\u0131n, H.L. (2018). End-to-end deep imitation learning: Robot soccer case study. arXiv."},{"key":"ref_13","unstructured":"Danel, M. (2017). POSTER, May, The University of Edinburgh."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Stulp, F., Buchli, J., Theodorou, E., and Schaal, S. (2010, January 6\u20138). Reinforcement learning of full-body humanoid motor skills. Proceedings of the 2010 10th IEEE-RAS International Conference on Humanoid Robots, Nashville, TN, USA.","DOI":"10.1109\/ICHR.2010.5686320"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1177\/0278364917710318","article-title":"Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection","volume":"37","author":"Levine","year":"2018","journal-title":"Int. J. Robot. Res."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Suay, H.B., and Chernova, S. (August, January 31). Effect of human guidance and state space size on interactive reinforcement learning. Proceedings of the 2011 Ro-Man, Atlanta, GA, USA.","DOI":"10.1109\/ROMAN.2011.6005223"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wang, C., Hindriks, K.V., and Babuska, R. (2014, January 18\u201320). Active learning of affordances for robot use of household objects. Proceedings of the 2014 IEEE-RAS International Conference on Humanoid Robots, Piscataway, NJ, USA.","DOI":"10.1109\/HUMANOIDS.2014.7041419"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kato, Y., Kanda, T., and Ishiguro, H. (2015, January 2\u20135). May I help you? Design of human-like polite approaching behavior. Proceedings of the 2015 10th ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Portland, OR, USA.","DOI":"10.1145\/2696454.2696463"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ozaki, Y., Ishihara, T., Matsumura, N., Nunobiki, T., and Yamada, T. (2018, January 27\u201331). Decision-making prediction for human-robot engagement between pedestrian and robot receptionist. Proceedings of the 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Nanjing, China.","DOI":"10.1109\/ROMAN.2018.8525814"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bergstrom, N., Kanda, T., Miyashita, T., Ishiguro, H., and Hagita, N. (2008, January 22\u201326). Modeling of natural human-robot encounters. Proceedings of the 2008 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Nice, France.","DOI":"10.1109\/IROS.2008.4650896"},{"key":"ref_21","unstructured":"Tuyen, N.T.V., Jeong, S., and Chong, N.Y. (2018, January 1\u20135). Emotional bodily expressions for culturally competent robots through long term human-robot interaction. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1109\/THMS.2019.2895753","article-title":"Modeling interaction structure for robot imitation learning of human social behavior","volume":"49","author":"Doering","year":"2019","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Xue, Y., Wang, F., Tian, H., Zhao, M., Li, J., Pan, H., and Dong, Y. (2021, January 23\u201327). Proactive interaction framework for intelligent social receptionist robots. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA48506.2021.9562115"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yal\u00e7inkaya, B., Couceiro, M.S., Soares, S.P., and Valente, A. (2023). Human-Aware Collaborative Robots in the Wild: Coping with Uncertainty in Activity Recognition. Sensors, 23.","DOI":"10.3390\/s23073388"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Khan, I.U., Afzal, S., and Lee, J.W. (2022). Human activity recognition via hybrid deep learning based model. Sensors, 22.","DOI":"10.3390\/s22010323"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Huang, C.M., and Mutlu, B. (2014, January 3\u20136). Learning-based modeling of multimodal behaviors for humanlike robots. Proceedings of the 2014 9th ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Bielefeld, Germany.","DOI":"10.1145\/2559636.2559668"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chi, T.C., Shen, M., Eric, M., Kim, S., and Hakkani-tur, D. (2020, January 7\u201312). Just ask: An interactive learning framework for vision and language navigation. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i03.5627"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Thobbi, A., Gu, Y., and Sheng, W. (2011, January 25\u201330). Using human motion estimation for human-robot cooperative manipulation. Proceedings of the 2011 IEEE\/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6094904"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2030","DOI":"10.1109\/TCYB.2014.2363664","article-title":"An integrated framework for human\u2013robot collaborative manipulation","volume":"45","author":"Sheng","year":"2014","journal-title":"IEEE Trans. Cybern."},{"key":"ref_30","unstructured":"(2020, May 20). Softbank NAO Humanoid Robot. Available online: https:\/\/www.softbankrobotics.com\/."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"6039","DOI":"10.1109\/LRA.2021.3089999","article-title":"Multi-sensor guided hand gesture recognition for a teleoperated robot using a recurrent neural network","volume":"6","author":"Qi","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2199","DOI":"10.1109\/JBHI.2019.2963048","article-title":"A multimodal wearable system for continuous and real-time breathing pattern monitoring during daily activity","volume":"24","author":"Qi","year":"2019","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"388","DOI":"10.1108\/AA-12-2021-0174","article-title":"A human activity-aware shared control solution for medical human\u2013robot interaction","volume":"42","author":"Su","year":"2022","journal-title":"Assem. Autom."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Schwung, D., Csaplar, F., Schwung, A., and Ding, S.X. (2017, January 8). An application of reinforcement learning algorithms to industrial multi-robot stations for cooperative handling operation. Proceedings of the 2017 IEEE 15th International Conference on Industrial Informatics (INDIN), Bangalore, India.","DOI":"10.1109\/INDIN.2017.8104770"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3303848","article-title":"Cooperative heterogeneous multi-robot systems: A survey","volume":"52","author":"Rizk","year":"2019","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.cirp.2011.03.017","article-title":"Dual arm robot for flexible and cooperative assembly","volume":"60","author":"Schreck","year":"2011","journal-title":"CIRP Ann."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Han, R., Chen, S., and Hao, Q. (August, January 31). Cooperative multi-robot navigation in dynamic environment with deep reinforcement learning. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197209"},{"key":"ref_38","unstructured":"(2020, May 04). The Vicon Motion Capture System. Available online: https:\/\/www.vicon.com\/."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"404","DOI":"10.7746\/jkros.2020.15.4.404","article-title":"Cooperative Robot for Table Balancing Using Q-learning","volume":"15","author":"Kim","year":"2020","journal-title":"J. Korea Robot. Soc."},{"key":"ref_40","unstructured":"Kim, Y. (2020). Cooperative Robot Development Based on Deep Reinforcement Learning for Table Balancing. [Master\u2019s Thesis, School of Mechanical Engineering]."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Jeon, H., Kim, Y., and Kang, B. (2021, January 1\u20136). Interactive Reinforcement Learning for Table Balancing Robot. Proceedings of the the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics (ACL-IJCNLP 2021) SpLU-RoboNLP Workshop, Bangkok, Thailand.","DOI":"10.18653\/v1\/2021.splurobonlp-1.8"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/11\/5235\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:46:16Z","timestamp":1760125576000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/11\/5235"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,31]]},"references-count":41,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["s23115235"],"URL":"https:\/\/doi.org\/10.3390\/s23115235","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,31]]}}}