{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T14:24:45Z","timestamp":1760711085778,"version":"3.41.2"},"reference-count":40,"publisher":"Emerald","issue":"2","license":[{"start":{"date-parts":[[2022,12,1]],"date-time":"2022-12-01T00:00:00Z","timestamp":1669852800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IR"],"published-print":{"date-parts":[[2023,2,17]]},"abstract":"<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title>\n<jats:p>This study aims to realize natural and effort-saving motion behavior and improve effectiveness for different operators in human\u2013robot force cooperation.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title>\n<jats:p>The parameter of admittance model is identified by deep deterministic policy gradient (DDPG) to realize human\u2013robot force cooperation for different operators in this paper. The movement coupling problem of hybrid robot is solved by realizing position and pose drags. In DDPG, minimum jerk trajectory is selected as the reward objective function, and the variable prioritized experience replay is applied to balance the exploration and exploitation.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Findings<\/jats:title>\n<jats:p>A series of simulations are implemented to validate the superiority and stability of DDPG. Furthermore, three sets of experiments involving mass parameter, damping parameter and DDPG are implemented, the effect of DDPG in real environment is validated and could meet the cooperation demand for different operators.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title>\n<jats:p>DDPG is applied in admittance model identification to realize human\u2013robot force cooperation for different operators. And minimum jerk trajectory is introduced into reward objective to meet requirement of human arm free movements. The algorithm proposed in this paper could be further extended in the other operation task.<\/jats:p>\n<\/jats:sec>","DOI":"10.1108\/ir-05-2022-0135","type":"journal-article","created":{"date-parts":[[2022,11,29]],"date-time":"2022-11-29T10:08:24Z","timestamp":1669716504000},"page":"287-298","source":"Crossref","is-referenced-by-count":2,"title":["Human-robot force cooperation analysis by deep reinforcement learning"],"prefix":"10.1108","volume":"50","author":[{"given":"Shaodong","family":"Li","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaogang","family":"Yuan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongjian","family":"Yu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","published-online":{"date-parts":[[2022,12,1]]},"reference":[{"key":"key2023021612331761700_ref001","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1007\/978-3-030-58147-3_56","article-title":"Adaptive fuzzy sliding mode controller design for a new hand rehabilitation robot","volume":"12272","year":"2020","journal-title":"Lecture Notes in Computer Science"},{"article-title":"Deep reinforcement learning for robotic manipulation-the state of the art","volume-title":"Computer Science","year":"2017","key":"key2023021612331761700_ref002"},{"issue":"4","key":"key2023021612331761700_ref003","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1108\/IR-05-2018-0111","article-title":"Improving stability in physical human\u2013robot interaction by estimating human hand stiffness and a vibration index","volume":"46","year":"2019","journal-title":"Industrial Robot: The International Journal of Robotics Research and Application"},{"issue":"4","key":"key2023021612331761700_ref004","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1108\/IR-12-2019-0258","article-title":"Dynamical system based variable admittance control for physical human-robot interaction","volume":"47","year":"2020","journal-title":"Industrial Robot: The International Journal of Robotics Research and Application"},{"year":"2007","key":"key2023021612331761700_ref005","first-page":"3639"},{"key":"key2023021612331761700_ref006","article-title":"Handover control for human-robot and robot-robot collaboration","volume":"8","year":"2021","journal-title":"Frontiers in Robotics and AI"},{"issue":"5","key":"key2023021612331761700_ref007","first-page":"124","article-title":"Fuzzy sarsa learning and the proof of existence of its stationary points","volume":"10","year":"2008","journal-title":"Asian Journal of Control"},{"year":"2014","key":"key2023021612331761700_ref008","first-page":"4770"},{"issue":"19","key":"key2023021612331761700_ref009","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.ifacol.2015.12.021","article-title":"Learning optimal variable admittance control for rotational motion in human-robot co-manipulation","volume":"48","year":"2015","journal-title":"IFAC-PapersOnLine"},{"year":"2015","key":"key2023021612331761700_ref010","first-page":"1011"},{"issue":"2","key":"key2023021612331761700_ref011","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1109\/TOH.2016.2518670","article-title":"Online stability in human-robot cooperation with admittance control","volume":"9","year":"2016","journal-title":"IEEE Transactions on Haptics"},{"issue":"5","key":"key2023021612331761700_ref012","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1108\/IR-02-2020-0039","article-title":"Technology jump in the industry: human\u2013robot cooperation in production","volume":"47","year":"2020","journal-title":"Industrial Robot: The International Journal of Robotics Research and Application"},{"issue":"4","key":"key2023021612331761700_ref013","article-title":"Variable admittance control based on fuzzy reinforcement learning for minimally invasive surgery manipulator","volume":"17","year":"2017","journal-title":"Sensors"},{"first-page":"446","article-title":"General model of human-robot cooperation using a novel velocity based variable impedance control","year":"2007","key":"key2023021612331761700_ref014"},{"year":"2008","key":"key2023021612331761700_ref015","first-page":"2189"},{"first-page":"165","article-title":"Redundancy resolution in human-robot co-manipulation with cartesian impedance control","year":"2016","key":"key2023021612331761700_ref016"},{"issue":"7","key":"key2023021612331761700_ref017","doi-asserted-by":"crossref","first-page":"1688","DOI":"10.1523\/JNEUROSCI.05-07-01688.1985","article-title":"The coordination of arm movements: an experimentally confirmed mathematical model","volume":"5","year":"1985","journal-title":"The Journal of Neuroscience"},{"article-title":"Human-robot collaboration and machine learning: a systematic review of recent research","volume-title":"arXiv","year":"2021","key":"key2023021612331761700_ref018"},{"first-page":"2829","article-title":"Continuous deep q-learning with model-based acceleration","year":"2016","key":"key2023021612331761700_ref019"},{"issue":"4","key":"key2023021612331761700_ref020","doi-asserted-by":"crossref","first-page":"750","DOI":"10.1109\/TOH.2021.3071626","article-title":"Adaptive human force scaling via admittance control for physical human-robot interaction","volume":"14","year":"2021","journal-title":"IEEE Transactions on Haptics"},{"key":"key2023021612331761700_ref021","doi-asserted-by":"crossref","first-page":"304","DOI":"10.23919\/ACC.1984.4788393","article-title":"Impedance control: an approach to manipulation","volume-title":"1984 American Control Conference","year":"1984"},{"key":"key2023021612331761700_ref022","first-page":"112","article-title":"Cooperative motion control of a robot and a human","volume-title":"3rd IEEE International Workshop on Robot and Human Communication","year":"1994"},{"key":"key2023021612331761700_ref023","first-page":"500","article-title":"Optimal variable impedance control for a robot and its application to lifting an object with a human","volume-title":"11th IEEE International Workshop on Robot and Human Interactive Communication","year":"2002"},{"key":"key2023021612331761700_ref024","first-page":"142","article-title":"Control of a robot handling an object in cooperation with a human","volume-title":"6th IEEE International Workshop on Robot and Human Communication","year":"1997"},{"year":"2012","key":"key2023021612331761700_ref025","first-page":"3903"},{"key":"key2023021612331761700_ref026","doi-asserted-by":"crossref","first-page":"51786","DOI":"10.1109\/ACCESS.2020.2979993","article-title":"A robot-assisted spine surgery system based on intraoperative 2D fluoroscopy navigation","volume":"8","year":"2020","journal-title":"IEEE Access"},{"article-title":"Continuous control with deep reinforcement learning","volume-title":"Computer Science","year":"2015","key":"key2023021612331761700_ref027"},{"key":"key2023021612331761700_ref028","first-page":"2240","article-title":"Human-robot cooperative manipulation with motion estimation","volume":"4","year":"2001","journal-title":"IEEE Conference on Intelligent Robots and Systems"},{"article-title":"Fuzzy variable impedance control based on stiffness identification for human-robot cooperation","volume-title":"3rd International Conference on Advances in Energy, Environment and Chemical Engineering","year":"2017","key":"key2023021612331761700_ref029"},{"article-title":"Playing Atari with deep reinforcement learning","volume-title":"Computer Science","year":"2013","key":"key2023021612331761700_ref030"},{"issue":"7540","key":"key2023021612331761700_ref031","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","year":"2015","journal-title":"Nature"},{"first-page":"676","article-title":"Investigating the impedance characteristic of human arm for development of robots to co-operate with human operators","year":"1999","key":"key2023021612331761700_ref032"},{"issue":"1","key":"key2023021612331761700_ref033","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1007\/s10846-019-00986-3","article-title":"Design of direct teaching behavior of collaborative robot based on force interaction","volume":"96","year":"2019","journal-title":"Journal of Intelligent & Robotic Systems"},{"issue":"4","key":"key2023021612331761700_ref034","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1049\/iet-cta:20070069","article-title":"Frequency domain stability observer and active damping control for stable haptic interaction","volume":"2","year":"2008","journal-title":"IET Control Theory & Applications"},{"article-title":"Prioritized experience replay","volume-title":"Computer Science","year":"2015","key":"key2023021612331761700_ref035"},{"first-page":"605","article-title":"Deterministic policy gradient algorithms","year":"2014","key":"key2023021612331761700_ref036"},{"first-page":"644","article-title":"Variable impedance control based on estimation of human arm stiffness for human-robot cooperative calligraphic task","year":"2017","key":"key2023021612331761700_ref037"},{"key":"key2023021612331761700_ref038","first-page":"2094","article-title":"Deep reinforcement learning with double q-learning","volume-title":"30th Association-for-the-Advancement-of-Artificial-Intelligence Conference on Artificial Intelligence","year":"2016"},{"first-page":"1995","article-title":"Dueling network architectures for deep reinforcement learning","year":"2016","key":"key2023021612331761700_ref039"},{"issue":"1","key":"key2023021612331761700_ref040","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1299\/jamdsm.1.113","article-title":"Stability analysis for impedance control of robot in human-robot cooperative task system","volume":"1","year":"2007","journal-title":"Journal of Advanced Mechanical Design, Systems, and Manufacturing"}],"container-title":["Industrial Robot: the international journal of robotics research and application"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IR-05-2022-0135\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IR-05-2022-0135\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T21:39:10Z","timestamp":1753393150000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/ir\/article\/50\/2\/287-298\/433232"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,1]]},"references-count":40,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,12,1]]},"published-print":{"date-parts":[[2023,2,17]]}},"alternative-id":["10.1108\/IR-05-2022-0135"],"URL":"https:\/\/doi.org\/10.1108\/ir-05-2022-0135","relation":{},"ISSN":["0143-991X","0143-991X"],"issn-type":[{"type":"print","value":"0143-991X"},{"type":"electronic","value":"0143-991X"}],"subject":[],"published":{"date-parts":[[2022,12,1]]}}}