{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T06:16:49Z","timestamp":1780640209876,"version":"3.54.1"},"reference-count":34,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2017,4,12]],"date-time":"2017-04-12T00:00:00Z","timestamp":1491955200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The National High Technology Research and Development Program of China","award":["2015AA043201"],"award-info":[{"award-number":["2015AA043201"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In order to get natural and intuitive physical interaction in the pose adjustment of the minimally invasive surgery manipulator, a hybrid variable admittance model based on Fuzzy Sarsa(\u03bb)-learning is proposed in this paper. The proposed model provides continuous variable virtual damping to the admittance controller to respond to human intentions, and it effectively enhances the comfort level during the task execution by modifying the generated virtual damping dynamically. A fuzzy partition defined over the state space is used to capture the characteristics of the operator in physical human-robot interaction. For the purpose of maximizing the performance index in the long run, according to the identification of the current state input, the virtual damping compensations are determined by a trained strategy which can be learned through the experience generated from interaction with humans, and the influence caused by humans and the changing dynamics in the robot are also considered in the learning process. To evaluate the performance of the proposed model, some comparative experiments in joint space are conducted on our experimental minimally invasive surgical manipulator.<\/jats:p>","DOI":"10.3390\/s17040844","type":"journal-article","created":{"date-parts":[[2017,4,12]],"date-time":"2017-04-12T10:15:06Z","timestamp":1491992106000},"page":"844","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":56,"title":["Variable Admittance Control Based on Fuzzy Reinforcement Learning for Minimally Invasive Surgery Manipulator"],"prefix":"10.3390","volume":"17","author":[{"given":"Zhijiang","family":"Du","sequence":"first","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, 2 Yikuang Street, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, 2 Yikuang Street, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhiyuan","family":"Yan","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, 2 Yikuang Street, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Dong","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, 2 Yikuang Street, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1236-8350","authenticated-orcid":false,"given":"Weidong","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics and System, Harbin Institute of Technology, 2 Yikuang Street, Harbin 150080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2017,4,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1119","DOI":"10.1016\/j.jtcvs.2006.10.078","article-title":"Endoscopic robotic mitral valve surgery","volume":"133","author":"Murphy","year":"2007","journal-title":"J. Thorac. Cardiovasc. Surg."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Rosen, J., Hannaford, B., and Satava, R.M. (2011). Applications of surgical robotics in general surgery. Surgical Robotics: Systems, Applications, and Visions, Springer.","DOI":"10.1007\/978-1-4419-1126-1"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1687814014568541","DOI":"10.1177\/1687814014568541","article-title":"Dimensional optimization of a minimally invasive surgical robot system based on NSGA-II algorithm","volume":"7","author":"Wang","year":"2015","journal-title":"Adv. Mech. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1016\/j.mechmachtheory.2007.03.003","article-title":"An atlas of physical human\u2013robot interaction","volume":"43","author":"Siciliano","year":"2008","journal-title":"Mech. Mach. Theory"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Mitchell, B., Koo, J., Iordachita, I., Kazanzides, P., Kapoor, A., Handa, J., and Taylor, R. (2007, January 10\u201314). Development and application of a new steady-hand manipulator for retinal surgery. Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Roma, Italy.","DOI":"10.1109\/ROBOT.2007.363056"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Lopez Infante, M., and Kyrki, V. (2011, January 6\u20139). Usability of force-based controllers in physical human-robot interaction. Proceedings of the ACM 6th International Conference on Human-Robot Interaction, Lausanne, Switzerland.","DOI":"10.1145\/1957656.1957790"},{"key":"ref_7","unstructured":"Pallegedara, A., Matsuda, Y., Egashira, N., Sugi, T., and Goto, S. (2012, January 17\u201321). Simulation and analysis of dynamics of the force-free control for industrial robot arms. Proceedings of the 2012 12th IEEE International Conference on Control, Automation and Systems (ICCAS), Jeju Island, Korea."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1016\/j.conengprac.2006.11.002","article-title":"Forcefree control with independent compensation for industrial articulated robot arms","volume":"15","author":"Goto","year":"2007","journal-title":"Control Eng. Pract."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1115\/1.3140702","article-title":"Impedance control: An approach to manipulation: Part I theory; Part II implementation; Part III applications","volume":"107","author":"Hogan","year":"1985","journal-title":"J. Dyn. Syst. Meas. Control"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1016\/j.rcim.2011.01.003","article-title":"Assisting manual welding with robot","volume":"27","author":"Erden","year":"2011","journal-title":"Rob. Comput. Integr. Manuf."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lecours, A., Mayer-St-Onge, B., and Gosselin, C. (2012, January 14\u201318). Variable admittance control of a four-degree-of-freedom intelligent assist device. Proceedings of the 2012 IEEE International Conference on Robotics and Automation (ICRA), Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6224586"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"850","DOI":"10.1109\/TRO.2015.2430053","article-title":"Variable Impedance Control of Redundant Manipulators for Intuitive Human\u2013Robot Physical Interaction","volume":"31","author":"Ficuciello","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Duchaine, V., and Gosselin, C.M. (2007, January 22\u201324). General model of human-robot cooperation using a novel velocity based variable impedance control. Proceedings of the IEEE Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC\u201907), Tsukuba, Japan.","DOI":"10.1109\/WHC.2007.59"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Duchaine, V., and Gosselin, C.M. (2008, January 19\u201323). Investigation of human-robot interaction stability using Lyapunov theory. Proceedings of the IEEE International Conference on Robotics and Automation, ICRA 2008, Pasadena, CA, USA.","DOI":"10.1109\/ROBOT.2008.4543531"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TSMCA.2011.2162501","article-title":"Operability of joystick-type steering device considering human arm impedance characteristics","volume":"42","author":"Kim","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Humans"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Mitsantisuk, C., Ohishi, K., and Katsura, S. (2011, January 13\u201315). Variable mechanical stiffness control based on human stiffness estimation. Proceedings of the 2011 IEEE International Conference on Mechatronics (ICM), Istanbul, Turkey.","DOI":"10.1109\/ICMECH.2011.5971211"},{"key":"ref_17","unstructured":"Tsumugiwa, T., Yokogawa, R., and Hara, K. (2002, January 11\u201315). Variable impedance control based on estimation of human arm stiffness for human-robot cooperative calligraphic task. Proceedings of the IEEE International Conference on Robotics and Automation, ICRA\u201902, Washington, DC, USA."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1080\/01691864.2014.900162","article-title":"Improved stability of haptic human\u2013robot interfaces using measurement of human arm stiffness","volume":"28","author":"Gallagher","year":"2014","journal-title":"Adv. Robot."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Corteville, B., Aertbeli\u00ebn, E., Bruyninckx, H., De Schutter, J., and Van Brussel, H. (2007, January 10\u201314). Human-inspired robot assistant for fast point-to-point movements. Proceedings 2007 IEEE International Conference on Robotics and Automation, Roma, Italy.","DOI":"10.1109\/ROBOT.2007.364036"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Dimeas, F., and Aspragathos, N. (2014, January 14\u201318). Fuzzy learning variable admittance control for human-robot cooperation. Proceedings of the 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA.","DOI":"10.1109\/IROS.2014.6943240"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1688","DOI":"10.1523\/JNEUROSCI.05-07-01688.1985","article-title":"The coordination of arm movements: an experimentally confirmed mathematical model","volume":"5","author":"Flash","year":"1985","journal-title":"J. Neurosci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2012\/365067","article-title":"Adaptive impedance control to enhance human skill on a haptic interface system","volume":"1","author":"Suzuki","year":"2012","journal-title":"J. Control Sci. Eng."},{"key":"ref_23","unstructured":"Furuta, K., Kado, Y., and Shiratori, S. (2006, January 4\u20136). Assisting control in Human Adaptive Mechatronics\u2014Single ball juggling. Proceedings of the IEEE International Conference on Control Applications, Munich, Germany."},{"key":"ref_24","unstructured":"Kurihara, K., Suzuki, S., Harashima, F., and Furuta, K. (2004, January 2\u20136). Human adaptive mechatronics (HAM) for haptic system. Proceedings of the 30th Annual Conference of IEEE Industrial Electronics Society, IECON 2004, Busan, Korea."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"918","DOI":"10.1109\/TRO.2011.2158251","article-title":"Human-like adaptation of force and impedance in stable and unstable interactions","volume":"27","author":"Yang","year":"2011","journal-title":"IEEE Trans. Robot."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"847","DOI":"10.1109\/TRO.2013.2256311","article-title":"Transferring human impedance behavior to heterogeneous variable impedance actuators","volume":"29","author":"Howard","year":"2013","journal-title":"IEEE Trans. Robot."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Kronander, K., and Billard, A. (2012, January 14\u201318). Online learning of varying stiffness through physical human-robot interaction. Proceedings of the 2012 IEEE International Conference on Robotics and Automation (ICRA), Saint Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6224877"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1907","DOI":"10.1109\/TMECH.2014.2309118","article-title":"Frequency-shaped impedance control for safe human\u2013robot interaction in reference tracking application","volume":"19","author":"Oh","year":"2014","journal-title":"IEEE\/ASME Trans. Mech."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1109\/TCYB.2015.2412554","article-title":"Optimized assistive human\u2013robot interaction using reinforcement learning","volume":"46","author":"Modares","year":"2016","journal-title":"IEEE Trans. Cybern."},{"key":"ref_30","unstructured":"Busoniu, L., Babuska, R., De Schutter, B., and Ernst, D. (2010). Reinforcement Learning and Dynamic Programming Using Function Approximators, CRC Press."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Dimeas, F., and Aspragathos, N. (October, January 28). Reinforcement learning of variable admittance control for human-robot co-manipulation. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353494"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1109\/5326.704563","article-title":"Fuzzy inference system learning by reinforcement methods","volume":"28","author":"Jouffe","year":"1998","journal-title":"IEEE Trans. Syst. Man Cybern. Part C Appl. Rev."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1002\/asjc.54","article-title":"Fuzzy Sarsa learning and the proof of existence of its stationary points","volume":"10","author":"Derhami","year":"2008","journal-title":"Asian J. Control"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/4\/844\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:32:32Z","timestamp":1760207552000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/4\/844"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,4,12]]},"references-count":34,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2017,4]]}},"alternative-id":["s17040844"],"URL":"https:\/\/doi.org\/10.3390\/s17040844","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,4,12]]}}}