{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T15:59:44Z","timestamp":1778687984942,"version":"3.51.4"},"reference-count":34,"publisher":"World Scientific Pub Co Pte Ltd","issue":"03n04","funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000196","name":"Canada Foundation for Innovation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000196","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000024","name":"Canadian Institutes of Health Research","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000024","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004543","name":"China Scholarship Council","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004543","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Alberta Jobs, Economy and Innovation Ministry"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Med. Robot. Res."],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:p> Recent studies in surgical robotics have focused on automating common surgical subtasks such as grasping and manipulation using deep reinforcement learning (DRL). In this work, we consider surgical endoscopic camera control for object tracking e.g. using the endoscopic camera manipulator (ECM) from the da Vinci Research Kit (dVRK) (Intuitive Inc., Sunnyvale, CA, USA) as a typical surgical robot learning task. A DRL policy for controlling the robot joint space movements is first trained in a simulation environment and then continues the learning in the real world. To speed up training and avoid significant failures (in this case, losing view of the object), human interventions are incorporated into the training process and regular DRL is combined with generative adversarial imitation learning (GAIL) to encourage imitating human behaviors. Experiments show that an average reward of 159.8 can be achieved within 1000 steps compared to only 121.8 without human interventions, and the view of the moving object is lost only twice during the training process out of 3 trials. These results show that human interventions can improve learning speed and significantly reduce failures during the training process. <\/jats:p>","DOI":"10.1142\/s2424905x23400044","type":"journal-article","created":{"date-parts":[[2023,10,14]],"date-time":"2023-10-14T04:11:33Z","timestamp":1697256693000},"source":"Crossref","is-referenced-by-count":8,"title":["Robot Learning Incorporating Human Interventions in the Real World for Autonomous Surgical Endoscopic Camera Control"],"prefix":"10.1142","volume":"08","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2989-9991","authenticated-orcid":false,"given":"Yafei","family":"Ou","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Alberta, 9211-116 Street NW, Edmonton, AB, T6G 1H9, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4406-915X","authenticated-orcid":false,"given":"Sadra","family":"Zargarzadeh","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Alberta, 9211-116 Street NW, Edmonton, AB, T6G 1H9, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7427-6961","authenticated-orcid":false,"given":"Mahdi","family":"Tavakoli","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Alberta, 9211-116 Street NW, Edmonton, AB, T6G 1H9, Canada"}]}],"member":"219","published-online":{"date-parts":[[2023,11,15]]},"reference":[{"key":"S2424905X23400044BIB001","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6907819"},{"key":"S2424905X23400044BIB002","doi-asserted-by":"publisher","DOI":"10.1109\/IROS45743.2020.9341710"},{"key":"S2424905X23400044BIB003","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9561673"},{"key":"S2424905X23400044BIB004","doi-asserted-by":"publisher","DOI":"10.1109\/RO-MAN47096.2020.9223543"},{"key":"S2424905X23400044BIB005","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2023.3254860"},{"key":"S2424905X23400044BIB006","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2983149"},{"key":"S2424905X23400044BIB007","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3052391"},{"key":"S2424905X23400044BIB009","first-page":"2067","volume-title":"International Conference on Autonomous Agents and MultiAgent Systems","author":"Saunders W.","year":"2018"},{"key":"S2424905X23400044BIB010","first-page":"410","volume-title":"Conference on Robot Learning","author":"Wang F.","year":"2018"},{"key":"S2424905X23400044BIB011","doi-asserted-by":"publisher","DOI":"10.1016\/j.eng.2022.05.017"},{"key":"S2424905X23400044BIB012","first-page":"1","author":"Wu J.","year":"2022","journal-title":"IEEE Trans. on Neural Netw. Learn. Syst."},{"key":"S2424905X23400044BIB013","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2023.3284380"},{"key":"S2424905X23400044BIB014","doi-asserted-by":"publisher","DOI":"10.1109\/ISMR57123.2023.10130214"},{"key":"S2424905X23400044BIB015","first-page":"6434","volume-title":"2014 IEEE Int. Conf. Robotics and Automation","author":"Kazanzides P.","year":"2014"},{"key":"S2424905X23400044BIB016","volume":"26","author":"Griffith S.","year":"2013","journal-title":"Adv. Neural Inform. Process. Syst."},{"issue":"1","key":"S2424905X23400044BIB017","first-page":"1545","volume":"32","author":"Warnell G.","year":"2018","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"S2424905X23400044BIB018","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3070252"},{"key":"S2424905X23400044BIB019","first-page":"332","volume-title":"Conference on Robot Learning","author":"Xu Y.","year":"2022"},{"key":"S2424905X23400044BIB022","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9812379"},{"key":"S2424905X23400044BIB023","doi-asserted-by":"publisher","DOI":"10.1109\/ISMR48346.2021.9661514"},{"key":"S2424905X23400044BIB024","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2010.5650301"},{"key":"S2424905X23400044BIB025","first-page":"1","volume-title":"ISR\/Robotik 2014; 41st Int. Symp. Robotics","author":"Bihlmaier A.","year":"2014"},{"key":"S2424905X23400044BIB026","doi-asserted-by":"publisher","DOI":"10.1109\/TMRB.2019.2949881"},{"key":"S2424905X23400044BIB027","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2965067"},{"key":"S2424905X23400044BIB028","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2021.707704"},{"key":"S2424905X23400044BIB029","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2017.11.011"},{"key":"S2424905X23400044BIB030","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989488"},{"key":"S2424905X23400044BIB031","doi-asserted-by":"publisher","DOI":"10.3390\/electronics8020224"},{"key":"S2424905X23400044BIB033","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9812010"},{"key":"S2424905X23400044BIB034","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3173442"},{"key":"S2424905X23400044BIB036","volume":"29","author":"Ho J.","year":"2016","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"S2424905X23400044BIB038","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.01.016"},{"key":"S2424905X23400044BIB039","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-022-06144-5"},{"key":"S2424905X23400044BIB040","doi-asserted-by":"publisher","DOI":"10.1109\/IROS51168.2021.9635867"}],"container-title":["Journal of Medical Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S2424905X23400044","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,2]],"date-time":"2024-02-02T09:45:26Z","timestamp":1706867126000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S2424905X23400044"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,15]]},"references-count":34,"journal-issue":{"issue":"03n04","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["10.1142\/S2424905X23400044"],"URL":"https:\/\/doi.org\/10.1142\/s2424905x23400044","relation":{},"ISSN":["2424-905X","2424-9068"],"issn-type":[{"value":"2424-905X","type":"print"},{"value":"2424-9068","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,15]]},"article-number":"2340004"}}