{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,27]],"date-time":"2026-04-27T20:53:41Z","timestamp":1777323221943,"version":"3.51.4"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"8","license":[{"start":{"date-parts":[[2023,10,21]],"date-time":"2023-10-21T00:00:00Z","timestamp":1697846400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,10,21]],"date-time":"2023-10-21T00:00:00Z","timestamp":1697846400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001779","name":"Monash University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001779","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Auton Robot"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Novice pilots find it difficult to operate and land unmanned aerial vehicles (UAVs), due to the complex UAV dynamics, challenges in depth perception, lack of expertise with the control interface and additional disturbances from the ground effect. Therefore we propose a shared autonomy approach to assist pilots in safely landing a UAV under conditions where depth perception is difficult and safe landing zones are limited. Our approach is comprised of two modules: a perception module that encodes information onto a compressed latent representation using two RGB-D cameras and a policy module that is trained with the reinforcement learning algorithm TD3 to discern the pilot\u2019s intent and to provide control inputs that augment the user\u2019s input to safely land the UAV. The policy module is trained in simulation using a population of simulated users. Simulated users are sampled from a parametric model with four parameters, which model a pilot\u2019s tendency to conform to the assistant, proficiency, aggressiveness and speed. We conduct a user study (<jats:inline-formula><jats:alternatives><jats:tex-math>$$n=28$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mrow>\n                    <mml:mi>n<\/mml:mi>\n                    <mml:mo>=<\/mml:mo>\n                    <mml:mn>28<\/mml:mn>\n                  <\/mml:mrow>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula>) where human participants were tasked with landing a physical UAV on one of several platforms under challenging viewing conditions. The assistant, trained with only simulated user data, improved task success rate from 51.4 to 98.2% despite being unaware of the human participants\u2019 goal or the structure of the environment a priori. With the proposed assistant, regardless of prior piloting experience, participants performed with a proficiency greater than the most experienced unassisted participants.\n<\/jats:p>","DOI":"10.1007\/s10514-023-10143-3","type":"journal-article","created":{"date-parts":[[2023,10,21]],"date-time":"2023-10-21T14:02:30Z","timestamp":1697896950000},"page":"1419-1438","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Reinforcement learning for shared autonomy drone landings"],"prefix":"10.1007","volume":"47","author":[{"given":"Kal","family":"Backman","sequence":"first","affiliation":[]},{"given":"Dana","family":"Kuli\u0107","sequence":"additional","affiliation":[]},{"given":"Hoam","family":"Chung","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,10,21]]},"reference":[{"issue":"9","key":"10143_CR1","doi-asserted-by":"publisher","first-page":"3312","DOI":"10.1109\/TMC.2021.3051273","volume":"21","author":"A Albanese","year":"2022","unstructured":"Albanese, A., Sciancalepore, V., & Costa-P\u00e9rez, X. (2022). Sardo: An automated searchand- rescue drone-based solution for victims localization. IEEE Transactions on Mobile Computing, 21(9), 3312\u20133325.","journal-title":"IEEE Transactions on Mobile Computing"},{"issue":"5","key":"10143_CR2","doi-asserted-by":"publisher","first-page":"874","DOI":"10.1002\/rob.21858","volume":"36","author":"T Baca","year":"2019","unstructured":"Baca, T., Stepan, P., Spurny, V., Hert, D., Penicka, R., Saska, M., & Kumar, V. (2019). Autonomous landing on a moving vehicle with an unmanned aerial vehicle. Journal of Field Robotics, 36(5), 874\u2013891.","journal-title":"Journal of Field Robotics"},{"issue":"2","key":"10143_CR3","doi-asserted-by":"publisher","first-page":"3192","DOI":"10.1109\/LRA.2021.3062572","volume":"6","author":"K Backman","year":"2021","unstructured":"Backman, K., Kuli\u0107, D., & Chung, H. (2021). Learning to assist drone landings. IEEE Robotics and Automation Letters, 6(2), 3192\u20133199.","journal-title":"IEEE Robotics and Automation Letters"},{"key":"10143_CR4","doi-asserted-by":"crossref","unstructured":"Bousmalis, K., Irpan, A., Wohlhart, P., Bai, Y., Kelcey, M., Kalakrishnan, M., & Vanhoucke, V. (2018). Using simulation and domain adaptation to improve efficiency of deep robotic grasping. In IEEE International Conference on Robotics and Automation, (pp. 4243-4250).","DOI":"10.1109\/ICRA.2018.8460875"},{"key":"10143_CR5","doi-asserted-by":"crossref","unstructured":"Carney, E., Castano, L., & Xu, H. (2019). Determination of safe landing zones for an autonomous uas using elevation and population density data. Aiaa scitech 2019 forum (p. 1-16).","DOI":"10.2514\/6.2019-1060"},{"key":"10143_CR6","doi-asserted-by":"crossref","unstructured":"Curran, W., Pocius, R., & Smart, W.D. (2017). Neural networks for incremental dimensionality reduced reinforcement learning. In: 2017 ieee\/rsj iros (pp. 1559-1565).","DOI":"10.1109\/IROS.2017.8205962"},{"issue":"4","key":"10143_CR7","doi-asserted-by":"publisher","first-page":"34","DOI":"10.3390\/drones2040034","volume":"2","author":"Y Feng","year":"2018","unstructured":"Feng, Y., Zhang, C., Baek, S., Rawashdeh, S., & Mohammadi, A. (2018). Autonomous landing of a uav on a moving platform using model predictive control. Drones, 2(4), 34.","journal-title":"Drones"},{"key":"10143_CR8","unstructured":"Fujimoto, S., van Hoof, H., & Meger, D. (2018). Addressing function approximation error in actor-critic methods. In Proceedings of the 35th int. conference on machine learning (Vol. 80, pp. 1587-1596)."},{"key":"10143_CR9","doi-asserted-by":"publisher","DOI":"10.1016\/j.autcon.2020.103200","volume":"115","author":"L Gonz\u00e1lez-deSantos","year":"2020","unstructured":"Gonz\u00e1lez-deSantos, L., Mart\u00ednez-S\u00e1nchez, J., Gonz\u00e1lez-Jorge, H., Navarro-Medina, F., & Arias, P. (2020). Uav payload with collision mitigation for contact inspection. Automation in Construction, 115, 103200.","journal-title":"Automation in Construction"},{"key":"10143_CR10","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1016\/S0166-4115(08)62386-9","volume":"52","author":"SG Hart","year":"1988","unstructured":"Hart, S. G., & Staveland, L. E. (1988). Development of nasa-tlx (task load index): Results of empirical and theoretical research. Human Mental Workload, 52, 139\u2013183.","journal-title":"Human Mental Workload"},{"key":"10143_CR11","unstructured":"Javdani, S. (2016). Ada assistance policy. https:\/\/github.com\/personalrobotics\/ ada assistance policy"},{"issue":"7","key":"10143_CR12","doi-asserted-by":"publisher","first-page":"717","DOI":"10.1177\/0278364918776060","volume":"37","author":"S Javdani","year":"2018","unstructured":"Javdani, S., Admoni, H., Pellegrinelli, S., Srinivasa, S. S., & Bagnell, J. A. (2018). Shared autonomy via hindsight optimization for teleoperation and teaming. The International Journal of Robotics Research, 37(7), 717\u2013742.","journal-title":"The International Journal of Robotics Research"},{"key":"10143_CR13","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1016\/j.eswa.2019.01.024","volume":"122","author":"MA Kaljahi","year":"2019","unstructured":"Kaljahi, M. A., Shivakumara, P., Idris, M. Y. I., Anisi, M. H., Lu, T., Blumenstein, M., & Noor, N. M. (2019). An automatic zone detection system for safe landing of uavs. Expert Systems with Applications, 122, 319\u2013333.","journal-title":"Expert Systems with Applications"},{"issue":"4","key":"10143_CR14","doi-asserted-by":"publisher","first-page":"3860","DOI":"10.1109\/LRA.2019.2929993","volume":"4","author":"X Kan","year":"2019","unstructured":"Kan, X., Thomas, J., Teng, H., Tanner, H. G., Kumar, V., & Karydis, K. (2019). Analysis of ground effect for small-scale uavs in forward flight. IEEE Robotics and Automation Letters, 4(4), 3860\u20133867.","journal-title":"IEEE Robotics and Automation Letters"},{"issue":"16","key":"10143_CR15","doi-asserted-by":"publisher","first-page":"5436","DOI":"10.3390\/app10165436","volume":"10","author":"D-H Kim","year":"2020","unstructured":"Kim, D.-H., Go, Y.-G., & Choi, S.-M. (2020). An aerial mixed-reality environment for firstperson\u2014view drone flying. Applied Sciences, 10(16), 5436.","journal-title":"Applied Sciences"},{"key":"10143_CR16","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., & Wierstra, D. (2016). Continuous control with deep reinforcement learning. In International conference on learning representations. [cs.LG]"},{"key":"10143_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TRO.2019.2942989","volume":"36","author":"A Loquercio","year":"2020","unstructured":"Loquercio, A., Kaufmann, E., Ranftl, R., Dosovitskiy, A., Koltun, V., & Scaramuzza, D. (2020). Deep drone racing: From simulation to reality with domain randomization. IEEE Transactions on Robotics, 36, 1\u201314.","journal-title":"IEEE Transactions on Robotics"},{"issue":"2","key":"10143_CR18","doi-asserted-by":"publisher","first-page":"1088","DOI":"10.1109\/LRA.2018.2795643","volume":"3","author":"A Loquercio","year":"2018","unstructured":"Loquercio, A., Maqueda, A. I., del-Blanco, C. R., & Scaramuzza, D. (2018). Dronet: Learning to fly by driving. IEEE Robotics and Automation Letters, 3(2), 1088\u20131095.","journal-title":"IEEE Robotics and Automation Letters"},{"key":"10143_CR19","doi-asserted-by":"crossref","unstructured":"Maturana, D., & Scherer, S. (2015). 3d convolutional neural networks for landing zone detection from lidar. In 2015 ieee international conference on robotics and automation (icra) (p. 3471-3478).","DOI":"10.1109\/ICRA.2015.7139679"},{"issue":"11","key":"10143_CR20","doi-asserted-by":"publisher","first-page":"5436","DOI":"10.3390\/drones6110347","volume":"6","author":"L Morando","year":"2022","unstructured":"Morando, L., Recchiuto, C. T., Calla, J., Scuteri, P., & Sgorbissa, A. (2022). Thermal and visual tracking of photovoltaic plants for autonomous uav inspection. Drones, 6(11), 5436.","journal-title":"Drones"},{"key":"10143_CR21","doi-asserted-by":"crossref","unstructured":"Nogar, S.M. (2020). Autonomous landing of a uav on a moving ground vehicle in a gps denied environment. In 2020 ieee international symposium on safety, security, and rescue robotics (ssrr) (p. 77-83).","DOI":"10.1109\/SSRR50563.2020.9292607"},{"key":"10143_CR22","doi-asserted-by":"crossref","unstructured":"Patrikar, J., Moon, B., Oh, J., & Scherer, S. (2022). Predicting like a pilot: Dataset and method to predict socially-aware aircraft trajectories in non-towered terminal airspace. In 2022 international conference on robotics and automation (icra) (p. 2525-2531).","DOI":"10.1109\/ICRA46639.2022.9811972"},{"key":"10143_CR23","doi-asserted-by":"crossref","unstructured":"Perez-Grau, F., Ragel, R., Caballero, F., Viguria, A., & Ollero, A. (2017). Semi-autonomous teleoperation of uavs in search and rescue scenarios. In International conference on unmanned aircraft systems (icuas) (p. 1066- 1074).","DOI":"10.1109\/ICUAS.2017.7991349"},{"issue":"3","key":"10143_CR24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0264471","volume":"17","author":"C Pfeiffer","year":"2022","unstructured":"Pfeiffer, C., Wengeler, S., Loquercio, A., & Scaramuzza, D. (2022). Visual attention prediction improves performance of autonomous drone racing agents. PLOS ONE, 17(3), 1\u201316.","journal-title":"PLOS ONE"},{"issue":"1","key":"10143_CR25","doi-asserted-by":"publisher","first-page":"8","DOI":"10.3390\/robotics9010008","volume":"9","author":"R Polvara","year":"2020","unstructured":"Polvara, R., Patacchiola, M., Hanheide, M., & Neumann, G. (2020). Sim-to-real quadrotor landing via sequential deep q-networks and domain randomization. Robotics, 9(1), 8.","journal-title":"Robotics"},{"key":"10143_CR26","unstructured":"Reddy, S. (2018). Deep assist. https:\/\/github.com\/rddy\/deepassist."},{"key":"10143_CR27","doi-asserted-by":"crossref","unstructured":"Reddy, S., Dragan, A., & Levine, S. (2018). Shared autonomy via deep reinforcement learning. In Proceedings of robotics: Science and systems.","DOI":"10.15607\/RSS.2018.XIV.005"},{"key":"10143_CR28","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1007\/s10846-018-0891-8","volume":"93","author":"A Rodriguez-Ramos","year":"2019","unstructured":"Rodriguez-Ramos, A., Sampedro, C., Bavle, H., Puente, P. D. L., & Campoy, P. (2019). A deep reinforcement learning strategy for uav autonomous landing on a moving platform. Journal of Intelligent and Robotic Systems, 93, 351\u2013366.","journal-title":"Journal of Intelligent and Robotic Systems"},{"key":"10143_CR29","doi-asserted-by":"publisher","first-page":"22003","DOI":"10.3390\/s150922003","volume":"15","author":"I Sa","year":"2015","unstructured":"Sa, I., Hrabar, S., & Corke, P. (2015). Inspection of pole-like structures using a visual-inertial aided vtol platform with shared autonomy. Sensors, 15, 22003\u201322048.","journal-title":"Sensors"},{"key":"10143_CR30","unstructured":"Salter, S., Rao, D., Wulfmeier, M., Hadsell, R., & Posner, H. (2021). Attention-privileged reinforcement learning. In Conference on Robot Learning."},{"key":"10143_CR31","doi-asserted-by":"crossref","unstructured":"Shah, S., Dey, D., Lovett, C., & Kapoor, A. (2018). Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Field and service robotics (pp. 621-635). Springer International Publishing.","DOI":"10.1007\/978-3-319-67361-5_40"},{"key":"10143_CR32","doi-asserted-by":"crossref","unstructured":"Shaqura, M., Alzuhair, K., Abdellatif, F., & Shamma, J.S. (2018). Human supervised multirotor uav system design for inspection applications. In Ieee international symposium on safety, security, and rescue robotics (ssrr) (p. 1-6).","DOI":"10.1109\/SSRR.2018.8468648"},{"key":"10143_CR33","doi-asserted-by":"crossref","unstructured":"Shi, G., Shi, X., O\u2019Connell, M., Yu, R., Azizzadenesheli, K., Anandkumar, A., & Chung, S. (2019). Neural lander: Stable drone landing control using learned dynamics. In Icra (p. 9784-9790).","DOI":"10.1109\/ICRA.2019.8794351"},{"key":"10143_CR34","doi-asserted-by":"publisher","first-page":"11","DOI":"10.3389\/frobt.2017.00011","volume":"4","author":"N Smolyanskiy","year":"2017","unstructured":"Smolyanskiy, N., & Gonzalez-Franco, M. (2017). Stereoscopic first person view system for drone navigation. Frontiers in Robotics and AI, 4, 11.","journal-title":"Frontiers in Robotics and AI"},{"key":"10143_CR35","doi-asserted-by":"crossref","unstructured":"Spurr, A., Song, J., Park, S., & Hilliges, O. (2018). Cross-modal deep variational hand pose estimation. In Ieee\/cvf conference on computer vision and pattern recognition (p. 89-98).","DOI":"10.1109\/CVPR.2018.00017"},{"issue":"61","key":"10143_CR36","first-page":"2023","volume":"16","author":"V Vapnik","year":"2015","unstructured":"Vapnik, V., & Izmailov, R. (2015). Learning using privileged information: Similarity control and knowledge transfer. Journal of Machine Learning Research, 16(61), 2023\u20132049.","journal-title":"Journal of Machine Learning Research"},{"key":"10143_CR37","doi-asserted-by":"crossref","unstructured":"Wang, P., Wang, C., Wang, J., & Meng, M.Q.-H. (2022). Quadrotor autonomous landing on moving platform. In Procedia Computer Science, 209 , 40-49. (Proceedings of the 2022 International Symposium on Biomimetic Intelligence and Robotics (ISBIR))","DOI":"10.1016\/j.procs.2022.10.097"},{"key":"10143_CR38","doi-asserted-by":"publisher","first-page":"105086","DOI":"10.1109\/ACCESS.2019.2932008","volume":"7","author":"Y Wang","year":"2019","unstructured":"Wang, Y., Bai, P., Liang, X., Wang, W., Zhang, J., & Fu, Q. (2019). Reconnaissance mission conducted by uav swarms based on distributed pso path planning algorithms. IEEE Access, 7, 105086\u2013105099.","journal-title":"IEEE Access"},{"key":"10143_CR39","doi-asserted-by":"crossref","unstructured":"Xia, B., Mantegh, I., & Xie, W. (2021). Integrated emergency self-landing method for autonomous uas in urban aerial mobility. In 2021 21st international conference on control, automation and systems (iccas) (p. 275- 282).","DOI":"10.23919\/ICCAS52745.2021.9649955"},{"key":"10143_CR40","doi-asserted-by":"crossref","unstructured":"Zhang, D., Tron, R., & Khurshid, R.P. (2021). Haptic feedback improves human-robot agreement and user satisfaction in sharedautonomy teleoperation. In 2021 ieee international conference on robotics and automation (icra) (p. 3306-3312).","DOI":"10.1109\/ICRA48506.2021.9560991"}],"container-title":["Autonomous Robots"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10514-023-10143-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10514-023-10143-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10514-023-10143-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,28]],"date-time":"2023-11-28T18:13:22Z","timestamp":1701195202000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10514-023-10143-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,21]]},"references-count":40,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["10143"],"URL":"https:\/\/doi.org\/10.1007\/s10514-023-10143-3","relation":{},"ISSN":["0929-5593","1573-7527"],"issn-type":[{"value":"0929-5593","type":"print"},{"value":"1573-7527","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,21]]},"assertion":[{"value":"28 July 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 September 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 October 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The authors declare that the submitted work is free from personal conflicts of interest. The user study was approved by the Monash University Human Research Ethics Committee (MUHREC), project ID 29565. All participants gave informed consent prior to participating in the user study.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}