{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T22:33:55Z","timestamp":1782772435838,"version":"3.54.5"},"reference-count":20,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2023,8,10]],"date-time":"2023-08-10T00:00:00Z","timestamp":1691625600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Engineering and Physical Sciences Research Council","award":["EP\/N509619\/1"],"award-info":[{"award-number":["EP\/N509619\/1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Reinforcement learning has been used on a variety of control tasks for drones, including, in previous work at the University of Bristol, on perching manoeuvres with sweep-wing aircraft. In this paper, a new aircraft model is presented representing flight up to very high angles of attack where the aerodynamic models are highly nonlinear. The model is employed to develop high-alpha manoeuvres, using reinforcement learning to exploit the nonlinearities at the edge of the flight envelope, enabling fixed-wing operations in tightly confined spaces. Training networks for multiple manoeuvres is also demonstrated. The approach is shown to generate controllers that take full advantage of the aircraft capability. It is suggested that a combination of these neural network-based controllers, together with classical model predictive control, could be used to operate efficiently within the low alpha flight regime and, yet, respond rapidly in confined spaces where high alpha, agile manoeuvres are required.<\/jats:p>","DOI":"10.3390\/a16080384","type":"journal-article","created":{"date-parts":[[2023,8,10]],"date-time":"2023-08-10T10:24:47Z","timestamp":1691663087000},"page":"384","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Reinforcement Learning Derived High-Alpha Aerobatic Manoeuvres for Fixed Wing Operation in Confined Spaces"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5485-2992","authenticated-orcid":false,"given":"Robert","family":"Clarke","sequence":"first","affiliation":[{"name":"Department of Aerospace Engineering, University of Bristol, Bristol BS8 1TR, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4840-2474","authenticated-orcid":false,"given":"Liam","family":"Fletcher","sequence":"additional","affiliation":[{"name":"Department of Aerospace Engineering, University of Bristol, Bristol BS8 1TR, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"East","sequence":"additional","affiliation":[{"name":"Department of Aerospace Engineering, University of Bristol, Bristol BS8 1TR, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7767-452X","authenticated-orcid":false,"given":"Thomas","family":"Richardson","sequence":"additional","affiliation":[{"name":"Department of Aerospace Engineering, University of Bristol, Bristol BS8 1TR, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1101","DOI":"10.1017\/aer.2022.48","article-title":"Improvements in learning to control perched landings","volume":"126","author":"Fletcher","year":"2022","journal-title":"Aeronaut. J."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Cano Lopes, G., Ferreira, M., da Silva Sim\u00f5es, A., and Luna Colombini, E. (2018, January 6\u201310). Intelligent Control of a Quadrotor with Proximal Policy Optimization Reinforcement Learning. Proceedings of the 2018 Latin American Robotic Symposium, 2018 Brazilian Symposium on Robotics (SBR) and 2018 Workshop on Robotics in Education (WRE), Jo\u00e3o Pessoa, Brazil.","DOI":"10.1109\/LARS\/SBR\/WRE.2018.00094"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Carvalho Melo, L., and Omena Albuquerque M\u00e1ximo, M.R. (2019, January 23\u201325). Learning Humanoid Robot Running Skills through Proximal Policy Optimization. Proceedings of the 2019 Latin American Robotics Symposium (LARS), 2019 Brazilian Symposium on Robotics (SBR) and 2019 Workshop on Robotics in Education (WRE), Rio Grande, Brazil.","DOI":"10.1109\/LARS-SBR-WRE48964.2019.00015"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"11945","DOI":"10.1038\/s41598-023-38259-7","article-title":"Controlling the Solo12 quadruped robot with deep reinforcement learning","volume":"13","author":"Aractingi","year":"2023","journal-title":"Sci. Rep."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.ifacol.2021.10.086","article-title":"Explainable AI methods on a deep reinforcement learning agent for automatic docking. (This work was supported by the Research Council of Norway through the EXAIGON project, project number 304843)","volume":"54","author":"Lekkas","year":"2021","journal-title":"IFAC-PapersOnLine"},{"key":"ref_6","first-page":"6611","article-title":"Guided Exploration with Proximal Policy Optimization using a Single Demonstration","volume":"Volume 139","author":"Meila","year":"2021","journal-title":"Proceedings of the 38th International Conference on Machine Learning"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"B\u00f8hn, E., Coates, E.M., Moe, S., and Johansen, T.A. (2019, January 11\u201314). Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimization. Proceedings of the 2019 International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta, GA, USA.","DOI":"10.1109\/ICUAS.2019.8798254"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Bohn, E., Coates, E.M., Reinhardt, D., and Johansen, T.A. (IEEE Trans. Neural Netw. Learn. Syst., 2023). Data-Efficient Deep Reinforcement Learning for Attitude Control of Fixed-Wing UAVs: Field Experiments, IEEE Trans. Neural Netw. Learn. Syst., early access.","DOI":"10.1109\/TNNLS.2023.3263430"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wei, W., Fang, Z., and Zhu, Y. (2023, January 23\u201327). Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning. Proceedings of the AIAA SCITECH 2023 Forum, National Harbor, MD, USA.","DOI":"10.2514\/6.2023-1072"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Clarke, S., and Hwang, I. (2020, January 6\u201310). Deep Reinforcement Learning Control for Aerobatic Maneuvering of Agile Fixed-Wing Aircraft. Proceedings of the AIAA Scitech 2020 Forum, Orlando, FL, USA.","DOI":"10.2514\/6.2020-0136"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lee, J.H., and Kampen, E.J.V. (19\u201321, January 11\u201315). Online Reinforcement Learning for Fixed-Wing Aircraft Longitudinal Control. Proceedings of the AIAA Scitech 2021 Forum, Virtual.","DOI":"10.2514\/6.2021-0392"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"107623","DOI":"10.1016\/j.ast.2022.107623","article-title":"Learning-based fixed-wing UAV reactive maneuver control for obstacle avoidance","volume":"126","author":"Wu","year":"2022","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhang, S., Du, X., Xiao, J., Huang, J., and He, K. (2021, January 22\u201324). Reinforcement Learning Control for 6 DOF Flight of Fixed-Wing Aircraft. Proceedings of the 2021 33rd Chinese Control and Decision Conference (CCDC), Kunming, China.","DOI":"10.1109\/CCDC52312.2021.9602605"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhen, Y., Hao, M., and Sun, W. (2020, January 27\u201328). Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs. Proceedings of the 2020 3rd International Conference on Unmanned Systems (ICUS), Harbin, China.","DOI":"10.1109\/ICUS50048.2020.9274875"},{"key":"ref_15","first-page":"1","article-title":"Stable-Baselines3: Reliable Reinforcement Learning Implementations","volume":"22","author":"Raffin","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_16","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv."},{"key":"ref_17","first-page":"1889","article-title":"Trust Region Policy Optimization","volume":"Volume 37","author":"Bach","year":"2015","journal-title":"Proceedings of the 32nd International Conference on Machine Learning"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Clarke, R.J., Fletcher, L.J., and Richardson, T.S. (2023, January 23\u201327). pyaerso: A Rust-backed Python Module for Accessible Flight Dynamics Modelling for Machine Learning. Proceedings of the AIAA SCITECH 2023 Forum, National Harbor, MD, USA.","DOI":"10.2514\/6.2023-0149"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1145\/2692956.2663188","article-title":"The rust language","volume":"34","author":"Matsakis","year":"2014","journal-title":"ACM Sigada Ada Lett."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Raz, A.K., Nolan, S.M., Levin, W., Mall, K., Mia, A., Mockus, L., Ezra, K., and Williams, K. (2022, January 5\u201312). Test and Evaluation of Reinforcement Learning via Robustness Testing and Explainable AI for High-Speed Aerospace Vehicles. Proceedings of the 2022 IEEE Aerospace Conference (AERO), Big Sky, MT, USA.","DOI":"10.1109\/AERO53065.2022.9843563"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/8\/384\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:30:16Z","timestamp":1760128216000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/8\/384"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,10]]},"references-count":20,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["a16080384"],"URL":"https:\/\/doi.org\/10.3390\/a16080384","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,10]]}}}