{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T05:32:27Z","timestamp":1772170347926,"version":"3.50.1"},"reference-count":16,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T00:00:00Z","timestamp":1711584000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Instituto de Engenharia de Sistemas e Computadores Investiga\u00e7\u00e3o e Desenvolvimento","award":["UIDB\/50021\/2020"],"award-info":[{"award-number":["UIDB\/50021\/2020"]}]},{"name":"Instituto de Engenharia de Sistemas e Computadores Investiga\u00e7\u00e3o e Desenvolvimento","award":["UIDB\/50009\/2020"],"award-info":[{"award-number":["UIDB\/50009\/2020"]}]},{"name":"Laboratory of Robotics and Engineering Systems","award":["UIDB\/50021\/2020"],"award-info":[{"award-number":["UIDB\/50021\/2020"]}]},{"name":"Laboratory of Robotics and Engineering Systems","award":["UIDB\/50009\/2020"],"award-info":[{"award-number":["UIDB\/50009\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Eng"],"abstract":"<jats:p>The problem addressed in this article consists of the motion control of a quadrotor affected by model disturbances and uncertainties. In order to tackle model uncertainty, adaptive control based on reinforcement learning is used. The distinctive feature of this article, in comparison with other works on quadrotor control using reinforcement learning, is the exploration of the underlying optimal control problem in which a quadratic cost and a linear dynamics allow for an algorithm that runs in real time. Instead of identifying a plant model, adaptation is obtained by approximating the performance index given by the Q-function using directional forgetting recursive least squares that rely on a linear regressor built from quadratic functions of input\/output data. The adaptive algorithm proposed is tested in simulation in a cascade control structure that drives a quadrotor. Simulations show the improvement in performance that results when the proposed algorithm is turned on.<\/jats:p>","DOI":"10.3390\/eng5020030","type":"journal-article","created":{"date-parts":[[2024,3,28]],"date-time":"2024-03-28T13:20:53Z","timestamp":1711632053000},"page":"544-561","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Adaptive Control of Quadrotors in Uncertain Environments"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-1457-3541","authenticated-orcid":false,"given":"Daniel","family":"Leit\u00e3o","sequence":"first","affiliation":[{"name":"Instituto Superior T\u00e9cnico, Universidade de Lisboa, Av. Rovisco Pais, 1049-001 Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8925-1273","authenticated-orcid":false,"given":"Rita","family":"Cunha","sequence":"additional","affiliation":[{"name":"Institute for Systems and Robotics, Instituto Superior T\u00e9cnico, Universidade de Lisboa, Av. Rovisco Pais, 1049-001 Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3149-8035","authenticated-orcid":false,"given":"Jo\u00e3o M.","family":"Lemos","sequence":"additional","affiliation":[{"name":"INESC-ID, Instituto Superior T\u00e9cnico, Universidade de Lisboa, Rua Alves Redol 9, 1000-029 Lisboa, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1832","DOI":"10.1007\/s40435-020-00737-5","article-title":"A review on drones controlled in real-time","volume":"9","author":"Kangunde","year":"2021","journal-title":"Int. J. Dyn. Control"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Azar, A.T., Koubaa, A., Ali Mohamed, N., Ibrahim, H.A., Ibrahim, Z.F., Kazim, M., Ammar, A., Benjdira, B., Khamis, A.M., and Hameed, I.A. (2021). Drone deep reinforcement learning: A review. Electronics, 10.","DOI":"10.3390\/electronics10090999"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Elmokadem, T., and Savkin, A.V. (2021). Towards fully autonomous UAVs: A survey. Sensors, 21.","DOI":"10.3390\/s21186223"},{"key":"ref_4","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3301273","article-title":"Reinforcement learning for UAV attitude control","volume":"3","author":"Koch","year":"2019","journal-title":"ACM Trans.-Cyber-Phys. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.ifacol.2021.11.158","article-title":"Robust deep reinforcement learning for quadcopter control","volume":"54","author":"Deshpande","year":"2021","journal-title":"IFAC-PapersOnLine"},{"key":"ref_7","unstructured":"Deshpande, A.M., Kumar, R., Minai, A.A., and Kumar, M. (2020, January 4\u20137). Developmental Reinforcement Learning of Control Policy of a Quadcopter UAV With Thrust Vectoring Rotors. Proceedings of the Dynamic Systems and Control Conference, American Society of Mechanical Engineers, Pittsburgh, PA, USA."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"106694","DOI":"10.1016\/j.asoc.2020.106694","article-title":"Real-time deep reinforcement learning based vehicle navigation","volume":"96","author":"Koh","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_9","unstructured":"Ramstedt, S., and Pal, C. (2019, January 8\u201314). Real-Time Reinforcement Learning. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_10","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2019). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/MCAS.2009.933854","article-title":"Reinforcement learning and adaptive dynamic programming for feedback control","volume":"9","author":"Lewis","year":"2009","journal-title":"IEEE Circuits Syst. Mag."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Siciliano, B., Sciavicco, L., Villani, L., and Oriolo, G. (2009). Robotics, Springer. Advanced textbooks in control and signal processing.","DOI":"10.1007\/978-1-84628-642-1"},{"key":"ref_13","unstructured":"Franklin, G.F., Powell, J.D., and Workman, M.L. (1998). Digital Control of Dynamic Systems, Addison-Wesley."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1146\/annurev-control-053018-023825","article-title":"A tour of reinforcement learning: The view from continuous control","volume":"2","author":"Recht","year":"2019","journal-title":"Annu. Rev. Control Robot. Auton. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1016\/0005-1098(87)90054-9","article-title":"Restricted exponential forgetting in real-time identification","volume":"23","year":"1987","journal-title":"Automatica"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"877","DOI":"10.1109\/TCST.2005.854318","article-title":"A dual approach to start-up of an adaptive predictive controller","volume":"13","author":"Filatov","year":"2005","journal-title":"IEEE Trans. Control. Syst. Technol."}],"container-title":["Eng"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2673-4117\/5\/2\/30\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:20:12Z","timestamp":1760106012000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2673-4117\/5\/2\/30"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,28]]},"references-count":16,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["eng5020030"],"URL":"https:\/\/doi.org\/10.3390\/eng5020030","relation":{},"ISSN":["2673-4117"],"issn-type":[{"value":"2673-4117","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,28]]}}}