{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T17:23:54Z","timestamp":1770225834280,"version":"3.49.0"},"reference-count":38,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,2,16]],"date-time":"2021-02-16T00:00:00Z","timestamp":1613433600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Major Project of the New Generation of Artificial Intelligence, China","award":["No. 2018AAA0102900"],"award-info":[{"award-number":["No. 2018AAA0102900"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Autopilot technology in the field of aviation has developed over many years. However, it is difficult for an autopilot system to autonomously operate a civil aircraft under bad weather conditions. In this paper, we present a reinforcement learning (RL) algorithm using multimodal data and preprocessing data to have a civil aircraft take off autonomously under crosswind conditions. The multimodal data include the common flight status and visual information. The preprocessing is a new design that maps some flight data by nonlinear functions based on the general flight dynamics before these data are fed into the RL model. Extensive experiments under different crosswind conditions with a professional flight simulator demonstrate that the proposed method can effectively control a civil aircraft to take off under various crosswind conditions and achieve better performance than trials without visual information or preprocessing data.<\/jats:p>","DOI":"10.3390\/s21041386","type":"journal-article","created":{"date-parts":[[2021,2,16]],"date-time":"2021-02-16T22:13:38Z","timestamp":1613513618000},"page":"1386","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Learning to Have a Civil Aircraft Take Off under Crosswind Conditions by Reinforcement Learning with Multimodal Data and Preprocessing Data"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9006-4520","authenticated-orcid":false,"given":"Feng","family":"Liu","sequence":"first","affiliation":[{"name":"State Key Laboratory of VR Technology &amp; Systems, Beihang University, Beijing 100191, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuling","family":"Dai","sequence":"additional","affiliation":[{"name":"State Key Laboratory of VR Technology &amp; Systems, Beihang University, Beijing 100191, China"},{"name":"Jiangxi Research Institute, Beihang University, Beijing 100191, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yongjia","family":"Zhao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of VR Technology &amp; Systems, Beihang University, Beijing 100191, China"},{"name":"Jiangxi Research Institute, Beihang University, Beijing 100191, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"197","DOI":"10.2514\/1.52672","article-title":"Autopilot abstraction and standardization for seamless integration of unmanned aircraft system applications","volume":"8","author":"Royo","year":"2011","journal-title":"J. Aerosp. Comput. Inf. Commun."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.conengprac.2018.04.010","article-title":"Robust autopilot design for landing a large civil aircraft in crosswind","volume":"76","author":"Theis","year":"2018","journal-title":"Control Eng. Pract."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Dehais, F., Peysakhovich, V., Scannella, S., Fongue, J., and Gateau, T. (2015, January 18\u201323). \u201cAutomation Surprise\u201d in Aviation: Real-Time Solutions. Proceedings of the 33rd annual ACM conference on Human Factors in Computing Systems, Seoul, Korea.","DOI":"10.1145\/2702123.2702521"},{"key":"ref_4","first-page":"693","article-title":"Considerations in autopilot litigation","volume":"48","author":"Cooling","year":"1982","journal-title":"J. Air L. Com."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"454","DOI":"10.2514\/1.1214","article-title":"Aircraft autopilot analysis and envelope protection for operation under icing conditions","volume":"27","author":"Sharma","year":"2004","journal-title":"J. Guid. Control Dyn."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1007\/BF02190056","article-title":"Control of an aircraft landing in windshear","volume":"83","author":"Patsko","year":"1994","journal-title":"J. Optim. Theor. Appl."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1007\/s13235-017-0215-9","article-title":"Aircraft control during cruise flight in windshear conditions: Viability approach","volume":"7","author":"Botkin","year":"2017","journal-title":"Dyn. Games Appl."},{"key":"ref_8","unstructured":"Blakelock, J.H. (1991). Automatic Control of Aircraft and Missiles., John Wiley & Sons, Inc."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Sanil, N., Rakesh, V., Mallapur, R., and Ahmed, M.R. (2020, January 10\u201312). Deep Learning Techniques for Obstacle Detection and Avoidance in Driverless Cars. Proceedings of the 2020 International Conference on Artificial Intelligence and Signal Processing (AISP), Amaravati, India.","DOI":"10.1109\/AISP48273.2020.9073155"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Hodges, C., An, S., Rahmani, H., and Bennamoun, M. (2019). Deep learning for driverless vehicles. Handbook of Deep Learning Applications, Springer.","DOI":"10.1007\/978-3-030-11479-4_4"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Han, M., Senellart, P., Bressan, S., and Wu, H. (2016, January 24\u201328). Routing an autonomous taxi with reinforcement learning. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, Indianapolis, Indiana.","DOI":"10.1145\/2983323.2983379"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kendall, A., Hawke, J., Janz, D., Mazur, P., Reda, D., Allen, J.M., and Shah, A. (2019, January 20\u201324). Learning to drive in a day. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793742"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Heim, E.H., Viken, E., Brandon, J.M., and Croom, M.A. (2018, January 25\u201329). NASA\u2019s Learn-to-Fly Project Overview. Proceedings of the 2018 Atmospheric Flight Mechanics Conference, Atlanta, Georgia.","DOI":"10.2514\/6.2018-3307"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Anwar, M.A., and Raychowdhury, A. (2018, January 20\u201322). NavREn-Rl: Learning to fly in real environment via end-to-end deep reinforcement learning using monocular images. Proceedings of the 25th International Conference on Mechatronics and Machine Vision in Practice (M2VIP), Stuttgart, Germany.","DOI":"10.1109\/M2VIP.2018.8600838"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Shaker, M., Smith, M.N., Yue, S., and Duckett, T. (2010, January 6\u20137). Vision-based landing of a simulated unmanned aerial vehicle with fast reinforcement learning. Proceedings of the 2010 International Conference on Emerging Security Technologies, Canterbury, UK.","DOI":"10.1109\/EST.2010.14"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Caruana, R., and Niculescu-Mizil, A. (2006, January 25\u201329). An empirical comparison of supervised learning algorithms. Proceedings of the 23rd international conference on Machine learning, Pittsburgh, PA, USA.","DOI":"10.1145\/1143844.1143865"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cunningham, P., Cord, M., and Delany, S.J. (2008). Supervised learning. Machine Learning Techniques for Multimedia, Springer.","DOI":"10.1007\/978-3-540-75171-7_2"},{"key":"ref_18","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_19","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2015). Deep reinforcement learning with double q-learning. arXiv, Available online: https:\/\/arxiv.org\/pdf\/1509.06461."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3301273","article-title":"Reinforcement learning for UAV attitude control","volume":"3","author":"Koch","year":"2019","journal-title":"ACM Trans. Cyber-Phys. Syst."},{"key":"ref_21","unstructured":"Lillicrap, T.P., Hunt, J.J., and Pritzel, A. (2015). Continuous control with deep reinforcement learning. arXiv, Available online: https:\/\/arxiv.org\/pdf\/1509.02971."},{"key":"ref_22","unstructured":"Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015, January 7\u20139). Trust region policy optimization. Proceedings of the 32nd international conference on machine learning, Lille, France."},{"key":"ref_23","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv, Available online: https:\/\/arxiv.org\/pdf\/1707.06347."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Xie, J., Peng, X., Wang, H., Niu, W., and Zheng, X. (2020). UAV Autonomous Tracking and Landing Based on Deep Reinforcement Learning Strategy. Sensors, 20.","DOI":"10.3390\/s20195630"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Dynkin, E.B. (1965). Markov processes. Markov Processes, Springer.","DOI":"10.1007\/978-3-662-00031-1"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1016\/S0019-9958(58)80003-0","article-title":"Dynamic programming and stochastic control processes","volume":"1","author":"Bellman","year":"1958","journal-title":"Inf. Control"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Lear."},{"key":"ref_28","unstructured":"Mnih, V., Kavukcuoglu, K., and Silver, D. (2013). Playing atari with deep reinforcement learning. arXiv, Available online: https:\/\/arxiv.org\/pdf\/1312.5602."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","article-title":"Deep reinforcement learning: A brief survey","volume":"34","author":"Arulkumaran","year":"2017","journal-title":"IEEE Signal Process Mag."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wei, F., Amaya-Bower, L., Gates, A., Rose, D., and Vasko, T. (2016, January 4\u20138). The Full-Scale Helicopter Flight Simulator Design and Fabrication at CCSU. Proceedings of the 57th AIAA\/ASCE\/AHS\/ASC Structures, Structural Dynamics, and Materials Conference, San Diego, CA, USA.","DOI":"10.2514\/6.2016-0582"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Jirgl, M., Boril, J., and Jalovecky, R. (2015, January 19\u201321). The identification possibilities of the measured parameters of an aircraft model and pilot behavior model on the flight simulator. Proceedings of the International Conference on Military Technologies (ICMT), Brno, Czech Republic.","DOI":"10.1109\/MILTECHS.2015.7153726"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"327","DOI":"10.14429\/dsj.64.4933","article-title":"Simulation of Flapping-wing Unmanned Aerial Vehicle using X-plane and Matlab\/Simulink","volume":"64","author":"Kaviyarasu","year":"2014","journal-title":"Defence Sci. J."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Baomar, H., and Bentley, P.J. (2016, January 7\u201310). An Intelligent Autopilot System that learns piloting skills from human pilots by imitation. Proceedings of the 2016 International Conference on Unmanned Aircraft Systems (ICUAS), Arlington, VA, USA.","DOI":"10.1109\/ICUAS.2016.7502578"},{"key":"ref_34","unstructured":"Teubert, C., and Watkins, J. (2020, August 02). The X-Plane Connect Toolbox (2020). Available online: https:\/\/github.com\/nasa\/XPlaneConnect."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wayllace, C., Ha, S., Han, Y., Hu, J., Monadjemi, S., Yeoh, W., and Ottley, A. (2020). DRAGON-V: Detection and Recognition of Airplane Goals with Navigational Visualization. AAAI 2020, AAAI.","DOI":"10.1609\/aaai.v34i09.7108"},{"key":"ref_36","unstructured":"Fremont, D.J., Chiu, J., Margineantu, D.D., Osipychev, D., and Seshia, S.A. (2005). Formal Analysis and Redesign of a Neural Network-Based Aircraft Taxiing System with VerifAI. arXiv, Available online: https:\/\/arxiv.org\/pdf\/2005.07173."},{"key":"ref_37","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized experience replay. arXiv, Available online: https:\/\/arxiv.org\/pdf\/1511.05952."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1145\/203330.203343","article-title":"Temporal difference learning and TD-Gammon","volume":"38","author":"Tesauro","year":"1995","journal-title":"Commun. ACM"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1386\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:24:58Z","timestamp":1760160298000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1386"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,16]]},"references-count":38,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["s21041386"],"URL":"https:\/\/doi.org\/10.3390\/s21041386","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,16]]}}}