{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T02:41:27Z","timestamp":1783996887739,"version":"3.55.0"},"reference-count":40,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2022,8,28]],"date-time":"2022-08-28T00:00:00Z","timestamp":1661644800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U20A20161"],"award-info":[{"award-number":["U20A20161"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The required navigation performance (RNP) procedure is one of the two basic navigation specifications for the performance-based navigation (PBN) procedure as proposed by the International Civil Aviation Organization (ICAO) through an integration of the global navigation infrastructures to improve the utilization efficiency of airspace and reduce flight delays and the dependence on ground navigation facilities. The approach stage is one of the most important and difficult stages in the whole flying. In this study, we proposed deep reinforcement learning (DRL)-based RNP procedure execution, DRL-RNP. By conducting an RNP approach procedure, the DRL algorithm was implemented, using a fixed-wing aircraft to explore a path of minimum fuel consumption with reward under windy conditions in compliance with the RNP safety specifications. The experimental results have demonstrated that the six degrees of freedom aircraft controlled by the DRL algorithm can successfully complete the RNP procedure whilst meeting the safety specifications for protection areas and obstruction clearance altitude in the whole procedure. In addition, the potential path with minimum fuel consumption can be explored effectively. Hence, the DRL method can be used not only to implement the RNP procedure with a simulated aircraft but also to help the verification and evaluation of the RNP procedure.<\/jats:p>","DOI":"10.3390\/s22176475","type":"journal-article","created":{"date-parts":[[2022,8,30]],"date-time":"2022-08-30T01:37:55Z","timestamp":1661823475000},"page":"6475","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["DRL-RNP: Deep Reinforcement Learning-Based Optimized RNP Flight Procedure Execution"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5047-7788","authenticated-orcid":false,"given":"Longtao","family":"Zhu","sequence":"first","affiliation":[{"name":"School of Aeronautics and Astronautics, Sichuan University, Chengdu 610065, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinlin","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Computer Science, Sichuan University, Chengdu 610065, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Computer Science, Sichuan University, Chengdu 610065, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yulong","family":"Ji","sequence":"additional","affiliation":[{"name":"School of Aeronautics and Astronautics, Sichuan University, Chengdu 610065, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6116-3194","authenticated-orcid":false,"given":"Jinchang","family":"Ren","sequence":"additional","affiliation":[{"name":"National Subsea Centre, Robert Gordon University, Aberdeen AB21 0BH, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1007\/s42405-019-00216-y","article-title":"Present and Future of Air Navigation: PBN Operations and Supporting Technologies","volume":"21","author":"Serna","year":"2020","journal-title":"Int. J. Aeronaut. Sp. Sci."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Israel, E., Justin Barnes, W., and Smith, L. (2020, January 8\u201310). Automating the Design of Instrument Flight Procedures. Proceedings of the 2020 Integrated Communications Navigation and Surveillance Conference (ICNS), Herndon, VA, USA.","DOI":"10.1109\/ICNS50378.2020.9222871"},{"key":"ref_3","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013, December 19). Playing Atari with Deep Reinforcement Learning. Available online: http:\/\/arxiv.org\/abs\/1312.5602."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"113456","DOI":"10.1016\/j.eswa.2020.113456","article-title":"Financial portfolio optimization with online deep reinforcement learning and restricted stacked autoencoder\u2014DeepBreath","volume":"156","author":"Soleymani","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Bayerlein, H., Theile, M., Caccamo, M., and Gesbert, D. (2020, January 7\u201311). UAV Path Planning for Wireless Data Harvesting: A Deep Reinforcement Learning Approach. Proceedings of the GLOBECOM 2020\u20142020 IEEE Global Communications Conference, Taipei, Taiwan.","DOI":"10.1109\/GLOBECOM42002.2020.9322234"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hua, J., Zeng, L., Li, G., and Ju, Z. (2021). Learning for a robot: Deep reinforcement learning, imitation learning, transfer learning. Sensors, 21.","DOI":"10.3390\/s21041278"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1007\/978-3-319-56991-8_32","article-title":"Deep Reinforcement Learning: An Overview","volume":"16","author":"Mousavi","year":"2018","journal-title":"Lect. Notes Netw. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1726","DOI":"10.1631\/FITEE.1900533","article-title":"Deep reinforcement learning: A survey","volume":"21","author":"Wang","year":"2020","journal-title":"Front. Inf. Technol. Electron. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Tang, C., and Lai, Y. (2020, January 1\u20134). Deep Reinforcement Learning Automatic Landing Control of Fixed-Wing Aircraft Using Deep Deterministic Policy Gradient. Proceedings of the 2020 International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece.","DOI":"10.1109\/ICUAS48674.2020.9213987"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Huang, X., Luo, W., and Liu, J. (2019, January 22\u201324). Attitude Control of Fixed-wing UAV Based on DDQN. Proceedings of the 2019 Chinese Automation Congress (CAC), Hangzhou, China.","DOI":"10.1109\/CAC48633.2019.8996970"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Bohn, E., Coates, E., Moe, S., and Johansen, T. (2019, January 11\u201314). Deep reinforcement learning attitude control of fixed-wing UAVs using proximal policy optimization. Proceedings of the 2019 International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta, GA, USA.","DOI":"10.1109\/ICUAS.2019.8798254"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zu, W., Yang, H., Liu, R., and Ji, Y. (2021). A multi-dimensional goal aircraft guidance approach based on reinforcement learning with a reward shaping algorithm. Sensors, 21.","DOI":"10.3390\/s21165643"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Pope, A.P., Ide, J.S., Mi\u0107ovi\u0107, D., Diaz, H., Rosenbluth, D., Ritholtz, L., Twedt, J.C., Walker, T.T., Alcedo, K., and Javorsek, D. (2021, January 15\u201318). Hierarchical Reinforcement Learning for Air-to-Air Combat. Proceedings of the 2021 International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece.","DOI":"10.1109\/ICUAS51884.2021.9476700"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Long, Y., and He, H. (2020, January 11\u201313). Robot path planning based on deep reinforcement learning. Proceedings of the 2020 IEEE Conference on Telecommunications, Optics and Computer Science (TOCS), Shenyang, China.","DOI":"10.1109\/TOCS50858.2020.9339752"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/s10846-019-01073-3","article-title":"Towards Real-Time Path Planning through Deep Reinforcement Learning for a UAV in Dynamic Environments","volume":"98","author":"Yan","year":"2020","journal-title":"J. Intell. Robot. Syst. Theory Appl."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Guo, S., Zhang, X., Zheng, Y., and Du, Y. (2020). An autonomous path planning model for unmanned ships based on deep reinforcement learning. Sensors, 20.","DOI":"10.3390\/s20020426"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"29064","DOI":"10.1109\/ACCESS.2020.2971780","article-title":"Path Planning for UAV Ground Target Tracking via Deep Reinforcement Learning","volume":"8","author":"Li","year":"2020","journal-title":"IEEE Access"},{"key":"ref_19","first-page":"5781591","article-title":"Dynamic Path Planning of Unknown Environment Based on Deep Reinforcement Learning","volume":"2018","author":"Lei","year":"2018","journal-title":"J. Robot."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Sun, W., Tsiotras, P., Lolla, T., Subramani, D., and Lermusiaux, P. (2017, January 24\u201326). Pursuit-evasion games in dynamic flow fields via reachability set analysis. Proceedings of the 2017 American Control Conference (ACC), Seattle, WA, USA.","DOI":"10.23919\/ACC.2017.7963664"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.automatica.2017.11.035","article-title":"Efficient path planning algorithms in reach-avoid problems","volume":"89","author":"Zhou","year":"2018","journal-title":"Automatica"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Takei, R., Huang, H., Ding, J., and Tomlin, C. (2012, January 14\u201318). Time-optimal multi-stage motion planning with guaranteed collision avoidance via an open-loop game formulation. Proceedings of the 2012 IEEE International Conference on Robotics and Automation, St. Paul, MN, USA.","DOI":"10.1109\/ICRA.2012.6225074"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ramana, M.V., and Kothari, M. (2015, January 15\u201318). A cooperative pursuit-evasion game of a high speed evader. Proceedings of the 2015 54th IEEE Conference on Decision and Control (CDC), Osaka, Japan.","DOI":"10.1109\/CDC.2015.7402668"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Berndt, J. (2004, January 16\u201319). JSBSim: An open source flight dynamics model in C++. Proceedings of the AIAA Modeling and Simulation Technologies Conference and Exhibit, Providence, RI, USA.","DOI":"10.2514\/6.2004-4923"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Gage, S. (2003, January 11\u201314). Creating a unified graphical wind turbulence model from multiple specifications. Proceedings of the AIAA Modeling and Simulation Technologies Conference and Exhibit, Austin, TX, USA.","DOI":"10.2514\/6.2003-5529"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"54910","DOI":"10.1109\/ACCESS.2020.2977693","article-title":"Wind Measurement and Simulation Techniques in Multi-Rotor Small Unmanned Aerial Vehicles","volume":"8","author":"Abichandani","year":"2022","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1016\/j.ast.2015.09.035","article-title":"Design, implementation and flight testing of advanced RNP to SBAS LPV approaches in Germany","volume":"47","author":"Dautermann","year":"2015","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_28","unstructured":"International Civil Aviation Organization (2006). Doc 8168 Aircraft Operations. Volume I\u2014Flight Procedures, Glory Master International Limited. [5th ed.]."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Yarats, D., Zhang, A., Kostrikov, I., Amos, B., Pineau, J., and Fergus, R. (2021, January 2\u20139). Improving Sample Efficiency in Model-Free Reinforcement Learning from Images. Proceedings of the AAAI Conference on Artificial Intelligence, Virtually.","DOI":"10.1609\/aaai.v35i12.17276"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Nagabandi, A., Kahn, G., Fearing, R., and Levine, S. (2018, January 21\u201325). Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8463189"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2017","DOI":"10.1162\/089976699300016070","article-title":"A Unified Analysis of Value-Function-Based Reinforcement- Learning Algorithms","volume":"11","author":"Algorithms","year":"1999","journal-title":"Neural Comput."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"103919","DOI":"10.1016\/j.engappai.2020.103919","article-title":"Policy-based reinforcement learning for time series anomaly detection","volume":"95","author":"Yu","year":"2020","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_33","first-page":"1057","article-title":"Policy Gradient Methods for Reinforcement Learning with Function Approximation","volume":"70","author":"Paczkowski","year":"1999","journal-title":"Pulp Pap."},{"key":"ref_34","first-page":"1928","article-title":"Asynchronous Methods for Deep Reinforcement Learning","volume":"48","author":"Mnih","year":"2013","journal-title":"Int. Conf. Mach. Learn."},{"key":"ref_35","unstructured":"Hull, D. (2007). Fundamentals of Airplane Flight Mechanics, Springer."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Clarke, S., and Hwang, I. (2020, January 6\u201310). Deep reinforcement learning control for aerobatic maneuvering of agile fixed-wing aircraft. Proceedings of the AIAA Scitech 2020 Forum, Orlando, FL, USA.","DOI":"10.2514\/6.2020-0136"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Varghese, N., and Mahmoud, Q. (2020). A survey of multi-task deep reinforcement learning. Electronics, 9.","DOI":"10.3390\/electronics9091363"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.patrec.2021.10.025","article-title":"Effective extraction of ventricles and myocardium objects from cardiac magnetic resonance images with a multi-task learning U-Net","volume":"155","author":"Ren","year":"2022","journal-title":"Pattern Recognit. Lett."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Hessel, M., Soyer, H., Espeholt, L., Czarnecki, W., Schmitt, S., and van Hasselt, H. (February, January 27). Multi-task deep reinforcement learning with PopArt. Proceedings of the 33rd AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.","DOI":"10.1609\/aaai.v33i01.33013796"},{"key":"ref_40","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 11\u201313). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/17\/6475\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:19:02Z","timestamp":1760141942000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/17\/6475"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,28]]},"references-count":40,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["s22176475"],"URL":"https:\/\/doi.org\/10.3390\/s22176475","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,28]]}}}