{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T04:51:51Z","timestamp":1785387111170,"version":"3.55.0"},"reference-count":83,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T00:00:00Z","timestamp":1725494400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T00:00:00Z","timestamp":1725494400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Institute of Information & communications Technology Planning & Evaluation","award":["No.2019-0-00231"],"award-info":[{"award-number":["No.2019-0-00231"]}]},{"name":"Information Technology Research Center","award":["IITP-2022-RS-2022-00156354"],"award-info":[{"award-number":["IITP-2022-RS-2022-00156354"]}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"crossref","award":["2020R1A6A1A03038540"],"award-info":[{"award-number":["2020R1A6A1A03038540"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Artif Intell Rev"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Recently, machine learning has been very useful in solving diverse tasks with drones, such as autonomous navigation, visual surveillance, communication, disaster management, and agriculture. Among these machine learning, two representative paradigms have been widely utilized in such applications: supervised learning and reinforcement learning. Researchers prefer to use supervised learning, mostly based on convolutional neural networks, because of its robustness and ease of use but yet data labeling is laborious and time-consuming. On the other hand, when traditional reinforcement learning is combined with the deep neural network, it can be a very powerful tool to solve high-dimensional input problems such as image and video. Along with the fast development of reinforcement learning, many researchers utilize reinforcement learning in drone applications, and it often outperforms supervised learning. However, it usually requires the agent to explore the environment on a trial-and-error basis which is high cost and unrealistic in the real environment. Recent advances in simulated environments can allow an agent to learn by itself to overcome these drawbacks, although the gap between the real environment and the simulator has to be minimized in the end. In this sense, a realistic and reliable simulator is essential for reinforcement learning training. This paper investigates various drone simulators that work with diverse reinforcement learning architectures. The characteristics of the reinforcement learning-based drone simulators are analyzed and compared for the researchers who would like to employ them for their projects. Finally, we shed light on some challenges and potential directions for future drone simulators.<\/jats:p>","DOI":"10.1007\/s10462-024-10933-w","type":"journal-article","created":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T07:05:36Z","timestamp":1725519936000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["Reinforcement learning-based drone simulators: survey, practice, and challenge"],"prefix":"10.1007","volume":"57","author":[{"given":"Jun Hoong","family":"Chan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kai","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"A. S. M. Sharifuzzaman","family":"Sagar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yong-Guk","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,9,5]]},"reference":[{"key":"10933_CR1","doi-asserted-by":"crossref","unstructured":"Afzal A, Katz DS, Goues CL, Timperley CS (2020) A study on the challenges of using robotics simulators for testing. arXiv:2004.07368","DOI":"10.1109\/ICST46399.2020.00020"},{"key":"10933_CR2","doi-asserted-by":"publisher","first-page":"172988141987093","DOI":"10.1177\/1729881419870937","volume":"16","author":"A Almousa","year":"2019","unstructured":"Almousa A, Sababha B, Al-Madi N, Barghouthi A, Younisse R (2019) Utsim: a framework and simulator for UAV air traffic integration, control, and communication. Int J Adv Robot Syst 16:172988141987093. https:\/\/doi.org\/10.1177\/1729881419870937","journal-title":"Int J Adv Robot Syst"},{"key":"10933_CR3","unstructured":"Altaweel M (2022) The use of drones in human and physical geography. https:\/\/www.gislounge.com\/use-drones-human-physical-geography\/. Accessed 27 Mar 2022"},{"key":"10933_CR4","doi-asserted-by":"crossref","unstructured":"Anwar A, Raychowdhury A (2019) Autonomous navigation via deep reinforcement learning for resource constraint edge nodes using transfer learning. arXiv:1910.05547","DOI":"10.1109\/ACCESS.2020.2971172"},{"key":"10933_CR5","unstructured":"Babushkin A (2022) jMAVSim. https:\/\/github.com\/DrTon\/jMAVSim. Accessed 27 Mar 2022"},{"key":"10933_CR6","doi-asserted-by":"crossref","unstructured":"Berndt J (2004) JSBSim: An open source flight dynamics model in c++. AIAA Modeling and Simulation Technologies Conference and Exhibit","DOI":"10.2514\/6.2004-4923"},{"key":"10933_CR7","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1108\/00022660910926890","volume":"81","author":"E Capello","year":"2009","unstructured":"Capello E, Guglieri G, Quagliotti F (2009) UAVS and simulation: an experience on MAVS. Aircr Eng Aerosp Technol 81:38\u201350. https:\/\/doi.org\/10.1108\/00022660910926890","journal-title":"Aircr Eng Aerosp Technol"},{"key":"10933_CR8","doi-asserted-by":"publisher","unstructured":"Carpin S, Lewis M, Wang J, Balakirsky S, Scrapper C (2007) Usarsim: a robot simulator for research and education. In: Proceedings 2007 IEEE international conference on robotics and automation, pp 1400\u20131405. https:\/\/doi.org\/10.1109\/ROBOT.2007.363180","DOI":"10.1109\/ROBOT.2007.363180"},{"key":"10933_CR9","unstructured":"Coumans E, Bai Y (2016\u20132021) PyBullet, a Python module for physics simulation for games, robotics and machine learning. http:\/\/pybullet.org"},{"key":"10933_CR10","doi-asserted-by":"publisher","unstructured":"Dankwa S, Zheng W (2019) Twin-delayed DDPG: a deep reinforcement learning technique to model a continuous movement of an intelligent robot agent. Association for Computing Machinery, New York. https:\/\/doi.org\/10.1145\/3387168.3387199","DOI":"10.1145\/3387168.3387199"},{"key":"10933_CR11","unstructured":"Dhariwal P, Hesse C, Klimov O, Nichol A, Plappert M, Radford A, Schulman J, Sidor S, Wu Y, Zhokhov P (2017) OpenAI Baselines. GitHub"},{"key":"10933_CR12","doi-asserted-by":"publisher","DOI":"10.1142\/S2301385019400065","author":"J Douthwaite","year":"2019","unstructured":"Douthwaite J, Zhao S, Mihaylova L (2019) Velocity obstacle approaches for multi-agent collision avoidance. Unmanned Syst. https:\/\/doi.org\/10.1142\/S2301385019400065","journal-title":"Unmanned Syst"},{"key":"10933_CR13","unstructured":"DroneSimPro: DroneSimPro Drone Simulator. https:\/\/www.dronesimpro.com"},{"key":"10933_CR14","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.micpro.2018.05.002","volume":"61","author":"E Ebeid","year":"2018","unstructured":"Ebeid E, Skriver M, Terkildsen KH, Jensen K, Schultz UP (2018) A survey of open-source UAV flight controllers and flight simulators. Microprocess Microsyst 61:11\u201320. https:\/\/doi.org\/10.1016\/j.micpro.2018.05.002","journal-title":"Microprocess Microsyst"},{"key":"10933_CR15","doi-asserted-by":"publisher","unstructured":"Echeverria G, Lemaignan S, Degroote A, Lacroix S, Karg M, Koch P, Lesire C, Stinckwich S (2012) Simulating complex robotic scenarios with morse, vol 7628. https:\/\/doi.org\/10.1007\/978-3-642-34327-8_20","DOI":"10.1007\/978-3-642-34327-8_20"},{"key":"10933_CR16","unstructured":"Encyclopedia (2022) Flight Simulator. https:\/\/www.newworldencyclopedia.org\/entry\/Flight_simulator. Accessed 27 Mar 2022"},{"key":"10933_CR17","unstructured":"Foundation AK (2022) Drones for hazard assessment and disaster management. https:\/\/www.akdn.org\/press-release\/drones-hazard- assessment-and-disaster-management. Accessed 27 Mar 2022"},{"key":"10933_CR18","doi-asserted-by":"publisher","first-page":"595","DOI":"10.1007\/978-3-319-26054-9_23","volume":"625","author":"F Furrer","year":"2016","unstructured":"Furrer F, Burri M, Achtelik M, Siegwart R (2016) RotorS\u2014a modular gazebo MAV simulator. Framework 625:595\u2013625. https:\/\/doi.org\/10.1007\/978-3-319-26054-9_23","journal-title":"Framework"},{"key":"10933_CR19","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1097\/SIH.0b013e31817bb8f6","volume":"3","author":"R Glavin","year":"2008","unstructured":"Glavin R, Gaba D (2008) Challenges and opportunities in simulation and assessment. Simul Healthc 3:69\u201371. https:\/\/doi.org\/10.1097\/SIH.0b013e31817bb8f6","journal-title":"Simul Healthc"},{"key":"10933_CR20","doi-asserted-by":"crossref","unstructured":"Guerra W, Tal E, Murali V, Ryou G, Karaman S (2019) FlightGoggles: photorealistic sensor simulation for perception-driven Robotics using Photogrammetry and Virtual Reality","DOI":"10.1109\/IROS40897.2019.8968116"},{"key":"10933_CR21","unstructured":"Haarnoja T, Zhou A, Abbeel P, Levine S (2018) Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: ICML"},{"key":"10933_CR22","unstructured":"Haas JK (2014) A history of the unity game engine"},{"key":"10933_CR24","unstructured":"Hartmann K, Steup C (2013) The vulnerability of UAVS to cyber attacks\u2014an approach to the risk assessment. In: 2013 5th international conference on cyber conflict (CYCON 2013), pp 1\u201323"},{"key":"10933_CR25","unstructured":"Harwood R (2019) The challenges to developing fully autonomous drone technology. Ansys, Com"},{"key":"10933_CR26","unstructured":"Hasselt H, Guez A, Silver D (2015) Deep reinforcement learning with double Q-learning. arXiv:1509.06461"},{"key":"10933_CR27","doi-asserted-by":"publisher","unstructured":"Hattenberger G, Bronz M, Gorraz M (2014). Using the paparazzi UAV system for scientific research. https:\/\/doi.org\/10.4233\/uuid:b38fbdb7-e6bd-440d-93be-f7dd1457be60","DOI":"10.4233\/uuid:b38fbdb7-e6bd-440d-93be-f7dd1457be60"},{"key":"10933_CR28","unstructured":"Hill A, Raffin A, Ernestus M, Gleave A, Kanervisto A, Traore R, Dhariwal P, Hesse C, Klimov O, Nichol A, Plappert M, Radford A, Schulman J, Sidor S, Wu Y (2018) Stable Baselines. GitHub"},{"key":"10933_CR29","unstructured":"Horizon\u00a0Hobby L. RealFlight\u00ae9.5 Flight Simulator. https:\/\/www.realflight.com\/"},{"key":"10933_CR30","doi-asserted-by":"publisher","first-page":"931","DOI":"10.1177\/0037549716666683","volume":"92","author":"Y Hu","year":"2016","unstructured":"Hu Y, Meng W (2016) Rosunitysim: development and experimentation of a real-time simulator for multi-UAV local planning. Simulation 92:931\u2013944. https:\/\/doi.org\/10.1177\/0037549716666683","journal-title":"Simulation"},{"key":"10933_CR31","doi-asserted-by":"publisher","unstructured":"Javaid AY, Sun W, Alam M (2013) Uavsim: a simulation testbed for unmanned aerial vehicle network cyber security analysis. In: 2013 IEEE Globecom workshops (GC Wkshps), pp 1432\u20131436 . https:\/\/doi.org\/10.1109\/GLOCOMW.2013.6825196","DOI":"10.1109\/GLOCOMW.2013.6825196"},{"issue":"4","key":"10933_CR32","doi-asserted-by":"publisher","DOI":"10.4108\/sis.2.4.e4","volume":"2","author":"AY Javaid","year":"2015","unstructured":"Javaid AY, Sun W, Alam M (2015) Single and multiple UAV cyber-attack simulation and performance evaluation. EAI Endors Trans Scalable Inf Syst 2(4):e4. https:\/\/doi.org\/10.4108\/sis.2.4.e4","journal-title":"EAI Endors Trans Scalable Inf Syst"},{"key":"10933_CR33","unstructured":"Karpowicz J (2022) UAVs as solutions to dull, dirty, and dangerous jobs. https:\/\/www.commercialuavnews.com\/construction\/uavs-solutions-dull-dirty-dangerous-jobs. Accessed 27 Mar 2022"},{"key":"10933_CR34","doi-asserted-by":"publisher","unstructured":"Kate B, Waterman J, Dantu K, Welsh M (2012) Simbeeotic: a simulator and testbed for micro-aerial vehicle swarm experiments. In: 2012 ACM\/IEEE 11th international conference on information processing in sensor networks (IPSN), pp 49\u201360.https:\/\/doi.org\/10.1109\/IPSN.2012.6920950","DOI":"10.1109\/IPSN.2012.6920950"},{"key":"10933_CR35","unstructured":"Koch W (2019) Flight controller synthesis via deep reinforcement learning. arXiv preprint arXiv:1909.06493"},{"issue":"2","key":"10933_CR36","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1145\/3301273","volume":"3","author":"W Koch","year":"2019","unstructured":"Koch W, Mancuso R, West R, Bestavros A (2019) Reinforcement learning for UAV attitude control. ACM Trans Cyber-Phys Syst 3(2):22","journal-title":"ACM Trans Cyber-Phys Syst"},{"key":"10933_CR37","doi-asserted-by":"publisher","unstructured":"Koenig N, Howard A (2004) Design and use paradigms for gazebo, an open-source multi-robot simulator. In: 2004 IEEE\/RSJ international conference on intelligent robots and systems (IROS) (IEEE Cat. No.04CH37566), vol 3, pp 2149\u201321543. https:\/\/doi.org\/10.1109\/IROS.2004.1389727","DOI":"10.1109\/IROS.2004.1389727"},{"key":"10933_CR38","doi-asserted-by":"publisher","unstructured":"Krishna CGL, Murphy RR (2017) A review on cybersecurity vulnerabilities for unmanned aerial vehicles. In: 2017 IEEE international symposium on safety, security and rescue robotics (SSRR), pp 194\u2013199. https:\/\/doi.org\/10.1109\/SSRR.2017.8088163","DOI":"10.1109\/SSRR.2017.8088163"},{"key":"10933_CR39","doi-asserted-by":"crossref","unstructured":"Krishnan S, Boroujerdian B, Fu W, Faust A, Reddi VJ (2021) Air learning: a deep reinforcement learning gym for autonomous aerial robot visual navigation. Mach Learn 1\u201340","DOI":"10.1007\/s10994-021-06006-6"},{"key":"10933_CR40","doi-asserted-by":"publisher","unstructured":"La WG, Park S, Kim H (2017) D-muns: distributed multiple UAVS\u2019 network simulator. In: 2017 ninth international conference on ubiquitous and future networks (ICUFN), pp 15\u201317. https:\/\/doi.org\/10.1109\/ICUFN.2017.7993738","DOI":"10.1109\/ICUFN.2017.7993738"},{"key":"10933_CR41","doi-asserted-by":"publisher","first-page":"1421","DOI":"10.1613\/jair.1.12412","volume":"69","author":"A Lazaridis","year":"2020","unstructured":"Lazaridis A, Fachantidis A, Vlahavas IP (2020) Deep reinforcement learning: a state-of-the-art walkthrough. J Artif Intell Res 69:1421\u20131471","journal-title":"J Artif Intell Res"},{"key":"10933_CR42","unstructured":"League TDR. The Drone Racing League Simulator. https:\/\/www.thedroneracingleague.com\/play\/"},{"key":"10933_CR43","doi-asserted-by":"publisher","unstructured":"Lepej P, Santamaria-Navarro A, Sol\u00e0 J (2017) A flexible hardware-in-the-loop architecture for uavs. In: 2017 international conference on unmanned aircraft systems (ICUAS), pp 1751\u20131756. https:\/\/doi.org\/10.1109\/ICUAS.2017.7991330","DOI":"10.1109\/ICUAS.2017.7991330"},{"key":"10933_CR44","doi-asserted-by":"publisher","DOI":"10.1155\/2015\/745303","author":"W Liang","year":"2015","unstructured":"Liang W, Li Z, Zhang H, Wang S, Bie R (2015) Vehicular ad hoc networks: architectures, research issues, methodologies, challenges, and trends. Int J Distrib Sens Netw. https:\/\/doi.org\/10.1155\/2015\/745303","journal-title":"Int J Distrib Sens Netw"},{"key":"10933_CR45","unstructured":"Little Arms\u00a0Studios L. Zephyr Simulator. https:\/\/zephyr-sim.com\/"},{"key":"10933_CR46","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1016\/j.simpat.2019.01.004","volume":"94","author":"A Mairaj","year":"2019","unstructured":"Mairaj A, Baba AI, Javaid AY (2019) Application specific drone simulators: recent advances and challenges. Simul Model Pract Theory 94:100\u2013117. https:\/\/doi.org\/10.1016\/j.simpat.2019.01.004","journal-title":"Simul Model Pract Theory"},{"key":"10933_CR47","doi-asserted-by":"crossref","unstructured":"Marconato EA, Rodrigues M, Melo\u00a0Pires R, Pigatto DF, Filho LCQ, Pinto ASR, Branco KC (2017) Avens\u2014a novel flying ad hoc network simulator with automatic code generation for unmanned aircraft system. In: HICSS","DOI":"10.24251\/HICSS.2017.760"},{"issue":"7540","key":"10933_CR48","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015a) Human-level control through deep reinforcement learning. Nature 518(7540):529\u2013533","journal-title":"Nature"},{"key":"10933_CR50","unstructured":"Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2015b) Playing Atari with Deep Reinforcement Learning. arXiv:1312.5602"},{"key":"10933_CR49","unstructured":"Mnih V, Badia AP, Mirza M, Graves A, Lillicrap T, Harley T, Silver D, Kavukcuoglu K (2016) Asynchronous methods for deep reinforcement learning. In: Balcan MF, Weinberger KQ (eds) Proceedings of The 33rd international conference on machine learning. proceedings of machine learning research, vol 48, pp 1928\u20131937. PMLR, New York. https:\/\/proceedings.mlr.press\/v48\/mniha16.html"},{"key":"10933_CR51","unstructured":"Museum NFL (2022) The Link Trainer Flight Simulator. https:\/\/www.nasflmuseum.com\/link-trainer.html. Accessed 27 Mar 2022"},{"key":"10933_CR52","unstructured":"Newman C (2022) Are drones and flying taxis the future of aviation? https:\/\/newseu.cgtn.com\/news\/2020-11-05\/Are-drones-and-flying-taxis-the-future-of-aviation--V9tpg618Bi\/index.html. Accessed 27 Mar 2022"},{"key":"10933_CR53","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/1818\/1\/012104","volume":"1818","author":"M Obaid","year":"2021","unstructured":"Obaid M, Mebayet S (2021) Drone controlled real live flight simulator. J Phys 1818:012104. https:\/\/doi.org\/10.1088\/1742-6596\/1818\/1\/012104","journal-title":"J Phys"},{"key":"10933_CR54","unstructured":"Ogre3D (2024) Ogre3D: Open Source 3D Graphics Engine. https:\/\/www.ogre3d.org. Accessed 18 Aug 2024"},{"key":"10933_CR55","unstructured":"Page RL (2004) Brief history of flight simulation"},{"key":"10933_CR56","doi-asserted-by":"crossref","unstructured":"Panerati J, Zheng H, Zhou S, Xu J, Prorok A, Schoellig AP (2021) Learning to fly\u2014a gym environment with pybullet physics for reinforcement learning of multi-agent quadcopter control. In: 2021 IEEE\/RSJ international conference on intelligent robots and systems (IROS)","DOI":"10.1109\/IROS51168.2021.9635857"},{"key":"10933_CR57","unstructured":"Pathmind (2022) A beginner\u2019s guide to deep reinforcement learning. https:\/\/wiki.pathmind.com\/deep-reinforcement-learning Accessed 27 Mar 2022"},{"key":"10933_CR58","doi-asserted-by":"publisher","unstructured":"Pianpak P, Son T, Toups Z (2018) A multi-agent simulator environment based on the robot operating system for human-robot interaction applications: 21st international conference, Tokyo, Japan, October 29-November 2, 2018, Proceedings, pp 612\u2013620. https:\/\/doi.org\/10.1007\/978-3-030-03098-8_48","DOI":"10.1007\/978-3-030-03098-8_48"},{"key":"10933_CR59","unstructured":"Plappert M (2016) keras-rl. GitHub"},{"key":"10933_CR60","doi-asserted-by":"publisher","DOI":"10.22214\/ijraset.2019.5475","author":"K Pradheep","year":"2019","unstructured":"Pradheep K (2019) Crop monitoring by drone for plant pathology. Int J Res Appl Sci Eng Technol. https:\/\/doi.org\/10.22214\/ijraset.2019.5475","journal-title":"Int J Res Appl Sci Eng Technol"},{"key":"10933_CR61","unstructured":"Reich L (2022) How drones are being used in disaster management? http:\/\/geoawesomeness.com\/drones-fly-rescue\/. Accessed 27 Mar 2022"},{"key":"10933_CR62","doi-asserted-by":"publisher","unstructured":"Rohmer E, Singh SPN, Freese M (2013) V-rep: a versatile and scalable robot simulation framework. In: 2013 IEEE\/RSJ international conference on intelligent robots and systems, pp 1321\u20131326. https:\/\/doi.org\/10.1109\/IROS.2013.6696520","DOI":"10.1109\/IROS.2013.6696520"},{"key":"10933_CR63","unstructured":"Schr\u00f6der D, Vorlaender M (2011) Raven: a real-time framework for the auralization of interactive virtual environments. In: Proceedings of forum acusticum, pp 1541\u20131546"},{"key":"10933_CR64","unstructured":"Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. arXiv:1707.06347"},{"key":"10933_CR65","doi-asserted-by":"crossref","unstructured":"Shah S, Dey D, Lovett C, Kapoor A (2017) Airsim: high-fidelity visual and physical simulation for autonomous vehicles. In: Field and service robotics. arXiv:1705.05065","DOI":"10.1007\/978-3-319-67361-5_40"},{"key":"10933_CR66","doi-asserted-by":"publisher","first-page":"48572","DOI":"10.1109\/ACCESS.2019.2909530","volume":"7","author":"H Shakhatreh","year":"2019","unstructured":"Shakhatreh H, Sawalmeh AH, Al-Fuqaha A, Dou Z, Almaita E, Khalil I, Othman NS, Khreishah A, Guizani M (2019) Unmanned aerial vehicles (UAVS): a survey on civil applications and key research challenges. IEEE Access 7:48572\u201348634. https:\/\/doi.org\/10.1109\/ACCESS.2019.2909530","journal-title":"IEEE Access"},{"issue":"24","key":"10933_CR67","doi-asserted-by":"publisher","first-page":"5571","DOI":"10.3390\/app9245571","volume":"9","author":"S-Y Shin","year":"2019","unstructured":"Shin S-Y, Kang Y-W, Kim Y-G (2019) Obstacle avoidance drone by deep reinforcement learning and its racing with human pilot. Appl Sci 9(24):5571","journal-title":"Appl Sci"},{"key":"10933_CR68","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2019.113064","volume":"143","author":"S-Y Shin","year":"2020","unstructured":"Shin S-Y, Kang Y-W, Kim Y-G (2020) Reward-driven u-net training for obstacle avoidance drone. Expert Syst Appl 143:113064","journal-title":"Expert Syst Appl"},{"key":"10933_CR69","unstructured":"Silver D (2022) Deep Reinforcement Learning. https:\/\/deepmind.com\/blog\/article\/deep-reinforcement-learning. Accessed 27 Mar 2022"},{"key":"10933_CR70","unstructured":"SIMULATOR RD. Real Drone Simulator. https:\/\/www.realdronesimulator.com\/"},{"key":"10933_CR71","unstructured":"Song Y, Naji S, Kaufmann E, Loquercio A, Scaramuzza D (2021) Flightmare: a flexible quadrotor simulator. In: Proceedings of the 2020 conference on robot learning, pp 1147\u20131157"},{"key":"10933_CR72","doi-asserted-by":"publisher","first-page":"3179","DOI":"10.1016\/j.comnet.2011.05.007","volume":"55","author":"R Stanica","year":"2011","unstructured":"Stanica R, Chaput E, Beylot A-L (2011) Simulation of vehicular ad-hoc networks: challenges, review of tools and recommendations. Comput Netw 55:3179\u20133188. https:\/\/doi.org\/10.1016\/j.comnet.2011.05.007","journal-title":"Comput Netw"},{"key":"10933_CR74","unstructured":"Sutton RS (2018) Reinforcement learning: an introduction. A Bradford Book"},{"key":"10933_CR73","unstructured":"Sutton RS, McAllester D, Singh S, Mansour Y (1999) Policy gradient methods for reinforcement learning with function approximation. In: Proceedings of the 12th international conference on neural information processing systems. NIPS\u201999, pp1057\u20131063. MIT Press, Cambridge"},{"key":"10933_CR75","unstructured":"Team AD (2020) OpenGLMetal,. https:\/\/developer.apple.com\/documentation\/metal\/"},{"key":"10933_CR76","unstructured":"team TR (2017) RLlib: Scalable Reinforcement Learning. GitHub"},{"key":"10933_CR77","unstructured":"Team VD (2020) OpenGLMetal, https:\/\/www.vulkan.org\/"},{"key":"10933_CR78","doi-asserted-by":"publisher","first-page":"199","DOI":"10.19062\/2247-3173.2016.18.1.26","volume":"18","author":"G Udeanu","year":"2016","unstructured":"Udeanu G, Dobrescu A, Oltean M (2016) Unmanned aerial vehicle in military operations. Sci Res Educ Air Force 18:199\u2013206. https:\/\/doi.org\/10.19062\/2247-3173.2016.18.1.26","journal-title":"Sci Res Educ Air Force"},{"key":"10933_CR79","unstructured":"Wikipedia (2022) Flight Simulator. https:\/\/en.wikipedia.org\/wiki\/Flight_simulator. Accessed 27 Mar 2022"},{"key":"10933_CR80","unstructured":"Woo M, Neider J, Davis T, Shreiner D (1999) Opengl programming guide: the official guide to learning opengl, version 1.2"},{"key":"10933_CR81","unstructured":"Wu Y, Mansimov E, Liao S, Grosse R, Ba J (2017) Scalable trust-region method for deep reinforcement learning using kronecker-factored approximation. In: Proceedings of the 31st international conference on neural information processing systems. NIPS\u201917, pp 5285\u20135294. Curran Associates Inc., Red Hook"},{"key":"10933_CR82","unstructured":"X-Plane (2022) X-Plane. https:\/\/www.x-plane.com\/. Accessed 27 Mar 2022"},{"key":"10933_CR84","unstructured":"Zhang K, Yang Z, Basar T (2019) Multi-agent reinforcement learning: a selective overview of theories and algorithms"},{"key":"10933_CR83","unstructured":"Zhang F, Hall D, Xu T, Boyle S, Bull D (2020) A simulation environment for drone cinematography. arXiv. arXiv:2010.01315"}],"container-title":["Artificial Intelligence Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-024-10933-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10462-024-10933-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-024-10933-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,25]],"date-time":"2024-09-25T03:46:11Z","timestamp":1727235971000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10462-024-10933-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,5]]},"references-count":83,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2024,10]]}},"alternative-id":["10933"],"URL":"https:\/\/doi.org\/10.1007\/s10462-024-10933-w","relation":{},"ISSN":["1573-7462"],"issn-type":[{"value":"1573-7462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,5]]},"assertion":[{"value":"28 August 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 September 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"281"}}