{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T17:48:59Z","timestamp":1776188939385,"version":"3.50.1"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,3,1]],"date-time":"2024-03-01T00:00:00Z","timestamp":1709251200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,5]],"date-time":"2024-03-05T00:00:00Z","timestamp":1709596800000},"content-version":"vor","delay-in-days":4,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Virtual Reality"],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Safe autonomous landing for Unmanned Aerial Vehicles (UAVs) in populated areas is a crucial aspect for successful integration of UAVs in populated environments. Nonetheless, validating autonomous landing in real scenarios is a challenging task with a high risk of injuring people. In this work, we propose a framework for safe real-time and thorough evaluation of vision-based autonomous landing in populated scenarios, using photo-realistic virtual environments and physics-based simulation. The proposed evaluation pipeline includes the use of Unreal graphics engine coupled with AirSim for realistic drone simulation to evaluate landing strategies. Then, Software-\/Hardware-In-The-Loop can be used to test beforehand the performance of the algorithms. The final validation stage consists in a Robot-In-The-Loop evaluation strategy where a real drone must perform autonomous landing maneuvers in real-time, with an avatar drone in a virtual environment mimicking its behavior, while the detection algorithms run in the virtual environment (virtual reality to the robot). This method determines the safe landing areas based on computer vision and convolutional neural networks to avoid colliding with people in static and dynamic scenarios. To test the robustness of the algorithms in adversary conditions, different urban-like environments were implemented, including moving agents and different weather conditions. We also propose different metrics to quantify the performance of the landing strategies, establishing a baseline for comparison with future works on this challenging task, and analyze them through several randomized iterations. The proposed approach allowed us to safely validate the autonomous landing strategies, providing an evaluation pipeline, and a benchmark for comparison. An extensive evaluation showed a 99% success rate in static scenarios and 87% in dynamic cases, demonstrating that the use of autonomous landing algorithms considerably prevents accidents involving humans, facilitating the integration of drones in human-populated spaces, which may help to unleash the full potential of drones in urban environments. Besides, this type of development helps to increase the safety of drone operations, which would advance drone flight regulations and allow their use in closer proximity to humans.<\/jats:p>","DOI":"10.1007\/s10055-024-00965-6","type":"journal-article","created":{"date-parts":[[2024,3,5]],"date-time":"2024-03-05T10:02:23Z","timestamp":1709632943000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Real-time safe validation of autonomous landing in populated areas: from virtual environments to Robot-In-The-Loop"],"prefix":"10.1007","volume":"28","author":[{"given":"Hector","family":"Tovanche-Picon","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Javier","family":"Gonz\u00e1lez-Trejo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"\u00c1ngel","family":"Flores-Abad","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Miguel \u00c1ngel","family":"Garc\u00eda-Ter\u00e1n","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Diego","family":"Mercado-Ravell","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,3,5]]},"reference":[{"key":"965_CR1","doi-asserted-by":"publisher","unstructured":"Abdollahzadeh S, Proulx P.-L, Allili M.S, Lapointe J.-F (2022) Safe landing zones detection for uavs using deep regression. In: 2022 19th conference on robots and vision (CRV), pp. 213\u2013218 . https:\/\/doi.org\/10.1109\/CRV55824.2022.00035","DOI":"10.1109\/CRV55824.2022.00035"},{"key":"965_CR2","unstructured":"Administration F.A (2021) Unmanned aircraft systems (UAS)"},{"key":"965_CR3","doi-asserted-by":"publisher","first-page":"115091","DOI":"10.1016\/j.eswa.2021.115091","volume":"179","author":"MS Alam","year":"2021","unstructured":"Alam MS, Oluoch J (2021) A survey of safe landing zone detection techniques for autonomous unmanned aerial vehicles (uavs). Expert Syst Appl 179:115091. https:\/\/doi.org\/10.1016\/j.eswa.2021.115091","journal-title":"Expert Syst Appl"},{"key":"965_CR4","doi-asserted-by":"publisher","unstructured":"Ariante G, Ponte S, Papa U, Del\u00a0Core G (2021) Safe landing area determination (slad) for unmanned aircraft systems by using rotary lidar. In: 2021 IEEE 8th international workshop on metrology for AeroSpace (MetroAeroSpace), pp. 110\u2013115 . https:\/\/doi.org\/10.1109\/MetroAeroSpace51421.2021.9511669","DOI":"10.1109\/MetroAeroSpace51421.2021.9511669"},{"key":"965_CR5","unstructured":"Authority U.K.C.A (2023) Rules and categories of drone flying. https:\/\/www.caa.co.uk\/drones\/rules-and-categories-of-drone-flying\/flying-in-the-open-category\/"},{"key":"965_CR6","doi-asserted-by":"publisher","unstructured":"Bassi E (2019) European drones regulation: today\u2019s legal challenges. In: 2019 International conference on unmanned aircraft systems (ICUAS), pp. 443\u2013450. https:\/\/doi.org\/10.1109\/ICUAS.2019.8798173","DOI":"10.1109\/ICUAS.2019.8798173"},{"issue":"29\u201347","key":"965_CR7","first-page":"5","volume":"20","author":"D Bruff","year":"2005","unstructured":"Bruff D (2005) The assignment problem and the hungarian method. Notes for Math 20(29\u201347):5","journal-title":"Notes for Math"},{"key":"965_CR8","doi-asserted-by":"publisher","first-page":"64534","DOI":"10.1109\/access.2020.2984768","volume":"8","author":"G Castellano","year":"2020","unstructured":"Castellano G, Castiello C, Mencar C, Vessio G (2020) Crowd detection in aerial images using spatial graphs and fully-convolutional neural networks. IEEE Access 8:64534\u201364544. https:\/\/doi.org\/10.1109\/access.2020.2984768","journal-title":"IEEE Access"},{"key":"965_CR9","doi-asserted-by":"publisher","DOI":"10.1007\/s10846-021-01544-6","author":"E Chatzikalymnios","year":"2022","unstructured":"Chatzikalymnios E, Moustakas K (2022) Landing site detection for autonomous rotor wing uavs using visual and structural information. J Intell Robot Syst Theory Appl. https:\/\/doi.org\/10.1007\/s10846-021-01544-6","journal-title":"J Intell Robot Syst Theory Appl"},{"key":"965_CR10","doi-asserted-by":"crossref","unstructured":"Delmerico J.A, Poranne R, Bogo F, Oleynikova H, Vollenweider E, Coros S, Nieto J.I, Pollefeys M (2022) Spatial computing and intuitive interaction: bringing mixed reality and robotics together. CoRR:2202.01493","DOI":"10.1109\/MRA.2021.3138384"},{"issue":"1","key":"965_CR11","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1016\/S0921-8890(01)00166-X","volume":"38","author":"PJ Garcia-Pardo","year":"2002","unstructured":"Garcia-Pardo PJ, Sukhatme GS, Montgomery JF (2002) Towards vision-based safe landing for an autonomous helicopter. Robot Auton Syst 38(1):19\u201329","journal-title":"Robot Auton Syst"},{"issue":"1","key":"965_CR12","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1007\/s10846-021-01380-8","volume":"102","author":"JA Gonzalez-Trejo","year":"2021","unstructured":"Gonzalez-Trejo JA, Mercado-Ravell DA (2021) Lightweight density map architecture for uavs safe landing in crowded areas. J Intell Robot Syst 102(1):7. https:\/\/doi.org\/10.1007\/s10846-021-01380-8","journal-title":"J Intell Robot Syst"},{"issue":"4","key":"965_CR13","doi-asserted-by":"publisher","first-page":"7901","DOI":"10.1109\/lra.2021.3101861","volume":"6","author":"J Gonzalez-Trejo","year":"2021","unstructured":"Gonzalez-Trejo J, Mercado-Ravell D, Becerra I, Murrieta-Cid R (2021) On the visual-based safe landing of uavs in populated areas: a crucial aspect for urban deployment. IEEE Robot Autom Lett 6(4):7901\u20137908. https:\/\/doi.org\/10.1109\/lra.2021.3101861","journal-title":"IEEE Robot Autom Lett"},{"key":"965_CR14","doi-asserted-by":"publisher","unstructured":"Guerin J, Delmas K, Guiochet J (2021) Certifying emergency landing for safe urban uav. In: 2021 51st Annual IEEE\/IFIP international conference on dependable systems and networks workshops (DSN-W), pp. 55\u201362. IEEE Computer Society, Los Alamitos, CA, USA . https:\/\/doi.org\/10.1109\/DSN-W52860.2021.00020. https:\/\/doi.ieeecomputersociety.org\/10.1109\/DSN-W52860.2021.00020","DOI":"10.1109\/DSN-W52860.2021.00020"},{"key":"965_CR15","doi-asserted-by":"publisher","unstructured":"Guerra W, Tal E, Murali V, Ryou G, Karaman S (2019) Flightgoggles: Photorealistic sensor simulation for perception-driven robotics using photogrammetry and virtual reality. In: 2019 IEEE\/RSJ International conference on intelligent robots and systems (IROS), pp. 6941\u20136948. https:\/\/doi.org\/10.1109\/IROS40897.2019.8968116","DOI":"10.1109\/IROS40897.2019.8968116"},{"key":"965_CR16","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1007\/s11370-022-00416-8","volume":"15","author":"S-N Heo","year":"2022","unstructured":"Heo S-N, Chen J, Liao Y-C, Lee H-H (2022) Auto-splitting d* lite path planning for large disaster area. Intell Serv Robot 15:289\u2013306. https:\/\/doi.org\/10.1007\/s11370-022-00416-8","journal-title":"Intell Serv Robot"},{"key":"965_CR17","unstructured":"Jaderberg M, Simonyan K, Zisserman A, Kavukcuoglu K (2015) Spatial transformer networks. preprint arXiv arXiv:1506.02025 [cs.CV]"},{"key":"965_CR18","doi-asserted-by":"publisher","unstructured":"Johnson A, Montgomery J, Matthies L (2005) Vision guided landing of an autonomous helicopter in hazardous terrain. In: IEEE international conference on robotics and automation. https:\/\/doi.org\/10.1109\/robot.2005.1570727","DOI":"10.1109\/robot.2005.1570727"},{"key":"965_CR19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3472288","volume":"54","author":"E Kakaletsis","year":"2022","unstructured":"Kakaletsis E, Symeonidis C, Tzelepi M, Mademlis I, Tefas A, Nikolaidis N, Pitas I (2022) Computer vision for autonomous uav flight safety: an overview and a vision-based safe landing pipeline example. ACM Comput Surv 54:1\u201337. https:\/\/doi.org\/10.1145\/3472288","journal-title":"ACM Comput Surv"},{"key":"965_CR20","doi-asserted-by":"publisher","unstructured":"Khazetdinov A, Zakiev A, Tsoy T, Svinin M, Magid E (2021) Embedded aruco: a novel approach for high precision uav landing. Institute of Electrical and Electronics Engineers Inc., https:\/\/doi.org\/10.1109\/SIBCON50419.2021.9438855","DOI":"10.1109\/SIBCON50419.2021.9438855"},{"key":"965_CR21","unstructured":"Kinahan J, Smeaton AF (2021) Image segmentation to identify safe landing zones for unmanned aerial vehicles. arXiv: 2111.14557"},{"key":"965_CR22","doi-asserted-by":"publisher","unstructured":"Lee W, Alkouz B, Shahzaad B, Bouguettaya A (2021) Package delivery using autonomous drones in skyways, vol. 5, pp. 48\u201350. ACM. https:\/\/doi.org\/10.1145\/3460418.3479289. https:\/\/dl.acm.org\/doi\/10.1145\/3460418.3479289","DOI":"10.1145\/3460418.3479289"},{"key":"965_CR23","doi-asserted-by":"publisher","unstructured":"Liu W, Lis K, Salzmann M, Fua P (2019) Geometric and physical constraints for drone-based head plane crowd density estimation. In: 2019 IEEE\/RSJ international conference on intelligent robots and systems (IROS). https:\/\/doi.org\/10.1109\/iros40897.2019.8967852","DOI":"10.1109\/iros40897.2019.8967852"},{"key":"965_CR24","doi-asserted-by":"publisher","unstructured":"Liu W, Salzmann M, Fua P (2019) Context-aware crowd counting. In: 2019 IEEE\/CVF Conference on computer vision and pattern recognition (CVPR), pp. 5094\u20135103. IEEE Computer Society, Los Alamitos, CA, USA. https:\/\/doi.org\/10.1109\/CVPR.2019.00524. https:\/\/doi.ieeecomputersociety.org\/10.1109\/CVPR.2019.00524","DOI":"10.1109\/CVPR.2019.00524"},{"key":"965_CR25","doi-asserted-by":"crossref","unstructured":"Marcu A, Costea D, Licaret V, Pirvu M, Leordeanu M, Slusanschi E (2018) Safeuav: learning to estimate depth and safe landing areas for uavs from synthetic data. In: European conference on computer vision (ECCV) UAVision Workshop","DOI":"10.1007\/978-3-030-11012-3_4"},{"key":"965_CR26","doi-asserted-by":"crossref","unstructured":"Ma Z, Wei X, Hong X, Gong Y (2019) Bayesian loss for crowd count estimation with point supervision. In: Proceedings of the IEEE international conference on computer vision, pp. 6142\u20136151","DOI":"10.1109\/ICCV.2019.00624"},{"key":"965_CR27","doi-asserted-by":"publisher","first-page":"822","DOI":"10.3390\/rs14040822","volume":"14","author":"SJ McTegg","year":"2022","unstructured":"McTegg SJ, Kurdi FT, Simmons S, Gharineiat Z (2022) Comparative approach of unmanned aerial vehicle restrictions in controlled airspaces. Remote. Sens. 14:822","journal-title":"Remote. Sens."},{"key":"965_CR28","doi-asserted-by":"crossref","unstructured":"Mitroudas T, Balaska V, Psomoulis A, Gasteratos A (2023) Embedded light-weight approach for safe landing in populated areas. arXiv:2302.14445 [cs.RO]","DOI":"10.1109\/ICRA57147.2024.10611639"},{"key":"965_CR29","unstructured":"Mittal M, Mohan R, Burgard W, Valada A (2019) Vision-based autonomous UAV navigation and landing for urban search and rescue. CoRR:1906.01304"},{"key":"965_CR30","doi-asserted-by":"publisher","DOI":"10.1007\/s10846-022-01655-8","author":"Y Nabavi","year":"2022","unstructured":"Nabavi Y, Asadi D, Ahmadi K (2022) Automatic landing control of a multi-rotor uav using a monocular camera. J Intell Robot Syst. https:\/\/doi.org\/10.1007\/s10846-022-01655-8","journal-title":"J Intell Robot Syst"},{"key":"965_CR31","doi-asserted-by":"publisher","unstructured":"Nguyen TH, Cao M, Nguyen T.-M, Xie L (2018) Post-mission autonomous return and precision landing of uav, pp. 1747\u20131752. IEEE, https:\/\/doi.org\/10.1109\/ICARCV.2018.8581117, https:\/\/ieeexplore.ieee.org\/document\/8581117\/","DOI":"10.1109\/ICARCV.2018.8581117"},{"key":"965_CR32","doi-asserted-by":"publisher","unstructured":"Orsolits H, Rauh S.F, Estrada J.G (2022) Using mixed reality based digital twins for robotics education. In: 2022 IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct), pp. 56\u201359. https:\/\/doi.org\/10.1109\/ISMAR-Adjunct57072.2022.00021","DOI":"10.1109\/ISMAR-Adjunct57072.2022.00021"},{"key":"965_CR33","doi-asserted-by":"publisher","first-page":"393","DOI":"10.1007\/s11370-019-00290-x","volume":"12","author":"N Pinkam","year":"2019","unstructured":"Pinkam N, Newaz AAR, Jeong S, Chong NY (2019) Rapid coverage of regions of interest for environmental monitoring. Intel Serv Robot 12:393\u2013406. https:\/\/doi.org\/10.1007\/s11370-019-00290-x","journal-title":"Intel Serv Robot"},{"key":"965_CR34","doi-asserted-by":"publisher","unstructured":"Saho K (2017) Kalman filter for moving object tracking: Performance analysis and filter design. In: de Oliveira\u00a0Serra, G.L. (ed.) Kalman Filters-Theory for advanced applications. IntechOpen, https:\/\/doi.org\/10.5772\/intechopen.71731","DOI":"10.5772\/intechopen.71731"},{"key":"965_CR35","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1007\/s11370-020-00312-z","volume":"13","author":"J-P Sanchez-Rodriguez","year":"2020","unstructured":"Sanchez-Rodriguez J-P, Aceves-Lopez A, Martinez-Carranza J, Flores-Wysocka G (2020) Onboard plane-wise 3d mapping using super-pixels and stereo vision for autonomous flight of a hexacopter. Intel Serv Robot 13:273\u2013287. https:\/\/doi.org\/10.1007\/s11370-020-00312-z","journal-title":"Intel Serv Robot"},{"key":"965_CR36","doi-asserted-by":"publisher","unstructured":"Sankaranarayanan S, Balaji Y, Jain A, Lim S, Chellappa R (2018) Learning from synthetic data: Addressing domain shift for semantic segmentation. In: 2018 IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 3752\u20133761. IEEE Computer Society, Los Alamitos, CA, USA. https:\/\/doi.org\/10.1109\/CVPR.2018.00395. https:\/\/doi.ieeecomputersociety.org\/10.1109\/CVPR.2018.00395","DOI":"10.1109\/CVPR.2018.00395"},{"key":"965_CR37","doi-asserted-by":"publisher","unstructured":"Shah S, Dey D, Lovett C, Kapoor A (2018) Airsim: High-fidelity visual and physical simulation for autonomous vehicles. Springer Proceedings in Advanced Robotics 5:621\u2013635. https:\/\/doi.org\/10.1007\/978-3-319-67361-5_40","DOI":"10.1007\/978-3-319-67361-5_40"},{"key":"965_CR38","unstructured":"SilverTm: city park environment collection LITE (2021). https:\/\/www.unrealengine.com\/marketplace\/en-US\/product\/city-park-environment-collection-lite"},{"key":"965_CR39","doi-asserted-by":"publisher","DOI":"10.1109\/tetci.2019.2897815","author":"M Tzelepi","year":"2019","unstructured":"Tzelepi M, Tefas A (2019) Graph embedded convolutional neural networks in human crowd detection for drone flight safety. IEEE Trans Emerg Topics Comput Intell. https:\/\/doi.org\/10.1109\/tetci.2019.2897815","journal-title":"IEEE Trans Emerg Topics Comput Intell"},{"key":"965_CR40","doi-asserted-by":"crossref","unstructured":"Wang J, Wei C (2023) A novel air-ground coordinated approach for UAV autonomous landing on a mobile platform. In: Fu W, Gu M, Niu Y (eds.) Proceedings of 2022 international conference on autonomous unmanned systems (ICAUS 2022), pp. 2033\u20132043. Springer, Singapore","DOI":"10.1007\/978-981-99-0479-2_189"},{"key":"965_CR41","doi-asserted-by":"publisher","first-page":"610","DOI":"10.1016\/j.isatra.2022.04.005","volume":"130","author":"L Yang","year":"2022","unstructured":"Yang L, Wang C, Wang L (2022) Autonomous uavs landing site selection from point cloud in unknown environments. ISA Trans 130:610\u2013628. https:\/\/doi.org\/10.1016\/j.isatra.2022.04.005","journal-title":"ISA Trans"},{"key":"965_CR42","doi-asserted-by":"publisher","unstructured":"Zhang Y, Zhou D, Chen S, Gao S, Ma Y (2016) Single-image crowd counting via multi-column convolutional neural network. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https:\/\/doi.org\/10.1109\/cvpr.2016.70","DOI":"10.1109\/cvpr.2016.70"},{"key":"965_CR43","unstructured":"Zhu P, Wen L, Du D, Bian X, Hu Q, Ling H (2020) Vision meets drones: past, present and future. preprint arXiv:2001.06303"}],"container-title":["Virtual Reality"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10055-024-00965-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10055-024-00965-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10055-024-00965-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,13]],"date-time":"2024-11-13T20:49:00Z","timestamp":1731530940000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10055-024-00965-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3]]},"references-count":43,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["965"],"URL":"https:\/\/doi.org\/10.1007\/s10055-024-00965-6","relation":{},"ISSN":["1359-4338","1434-9957"],"issn-type":[{"value":"1359-4338","type":"print"},{"value":"1434-9957","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3]]},"assertion":[{"value":"13 June 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}}],"article-number":"66"}}