{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T06:04:40Z","timestamp":1771308280798,"version":"3.50.1"},"reference-count":18,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T00:00:00Z","timestamp":1770940800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Autonomous drone navigation in confined tubular environments remains a major challenge due to the constraining geometry of the conduits, the proximity of the walls, and the perceptual limitations inherent to such scenarios. We propose a reinforcement learning (RL) approach enabling a drone to navigate unknown three-dimensional tubes without any prior knowledge of their geometry, relying solely on local observations from a Light Detection and Ranging (LiDAR) sensor and a conditional visual detection of the tube center. In contrast, the Pure Pursuit algorithm, used as a deterministic baseline, benefits from explicit access to the centerline, creating an information asymmetry designed to assess the ability of RL to compensate for the absence of a geometric model. The agent is trained through a progressive curriculum learning strategy that gradually exposes it to increasingly curved geometries, where the tube center frequently disappears from the visual field. A turning-negotiation mechanism, based on the combination of direct visibility, directional memory, and LiDAR symmetry cues, proves essential for ensuring stable navigation under such partial observability conditions. Experiments show that the Proximal Policy Optimization (PPO) policy acquires robust and generalizable behavior, consistently outperforming the deterministic controller despite its limited access to geometric information. Validation in a high-fidelity three-dimensional environment further confirms the transferability of the learned behavior to continuous physical dynamics. In particular, this work introduces an explicit formulation of the turn negotiation problem in tubular navigation, coupled with a reward design and evaluation metrics that make turn-handling behavior measurable and analyzable. This explicit focus on turn negotiation distinguishes our approach from prior learning-based and heuristic methods. The proposed approach thus provides a complete framework for autonomous navigation in unknown tubular environments and opens perspectives for industrial, underground, or medical applications where progressing through narrow and weakly perceptive conduits represents a central challenge.<\/jats:p>","DOI":"10.3390\/s26041236","type":"journal-article","created":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T16:09:32Z","timestamp":1770998972000},"page":"1236","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Curriculum-Based Reinforcement Learning for Autonomous UAV Navigation in Unknown Curved Tubular Conduits"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-3277-7010","authenticated-orcid":false,"given":"Zamirddine","family":"Mari","sequence":"first","affiliation":[{"name":"DGA Techniques Navales\u2014Direction G\u00e9n\u00e9rale de l\u2019Armement, Toulon, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7993-4724","authenticated-orcid":false,"given":"J\u00e9r\u00f4me","family":"Pasquet","sequence":"additional","affiliation":[{"name":"LIRMM, TETIS, CNRS, Universit\u00e9 de Montpellier Paul-Val\u00e9ry, Montpellier, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8996-6287","authenticated-orcid":false,"given":"Julien","family":"Seinturier","sequence":"additional","affiliation":[{"name":"Universit\u00e9 de Toulon, Aix Marseille Univ, CNRS, LIS, Marseille, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2230","DOI":"10.1109\/TRO.2025.3548525","article-title":"Autonomous Flights inside Narrow Tunnels","volume":"41","author":"Wang","year":"2025","journal-title":"IEEE Trans. Robot."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"5489","DOI":"10.1109\/LRA.2022.3154024","article-title":"Neither Fast Nor Slow: How to Fly Through Narrow Tunnels","volume":"7","author":"Wang","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Mansouri, S.S., Kanellakis, C., Karvelis, P., Kominiak, D., and Nikolakopoulos, G. (2020, January 12\u201315). MAV Navigation in Unknown Dark Underground Mines Using Deep Learning. Proceedings of the 2020 European Control Conference (ECC), St. Petersburg, Russia.","DOI":"10.23919\/ECC51009.2020.9143842"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"835","DOI":"10.1017\/S0263574721000849","article-title":"Autonomous Collision-Free Navigation of a Quadrotor UAV in Unknown Tunnel-like Environments","volume":"40","author":"Elmokadem","year":"2022","journal-title":"Robotica"},{"key":"ref_5","unstructured":"Tardioli, D., Cano, R., and Mosteo, A.R. (2024). UAV Navigation in Tunnels with 2D Tilted LiDARs. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Mansouri, S.S., Karvelis, P., Kanellakis, C., Kominiak, D., and Nikolakopoulos, G. (2019, January 14\u201317). Vision-based MAV Navigation in Underground Mine Using Convolutional Neural Network. Proceedings of the IECON 2019\u201445th Annual Conference of the IEEE Industrial Electronics Society, Lisbon, Portugal.","DOI":"10.1109\/IECON.2019.8927168"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Tagliabue, A., Tordesillas, J., Cai, X., Santamaria-Navarro, A., How, J.P., Carlone, L., and Agha-mohammadi, A. (2021). LION: Lidar-Inertial Observability-Aware Navigator for Vision-Denied Environments. arXiv.","DOI":"10.1007\/978-3-030-71151-1_34"},{"key":"ref_8","unstructured":"Sie, E., Liu, Z., and Vasisht, D. (2023, January 2\u20136). BatMobility: Towards Flying Without Seeing for Autonomous Drones. Proceedings of the 27th Annual International Conference on Mobile Computing and Networking (MobiCom\u201923), Madrid, Spain."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Lam, M., Dodds, L., Eid, A., Hester, J., and Adib, F. (2025, January 19\u201322). 6D Self-Localization of Drones Using a Single Millimeter-Wave Backscatter Anchor. Proceedings of the IEEE INFOCOM 2025\u2014IEEE Conference on Computer Communications, London, UK.","DOI":"10.1109\/INFOCOM55648.2025.11044655"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Tan, M., Tao, Y., Zheng, B., Xie, G., Feng, L., Xia, Z., and Xiong, J. (2024). Safe Navigation for Robotic Digestive Endoscopy via Human Intervention-based Reinforcement Learning. arXiv.","DOI":"10.1016\/j.eswa.2025.128841"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Corsi, D., Marzari, L., Pore, A., Farinelli, A., Casals, A., Fiorini, P., and Dall\u2019Alba, D. (2023). Constrained Reinforcement Learning and Formal Verification for Safe Colonoscopy Navigation. arXiv.","DOI":"10.1109\/IROS55552.2023.10341789"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Pore, A., Finocchiaro, M., Dall\u2019Alba, D., Hernansanz, A., Ciuti, G., Arezzo, A., Menciassi, A., Casals, A., and Fiorini, P. (2022). Colonoscopy Navigation using End-to-End Deep Visuomotor Control: A User Study. arXiv.","DOI":"10.1109\/IROS47612.2022.9981480"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1007\/s10514-021-10011-y","article-title":"AlphaPilot: Autonomous drone racing","volume":"46","author":"Foehn","year":"2021","journal-title":"Auton. Robot."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"982","DOI":"10.1038\/s41586-023-06419-4","article-title":"Champion-level drone racing using deep reinforcement learning","volume":"620","author":"Kaufmann","year":"2023","journal-title":"Nature"},{"key":"ref_15","unstructured":"Coulter, R. (1990). Implementation of the Pure Pursuit Path Tracking Algorithm, Carnegie Mellon University."},{"key":"ref_16","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ren, Y., Cai, Y., Li, H., Chen, N., Zhu, F., Yin, L., Kong, F., Li, R., and Zhang, F. (2025). A Survey on LiDAR-based Autonomous Aerial Vehicles. arXiv.","DOI":"10.1109\/TMECH.2025.3599633"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"OpenAI, Andrychowicz, M., Baker, B., Chociej, M., Jozefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., and Powell, G. (2018). Learning Dexterous In-Hand Manipulation. arXiv.","DOI":"10.1177\/0278364919887447"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/4\/1236\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T05:12:26Z","timestamp":1771305146000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/4\/1236"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,13]]},"references-count":18,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["s26041236"],"URL":"https:\/\/doi.org\/10.3390\/s26041236","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,13]]}}}