{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T21:27:58Z","timestamp":1783114078744,"version":"3.54.6"},"reference-count":47,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,3,25]],"date-time":"2025-03-25T00:00:00Z","timestamp":1742860800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>In the realm of real-time environmental monitoring and hazard detection, multi-robot systems present a promising solution for exploring and mapping dynamic fields, particularly in scenarios where human intervention poses safety risks. This research introduces a strategy for path planning and control of a group of mobile sensing robots to efficiently explore and reconstruct a dynamic field consisting of multiple non-overlapping diffusion sources. Our approach integrates a reinforcement learning-based path planning algorithm to guide the multi-robot formation in identifying diffusion sources, with a clustering-based method for destination selection once a new source is detected, to enhance coverage and accelerate exploration in unknown environments. Simulation results and real-world laboratory experiments demonstrate the effectiveness of our approach in exploring and reconstructing dynamic fields. This study advances the field of multi-robot systems in environmental monitoring and has practical implications for rescue missions and field explorations.<\/jats:p>","DOI":"10.3389\/frobt.2025.1492526","type":"journal-article","created":{"date-parts":[[2025,3,26]],"date-time":"2025-03-26T00:47:17Z","timestamp":1742950037000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Reinforcement learning-based dynamic field exploration and reconstruction using multi-robot systems for environmental monitoring"],"prefix":"10.3389","volume":"12","author":[{"given":"Thinh","family":"Lu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Divyam","family":"Sobti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Deepak","family":"Talwar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wencen","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,3,25]]},"reference":[{"key":"B1","volume-title":"Deep learning using rectified linear units (ReLU)","author":"Agarap","year":"2018"},{"key":"B2","doi-asserted-by":"publisher","first-page":"3773","DOI":"10.1109\/tase.2023.3285300","article-title":"Cure: a hierarchical framework for multi-robot autonomous exploration inspired by centroids of unknown regions","volume":"21","author":"Bi","year":"2024","journal-title":"IEEE Trans. Automation Sci. Eng."},{"key":"B3","doi-asserted-by":"publisher","first-page":"376","DOI":"10.1109\/tro.2004.839232","article-title":"Coordinated multi-robot exploration","volume":"21","author":"Burgard","year":"2005","journal-title":"Robotics, IEEE Trans."},{"key":"B4","doi-asserted-by":"publisher","first-page":"5567","DOI":"10.1109\/lra.2024.3396097","article-title":"Hma-sar: multi-agent search and rescue for unknown located dynamic targets in completely unknown environments","volume":"9","author":"Cao","year":"2024","journal-title":"IEEE Robotics Automation Lett."},{"key":"B5","doi-asserted-by":"publisher","first-page":"7520","DOI":"10.1109\/tnnls.2022.3214681","article-title":"Adaptive optimal tracking control of an underactuated surface vessel using actor\u2013critic reinforcement learning","volume":"35","author":"Chen","year":"2024","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"B6","first-page":"4450","volume-title":"Adaptive sensing for instantaneous gas release parameter estimation","author":"Christopoulos","year":"2005"},{"key":"B7","doi-asserted-by":"publisher","first-page":"853","DOI":"10.1109\/tcst.2013.2267623","article-title":"Coupled controls-computational fluids approach for the estimation of the concentration from a moving gaseous source in a 2-D domain with a Lyapunov-guided sensing aerial vehicle","volume":"22","author":"Demetriou","year":"2013","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"B8","unstructured":"Developer Nvidia\n          \n          \n          2024"},{"key":"B9","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1109\/mra.2011.2181683","article-title":"Robots for environmental monitoring: significant advancements and applications","volume":"19","author":"Dunbabin","year":"2012","journal-title":"IEEE Robot. Autom. Mag."},{"key":"B10","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1109\/SMC.2019.8914401","article-title":"A graph partitioning approach for fast exploration with multi-robot coordination","volume-title":"2019 IEEE international conference on systems, man and cybernetics (SMC)","author":"Gautam","year":"2019"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TNNLS.2023.3329530","article-title":"Graph soft actor\u2013critic reinforcement learning for large-scale distributed multirobot coordination","author":"Hu","year":"2023","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"B12","unstructured":"Adafruit 9-dof absolute orientation imu fusion breakout - bno055\n          \n          \n            \n              Industries\n              A.\n            \n          \n          \n          2024"},{"key":"B13","doi-asserted-by":"publisher","first-page":"765","DOI":"10.1080\/104077890504456","article-title":"On the rate of spread for some reaction-diffusion models of forest fire propagation","volume":"46","author":"Khaled","year":"2004","journal-title":"Numer. Heat. Transf. Part A Appl."},{"key":"B14","first-page":"1060","article-title":"Early forest fire detection using drones and artificial intelligence","volume-title":"2019 42nd international convention on information and communication technology, electronics and microelectronics","author":"Kinaneva","year":"2019"},{"key":"B15","article-title":"Adam: a method for stochastic optimization","volume-title":"International conference on learning representations (ICLR)","author":"Kingma","year":"2015"},{"key":"B16","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1109\/IRCE62232.2024.10739833","article-title":"Multi-robots path planning and mapping for exploring unknown environment","volume-title":"2024 7th international conference on intelligent robotics and control engineering (IRCE)","author":"Long","year":"2024"},{"key":"B17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/AUV53081.2022.9965934","article-title":"Robotic swarm for marine and submarine missions: challenges and perspectives","volume-title":"2022 IEEE\/OES autonomous underwater vehicles symposium (AUV)","author":"Luvisutto","year":"2022"},{"key":"B18","first-page":"1494","article-title":"Ux 1 system design - a robotic system for underwater mining exploration","volume-title":"IEEE\/RSJ international conference on intelligent robots and systems IROS","author":"Martins","year":"2018"},{"key":"B19","doi-asserted-by":"publisher","first-page":"666","DOI":"10.1109\/tro.2006.878957","article-title":"Performance analysis of multirobot cooperative localization","volume":"22","author":"Mourikis","year":"2006","journal-title":"IEEE Trans. Robotics"},{"key":"B20","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1109\/IITSI.2010.74","article-title":"Research on k-means clustering algorithm: an improved k-means clustering algorithm","volume-title":"2010 third international symposium on intelligent information technology and security informatics","author":"Na","year":"2010"},{"key":"B21","doi-asserted-by":"publisher","first-page":"610","DOI":"10.1109\/lra.2019.2891991","article-title":"Deep reinforcement learning robot for search and rescue applications: exploration in unknown cluttered environments","volume":"4","author":"Niroui","year":"2019","journal-title":"IEEE Robotics Automation Lett."},{"key":"B22","unstructured":"Teensy\u00ae 4.0 development board\n          \n          \n          2024"},{"key":"B23","doi-asserted-by":"publisher","first-page":"191617","DOI":"10.1109\/access.2020.3030190","article-title":"Collaborative multi-robot search and rescue: planning, coordination, perception, and active vision","volume":"8","author":"Queralta","year":"2020","journal-title":"IEEE Access"},{"key":"B24","article-title":"ROS: an open-source robot operating system","volume-title":"ICRA workshop on open source software","author":"Quigley","year":"2009"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.5555\/3546258.3546526","article-title":"Stable-baselines3: reliable reinforcement learning implementations","volume":"22","author":"Raffin","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"B26","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1016\/j.camwa.2024.01.024","article-title":"Analytical and numerical insights into wildfire dynamics: exploring the advection\u2013diffusion\u2013reaction model","volume":"158","author":"Reisch","year":"2024","journal-title":"Comput. Math. Appl."},{"key":"B27","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-84800-015-5","volume-title":"Distributed consensus in multi-vehicle cooperative control","author":"Ren","year":"2008"},{"key":"B28","first-page":"3736","article-title":"Kalman and extended kalman filters: concept, derivation and properties","volume":"43","author":"Ribeiro","year":"2004","journal-title":"Inst. Syst. Robotics"},{"key":"B29","doi-asserted-by":"publisher","first-page":"765","DOI":"10.1109\/tim.2015.2506319","article-title":"Autonomous gas detection and mapping with unmanned aerial vehicles","volume":"65","author":"Rossi","year":"2016","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"B30","first-page":"1889","article-title":"Trust region policy optimization","volume-title":"Proceedings of the 32nd international Conference on international Conference on machine learning - volume 37, Lille, France ICML\u201915","author":"Schulman","year":"2015"},{"key":"B31","doi-asserted-by":"publisher","first-page":"06347","DOI":"10.48550\/arXiv.1707.06347","article-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017","journal-title":"CoRR"},{"key":"B32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IECON51785.2023.10312013","article-title":"A novel collaborative knowledge sharing and self-learning framework for robotic systems in search and rescue operations","volume-title":"IECON 2023- 49th annual conference of the IEEE industrial electronics society","author":"Shuvo","year":"2023"},{"key":"B33","doi-asserted-by":"crossref","DOI":"10.31979\/etd.e36p-dxjt","volume-title":"Deep reinforcement learning based path-planning for multi-agent systems in advection-diffusion field reconstruction tasks","author":"Talwar","year":"2020"},{"key":"B34","doi-asserted-by":"crossref","first-page":"3211","DOI":"10.1109\/ACC.2010.5530762","article-title":"Optimal trajectories of mobile remote sensors for parameter estimation in distributed cyber-physical systems","volume-title":"Proceedings of the 2010 American control conference","author":"Tricaud","year":"2010"},{"key":"B35","article-title":"Optimal measurement methods for distributed parameter system identification","volume-title":"Taylor and Francis series in systems and control","author":"Ucinski","year":"2005"},{"key":"B36","doi-asserted-by":"crossref","first-page":"5257","DOI":"10.1109\/CDC.2005.1582997","article-title":"Time-optimal path planning of moving sensors for parameter estimation of distributed systems","volume-title":"Proceedings of the 44th IEEE conference on decision and control","author":"Ucinski","year":"2005"},{"key":"B37","doi-asserted-by":"publisher","first-page":"2740","DOI":"10.1109\/lra.2023.3251193","article-title":"Curriculum reinforcement learning from avoiding collisions to navigating among movable obstacles in diverse environments","volume":"8","author":"Wang","year":"2023","journal-title":"IEEE Robotics Automation Lett."},{"key":"B38","first-page":"4555","article-title":"A survey on curriculum learning","volume-title":"IEEE Trans. Pattern Analysis Mach. Intell.","author":"Wang","year":"2022"},{"key":"B39","doi-asserted-by":"publisher","first-page":"842","DOI":"10.1017\/s0263574720000788","article-title":"Parameter identification of spatial\u2013temporal varying processes by a multi-robot system in realistic diffusion fields","volume":"39","author":"Wu","year":"2020","journal-title":"Robotica"},{"key":"B40","first-page":"828","article-title":"Robust cooperative exploration with a switching strategy","volume-title":"IEEE Trans. Robotics","author":"Wu","year":"2012"},{"key":"B41","unstructured":"YDLIDAR-G4-Datasheet\n          \n          \n          2024"},{"key":"B42","doi-asserted-by":"crossref","first-page":"1949","DOI":"10.1109\/CDC.2018.8619486","article-title":"Geometric reinforcement learning based path planning for mobile sensor networks in advection-diffusion field reconstruction","volume-title":"2018 IEEE conference on decision and control (CDC)","author":"You","year":"2018"},{"key":"B43","doi-asserted-by":"crossref","first-page":"4327","DOI":"10.1109\/CDC.2016.7798925","article-title":"Cooperative filtering for parameter identification of diffusion processes","volume-title":"2016 IEEE 55th conference on decision and control","author":"You","year":"2016"},{"key":"B44","doi-asserted-by":"publisher","first-page":"527","DOI":"10.1109\/tcst.2022.3183585","article-title":"Cooperative filtering and parameter identification for advection\u2013diffusion processes using a mobile sensor network","volume":"31","author":"You","year":"2022","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"B45","first-page":"650","article-title":"Cooperative filters and control for cooperative exploration","volume-title":"IEEE Trans. Automatic Control","author":"Zhang","year":"2010"},{"key":"B46","doi-asserted-by":"publisher","first-page":"1175418","DOI":"10.3389\/frobt.2023.1175418","article-title":"Distributed cooperative kalman filter constrained by advection\u2013diffusion equation for mobile sensor networks","volume":"10","author":"Zhang","year":"2023","journal-title":"Front. Robotics AI"},{"key":"B47","doi-asserted-by":"crossref","first-page":"7639","DOI":"10.1109\/CAC59555.2023.10452014","article-title":"Multi-robot autonomous exploration in unknown environment: a review","volume-title":"2023 China automation congress (CAC)","author":"Zhu","year":"2023"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1492526\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,26]],"date-time":"2025-03-26T00:50:28Z","timestamp":1742950228000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1492526\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,25]]},"references-count":47,"alternative-id":["10.3389\/frobt.2025.1492526"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2025.1492526","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,25]]},"article-number":"1492526"}}