{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:22:58Z","timestamp":1760242978373,"version":"build-2065373602"},"reference-count":31,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2015,3,27]],"date-time":"2015-03-27T00:00:00Z","timestamp":1427414400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Maintaining contact between the robot and plume is significant in chemical plume tracing (CPT). In the time immediately following the loss of chemical detection during the process of CPT, Track-Out activities bias the robot heading relative to the upwind direction, expecting to rapidly re-contact the plume. To determine the bias angle used in the Track-Out activity, we propose an online instance-based reinforcement learning method, namely virtual trail following (VTF). In VTF, action-value is generalized from recently stored instances of successful Track-Out activities. We also propose a collaborative VTF (cVTF) method, in which multiple robots store their own instances, and learn from the stored instances, in the same database. The proposed VTF and cVTF methods are compared with biased upwind surge (BUS) method, in which all Track-Out activities utilize an offline optimized universal bias angle, in an indoor environment with three different airflow fields. With respect to our experimental conditions, VTF and cVTF show stronger adaptability to different airflow environments than BUS, and furthermore, cVTF yields higher success rates and time-efficiencies than VTF.<\/jats:p>","DOI":"10.3390\/s150407512","type":"journal-article","created":{"date-parts":[[2015,3,27]],"date-time":"2015-03-27T17:12:08Z","timestamp":1427476328000},"page":"7512-7536","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Learning to Rapidly Re-Contact the Lost Plume in Chemical Plume Tracing"],"prefix":"10.3390","volume":"15","author":[{"given":"Meng-Li","family":"Cao","sequence":"first","affiliation":[{"name":"Institute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, China"}]},{"given":"Qing-Hao","family":"Meng","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, China"}]},{"given":"Jia-Ying","family":"Wang","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, China"}]},{"given":"Bing","family":"Luo","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, China"}]},{"given":"Ya-Qi","family":"Jing","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, China"}]},{"given":"Shu-Gen","family":"Ma","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Autonomous Systems, Tianjin Key Laboratory of Process Measurement and Control, School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072, China"},{"name":"Department of Robotics, Ritsumeikan University, 1-1-1 Nojihigashi, Kusatsu-Shi 525-8577, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2015,3,27]]},"reference":[{"key":"ref_1","unstructured":"Quinn, T.P. (2005). The Behavior and Ecology of Pacific Salmon and Trout, University of Washington Press."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1093\/beheco\/arq029","article-title":"Ecological determinants and sensory mechanisms in habitat selection of crustacean postlarvae","volume":"21","author":"Lecchini","year":"2010","journal-title":"Behav. Ecol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3387","DOI":"10.1242\/jeb.205.21.3387","article-title":"Behavioral observations and computer simulations of blue crab movement to a chemical source in a controlled turbulent flow","volume":"205","author":"Weissburg","year":"2002","journal-title":"J. Exp. Biol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"16591","DOI":"10.3390\/s131216591","article-title":"Simultaneous sampling of flow and odorants by crustaceans can aid searches within a turbulent plume","volume":"13","author":"Pravin","year":"2013","journal-title":"Sensors"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"854","DOI":"10.1007\/s10886-008-9484-5","article-title":"Navigational strategies used by insects to find distant, wind-borne sources of odor","volume":"34","author":"Willis","year":"2008","journal-title":"J. Chem. Ecol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1038\/nature05464","article-title":"\u201cInfotaxis\u201d as a strategy for searching without gradients","volume":"445","author":"Vergassola","year":"2007","journal-title":"Nature"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"11444","DOI":"10.3390\/s140711444","article-title":"Distributed least-squares estimation of a remote chemical source via convex combination in wireless sensor networks","volume":"14","author":"Cao","year":"2014","journal-title":"Sensors"},{"key":"ref_8","unstructured":"Ishida, H. (2007, January 10\u201314). Robotic systems for gas\/odor source localization: Gap between experiments and real-life situations. Proceedings of the IEEE International Conference on Robotics and Automation, Roma, Italy."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1177\/10597123010093003","article-title":"Tracking of fluid-advected odor plumes: Strategies inspired by insect orientation to pheromone","volume":"9","author":"Li","year":"2001","journal-title":"Adapt. Behav."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1109\/TRO.2006.870627","article-title":"Moth-inspired chemical plume tracing on an autonomous underwater vehicle","volume":"22","author":"Li","year":"2006","journal-title":"IEEE Trans. Robot."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/S0040-6090(02)00593-X","article-title":"Olfaction-based mobile robot navigation","volume":"418","author":"Marques","year":"2002","journal-title":"Thin Solid Films"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dillmann, R., W\u00f6rn, H., and Gockel, T. (2003). Autonome Mobile Systeme, Springer.","DOI":"10.1007\/978-3-642-18986-9"},{"key":"ref_13","unstructured":"Nakamoto, T. (2008). Olfactory Display: Multimedia Tool for Presenting Scents, Fragrance Journal Ltd."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3163","DOI":"10.1109\/JSEN.2012.2208740","article-title":"Chemical sensing in robotic applications: A review","volume":"12","author":"Ishida","year":"2012","journal-title":"IEEE Sens. J."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1616","DOI":"10.3390\/s6111616","article-title":"Airborne chemical sensing review","volume":"6","author":"Lilienthal","year":"2006","journal-title":"Sensors"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1109\/MRA.2012.2184671","article-title":"Autonomous gas-sensitive microdrone: Wind vector estimation and gas distribution mapping","volume":"19","author":"Neumann","year":"2012","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1080\/01691864.2013.779052","article-title":"Gas source localization with a micro-drone using bio-inspired and particle filter-based algorithms and particle filter-based algorithms","volume":"27","author":"Neumann","year":"2013","journal-title":"Adv. Robot."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1109\/MRA.2011.2181683","article-title":"Robots for environmental monitoring: Significant advancements and applications","volume":"19","author":"Dunbabin","year":"2012","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, Cambridge University Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1007\/s10514-006-7099-7","article-title":"Olfactory coordinated area coverage","volume":"20","author":"Larionova","year":"2006","journal-title":"Auton. Robot."},{"key":"ref_21","unstructured":"Gu, D.-B., and Yang, E.-F. (2004). Multiagent Reinforcement Learning for Multi-Robot Systems: A Survey, University of Essex. Teachnical Report CSM-404."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1016\/j.robot.2004.03.006","article-title":"Self-valuing learning and generalization with application in visually guided grasping of complex objects","volume":"47","author":"Zhang","year":"2004","journal-title":"Robot. Auton. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1023\/A:1006559212014","article-title":"Locally weighted learning","volume":"11","author":"Atkeson","year":"1997","journal-title":"Artif. Intell. Rev."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1017\/S0263574704000803","article-title":"Teaching robots to plan through q-learning","volume":"23","author":"Gu","year":"2005","journal-title":"Robotica"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"10415","DOI":"10.3390\/s111110415","article-title":"Collective odor source estimation and search in time-varient airflow environment","volume":"11","author":"Meng","year":"2011","journal-title":"Sensors"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4737","DOI":"10.3390\/s120404737","article-title":"Adapting ant colony metaphor for multi-robot chemical plume tracing","volume":"12","author":"Meng","year":"2012","journal-title":"Sensors"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1007\/s10514-011-9219-2","article-title":"Odor source localization using a mobile robot in outdoor airflow environments with a particle filter algorithm","volume":"30","author":"Li","year":"2011","journal-title":"Auton. Robot."},{"key":"ref_29","unstructured":"Lilienthal, A.J. (2004). Gas Distribution Mapping and Gas Source Localization with a Mobile Robot. [Ph.D. Thesis, University of Tubingen]."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.robot.2008.02.005","article-title":"Velocity planning for a mobile robot to track a moving target\u2014A potential field approach","volume":"57","author":"Huang","year":"2009","journal-title":"Robot. Auton. Syst."},{"key":"ref_31","unstructured":"Lochmatter, T. (2010). Bio-Inspired and Probabilistic Algorithms for Distributed Odor Source Localization Using Mobile Robots. [Ph.D. Thesis, Swiss federal Institute of Technology in Lausanne]."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/15\/4\/7512\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:44:04Z","timestamp":1760215444000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/15\/4\/7512"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,3,27]]},"references-count":31,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2015,4]]}},"alternative-id":["s150407512"],"URL":"https:\/\/doi.org\/10.3390\/s150407512","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2015,3,27]]}}}