{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:35:21Z","timestamp":1760243721829,"version":"build-2065373602"},"reference-count":40,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2012,10,9]],"date-time":"2012-10-09T00:00:00Z","timestamp":1349740800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Given multiple widespread stationary data sources such as ground-based sensors, an unmanned aircraft can fly over the sensors and gather the data via a wireless link. Performance criteria for such a network may incorporate costs such as trajectory length for the aircraft or the energy required by the sensors for radio transmission. Planning is hampered by the complex vehicle and communication dynamics and by uncertainty in the locations of sensors, so we develop a technique based on model-free learning. We present a stochastic optimisation method that allows the data-ferrying aircraft to optimise data collection trajectories through an unknown environment in situ, obviating the need for system identification. We compare two trajectory representations, one that learns near-optimal trajectories at low data requirements but that fails at high requirements, and one that gives up some performance in exchange for a data collection guarantee. With either encoding the ferry is able to learn significantly improved trajectories compared with alternative heuristics. To demonstrate the versatility of the model-free learning approach, we also learn a policy to minimise the radio transmission energy required by the sensor nodes, allowing prolonged network lifetime.<\/jats:p>","DOI":"10.3390\/rs4102971","type":"journal-article","created":{"date-parts":[[2012,10,10]],"date-time":"2012-10-10T02:55:18Z","timestamp":1349837718000},"page":"2971-3005","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Model-Free Trajectory Optimisation for Unmanned Aircraft Serving as Data Ferries for Widespread Sensors"],"prefix":"10.3390","volume":"4","author":[{"given":"Ben","family":"Pearre","sequence":"first","affiliation":[{"name":"Computer Science, University of Colorado at Boulder, Boulder, CO 80309, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Timothy X.","family":"Brown","sequence":"additional","affiliation":[{"name":"Electrical, Computer, and Energy Engineering, University of Colorado at Boulder, Boulder, CO 80309, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2012,10,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"990","DOI":"10.3390\/rs2040990","article-title":"Monitoring vegetation phenological cycles in two different semi-arid environmental settings using a ground-based NDVI system: A potential approach to improve satellite data interpretation","volume":"2","author":"Baghzouz","year":"2010","journal-title":"Remote Sens"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1648\/0273-8570-71.1.167","article-title":"Methodological considerations for monitoring wild bird nests using video technology","volume":"71","author":"McQuillen","year":"2000","journal-title":"J. Field Ornithol"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"378","DOI":"10.3390\/rs3020378","article-title":"Alaskan permafrost groundwater storage changes derived from GRACE and ground measurements","volume":"3","author":"Muskett","year":"2011","journal-title":"Remote Sens"},{"key":"ref_4","unstructured":"Jenkins, A., Henkel, D., and Brown, T. (March, January 11\u2013). Sensor Data Collection through Gateways in a Highly Mobile Mesh Network. Hong Kong, China."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Henkel, D., and Brown, T.X. (2008, January 23\u201326). Towards Autonomous Data Ferry Route Design through Reinforcement Learning. Newport Beach, CA, USA.","DOI":"10.1109\/WOWMOM.2008.4594888"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3449","DOI":"10.1016\/j.comnet.2006.01.008","article-title":"Data harvesting with mobile elements in wireless sensor networks","volume":"50","author":"Gu","year":"2006","journal-title":"Comput. Netw"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1109\/TMC.2007.57","article-title":"Mobile element scheduling with dynamic deadlines","volume":"6","author":"Somasundara","year":"2007","journal-title":"IEEE Trans. Mobile Comput"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"He, T., won Lee, K., and Swami, A. (2010, January 20\u201324). Flying in the Dark: Controlling Autonomous Data Ferries with Partial Observations. Chicago, IL, USA.","DOI":"10.1145\/1860093.1860113"},{"key":"ref_9","unstructured":"Zhao, W., and Ammar, M.H. (May, January 28\u2013). Message Ferrying: Proactive Routing in Highly-Partitioned Wireless Ad Hoc Networks. San Juan, Puerto Rico."},{"key":"ref_10","unstructured":"Dunbabin, M., Corke, P., Vasilescu, I., and Rus, D. (May, January 15\u2013). Data Muling over Underwater Wireless Sensor Networks Using an Autonomous Underwater Vehicle. Orlando, FL, USA."},{"key":"ref_11","unstructured":"Bin Tariq, M.M., Ammar, M., and Zegura, E. (May, January 22\u2013). Message Ferry Route Design for Sparse Ad Hoc Networks with Mobile Nodes. Florence, Italy."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1476","DOI":"10.1109\/TPDS.2007.1070","article-title":"SenCar: An energy-efficient data gathering mechanism for large-scale multihop sensor networks","volume":"18","author":"Ma","year":"2007","journal-title":"IEEE Trans. Parallel Distrib. Syst"},{"key":"ref_13","first-page":"22","article-title":"Using mobile robots to harvest data from sensor fields","volume":"16","author":"Tekdas","year":"2008","journal-title":"IEEE Wirel. Commun. Special Issue Wirel. Commun. Netw. Robot"},{"key":"ref_14","unstructured":"Sugihara, R., and Gupta, R.K. (June, January 11\u2013). Improving the Data Delivery Latency in Sensor Networks with Controlled Mobility. Santorini Island, Greece."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Sugihara, R., and Gupta, R.K. (2011). Path Planning of Data Mules in Sensor Networks, ACM.","DOI":"10.1145\/1993042.1993043"},{"key":"ref_16","unstructured":"Henkel, D., and Brown, T.X. (2008, January 2\u20134). On Controlled Node Mobility in Delay-Tolerant Networks of Unmanned Aerial Vehicles. Boulder, CO, USA."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Carfang, A., Frew, E.W., and Brown, T.X. (2010, January 2\u20135). Improved Delay-Tolerant Communication by Considering Radio Propagation in Planning Data Ferry Navigation. Toronto, ON, Canada.","DOI":"10.2514\/6.2010-8172"},{"key":"ref_18","unstructured":"Stachura, M., Carfang, A., and Frew, E.W. (2009, January 8\u201310). Cooperative Target Tracking with a Communication Limited Active Sensor Network. Marina Del Rey, CA, USA."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"993","DOI":"10.1109\/JSAC.2012.120614","article-title":"Optimization of UAV heading for the ground-to-air uplink","volume":"30","author":"Jiang","year":"2012","journal-title":"IEEE J. Sel. Areas Commun"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1016\/j.adhoc.2006.02.001","article-title":"Trading latency for energy in densely deployed wireless ad hoc networks using message ferrying","volume":"5","author":"Jun","year":"2007","journal-title":"Ad Hoc Netw"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1016\/j.peva.2009.08.005","article-title":"Reliable and energy-efficient data collection in sparse sensor networks with mobile elements","volume":"66","author":"Anastasi","year":"2009","journal-title":"Perform. Eval"},{"key":"ref_22","unstructured":"Sugihara, R., and Gupta, R.K. (April, January 19\u2013). Optimizing Energy-Latency Trade-off in Sensor Networks with Controlled Mobility. Rio de Janeiro, Brazil."},{"key":"ref_23","unstructured":"Ciullo, D., Celik, G., and Modiano, E. (4, January 31). Minimizing Transmission Energy in Sensor Networks via Trajectory Control. Avignon, France."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1002\/wcm.584","article-title":"Should I send now or send later? A decision-theoretic approach to transmission scheduling in sensor networks with mobile sinks","volume":"8","author":"Turgut","year":"2008","journal-title":"Wirel. Commun. Mobile Comput"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Taylor, S.G., Farinholt, K.M., Flynn, E.B., Figueiredo, E., Mascarenas, D.L., Moro, E.A., Park, G., Todd, M.D., and Farrar, C.R. (2009). A mobile-agent\u2013based wireless sensing network for structural monitoring applications. Meas. Sci. Technol, 20.","DOI":"10.1088\/0957-0233\/20\/4\/045201"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"537","DOI":"10.1016\/j.adhoc.2008.06.003","article-title":"Energy conservation in wireless sensor networks: A survey","volume":"7","author":"Anastasi","year":"2009","journal-title":"Ad Hoc Netw"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1671","DOI":"10.3390\/rs4061671","article-title":"Unmanned aircraft systems in remote sensing and scientific research: Classification and considerations of use","volume":"4","author":"Watts","year":"2012","journal-title":"Remote Sens"},{"key":"ref_28","unstructured":"Pearre, B., and Brown, T.X. (4, January 31). Fast, Scalable, Model-free Trajectory Optimization for Wireless Data Ferries. Maui, HI, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wagle, N., and Frew, E.W. (2010, January 2\u20135). A Particle Filter Approach to WiFi Target Localization. Toronto, ON, Canada.","DOI":"10.2514\/6.2010-7750"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sadegh, P., and Spall, J. (1997, January 4\u20136). Optimal Random Perturbations for Stochastic Approximation Using a Simultaneous Perturbation Gradient Approximation. Albuquerque, NM, USA.","DOI":"10.1109\/ACC.1997.609490"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1002\/eej.20239","article-title":"Parameter estimation using simultaneous perturbation stochastic approximation","volume":"154","author":"Hirokami","year":"2006","journal-title":"Electr. Eng. Jpn"},{"key":"ref_32","unstructured":"Kohl, N., and Stone, P. (1, January 26). Policy Gradient Reinforcement Learning for Fast Quadrupedal Locomotion. New Orleans, LA, USA."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"682","DOI":"10.1016\/j.neunet.2008.02.003","article-title":"Reinforcement learning of motor skills with policy gradients","volume":"21","author":"Peters","year":"2008","journal-title":"Neural Netw"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Sigaud, O., and Peters, J. (2009). From Motor to Interaction Learning in Robots, Springer.","DOI":"10.1007\/978-3-642-05181-4"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Pearre, B., and Brown, T. (2010, January 6\u201310). Model-free Trajectory Optimization for Wireless Data Ferries among Multiple Sources. Miami, FL, USA.","DOI":"10.1109\/LCN.2010.5735811"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Glynn, P. (1987, January 14\u201316). Likelihood Ratio Gradient Estimation: An Overview. Atlanta, GA, USA.","DOI":"10.21236\/ADA197085"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1007\/BF00992696","article-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","volume":"8","author":"Williams","year":"1992","journal-title":"Mach. Learn"},{"key":"ref_38","first-page":"1057","article-title":"Policy gradient methods for reinforcement learning with function approximation","volume":"12","author":"Sutton","year":"2000","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1613\/jair.806","article-title":"Infinite-horizon policy-gradient estimation","volume":"15","author":"Baxter","year":"2001","journal-title":"J. Artif. Intell. Res"},{"key":"ref_40","unstructured":"Tang, J., and Abbeel, P. (2010, January 6\u201311). On a Connection between Importance Sampling and the Likelihood Ratio Policy Gradient. Vancouver, BC, Canada."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/4\/10\/2971\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:52:42Z","timestamp":1760219562000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/4\/10\/2971"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,10,9]]},"references-count":40,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2012,10]]}},"alternative-id":["rs4102971"],"URL":"https:\/\/doi.org\/10.3390\/rs4102971","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2012,10,9]]}}}