{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T20:26:46Z","timestamp":1779136006157,"version":"3.51.4"},"reference-count":33,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2020,7,4]],"date-time":"2020-07-04T00:00:00Z","timestamp":1593820800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61873224, 41976182, 61873223 and 61773333"],"award-info":[{"award-number":["61873224, 41976182, 61873223 and 61773333"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The autonomous underwater glider has attracted enormous interest for underwater activities, especially in long-term and large-scale underwater data collection. In this paper, we focus on the application of gliders gathering data from underwater sensor networks over underwater acoustic channels. However, this application suffers from a rapidly time-varying environment and limited energy. To optimize the performance of data collection and maximize the network lifetime, we propose a distributed, energy-efficient sensor scheduling algorithm based on the multi-armed bandit formulation. Besides, we design an indexable threshold policy to tradeoff between the data quality and the collection delay. Moreover, to reduce the computational complexity, we divide the proposed algorithm into off-line computation and on-line scheduling parts. Simulation results indicate that the proposed policy significantly improves the performance of the data collection and reduces the energy consumption. They prove the effectiveness of the threshold, which could reduce the collection delay by at least 10% while guaranteeing the data quality.<\/jats:p>","DOI":"10.3390\/s20133758","type":"journal-article","created":{"date-parts":[[2020,7,6]],"date-time":"2020-07-06T09:49:11Z","timestamp":1594028951000},"page":"3758","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Energy-Efficient Data Collection Using Autonomous Underwater Glider: A Reinforcement Learning Formulation"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1116-963X","authenticated-orcid":false,"given":"Xinbin","family":"Li","sequence":"first","affiliation":[{"name":"Institute of Electrical Engineering, Yanshan University, Qinhuangdao 066004, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xianglin","family":"Xu","sequence":"additional","affiliation":[{"name":"Institute of Electrical Engineering, Yanshan University, Qinhuangdao 066004, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lei","family":"Yan","sequence":"additional","affiliation":[{"name":"Institute of Electrical Engineering, Yanshan University, Qinhuangdao 066004, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haihong","family":"Zhao","sequence":"additional","affiliation":[{"name":"Institute of Electrical Engineering, Yanshan University, Qinhuangdao 066004, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tongwei","family":"Zhang","sequence":"additional","affiliation":[{"name":"National Deep Sea Center, Qingdao 266237, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,7,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"9280","DOI":"10.1109\/TVT.2019.2931312","article-title":"A Scheme for Delay-Sensitive Spatiotemporal Routing in SDN-Enabled Underwater Acoustic Sensor Networks","volume":"68","author":"Lin","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1021","DOI":"10.1017\/S0373463319000080","article-title":"Review of navigation and positioning of deep-sea manned submersibles","volume":"72","author":"Zhang","year":"2019","journal-title":"J. Navig."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1016\/j.apacoust.2019.06.019","article-title":"Development and calibration of an underwater acoustic data collection system for monitoring coarse bedload transport","volume":"155","author":"Goodwiller","year":"2019","journal-title":"Appl. Acoust."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"10671","DOI":"10.1109\/TVT.2018.2867021","article-title":"A stratification-based data collection scheme in underwater acoustic sensor networks","volume":"67","author":"Han","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.marpolbul.2019.01.009","article-title":"A review of unmanned vehicles for the detection and monitoring of marine fauna","volume":"140","author":"Verfuss","year":"2019","journal-title":"Mar. Pollut. Bull."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Jiang, C., Li, J., and Xu, W. (2019). The Use of Underwater Gliders as Acoustic Sensing Platforms. Appl. Sci., 9.","DOI":"10.3390\/app9224839"},{"key":"ref_7","unstructured":"Lee, K.M.B., Lee, J.J.H., Yoo, C., Hollings, B., and Fitch, R. (2018, January 4\u20136). Active perception for plume source localisation with underwater gliders. Proceedings of the Australasian Conference on Robotics and Automation, Lincoln, New Zealand."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Leccese, F., Cagnetti, M., Giarnetti, S., Petritoli, E., Luisetto, I., Tuti, S., \u00d0urovi\u0107-Pej\u010dev, R., \u00d0or\u0111evi\u0107, T., Toma\u0161evi\u0107, A., and Bursi\u0107, V. (2018, January 8\u201310). A simple takagi-sugeno fuzzy modelling case study for an underwater glider control system. Proceedings of the 2018 IEEE International Workshop on Metrology for the Sea; Learning to Measure Sea Health Parameters (MetroSea), Bari, Italy.","DOI":"10.1109\/MetroSea.2018.8657877"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.dsr.2015.08.009","article-title":"Sea surface cooling in the Northern South China Sea observed using Chinese sea-wing underwater glider measurements","volume":"105","author":"Qiu","year":"2015","journal-title":"Deep Sea Res. Part I"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/s40857-018-0126-y","article-title":"Using Petrel II Glider to Analyze Underwater Noise Spectrogram in the South China Sea","volume":"46","author":"Liu","year":"2018","journal-title":"Acoust. Aust."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1109\/JOE.2012.2227551","article-title":"Motion parameter optimization and sensor scheduling for the sea-wing underwater glider","volume":"38","author":"Yu","year":"2013","journal-title":"IEEE J. Ocean. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"977","DOI":"10.1002\/wcm.654","article-title":"Prospects and problems of wireless communication for underwater sensor networks","volume":"8","author":"Lanbo","year":"2008","journal-title":"Wirel. Commun. Mob. Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1016\/j.adhoc.2005.01.004","article-title":"Underwater acoustic sensor networks: Research challenges","volume":"3","author":"Akyildiz","year":"2005","journal-title":"Ad Hoc Netw."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Li, K., Nuchkrua, T., Zhao, H., Yuan, Y., and Boonto, S. (2018, January 20\u201324). Learning-based adaptive robust control of manipulated pneumatic artificial muscle driven by H 2-based metal hydride. Proceedings of the 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE), Munich, Germany.","DOI":"10.1109\/COASE.2018.8560584"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Heidemann, J., Ye, W., Wills, J., Syed, A., and Li, Y. (2006, January 3\u20136). Research Challenges and Applications for underwater sensor networking. Proceedings of the IEEE Wireless Communications and Networking Conference, Las Vegas, NV, USA.","DOI":"10.1109\/WCNC.2006.1683469"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"899","DOI":"10.1109\/JSAC.2012.120606","article-title":"Underwater data collection using robotic sensor networks","volume":"30","author":"Hollinger","year":"2012","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1010","DOI":"10.1109\/TMC.2019.2907854","article-title":"A High-Availability Data Collection Scheme based on Multi-AUVs for Underwater Sensor Networks","volume":"19","author":"Han","year":"2019","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Leccese, F., Cagnetti, M., Giarnetti, S., Petritoli, E., Luisetto, I., Tuti, S., Leccisi, M., \u00d0urovi\u0107-Pej\u010dev, R., \u00d0or\u0111evi\u0107, T., and Toma\u0161evi\u0107, A. (2018, January 22\u201324). Comparison between routing protocols for wide archeological site. Proceedings of the 2018 Metrology for Archaeology and Cultural Heritage (MetroArchaeo), Cassino FR, Italy.","DOI":"10.1109\/MetroArchaeo43810.2018.9089808"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"5755","DOI":"10.1109\/TII.2019.2912320","article-title":"District Partition-based Data Collection Algorithm with Event Dynamic Competition in Underwater Acoustic Sensor Networks","volume":"15","author":"Han","year":"2019","journal-title":"IEEE Trans. Ind. Inf."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"9091","DOI":"10.1109\/ACCESS.2019.2891590","article-title":"DQELR: An adaptive deep Q-network-based energy-and latency-aware routing protocol design for underwater acoustic sensor networks","volume":"7","author":"Su","year":"2019","journal-title":"IEEE Access"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Cheung, M.Y., Leighton, J., and Hover, F.S. (2013, January 6\u201310). Multi-armed bandit formulation for autonomous mobile acoustic relay adaptive positioning. Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6631165"},{"key":"ref_22","unstructured":"Bhatta, P., Fiorelli, E., Lekien, F., Leonard, N.E., Paley, D., Zhang, F., Bachmayer, R., Davis, R.E., Fratantoni, D.M., and Sepulchre, R. (2005, January 9\u201311). Coordination of an underwater glider fleet for adaptive ocean sampling. Proceedings of the International Workshop on Underwater Robotics International Advanced Robotics Programmed (IARP), Genoa, Italy."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Petritoli, E., Leccese, F., and Cagnetti, M. (2018, January 8\u201310). A high accuracy buoyancy system control for an underwater glider. Proceedings of the 2018 IEEE International Workshop on Metrology for the Sea; Learning to Measure Sea Health Parameters (MetroSea), Bari, Italy.","DOI":"10.1109\/MetroSea.2018.8657831"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1038\/s41928-019-0355-6","article-title":"What should 6G be","volume":"3","author":"Dang","year":"2020","journal-title":"Nat. Electron."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1109\/TAES.2015.140935","article-title":"Dynamic underwater glider network for environmental field estimation","volume":"52","author":"Grasso","year":"2016","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1109\/JOE.2013.2278787","article-title":"Statistical characterization and computationally efficient modeling of a class of underwater acoustic communication channels","volume":"38","author":"Qarabaqi","year":"2013","journal-title":"IEEE J. Ocean. Eng."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Pignieri, F., De Rango, F., Veltri, F., and Marano, S. (2008, January 16\u201319). Markovian approach to model underwater acoustic channel: Techniques comparison. Proceedings of the MILCOM 2008-2008 IEEE Military Communications Conference, San Diego, CA, USA.","DOI":"10.1109\/MILCOM.2008.4753161"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Xiaohong, S., Haiyan, W., Yuzhi, Z., and Ruiqin, Z. (2012). Adaptive Technique for Underwater Acoustic Communication. Underwater Acoust., 59\u201374.","DOI":"10.5772\/33985"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2149","DOI":"10.1109\/TVT.2010.2041803","article-title":"Distributed optimal relay selection in wireless cooperative networks with finite-state Markov channels","volume":"59","author":"Wei","year":"2010","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"4594","DOI":"10.1109\/TWC.2009.080550","article-title":"Distributed sender scheduling for multimedia transmission in wireless mobile peer-to-peer networks","volume":"8","author":"Si","year":"2009","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1111\/j.2517-6161.1979.tb01068.x","article-title":"Bandit processes and dynamic allocation indices","volume":"41","author":"Gittins","year":"1979","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Mahajan, A., and Teneketzis, D. (2008). Multi-armed bandit problems. Foundations and Applications of Sensor Management, Springer.","DOI":"10.1007\/978-0-387-49819-5_6"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1093\/biomet\/66.3.561","article-title":"A dynamic allocation index for the discounted multiarmed bandit problem","volume":"66","author":"Gittins","year":"1979","journal-title":"Biometrika"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/13\/3758\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:47:32Z","timestamp":1760176052000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/13\/3758"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,4]]},"references-count":33,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2020,7]]}},"alternative-id":["s20133758"],"URL":"https:\/\/doi.org\/10.3390\/s20133758","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,4]]}}}