{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T20:15:42Z","timestamp":1768421742185,"version":"3.49.0"},"reference-count":37,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T00:00:00Z","timestamp":1667779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["62072211"],"award-info":[{"award-number":["62072211"]}]},{"name":"National Natural Science Foundation of China","award":["51939003"],"award-info":[{"award-number":["51939003"]}]},{"name":"National Natural Science Foundation of China","award":["U20A20285"],"award-info":[{"award-number":["U20A20285"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Target search for moving and invisible objects has always been considered a challenge, as the floating objects drift with the flows. This study focuses on target search by multiple autonomous underwater vehicles (AUV) and investigates a multi-agent target search method (MATSMI) for moving and invisible objects. In the MATSMI algorithm, based on the multi-agent deep deterministic policy gradient (MADDPG) method, we add spatial and temporal information to the reinforcement learning state and set up specialized rewards in conjunction with a maritime target search scenario. Additionally, we construct a simulation environment to simulate a multi-AUV search for the floating object. The simulation results show that the MATSMI method has about 20% higher search success rate and about 70 steps shorter search time than the traditional search method. In addition, the MATSMI method converges faster than the MADDPG method. This paper provides a novel and effective method for solving the maritime target search problem.<\/jats:p>","DOI":"10.3390\/s22218562","type":"journal-article","created":{"date-parts":[[2022,11,8]],"date-time":"2022-11-08T08:17:12Z","timestamp":1667895432000},"page":"8562","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["A Multi-AUV Maritime Target Search Method for Moving and Invisible Objects Based on Multi-Agent Deep Reinforcement Learning"],"prefix":"10.3390","volume":"22","author":[{"given":"Guangcheng","family":"Wang","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"}]},{"given":"Fenglin","family":"Wei","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9025-3375","authenticated-orcid":false,"given":"Yu","family":"Jiang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"},{"name":"State Key Lab of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, China"}]},{"given":"Minghao","family":"Zhao","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"}]},{"given":"Kai","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"}]},{"given":"Hong","family":"Qi","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"},{"name":"State Key Lab of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2364","DOI":"10.1109\/TNNLS.2015.2482501","article-title":"Multi-AUV Target Search Based on Bioinspired Neurodynamics Model in 3-D Underwater Environments","volume":"27","author":"Cao","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.oceaneng.2017.12.037","article-title":"Multi-AUV cooperative target search and tracking in unknown underwater environment","volume":"150","author":"Cao","year":"2018","journal-title":"Ocean Eng."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Li, J., Li, C., Chen, T., and Zhang, Y. (2022). Improved RRT Algorithm for AUV Target Search in Unknown 3D Environment. J. Mar. Sci., 10.","DOI":"10.3390\/jmse10060826"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1137\/1023109","article-title":"Search and screening: General principles with historical applications (B. O. Koopman)","volume":"23","author":"Stone","year":"1981","journal-title":"SIAM Rev."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Fallon, M.F., Kaess, M., Johannsson, H., and Leonard, J.J. (2011, January 9\u201313). Efficient AUV navigation fusing acoustic ranging and side-scan sonar. Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China.","DOI":"10.1109\/ICRA.2011.5980302"},{"key":"ref_6","unstructured":"Chew, J.L., and Chitre, M. (2013, January 21\u201325). Object Detection with Sector Scanning Sonar. Proceedings of the OCEANS 2013 MTS\/IEEE, Oceans-IEEE, San Diego, CA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2848","DOI":"10.1016\/j.procs.2015.05.446","article-title":"Computational System for Planning Search and Rescue Operations at Sea","volume":"51","author":"Vettor","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"107399","DOI":"10.1016\/j.oceaneng.2020.107399","article-title":"A decision support method for design and operationalization of search and rescue in maritime emergency","volume":"207","author":"Xiong","year":"2020","journal-title":"Ocean Eng."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"110098","DOI":"10.1016\/j.oceaneng.2021.110098","article-title":"Coverage path planning for maritime search and rescue using reinforcement learning","volume":"241","author":"Ai","year":"2021","journal-title":"Ocean Eng."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"546","DOI":"10.1007\/s12555-019-0666-4","article-title":"Collaborative Mission and Route Planning of Multi-vehicle Systems for Autonomous Search in Marine Environment","volume":"18","author":"Yoon","year":"2020","journal-title":"Int. J. Control Autom. Syst."},{"key":"ref_11","first-page":"85","article-title":"Distributed MPC for Formation Path-Following of Multi-Vehicle Systems","volume":"55","author":"Varagnolo","year":"2022","journal-title":"IFAC-PapersOnLine"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1504\/IJOR.2010.035523","article-title":"An efficient multi-vehicle multi-criteria mission planning and control system for autonomous underwater vehicles","volume":"9","author":"Tavana","year":"2010","journal-title":"Int. J. Oper. Res."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Scharff Willners, J., Gonzalez-Adell, D., Hern\u00e1ndez, J., Pairet, \u00c8., and Petillot, Y. (2021). Online 3-Dimensional Path Planning with Kinematic Constraints in Unknown Environments Using Hybrid A* with Tree Pruning. Sensors, 21.","DOI":"10.3390\/s21041152"},{"key":"ref_14","unstructured":"Meghjani, M., and Dudek, G. (2015, January 26\u201330). Search for a rendezvous with lost target at sea. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (IEEE ICRA 2015), Seattle, WA, USA."},{"key":"ref_15","unstructured":"Shalev-Shwartz, S., Shammah, S., and Shashua, A. (2016). Safe, multi-agent, reinforcement learning for autonomous driving. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Pan, X., You, Y., Wang, Z., and Lu, C. (2017). Virtual to real reinforcement learning for autonomous driving. arXiv.","DOI":"10.5244\/C.31.11"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3301273","article-title":"Reinforcement learning for UAV attitude control","volume":"3","author":"Koch","year":"2019","journal-title":"ACM Trans. Cyber-Phys. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mohammed, A., Sultan, S., Cho, S., and Pyun, J. (2022). Powering UAV with Deep Q-Network for Air Quality Tracking. Sensors, 22.","DOI":"10.3390\/s22166118"},{"key":"ref_19","unstructured":"Li, S., Wu, Y., Cui, X., Dong, H., Fang, F., and Russell, S. (February, January 27). Robust Multi-Agent Reinforcement Learning via Minimax Deep Deterministic Policy Gradient. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), Honolulu, HI, USA."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1109\/TITS.2019.2901791","article-title":"Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control","volume":"21","author":"Chu","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Jiang, W., Yu, W., Wang, W., and Huang, T. (2022). Multi-Agent Reinforcement Learning for Joint Cooperative Spectrum Sensing and Channel Access in Cognitive UAV Networks. Sensors, 22.","DOI":"10.3390\/s22041651"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jo, S., Yang, W., Choi, H., Noh, E., Jo, H., and Park, J. (2022). Deep Q-Learning-Based Transmission Power Control of a High Altitude Platform Station with Spectrum Sharing. Sensors, 22.","DOI":"10.3390\/s22041630"},{"key":"ref_23","first-page":"51","article-title":"Multi-agent actor-critic for mixed cooperative-competitive environments","volume":"30","author":"Lowe","year":"2017","journal-title":"Adv. Neural Inf. Process Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Stone, L.D., Brown, S.S., Buemi, R.P., and Hopkins, C.R. (1978). Numerical Optimization of Search for a Moving Target.","DOI":"10.21236\/ADA058470"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1154","DOI":"10.1287\/opre.29.6.1154","article-title":"Optimal whereabouts search for a moving target","volume":"29","author":"Stone","year":"1979","journal-title":"Oper. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"739","DOI":"10.1287\/opre.31.4.739","article-title":"Search for a moving target: The Fab Algorithm","volume":"31","author":"Washburn","year":"1983","journal-title":"Oper. Res."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1287\/moor.6.4.518","article-title":"Constrained Optimization of functionals with search theory applications","volume":"6","author":"Stromquist","year":"1981","journal-title":"Math. Oper. Res."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, J., and Liu, Y. (2021, January 5\u20136). Deep Reinforcement Learning based Adaptive Real-Time Path Planning for UAV. Proceedings of the 8th International Conference on Dependable Systems and Their Applications, Yinchuan, China.","DOI":"10.1109\/DSA52907.2021.00077"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Yu, J., Liu, G., Xu, J., Zhao, Z., Chen, Z., Yang, M., Wang, X., and Bai, Y. (2022). A Hybrid Multi-Target Path Planning Algorithm for Unmanned Cruise Ship in an Unknown Obstacle Environment. Sensors, 22.","DOI":"10.3390\/s22072429"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Nussbaum, D., and Y\u00f6r\u00fck\u00e7\u00fc, A. (2015, January 11\u201313). Moving target search with subgoal graphs. Proceedings of the Eighth Annual Symposium on Combinatorial Search, Ein Gedi, Israel.","DOI":"10.1609\/icaps.v25i1.13709"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Botea, A., Baier, J., Harabor, D., and Hern\u00e1ndez, C. (2013, January 10\u201314). Moving target search with compressed path databases. Proceedings of the Twenty-Third International Conference on Automated Planning and Scheduling, Rome, Italy.","DOI":"10.1609\/icaps.v23i1.13599"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Song, T., Huo, X., and Wu, X. (2020). A Two-Stage Method for Target Searching in the Path Planning for Mobile Robots. Sensors, 20.","DOI":"10.3390\/s20236919"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1109\/LRA.2019.2891991","article-title":"Deep Reinforcement Learning Robot for Search and Rescue Applications: Exploration in Unknown Cluttered Environments","volume":"4","author":"Niroui","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Liu, Z., Gao, X., and Fu, X. (2018). A Cooperative Search and Coverage Algorithm with Controllable Revisit and Connectivity Maintenance for Multiple Unmanned Aerial Vehicles. Sensors, 18.","DOI":"10.3390\/s18051472"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"McCalmon, J. (2021, January 2\u20139). Exploration of Unknown Environments Using Deep Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual Event.","DOI":"10.1609\/aaai.v35i18.17981"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Walker, O., Vanegas, F., and Gonzalez, F. (2020). A Framework for Multi-Agent UAV Exploration and Target-Finding in GPS-Denied and Partially Observable Environments. Sensors, 20.","DOI":"10.3390\/s20174739"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/s43154-021-00048-3","article-title":"Multi-Agent Systems for Search and Rescue Applications","volume":"2","author":"Drew","year":"2021","journal-title":"Curr. Robot. Rep."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8562\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:11:43Z","timestamp":1760145103000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8562"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,7]]},"references-count":37,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["s22218562"],"URL":"https:\/\/doi.org\/10.3390\/s22218562","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,7]]}}}