{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T15:45:51Z","timestamp":1784303151527,"version":"3.55.0"},"reference-count":36,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2022,9,6]],"date-time":"2022-09-06T00:00:00Z","timestamp":1662422400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, a cooperative search method for multiple UAVs is proposed to solve the problem of low efficiency of multi-UAV task execution by using a cooperative game with incomplete information. To improve search efficiency, CBBA (Consensus-Based Bundle Algorithm) is applied to designate the tasks area for each UAV. Then, Independent Deep Reinforcement Learning (IDRL) is used to solve Nash equilibrium to improve UAVs\u2019 collaborations. The proposed reward function is smartly developed to guide UAVs to fly along the path with higher reward value while avoiding the collisions between UAVs during flights. Finally, extensive experiments are carried out to compare our proposed method with other algorithms. Simulation results show that the proposed method can obtain more rewards in the same period of time as other algorithms.<\/jats:p>","DOI":"10.3390\/s22186737","type":"journal-article","created":{"date-parts":[[2022,9,8]],"date-time":"2022-09-08T04:18:32Z","timestamp":1662610712000},"page":"6737","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Cooperative Search Method for Multiple UAVs Based on Deep Reinforcement Learning"],"prefix":"10.3390","volume":"22","author":[{"given":"Mingsheng","family":"Gao","sequence":"first","affiliation":[{"name":"College of Internet of Things, Hohai University, Changzhou 213002, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoxuan","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Internet of Things, Hohai University, Changzhou 213002, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"15441","DOI":"10.1109\/JIOT.2021.3073973","article-title":"Deep Reinforcement Learning Multi-UAV Trajectory Control for Target Tracking","volume":"8","author":"Moon","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Yan, C., and Xiang, X. (2018, January 23\u201325). A Path Planning Algorithm for UAV Based on Improved Q-Learning. Proceedings of the 2018 2nd International Conference on Robotics and Automation Sciences (ICRAS), Wuhan, China.","DOI":"10.1109\/ICRAS.2018.8443226"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Pei-bei, M., Zuo-e, F., and Jun, J. (2014, January 8\u201310). Cooperative control of multi-UAV with time constraint in the threat environment. Proceedings of the 2014 IEEE Chinese Guidance, Navigation and Control Conference, Yantai, China.","DOI":"10.1109\/CGNCC.2014.7007549"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Schouwenaars, T., How, J., and Feron, E. (2004, January 16\u201319). Decentralized Cooperative Trajectory Planning of Multiple Aircraft with Hard Safety Guarantees. Proceedings of the AIAA Guidance, Navigation, and Control Conference and Exhibit, American Institute of Aeronautics and Astronautics, Providence, RI, USA.","DOI":"10.2514\/6.2004-5141"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Li, L., Gu, Q., and Liu, L. (2020, January 26\u201329). Research on Path Planning Algorithm for Multi-UAV Maritime Targets Search Based on Genetic Algorithm. Proceedings of the 2020 IEEE International Conference on Information Technology, Big Data and Artificial Intelligence (ICIBA), Chongqing, China.","DOI":"10.1109\/ICIBA50161.2020.9277470"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhao, H., Liu, Q., Ge, Y., Kong, R., and Chen, E. (2016, January 12\u201315). Group Preference Aggregation: A Nash Equilibrium Approach. Proceedings of the 2016 IEEE 16th International Conference on Data Mining (ICDM), Barcelona, Spain.","DOI":"10.1109\/ICDM.2016.0079"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"912","DOI":"10.1109\/TRO.2009.2022423","article-title":"Consensus-Based Decentralized Auctions for Robust Task Allocation","volume":"25","author":"Choi","year":"2009","journal-title":"IEEE Trans. Robot."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"478","DOI":"10.1016\/j.procs.2015.08.169","article-title":"A Hierarchical Decomposition Framework for Modeling Combinatorial Optimization Problems","volume":"60","author":"Chaieb","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_9","unstructured":"Wang, Y., Cai, W., and Zheng, Y.R. (2017, January 18\u201321). Dubins curves for 3D multi-vehicle path planning using spline interpolation. Proceedings of the OCEANS 2017-Anchorage, Anchorage, AK, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Liu, J., Zhang, Y., Wang, X., Xu, C., and Ma, X. (2020, January 21\u201323). Min-max Path Planning of Multiple UAVs for Autonomous Inspection. Proceedings of the 2020 International Conference on Wireless Communications and Signal Processing (WCSP), Nanjing, China.","DOI":"10.1109\/WCSP49889.2020.9299869"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Qingtian, H. (2021, January 22\u201324). Research on Cooperate Search Path Planning of Multiple UAVs Using Dubins Curve. Proceedings of the 2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA), Bhubaneswar, India.","DOI":"10.1109\/ICPECA51329.2021.9362518"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Han, W., and Li, W. (2021, January 10\u201312). Research on Path Planning Problem of Multi-UAV Data Acquisition System for Emergency Scenario. Proceedings of the 2021 International Conference on Electronic Information Technology and Smart Agriculture (ICEITSA), Huaihua, China.","DOI":"10.1109\/ICEITSA54226.2021.00049"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yaguchi, Y., and Tomeba, T. (2021, January 15\u201318). Region Coverage Flight Path Planning Using Multiple UAVs to Monitor the Huge Areas. Proceedings of the 2021 International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece.","DOI":"10.1109\/ICUAS51884.2021.9476775"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Krusniak, M., James, A., Flores, A., and Shang, Y. (2021, January 10\u201312). A Multiple UAV Path-Planning Approach to Small Object Counting with Aerial Images. Proceedings of the 2021 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA.","DOI":"10.1109\/ICCE50685.2021.9427712"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1109\/TAI.2021.3103143","article-title":"Bipartite Cooperative Coevolution for Energy-Aware Coverage Path Planning of UAVs","volume":"3","author":"Shao","year":"2022","journal-title":"IEEE Trans. Artif. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"51770","DOI":"10.1109\/ACCESS.2020.2980203","article-title":"Path Planning for UAV to Cover Multiple Separated Convex Polygonal Regions","volume":"8","author":"Xie","year":"2020","journal-title":"IEEE Access"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Xie, J., and Chen, J. (2022). Multiregional Coverage Path Planning for Multiple Energy Constrained UAVs. IEEE Trans. Intell. Transp. Syst.","DOI":"10.1109\/TITS.2022.3160402"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Pan, S. (2019, January 14\u201316). UAV Delivery Planning Based on K-Means++ Clustering and Genetic Algorithm. Proceedings of the 2019 5th International Conference on Control Science and Systems Engineering (ICCSSE), Shanghai, China.","DOI":"10.1109\/ICCSSE.2019.00011"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Yue, X., and Zhang, W. (2018, January 25\u201327). UAV Path Planning Based on K-Means Algorithm and Simulated Annealing Algorithm. Proceedings of the 2018 37th Chinese Control Conference (CCC), Wuhan, China.","DOI":"10.23919\/ChiCC.2018.8483993"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s11047-019-09749-3","article-title":"Cooperative search method for multiple AUVs based on target clustering and path optimization","volume":"20","author":"Ling","year":"2019","journal-title":"Nat. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Steven, A., Hertono, G.F., and Handari, B.D. (2017, January 15\u201316). Implementation of clustered ant colony optimization in solving fixed destination multiple depot multiple traveling salesman problem. Proceedings of the 2017 1st International Conference on Informatics and Computational Sciences (ICICoS), Semarang, Indonesia.","DOI":"10.1109\/ICICOS.2017.8276351"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Almansoor, M., and Harrath, Y. (2021, January 29\u201330). Big Data Analytics, Greedy Approach, and Clustering Algorithms for Real-Time Cash Management of Automated Teller Machines. Proceedings of the 2021 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT), Zallaq, Bahrain.","DOI":"10.1109\/3ICT53449.2021.9581890"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhou, S., Lin, K.-J., and Shih, C.-S. (2015, January 14\u201316). Device clustering for fault monitoring in Internet of Things systems. Proceedings of the 2015 IEEE 2nd World Forum on Internet of Things (WF-IoT), Milan, Italy.","DOI":"10.1109\/WF-IoT.2015.7389057"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Trigui, S., Koub\u00e2a, A., Cheikhrouhou, O., Qureshi, B., and Youssef, H. (2016, January 4\u20136). A Clustering Market-Based Approach for Multi-robot Emergency Response Applications. Proceedings of the 2016 International Conference on Autonomous Robot Systems and Competitions (ICARSC), Bragana, Portugal.","DOI":"10.1109\/ICARSC.2016.14"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Tang, Y. (2021, January 10\u201312). UAV Detection Based on Clustering Analysis and Improved Genetic Algorithm. Proceedings of the 2021 International Conference on Electronic Communications, Internet of Things and Big Data (ICEIB), Yilan County, Taiwan.","DOI":"10.1109\/ICEIB53692.2021.9686421"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Wang, J., and Meng, Q.H. (2020\u201324, January 24). Path Planning for Nonholonomic Multiple Mobile Robot System with Applications to Robotic Autonomous Luggage Trolley Collection at Airports. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341403"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hassanpour, F., and Akbarzadeh-T, M.-R. (2020, January 29\u201330). Solving a Multi-Traveling Salesmen Problem using a Mamdani Fuzzy Inference Engine and Simulated Annealing Search Algorithm. Proceedings of the 2020 10th International Conference on Computer and Knowledge Engineering (ICCKE), Mashhad, Iran.","DOI":"10.1109\/ICCKE50421.2020.9303696"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"45695","DOI":"10.1109\/ACCESS.2022.3170583","article-title":"Decentralized Multi-UAV Path Planning Based on Two-Layer Coordinative Framework for Formation Rendezvous","volume":"10","author":"Cheng","year":"2022","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Baranwal, M., Roehl, B., and Salapaka, S.M. (2017, January 24\u201326). Multiple traveling salesmen and related problems: A maximum-entropy principle based approach. Proceedings of the 2017 American Control Conference (ACC), Seattle, WA, USA.","DOI":"10.23919\/ACC.2017.7963559"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ze-ling, C., Qi, W., and Ye-qing, Y. (2018, January 10\u201312). Research on Optimization Method of Multi-UAV Collaborative Task Planning. Proceedings of the 2018 IEEE CSAA Guidance, Navigation and Control Conference (CGNCC), Xiamen, China.","DOI":"10.1109\/GNCC42960.2018.9018868"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1171","DOI":"10.1109\/OJCOMS.2021.3081996","article-title":"Multi-UAV Path Planning for Wireless Data Harvesting with Deep Reinforcement Learning","volume":"2","author":"Bayerlein","year":"2021","journal-title":"IEEE Open J. Commun. Soc."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wang, Z., Wan, R., Gui, X., and Zhou, G. (2020, January 6\u20138). Deep Reinforcement Learning of Cooperative Control with Four Robotic Agents by MADDPG. Proceedings of the 2020 International Conference on Computer Engineering and Intelligent Control (ICCEIC), Chongqing, China.","DOI":"10.1109\/ICCEIC51584.2020.00061"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Miyazaki, K., Matsunaga, N., and Murata, K. (2021, January 12\u201315). Formation path learning for cooperative transportation of multiple robots using MADDPG. Proceedings of the 2021 21st International Conference on Control, Automation and Systems (ICCAS), Jeju, Korea.","DOI":"10.23919\/ICCAS52745.2021.9649891"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Chen, W., Hua, L., Xu, L., Zhang, B., Li, M., Ma, T., and Chen, Y.-Y. (2021, January 15\u201317). MADDPG Algorithm for Coordinated Welding of Multiple Robots. Proceedings of the 2021 6th International Conference on Automation, Control and Robotics Engineering (CACRE), Dalian, China.","DOI":"10.1109\/CACRE52464.2021.9501327"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"146264","DOI":"10.1109\/ACCESS.2019.2943253","article-title":"Joint Optimization of Multi-UAV Target Assignment and Path Planning Based on Multi-Agent Reinforcement Learning","volume":"7","author":"Han","year":"2019","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Baum, M., and Passino, K. (2002, January 5\u20138). A Search-Theoretic Approach to Cooperative Control for Uninhabited Air Vehicles. Proceedings of the AIAA Guidance, Navigation, and Control Conference and Exhibit, Monterey, CA, USA.","DOI":"10.2514\/6.2002-4589"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6737\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:24:18Z","timestamp":1760142258000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6737"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,6]]},"references-count":36,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["s22186737"],"URL":"https:\/\/doi.org\/10.3390\/s22186737","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,6]]}}}