{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T09:06:47Z","timestamp":1762938407509,"version":"3.45.0"},"reference-count":25,"publisher":"Wiley","issue":"25-26","license":[{"start":{"date-parts":[[2025,10,1]],"date-time":"2025-10-01T00:00:00Z","timestamp":1759276800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/501100005089","name":"Natural Science Foundation of Beijing Municipality","doi-asserted-by":"publisher","award":["4232024"],"award-info":[{"award-number":["4232024"]}],"id":[{"id":"10.13039\/501100005089","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2022YFF0604502"],"award-info":[{"award-number":["2022YFF0604502"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Concurrency and Computation"],"published-print":{"date-parts":[[2025,11,30]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:p>With the rapid development of artificial intelligence, big data, and distributed computing technologies, hierarchical federated learning has emerged as a widely studied distributed machine learning framework. In hierarchical federated learning, edge servers are deployed between cloud servers and mobile devices, efficiently receiving local models from nearby mobile devices and performing edge model aggregation. Node collaboration in hierarchical federated learning can reduce training costs and improve model quality while protecting data privacy. However, data security risks and resource consumption during model training can reduce the willingness of mobile devices to participate. Additionally, collaborative nodes are often heterogeneous, facing issues such as skewed datasets and imbalanced capabilities. Therefore, this paper proposes a deep reinforcement learning\u2010based incentive mechanism for node collaboration, aimed at maximizing node benefits. A node collaboration strategy optimization model is then constructed using the Markov decision process framework, and the NCIA algorithm, based on deep reinforcement learning networks, is designed. Finally, through extensive simulation experiments, the proposed NCIA algorithm is demonstrated to improve model accuracy by 5.28% and 14.22% compared with the CCEG and FedAvg algorithms, respectively.<\/jats:p>","DOI":"10.1002\/cpe.70320","type":"journal-article","created":{"date-parts":[[2025,10,2]],"date-time":"2025-10-02T01:24:32Z","timestamp":1759368272000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Design of Incentive Mechanism for Node Collaboration in Hierarchical Federated Learning Based on Deep Reinforcement Learning"],"prefix":"10.1002","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1444-9469","authenticated-orcid":false,"given":"Zhuo","family":"Li","sequence":"first","affiliation":[{"name":"Beijing Key Laboratory of Internet Culture and Digital Dissemination Research Beijing Information Science and Technology University  Beijing China"},{"name":"School of Computer Science Beijing Information Science and Technology University  Beijing China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-0020-1764","authenticated-orcid":false,"given":"Yu","family":"Xin","sequence":"additional","affiliation":[{"name":"School of Computer Science Beijing Information Science and Technology University  Beijing China"}]},{"given":"Fangxing","family":"Geng","sequence":"additional","affiliation":[{"name":"School of Computer Science Beijing Information Science and Technology University  Beijing China"}]}],"member":"311","published-online":{"date-parts":[[2025,10]]},"reference":[{"volume-title":"Deep Learning","year":"2016","author":"Goodfellow I.","key":"e_1_2_11_2_1"},{"key":"e_1_2_11_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCOMM.2019.2960361"},{"key":"e_1_2_11_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3134077"},{"key":"e_1_2_11_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2021.3082561"},{"key":"e_1_2_11_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.121390"},{"key":"e_1_2_11_7_1","doi-asserted-by":"publisher","DOI":"10.3390\/e25111551"},{"key":"e_1_2_11_8_1","first-page":"1273","volume-title":"Artificial Intelligence and Statistics","author":"McMahan B.","year":"2017"},{"key":"e_1_2_11_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2024.110424"},{"key":"e_1_2_11_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13042-022-01647-y"},{"key":"e_1_2_11_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2019.2904348"},{"key":"e_1_2_11_12_1","first-page":"1","volume-title":"ICC 2020\u20132020 IEEE International Conference on Communications (ICC)","author":"Liu L.","year":"2020"},{"key":"e_1_2_11_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP40776.2020.9054634"},{"key":"e_1_2_11_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2023.3279983"},{"key":"e_1_2_11_15_1","doi-asserted-by":"publisher","DOI":"10.3390\/app13095821"},{"key":"e_1_2_11_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData47090.2019.9006327"},{"key":"e_1_2_11_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2020.2987774"},{"issue":"4","key":"e_1_2_11_18_1","first-page":"1366","article-title":"Research on Hierarchical Federated Learning Incentive Mechanism Based on Master\u2010Slave Game","volume":"45","author":"Jia Y.","year":"2023","journal-title":"Journal of Electronics and Information Technology"},{"key":"e_1_2_11_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM42981.2021.9488743"},{"key":"e_1_2_11_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TWC.2017.2726059"},{"key":"e_1_2_11_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/VTS-APWCS.2019.8851649"},{"key":"e_1_2_11_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2023.3264677"},{"key":"e_1_2_11_23_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i6.25911"},{"key":"e_1_2_11_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2019.07.034"},{"key":"e_1_2_11_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2020.2985694"},{"key":"e_1_2_11_26_1","first-page":"8927","volume-title":"International Conference on Machine Learning","author":"Sim R. H. L.","year":"2020"}],"container-title":["Concurrency and Computation: Practice and Experience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cpe.70320","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T09:04:16Z","timestamp":1762938256000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cpe.70320"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10]]},"references-count":25,"journal-issue":{"issue":"25-26","published-print":{"date-parts":[[2025,11,30]]}},"alternative-id":["10.1002\/cpe.70320"],"URL":"https:\/\/doi.org\/10.1002\/cpe.70320","archive":["Portico"],"relation":{},"ISSN":["1532-0626","1532-0634"],"issn-type":[{"type":"print","value":"1532-0626"},{"type":"electronic","value":"1532-0634"}],"subject":[],"published":{"date-parts":[[2025,10]]},"assertion":[{"value":"2025-02-10","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-16","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70320"}}