{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T18:36:56Z","timestamp":1772908616498,"version":"3.50.1"},"reference-count":36,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2022,1,28]],"date-time":"2022-01-28T00:00:00Z","timestamp":1643328000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>As an emerging paradigm considering data privacy and transmission efficiency, decentralized learning aims to acquire a global model using the training data distributed over many user devices. It is a challenging problem since link loss, partial device participation, and non-independent and identically distributed (non-iid) data distribution would all deteriorate the performance of decentralized learning algorithms. Existing work may restrict to linear models or show poor performance over non-iid data. Therefore, in this paper, we propose a decentralized learning scheme based on distributed parallel stochastic gradient descent (DPSGD) and graph neural network (GNN) to deal with the above challenges. Specifically, each user device participating in the learning task utilizes local training data to compute local stochastic gradients and updates its own local model. Then, each device utilizes the GNN model and exchanges the model parameters with its neighbors to reach the average of resultant global models. The iteration repeats until the algorithm converges. Extensive simulation results over both iid and non-iid data validate the algorithm\u2019s convergence to near optimal results and robustness to both link loss and partial device participation.<\/jats:p>","DOI":"10.3390\/s22031030","type":"journal-article","created":{"date-parts":[[2022,1,29]],"date-time":"2022-01-29T01:43:27Z","timestamp":1643420607000},"page":"1030","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["A Graph Neural Network Based Decentralized Learning Scheme"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2191-4912","authenticated-orcid":false,"given":"Huiguo","family":"Gao","sequence":"first","affiliation":[{"name":"College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China"},{"name":"Zhejiang Provincial Key Laboratory of Information Processing, Communication and Networking (IPCAN), Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7686-0507","authenticated-orcid":false,"given":"Mengyuan","family":"Lee","sequence":"additional","affiliation":[{"name":"College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China"},{"name":"Zhejiang Provincial Key Laboratory of Information Processing, Communication and Networking (IPCAN), Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7296-1490","authenticated-orcid":false,"given":"Guanding","family":"Yu","sequence":"additional","affiliation":[{"name":"College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China"},{"name":"Zhejiang Provincial Key Laboratory of Information Processing, Communication and Networking (IPCAN), Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhaolin","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Agarwal, A., and Duchi, J.C. (2012, January 10\u201313). Distributed delayed stochastic optimization. Proceedings of the 51st IEEE Conference on Decision and Control, Maui, HI, USA.","DOI":"10.1109\/CDC.2012.6426626"},{"key":"ref_2","first-page":"551","article-title":"Online passive-aggressive algorithms","volume":"7","author":"Crammer","year":"2006","journal-title":"J. Mach. Learn. Res."},{"key":"ref_3","unstructured":"Lian, X., Huang, Y., Li, Y., and Liu, J. (2015, January 7\u201312). Asynchronous parallel stochastic gradient for nonconvex optimization. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lian, Z., Wang, W., and Su, C. (2021, January 14\u201323). COFEL: Communication-Efficient and Optimized Federated Learning with Local Differential Privacy. Proceedings of the IEEE International Conference on Communications, Montreal, QC, Canada.","DOI":"10.1109\/ICC42927.2021.9500632"},{"key":"ref_5","unstructured":"Wang, W., Memon, F.H., Lian, Z., Yin, Z., Gadekallu, T.R., Phan, Q., Dev, K., and Su, C. (2021). Secure-Enhanced Federated Learning for AI-Empowered Electric Vehicle Energy Prediction. IEEE Consum. Electron. Mag."},{"key":"ref_6","unstructured":"Yuan, K., Ling, Q., and Yin, W. (2013). On the convergence of decentralized gradient descent. arxiv."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1109\/TAC.2008.2009515","article-title":"Distributed subgradient methods for multiagent optimization","volume":"54","author":"Ozdaglar","year":"2009","journal-title":"IEEE Trans. Automat. Contr."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Jakovetic, D., Xavier, J., and Moura, J.M. (2012, January 20\u201322). Convergence rate analysis of distributed gradient methods for smooth optimization. Proceedings of the 20th Telecommunications Forum, Belgrade, Serbia.","DOI":"10.1109\/TELFOR.2012.6419345"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"944","DOI":"10.1137\/14096668X","article-title":"Extra: An exact first-order algorithm for decentralized consensus optimization","volume":"25","author":"Shi","year":"2015","journal-title":"SIAM J. Optim."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Nedi\u0107, A., Olshevsky, A., and Shi, W. (2016). Achieving geometric convergence for distributed optimization over time-varying graphs. arXiv.","DOI":"10.1109\/CDC.2016.7798402"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1750","DOI":"10.1109\/TSP.2014.2304432","article-title":"On the linear convergence of the ADMM in decentralized consensus optimization","volume":"62","author":"Shi","year":"2014","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_12","unstructured":"He, L., Bian, A., and Jaggi, M. (2018, January 3\u20138). Cola: Decentralized linear learning. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_13","unstructured":"Lian, X., Zhang, C., Zhang, H., Hsieh, C.-J., Zhang, W., and Liu, J. (2017, January 4\u20139). Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_14","unstructured":"Ram, S.S., Nedi\u0107, A., and Veeravalli, V.V. (2009, January 15\u201318). Asynchronous gossip algorithms for stochastic optimization. Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference, Shanghai, China."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Diehl, M., Glineur, F., Jarlebring, E., and Michiels, W. (2010). Asynchronous gossip algorithm for stochastic optimization: Constant stepsize analysis. Recent Advances in Optimization and its Applications in Engineering, Springer.","DOI":"10.1007\/978-3-642-12598-0"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"772","DOI":"10.1109\/JSTSP.2011.2118740","article-title":"Distributed asynchronous constrained stochastic optimization","volume":"5","author":"Srivastava","year":"2011","journal-title":"IEEE J. Sel. Topics. Signal Process."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"516","DOI":"10.1007\/s10957-010-9737-7","article-title":"Distributed stochastic sub-gradient projection algorithms for convex optimization","volume":"147","author":"Ram","year":"2010","journal-title":"J. Optim. Theory Appl."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1109\/JPROC.2006.887293","article-title":"Consensus and Cooperation in Networked Multi-Agent Systems","volume":"95","author":"Fax","year":"2007","journal-title":"Proc. IEEE"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2282","DOI":"10.1109\/TWC.2020.3040983","article-title":"Graph Embedding-Based Wireless Link Scheduling with Few Training Samples","volume":"20","author":"Lee","year":"2021","journal-title":"IEEE Trans. Wireless Commun."},{"key":"ref_20","unstructured":"Lee, M., Hosseinalipour, S., Brinton, C.G., Yu, G., and Dai, H. (2020). A Fast Graph Neural Network-Based Method for Winner Determination in Multi-Unit Combinatorial Auctions. IEEE Trans. Cloud Comput."},{"key":"ref_21","unstructured":"Lee, M., Yu, G., and Dai, H. (2021). Decentralized Inference with Graph Neural Networks in Wireless Communication Systems. IEEE Trans. Mobile Comput."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1088\/1367-2630\/9\/6\/189","article-title":"Worm epidemics in wireless ad hoc networks","volume":"9","author":"Nekovee","year":"2007","journal-title":"New J. Phys."},{"key":"ref_23","unstructured":"Tang, H., Lian, X., Yan, M., Zhang, C., and Liu, J. (2018, January 10\u201315). D2: Decentralized training over decentralized data. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Lu, S., Zhang, X., Sun, H., and Hong, M. (2019, January 2\u20135). GNSD: A gradient-tracking based nonconvex stochastic algorithm for decentralized optimization. Proceedings of the IEEE Data Science Workshop, Minneapolis, MN, USA.","DOI":"10.1109\/DSW.2019.8755807"},{"key":"ref_25","unstructured":"Hosseinalipour, S., Azam, S.S., Brinton, C.G., Michelusi, N., Aggarwal, V., Love, D.J., and Dai, H. (2020). Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks. arXiv."},{"key":"ref_26","unstructured":"Xiao, L., and Boyd, S. (2003, January 9\u201312). Fast linear iterations for distributed averaging. Proceedings of the 42nd IEEE International Conference on Decision and Control, Maui, HI, USA."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"142404","DOI":"10.1109\/ACCESS.2020.3014148","article-title":"Deep learning-based average consensus","volume":"8","author":"Kishida","year":"2020","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Sandryhaila, A., Kar, S., and Moura, J.M.F. (2014, January 4\u20139). Finite-time distributed consensus through graph filters. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing, Florence, Italy.","DOI":"10.1109\/ICASSP.2014.6853763"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Iancu, B., and Isufi, E. (2021, January 18\u201322). Towards Finite-Time Consensus with Graph Convolutional Neural Networks. Proceedings of the 28th European Signal Processing Conference, Virtual.","DOI":"10.23919\/Eusipco47968.2020.9287610"},{"key":"ref_30","unstructured":"Kingma, D.P., and Ba, J. (2017). Adam: A method for stochastic optimization. arxiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gama, F., Isufi, E., Leus, G., and Ribeiro, A. (2020). From graph filters to graph neural networks. arXiv.","DOI":"10.1109\/IEEECONF44664.2019.9048767"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"7690","DOI":"10.1109\/TWC.2020.3015671","article-title":"Scheduling for cellular federated edge learning with importance and channel awareness","volume":"19","author":"Ren","year":"2020","journal-title":"IEEE Trans. Wireless Commun."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"13593","DOI":"10.1109\/TVT.2020.3015268","article-title":"Importance-aware data selection and resource allocation in federated edge learning system","volume":"69","author":"He","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Kempe, D., and McSherry, F. (2004, January 13\u201316). A decentralized algorithm for spectral analysis. Proceedings of the 36th Annual ACM Symposium on Theory of Computing, Chicago, IL, USA.","DOI":"10.1145\/1007352.1007438"},{"key":"ref_35","unstructured":"Han, X., Kashif, R., and Roland, V. (2017). Fashion-MNIST: A Novel Image Dataset for Benchmarking Machine Learning Algorithms. arXiv."},{"key":"ref_36","unstructured":"Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., and Chandra, V. (2018). Federated learning with non-IID data. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/3\/1030\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:10:05Z","timestamp":1760134205000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/3\/1030"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,28]]},"references-count":36,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["s22031030"],"URL":"https:\/\/doi.org\/10.3390\/s22031030","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,28]]}}}