{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,18]],"date-time":"2026-02-18T04:36:54Z","timestamp":1771389414095,"version":"3.50.1"},"reference-count":36,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2023,6,10]],"date-time":"2023-06-10T00:00:00Z","timestamp":1686355200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this paper, we study distributed inference and learning over networks which can be modeled by a directed graph. A subset of the nodes observes different features, which are all relevant\/required for the inference task that needs to be performed at some distant end (fusion) node. We develop a learning algorithm and an architecture that can combine the information from the observed distributed features, using the processing units available across the networks. In particular, we employ information-theoretic tools to analyze how inference propagates and fuses across a network. Based on the insights gained from this analysis, we derive a loss function that effectively balances the model\u2019s performance with the amount of information transmitted across the network. We study the design criterion of our proposed architecture and its bandwidth requirements. Furthermore, we discuss implementation aspects using neural networks in typical wireless radio access and provide experiments that illustrate benefits over state-of-the-art techniques.<\/jats:p>","DOI":"10.3390\/e25060920","type":"journal-article","created":{"date-parts":[[2023,6,12]],"date-time":"2023-06-12T02:28:42Z","timestamp":1686536922000},"page":"920","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["In-Network Learning: Distributed Training and Inference in Networks"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0773-7905","authenticated-orcid":false,"given":"Matei","family":"Moldoveanu","sequence":"first","affiliation":[{"name":"Laboratoire d\u2019Informatique Gaspard-Monge, Universit\u00e9 Paris-Est, 77454 Marne-la-Vall\u00e9e, France"},{"name":"Mathematical and Algorithmic Sciences Lab, Paris Research Center, Huawei Technologies, 92100 Boulogne-Billancourt, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2023-9476","authenticated-orcid":false,"given":"Abdellatif","family":"Zaidi","sequence":"additional","affiliation":[{"name":"Laboratoire d\u2019Informatique Gaspard-Monge, Universit\u00e9 Paris-Est, 77454 Marne-la-Vall\u00e9e, France"},{"name":"Mathematical and Algorithmic Sciences Lab, Paris Research Center, Huawei Technologies, 92100 Boulogne-Billancourt, France"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,10]]},"reference":[{"key":"ref_1","unstructured":"Zou, Z., Shi, Z., Guo, Y., and Ye, J. (2019). Object Detection in 20 Years: A Survey. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/j.pneurobio.2019.01.008","article-title":"The roles of supervised machine learning in systems neuroscience","volume":"175","author":"Glaser","year":"2019","journal-title":"Prog. Neurobiol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"986","DOI":"10.1109\/TMI.2003.815867","article-title":"Mutual-information-based registration of medical images: A survey","volume":"22","author":"Pluim","year":"2003","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1177\/0278364913495721","article-title":"Reinforcement Learning in Robotics: A Survey","volume":"32","author":"Kober","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_5","unstructured":"Vinyals, O., and Le, Q.V. (2015). A Neural Conversational Model. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1887","DOI":"10.1109\/COMST.2016.2527741","article-title":"A Comprehensive Survey of Recent Advancements in Molecular Communication","volume":"18","author":"Farsad","year":"2016","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Peter Hong, Y.W., and Wang, C.C. (2021, January 27\u201330). In-Network Learning via Over-the-Air Computation in Internet-of-Things. Proceedings of the 2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Lucca, Italy.","DOI":"10.1109\/SPAWC51858.2021.9593183"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1109\/MCOM.001.2000015","article-title":"The Internet of Things as a deep neural network","volume":"58","author":"Du","year":"2020","journal-title":"IEEE Commun. Mag."},{"key":"ref_9","unstructured":"McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B.A. (2017, January 20\u201322). Communication-Efficient Learning of Deep Networks from Decentralized Data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, FL, USA."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/MSP.2006.1657815","article-title":"Distributed compression-estimation using wireless sensor networks","volume":"23","author":"Xiao","year":"2006","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_11","unstructured":"Kreidl, O.P., Tsitsiklis, J.N., and Zoumpoulis, S.I. (October, January 29). Decentralized detection in sensor network architectures with feedback. Proceedings of the 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, IL, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1109\/MSP.2007.361598","article-title":"Wireless Sensors in Distributed Detection Applications","volume":"24","author":"Chamberland","year":"2007","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_13","unstructured":"Tsitsiklis, J.N. (1993). Advances in Statistical Signal Processing, JAI Press."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/MPRV.2003.1251168","article-title":"A learning-theory approach to sensor networks","volume":"2","author":"Simic","year":"2003","journal-title":"IEEE Pervasive Comput."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1109\/MSP.2006.1657817","article-title":"Distributed learning in wireless sensor networks","volume":"23","author":"Predd","year":"2006","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"4053","DOI":"10.1109\/TSP.2005.857020","article-title":"Nonparametric decentralized detection using kernel methods","volume":"53","author":"Nguyen","year":"2005","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Jagyasi, B., and Raval, J. (2015, January 8\u201310). Data aggregation in multihop wireless mesh sensor Neural Networks. Proceedings of the 2015 9th International Conference on Sensing Technology (ICST), Auckland, New Zealand.","DOI":"10.1109\/ICSensT.2015.7438366"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Tran, N.H., Bao, W., Zomaya, A., Nguyen, M.N.H., and Hong, C.S. (May, January 29). Federated Learning over Wireless Networks: Optimization Model Design and Analysis. Proceedings of the IEEE INFOCOM 2019\u2014IEEE Conference on Computer Communications, Paris, France.","DOI":"10.1109\/INFOCOM.2019.8737464"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3546","DOI":"10.1109\/TWC.2020.2974748","article-title":"Federated learning over wireless fading channels","volume":"19","author":"Amiri","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1109\/TCOMM.2019.2944169","article-title":"Scheduling Policies for Federated Learning in Wireless Networks","volume":"68","author":"Yang","year":"2020","journal-title":"IEEE Trans. Commun."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jnca.2018.05.003","article-title":"Distributed learning of deep neural network over multiple agents","volume":"116","author":"Gupta","year":"2018","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_22","unstructured":"Ceballos, I., Sharma, V., Mugica, E., Singh, A., Roman, A., Vepakomma, P., and Raskar, R. (2020). SplitNN-driven Vertical Partitioning. arXiv."},{"key":"ref_23","unstructured":"National Institutes of Health (2003). NIH Data Sharing Policy and Implementation Guidance, National Institutes of Health."},{"key":"ref_24","unstructured":"Aguerri, I.E., and Zaidi, A. (2018, January 21\u201323). Distributed Information Bottleneck Method for Discrete and Gaussian Sources. Proceedings of the IEEE International Zurich Seminar on Information and Communications, Zurich, Switzerland."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1109\/TPAMI.2019.2928806","article-title":"Distributed Variational Representation Learning","volume":"43","author":"Aguerri","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zaidi, A., Aguerri, I.E., and Shamai (Shitz), S. (2020). On the Information Bottleneck Problems: Models, Connections, Applications and Information Theoretic Views. Entropy, 22.","DOI":"10.3390\/e22020151"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Moldoveanu, M., and Zaidi, A. (2021, January 27\u201330). On in-network learning. A comparative study with federated and split learning. Proceedings of the 2021 IEEE 22nd International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Lucca, Italy.","DOI":"10.1109\/SPAWC51858.2021.9593182"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Moldoveanu, M., and Zaidi, A. (2021, January 7\u201311). In-network Learning for Distributed Training and Inference in Networks. Proceedings of the IEEE Globecom 2021 Workshops, Madrid, Spain.","DOI":"10.1109\/GCWkshps52748.2021.9682062"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3339474","article-title":"Federated machine learning: Concept and applications","volume":"10","author":"Yang","year":"2019","journal-title":"ACM Trans. Intell. Syst. Technol. (TIST)"},{"key":"ref_30","unstructured":"Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv."},{"key":"ref_31","first-page":"16131","article-title":"Compressing images by encoding their latent representations with relative entropy coding","volume":"33","author":"Flamich","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"6967","DOI":"10.1109\/TIT.2018.2865570","article-title":"Strong Functional Representation Lemma and Applications to Coding Theorems","volume":"64","author":"Li","year":"2018","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1109\/18.32119","article-title":"Multiterminal source encoding with one distortion criterion","volume":"35","author":"Berger","year":"1989","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"El Gamal, A., and Kim, Y.H. (2011). Network Information Theory, Cambridge University Press.","DOI":"10.1017\/CBO9781139030687"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Liu, S., and Deng, W. (2015, January 3\u20136). Very deep convolutional neural network based image classification using small training sample size. Proceedings of the 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ACPR.2015.7486599"},{"key":"ref_36","unstructured":"Liu, H., Chen, M., Er, S., Liao, W., Zhang, T., and Zhao, T. (2022, January 17\u201323). Benefits of overparameterized convolutional residual networks: Function approximation under smoothness constraint. Proceedings of the International Conference on Machine Learning. PMLR, Baltimore, MD, USA."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/6\/920\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:52:18Z","timestamp":1760125938000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/6\/920"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,10]]},"references-count":36,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["e25060920"],"URL":"https:\/\/doi.org\/10.3390\/e25060920","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,10]]}}}