{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T18:14:06Z","timestamp":1778177646926,"version":"3.51.4"},"reference-count":38,"publisher":"Public Library of Science (PLoS)","issue":"5","license":[{"start":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T00:00:00Z","timestamp":1778112000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"\n Federated learning offers a promising path toward privacy-preserving speech emotion recognition, yet existing approaches remain confined to single-corpus evaluation, lack formal differential privacy guarantees, and provide no mechanism for model interpretability. Meanwhile, cross-corpus generalization continues to challenge even centralized systems, with typical accuracy drops of 20\u201340% on unseen datasets due to domain shift in recording conditions, speaker demographics, and cultural expression norms. This paper introduces FedEmoNet, a unified framework that jointly addresses these open problems by combining FedProx-based distributed optimization, a hybrid Temporal Convolutional Network\u2013Transformer (TCN-Transformer) architecture, Particle Swarm Optimization (PSO) feature selection, and calibrated (\n \n \n \n \n \n \u03f5<\/mml:mi>\n =<\/mml:mo>\n 1.0<\/mml:mn>\n ,<\/mml:mo>\n \u03b4<\/mml:mi>\n =<\/mml:mo>\n \n 10<\/mml:mn>\n \n \u2212<\/mml:mo>\n 5<\/mml:mn>\n <\/mml:mrow>\n <\/mml:msup>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n )-differential privacy. Five heterogeneous clients\u2014two German-speech (EmoDB), two English-speech (RAVDESS), and one mixed\u2014collaborate under non-IID conditions (Dirichlet\n \n \n \n \n \n \u03b1<\/mml:mi>\n =<\/mml:mo>\n 0.5<\/mml:mn>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n ) without exchanging raw audio. Each client extracts multi-scale phase space reconstructions at micro (25\u2009ms), meso (250\u2009ms), and macro (2.5\u2009s) temporal resolutions alongside spectral and handcrafted features, which are fused through multi-head attention across the TCN-Transformer branches. On held-out, speaker-independent test sets the framework achieves 99.07%\u2009\u00b1\u20090.35% accuracy on EmoDB (107 samples) and 98.96%\u2009\u00b1\u20090.42% on RAVDESS (288 samples). Zero-shot cross-corpus evaluation on CREMA-D (1,488 samples) yields 68.15%\u2009\u00b1\u20091.23% overall, with a clear arousal-dependent pattern: high-arousal emotions (angry, happy, sad) transfer at 71.9% versus 62.1% for low-arousal categories (neutral, disgust, fear). Ablation experiments confirm that PSO selection (+2.80%), Transformer blocks (+2.10%), and the FedProx protocol (+2.62%) each contribute significantly, and a monotonic reduced-data curve rules out memorization. Membership inference attack resistance drops to near-chance levels (AUC\u2009\u2009=\u2009\u20090.52) under differential privacy while retaining 98.5% accuracy. A dual SHAP\u2013LIME explainability analysis reveals high inter-method agreement (\n r<\/jats:italic>\n \u2009=\u20090.997) and confirms that prosodic features\u2014particularly fundamental frequency statistics\u2014serve as language-invariant emotion indicators across all three corpora (\n r<\/jats:italic>\n \u2009=\u20090.94 cross-corpus consistency).\n <\/jats:p>","DOI":"10.1371\/journal.pone.0342953","type":"journal-article","created":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T17:44:54Z","timestamp":1778175894000},"page":"e0342953","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["FedEmoNet: Privacy-preserving federated learning with TCN-Transformer fusion for cross-corpus speech emotion recognition"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-7622-5991","authenticated-orcid":true,"given":"Mohammed","family":"Tawfik","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Razan Ali","family":"Obeidat","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Saddam","family":"Kamel","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Njood Anwer","family":"Aljarrah","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haneen Hussein","family":"Shehadeh","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ahmad","family":"Dalalah","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2026,5,7]]},"reference":[{"key":"pone.0342953.ref001","doi-asserted-by":"crossref","first-page":"784514","DOI":"10.3389\/fnbot.2021.784514","article-title":"Deep Cross-Corpus Speech Emotion Recognition: Recent Advances and Perspectives","volume":"15","author":"S Zhang","year":"2021","journal-title":"Front Neurorobot"},{"key":"pone.0342953.ref002","doi-asserted-by":"crossref","unstructured":"Li Z, Xing X, Fang Y, Zhang W, Fan H, Xu X. Multi-Scale Temporal Transformer for Speech Emotion Recognition. 2024. https:\/\/doi.org\/10.48550\/arXiv.2410.00390","DOI":"10.21437\/Interspeech.2023-1170"},{"key":"pone.0342953.ref003","doi-asserted-by":"crossref","unstructured":"Tang X, Lin Y, Dang T, Zhang Y, Cheng J. Speech emotion recognition via CNN-transformer and multidimensional attention mechanism. In: 2024. https:\/\/doi.org\/10.48550\/arXiv.2403.04743","DOI":"10.1016\/j.specom.2025.103242"},{"key":"pone.0342953.ref004","doi-asserted-by":"crossref","unstructured":"Ma Z, Chen M, Zhang H, Zheng Z, Chen W, Li X, et al. EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark. In: Interspeech 2024, 2024. 1580\u20134. https:\/\/doi.org\/10.21437\/interspeech.2024-788","DOI":"10.21437\/Interspeech.2024-788"},{"key":"pone.0342953.ref005","doi-asserted-by":"crossref","unstructured":"Li M, Zheng Y, Li D, Wu Y, Wang Y, Fei H. MS-SENet: Enhancing Speech Emotion Recognition Through Multi-Scale Feature Fusion with Squeeze-and-Excitation Blocks. In: Proc. IEEE ICASSP; 2024. 10111\u20135. https:\/\/doi.org\/10.1109\/ICASSP48485.2024.10447232","DOI":"10.1109\/ICASSP48485.2024.10447232"},{"key":"pone.0342953.ref006","doi-asserted-by":"crossref","first-page":"7046","DOI":"10.1007\/s10489-024-05536-5","article-title":"Unveiling hidden factors: explainable AI for feature boosting in speech emotion recognition","volume":"54","author":"A Nfissi","year":"2024","journal-title":"Appl Intell"},{"issue":"1","key":"pone.0342953.ref007","doi-asserted-by":"crossref","first-page":"26158","DOI":"10.1038\/s41598-025-08703-x","article-title":"Speech emotion recognition based on a stacked autoencoders optimized by PSO based grass fibrous root optimization","volume":"15","author":"C Zeng","year":"2025","journal-title":"Sci Rep"},{"key":"pone.0342953.ref008","doi-asserted-by":"crossref","first-page":"110905","DOI":"10.1016\/j.apacoust.2025.110905","article-title":"Multilingual speech emotion recognition using IGRFXG \u2013 Ensemble feature selection approach","volume":"240","author":"A Tripathi","year":"2025","journal-title":"Applied Acoustics"},{"key":"pone.0342953.ref009","doi-asserted-by":"crossref","unstructured":"Hjuler MJ, Clemmensen LH, Das S. Exploring local interpretable model-agnostic explanations for speech emotion recognition with distribution-shift. In: Proc. IEEE ICASSP, 2025. https:\/\/doi.org\/10.1109\/ICASSP49660.2025.10889825","DOI":"10.1109\/ICASSP49660.2025.10889825"},{"issue":"1","key":"pone.0342953.ref010","doi-asserted-by":"crossref","first-page":"43375","DOI":"10.1038\/s41598-025-28686-z","article-title":"FedSER-XAI: PSO-optimized multi-stream cross-attention transformer with graph features for explainable federated speech emotion recognition","volume":"15","author":"EA Alkhamali","year":"2025","journal-title":"Sci Rep"},{"key":"pone.0342953.ref011","doi-asserted-by":"crossref","unstructured":"Zhou W, Lu J, Xiong Z, Wang W. Leveraging TCN and Transformer for Effective Visual-Audio Fusion in Continuous Emotion Recognition. In: Proc. IEEE\/CVF CVPR Workshops, 2023. 5756\u201363. https:\/\/doi.org\/10.1109\/CVPRW59228.2023.00610","DOI":"10.1109\/CVPRW59228.2023.00610"},{"key":"pone.0342953.ref012","doi-asserted-by":"crossref","first-page":"108841","DOI":"10.1016\/j.compbiomed.2024.108841","article-title":"Cross-corpus speech emotion recognition with transformers: Leveraging handcrafted features and data augmentation","volume":"179","author":"R Alroobaea","year":"2024","journal-title":"Comput Biol Med"},{"key":"pone.0342953.ref013","doi-asserted-by":"crossref","first-page":"18237","DOI":"10.1109\/ACCESS.2024.3360483","article-title":"MaxMViT-MLP: Multiaxis and Multiscale Vision Transformers Fusion Network for Speech Emotion Recognition","volume":"12","author":"KL Ong","year":"2024","journal-title":"IEEE Access"},{"issue":"1","key":"pone.0342953.ref014","doi-asserted-by":"crossref","first-page":"4484","DOI":"10.1038\/s41598-024-52989-2","article-title":"Speech emotion recognition via graph-based representations","volume":"14","author":"A Pentari","year":"2024","journal-title":"Sci Rep"},{"key":"pone.0342953.ref015","doi-asserted-by":"crossref","unstructured":"Latif S, Khalifa S, Rana R, Jurdak R. Federated Learning for Speech Emotion Recognition Applications. In: Proc. ACM\/IEEE IPSN, 2020. 341\u20132. https:\/\/doi.org\/10.1109\/IPSN48710.2020.00-16","DOI":"10.1109\/IPSN48710.2020.00-16"},{"key":"pone.0342953.ref016","first-page":"101268","article-title":"A lightweight and privacy preserved federated learning ecosystem for analyzing verbal communication emotions in identical and non-identical databases","volume":"34","author":"M Chawla","year":"2024","journal-title":"Measurement: Sensors"},{"key":"pone.0342953.ref017","doi-asserted-by":"crossref","first-page":"106353","DOI":"10.1016\/j.bspc.2024.106353","article-title":"AFLEMP: Attention-based Federated Learning for Emotion recognition using Multi-modal Physiological data","volume":"94","author":"N Gahlan","year":"2024","journal-title":"Biomedical Signal Processing and Control"},{"issue":"3","key":"pone.0342953.ref018","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3650040","article-title":"FedCMD: A Federated Cross-modal Knowledge Distillation for Drivers\u2019 Emotion Recognition","volume":"15","author":"S Bano","year":"2024","journal-title":"ACM Trans Intell Syst Technol"},{"issue":"5","key":"pone.0342953.ref019","article-title":"A Deep Learning Model for Speech Emotion Recognition on RAVDESS Dataset","volume":"16","author":"Z Wei","year":"2025","journal-title":"ijacsa"},{"issue":"2","key":"pone.0342953.ref020","first-page":"213","article-title":"A novel multi-task and ensembled optimized parallel convolutional autoencoder and transformer for speech emotion recognition","volume":"56","author":"J Sharifzadeh","year":"2024","journal-title":"AUT Journal of Electrical Engineering"},{"issue":"1","key":"pone.0342953.ref021","doi-asserted-by":"crossref","first-page":"13126","DOI":"10.1038\/s41598-024-63776-4","article-title":"An enhanced speech emotion recognition using vision transformer","volume":"14","author":"S Akinpelu","year":"2024","journal-title":"Sci Rep"},{"issue":"1","key":"pone.0342953.ref022","first-page":"012056","article-title":"Speech Emotion Recognition Based on Swin-Transformer","volume":"2508","author":"Z Liao","year":"2023","journal-title":"Journal of Physics: Conference Series"},{"key":"pone.0342953.ref023","doi-asserted-by":"crossref","first-page":"101894","DOI":"10.1016\/j.bspc.2020.101894","article-title":"Speech emotion recognition with deep convolutional neural networks","volume":"59","author":"D Issa","year":"2020","journal-title":"Biomedical Signal Processing and Control"},{"issue":"17","key":"pone.0342953.ref024","doi-asserted-by":"crossref","first-page":"5797","DOI":"10.3390\/s24175797","article-title":"A combined CNN architecture for speech emotion recognition","volume":"24","author":"R Begazo","year":"2024","journal-title":"Sensors"},{"key":"pone.0342953.ref025","doi-asserted-by":"crossref","first-page":"1075624","DOI":"10.3389\/fpsyg.2022.1075624","article-title":"Speech emotion recognition based on improved masking EMD and convolutional recurrent neural network","volume":"13","author":"C Sun","year":"2023","journal-title":"Front Psychol"},{"key":"pone.0342953.ref026","doi-asserted-by":"crossref","first-page":"105708","DOI":"10.1016\/j.bspc.2023.105708","article-title":"Improvement of emotion classification performance using multi-resolution variational mode decomposition method","volume":"89","author":"SP Mishra","year":"2024","journal-title":"Biomedical Signal Processing and Control"},{"key":"pone.0342953.ref027","doi-asserted-by":"crossref","first-page":"110821","DOI":"10.1016\/j.apacoust.2025.110821","article-title":"Sub-band based analysis for speech emotion classification","volume":"239","author":"DP Pattnaik","year":"2025","journal-title":"Applied Acoustics"},{"issue":"5","key":"pone.0342953.ref028","doi-asserted-by":"crossref","first-page":"2598","DOI":"10.3390\/app15052598","article-title":"Enhancing Embedded Space with Low\u2013Level Features for Speech Emotion Recognition","volume":"15","author":"L Smietanka","year":"2025","journal-title":"Applied Sciences"},{"issue":"1","key":"pone.0342953.ref029","doi-asserted-by":"crossref","first-page":"21607","DOI":"10.1038\/s41598-025-94727-2","article-title":"MS-EmoBoost: a novel strategy for enhancing self-supervised speech emotion representations","volume":"15","author":"H Song","year":"2025","journal-title":"Sci Rep"},{"issue":"4","key":"pone.0342953.ref030","doi-asserted-by":"crossref","first-page":"1325","DOI":"10.3390\/app14041325","article-title":"Enhancing Emotion Recognition through Federated Learning: A Multimodal Approach with Convolutional Neural Networks","volume":"14","author":"N Simi\u0107","year":"2024","journal-title":"Applied Sciences"},{"key":"pone.0342953.ref031","doi-asserted-by":"crossref","unstructured":"Feng T, Narayanan S. Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labeling. In: Interspeech 2022, 2022. 5050\u20134. https:\/\/doi.org\/10.21437\/interspeech.2022-141","DOI":"10.21437\/Interspeech.2022-141"},{"key":"pone.0342953.ref032","doi-asserted-by":"crossref","unstructured":"Burkhardt F, Paeschke A, Rolfes M, Sendlmeier WF, Weiss B. A database of German emotional speech. In: Interspeech 2005, 2005. 1517\u201320. https:\/\/doi.org\/10.21437\/interspeech.2005-446","DOI":"10.21437\/Interspeech.2005-446"},{"issue":"5","key":"pone.0342953.ref033","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0196391","article-title":"The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English","volume":"13","author":"SR Livingstone","year":"2018","journal-title":"PLoS One"},{"issue":"4","key":"pone.0342953.ref034","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1109\/TAFFC.2014.2336244","article-title":"CREMA-D: Crowd-sourced Emotional Multimodal Actors Dataset","volume":"5","author":"H Cao","year":"2014","journal-title":"IEEE Transactions on Affective Computing"},{"issue":"9","key":"pone.0342953.ref035","doi-asserted-by":"crossref","first-page":"1492","DOI":"10.1002\/tee.24110","article-title":"EmoBone: A Multinational Audio Dataset of Emotional Bone Conducted Speech","volume":"19","author":"MdS Hosain","year":"2024","journal-title":"IEEJ Transactions Elec Engng"},{"key":"pone.0342953.ref036","unstructured":"Li T, Sahu AK, Zaheer M, Sanjabi M, Talwalkar A, Smith V. In: Proceedings of Machine Learning and Systems (MLSys), 2020. 429\u201350."},{"issue":"9","key":"pone.0342953.ref037","doi-asserted-by":"crossref","first-page":"361","DOI":"10.3390\/computers14090361","article-title":"Prosodic Spatio-Temporal Feature Fusion with Attention Mechanisms for Speech Emotion Recognition","volume":"14","author":"K Nugroho","year":"2025","journal-title":"Computers"},{"key":"pone.0342953.ref038","doi-asserted-by":"crossref","first-page":"96994","DOI":"10.1109\/ACCESS.2020.2991811","article-title":"Emotion Recognition From Speech Using Wavelet Packet Transform Cochlear Filter Bank and Random Forest Classifier","volume":"8","author":"S Hamsa","year":"2020","journal-title":"IEEE Access"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0342953","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T17:45:02Z","timestamp":1778175902000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0342953"}},"subtitle":[],"editor":[{"given":"Viacheslav","family":"Kovtun","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2026,5,7]]},"references-count":38,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,5,7]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0342953","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,7]]}}}