{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T05:54:30Z","timestamp":1791266070520,"version":"4.1.0"},"reference-count":69,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T00:00:00Z","timestamp":1791244800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>Deepfakes have raised widespread concern owing to their threats to privacy, security, and societal trust, driving growing research interest in effective detection methods. Audio deepfake detection has moved from handcrafted-feature classifiers to end-to-end deep learning architectures and, more recently, to self-supervised speech foundation models and general-purpose speech recognizers such as Whisper. The use of instruction-following speech large language models (LLMs), of which Speech Audio Language Music Open Neural Network (SALMONN) is a representative example, remains comparatively less explored, and forms the setting of this study. We argue that large language models (LLMs), with their powerful representational and information retrieval capabilities, hold significant potential for this task\u2014provided that relevant audio features are carefully selected and prioritized. To this end, we employ a speech LLM, specifically SALMONN, for deepfake audio detection, incorporating a two-stage training strategy in which Stage 1 injects acoustic-feature awareness into the Low-Rank Adaptation (LoRA) adapters via an auxiliary feature-prediction pretext task (referred to as feature dropin), and Stage 2 performs windowed random retention on the encoder token sequence (referred to as feature dropout) before the final real\/fake decision. Our approach achieves an absolute accuracy improvement of 0.188 on the FakeOrReal benchmark and attains state-of-the-art performance on the Automatic Speaker Verification Spoofing and Countermeasures (ASVspoof) 2019 LA subset with an Equal Error Rate (EER) of 0.00496, representing a 33.0% relative improvement over prior methods. Furthermore, the model transfers non-trivially to an out-of-domain artificial intelligence (AI)-generated music dataset (M6), reaching accuracy comparable to a ResNet18 baseline reported in prior study, which we interpret as evidence of transferable audio representations rather than broad cross-domain robustness.<\/jats:p>","DOI":"10.3389\/frai.2026.1928592","type":"journal-article","created":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T05:21:16Z","timestamp":1791264076000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Defending against deepfake speech attacks via feature dropout and dropin regularization in large audio language models"],"prefix":"10.3389","volume":"9","author":[{"given":"Yupei","family":"Li","sequence":"first","affiliation":[{"name":"Group on Language, Audio, & Music (GLAM), Imperial College London","place":["London, United Kingdom"]},{"name":"Chair of Health Informatics (CHI), TUM University Hospital","place":["Munich, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yixiong","family":"Fang","sequence":"additional","affiliation":[{"name":"Shanghai Jiaotong University","place":["Shanghai, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiahao","family":"Xue","sequence":"additional","affiliation":[{"name":"Queen Mary University of London","place":["London, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manuel","family":"Milling","sequence":"additional","affiliation":[{"name":"Chair of Health Informatics (CHI), TUM University Hospital","place":["Munich, Germany"]},{"name":"Munich Center for Machine Learning (MCML)","place":["M\u00fcnchen, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bj\u00f6rn W.","family":"Schuller","sequence":"additional","affiliation":[{"name":"Group on Language, Audio, & Music (GLAM), Imperial College London","place":["London, United Kingdom"]},{"name":"Chair of Health Informatics (CHI), TUM University Hospital","place":["Munich, Germany"]},{"name":"Munich Center for Machine Learning (MCML)","place":["M\u00fcnchen, Germany"]},{"name":"Munich Data Science Institute (MDSI)","place":["M\u00fcnchen, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,10,6]]},"reference":[{"key":"B1","article-title":"Detecting music deepfakes is easy but actually hard","author":"Afchar","year":"2024","journal-title":"arXiv preprint arXiv:2405.04181"},{"key":"B2","doi-asserted-by":"publisher","first-page":"155","DOI":"10.3390\/a15050155","article-title":"A review of modern audio deepfake detection methods: challenges and future directions","volume":"15","author":"Almutairi","year":"2022","journal-title":"Algorithms"},{"key":"B3","article-title":"Exhubert: Enhancing hubert through block extension and fine-tuning on 37 emotion datasets","author":"Amiriparian","year":"2024","journal-title":"arXiv preprint arXiv:2406.10275"},{"key":"B4","article-title":"Audio spoofing verification using deep convolutional neural networks by transfer learning","author":"Aravind","year":"2020","journal-title":"arXiv preprint arXiv:2008.03464"},{"key":"B5","doi-asserted-by":"publisher","first-page":"162857","DOI":"10.1109\/ACCESS.2021.3133134","article-title":"Voice spoofing countermeasure for logical access attacks detection","volume":"9","author":"Arif","year":"2021","journal-title":"IEEE Access"},{"key":"B6","first-page":"12449","article-title":"\u201cwav2vec 2.0: a framework for self-supervised learning of speech representations,\u201d","author":"Baevski","year":"2020","journal-title":"Advances in neural information processing systems"},{"key":"B7","doi-asserted-by":"publisher","first-page":"130722","DOI":"10.1016\/j.eswa.2025.130722","article-title":"A multi-level and iterative feature engineering framework for deepfake audio detection","volume":"303","author":"Celik","year":"2026","journal-title":"Expert Syst. Appl"},{"key":"B8","doi-asserted-by":"publisher","first-page":"11820","DOI":"10.1038\/s41598-026-37590-z","article-title":"ETNeXt: Integrated feature engineering and classification framework for BLDC motor fault detection","volume":"16","author":"Celik","year":"","journal-title":"Sci. Rep"},{"key":"B9","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1016\/j.aej.2026.01.009","article-title":"VoxVeritasNet: a new feature engineering model leveraging iterative feature selection for detecting fake or real speech","volume":"136","author":"Celik","year":"","journal-title":"Alexandr. Eng. J"},{"key":"B10","article-title":"Toward robust real-world audio deepfake detection: closing the explainability gap","author":"Channing","year":"2024","journal-title":"arXiv preprint arXiv:2410.07436"},{"key":"B11","article-title":"Beats: audio pre-training with acoustic tokenizers","author":"Chen","year":"2022","journal-title":"arXiv preprint arXiv:2212.09058"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2024-698","article-title":"Rawbmamba: end-to-end bidirectional state space model for audio deepfake detection","author":"Chen","year":"2024","journal-title":"arXiv preprint arXiv:2406.06086"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1024","DOI":"10.1109\/JSTSP.2020.2999185","article-title":"Recurrent convolutional structures for audio spoof and video deepfake detection","volume":"14","author":"Chintha","year":"2020","journal-title":"IEEE J. Sel. Top. Signal Process"},{"key":"B14","article-title":"Qwen2-audio technical report","author":"Chu","year":"2024","journal-title":"arXiv preprint arXiv:2407.10759"},{"key":"B15","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1023\/A:1022627411411","article-title":"Support-vector networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn"},{"key":"B16","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1109\/TASSP.1980.1163420","article-title":"\u201cComparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,\u201d","author":"Davis","year":"1980","journal-title":"IEEE transactions on acoustics, speech, and signal processing"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2020-2650","article-title":"Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification","author":"Desplanques","year":"2020","journal-title":"arXiv preprint arXiv:2005.07143"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1145\/3658644.3690311","article-title":"\u201cTrident of poseidon: A generalized approach for detecting deepfake voices,\u201d","author":"Doan","year":"2024","journal-title":"Proceedings of the 2024 on ACM SIGSAC conference on computer and communications security"},{"key":"B19","article-title":"Adaptive re-calibration of channel-wise features for adversarial audio classification","author":"Dongre","year":"2022","journal-title":"arXiv preprint arXiv:2210.11722"},{"key":"B20","article-title":"An image is worth 16x16 words: transformers for image recognition at scale","author":"Dosovitskiy","year":"2021","journal-title":"arXiv preprint arXiv:2010.11929"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.52202\/085713-4991","article-title":"From uncertainty to trust: enhancing reliability in vision-language models with uncertainty-guided dropout decoding","author":"Fang","year":"2024","journal-title":"arXiv preprint arXiv:2412.06474"},{"key":"B22","article-title":"Listen, think, and understand","author":"Gong","year":"2023","journal-title":"arXiv preprint arXiv:2305.10790"},{"key":"B23","first-page":"1319","article-title":"\u201cMaxout networks,\u201d","volume-title":"International conference on machine learning","author":"Goodfellow","year":"2013"},{"key":"B24","doi-asserted-by":"publisher","first-page":"134018","DOI":"10.1109\/ACCESS.2022.3231480","article-title":"Deepfake audio detection via mfcc features using machine learning","volume":"10","author":"Hamza","year":"2022","journal-title":"IEEE Access"},{"key":"B25","article-title":"Lora: Low-rank adaptation of large language models","author":"Hu","year":"2021","journal-title":"arXiv preprint arXiv:2106.09685"},{"key":"B26","doi-asserted-by":"publisher","first-page":"1265","DOI":"10.1109\/LSP.2021.3089437","article-title":"Towards end-to-end synthetic speech detection","volume":"28","author":"Hua","year":"2021","journal-title":"IEEE Signal Process. Lett"},{"key":"B27","article-title":"Text clustering as classification with llms","author":"Huang","year":"2024","journal-title":"arXiv preprint arXiv:2410.00927"},{"key":"B28","article-title":"Self-attention and hybrid features for replay and deep-fake audio detection","author":"Huang","year":"2024","journal-title":"arXiv preprint arXiv:2401.05614"},{"key":"B29","doi-asserted-by":"publisher","first-page":"103149","DOI":"10.1016\/j.specom.2024.103149","article-title":"AFP-Conformer: asymptotic feature pyramid conformer for spoofing speech detection","volume":"166","author":"Huang","year":"2025","journal-title":"Speech Commun"},{"key":"B30","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1007\/978-3-031-25066-8_14","article-title":"\u201cFrequency dropout: feature-level regularization via randomized filtering,\u201d","author":"Islam","year":"2022","journal-title":"European conference on computer vision"},{"key":"B31","doi-asserted-by":"publisher","first-page":"6367","DOI":"10.1109\/ICASSP43922.2022.9747766","article-title":"\u201cAASIST: audio anti-spoofing using integrated spectro-temporal graph attention networks,\u201d","author":"Jung","year":"2022","journal-title":"ICASSP 2022-IEEE international conference on acoustics, speech and signal processing"},{"key":"B32","article-title":"How deep are the fakes? focusing on audio deepfake: a survey","author":"Khanjani","year":"2021","journal-title":"arXiv preprint arXiv:2111.14203"},{"key":"B33","article-title":"Source tracing of audio deepfake systems","author":"Klein","year":"2024","journal-title":"arXiv preprint arXiv:2407.08016"},{"key":"B34","article-title":"Assert: anti-spoofing with squeeze-excitation and residual networks","author":"Lai","year":"2019","journal-title":"arXiv preprint arXiv:1904.01120"},{"key":"B35","first-page":"19730","article-title":"\u201cBlip-2: bootstrapping language-image pre-training with frozen image encoders and large language models,\u201d","volume-title":"International conference on machine learning","author":"Li","year":"2023"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3714458","article-title":"A survey on speech deepfake detection","volume":"57","author":"Li","year":"2025","journal-title":"ACM Comput. Surv"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1145\/3658644.3670285","article-title":"\u201cSafeear: content privacy-preserving audio deepfake detection,\u201d","author":"Li","year":"2024","journal-title":"Proceedings of the 2024 on ACM SIGSAC conference on computer and communications security"},{"key":"B38","doi-asserted-by":"publisher","first-page":"9237","DOI":"10.1038\/s41598-026-36044-w","article-title":"M6: multi-generator, multi-domain, multi-lingual and cultural, multi-genres, multi-instrument machine-generated music detection databases","volume":"16","author":"Li","year":"2026","journal-title":"Sci. Rep"},{"key":"B39","article-title":"Neuroplasticity in artificial intelligence-an overview and inspirations on drop in &out learning","author":"Li","year":"","journal-title":"arXiv preprint arXiv:2503.21419"},{"key":"B40","article-title":"From audio deepfake detection to ai-generated music detection-a pathway and overview","author":"Li","year":"","journal-title":"arXiv preprint arXiv:2412.00571"},{"key":"B41","article-title":"Detecting machine-generated music with explainability-a challenge and early benchmarks","author":"Li","year":"","journal-title":"arXiv e-prints, arXiv-2412"},{"key":"B42","article-title":"Dfallm: achieving generalizable multitask deepfake detection by optimizing audio llm components","author":"Li","year":"","journal-title":"arXiv preprint arXiv:2512.08403"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1145\/3589335.3641299","article-title":"\u201cInformation retrieval meets large language models,\u201d","author":"Liu","year":"2024","journal-title":"Companion proceedings of the ACM on web conference 2024"},{"key":"B44","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1049\/icp.2025.2329","article-title":"Speech anti-spoofing method based on graph attention mechanism and adversarial training","volume":"41","author":"Lu","year":"2025","journal-title":"J. Signal Proc"},{"key":"B45","article-title":"Continuous learning of transformer-based audio deepfake detection","author":"Nguyen Le","year":"2024","journal-title":"arXiv e-prints, pages arXiv-2409"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2024-1472","article-title":"\u201cAttentive merging of hidden embeddings from pre-trained speech model for anti-spoofing detection,\u201d","author":"Pan","year":"2024","journal-title":"Interspeech 2024"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1109\/IS262782.2024.10704095","article-title":"\u201cDeepfake audio detection using spectrogram-based feature and ensemble of deep learning models,\u201d","author":"Pham","year":"2024","journal-title":"2024 IEEE 5th international symposium on the internet of sounds (IS2)"},{"key":"B48","article-title":"DIN-CTS: low-complexity depthwise-inception neural network with contrastive training strategy for deepfake speech detection","author":"Pham","year":"2025","journal-title":"arXiv preprint arXiv:2502.20225"},{"key":"B49","doi-asserted-by":"publisher","first-page":"735","DOI":"10.1109\/ESIC60604.2024.10481602","article-title":"\u201cDetection of ai-generated text using large language model,\u201d","author":"Prajapati","year":"2024","journal-title":"2024 international conference on emerging systems and intelligent computing (ESIC)"},{"key":"B50","volume-title":"Fundamentals of Speech Recognition","author":"Rabiner","year":"1993"},{"key":"B51","first-page":"28492","article-title":"\u201cRobust speech recognition via large-scale weak supervision,\u201d","volume-title":"International conference on machine learning","author":"Radford","year":"2022"},{"key":"B52","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/IJCB54206.2022.10007949","article-title":"\u201cStatnet: spectral and temporal features based multi-task network for audio spoofing detection,\u201d","author":"Ranjan","year":"2022","journal-title":"2022 IEEE international joint conference on biometrics (IJCB)"},{"key":"B53","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/SPED.2019.8906599","article-title":"\u201cFor: a dataset for synthetic speech detection,\u201d","author":"Reimao","year":"2019","journal-title":"2019 international conference on speech technology and human-computer dialogue (SpeD)"},{"key":"B54","doi-asserted-by":"publisher","author":"Sahidullah","year":"2015","DOI":"10.21437\/Interspeech.2015-472"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.515","article-title":"\u201cGradual dropin of layers to train very deep neural networks,\u201d","author":"Smith","year":"2016","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)"},{"key":"B56","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res"},{"key":"B57","article-title":"Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation","author":"Tak","year":"2022","journal-title":"arXiv preprint arXiv:2202.12233"},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.21437\/ASVSPOOF.2021-1","article-title":"\u201cEnd-to-end spectro-temporal graph attention networks for speaker verification anti-spoofing and speech deepfake detection,\u201d","author":"Tak","year":"2021","journal-title":"Proceedings of the 2021 edition of the automatic speaker verification and spoofing countermeasures challenge"},{"key":"B59","article-title":"A survey on neural speech synthesis","author":"Tan","year":"2021","journal-title":"arXiv preprint arXiv:2106.15561"},{"key":"B60","article-title":"Salmonn: towards generic hearing abilities for large language models","author":"Tang","year":"2023","journal-title":"arXiv preprint arXiv:2310.13289"},{"key":"B61","article-title":"Llama: Open and efficient foundation language models","author":"Touvron","year":"2023","journal-title":"arXiv preprint arXiv:2407.21783"},{"key":"B62","first-page":"2523","article-title":"Measuring spectral centroid: a comparison of estimation methods","volume":"116","author":"Tschacher","year":"2004","journal-title":"J. Acoust. Soc. Am"},{"key":"B63","first-page":"1058","article-title":"\u201cRegularization of neural networks using dropconnect,\u201d","volume-title":"International conference on machine learning","author":"Wan","year":"2013"},{"key":"B64","first-page":"118","article-title":"\u201cFast dropout training,\u201d","volume-title":"International conference on machine learning","author":"Wang","year":"2013"},{"key":"B65","doi-asserted-by":"publisher","first-page":"101114","DOI":"10.1016\/j.csl.2020.101114","article-title":"Asvspoof 2019: a large-scale public database of synthesized, converted and replayed speech","volume":"64","author":"Wang","year":"2020","journal-title":"Comput. Speech Lang"},{"key":"B66","doi-asserted-by":"publisher","first-page":"99894","DOI":"10.1109\/ACCESS.2024.3421281","article-title":"Toward improving synthetic audio spoofing detection robustness via meta-learning and disentangled training with adversarial examples","volume":"12","author":"Wang","year":"2024","journal-title":"IEEE Access"},{"key":"B67","article-title":"Llama pro: Progressive llama with block expansion","author":"Wu","year":"2024","journal-title":"arXiv preprint arXiv:2401.02415"},{"key":"B68","article-title":"Enhancing transformer training efficiency with dynamic dropout","author":"Yan","year":"2024","journal-title":"arXiv preprint arXiv:2411.03236"},{"key":"B69","doi-asserted-by":"publisher","DOI":"10.52202\/079017-2168","article-title":"\u201cSlim: style-linguistics mismatch model for generalized audio deepfake detection,\u201d","author":"Zhu","year":"2024","journal-title":"Advances in neural information processing systems"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1928592\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T05:21:21Z","timestamp":1791264081000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1928592\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,6]]},"references-count":69,"alternative-id":["10.3389\/frai.2026.1928592"],"URL":"https:\/\/doi.org\/10.3389\/frai.2026.1928592","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,6]]},"article-number":"1928592"}}