{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,3]],"date-time":"2026-09-03T17:07:04Z","timestamp":1788455224803,"version":"build-2803163510"},"reference-count":69,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,4,9]],"date-time":"2025-04-09T00:00:00Z","timestamp":1744156800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Hum. Neurosci."],"abstract":"<jats:p>The audio-visual benefit in speech perception\u2014where congruent visual input enhances auditory processing\u2014is well-documented across age groups, particularly in challenging listening conditions and among individuals with varying hearing abilities. However, most studies rely on highly controlled laboratory environments with scripted stimuli. Here, we examine the audio-visual benefit using unscripted, natural speech from untrained speakers within a virtual acoustic environment. Using electroencephalography (EEG) and cortical speech tracking, we assessed neural responses across audio-visual, audio-only, visual-only, and masked-lip conditions to isolate the role of lip movements. Additionally, we analysed individual differences in acoustic and visual features of the speakers, including pitch, jitter, and lip-openness, to explore their influence on the audio-visual speech tracking benefit. Results showed a significant audio-visual enhancement in speech tracking with background noise, with the masked-lip condition performing similarly to the audio-only condition, emphasizing the importance of lip movements in adverse listening situations. Our findings reveal the feasibility of cortical speech tracking with naturalistic stimuli and underscore the impact of individual speaker characteristics on audio-visual integration in real-world listening contexts.<\/jats:p>","DOI":"10.3389\/fnhum.2025.1560558","type":"journal-article","created":{"date-parts":[[2025,4,9]],"date-time":"2025-04-09T05:23:31Z","timestamp":1744176211000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Neural speech tracking in a virtual acoustic environment: audio-visual benefit for unscripted continuous speech"],"prefix":"10.3389","volume":"19","author":[{"given":"Mareike","family":"Daeglau","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"J\u00fcrgen","family":"Otten","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Giso","family":"Grimm","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bojana","family":"Mirkovic","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Volker","family":"Hohmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefan","family":"Debener","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,4,9]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"390","DOI":"10.1016\/j.tics.2012.05.003","article-title":"Cortical oscillations and sensory predictions","volume":"16","author":"Arnal","year":"2012","journal-title":"Trends Cogn. Sci."},{"key":"ref2","doi-asserted-by":"publisher","first-page":"738408","DOI":"10.3389\/fnins.2021.738408","article-title":"Neural measures of pitch processing in EEG responses to running speech","volume":"15","author":"Bachmann","year":"2021","journal-title":"Front. Neurosci."},{"key":"ref3","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1016\/j.neuroimage.2015.09.020","article-title":"Functional changes in inter- and intra-hemispheric cortical processing underlying degraded speech perception","volume":"124","author":"Bidelman","year":"2016","journal-title":"NeuroImage"},{"key":"ref4","author":"Boersma","year":"2024"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"828546","DOI":"10.3389\/fnins.2022.828546","article-title":"Cortical tracking of voice pitch in the presence of multiple speakers depends on selective attention","volume":"16","author":"Brodbeck","year":"2022","journal-title":"Front. Neurosci."},{"key":"ref6","doi-asserted-by":"publisher","first-page":"1001","DOI":"10.1098\/rstb.2007.2155","article-title":"The processing of audio-visual speech: empirical and neural bases","volume":"363","author":"Campbell","year":"2007","journal-title":"Philos. Trans. Soc B Biol. Sci."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"e1000436","DOI":"10.1371\/journal.pcbi.1000436","article-title":"The natural statistics of audiovisual speech","volume":"5","author":"Chandrasekaran","year":"2009","journal-title":"PLoS Comput. Biol."},{"key":"ref9004","doi-asserted-by":"publisher","DOI":"10.4324\/9780203771587","article-title":"Statistical Power Analysis for the Behavioral Sciences","author":"Cohen","year":"2013","journal-title":"(2nd ed.)Routledge."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"14195","DOI":"10.1523\/JNEUROSCI.1829-15.2015","article-title":"Congruent visual speech enhances cortical entrainment to continuous auditory speech in noise-free conditions","volume":"35","author":"Crosse","year":"2015","journal-title":"J. Neurosci."},{"key":"ref9","doi-asserted-by":"publisher","first-page":"604","DOI":"10.3389\/fnhum.2016.00604","article-title":"The multivariate temporal response function (mTRF) toolbox: a MATLAB toolbox for relating neural signals to continuous stimuli","volume":"10","author":"Crosse","year":"","journal-title":"Front. Hum. Neurosci."},{"key":"ref10","doi-asserted-by":"publisher","first-page":"9888","DOI":"10.1523\/JNEUROSCI.1396-16.2016","article-title":"Eye can hear clearly now: inverse effectiveness in natural audiovisual speech processing relies on long-term Crossmodal temporal integration","volume":"36","author":"Crosse","year":"","journal-title":"J. Neurosci."},{"key":"ref11","volume-title":"Audiovisual recordings of unscripted monologues [video recording]","author":"Daeglau","year":"2023"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.jneumeth.2003.10.009","article-title":"EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis","volume":"134","author":"Delorme","year":"2004","journal-title":"J. Neurosci. Methods"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"311","DOI":"10.3389\/fnhum.2014.00311","article-title":"Cortical entrainment to continuous speech: functional roles and interpretations","volume":"8","author":"Ding","year":"2014","journal-title":"Front. Hum. Neurosci."},{"key":"ref14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jvoice.2004.02.002","article-title":"Classification of dysphonic voice: acoustic and auditory-perceptual measures","volume":"19","author":"Eadie","year":"2005","journal-title":"J. Voice"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"5750","DOI":"10.1523\/JNEUROSCI.1828-18.2019","article-title":"Neural speech tracking in the Theta and in the Delta frequency band differentially encode clarity and comprehension of speech in noise","volume":"39","author":"Etard","year":"2019","journal-title":"J. Neurosci."},{"key":"ref16","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1108\/13522750610640549","article-title":"Identifying likeable attributes: a qualitative study of television advertisements in Asia","volume":"9","author":"Fam","year":"2006","journal-title":"Qual. Mark. Res. Int. J."},{"key":"ref17","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1007\/s10919-008-0054-x","article-title":"Attaining status at the expense of likeability: pilfering Power through conversational interruption","volume":"32","author":"Farley","year":"2008","journal-title":"J. Nonverbal Behav."},{"key":"ref9001","doi-asserted-by":"publisher","first-page":"175","DOI":"10.3758\/BF03193146","article-title":"G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences","volume":"39","author":"Faul","year":"2007","journal-title":"Behavior Research Methods"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"066033","DOI":"10.1088\/1741-2552\/ab4340","article-title":"Congruent audiovisual speech enhances auditory attention decoding with EEG","volume":"16","author":"Fu","year":"2019","journal-title":"J. Neural Eng."},{"key":"ref19","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1109\/MSP.2021.3075932","article-title":"Electroencephalography-based auditory attention decoding: toward Neurosteered hearing devices","volume":"38","author":"Geirnaert","year":"2021","journal-title":"IEEE Signal Process. Mag."},{"key":"ref20","doi-asserted-by":"publisher","first-page":"566","DOI":"10.3813\/AAA.919337","article-title":"A toolbox for rendering virtual acoustic environments in the context of audiology","volume":"105","author":"Grimm","year":"2019","journal-title":"Acta Acust. United Acust"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"e1001752","DOI":"10.1371\/journal.pbio.1001752","article-title":"Speech rhythms and multiplexed oscillatory sensory coding in the human brain","volume":"11","author":"Gross","year":"2013","journal-title":"PLoS Biol."},{"key":"ref22","first-page":"498","article-title":"Improved numerical methods for Gammatone Filterbank analysis and synthesis","volume":"93","author":"Herzke","year":"2007","journal-title":"Acta Acust United Acust"},{"key":"ref23","first-page":"433","article-title":"Frequency analysis and synthesis using a Gammatone filterbank","volume":"88","author":"Hohmann","year":"2002","journal-title":"Acta Acustica United with Acustica"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"31S","DOI":"10.1097\/AUD.0000000000000945","article-title":"The virtual reality lab: realization and application of virtual sound environments","volume":"41","author":"Hohmann","year":"2020","journal-title":"Ear Hear."},{"key":"ref25","doi-asserted-by":"publisher","first-page":"721","DOI":"10.3389\/fpsyg.2020.00721","article-title":"The \u2018real-world approach\u2019 and its problems: a critique of the term ecological validity","volume":"11","author":"Holleman","year":"2020","journal-title":"Front. Psychol."},{"key":"ref26","first-page":"65","article-title":"A simple sequentially Rejective multiple test procedure","volume":"6","author":"Holm","year":"1979","journal-title":"Scand. J. Stat."},{"key":"ref27","doi-asserted-by":"publisher","first-page":"177","DOI":"10.5709\/acp-0387-6","article-title":"Eye-blink patterns reflect attention to continuous speech","volume":"19","author":"Holtze","year":"2023","journal-title":"Adv. Cogn. Psychol."},{"key":"ref28","doi-asserted-by":"publisher","first-page":"603","DOI":"10.3389\/fnins.2020.00603","article-title":"Decoding the attended speaker from EEG using adaptive evaluation intervals captures fluctuations in attentional listening","volume":"14","author":"Jaeger","year":"2020","journal-title":"Front. Neurosci."},{"key":"ref29","doi-asserted-by":"publisher","first-page":"1193","DOI":"10.1037\/a0023100","article-title":"Psychophysics of the McGurk and other audiovisual speech integration effects","volume":"37","author":"Jiang","year":"2011","journal-title":"J. Exp. Psychol. Hum. Percept. Perform."},{"key":"ref30","doi-asserted-by":"publisher","first-page":"5S","DOI":"10.1097\/AUD.0000000000000944","article-title":"The quest for ecological validity in hearing science: what it is, why it matters, and how to advance it","volume":"41","author":"Keidser","year":"2020","journal-title":"Ear Hear."},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1101\/2024.02.13.580071","article-title":"The lab streaming layer for synchronized multimodal recording","author":"Kothe","year":"2024","journal-title":"Biorxiv"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"11080","DOI":"10.1093\/cercor\/bhad347","article-title":"EEG-based speaker\u2013listener neural coupling reflects speech-selective attentional mechanisms beyond the speech stimulus","volume":"33","author":"Li","year":"2023","journal-title":"Cereb. Cortex"},{"key":"ref9002","doi-asserted-by":"publisher","first-page":"342","DOI":"10.1093\/joc\/jqad003","article-title":"Conversational dynamics of joint attention and shared emotion predict outcomes in interpersonal influence situations: An interaction ritual perspective","volume":"73","author":"Liao","year":"2023","journal-title":"Journal of Communication"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"1001","DOI":"10.1016\/j.neuron.2007.06.004","article-title":"Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex","volume":"54","author":"Luo","year":"2007","journal-title":"Neuron"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"1299","DOI":"10.3758\/s13423-015-0817-4","article-title":"Variability and stability in the McGurk effect: contributions of participants, stimuli, time, and response type","volume":"22","author":"Mallick","year":"2015","journal-title":"Psychon. Bull. Rev."},{"key":"ref35","doi-asserted-by":"publisher","first-page":"746","DOI":"10.1038\/264746a0","article-title":"Hearing lips and seeing voices","volume":"264","author":"Mcgurk","year":"1976","journal-title":"Nature"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"349","DOI":"10.3389\/fnins.2016.00349","article-title":"Target speaker detection with concealed EEG around the ear","volume":"10","author":"Mirkovic","year":"2016","journal-title":"Front. Neurosci."},{"key":"ref37","doi-asserted-by":"publisher","first-page":"046007","DOI":"10.1088\/1741-2560\/12\/4\/046007","article-title":"Decoding the attended speech stream with multi-channel EEG: implications for online, daily-life applications","volume":"12","author":"Mirkovic","year":"2015","journal-title":"J. Neural Eng."},{"key":"ref38","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1111\/j.0963-7214.2004.01502010.x","article-title":"Visual prosody and speech intelligibility: head movement improves auditory speech perception","volume":"15","author":"Munhall","year":"2004","journal-title":"Psychol. Sci."},{"key":"ref39","first-page":"2800","article-title":"The effects of attention and visual input on the representation of natural speech in EEG","author":"O\u2019Sullivan","year":"2013"},{"key":"ref40","doi-asserted-by":"publisher","first-page":"2105","DOI":"10.1016\/j.neuron.2023.04.004","article-title":"Vowel and formant representation in the human auditory speech cortex","volume":"111","author":"Oganian","year":"2023","journal-title":"Neuron"},{"key":"ref41","doi-asserted-by":"publisher","first-page":"e14521","DOI":"10.7554\/eLife.14521","article-title":"Lip movements entrain the observers\u2019 low-frequency brain oscillations to facilitate speech intelligibility","volume":"5","author":"Park","year":"2016","journal-title":"Elife"},{"key":"ref42","doi-asserted-by":"publisher","first-page":"320","DOI":"10.3389\/fpsyg.2012.00320","article-title":"Neural oscillations carry speech rhythm through to comprehension","volume":"3","author":"Peelle","year":"2012","journal-title":"Front. Psychol."},{"key":"ref43","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1016\/j.cortex.2015.03.006","article-title":"Prediction and constraint in audiovisual speech perception","volume":"68","author":"Peelle","year":"2015","journal-title":"Cortex"},{"key":"ref44","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1016\/j.neuroimage.2019.05.026","article-title":"ICLabel: an automated electroencephalographic independent component classifier, dataset, and website","volume":"198","author":"Pion-Tonachini","year":"2019","journal-title":"NeuroImage"},{"key":"ref45","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1016\/j.neuroimage.2019.04.017","article-title":"Hearing-impaired listeners show increased audiovisual benefit when listening to speech in noise","volume":"196","author":"Puschmann","year":"2019","journal-title":"NeuroImage"},{"key":"ref46","doi-asserted-by":"publisher","first-page":"11505","DOI":"10.1523\/JNEUROSCI.1007-17.2017","article-title":"The right Temporoparietal junction supports speech tracking during selective listening: evidence from concurrent EEG-fMRI","volume":"37","author":"Puschmann","year":"2017","journal-title":"J. Neurosci."},{"key":"ref47","doi-asserted-by":"publisher","first-page":"e0253874","DOI":"10.1371\/journal.pone.0253874","article-title":"Influence of surgical and N95 face masks on speech perception and listening effort in noise","volume":"16","author":"Rahne","year":"2021","journal-title":"PLoS One"},{"key":"ref48","doi-asserted-by":"publisher","first-page":"685774","DOI":"10.3389\/fnins.2021.685774","article-title":"EEG-based Intersubject correlations reflect selective attention in a competing speaker scenario","volume":"15","author":"Rosenkranz","year":"2021","journal-title":"Front. Neurosci."},{"key":"ref49","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1080\/14992027.2020.1851401","article-title":"Impacts of face coverings on communication: an indirect impact of COVID-19","volume":"60","author":"Saunders","year":"2021","journal-title":"Int. J. Audiol."},{"key":"ref50","doi-asserted-by":"publisher","first-page":"508","DOI":"10.3389\/fpsyg.2019.00508","article-title":"Dynamic facial expression of emotion and observer inference","volume":"10","author":"Scherer","year":"2019","journal-title":"Front. Psychol."},{"key":"ref9003","doi-asserted-by":"publisher","first-page":"1137","DOI":"10.1093\/scan\/nsu168","article-title":"Engaged listeners: Shared neural processing of powerful political speeches","volume":"10","author":"Schm\u00e4lzle","year":"2015","journal-title":"Social Cognitive and Affective Neuroscience"},{"key":"ref51","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1016\/j.tics.2008.01.002","article-title":"Neuronal oscillations and visual amplification of speech","volume":"12","author":"Schroeder","year":"2008","journal-title":"Trends Cogn. Sci."},{"key":"ref52","doi-asserted-by":"publisher","first-page":"3081","DOI":"10.1044\/2017_JSLHR-S-16-0130","article-title":"Acoustics of clear and noise-adapted speech in children, young, and older adults","volume":"60","author":"Smiljanic","year":"2017","journal-title":"J. Speech Lang. Hear. Research: JSLHR"},{"key":"ref53","doi-asserted-by":"publisher","first-page":"282","DOI":"10.1097\/MAO.0000000000003458","article-title":"How face masks interfere with speech understanding of Normal-hearing individuals: vision makes the difference","volume":"43","author":"S\u00f6nnichsen","year":"2022","journal-title":"Otol. Neurotol."},{"key":"ref54","doi-asserted-by":"publisher","first-page":"514","DOI":"10.1016\/j.nicl.2017.09.001","article-title":"Auditory cross-modal reorganization in cochlear implant users indicates audio-visual integration","volume":"16","author":"Stropahl","year":"2017","journal-title":"NeuroImage"},{"key":"ref55","doi-asserted-by":"publisher","first-page":"863","DOI":"10.3758\/s13423-016-1148-9","article-title":"McGurk stimuli for the investigation of multisensory integration in cochlear implant users: the Oldenburg audio visual speech stimuli (OLAVS)","volume":"24","author":"Stropahl","year":"2017","journal-title":"Psychon. Bull. Rev."},{"key":"ref56","doi-asserted-by":"publisher","first-page":"212","DOI":"10.1121\/1.1907309","article-title":"Visual contribution to speech intelligibility in noise","volume":"26","author":"Sumby","year":"2005","journal-title":"J. Acoust. Soc. Am."},{"key":"ref57","doi-asserted-by":"publisher","first-page":"1397","DOI":"10.1007\/s41870-023-01697-7","article-title":"Fusing facial and speech cues for enhanced multimodal emotion recognition","volume":"16","author":"Tomar","year":"2024","journal-title":"Int. J. Inf. Technol."},{"key":"ref58","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1007\/s10162-018-0654-z","article-title":"Speech intelligibility predicted from neural entrainment of the speech envelope","volume":"19","author":"Vanthornhout","year":"2018","journal-title":"J. Assoc. Res. Otolaryngol."},{"key":"ref59","doi-asserted-by":"publisher","first-page":"10972","DOI":"10.1093\/cercor\/bhad325","article-title":"EEG-based auditory attention decoding with audiovisual speech for hearing-impaired listeners","volume":"33","author":"Wang","year":"2023","journal-title":"Cereb. Cortex"},{"key":"ref60","first-page":"1439","volume-title":"The influence of likeability ratings of audio-visual stimuli on cortical speech tracking with Mobile EEG in virtual environments","author":"Wiedenmann","year":"2023"},{"key":"ref61","doi-asserted-by":"publisher","first-page":"682677","DOI":"10.3389\/fpsyg.2021.682677","article-title":"Effects of wearing face masks while using different speaking styles in noise on speech intelligibility during the COVID-19 pandemic","volume":"12","author":"Yi","year":"2021","journal-title":"Front. Psychol."},{"key":"ref62","doi-asserted-by":"publisher","first-page":"785677","DOI":"10.3389\/fpsyg.2021.785677","article-title":"Likeability and expert persuasion: Dislikeability reduces the perceived persuasiveness of expert evidence","volume":"12","author":"Younan","year":"2021","journal-title":"Front. Psychol."},{"key":"ref63","doi-asserted-by":"publisher","first-page":"980","DOI":"10.1016\/j.neuron.2012.12.037","article-title":"Mechanisms underlying selective neuronal tracking of attended speech at a \u2018cocktail party\u2019","volume":"77","author":"Zion Golumbic","year":"2013","journal-title":"Neuron"},{"key":"ref64","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1016\/j.bandl.2011.12.010","article-title":"Temporal context in speech processing and attentional stream selection: a behavioral and neural perspective","volume":"122","author":"Zion Golumbic","year":"2012","journal-title":"Brain Lang."},{"key":"ref65","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1007\/BF00990791","article-title":"What sounds beautiful is good: the vocal attractiveness stereotype","volume":"13","author":"Zuckerman","year":"1989","journal-title":"J. Nonverbal Behav."}],"container-title":["Frontiers in Human Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnhum.2025.1560558\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,9]],"date-time":"2025-04-09T05:23:33Z","timestamp":1744176213000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnhum.2025.1560558\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,9]]},"references-count":69,"alternative-id":["10.3389\/fnhum.2025.1560558"],"URL":"https:\/\/doi.org\/10.3389\/fnhum.2025.1560558","relation":{},"ISSN":["1662-5161"],"issn-type":[{"value":"1662-5161","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,9]]},"article-number":"1560558"}}