{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T16:39:02Z","timestamp":1787503142793,"version":"build-2736575974"},"reference-count":49,"publisher":"Public Library of Science (PLoS)","issue":"11","license":[{"start":{"date-parts":[[2023,11,29]],"date-time":"2023-11-29T00:00:00Z","timestamp":1701216000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Carleton College and National Institute on Deafness and Other Communication Disorders","award":["R15-DC018114"],"award-info":[{"award-number":["R15-DC018114"]}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>Among the most robust findings in speech research is that the presence of a talking face improves the intelligibility of spoken language. Talking faces supplement the auditory signal by providing fine phonetic cues based on the placement of the articulators, as well as temporal cues to when speech is occurring. In this study, we varied the amount of information contained in the visual signal, ranging from temporal information alone to a natural talking face. Participants were presented with spoken sentences in energetic or informational masking in four different visual conditions: audio-only, a modulating circle providing temporal cues to salient features of the speech, a digitally rendered point-light display showing lip movement, and a natural talking face. We assessed both sentence identification accuracy and self-reported listening effort. Audiovisual benefit for intelligibility was observed for the natural face in both informational and energetic masking, but the digitally rendered point-light display only provided benefit in energetic masking. Intelligibility for speech accompanied by the modulating circle did not differ from the audio-only conditions in either masker type. Thus, the temporal cues used here were insufficient to improve speech intelligibility in noise, but some types of digital point-light displays may contain enough phonetic detail to produce modest improvements in speech identification in noise.<\/jats:p>","DOI":"10.1371\/journal.pone.0290826","type":"journal-article","created":{"date-parts":[[2023,11,29]],"date-time":"2023-11-29T13:36:46Z","timestamp":1701265006000},"page":"e0290826","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["The effects of temporal cues, point-light displays, and faces on speech identification and listening effort"],"prefix":"10.1371","volume":"18","author":[{"given":"Katrina","family":"Sewell","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Violet A.","family":"Brown","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Grace","family":"Farwell","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-8747-334X","authenticated-orcid":true,"given":"Maya","family":"Rogers","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3763-8641","authenticated-orcid":true,"given":"Xingyi","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5950-0139","authenticated-orcid":true,"given":"Julia F.","family":"Strand","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2023,11,29]]},"reference":[{"key":"pone.0290826.ref001","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1044\/jshr.1202.423","article-title":"Interaction of audition and vision in the recognition of oral speech stimuli","volume":"12","author":"NP Erber","year":"1969","journal-title":"Journal of Speech and Hearing Research"},{"key":"pone.0290826.ref002","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1121\/1.1907309","article-title":"Visual contributions to speech intelligibility in noise","volume":"26","author":"WH Sumby","year":"1954","journal-title":"J Acoust Soc Am"},{"key":"pone.0290826.ref003","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1044\/jshd.4004.481","article-title":"Auditory-visual perception of speech.","volume":"40","author":"NP Erber","year":"1975","journal-title":"J Speech Hear Disord"},{"key":"pone.0290826.ref004","doi-asserted-by":"crossref","first-page":"2677","DOI":"10.1121\/1.422788","article-title":"Auditory-visual speech recognition by hearing-impaired subjects: Consonant recognition, sentence recognition, and auditory-visual integration","volume":"103","author":"KW Grant","year":"1998","journal-title":"J Acoust Soc Am"},{"key":"pone.0290826.ref005","doi-asserted-by":"crossref","first-page":"390","DOI":"10.1044\/1092-4388(2003\/032)","article-title":"Talker and lexical effects on audiovisual word recognition by adults with cochlear implants","volume":"46","author":"AR Kaiser","year":"2003","journal-title":"J Speech Lang Hear Res"},{"key":"pone.0290826.ref006","doi-asserted-by":"crossref","first-page":"786","DOI":"10.3109\/14992027.2011.599870","article-title":"Older adults expend more listening effort than young adults recognizing audiovisual speech in noise.","volume":"50","author":"PA Gosselin","year":"2011","journal-title":"Int J Audiol"},{"key":"pone.0290826.ref007","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1044\/1092-4388(2002\/046)","article-title":"Listening effort and fatigue in school-age children with and without hearing loss","volume":"45","author":"C Bourland-Hicks","year":"2002","journal-title":"J Speech Lang Hear Res"},{"key":"pone.0290826.ref008","doi-asserted-by":"crossref","first-page":"2438","DOI":"10.1121\/1.423751","article-title":"Measures of auditory\u2013visual integration in nonsense syllables and sentences","volume":"104","author":"KW Grant","year":"1998","journal-title":"Journal of the Acoustical Society of America"},{"key":"pone.0290826.ref009","doi-asserted-by":"crossref","first-page":"842","DOI":"10.1121\/1.1836832","article-title":"The role of visual speech cues in reducing energetic and informational masking","volume":"117","author":"KS Helfer","year":"2005","journal-title":"J Acoust Soc Am"},{"key":"pone.0290826.ref010","doi-asserted-by":"crossref","first-page":"1908","DOI":"10.1044\/JSLHR-H-13-0076","article-title":"Enhancing speech intelligibility: Interactions among context, modality, speech style, and masker","volume":"57","author":"KJ Van Engen","year":"2014","journal-title":"J Speech Lang Hear Res"},{"key":"pone.0290826.ref011","doi-asserted-by":"crossref","first-page":"396","DOI":"10.3758\/s13414-016-1238-9","article-title":"Audiovisual sentence recognition not predicted by susceptibility to the McGurk effect.","volume":"79","author":"KJ Van Engen","year":"2017","journal-title":"Atten Percept Psychophys"},{"key":"pone.0290826.ref012","first-page":"97","volume-title":"Hearing by eye: The psychology of lip-reading, (pp.","author":"D Reisberg","year":"1987"},{"key":"pone.0290826.ref013","article-title":"An examination of cues for visual and audio-visual speech perception using natural and computer-generated faces.","author":"M. McGrath","year":"1985","journal-title":"Ph D Thesis, Univ of Nottingham."},{"key":"pone.0290826.ref014","doi-asserted-by":"crossref","first-page":"1159","DOI":"10.1044\/jshr.3906.1159","article-title":"Point-light facial displays enhance comprehension of speech in noise","volume":"39","author":"LD Rosenblum","year":"1996","journal-title":"J Speech Hear Res"},{"key":"pone.0290826.ref015","doi-asserted-by":"crossref","first-page":"1197","DOI":"10.1121\/1.1288668","article-title":"The use of visible speech cues for improving auditory detection of spoken sentences","volume":"108","author":"KW Grant","year":"2000","journal-title":"J Acoust Soc Am"},{"key":"pone.0290826.ref016","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1037\/pag0000094","article-title":"Lipreading and audiovisual speech recognition across the adult lifespan: Implications for audiovisual integration.","volume":"31","author":"N Tye-Murray","year":"2016","journal-title":"Psychol Aging."},{"key":"pone.0290826.ref017","doi-asserted-by":"crossref","first-page":"2016","DOI":"10.3758\/s13414-022-02440-3","article-title":"Independent mechanisms of temporal and linguistic cue correspondence benefiting audiovisual speech processing.","volume":"84","author":"S Fiscella","year":"2022","journal-title":"Atten Percept Psychophys"},{"key":"pone.0290826.ref018","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.specom.2004.10.011","article-title":"Auditory speech detection in noise enhanced by lipreading.","volume":"44","author":"LE Bernstein","year":"2004","journal-title":"Speech Commun"},{"key":"pone.0290826.ref019","doi-asserted-by":"crossref","first-page":"650","DOI":"10.1097\/AUD.0b013e31821a4578","article-title":"Cross-modal enhancement of speech detection in young and older adults: does signal content matter?","volume":"32","author":"N Tye-Murray","year":"2011","journal-title":"Ear Hear"},{"key":"pone.0290826.ref020","doi-asserted-by":"crossref","first-page":"B69","DOI":"10.1016\/j.cognition.2004.01.006","article-title":"Seeing to hear better: evidence for early audio-visual interactions in speech identification.","volume":"93","author":"L Schwartz J-","year":"2004","journal-title":"Cognition"},{"key":"pone.0290826.ref021","article-title":"Talking points: A modulating circle increases listening effort without improving speech recognition in young adults.","author":"JF Strand","year":"2020","journal-title":"Psychon Bull Rev."},{"key":"pone.0290826.ref022","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1159\/000259969","article-title":"Use of visual information for phonetic perception","volume":"36","author":"Q. Summerfield","year":"1979","journal-title":"Phonetica"},{"key":"pone.0290826.ref023","doi-asserted-by":"crossref","first-page":"EL246","DOI":"10.1121\/10.0000737","article-title":"Visual analog of the acoustic amplitude envelope benefits speech perception in noise","volume":"147","author":"Y Yuan","year":"2020","journal-title":"J Acoust Soc Am"},{"key":"pone.0290826.ref024","doi-asserted-by":"crossref","first-page":"2845","DOI":"10.1044\/2021_JSLHR-20-00688","article-title":"Effects of Visual Speech Envelope on Audiovisual Speech Perception in Multitalker Listening Environments.","volume":"64","author":"Y Yuan","year":"2021","journal-title":"J Speech Lang Hear Res"},{"key":"pone.0290826.ref025","doi-asserted-by":"crossref","first-page":"1181","DOI":"10.1073\/pnas.0408949102","article-title":"Visual speech speeds up the neural processing of auditory speech","volume":"102","author":"V van Wassenhove","year":"2005","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pone.0290826.ref026","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1037\/0096-3445.109.2.160","article-title":"Attention and the detection of signals.","volume":"109","author":"MI Posner","year":"1980","journal-title":"J Exp Psychol."},{"key":"pone.0290826.ref027","doi-asserted-by":"crossref","first-page":"982","DOI":"10.1121\/1.1953167","article-title":"Combining energetic and informational masking for speech identification","volume":"118","author":"G Kidd","year":"2005","journal-title":"J Acoust Soc Am"},{"key":"pone.0290826.ref028","doi-asserted-by":"crossref","first-page":"5S","DOI":"10.1097\/AUD.0000000000000312","article-title":"Hearing impairment and cognitive energy: The Framework for Understanding Effortful Listening (FUEL).","volume":"371","author":"MK Pichora-Fuller","year":"2016","journal-title":"Ear Hear."},{"key":"pone.0290826.ref029","doi-asserted-by":"crossref","first-page":"433","DOI":"10.3109\/14992027.2014.890296","article-title":"Listening effort and fatigue: What exactly are we measuring? A British Society of Audiology Cognition in Hearing Special Interest Group \u201cwhite paper.\u201d","volume":"53","author":"R McGarrigle","year":"2014","journal-title":"Int J Audiol"},{"key":"pone.0290826.ref030","first-page":"2","article-title":"About face: Seeing the talker improves spoken word recognition but increases listening effort","author":"VA Brown","year":"2019","journal-title":"Journal of Cognition"},{"key":"pone.0290826.ref031","article-title":"Face mask type affects audiovisual speech intelligibility and subjective listening effort in young and older adults.","volume":"6","author":"VA Brown","year":"2021","journal-title":"Cognitive Research: Principles & Implications."},{"key":"pone.0290826.ref032","first-page":"1685","article-title":"Face Alignment at 3000 FPS via Regressing Local Binary Features","author":"S Ren","year":"2014","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"pone.0290826.ref033","article-title":"The OpenCV Library","author":"G. Bradski","year":"2000","journal-title":". Dr Dobb\u2019s J Softw Tools Prof Program"},{"key":"pone.0290826.ref034","doi-asserted-by":"crossref","first-page":"108","DOI":"10.3109\/03005367909078884","article-title":"The BKB (Bamford-Kowal-Bench) sentence lists for partially-hearing children.","volume":"13","author":"J Bench","year":"1979","journal-title":"Br J Audiol"},{"key":"pone.0290826.ref035","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1016\/j.specom.2010.05.002","article-title":"Similarity and familiarity: Second language sentence recognition in first- and second-language multi-talker babble.","volume":"52","author":"KJ Van Engen","year":"2010","journal-title":"Speech Commun."},{"key":"pone.0290826.ref036","author":"MB Winn","year":"2018","journal-title":"Praat script for creating speech-shaped noise [software] version 12."},{"key":"pone.0290826.ref037","doi-asserted-by":"crossref","first-page":"2064","DOI":"10.3758\/s13414-017-1361-2","article-title":"Headphone screening to facilitate web-based auditory experiments.","volume":"79","author":"KJP Woods","year":"2017","journal-title":"Atten Percept Psychophys"},{"key":"pone.0290826.ref038","doi-asserted-by":"crossref","first-page":"1781","DOI":"10.1044\/2015_JSLHR-H-14-0180","article-title":"Comparison of psychophysiological and dual-task measures of listening effort","volume":"58","author":"S Seeman","year":"2015","journal-title":"J Speech Lang Hear Res"},{"key":"pone.0290826.ref039","doi-asserted-by":"crossref","first-page":"1463","DOI":"10.1044\/2018_JSLHR-H-17-0257","article-title":"Measuring listening effort: Convergent validity, sensitivity, and links with cognitive and personality measures","volume":"61","author":"JF Strand","year":"2018","journal-title":"J Speech Lang Hear Res"},{"key":"pone.0290826.ref040","first-page":"1","article-title":"Fitting Linear Mixed-Effects Models Using lme4.","volume":"67","author":"D Bates","year":"2015","journal-title":"Journal of Statistical Software, Articles."},{"key":"pone.0290826.ref041","author":"R Core Team","year":"2022","journal-title":"R 4.2.1. R Foundation for Statistical Computing Vienna, Austria"},{"key":"pone.0290826.ref042","first-page":"1","article-title":"lmerTest Package: Tests in Linear Mixed Effects Models","volume":"82","author":"A Kuznetsova","year":"2017","journal-title":"Journal of Statistical Software, Articles"},{"key":"pone.0290826.ref043","doi-asserted-by":"crossref","first-page":"1686","DOI":"10.21105\/joss.01686","article-title":"Welcome to the Tidyverse.","volume":"4","author":"H Wickham","year":"2019","journal-title":"JOSS"},{"key":"pone.0290826.ref044","first-page":"68","article-title":"Random effects structure for confirmatory hypothesis testing: Keep it maximal","author":"DJ Barr","year":"2013","journal-title":"J Mem Lang"},{"key":"pone.0290826.ref045","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1177\/1084713807307409","article-title":"Auditory and visual lexical neighborhoods in audiovisual speech perception.","volume":"11","author":"N Tye-Murray","year":"2007","journal-title":"Trends Amplif."},{"key":"pone.0290826.ref046","doi-asserted-by":"crossref","first-page":"1101","DOI":"10.1121\/1.1345696","article-title":"Informational and energetic masking effects in the perception of two simultaneous talkers","volume":"109","author":"DS Brungart","year":"2001","journal-title":"J Acoust Soc Am"},{"key":"pone.0290826.ref047","doi-asserted-by":"crossref","first-page":"1725","DOI":"10.1121\/1.400247","article-title":"Effects of fluctuating noise and interfering speech on the speech-reception threshold for impaired and normal hearing","volume":"88","author":"JM Festen","year":"1990","journal-title":"The Journal of the Acoustical Society of America"},{"key":"pone.0290826.ref048","doi-asserted-by":"crossref","first-page":"1230","DOI":"10.1044\/1092-4388(2009\/08-0111)","article-title":"Objective measures of listening effort: Effects of background noise and noise reduction","volume":"52","author":"A Sarampalis","year":"2009","journal-title":"J Speech Lang Hear Res"},{"key":"pone.0290826.ref049","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1097\/AUD.0000000000000028","article-title":"The effect of hearing aid noise reduction on listening effort in hearing-impaired adults.","volume":"35","author":"JL Desjardins","year":"2014","journal-title":"Ear Hear"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0290826","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,29]],"date-time":"2023-11-29T13:37:14Z","timestamp":1701265034000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0290826"}},"subtitle":[],"editor":[{"given":"Andrew R.","family":"Dykstra","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2023,11,29]]},"references-count":49,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2023,11,29]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0290826","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,29]]}}}