{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T22:24:31Z","timestamp":1784154271786,"version":"3.55.0"},"reference-count":58,"publisher":"Public Library of Science (PLoS)","issue":"3","license":[{"start":{"date-parts":[[2024,3,29]],"date-time":"2024-03-29T00:00:00Z","timestamp":1711670400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>To examine visual speech perception (i.e., lip-reading), we created a multi-layer network (the AV-net) that contained: (1) an auditory layer with nodes representing phonological word-forms and edges connecting words that were phonologically related, and (2) a visual layer with nodes representing the viseme representations of words and edges connecting viseme representations that differed by a single viseme (and additional edges to connect related nodes in the two layers). The results of several computer simulations (in which activation diffused across the network to simulate word identification) are reported and compared to the performance of human participants who identified the same words in a condition in which audio and visual information were both presented (Simulation 1), in an audio-only presentation condition (Simulation 2), and a visual-only presentation condition (Simulation 3). Another simulation (Simulation 4) examined the influence of phonological information on visual speech perception by comparing performance in the multi-layer AV-net to a single-layer network that contained only a visual layer with nodes representing the viseme representations of words and edges connecting viseme representations that differed by a single viseme. We also report the results of several analyses of the errors made by human participants in the visual-only presentation condition. The results of our analyses have implications for future research and training of lip-reading, and for the development of automatic lip-reading devices and software for individuals with certain developmental or acquired disorders or for listeners with normal hearing in noisy conditions.<\/jats:p>","DOI":"10.1371\/journal.pone.0300926","type":"journal-article","created":{"date-parts":[[2024,3,29]],"date-time":"2024-03-29T17:29:51Z","timestamp":1711733391000},"page":"e0300926","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["Using network science to examine audio-visual speech perception with a multi-layer graph"],"prefix":"10.1371","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1209-0838","authenticated-orcid":true,"given":"Michael S.","family":"Vitevitch","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lorin","family":"Lachs","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2024,3,29]]},"reference":[{"key":"pone.0300926.ref001","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1038\/nphys2188","article-title":"The network takeover","volume":"8","author":"AL Barab\u00e1si","year":"2012","journal-title":"Nat Phys"},{"key":"pone.0300926.ref002","doi-asserted-by":"crossref","DOI":"10.4324\/9780367853259","volume-title":"Network Science in Cognitive Psychology","author":"MS Vitevitch","year":"2019"},{"key":"pone.0300926.ref003","author":"GA Carpenter","year":"2016"},{"issue":"4","key":"pone.0300926.ref004","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1017\/S0140525X00003356","article-title":"Connectionist modelling in psychology: A localist manifesto","volume":"23","author":"M. Page","year":"2000","journal-title":"Behavioral and Brain Sciences"},{"key":"pone.0300926.ref005","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1111\/cogs.12148","article-title":"Parallel Distributed Processing at 25: Further Explorations in the Microstructure of Cognition","volume":"38","author":"TT Rogers","year":"2014","journal-title":"Cogn Sci"},{"key":"pone.0300926.ref006","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1207\/s15516709cog2901_3","article-title":"The large-scale structure of semantic networks: Statistical analyses and a model of semantic growth","volume":"29","author":"M Steyvers","year":"2005","journal-title":"Cognitive Sci"},{"key":"pone.0300926.ref007","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1044\/1092-4388(2008\/030)","article-title":"What can graph theory tell us about word learning and lexical retrieval?","volume":"51","author":"MS Vitevitch","year":"2008","journal-title":"J Speech Lang Hear R"},{"key":"pone.0300926.ref008","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1037\/xge0000575","article-title":"The phonographic language network: Using network science to investigate the phonological and orthographic similarity structure of language","volume":"148","author":"CSQ Siew","year":"2019","journal-title":"J Exp Psychol Gen"},{"issue":"1","key":"pone.0300926.ref009","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1177\/00238309221095455","article-title":"Using network science and psycholinguistic megastudies to examine the dimensions of phonological similarity","volume":"66","author":"N Castro","year":"2023","journal-title":"Lang Speech"},{"key":"pone.0300926.ref010","doi-asserted-by":"crossref","first-page":"1211","DOI":"10.1037\/0278-7393.18.6.1211","article-title":"Form-based priming in spoken word recognition: The roles of competition and bias","volume":"18","author":"SD Goldinger","year":"1992","journal-title":"J Exp Psychol Learn"},{"key":"pone.0300926.ref011","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1097\/00003446-199802000-00001","article-title":"Recognizing spoken words: The neighborhood activation model","volume":"19","author":"PA Luce","year":"1998","journal-title":"Ear Hearing"},{"key":"pone.0300926.ref012","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1111\/tops.12532","article-title":"What can network science tell us about phonology and language processing?","volume":"14","author":"MS Vitevitch","year":"2022","journal-title":"Top Cogn Sci"},{"issue":"5","key":"pone.0300926.ref013","doi-asserted-by":"crossref","first-page":"E19348","DOI":"10.1371\/journal.pone.0019348","article-title":"Small worlds and semantic network growth in typical and late talkers","volume":"6","author":"N Beckage","year":"2011","journal-title":"PLOS One"},{"issue":"9","key":"pone.0300926.ref014","doi-asserted-by":"crossref","first-page":"2275","DOI":"10.1044\/2018_JSLHR-L-18-0036","article-title":"An Application of Network Science to Phonological Sequence Learning in Children With Developmental Language Disorder","volume":"61","author":"S Benham","year":"2018","journal-title":"J Speech Lang Hear R"},{"key":"pone.0300926.ref015","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.jml.2014.05.005","article-title":"How children explore the phonological network in child-directed speech: A survival analysis of children\u2019s first word productions","volume":"75","author":"MT Carlson","year":"2014","journal-title":"J Mem Lang"},{"key":"pone.0300926.ref016","doi-asserted-by":"crossref","first-page":"1911","DOI":"10.1044\/2017_JSLHR-S-16-0298","article-title":"Using network science measures to predict lexical decision performance of adults who stutter","volume":"60","author":"N Castro","year":"2017","journal-title":"J Speech Lang Hear R"},{"key":"pone.0300926.ref017","doi-asserted-by":"crossref","first-page":"13","DOI":"10.3109\/17549507.2014.987819","article-title":"Using network science in the language sciences and clinic","volume":"17","author":"MS Vitevitch","year":"2015","journal-title":"INT J SPEECH-LANG PA"},{"key":"pone.0300926.ref018","doi-asserted-by":"crossref","first-page":"2259","DOI":"10.1038\/s41598-018-20730-5","article-title":"Multiplex model of mental lexicon reveals explosive learning in humans","volume":"8","author":"M Stella","year":"2018","journal-title":"Sci Rep-UK"},{"key":"pone.0300926.ref019","doi-asserted-by":"crossref","first-page":"e12881","DOI":"10.1111\/cogs.12881","article-title":"Quantifying the interplay of semantics and phonology during failures of word retrieval by people with aphasia using a multiplex lexical network","volume":"44","author":"N Castro","year":"2020","journal-title":"Cognitive Sci"},{"issue":"4","key":"pone.0300926.ref020","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1044\/jshr.1104.796","article-title":"Confusions among visually perceived consonants","volume":"11","author":"CG Fisher","year":"1968","journal-title":"J Speech Hear Res"},{"issue":"6","key":"pone.0300926.ref021","doi-asserted-by":"crossref","first-page":"3704","DOI":"10.1121\/1.420402","article-title":"Speechreading and the structure of the lexicon: Computationally modeling the effects of reduced phonetic distinctiveness on lexical uniqueness","volume":"102","author":"ET Auer","year":"1997","journal-title":"J Acoust Soc Am"},{"key":"pone.0300926.ref022","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1121\/1.1907309","article-title":"Visual contribution to speech intelligibility in noise","volume":"26","author":"WH Sumby","year":"1954","journal-title":"J Acoust Soc Am"},{"key":"pone.0300926.ref023","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1348\/000712601162220","article-title":"Bisensory augmentation: A speechreading advantage when speech is clearly audible and intact","volume":"92","author":"P Arnold","year":"2001","journal-title":"Brit J Psychol"},{"issue":"5","key":"pone.0300926.ref024","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1111\/j.1467-9450.2009.00751.x","article-title":"Spoken word recognition by eye","volume":"50","author":"ET Auer","year":"2009","journal-title":"Scand J Psychol"},{"key":"pone.0300926.ref025","doi-asserted-by":"crossref","first-page":"341","DOI":"10.3758\/BF03196291","article-title":"The influence of the lexicon on speech read word recognition: Contrasting segmental and lexical distinctiveness","volume":"9","author":"ET Auer","year":"2002","journal-title":"Psychon B Rev"},{"key":"pone.0300926.ref026","doi-asserted-by":"crossref","first-page":"218","DOI":"10.3758\/BF03193744","article-title":"Discriminating languages by speech-reading","volume":"69","author":"S Soto-Faraco","year":"2007","journal-title":"Percept Psychophys"},{"key":"pone.0300926.ref027","unstructured":"https:\/\/www.who.int\/news-room\/fact-sheets\/detail\/ageing-and-health."},{"key":"pone.0300926.ref028","article-title":"EXPRESS: Examining listeners\u2019 perception of spoken words with different face masks","author":"B Cox","year":"2023","journal-title":"Q J Exp Psychol"},{"key":"pone.0300926.ref029","unstructured":"https:\/\/www.consumerreports.org\/disability-rights\/auto-captions-often-fall-short-on-zoom-facebook-and-others-a9742392879\/."},{"key":"pone.0300926.ref030","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1038\/35065725","article-title":"Exploring complex networks","volume":"410","author":"SH Strogatz","year":"2001","journal-title":"Nature"},{"key":"pone.0300926.ref031","unstructured":"Lachs L, Hernandez LR. Update: The Hoosier Audiovisual Multi-Talker Database. In Research on Spoken Language Processing Progress Report No. 22 (pp. 377\u2013388). Bloomington, IN: Speech Research laboratory, Indiana University. (1998)."},{"key":"pone.0300926.ref032","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1177\/00238309020450040501","article-title":"Naturalistic and experimental analyses of word frequency and neighborhood density effects in slips of the ear","volume":"45","author":"MS Vitevitch","year":"2002","journal-title":"Lang Speech"},{"key":"pone.0300926.ref033","first-page":"357","article-title":"Sizing up the Hoosier Mental Lexicon: Measuring the familiarity of 20,000 words","volume":"10","author":"HC Nusbaum","year":"1984","journal-title":"Research on Speech Perception Progress Report"},{"issue":"3","key":"pone.0300926.ref034","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1044\/jshd.1703.321","article-title":"Development of materials for speech audiometry","volume":"17","author":"IJ Hirsh","year":"1952","journal-title":"J Speech Hear Disord"},{"issue":"1","key":"pone.0300926.ref035","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1097\/AUD.0000000000001395","article-title":"Using complex networks in the hearing sciences","volume":"45","author":"MS Vitevitch","year":"2024","journal-title":"Ear Hearing"},{"issue":"1","key":"pone.0300926.ref036","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1044\/1059-0889(2013\/13-0009)","article-title":"Identification of acoustically similar and dissimilar vowels in profoundly deaf adults who use hearing aids and\/or cochlear implants: some preliminary findings","volume":"23","author":"MJ Hay-McCutcheon","year":"2014","journal-title":"Am J Audiol"},{"key":"pone.0300926.ref037","doi-asserted-by":"crossref","first-page":"481","DOI":"10.3758\/BF03195594","article-title":"A web-based interface to calculate phonotactic probability for words and nonwords in English","volume":"36","author":"MS Vitevitch","year":"2004","journal-title":"Behav Res Methods Ins C"},{"issue":"1\u20132","key":"pone.0300926.ref038","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/S0167-6393(98)00049-1","article-title":"Modeling the interaction of phonemic intelligibility and lexical structure in audiovisual word recognition","volume":"26","author":"P Iverson","year":"1998","journal-title":"Speech Commun"},{"key":"pone.0300926.ref039","doi-asserted-by":"crossref","first-page":"910","DOI":"10.3758\/s13428-018-1186-5","article-title":"Spreadr: An R package to simulate spreading activation in a network","volume":"51","author":"CSQ Siew","year":"2019","journal-title":"Behav Res Methods"},{"key":"pone.0300926.ref040","first-page":"369","article-title":"Simulating retrieval from a highly clustered network: Implications for spoken word recognition","volume":"2","author":"MS Vitevitch","year":"2011","journal-title":"Front. Lang. Sci"},{"issue":"2","key":"pone.0300926.ref041","doi-asserted-by":"crossref","first-page":"188","DOI":"10.3390\/brainsci13020188","article-title":"The Resilience of the Phonological Network May Have Implications for Developmental and Acquired Disorders","volume":"13","author":"MS Vitevitch","year":"2023","journal-title":"Brain Sciences"},{"issue":"3","key":"pone.0300926.ref042","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1037\/0033-295X.93.3.283","article-title":"A spreading-activation theory of retrieval in sentence production","volume":"93","author":"GS Dell","year":"1986","journal-title":"Psychol Rev"},{"key":"pone.0300926.ref043","unstructured":"JASP Team (2022). JASP (Version 0.16.3) [Computer software]."},{"key":"pone.0300926.ref044","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1016\/j.physa.2017.10.048","article-title":"Are crossing dependencies really scarce?","volume":"493","author":"R Ferrer-i-Cancho","year":"2018","journal-title":"Physica A"},{"key":"pone.0300926.ref045","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1038\/35022643","article-title":"Navigation in a small world","volume":"406","author":"J.M. Kleinberg","year":"2000","journal-title":"Nature"},{"key":"pone.0300926.ref046","doi-asserted-by":"crossref","first-page":"198701","DOI":"10.1103\/PhysRevLett.87.198701","article-title":"Efficient behavior of small-world networks","volume":"87","author":"V Latora","year":"2001","journal-title":"Phys. Rev. Lett"},{"issue":"7","key":"pone.0300926.ref047","doi-asserted-by":"crossref","first-page":"526","DOI":"10.3390\/e20070526","article-title":"Universal Features in Phonological Neighbor Networks","volume":"20","author":"KS Brown","year":"2018","journal-title":"Entropy"},{"key":"pone.0300926.ref048","first-page":"596","article-title":"The lexical restructuring hypothesis and graph theoretical analyses of networks based on random lexicons. J. Speech Lang","volume":"52","author":"T.M. Gruenenfelder","year":"2009","journal-title":"Hear. Res"},{"issue":"5","key":"pone.0300926.ref049","doi-asserted-by":"crossref","first-page":"P05006","DOI":"10.1088\/1742-5468\/2015\/05\/P05006","article-title":"Patterns in the English language: phonological networks, percolation and assembly models","volume":"2015","author":"M Stella","year":"2015","journal-title":"J Stat Mech-Theory E"},{"key":"pone.0300926.ref050","doi-asserted-by":"crossref","first-page":"22313","DOI":"10.1103\/PhysRevE.95.022313","article-title":"Modeling structure and resilience of the dark network","volume":"95","author":"M De Domenico","year":"2017","journal-title":"Phys. Rev. E"},{"key":"pone.0300926.ref051","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1177\/0023830908099070","article-title":"Word length and lexical competition: Longer is the same as shorter","volume":"51","author":"MS Vitevitch","year":"2008","journal-title":"Lang Speech"},{"issue":"4","key":"pone.0300926.ref052","doi-asserted-by":"crossref","first-page":"2074","DOI":"10.1121\/1.427952","article-title":"Recognition of spoken words by native and non-native listeners: Talker-, listener-, and item-related factors","volume":"106","author":"AR Bradlow","year":"1999","journal-title":"J Acoust Soc Am"},{"issue":"5","key":"pone.0300926.ref053","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1097\/00003446-199510000-00004","article-title":"Lexical effects on spoken word recognition by pediatric cochlear implant users","volume":"16","author":"KI Kirk","year":"1995","journal-title":"Ear Hearing"},{"issue":"3","key":"pone.0300926.ref054","doi-asserted-by":"crossref","first-page":"1242","DOI":"10.1121\/1.408177","article-title":"Training Japanese listeners to identify English \/r\/ and \/l\/: II. The role of phonetic environment and talker variability in new perceptual categories","volume":"94","author":"S Lively","year":"1993","journal-title":"J Acoust Soc Am"},{"issue":"4","key":"pone.0300926.ref055","doi-asserted-by":"crossref","first-page":"667","DOI":"10.3758\/BF03194734","article-title":"Stimulus-based lexical distinctiveness as a general word-recognition mechanism","volume":"64","author":"S Mattys","year":"2002","journal-title":"Percept Psychophys"},{"key":"pone.0300926.ref056","doi-asserted-by":"crossref","first-page":"1474","DOI":"10.1038\/s41598-022-27029-6","article-title":"Feature-rich multiplex lexical networks reveal mental strategies of early language learning","volume":"13","author":"S Citraro","year":"2023","journal-title":"Sci Rep-UK"},{"key":"pone.0300926.ref057","doi-asserted-by":"crossref","first-page":"58","DOI":"10.21832\/9781783092338-006","volume-title":"Motor Speech Disorders A Cross-Language Perspective","author":"MS Vitevitch","year":"2014"},{"issue":"2","key":"pone.0300926.ref058","doi-asserted-by":"crossref","first-page":"1367","DOI":"10.1121\/10.0009584","article-title":"Using acoustic distance and acoustic absement to quantify lexical competition","volume":"151","author":"MC Kelley","year":"2022","journal-title":"J Acoust Soc Am"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0300926","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,29]],"date-time":"2024-03-29T17:30:25Z","timestamp":1711733425000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0300926"}},"subtitle":[],"editor":[{"given":"Jie","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2024,3,29]]},"references-count":58,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3,29]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0300926","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,29]]}}}