{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,19]],"date-time":"2026-08-19T05:24:50Z","timestamp":1787117090074,"version":"build-2736575974"},"reference-count":46,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,8,19]],"date-time":"2026-08-19T00:00:00Z","timestamp":1787097600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010661","name":"Horizon 2020 Framework Programme","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Psychol."],"abstract":"\n Introduction<\/jats:title>\n Previous research has shown that speech and gesture are integrated during comprehension, with differences in response times and electrophysiological activity observed between incongruent and congruent gesture\u2014speech pairings. Beyond these incongruency effects, it remains unclear whether comprehenders use gestures to adapt or extend conventional interpretations for novel communicative purposes.<\/jats:p>\n <\/jats:sec>\n \n Method<\/jats:title>\n \n The current study examined the bidirectional influence of speech and gesture by employing a novel levels-of-congruence paradigm, in which the same metaphor (e.g.,\n Men are oaks<\/jats:italic>\n ) was paired with a gesture depicting either its preferred interpretation (e.g., men are robust) or a less preferred but possible interpretation (e.g., men are tall). In Experiment 1, participants rated the meaningfulness of preferred and possible interpretations across three gesture conditions (gesture-of-preferred, gesture-of-possible, non-gestured). In Experiment 2, participants selected a single interpretation under the same conditions. In Experiment 3, participants viewed clips of an actor producing the same gesture and utterance combination (e.g., flexing both arms while saying\n Men are oaks<\/jats:italic>\n ) in one of three multimodal conditions: no additional cues (muted, blurred lips), only lip movements (muted, visible lips), or audible speech. Then they rated how well the gesture matched the gestured interpretation (e.g.,\n robust<\/jats:italic>\n ), an alternative interpretation consistent with the speech but not the gesture (e.g.,\n tall<\/jats:italic>\n ), and an unrelated interpretation (e.g.,\n hikers<\/jats:italic>\n ).\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n In Experiments 1 and 2, participants consistently rated the preferred interpretation as the most meaningful and selected it most often across all gesture conditions. However, gestures depicting the possible interpretation significantly increased both the meaningfulness ratings and the likelihood of being selected of the possible interpretation. In Experiment 3, the presence of speech cues significantly increased ratings for the alternative interpretation compared with the condition without additional cues.<\/jats:p>\n <\/jats:sec>\n \n Discussion<\/jats:title>\n These findings demonstrate bidirectional interactions between speech and gesture during metaphor comprehension. Gestures constrained metaphor interpretation, whereas speech broadened the range of interpretations attributed to gestures. Overall, the results support the integrated-systems hypothesis by showing that speech and gesture jointly contribute to the construction of meaning.<\/jats:p>\n <\/jats:sec>","DOI":"10.3389\/fpsyg.2026.1851588","type":"journal-article","created":{"date-parts":[[2026,8,19]],"date-time":"2026-08-19T04:35:02Z","timestamp":1787114102000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Bidirectional effects of speech and gesture: evidence from a novel levels-of-congruence paradigm"],"prefix":"10.3389","volume":"17","author":[{"given":"Laura","family":"Pissani","sequence":"first","affiliation":[{"name":"Department of Language Science and Technology, Saarland University","place":["Saarbr\u00fccken, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vera","family":"Demberg","sequence":"additional","affiliation":[{"name":"Department of Language Science and Technology, Saarland University","place":["Saarbr\u00fccken, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,8,19]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1515\/sem-2012-0067","article-title":"Do metaphoric gestures influence how a message is perceived? The effects of metaphoric gesture-speech matches and mismatches on semantic communication and social judgment","volume":"2012","author":"Beattie","year":"2012","journal-title":"Semiotica"},{"key":"B2","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1207\/s15327868ms1204_4","article-title":"Reading and recall of metaphorical sentences: effects of familiarity and context","volume":"12","author":"Blasko","year":"1997","journal-title":"Metaphor Symbol"},{"key":"B3","first-page":"658","article-title":"\u201cTime course of metaphor processing: effects of subjective familiarity and aptness,\u201d","volume":"13","author":"Blasko","year":"1991","journal-title":"Proceedings of the Annual Meeting of the Cognitive Science Society"},{"key":"B4","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1037\/0278-7393.19.2.295","article-title":"Effects of familiarity and aptness on metaphor processing","volume":"19","author":"Blasko","year":"1993","journal-title":"J. Experim. Psychol.: Learn. Memory Cognit"},{"key":"B5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v080.i01","article-title":"BRMS: an R package for Bayesian multilevel models using Stan","volume":"80","author":"B\u00fcrkner","year":"2017","journal-title":"J. Statist. Softw"},{"key":"B6","doi-asserted-by":"publisher","first-page":"395","DOI":"10.32614\/RJ-2018-017","article-title":"Advanced Bayesian multilevel modeling with the R package BRMS","volume":"10","author":"B\u00fcrkner","year":"2018","journal-title":"R J"},{"key":"B7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v100.i05","article-title":"Bayesian item response modeling in R with brms and Stan","volume":"100","author":"B\u00fcrkner","year":"2021","journal-title":"J. Statist. Softw"},{"key":"B8","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.bandc.2008.12.005","article-title":"Gesture and metaphor comprehension: electrophysiological evidence of cross-modal coordination by audiovisual stimulation","volume":"70","author":"Cornejo","year":"2009","journal-title":"Brain Cognit"},{"key":"B9","doi-asserted-by":"publisher","first-page":"49","DOI":"10.5406\/amerjpsyc.129.1.0049","article-title":"Familiarity and aptness in metaphor comprehension","volume":"129","author":"Damerall","year":"2016","journal-title":"Am. J. Psychol"},{"key":"B10","doi-asserted-by":"publisher","first-page":"1935","DOI":"10.3389\/fpsyg.2020.01935","article-title":"What's new? gestures accompany inferable rather than brand-new referents in discourse","volume":"11","author":"Debreslioska","year":"2020","journal-title":"Front. Psychol"},{"key":"B11","doi-asserted-by":"crossref","unstructured":"Guo\n X. H.\n \n \n Goldin-Meadow\n S.\n \n \n Bainbridge\n W. A.\n \n \n What Makes Co-speech Gestures Memorable\n \n 2024","DOI":"10.31234\/osf.io\/7shx2"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1845","DOI":"10.1162\/jocn.2010.21462","article-title":"The role of synchrony and ambiguity in speech-gesture integration during comprehension","volume":"23","author":"Habets","year":"2011","journal-title":"J. Cognit. Neurosci"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1175","DOI":"10.1162\/jocn.2007.19.7.1175","article-title":"The role of iconic gestures in speech disambiguation: Erp evidence","volume":"19","author":"Holle","year":"2007","journal-title":"Journal of cognitive neuroscience"},{"key":"B14","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1007\/s10919-008-0063-9","article-title":"Do iconic hand gestures really contribute to the communication of semantic information in a face-to-face context?","volume":"33","author":"Holler","year":"2009","journal-title":"J. Nonverbal Behav"},{"key":"B15","author":"Huang","year":"2025","journal-title":"Dissecting the Hierarchy of Gesture Comprehension: Evidence from a Multilevel Priming Paradigm and Drift-Diffusion Model"},{"key":"B16","doi-asserted-by":"publisher","first-page":"110","DOI":"10.1016\/j.jml.2017.05.004","article-title":"Mechanisms underlying the beneficial effect of a speaker's gestures on the listener","volume":"96","author":"Iani","year":"2017","journal-title":"J. Memory Lang"},{"key":"B17","doi-asserted-by":"publisher","first-page":"634074","DOI":"10.3389\/fpsyg.2021.634074","article-title":"The role of iconic gestures in speech comprehension: an overview of various methodologies","volume":"12","author":"Kananda Arachchige","year":"2021","journal-title":"Front. Psychol"},{"key":"B18","doi-asserted-by":"publisher","first-page":"825","DOI":"10.1007\/s10956-012-9433-5","article-title":"The different benefits from different gestures in understanding a concept","volume":"2013","author":"Kang","year":"2013","journal-title":"J. Sci. Educ. Technol"},{"key":"B19","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1186\/s41235-016-0004-9","article-title":"From hands to minds: gestures promote understanding","volume":"1","author":"Kang","year":"2016","journal-title":"Cognit. Res.: Princ. Implicat"},{"key":"B20","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1006\/jmla.1999.2634","article-title":"Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory","volume":"40","author":"Kelly","year":"1999","journal-title":"J. Memory Lang"},{"key":"B21","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1162\/jocn.2009.21254","article-title":"Integrating speech and iconic gestures in a stroop-like task: evidence for automatic processing","volume":"22","author":"Kelly","year":"","journal-title":"J. Cognit. Neurosci"},{"key":"B22","doi-asserted-by":"publisher","first-page":"517","DOI":"10.3758\/s13423-014-0681-7","article-title":"The processing of speech, gesture, and action during language comprehension","volume":"22","author":"Kelly","year":"2015","journal-title":"Psychon. Bullet. Rev"},{"key":"B23","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1177\/0956797609357327","article-title":"Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension","volume":"21","author":"Kelly","year":"","journal-title":"Psychol. Sci"},{"key":"B24","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1016\/S0093-934X(03)00335-3","article-title":"Neural correlates of bimodal speech and gesture comprehension","volume":"89","author":"Kelly","year":"2004","journal-title":"Brain Lang"},{"key":"B25","doi-asserted-by":"publisher","first-page":"239","DOI":"10.3390\/bs12070239","article-title":"A study of the use of iconic and metaphoric gestures with motion-based, static space-based, static object-based, and static event-based statements","volume":"12","author":"Khatin-Zadeh","year":"2022","journal-title":"Behav. Sci"},{"key":"B26","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1016\/S0749-596X(02)00505-3","article-title":"What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking","volume":"48","author":"Kita","year":"2003","journal-title":"J. Memory Lang"},{"key":"B27","volume-title":"Hand and mind: What gestures reveal about thought","author":"McNeill","year":"1992"},{"key":"B28","doi-asserted-by":"crossref","DOI":"10.7208\/chicago\/9780226514642.001.0001","volume-title":"Gesture and Thought","author":"McNeill","year":"2005"},{"key":"B29","author":"Meghdadi","year":"2025","journal-title":"Integrating Language Model Embeddings into the ACT-R Cognitive Modeling Framework"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1301.3781","article-title":"Efficient estimation of word representations in vector space","author":"Mikolov","year":"","journal-title":"arXiv"},{"key":"B31","article-title":"\u201cDistributed representations of words and phrases and their compositionality,\u201d","author":"Mikolov","year":"","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B32","doi-asserted-by":"publisher","first-page":"916","DOI":"10.1037\/xge0001867","article-title":"Distinct temporal dynamics of speech and gesture processing: Insights from event-related potentials across L1 and L2","volume":"155","author":"\u00d6zer","year":"2026","journal-title":"J. Exp. Psychol. Gen."},{"key":"B33","unstructured":"Pissani\n L.\n \n \n Repository of Gestures and Multimodal Cues\n \n 2026"},{"key":"B34","doi-asserted-by":"publisher","first-page":"253","DOI":"10.3758\/s13423-021-01985-y","article-title":"Can you mend a broken heart? Awakening conventional metaphors in the maze","volume":"29","author":"Pissani","year":"2022","journal-title":"Psychon. Bullet. Rev"},{"key":"B35","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1080\/10926488.2023.2225561","article-title":"Early birds can fly: awakening the literal meaning of conventional metaphors further downstream","volume":"38","author":"Pissani","year":"2023","journal-title":"Metaphor Symbol"},{"key":"B36","unstructured":"Speech cues influence the interpretation of metaphoric gestures\n \n \n \n Pissani\n L.\n \n \n Demberg\n V.\n \n \n 10.31234\/osf.io\/pqk3d_v2\n PsyArXiv\n \n 2026"},{"key":"B37","unstructured":"Working memory and vocabulary knowledge sustain activation of the literal meaning of metaphors during sentence comprehension\n \n \n \n Pissani\n L.\n \n \n Scholman\n M. C.\n \n \n Demberg\n V.\n \n \n 10.31234\/osf.io\/dqanh_v1\n PsyArXiv\n \n 2026"},{"key":"B38","first-page":"57","article-title":"\u201cLingoTurk: Managing crowdsourced tasks for psycholinguistics,\u201d","author":"Pusse","year":"2016","journal-title":"Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)"},{"key":"B39","volume-title":"R: A Language and Environment for Statistical Computing","year":"2024"},{"key":"B40","article-title":"\u201cIntegratitng co-speech gestures into sentence meaning comprehension,\u201d","author":"Reimer","year":"2024","journal-title":"Proceedings of the Annual Meeting of the Cognitive Science Society"},{"key":"B41","doi-asserted-by":"publisher","first-page":"800","DOI":"10.3758\/s13428-014-0502-y","article-title":"Semantic properties, aptness, familiarity, conventionality, and interpretive diversity scores for 84 metaphors and similes","volume":"47","author":"Roncero","year":"2015","journal-title":"Behav. Res. Methods"},{"key":"B42","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1016\/j.jml.2018.12.003","article-title":"Metaphor comprehension: An individual-differences approach","volume":"105","author":"Stamenkovi\u0107","year":"2019","journal-title":"J. Memory Lang"},{"key":"B43","doi-asserted-by":"publisher","first-page":"821","DOI":"10.1162\/jocn.2009.21053","article-title":"Memory effects of speech and gesture binding: cortical and hippocampal activation in relation to subsequent memory performance","volume":"21","author":"Straube","year":"2009","journal-title":"J. Cognit. Neurosci"},{"key":"B44","doi-asserted-by":"crossref","DOI":"10.31234\/osf.io\/b5zq7","article-title":"\u201cThe predictive potential of hand gestures during conversation: An investigation of the timing of gestures in relation to speech,\u201d","volume-title":"Gesture and Speech in Interaction Conference","author":"Ter Bekke","year":"2020"},{"key":"B45","doi-asserted-by":"publisher","first-page":"583","DOI":"10.3389\/fnhum.2014.00583","article-title":"The role of literal meaning in figurative language comprehension: evidence from masked priming ERP","volume":"8","author":"Weiland","year":"2014","journal-title":"Front. Human Neurosci"},{"key":"B46","doi-asserted-by":"publisher","first-page":"654","DOI":"10.1111\/j.1469-8986.2005.00356.x","article-title":"Meaningful gestures: electrophysiological indices of iconic gesture comprehension","volume":"42","author":"Wu","year":"2005","journal-title":"Psychophysiology"}],"container-title":["Frontiers in Psychology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpsyg.2026.1851588\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,19]],"date-time":"2026-08-19T04:35:18Z","timestamp":1787114118000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpsyg.2026.1851588\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,19]]},"references-count":46,"alternative-id":["10.3389\/fpsyg.2026.1851588"],"URL":"https:\/\/doi.org\/10.3389\/fpsyg.2026.1851588","relation":{},"ISSN":["1664-1078"],"issn-type":[{"value":"1664-1078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,19]]},"article-number":"1851588"}}