{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T06:05:03Z","timestamp":1790057103872,"version":"4.0.1"},"reference-count":32,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T00:00:00Z","timestamp":1790035200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Word grounding refers to the ability of agents to associate linguistic terms with the perceptual concepts they represent, such as color, shape, and spatial features. This study quantitatively evaluates how syntactic and semantic information affects word-grounding performance.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We evaluated five configurations within a generative multimodal Bayesian framework. The baseline grounds words using perceptual cues, while the additional configurations independently incorporate word-position indices, part-of-speech tags, FastText embeddings, and transformer-based contextual embeddings. The models were evaluated using a CLEVR-derived 3D scene dataset and synonym-substituted descriptions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Linguistic information improved grounding accuracy across the Color, Geometry, and Spatial Relation modalities. Contextual embeddings produced a 53.8% absolute improvement in object-grounding accuracy over the baseline. Only models incorporating semantic embeddings generalized effectively to synonym-substituted descriptions without retraining.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>These findings demonstrate the contribution of linguistic structure and semantic representations to perceptual alignment and provide empirical foundations for developing language-grounding systems.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/frai.2026.1803426","type":"journal-article","created":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T05:39:38Z","timestamp":1790055578000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["The effect of syntactic and semantic information on word grounding through visual perception"],"prefix":"10.3389","volume":"9","author":[{"given":"Saima","family":"Shaukat","sequence":"first","affiliation":[{"name":"School of Computing, Engineering, and Mathematics, University of Plymouth","place":["Plymouth, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amir","family":"Aly","sequence":"additional","affiliation":[{"name":"School of Computing, Engineering, and Mathematics, University of Plymouth","place":["Plymouth, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anouar","family":"Chibani","sequence":"additional","affiliation":[{"name":"School of Computing, Engineering, and Mathematics, University of Plymouth","place":["Plymouth, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Wennekers","sequence":"additional","affiliation":[{"name":"School of Computing, Engineering, and Mathematics, University of Plymouth","place":["Plymouth, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Angelo","family":"Cangelosi","sequence":"additional","affiliation":[{"name":"School of Computer Science, University of Manchester","place":["Manchester, United Kingdom"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,22]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"376","DOI":"10.1109\/DEVLRN.2017.8329833","article-title":"\u201cA generative framework for multimodal learning of spatial concepts and object categories: an unsupervised part-of-speech tagging and 3d visual perception based approach,\u201d","author":"Aly","year":"2017","journal-title":"2017 joint IEEE international conference on development and learning and epigenetic robotics (ICDL-EpiRob)"},{"key":"B2","doi-asserted-by":"publisher","first-page":"7143","DOI":"10.1109\/ICRA.2018.8461231","article-title":"\u201cTowards understanding object-directed actions: a generative model for grounding syntactic categories of speech through visual perception,\u201d","author":"Aly","year":"2018","journal-title":"2018 IEEE international conference on robotics and automation (ICRA)"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/HUMANOIDS.2018.8625009","article-title":"\u201cA probabilistic approach to unsupervised induction of combinatory categorial grammar in situated human-robot interaction,\u201d","author":"Aly","year":"2018","journal-title":"2018 IEEE-RAS 18th international conference on humanoid robots (humanoids)"},{"key":"B4","doi-asserted-by":"publisher","first-page":"2425","DOI":"10.1109\/ICCV.2015.279","article-title":"\u201cVqa: visual question answering,\u201d","author":"Antol","year":"2015","journal-title":"Proceedings of the IEEE international conference on computer vision"},{"key":"B5","doi-asserted-by":"publisher","first-page":"13633","DOI":"10.63317\/52jyi873avpy","article-title":"\u201cRecap: semantic role enhanced caption generation,\u201d","author":"Bhattacharyya","year":"2024","journal-title":"Proceedings of the 2024 joint international conference on computational linguistics, language resources and evaluation (LREC-COLING 2024)"},{"key":"B6","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1162\/tacl_a_00051","article-title":"Enriching word vectors with subword information","volume":"5","author":"Bojanowski","year":"2017","journal-title":"Trans. Assoc. Comput. Linguist"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1423","article-title":"\u201cBert: pre-training of deep bidirectional transformers for language understanding,\u201d","author":"Devlin","year":"2019","journal-title":"Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, Vol. 1 (long and short papers)"},{"key":"B8","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-031-01553-3","volume-title":"Visual Object Recognition","author":"Grauman","year":"2011"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1640","DOI":"10.1109\/IROS.2013.6696569","article-title":"\u201cGrounding spatial relations for human-robot interaction,\u201d","author":"Guadarrama","year":"2013","journal-title":"2013 IEEE\/RSJ international conference on intelligent robots and systems"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1016\/0167-2789(90)90087-6","article-title":"The symbol grounding problem","author":"Harnad","year":"1990","journal-title":"Physica D"},{"key":"B11","doi-asserted-by":"publisher","first-page":"2901","DOI":"10.1109\/CVPR.2017.215","article-title":"\u201cClevr: a diagnostic dataset for compositional language and elementary visual reasoning,\u201d","author":"Johnson","year":"2017","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"B12","doi-asserted-by":"publisher","first-page":"142","DOI":"10.5455\/jmood.20150920115439","article-title":"Kappa test","volume":"5","author":"Kili\u00e7","year":"2015","journal-title":"Psychiatry Behav. Sci"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2021.XVII.047","article-title":"\u201cLanguage conditioned imitation learning over unstructured data,\u201d","author":"Lynch","year":"2021","journal-title":"Proceedings of robotics: science and systems"},{"key":"B14","unstructured":"A joint model of language and perception for grounded attribute learning\n          \n          \n            \n              Matuszek\n              C.\n            \n            \n              FitzGerald\n              N.\n            \n            \n              Zettlemoyer\n              L.\n            \n            \n              Bo\n              L.\n            \n            \n              Fox\n              D.\n            \n          \n          arXiv\n          \n          2012"},{"key":"B15","first-page":"3111","article-title":"\u201cDistributed representations of words and phrases and their compositionality,\u201d","volume-title":"Proceedings of the 27th international conference on neural information processing systems","author":"Mikolov","year":"2013"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1532","DOI":"10.3115\/v1\/D14-1162","article-title":"\u201cGlove: global vectors for word representation,\u201d","author":"Pennington","year":"2014","journal-title":"Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)"},{"key":"B17","first-page":"8748","article-title":"\u201cLearning transferable visual models from natural language supervision,\u201d","volume-title":"International conference on machine learning","author":"Radford","year":"2021"},{"key":"B18","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1016\/j.tics.2005.06.013","article-title":"Grounding words in perception and action: computational insights","volume":"9","author":"Roy","year":"2005","journal-title":"Trends Cogn. Sci"},{"key":"B19","doi-asserted-by":"publisher","first-page":"70","DOI":"10.3115\/1119212.1119222","article-title":"\u201cConversational robots: building blocks for grounding word meaning,\u201d","author":"Roy","year":"2003","journal-title":"Proceedings of the HLT-NAACL 2003 workshop on Learning word meaning from non-linguistic data"},{"key":"B20","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1016\/S0885-2308(02)00024-4","article-title":"Learning visually grounded words and syntax for a scene description task","volume":"16","author":"Roy","year":"2002","journal-title":"Comput. Speech Lang"},{"key":"B21","doi-asserted-by":"publisher","first-page":"2155","DOI":"10.1109\/IROS.2010.5651280","article-title":"\u201cFast 3d recognition and pose using the viewpoint feature histogram,\u201d","author":"Rusu","year":"2010","journal-title":"2010 IEEE\/RSJ international conference on intelligent robots and systems"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1362463","DOI":"10.3389\/frobt.2024.1362463","article-title":"Evaluation of co-speech gestures grounded in word-distributed representation","volume":"11","author":"Sasaki","year":"2024","journal-title":"Front. Robot. AI"},{"key":"B23","first-page":"484","article-title":"\u201cEvaluating semantic representations in multimodal word grounding,\u201d","volume-title":"Pacific rim international conference on artificial intelligence","author":"Shaukat","year":""},{"key":"B24","article-title":"\u201cGrounding word meaning through perception: toward compositional language understanding in human-robot interaction,\u201d","volume-title":"The international conference on robot and human interactive communication (Ro-MAN)","author":"Shaukat","year":""},{"key":"B25","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1093\/acprof:oso\/9780199217274.003.0012","article-title":"\u201cThe symbol grounding problem has been solved. So what's next,\u201d","author":"Steels","year":"2008","journal-title":"Symbols Embodiment: Debates on Meaning and Cognition"},{"key":"B26","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1162\/tacl_a_00096","article-title":"Unsupervised part-of-speech tagging with anchor hidden markov models","volume":"4","author":"Stratos","year":"2016","journal-title":"Trans. Assoc. Comput. Linguist"},{"key":"B27","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1609\/aimag.v32i4.2384","article-title":"Approaching the symbol grounding problem with probabilistic graphical models","volume":"32","author":"Tellex","year":"2011","journal-title":"AI Mag"},{"key":"B28","first-page":"1691","article-title":"\u201cLanguage grounding with 3d objects,\u201d","volume-title":"Conference on robot learning","author":"Thomason","year":"2022"},{"key":"B29","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1016\/S1389-0417(02)00051-7","article-title":"The physical symbol grounding problem","volume":"3","author":"Vogt","year":"2002","journal-title":"Cogn. Syst. Res"},{"key":"B30","doi-asserted-by":"publisher","first-page":"639","DOI":"10.1109\/TASLP.2023.3337643","article-title":"Lost in context? On the sense-wise variance of contextualized word embeddings","volume":"32","author":"Wang","year":"2023","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process"},{"key":"B31","first-page":"1039","article-title":"\u201cNeural-symbolic VQA: disentangling reasoning from vision and language understanding,\u201d","volume-title":"Proceedings of the 32nd international conference on neural information processing systems","author":"Yi","year":"2018"},{"key":"B32","first-page":"53","article-title":"\u201cGrounded language learning from video described with sentences,\u201d","volume-title":"Proceedings of the 51st annual meeting of the association for computational linguistics (Vol. 1: long papers)","author":"Yu","year":"2013"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1803426\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T05:39:41Z","timestamp":1790055581000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1803426\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,22]]},"references-count":32,"alternative-id":["10.3389\/frai.2026.1803426"],"URL":"https:\/\/doi.org\/10.3389\/frai.2026.1803426","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,22]]},"article-number":"1803426"}}