{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T10:54:12Z","timestamp":1774522452169,"version":"3.50.1"},"reference-count":45,"publisher":"Wiley","issue":"2","license":[{"start":{"date-parts":[[2007,3,1]],"date-time":"2007-03-01T00:00:00Z","timestamp":1172707200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Cognitive Science"],"published-print":{"date-parts":[[2007,3,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We introduce a computational theory of situated language understanding in which the meaning of words and utterances depends on the physical environment and the goals and plans of communication partners. According to the theory, concepts that ground linguistic meaning are neither internal nor external to language users, but instead span the objective\u2010subjective boundary. To model the possible interactions between subject and object, the theory relies on the notion of <jats:italic>perceived affordances<\/jats:italic>: structured units of interaction that can be used for prediction at multiple levels of abstraction. Language understanding is treated as a process of filtering perceived affordances. The theory accounts for many aspects of the situated nature of human language use and provides a unified solution to a number of demands on any theory of language understanding including conceptual combination, prototypicality effects, and the generative nature of lexical items.<\/jats:p><jats:p>To support the theory, we describe an implemented system that understands verbal commands situated in a virtual gaming environment. The implementation uses probabilistic hierarchical plan recognition to generate perceived affordances. The system has been evaluated on its ability to correctly interpret free\u2010form spontaneous verbal commands recorded from unrehearsed game play between human players. The system is able to \u201cstep into the shoes\u201d of human players and correctly respond to a broad range of verbal commands in which linguistic meaning depends on social and physical context. We quantitatively compare the system's predictions in response to direct player commands with the actions taken by human players and show generalization to unseen data across a range of situations and verbal constructions.<\/jats:p>","DOI":"10.1080\/15326900701221199","type":"journal-article","created":{"date-parts":[[2022,9,21]],"date-time":"2022-09-21T20:42:43Z","timestamp":1663792963000},"page":"197-231","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Situated Language Understanding as Filtering Perceived Affordances"],"prefix":"10.1111","volume":"31","author":[{"given":"Peter","family":"Gorniak","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Deb","family":"Roy","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2007,3]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(80)90042-9"},{"issue":"4","key":"e_1_2_8_3_1","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1017\/S0140525X99002149","article-title":"Perceptual symbol systems","volume":"22","author":"Barsalou L.","year":"1999","journal-title":"Behavioural and Brain Sciences"},{"key":"e_1_2_8_4_1","first-page":"459","volume-title":"Computing Anticipatory Systems. CASYS 2000\u2014Fourth International Conference","author":"Bickhard M. H.","year":"2001"},{"key":"e_1_2_8_5_1","unstructured":"Bobick A. F. Ivanov Y. A.Action recognition using probabilistic parsing1998Proceedings of the IEEE Conference on Computer Vision and Pattern RecognitionSanta Barbar CA"},{"key":"e_1_2_8_6_1","first-page":"1","article-title":"Decision\u2010theoretic planning: Structural assumptions and computational leverage","volume":"11","author":"Boutilier C.","year":"1999","journal-title":"Journal of AI Research"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.839"},{"key":"e_1_2_8_8_1","volume-title":"Vision, Instruction and Action","author":"Chapman D.","year":"1991"},{"key":"e_1_2_8_9_1","volume-title":"Being There: Putting Brain, Body and World Together Again","author":"Clark A.","year":"1998"},{"key":"e_1_2_8_10_1","doi-asserted-by":"publisher","DOI":"10.1162\/089120103322753356"},{"key":"e_1_2_8_11_1","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/4378.001.0001","volume-title":"Made\u2010up minds","author":"Drescher G.","year":"1991"},{"key":"e_1_2_8_12_1","volume-title":"Rethinking Context: Language as an Interactive Phenomenon","author":"Duranti A.","year":"1992"},{"issue":"8","key":"e_1_2_8_13_1","first-page":"451","article-title":"An efficient context\u2010free parsing algorithm","volume":"6","author":"Earley J.","year":"1970","journal-title":"Communications of the ACM"},{"key":"e_1_2_8_14_1","unstructured":"Erol K. Hendler J. Nau D.HTN planning: Complexity and expressivity.Proceedings of the American Association for Artificial Intelligence1994Seattl WA"},{"key":"e_1_2_8_15_1","unstructured":"Geib C. Goldman R.Partial observability and probabilistic plan\/goal recognition2005IJCAI\u201005 workshop on Modeling Others from Observations"},{"key":"e_1_2_8_16_1","first-page":"67","volume-title":"Perceiving, Acting and Knowing","author":"Gibson J.","year":"1977"},{"key":"e_1_2_8_17_1","first-page":"1","article-title":"What memory is for Behavioural and Brain","volume":"20","author":"Glenberg A. M.","year":"1997","journal-title":"Sciences"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03196313"},{"key":"e_1_2_8_19_1","doi-asserted-by":"crossref","unstructured":"Gorniak P. Roy D.Probabilistic grounding of situated speech using plan recognition and reference resolution2005aProceedings of the International Conference on Multimodal InterfacesTrento Italy","DOI":"10.1145\/1088463.1088489"},{"key":"e_1_2_8_20_1","unstructured":"Gorniak P. Roy D.Speaking with your Sidekick: Understanding situated speech in computer role playing games2005bProceedings of Artificial Intelligence and Digital EntertainmentMarina Del Ray"},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1327"},{"key":"e_1_2_8_22_1","unstructured":"Hsiao K. Mavridis N. Roy D.Coupling perception and simulation: Steps towards conversational robotics2003Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)Las Vega NE"},{"key":"e_1_2_8_23_1","doi-asserted-by":"crossref","unstructured":"Klein D. Manning C. D.Accurate unlexicalized parsing2003Proceedings of the 41st Meeting of the Association of Computational LinguisticsSapporo Japan","DOI":"10.3115\/1075096.1075150"},{"key":"e_1_2_8_24_1","first-page":"3","volume-title":"Concepts: Core Readings","author":"Laurence S.","year":"1999"},{"key":"e_1_2_8_25_1","unstructured":"Litman D. J. Allen J. F.A plan recognition model for clarification subdialogues.COLING1984302\u2013311"},{"key":"e_1_2_8_26_1","unstructured":"Littman M. Sutton R. Singh S.Predictive representations of state2001Proceedings of the Neural Information Processing Systems ConferenceVancouver Canada"},{"key":"e_1_2_8_27_1","doi-asserted-by":"crossref","first-page":"495","DOI":"10.21437\/Eurospeech.1999-127","article-title":"Finding consensus among words: Latticebased word error minimization","volume":"1","author":"Mangu L.","year":"1999","journal-title":"Proceedings of EUROSPEECH1999"},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1037\/10039-000"},{"key":"e_1_2_8_29_1","volume-title":"Society of Mind","author":"Minsky M.","year":"1985"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1141"},{"key":"e_1_2_8_31_1","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3169.001.0001"},{"key":"e_1_2_8_32_1","doi-asserted-by":"publisher","DOI":"10.1006\/brln.2000.2390"},{"key":"e_1_2_8_33_1","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511625251","volume-title":"Philosophical Papers, Vol. 2: Mind, Language and Reality","author":"Putnam H.","year":"1975"},{"key":"e_1_2_8_34_1","volume-title":"Proceedings of the Conference on Uncertainty in Artificial Intelligence, UAI2000","author":"Pynadath D. V.","year":"2000"},{"key":"e_1_2_8_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2005.04.007"},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2004.823327"},{"key":"e_1_2_8_37_1","volume-title":"Scripts, Plans, Goals and Understanding","author":"Schank R. C.","year":"1977"},{"key":"e_1_2_8_38_1","doi-asserted-by":"crossref","unstructured":"Schuler W.Using model\u2010theoretic semantic interpretation to guide statistical parsing and word recognition in a spoken language interface2003Proceedings of the Association for Computational LinguisticsSapporo Japan","DOI":"10.3115\/1075096.1075163"},{"key":"e_1_2_8_39_1","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/5128.001.0001","volume-title":"On the Origin of Objects","author":"Smith B. C.","year":"1996"},{"key":"e_1_2_8_40_1","doi-asserted-by":"crossref","unstructured":"Steedman M.Formalizing affordance2002Proceedings of the 24th Annual Meeting of the Cognitive Science SocietyFairfa VA834\u2013839","DOI":"10.4324\/9781315782379-178"},{"issue":"2","key":"e_1_2_8_41_1","first-page":"165","article-title":"An efficient probabilistic context\u2010free parsing algorithm that computes prefix probabilities","volume":"21","author":"Stolcke A.","year":"1995","journal-title":"Computational Linguistics"},{"key":"e_1_2_8_42_1","unstructured":"Stone M.Representing communicative intentions in collaborative conversational agents2001AAAI Fall Symposium on Intent Inference for Collaborative TasksNorth Falmout MA"},{"key":"e_1_2_8_43_1","unstructured":"Stoytchev A.Behavior\u2010grounded representation of tool affordances2005Proceedings of IEEE International Conference on Robotics and Automation (ICRA)Barcelona Spane"},{"key":"e_1_2_8_44_1","unstructured":"Winograd T.Procedures as a representation for data in a computer program for understanding natural language1970Massachusetts Institute of TechnologyPh.D. thesis"},{"key":"e_1_2_8_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/18.87000"},{"key":"e_1_2_8_46_1","unstructured":"Zwaan R. A.The immersed experiencer: Toward an embodied theory of language comprehension.The Psychology of Learning and Motivation200344"}],"container-title":["Cognitive Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1080\/15326900701221199","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1080\/15326900701221199","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1080\/15326900701221199","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,26]],"date-time":"2024-10-26T16:46:09Z","timestamp":1729961169000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1080\/15326900701221199"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2007,3]]},"references-count":45,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2007,3,4]]}},"alternative-id":["10.1080\/15326900701221199"],"URL":"https:\/\/doi.org\/10.1080\/15326900701221199","archive":["Portico"],"relation":{},"ISSN":["0364-0213","1551-6709"],"issn-type":[{"value":"0364-0213","type":"print"},{"value":"1551-6709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2007,3]]},"assertion":[{"value":"2007-03-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}