{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T01:08:23Z","timestamp":1783386503362,"version":"3.54.6"},"reference-count":53,"publisher":"SAGE Publications","issue":"10","license":[{"start":{"date-parts":[[2018,6,25]],"date-time":"2018-06-25T00:00:00Z","timestamp":1529884800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"Robotics Con- sortium of the U.S Army Research Laboratory under the Collaborative Technology Alliance Program"},{"name":"National Science Foundation"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2018,9]]},"abstract":"<jats:p>Our goal is to develop models that allow a robot to efficiently understand or \u201cground\u201d natural language instructions in the context of its world representation. Contemporary approaches estimate correspondences between language instructions and possible groundings such as objects, regions, and goals for actions that the robot should execute. However, these approaches typically reason in relatively small domains and do not model abstract spatial concepts such as as \u201crows,\u201d \u201ccolumns,\u201d or \u201cgroups\u201d of objects and, hence, are unable to interpret an instruction such as \u201cpick up the middle block in the row of five blocks.\u201d In this paper, we introduce two new models for efficient natural language understanding of robot instructions. The first model, which we call the adaptive distributed correspondence graph (ADCG), is a probabilistic model for interpreting abstract concepts that require hierarchical reasoning over constituent concrete entities as well as notions of cardinality and ordinality. Abstract grounding variables form a Markov boundary over concrete groundings, effectively de-correlating them from the remaining variables in the graph. This structure reduces the complexity of model training and inference. Inference in the model is posed as an approximate search procedure that orders factor computation such that the estimated probable concrete groundings focus the search for abstract concepts towards likely hypothesis, pruning away improbable portions of the exponentially large space of abstractions. Further, we address the issue of scalability to complex domains and introduce a hierarchical extension to a second model termed the hierarchical adaptive distributed correspondence graph (HADCG). The model utilizes the abstractions in the ADCG but infers a coarse symbolic structure from the utterance and the environment model and then performs fine-grained inference over the reduced graphical model, further improving the efficiency of inference. Empirical evaluation demonstrates accurate grounding of abstract concepts embedded in complex natural language instructions commanding a robotic torso and a mobile robot. Further, the proposed approximate inference method allows significant efficiency gains compared with the baseline, with minimal trade-off in accuracy.<\/jats:p>","DOI":"10.1177\/0278364918777627","type":"journal-article","created":{"date-parts":[[2018,6,26]],"date-time":"2018-06-26T00:42:20Z","timestamp":1529973740000},"page":"1269-1299","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":43,"title":["Efficient grounding of abstract spatial concepts for natural language interaction with robot platforms"],"prefix":"10.1177","volume":"37","author":[{"given":"Rohan","family":"Paul","sequence":"first","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jacob","family":"Arkin","sequence":"additional","affiliation":[{"name":"Robotics and Artificial Intelligence Laboratory (RAIL), Electrical and Computer Engineering, University of Rochester, Rochester, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Derya","family":"Aksaray","sequence":"additional","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicholas","family":"Roy","sequence":"additional","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas M.","family":"Howard","sequence":"additional","affiliation":[{"name":"Robotics and Artificial Intelligence Laboratory (RAIL), Electrical and Computer Engineering, University of Rochester, Rochester, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2018,6,25]]},"reference":[{"key":"bibr1-0278364918777627","first-page":"505","volume-title":"Proceedings, 15th International Conference on Principles of Knowledge Representation and Reasoning","author":"Alomari M","year":"2016"},{"key":"bibr2-0278364918777627","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1138"},{"key":"bibr3-0278364918777627","author":"Andreas J","year":"2015","journal-title":"EMNLP"},{"key":"bibr4-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00209"},{"key":"bibr5-0278364918777627","first-page":"751","author":"Bisk Y","year":"2016","journal-title":"Proceedings of NAACL-HLT"},{"key":"bibr6-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2016.7759412"},{"key":"bibr7-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139457"},{"key":"bibr8-0278364918777627","doi-asserted-by":"publisher","DOI":"10.3115\/1687878.1687892"},{"key":"bibr9-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1177\/0278364917706418"},{"key":"bibr10-0278364918777627","first-page":"275","volume-title":"Proceedings of the 5th ACM\/IEEE International Conference on Human\u2013Robot Interaction","author":"Cantrell R","year":"2010"},{"key":"bibr11-0278364918777627","first-page":"317","volume-title":"2010 5th ACM\/IEEE International Conference on Human\u2013Robot Interaction (HRI)","author":"Chao C","year":"2010"},{"key":"bibr12-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1613\/jair.2962"},{"key":"bibr13-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2015.7354117"},{"key":"bibr14-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(03)00021-6"},{"key":"bibr15-0278364918777627","volume-title":"Proceedings International Symposium on Experimental Robotics (ISER)","author":"Duvallet F","year":"2014"},{"key":"bibr16-0278364918777627","doi-asserted-by":"publisher","DOI":"10.3115\/1706543.1706546"},{"key":"bibr17-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1162\/089120102760275983"},{"key":"bibr18-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2013.6696569"},{"key":"bibr19-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1016\/0167-2789(90)90087-6"},{"key":"bibr20-0278364918777627","author":"Heim I","year":"1998","journal-title":"Semantics in generative grammar"},{"key":"bibr21-0278364918777627","volume-title":"IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) Workshop on Rehabilitation and Assistive Robotics","author":"Howard TM","year":"2014"},{"key":"bibr22-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6907841"},{"key":"bibr23-0278364918777627","volume-title":"Semantics and Cognition","volume":"8","author":"Jackendoff R","year":"1983"},{"key":"bibr24-0278364918777627","first-page":"259","volume-title":"2010 5th ACM\/IEEE International Conference on Human\u2013Robot Interaction (HRI)","author":"Kollar T","year":"2010"},{"key":"bibr25-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-28572-1_3"},{"key":"bibr26-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1163\/156855308X344864"},{"key":"bibr27-0278364918777627","author":"Kulick J","year":"2013","journal-title":"IJCAI"},{"key":"bibr28-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-011-0123-x"},{"key":"bibr29-0278364918777627","doi-asserted-by":"publisher","DOI":"10.3115\/1220175.1220271"},{"key":"bibr30-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00127"},{"issue":"6","key":"bibr31-0278364918777627","first-page":"4","volume":"2","author":"MacMahon M","year":"2006","journal-title":"AAAI"},{"issue":"2","key":"bibr32-0278364918777627","first-page":"313","volume":"19","author":"Marcus MP","year":"1993","journal-title":"Computational linguistics"},{"key":"bibr33-0278364918777627","first-page":"710","author":"Martin JH","year":"2000","journal-title":"International Edition"},{"key":"bibr34-0278364918777627","volume-title":"Proceedings of the 2012 International Conference on Machine Learning","author":"Matuszek C","year":"2012"},{"key":"bibr35-0278364918777627","first-page":"251","volume-title":"2010 5th ACM\/IEEE International Conference on Human\u2013Robot Interaction (HRI)","author":"Matuszek C","year":"2010"},{"key":"bibr36-0278364918777627","volume-title":"Procceedings of the 13th International Symposium on Experimental Robotics (ISER)","author":"Matuszek C","year":"2012"},{"key":"bibr37-0278364918777627","first-page":"2772","volume-title":"Proceedings of the National Conference on Artificial Intelligence (AAAI)","author":"Mei H","year":"2016"},{"key":"bibr38-0278364918777627","volume-title":"Proceedings of Robotics: Science and Systems (RSS)","author":"Misra DK","year":"2014"},{"key":"bibr39-0278364918777627","volume-title":"Proceedings of the Workshops at AAAI","author":"Mohan S","year":"2012"},{"key":"bibr40-0278364918777627","unstructured":"Nilsson NJ (1984) Shakey the robot. Technical report, DTIC Document."},{"key":"bibr41-0278364918777627","first-page":"309","volume-title":"Proceedings of the 2016 International Symposium on Experimental Robotics","author":"Oh J","year":"2016"},{"key":"bibr42-0278364918777627","first-page":"1371","volume-title":"Proceedings of the National Conference on Artificial Intelligence (AAAI)","author":"Oh J","year":"2015"},{"key":"bibr43-0278364918777627","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2016.XII.040"},{"key":"bibr44-0278364918777627","first-page":"1488","volume":"9","author":"Shimizu N","year":"2009","journal-title":"IJCAI"},{"key":"bibr45-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.99"},{"key":"bibr46-0278364918777627","volume-title":"Proceedings of the National Conference on Artificial Intelligence (AAAI)","author":"Tellex S","year":"2011"},{"key":"bibr47-0278364918777627","first-page":"806","volume-title":"Proceedings of the 48th Annual Meeting of the Association for Computational Linguisticsi (ACL)","author":"Vogel A","year":"2010"},{"key":"bibr48-0278364918777627","unstructured":"Winograd T (1971) Procedures as a representation for data in a computer program for understanding natural language. Technical report, DTIC Document."},{"key":"bibr49-0278364918777627","first-page":"149","author":"Yang S","year":"2016","journal-title":"Proceedings of NAACL-HLT"},{"key":"bibr50-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1016\/S0019-9958(67)80007-X"},{"key":"bibr51-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1613\/jair.4556"},{"key":"bibr52-0278364918777627","first-page":"678","volume-title":"Proceedings of Empirical Methods in Natural Language Processing and Computational Nautal Language Learning (EMNLP-CoNLL)","author":"Zettlemoyer LS","year":"2007"},{"key":"bibr53-0278364918777627","doi-asserted-by":"publisher","DOI":"10.1145\/279232.279236"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364918777627","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0278364918777627","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364918777627","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:15:41Z","timestamp":1777457741000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0278364918777627"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,25]]},"references-count":53,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2018,9]]}},"alternative-id":["10.1177\/0278364918777627"],"URL":"https:\/\/doi.org\/10.1177\/0278364918777627","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,6,25]]}}}