{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T10:08:14Z","timestamp":1777716494678,"version":"3.51.4"},"reference-count":51,"publisher":"SAGE Publications","issue":"9","license":[{"start":{"date-parts":[[2014,8,1]],"date-time":"2014-08-01T00:00:00Z","timestamp":1406851200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2014,8]]},"abstract":"<jats:p>This paper describes a framework that enables robots to efficiently learn human-centric models of their environment from natural language descriptions. Typical semantic mapping approaches are limited to augmenting metric maps with higher-level properties of the robot\u2019s surroundings (e.g. place type, object locations) that can be inferred from the robot\u2019s sensor data, but do not use this information to improve the metric map. The novelty of our algorithm lies in fusing high-level knowledge that people can uniquely provide through speech with metric information from the robot\u2019s low-level sensor streams. Our method jointly estimates a hybrid metric, topological, and semantic representation of the environment. This semantic graph provides a common framework in which we integrate information that the user communicates (e.g. labels and spatial relations) with metric observations from low-level sensors. Our algorithm efficiently maintains a factored distribution over semantic graphs based upon the stream of natural language and low-level sensor information. We detail the means by which the framework incorporates knowledge conveyed by the user\u2019s descriptions, including the ability to reason over expressions that reference yet unknown regions in the environment. We evaluate the algorithm\u2019s ability to learn human-centric maps of several different environments and analyze the knowledge inferred from language and the utility of the learned maps. The results demonstrate that the incorporation of information from free-form descriptions increases the metric, topological, and semantic accuracy of the recovered environment model.<\/jats:p>","DOI":"10.1177\/0278364914537359","type":"journal-article","created":{"date-parts":[[2014,8,6]],"date-time":"2014-08-06T05:48:44Z","timestamp":1407304124000},"page":"1167-1190","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":22,"title":["A framework for learning semantic maps from grounded natural language descriptions"],"prefix":"10.1177","volume":"33","author":[{"given":"Matthew R.","family":"Walter","sequence":"first","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sachithra","family":"Hemachandra","sequence":"additional","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bianca","family":"Homberg","sequence":"additional","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stefanie","family":"Tellex","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Brown University, Providence, RI, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seth","family":"Teller","sequence":"additional","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2014,8,6]]},"reference":[{"key":"bibr1-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1177\/0278364909100586"},{"key":"bibr2-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1177\/0278364904049393"},{"key":"bibr3-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2007.4399611"},{"key":"bibr4-0278364914537359","first-page":"96","author":"Bugmann G","year":"2004","journal-title":"Proceedings of intelligent autonomous systems"},{"key":"bibr5-0278364914537359","first-page":"859","volume-title":"Proceedings of the national conference on artificial intelligence (AAAI)","author":"Chen DL","year":"2011"},{"key":"bibr6-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1177\/0278364908090961"},{"key":"bibr7-0278364914537359","first-page":"176","volume-title":"Proceedings of the conference on uncertainty in artificial intelligence (UAI)","author":"Doucet A","year":"2000"},{"key":"bibr8-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2009.5152776"},{"key":"bibr9-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2005.1570475"},{"key":"bibr10-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1111\/1467-9868.00421"},{"key":"bibr11-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2005.1545511"},{"key":"bibr12-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/CIRA.1999.810068"},{"key":"bibr13-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1016\/0167-2789(90)90087-6"},{"key":"bibr14-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5980209"},{"key":"bibr15-0278364914537359","unstructured":"Hemachandra S, Walter MR, Tellex S, (2013) Learning semantic maps from natural language descriptions. Available at: http:\/\/vimeo.com\/67438012."},{"key":"bibr16-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6907235"},{"key":"bibr17-0278364914537359","volume-title":"Semantics and Cognition","author":"Jackendoff R","year":"1985"},{"key":"bibr18-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2008.2006706"},{"key":"bibr19-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2009.5152831"},{"key":"bibr20-0278364914537359","first-page":"259","volume-title":"Proceedings of the ACM\/IEEE international conference on human\u2013robot interaction (HRI)","author":"Kollar T","year":"2010"},{"key":"bibr21-0278364914537359","first-page":"457","volume-title":"Proceedings of the national conference on artificial intelligence (AAAI)","author":"Konolige K","year":"2004"},{"key":"bibr22-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-32255-9_22"},{"key":"bibr23-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(00)00017-5"},{"key":"bibr24-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2004.1302485"},{"key":"bibr25-0278364914537359","first-page":"1143","volume-title":"Proceedings of the international joint conference on artificial intelligence (IJCAI)","author":"Leonard J","year":"2003"},{"key":"bibr26-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1007\/BF00162521"},{"key":"bibr27-0278364914537359","volume-title":"The Image of the City","author":"Lynch K","year":"1960"},{"key":"bibr28-0278364914537359","first-page":"1475","volume-title":"Proceedings of the national conference on artificial intelligence (AAAI)","author":"MacMahon M","year":"2006"},{"key":"bibr29-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2006.12.003"},{"key":"bibr30-0278364914537359","first-page":"251","volume-title":"Proceedings of the ACM\/IEEE international conference on human\u2013robot interaction (HRI)","author":"Matuszek C","year":"2010"},{"key":"bibr31-0278364914537359","first-page":"403","volume-title":"Proceedings of the international symposium on experimental robotics (ISER)","author":"Matuszek C","year":"2012"},{"key":"bibr32-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2008.03.008"},{"key":"bibr33-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2004.1389613"},{"key":"bibr34-0278364914537359","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2007.III.010"},{"key":"bibr35-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2012.6224637"},{"key":"bibr36-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1177\/0278364909356483"},{"key":"bibr37-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2009.5152376"},{"key":"bibr38-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1177\/0278364910393287"},{"key":"bibr39-0278364914537359","unstructured":"Russell SJ, Norvig P (2003) Artificial Intelligence: A Modern Approach. 2nd edition. Upper Saddle River, NJ: Prentice Hall, pp. 97\u2013104."},{"key":"bibr40-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2004.839228"},{"key":"bibr41-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2004.826273"},{"key":"bibr42-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1177\/027836498600500404"},{"key":"bibr43-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v25i1.7979"},{"key":"bibr44-0278364914537359","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2012.VIII.052"},{"key":"bibr45-0278364914537359","first-page":"989","volume-title":"Proceedings of the national conference on artificial intelligence (AAAI)","author":"Thrun S","year":"1998"},{"key":"bibr46-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1177\/0278364904045479"},{"key":"bibr47-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2003.1238354"},{"key":"bibr48-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2008.03.005"},{"key":"bibr49-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1177\/0278364906075026"},{"key":"bibr50-0278364914537359","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2013.IX.004"},{"key":"bibr51-0278364914537359","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2008.03.007"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364914537359","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0278364914537359","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364914537359","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:18:35Z","timestamp":1777457915000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0278364914537359"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,8]]},"references-count":51,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2014,8]]}},"alternative-id":["10.1177\/0278364914537359"],"URL":"https:\/\/doi.org\/10.1177\/0278364914537359","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,8]]}}}