{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T10:52:47Z","timestamp":1777891967293,"version":"3.51.4"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,9,8]],"date-time":"2022-09-08T00:00:00Z","timestamp":1662595200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100008902","name":"Los Alamos National Laboratory","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100008902","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001942","name":"CHIST-ERA","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001942","id-type":"DOI","asserted-by":"crossref"}]},{"name":"InDex, Estonian Centre of Excellence in IT"},{"name":"IT Academy program in Estonia","award":["COVSG24 and PSG605"],"award-info":[{"award-number":["COVSG24 and PSG605"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2022,12,31]]},"abstract":"<jats:p>\n            To facilitate continuous development of novel HRI systems, it is beneficial to have tools that enable quick adjustments, flexibility, or re-invention of the human interfaces when system requirements change due to updates in the state-of-the-art, application domain, etc. Thus, modularity is a key design principle which promotes software reuse and scalability, and reduces development time and cost. Hence, a robot\u2019s autonomous capabilities should not depend on the command interface and should be decoupled via a common format that possesses the descriptive capabilities for outlining tasks and has a sensible syntax for HRI. In this paper, we propose the\n            <jats:bold>Unified Meaning Representation Format (UMRF)<\/jats:bold>\n            , which provides the syntax and semantics for passing both simple and complex commands modelled as control flow graphs. UMRF is a standalone meaning representation container that supports embedding other meaning representation formalisms, such as predicate-argument semantics and graphical meaning representation formats, making it adoptable as a standard task description format for semi-autonomous systems in HRI domains. In this article, we define the UMRF syntax and semantics, summarize its unique aspects relative to related task description formats, and demonstrate its descriptiveness by navigating a robot via concurrent (e.g., gestures and speech) and interchangeable input systems (e.g., Google Assistant, Amazon Alexa).\n          <\/jats:p>","DOI":"10.1145\/3522580","type":"journal-article","created":{"date-parts":[[2022,3,26]],"date-time":"2022-03-26T11:19:37Z","timestamp":1648293577000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Unified Meaning Representation Format (UMRF) - A Task Description and Execution Formalism for HRI"],"prefix":"10.1145","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0174-3116","authenticated-orcid":false,"given":"Robert","family":"Valner","sequence":"first","affiliation":[{"name":"University of Tartu, Tartu, Estonia"}]},{"given":"Selma","family":"Wanna","sequence":"additional","affiliation":[{"name":"University of Texas at Austin, Austin, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1720-1509","authenticated-orcid":false,"given":"Karl","family":"Kruusam\u00e4e","sequence":"additional","affiliation":[{"name":"University of Tartu, Tartu, Estonia"}]},{"given":"Mitch","family":"Pryor","sequence":"additional","affiliation":[{"name":"University of Texas at Austin, Austin, USA"}]}],"member":"320","published-online":{"date-parts":[[2022,9,8]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISR.2013.6695630"},{"key":"e_1_3_3_3_2","unstructured":"Accessed 26\/07\/2021. PyTransitions. https:\/\/github.com\/pytransitions\/transitions."},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1008"},{"key":"e_1_3_3_5_2","doi-asserted-by":"crossref","unstructured":"Peter Anderson Basura Fernando Mark Johnson and Stephen Gould. 2016. SPICE: Semantic propositional image caption evaluation. (7 2016).","DOI":"10.1007\/978-3-319-46454-1_24"},{"key":"e_1_3_3_6_2","first-page":"86","volume-title":"ACL\u201998\/COLING\u201998 Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics","author":"Baker Colin","year":"1983","unstructured":"Colin Baker, Charles Fillmore, and John Lowe. 1983. The Berkeley FrameNet project. In ACL\u201998\/COLING\u201998 Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics. 86\u201390."},{"key":"e_1_3_3_7_2","first-page":"178","article-title":"Abstract meaning representation for Sembanking","author":"Banarescu Laura","year":"2013","unstructured":"Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract meaning representation for Sembanking. LAW 2013 and ID 2013-7th Linguistic Annotation Workshop and Interoperability with Discourse, Proceedings of the Workshop (2013), 178\u2013186.","journal-title":"LAW 2013 and ID 2013-7th Linguistic Annotation Workshop and Interoperability with Discourse, Proceedings of the Workshop"},{"key":"e_1_3_3_8_2","first-page":"4519","article-title":"HuRIC: A human robot interaction corpus","author":"Bastianelli Emanuele","year":"2014","unstructured":"Emanuele Bastianelli, Giuseppe Castellucci, Danilo Croce, Luca Iocchi, Roberto Basili, and Daniele Nardi. 2014. HuRIC: A human robot interaction corpus. Proceedings of the 9th International Conference on Language Resources and Evaluation, LREC 2014 (2014), 4519\u20134526.","journal-title":"Proceedings of the 9th International Conference on Language Resources and Evaluation, LREC 2014"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364918755924"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2010.938836"},{"key":"e_1_3_3_11_2","doi-asserted-by":"crossref","unstructured":"Claire Bonial Lucia Donatelli Mitchell Abrams Stephanie M. Lukin Stephen Tratz Matthew Marge Ron Artstein David Traum and Clare Voss. 2020. Dialogue-AMR: Abstract Meaning Representation for Dialogue 684\u2013695.","DOI":"10.18653\/v1\/W19-3322"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-3322"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2005.1545523"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2016.7759506"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3314221.3314594"},{"key":"e_1_3_3_16_2","article-title":"OpenPose: Realtime multi-person 2D pose estimation using part affinity fields","author":"Cao Zhe","year":"2018","unstructured":"Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2018. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. arXiv preprint arXiv:1812.08008 (2018).","journal-title":"arXiv preprint arXiv:1812.08008"},{"key":"e_1_3_3_17_2","article-title":"Component based development and advanced OO design. Whitepaper","author":"Collins-Cope M.","year":"2001","unstructured":"M. Collins-Cope. 2001. Component based development and advanced OO design. Whitepaper. Ratio Group Ltd (2001).","journal-title":"Ratio Group Ltd"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCSE.2019.8845336"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICECCS.2007.7"},{"key":"e_1_3_3_20_2","unstructured":"Shibhansh Dohare Harish Karnick and Vivek Gupta. 2017. Text Summarization Using Abstract Meaning Representation. arXiv:cs.CL\/1706.01678."},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/CACSD-CCA-ISIC.2006.4776814"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1177\/1557234X13510679"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1129"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377812.3382143"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/1925805.1925814"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1300"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40708-016-0042-6"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2019.00105"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3208975"},{"key":"e_1_3_3_30_2","volume-title":"Speech and Language Processing 3rd ed. Draft","author":"Jurafsky Dan","year":"2019","unstructured":"Dan Jurafsky. 2019. Speech and Language Processing 3rd ed. Draft. 295\u2013323 pages."},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2909437.2909446"},{"key":"e_1_3_3_32_2","volume-title":"Natural Language For Human Robot Interaction","author":"Khayrallah Huda","year":"2015","unstructured":"Huda Khayrallah, Sean Trott, and Jerome Feldman. 2015. Natural Language For Human Robot Interaction. Technical Report."},{"key":"e_1_3_3_33_2","unstructured":"Markus Klotzb\u00fccher and Herman Bruyninckx. 2012. Coordinating robotic tasks and systems with rFSM statecharts. (2012)."},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-3022"},{"key":"e_1_3_3_35_2","article-title":"Generalized grounding graphs: A probabilistic framework for understanding grounded commands","volume":"1712","author":"Kollar Thomas","year":"2017","unstructured":"Thomas Kollar, Stefanie Tellex, Matthew R. Walter, Albert Huang, Abraham Bachrach, Sachithra Hemachandra, Emma Brunskill, Ashis Gopal Banerjee, Deb Roy, Seth J. Teller, and Nicholas Roy. 2017. Generalized grounding graphs: A probabilistic framework for understanding grounded commands. CoRR abs\/1712.01097 (2017). arXiv:1712.01097 http:\/\/arxiv.org\/abs\/1712.01097.","journal-title":"CoRR"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.5555\/3271461"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2020.2983930"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3385009"},{"key":"e_1_3_3_39_2","doi-asserted-by":"crossref","unstructured":"C. Matuszek L. Bo L. Zettlemoyer and D. Fox. 2014. Learning from Unscripted Deictic Gesture and Language for Human-Robot Interactions. Proceedings of the AAAI Conference on Artificial Intelligence . 28 1 (Jun. 2014).","DOI":"10.1609\/aaai.v28i1.9051"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21439"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2013.6630616"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1162\/0891201053630264"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/1764810.1764828"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.21437\/AI-MHRI.2018-15"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.5772\/56906"},{"key":"e_1_3_3_46_2","volume-title":"A Task Scheduler for ROS","author":"Pradalier C\u00e9dric","year":"2017","unstructured":"C\u00e9dric Pradalier. 2017. A Task Scheduler for ROS. Research Report. UMI 2958 GeorgiaTech-CNRS. https:\/\/hal.archives-ouvertes.fr\/hal-01435823. ROBOTICS."},{"key":"e_1_3_3_47_2","doi-asserted-by":"crossref","unstructured":"Sarah Pratt Mark Yatskar Luca Weihs Ali Farhadi and Aniruddha Kembhavi. 2020. Grounded situation recognition. (2020). arXiv:cs.CV\/2003.12058.","DOI":"10.1007\/978-3-030-58548-8_19"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-54927-9_4"},{"key":"e_1_3_3_49_2","unstructured":"J. Ruppenhofer M. Ellsworth M. R. L. Petruck C. R. Johnson and J. Scheffczyk. 2006. {FrameNet} {II:} Extended theory and practice. 1 (2006)."},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487442"},{"key":"e_1_3_3_51_2","volume-title":"VerbNet: A Broad-coverage, Comprehensive Verb Lexicon","author":"Schuler Karin K.","year":"2005","unstructured":"Karin K. Schuler. 2005. VerbNet: A Broad-coverage, Comprehensive Verb Lexicon. Ph.D. Dissertation. Copyright - Database copyright ProQuest LLC; ProQuest does not claim copyright in the individual underlying works; Last updated - 2019-10-18."},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.smhl.2018.03.002"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1186\/s40648-020-00177-y"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1080\/14494035.2020.1782627"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2012.6225172"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/EUROMICRO.2005.18"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2012.6225145"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2022.3173647"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3102113.3102147"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2019.103181"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10846-020-01234-9"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2016.00033"},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/p19-1009"}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3522580","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3522580","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3522580","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:09:33Z","timestamp":1750183773000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3522580"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,8]]},"references-count":62,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,12,31]]}},"alternative-id":["10.1145\/3522580"],"URL":"https:\/\/doi.org\/10.1145\/3522580","relation":{},"ISSN":["2573-9522","2573-9522"],"issn-type":[{"value":"2573-9522","type":"print"},{"value":"2573-9522","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,8]]},"assertion":[{"value":"2020-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-09-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}