{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T11:21:46Z","timestamp":1780053706750,"version":"3.54.0"},"reference-count":49,"publisher":"SAGE Publications","issue":"12-13","license":[{"start":{"date-parts":[[2020,4,12]],"date-time":"2020-04-12T00:00:00Z","timestamp":1586649600000},"content-version":"vor","delay-in-days":366,"URL":"http:\/\/www.sagepub.com\/licence-information-for-chorus"}],"funder":[{"DOI":"10.13039\/100000185","name":"Defense Advanced Research Projects Agency","doi-asserted-by":"publisher","award":["D15AP00102"],"award-info":[{"award-number":["D15AP00102"]}],"id":[{"id":"10.13039\/100000185","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006602","name":"Air Force Research Laboratory","doi-asserted-by":"publisher","award":["FA9550-17-1-0124"],"award-info":[{"award-number":["FA9550-17-1-0124"]}],"id":[{"id":"10.13039\/100006602","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2019,10]]},"abstract":"<jats:p>Efficient motion intent communication is necessary for safe and collaborative work environments with co-located humans and robots. Humans efficiently communicate their motion intent to other humans through gestures, gaze, and other non-verbal cues, and can replan their motions in response. However, robots often have difficulty using these methods. Many existing methods for robot motion intent communication rely on 2D displays, which require the human to continually pause their work to check a visualization. We propose a mixed-reality head-mounted display (HMD) visualization of the intended robot motion over the wearer\u2019s real-world view of the robot and its environment. In addition, our interface allows users to adjust the intended goal pose of the end effector using hand gestures. We describe its implementation, which connects a ROS-enabled robot to the HoloLens using ROS Reality, using MoveIt for motion planning, and using Unity to render the visualization. To evaluate the effectiveness of this system against a 2D display visualization and against no visualization, we asked 32 participants to label various arm trajectories as either colliding or non-colliding with blocks arranged on a table. We found a 15% increase in accuracy with a 38% decrease in the time it took to complete the task compared with the next best system. These results demonstrate that a mixed-reality HMD allows a human to determine where the robot is going to move more quickly and accurately than existing baselines.<\/jats:p>","DOI":"10.1177\/0278364919842925","type":"journal-article","created":{"date-parts":[[2019,4,12]],"date-time":"2019-04-12T07:50:45Z","timestamp":1555055445000},"page":"1513-1526","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":113,"title":["Communicating and controlling robot arm motion intent through mixed-reality head-mounted displays"],"prefix":"10.1177","volume":"38","author":[{"given":"Eric","family":"Rosen","sequence":"first","affiliation":[{"name":"Humans To Robots Laboratory, Department of Computer Science, Providence, RI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Whitney","sequence":"additional","affiliation":[{"name":"Humans To Robots Laboratory, Department of Computer Science, Providence, RI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elizabeth","family":"Phillips","sequence":"additional","affiliation":[{"name":"Humans To Robots Laboratory, Department of Computer Science, Providence, RI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gary","family":"Chien","sequence":"additional","affiliation":[{"name":"Humans To Robots Laboratory, Department of Computer Science, Providence, RI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James","family":"Tompkin","sequence":"additional","affiliation":[{"name":"Humans To Robots Laboratory, Department of Computer Science, Providence, RI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"George","family":"Konidaris","sequence":"additional","affiliation":[{"name":"Humans To Robots Laboratory, Department of Computer Science, Providence, RI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefanie","family":"Tellex","sequence":"additional","affiliation":[{"name":"Humans To Robots Laboratory, Department of Computer Science, Providence, RI, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2019,4,12]]},"reference":[{"key":"bibr1-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/SII.2016.7844004"},{"key":"bibr2-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2016.7745145"},{"key":"bibr3-0278364919842925","unstructured":"Bischoff M (2017) ROS Sharp. https:\/\/github.com\/siemens\/ros-sharp."},{"issue":"194","key":"bibr4-0278364919842925","first-page":"4","volume":"189","author":"Brooke J","year":"1996","journal-title":"Usability Evaluation in Industry"},{"key":"bibr5-0278364919842925","unstructured":"Burke JL, Murphy RR (2004) Situation awareness and task performance in robot-assisted technical search: Bujold goes to Bridgeport. Technical Report CRASAR-TR2004-23, Center for Robot-Assisted Search and Rescue, Tampa, FL."},{"key":"bibr6-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1207\/s15327051hci1901&2_5"},{"key":"bibr7-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1561\/2300000057"},{"key":"bibr8-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/ECMR.2015.7403771"},{"key":"bibr9-0278364919842925","volume-title":"RSS Workshop on Social Trust in Autonomous Robots","author":"Chadalavada RT","year":"2016"},{"key":"bibr10-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1145\/2814347.2814350"},{"key":"bibr11-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2011.2181749"},{"key":"bibr12-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2006.42"},{"key":"bibr13-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/HRI.2013.6483603"},{"key":"bibr14-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(02)00372-X"},{"key":"bibr15-0278364919842925","unstructured":"Han Y (2016) The Social Behavior Guide for Confused Autonomous Machines. Master\u2019s Thesis, Rhode Island School of Design. https:\/\/issuu.com\/horatiohan\/docs\/the_soical_behavior_guide_for_confu"},{"key":"bibr16-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1007\/s11235-015-0034-5"},{"key":"bibr17-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2002.999788"},{"key":"bibr18-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/IWAR.1999.803809"},{"key":"bibr19-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1145\/2157689.2157691"},{"key":"bibr20-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1002\/0471214426.pas0402"},{"key":"bibr21-0278364919842925","unstructured":"Microsoft (2017) MixedRealityToolKit. https:\/\/github.com\/Microsoft\/MixedRealityToolkit-Unity"},{"key":"bibr22-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.1993.583833"},{"key":"bibr23-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1145\/1514095.1514110"},{"key":"bibr24-0278364919842925","volume-title":"International Conference on Autonomous Systems","author":"Nakata T","year":"1998"},{"key":"bibr25-0278364919842925","unstructured":"NASA Human Performance Research Group and others (1987) Task Load Index (NASA-TLX) v1.0 Computerised Version. NASA Ames Research Centre."},{"key":"bibr26-0278364919842925","doi-asserted-by":"crossref","unstructured":"Nikolaidis S, Kwon M, Forlizzi J, Srinivasa S (2017) Planning with verbal communication for human\u2013robot collaboration.\n                      arXiv preprint arXiv:1706.04694\n                      .","DOI":"10.1145\/3203305"},{"key":"bibr27-0278364919842925","first-page":"268","volume-title":"Proceedings IEEE 1998 Virtual Reality Annual International Symposium, 1998","author":"Ohshima T","year":"1998"},{"key":"bibr28-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/VRAIS.1993.378265"},{"key":"bibr29-0278364919842925","first-page":"5","volume-title":"ICRA Workshop on Open Source Software","volume":"3","author":"Quigley M","year":"2009"},{"key":"bibr30-0278364919842925","first-page":"18","volume":"96","author":"Rekimoto J","year":"1996","journal-title":"Virtual Systems and Multimedia"},{"key":"bibr31-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1162\/105474699566143"},{"key":"bibr32-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-40621-3_35"},{"key":"bibr33-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-008-0223-2"},{"key":"bibr34-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1145\/2685328.2685335"},{"key":"bibr35-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1016\/j.cogsys.2017.02.002"},{"key":"bibr36-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/HRI.2016.7451837"},{"key":"bibr37-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/AIM.2016.7577007"},{"key":"bibr38-0278364919842925","first-page":"72","volume":"163","author":"Slater M","year":"1996","journal-title":"ACM Virtual Reality Software and Technology"},{"key":"bibr39-0278364919842925","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2016.00074"},{"key":"bibr40-0278364919842925","doi-asserted-by":"publisher","DOI":"10.3758\/BF03207704"},{"key":"bibr41-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1145\/2559636.2559672"},{"key":"bibr42-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1145\/2696454.2696475"},{"key":"bibr43-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1145\/1957656.1957674"},{"key":"bibr44-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1037\/h0058700"},{"key":"bibr45-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2015.7354021"},{"key":"bibr46-0278364919842925","unstructured":"Unity Technologies (2018) Unity Software. https:\/\/unity.com"},{"key":"bibr47-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1145\/3171221.3171253"},{"key":"bibr48-0278364919842925","doi-asserted-by":"publisher","DOI":"10.1109\/VL.1994.363621"},{"key":"bibr49-0278364919842925","volume-title":"International Symposium on Robotics Research","author":"Whitney D","year":"2017"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364919842925","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0278364919842925","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364919842925","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364919842925","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:15:54Z","timestamp":1777457754000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0278364919842925"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,12]]},"references-count":49,"journal-issue":{"issue":"12-13","published-print":{"date-parts":[[2019,10]]}},"alternative-id":["10.1177\/0278364919842925"],"URL":"https:\/\/doi.org\/10.1177\/0278364919842925","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,4,12]]}}}