{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T17:53:03Z","timestamp":1781545983986,"version":"3.54.5"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T00:00:00Z","timestamp":1781481600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2026,6,15]]},"abstract":"<jats:p>\n                    End-users of immersive Augmented Reality (AR) applications struggle to discover and execute unfamiliar mid-air gestures. We introduce ARG\n                    <jats:sc>esture<\/jats:sc>\n                    A\n                    <jats:sc>id<\/jats:sc>\n                    , a novel proof-of-concept conversational assistant that provides adaptive, context-aware assistance tailored to users' tasks and gestural errors. By combining users' voice-based task descriptions with formalized representations of hand gesture data together with LLMs, ARG\n                    <jats:sc>esture<\/jats:sc>\n                    A\n                    <jats:sc>id<\/jats:sc>\n                    dynamically supports users through: (1)\n                    <jats:italic toggle=\"yes\">feedforward<\/jats:italic>\n                    , directing users toward correct pose, position, and placement before initiating gestures, and (2)\n                    <jats:italic toggle=\"yes\">corrective feedback<\/jats:italic>\n                    , offering targeted, adaptive instructions to address user errors during discovery and execution. Our initial feasibility study (N=15), which compared users' real-world interaction data against a manually labeled reference dataset, shows that ARG\n                    <jats:sc>esture<\/jats:sc>\n                    A\n                    <jats:sc>id<\/jats:sc>\n                    achieved an overall accuracy of 81.3%. Our follow-up user study (N=15) evaluated ARG\n                    <jats:sc>esture<\/jats:sc>\n                    A\n                    <jats:sc>id<\/jats:sc>\n                    against a BASELINE LLM-based voice assistant across unimanual and bimanual gestures. Results indicate that ARG\n                    <jats:sc>esture<\/jats:sc>\n                    A\n                    <jats:sc>id<\/jats:sc>\n                    achieved higher task completion rates and was rated more favorably for perceived discoverability and usefulness. These findings suggest that adaptive, in-context feedforward and corrective feedback, grounded in formal gesture representations and observed error patterns, may support users during mid-air gesture discovery and execution, and help them refine their understanding of required gestures. We discuss key considerations for designing future adaptive, context-aware AR systems in AR environments.\n                  <\/jats:p>","DOI":"10.1145\/3810212","type":"journal-article","created":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T17:06:41Z","timestamp":1781543201000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["ARGESTUREAID: A Voice-Based, Adaptive, and Context-Aware Conversational Assistant for Supporting Mid-Air Gesture Discovery and Execution"],"prefix":"10.1145","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2730-5512","authenticated-orcid":false,"given":"Anjali","family":"Khurana","sequence":"first","affiliation":[{"name":"Computing Science, Simon Fraser University, Burnaby, British Columbia, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8934-7761","authenticated-orcid":false,"given":"Amy","family":"Karlson","sequence":"additional","affiliation":[{"name":"Reality Labs Research, Meta Inc., Redmond, Washington, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4520-7000","authenticated-orcid":false,"given":"Christopher","family":"Collins","sequence":"additional","affiliation":[{"name":"Meta Reality Labs Research, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6105-1272","authenticated-orcid":false,"given":"Mengjie","family":"Yu","sequence":"additional","affiliation":[{"name":"Reality Labs Research, Meta Inc., Redmond, Washington, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8581-4921","authenticated-orcid":false,"given":"Vinay","family":"Jayaram","sequence":"additional","affiliation":[{"name":"Meta Reality Labs, New York, New York, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2059-3558","authenticated-orcid":false,"given":"Hrvoje","family":"Benko","sequence":"additional","affiliation":[{"name":"Reality Labs Research, Meta Inc., Redmond, Washington, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-0173-1752","authenticated-orcid":false,"given":"Parmit K.","family":"Chilana","sequence":"additional","affiliation":[{"name":"Computing Science, Simon Fraser University, Burnaby, British Columbia, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,15]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"2020. Get holomeeting 2. https:\/\/www.microsoft.com\/en-us\/p\/holomeeting-2\/9nh5zdb4ggbq?activetab=pivot%3Aoverviewtab [Accessed 21-04-2026]."},{"key":"e_1_2_2_2_1","unstructured":"2024. Figma AI: Your Creativity unblocked with Figma AI \u2014 figma.com. https:\/\/www.figma.com\/ai\/. [Accessed 11-09-2024]."},{"key":"e_1_2_2_3_1","unstructured":"2025. ChatGPT Voice mode \u2014 chatgpt.com. https:\/\/chatgpt.com\/features\/voice\/. [Accessed 30-10-2025]."},{"key":"e_1_2_2_4_1","unstructured":"2025. Learn more about Meta AI on Meta Quest | Quest Help | Meta Store \u2014 meta.com. https:\/\/www.meta.com\/help\/quest\/ 1008196844074415\/?srsltid=AfmBOorUaGyNAf_fDGJKAzsM297_c3XjrTp1KsvfW2v-Vxuc45Af3lLC. [Accessed 30-10-2025]."},{"key":"e_1_2_2_5_1","unstructured":"2025. OpenAI Platform \u2014 platform.openai.com. https:\/\/platform.openai.com\/docs\/guides\/realtime-vad. [Accessed 30-10-2025]."},{"key":"e_1_2_2_6_1","unstructured":"2025. OVRSkeleton Class | Unity: Core SDK v69 \u2014 developers.meta.com. https:\/\/developers.meta.com\/horizon\/reference\/unity\/v69\/class_o_v_r_skeleton\/. [Accessed 23-04-2025]."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISCV60512.2024.10620105"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/0953-5438(92)90021-7"},{"key":"e_1_2_2_9_1","unstructured":"Adobe. 2023. Adobe unveils Firefly a family of new creative generative ai. https:\/\/news.adobe.com\/news\/news-details\/2023\/Adobe-Unveils-Firefly-a-Family-of-new-Creative-Generative-AI\/default.aspx [Accessed 21-09-2024]."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3205873.3205887"},{"key":"e_1_2_2_11_1","unstructured":"Anthropic. 2024. Model Context Protocol (MCP) - Claude Docs \u2014 docs.claude.com. https:\/\/docs.claude.com\/en\/docs\/mcp. [Accessed 30-10-2025]."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208339"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1449715.1449724"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.3389\/frvir.2021.578080"},{"key":"e_1_2_2_15_1","first-page":"2","article-title":"2015. A survey of augmented reality","volume":"8","author":"Billinghurst Mark","year":"2015","unstructured":"Mark Billinghurst, Adrian Clark, Gun Lee, et al. 2015. A survey of augmented reality. Foundations and Trends\u00ae in Human-Computer Interaction 8, 2\u20133 (2015), 73\u2013272.","journal-title":"Foundations and Trends\u00ae in Human-Computer Interaction"},{"key":"e_1_2_2_16_1","volume-title":"Augmented and Mixed Reality: 9th International Conference, VAMR 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9-14, 2017, Proceedings 9. Springer, 215\u2013226","author":"Bozgeyikli Lal","year":"2017","unstructured":"Lal Bozgeyikli, Andrew Raij, Srinivas Katkoori, and Redwan Alqasemi. 2017. Effects of instruction methods on user experience in virtual reality serious games. In Virtual, Augmented and Mixed Reality: 9th International Conference, VAMR 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9-14, 2017, Proceedings 9. Springer, 215\u2013226."},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.3389\/frvir.2021.728461"},{"key":"e_1_2_2_18_1","volume-title":"Blind variation and selective retentions in creative thought as in other knowledge processes. Psychological review 67, 6","author":"Campbell Donald T","year":"1960","unstructured":"Donald T Campbell. 1960. Blind variation and selective retentions in creative thought as in other knowledge processes. Psychological review 67, 6 (1960), 380."},{"key":"e_1_2_2_19_1","volume-title":"Carroll and Mary Beth Rosson","author":"John","year":"1987","unstructured":"John M. Carroll and Mary Beth Rosson. 1987. Paradox of the Active User. MIT Press, Cambridge, MA, USA, 80\u2013111."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581211"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/266180.266328"},{"key":"e_1_2_2_22_1","volume-title":"Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative sociology 13, 1","author":"Corbin Juliet M","year":"1990","unstructured":"Juliet M Corbin and Anselm Strauss. 1990. Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative sociology 13, 1 (1990), 3\u201321."},{"key":"e_1_2_2_23_1","volume-title":"ACM CHI Conference on Human Factors in Computing Systems (January","author":"Cranshaw Justin","year":"2017","unstructured":"Justin Cranshaw, Emad Elwany, Todd Newman, Rafal Kocielnik, Bowen Yu, Sandeep Soni, Jaime Teevan, and Andr\u00e9s Monroy-Hern\u00e1ndez. 2017. Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop. ACM CHI Conference on Human Factors in Computing Systems (January 2017). https:\/\/www.microsoft.com\/en-us\/research\/publication\/calendar-help-designing-workflow-based-scheduling-agent-humans-loop\/"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2901790.2901834"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0295817"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3532106.3533455"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3385378.3385384"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1378773.1378804"},{"key":"e_1_2_2_29_1","unstructured":"Ari Grobman. 2018. AR's success depends on perfecting input methods. https:\/\/thenextweb.com\/news\/ars-success-depends-on-perfecting-input-methods"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1519048"},{"key":"e_1_2_2_31_1","volume-title":"Horrigan and Sydney Jones","author":"John","year":"2020","unstructured":"John B. Horrigan and Sydney Jones. 2020. When technology fails. https:\/\/www.pewresearch.org\/internet\/2008\/11\/16\/when-technology-fails\/"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581420"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/VL\/HCC51201.2021.9576440"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/VL\/HCC51201.2021.9576440"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3631451"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713431"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3640543.3645200"},{"key":"e_1_2_2_38_1","volume-title":"Biorxiv (2024)","author":"Reality Labs Ctrl","year":"2024","unstructured":"Ctrl labs at Reality Labs, David Sussillo, Patrick Kaifosh, and Thomas Reardon. 2024. A generic noninvasive neuromotor interface for human-computer interaction. Biorxiv (2024), 2024\u201302."},{"key":"e_1_2_2_39_1","volume-title":"2016 IEEE Third VR International Workshop on Collaborative Virtual Environments (3DCVE). IEEE, 9\u201312","author":"Ch\u00e9n\u00e9chal Morgan Le","year":"2016","unstructured":"Morgan Le Ch\u00e9n\u00e9chal, Thierry Duval, Val\u00e9rie Gouranton, J\u00e9r\u00f4me Royan, and Bruno Arnaldi. 2016. Vishnu: virtual immersive support for helping users an interaction paradigm for collaborative remote guiding in mixed reality. In 2016 IEEE Third VR International Workshop on Collaborative Virtual Environments (3DCVE). IEEE, 9\u201312."},{"key":"e_1_2_2_40_1","volume-title":"Augmenting reality to diminish distractions for cognitive enhancement. arXiv preprint arXiv:2403.03875","author":"Lee JangHyeon","year":"2024","unstructured":"JangHyeon Lee and Lawrence H Kim. 2024. Augmenting reality to diminish distractions for cognitive enhancement. arXiv preprint arXiv:2403.03875 (2024)."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3462244.3479902"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642230"},{"key":"e_1_2_2_43_1","volume-title":"Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters. In The Thirteenth International Conference on Learning Representations.","author":"Li Kevin","year":"2024","unstructured":"Kevin Li, Sachin Goyal, Jo\u00e3o D Semedo, and J Zico Kolter. 2024. Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters. In The Thirteenth International Conference on Learning Representations."},{"key":"e_1_2_2_44_1","volume-title":"AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap. arXiv preprint arXiv:2306.01941","author":"Vera Liao Q","year":"2023","unstructured":"Q Vera Liao and Jennifer Wortman Vaughan. 2023. AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap. arXiv preprint arXiv:2306.01941 (2023)."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR59233.2023.00117"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1463160.1463190"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858288"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3530886"},{"key":"e_1_2_2_49_1","volume-title":"\u201cfirst hand","author":"Introducing","unstructured":"Meta. 2022. Introducing \u201cfirst hand,\u201d our official hand tracking demo built with presence platform's interaction SDK. https:\/\/developer.oculus.com\/blog\/introducing-first-hand\/"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3603623"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/1124772.1124832"},{"key":"e_1_2_2_52_1","unstructured":"Donald Norman. 1990. A.(1990) The Design of Everyday Things."},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2513383.2513455"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174214"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3714310"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/1040830.1040893"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.4324\/9781315542072-29"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3526113.3545628"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3654777.3676378"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.3390\/s21175856"},{"key":"e_1_2_2_61_1","unstructured":"Jared Spataro. 2023. Introducing Microsoft 365 copilot - your copilot for work. https:\/\/blogs.microsoft.com\/blog\/2023\/03\/16\/introducing-microsoft-365-copilot-your-copilot-for-work\/"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3332165.3347872"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2466255"},{"key":"e_1_2_2_64_1","unstructured":"VRChat.2025.VRChat-hello.vrchat.com. https:\/\/hello.vrchat.com\/. [Accessed 22-04-2025]."},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308561.3353773"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.3390\/infol3120566"},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642935"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/30851.275627"},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.5555\/3600270.3602070"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501979"},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3643834.3660691"},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3503537"},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3698145"},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/302979.303053"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","unstructured":"Yuchong Zhang Adam Nowak Yueming Xuan Andrzej Romanowski and Morten Fjeld. 2023. See or Hear? Exploring the Effect of Visual\/Audio Hints and Gaze-assisted Instant Post-task Feedback for Visual Search Tasks in AR. In 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 1113\u20131122. doi:10.1109\/ISMAR59233.2023.00128","DOI":"10.1109\/ISMAR59233.2023.00128"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3810212","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T17:07:26Z","timestamp":1781543246000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3810212"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,15]]},"references-count":75,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,6,15]]}},"alternative-id":["10.1145\/3810212"],"URL":"https:\/\/doi.org\/10.1145\/3810212","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,15]]},"assertion":[{"value":"2026-06-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}