{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T05:01:41Z","timestamp":1784696501170,"version":"3.55.0"},"reference-count":80,"publisher":"Association for Computing Machinery (ACM)","issue":"EICS","license":[{"start":{"date-parts":[[2021,5,27]],"date-time":"2021-05-27T00:00:00Z","timestamp":1622073600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Research Grants Council of Hong Kongand the 5GEAR and FIT projects from Academy of Finland","award":["16214817"],"award-info":[{"award-number":["16214817"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,5,27]]},"abstract":"<jats:p>The continued advancement in user interfaces comes to the era of virtual reality that requires a better understanding of how users will interact with 3D buttons in mid-air. Although virtual reality owns high levels of expressiveness and demonstrates the ability to simulate the daily objects in the physical environment, the most fundamental issue of designing virtual buttons is surprisingly ignored. To this end, this paper presents four variants of virtual buttons, considering two design dimensions of key representations and multi-modal cues (audio, visual, haptic). We conduct two multi-metric assessments to evaluate the four virtual variants and the baselines of physical variants. Our results indicate that the 3D-lookalike buttons help users with more refined and subtle mid-air interactions (i.e. lesser press depth) when haptic cues are available; while the users with 2D-lookalike buttons unintuitively achieve better keystroke performance than the 3D counterparts. We summarize the findings, and accordingly, suggest the design choices of virtual reality buttons among the two proposed design dimensions.<\/jats:p>","DOI":"10.1145\/3457141","type":"journal-article","created":{"date-parts":[[2021,5,30]],"date-time":"2021-05-30T01:31:19Z","timestamp":1622338279000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":41,"title":["Exploring Button Designs for Mid-air Interaction in Virtual Reality: A Hexa-metric Evaluation of Key Representations and Multi-modal Cues"],"prefix":"10.1145","volume":"5","author":[{"given":"Carlos","family":"Bermejo","sequence":"first","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lik Hang","family":"Lee","sequence":"additional","affiliation":[{"name":"KAIST, South Korea and The University of Oulu, South Korea, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul","family":"Chojecki","sequence":"additional","affiliation":[{"name":"Fraunhofer Heinrich Hertz Institute HHI, Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Przewozny","sequence":"additional","affiliation":[{"name":"Fraunhofer Heinrich Hertz Institute HHI, Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pan","family":"Hui","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology &amp; University of Helsinki, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,5,29]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359996.3364246"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/HAVE.2015.7359447"},{"key":"e_1_2_1_3_1","volume-title":"Universal Access in Human-Computer Interaction. Access to Today's Technologies","author":"Berget Gerd","unstructured":"Gerd Berget and Frode Eika Sandnes . 2015. On the Understandability of Public Domain Icons: Effects of Gender and Age . In Universal Access in Human-Computer Interaction. Access to Today's Technologies , Margherita Antona and Constantine Stephanidis (Eds.). Springer International Publishing , Cham , 387--396. Gerd Berget and Frode Eika Sandnes. 2015. On the Understandability of Public Domain Icons: Effects of Gender and Age. In Universal Access in Human-Computer Interaction. Access to Today's Technologies, Margherita Antona and Constantine Stephanidis (Eds.). Springer International Publishing, Cham, 387--396."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2002.1011262"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2933242.2935874"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2330667.2330687"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1177\/154193120204602611"},{"key":"e_1_2_1_8_1","volume-title":"Benjamin Long, Bruce Drinkwater, and Sriram Subramanian.","author":"Carter Tom","year":"2013","unstructured":"Tom Carter , Sue Ann Seah , Benjamin Long, Bruce Drinkwater, and Sriram Subramanian. 2013 a. UltraHaptics : Multi- Point Mid-Air Haptic Feedback for Touch Surfaces . (2013). Tom Carter, Sue Ann Seah, Benjamin Long, Bruce Drinkwater, and Sriram Subramanian. 2013a. UltraHaptics : Multi-Point Mid-Air Haptic Feedback for Touch Surfaces. (2013)."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2501988.2502018"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1080\/00140130500197260"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1943403.1943445"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2380296.2380308"},{"key":"e_1_2_1_13_1","volume-title":"Proc. EuroHaptics. 41--46","author":"Doerrer C","year":"2002","unstructured":"C Doerrer and R Werthschuetzky . 2002 . Simulating push-buttons using a haptic display: Requirements on force resolution and force-displacement curve . In Proc. EuroHaptics. 41--46 . C Doerrer and R Werthschuetzky. 2002. Simulating push-buttons using a haptic display: Requirements on force resolution and force-displacement curve. In Proc. EuroHaptics. 41--46."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-22643-5_33"},{"key":"e_1_2_1_15_1","volume-title":"Performance Envelopes of Virtual Keyboard Text Input Strategies in Virtual Reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 289--300","author":"Dudley John","year":"2019","unstructured":"John Dudley , Hrvoje Benko , Daniel Wigdor , and Per Ola Kristensson . 2019 . Performance Envelopes of Virtual Keyboard Text Input Strategies in Virtual Reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 289--300 . John Dudley, Hrvoje Benko, Daniel Wigdor, and Per Ola Kristensson. 2019. Performance Envelopes of Virtual Keyboard Text Input Strategies in Virtual Reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 289--300."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2947617"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2535923"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/HAPTICS.2016.7463168"},{"key":"e_1_2_1_19_1","volume-title":"New Trends on Human-Computer Interaction","author":"Gonz\u00e1lez Gabriel","unstructured":"Gabriel Gonz\u00e1lez , Jos\u00e9 P Molina , Arturo S Garc'ia , Diego Mart'inez , and Pascual Gonz\u00e1lez . 2009. Evaluation of text input techniques in immersive virtual environments . In New Trends on Human-Computer Interaction . Springer , 109--118. Gabriel Gonz\u00e1lez, Jos\u00e9 P Molina, Arturo S Garc'ia, Diego Mart'inez, and Pascual Gonz\u00e1lez. 2009. Evaluation of text input techniques in immersive virtual environments. In New Trends on Human-Computer Interaction. Springer, 109--118."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300603"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300715"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.apergo.2014.04.001"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357300"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/WHC.2011.5945548"},{"key":"e_1_2_1_25_1","volume-title":"Rehabilitation Robotics (ICORR), 2011 IEEE International Conference on. IEEE, 1--6.","author":"In HyunKi","year":"2011","unstructured":"HyunKi In , Kyu-Jin Cho , KyuRi Kim , and BumSuk Lee . 2011 . Jointless structure and under-actuation mechanism for compact hand exoskeleton . In Rehabilitation Robotics (ICORR), 2011 IEEE International Conference on. IEEE, 1--6. HyunKi In, Kyu-Jin Cho, KyuRi Kim, and BumSuk Lee. 2011. Jointless structure and under-actuation mechanism for compact hand exoskeleton. In Rehabilitation Robotics (ICORR), 2011 IEEE International Conference on. IEEE, 1--6."},{"key":"e_1_2_1_26_1","unstructured":"Thalmic Labs Inc. 2017 (accessed August 6 2017) a. Myo. https:\/\/www.myo.com\/.  Thalmic Labs Inc. 2017 (accessed August 6 2017) a. Myo. https:\/\/www.myo.com\/."},{"key":"e_1_2_1_27_1","unstructured":"Thalmic Labs Inc. 2017 (accessed August 6 2017) b. Unity - Game Engine. https:\/\/unity3d.com\/.  Thalmic Labs Inc. 2017 (accessed August 6 2017) b. Unity - Game Engine. https:\/\/unity3d.com\/."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357089"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2592234"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1385569.1385653"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3139131.3141221"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3084822.3084824"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2642918.2647376"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858196"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858196"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3334480.3382914"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173919"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-39516-6_17"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1520340.1520568"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1101616.1101635"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2536146.2536166"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.5555\/580521.835816"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2984751.2985708"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.5555\/1531514.1531540"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2831081"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351252"},{"key":"e_1_2_1_47_1","volume-title":"One-thumb Text Acquisition on Force-assisted Miniature Interfaces for Mobile Headsets. In 2020 IEEE International Conference on Pervasive Computing and Communications (PerCom). 1--10","author":"Lee L.H.","unstructured":"L.H. Lee , Y. Zhu , Y.P. Yau , T. Braud , X. SU, and P. Hui . 2020 . One-thumb Text Acquisition on Force-assisted Miniature Interfaces for Mobile Headsets. In 2020 IEEE International Conference on Pervasive Computing and Communications (PerCom). 1--10 . L.H. Lee, Y. Zhu, Y.P. Yau, T. Braud, X. SU, and P. Hui. 2020. One-thumb Text Acquisition on Force-assisted Miniature Interfaces for Mobile Headsets. In 2020 IEEE International Conference on Pervasive Computing and Communications (PerCom). 1--10."},{"key":"e_1_2_1_48_1","volume-title":"HIBEY: Hide the Keyboard in Augmented Reality. In 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom. 1--10","author":"Lee L. H.","year":"2019","unstructured":"L. H. Lee , K. Yung Lam , Y. P. Yau , T. Braud , and P. Hui . 2019 . HIBEY: Hide the Keyboard in Augmented Reality. In 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom. 1--10 . https:\/\/doi.org\/10.1109\/PERCOM. 2019 .8767420 10.1109\/PERCOM.2019.8767420 L. H. Lee, K. Yung Lam, Y. P. Yau, T. Braud, and P. Hui. 2019. HIBEY: Hide the Keyboard in Augmented Reality. In 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom. 1--10. https:\/\/doi.org\/10.1109\/PERCOM.2019.8767420"},{"key":"e_1_2_1_49_1","unstructured":"Minkyung Lee Woontack Woo etal 2003. ARKB: 3D vision-based Augmented Reality Keyboard. In ICAT.  Minkyung Lee Woontack Woo et al. 2003. ARKB: 3D vision-based Augmented Reality Keyboard. In ICAT."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518750"},{"key":"e_1_2_1_51_1","volume-title":"Design, User Experience, and Usability: Users","author":"Li Wei","unstructured":"Wei Li , Xuan Zhang , and Yi Shen Zhang . 2018. How Animation Improve Children's Cognition in User Interface: A Study of the Kids VOD Application . In Design, User Experience, and Usability: Users , Contexts and Case Studies, Aaron Marcus and Wentao Wang (Eds.). Springer International Publishing , Cham , 196--217. Wei Li, Xuan Zhang, and Yi Shen Zhang. 2018. How Animation Improve Children's Cognition in User Interface: A Study of the Kids VOD Application. In Design, User Experience, and Usability: Users, Contexts and Case Studies, Aaron Marcus and Wentao Wang (Eds.). Springer International Publishing, Cham, 196--217."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/302979.302995"},{"key":"e_1_2_1_53_1","volume-title":"IEEE World Haptics Conference, WHC 2015 (2015","author":"Ma Zhaoyuan","year":"2015","unstructured":"Zhaoyuan Ma , Darren Edge , Leah Findlater , and Hong Z. Tan . 2015. Haptic keyclick feedback improves typing speed and reduces typing errors on a flat keyboard . IEEE World Haptics Conference, WHC 2015 (2015 ), 220--227. https:\/\/doi.org\/10.1109\/WHC. 2015 .7177717 10.1109\/WHC.2015.7177717 Zhaoyuan Ma, Darren Edge, Leah Findlater, and Hong Z. Tan. 2015. Haptic keyclick feedback improves typing speed and reduces typing errors on a flat keyboard. IEEE World Haptics Conference, WHC 2015 (2015), 220--227. https:\/\/doi.org\/10.1109\/WHC.2015.7177717"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15327051hci0701_3"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2556964"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174227"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/1551788.1551847"},{"key":"e_1_2_1_58_1","volume-title":"Telemanipulator and Telepresence Technologies","volume":"2351","author":"Milgram Paul","year":"1994","unstructured":"Paul Milgram , Haruo Takemura , Akira Utsumi , and Fumio Kishino . 1994 . Augmented reality: A class of displays on the reality-virtuality continuum . Telemanipulator and Telepresence Technologies , Vol. 2351 (01 1994). https:\/\/doi.org\/10.1117\/12.197321 10.1117\/12.197321 Paul Milgram, Haruo Takemura, Akira Utsumi, and Fumio Kishino. 1994. Augmented reality: A class of displays on the reality-virtuality continuum. Telemanipulator and Telepresence Technologies, Vol. 2351 (01 1994). https:\/\/doi.org\/10.1117\/12.197321"},{"key":"e_1_2_1_59_1","volume-title":"Proc. SPIE","volume":"6794","author":"Nam Y","year":"2007","unstructured":"Y Nam , M Park , and R Yamane . 2007 . Smart glove: hand master using magnetorheological fluid actuators . In Proc. SPIE , Vol. 6794 . 679434--1. Y Nam, M Park, and R Yamane. 2007. Smart glove: hand master using magnetorheological fluid actuators. In Proc. SPIE, Vol. 6794. 679434--1."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979303"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858489"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/AINA.2018.00059"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3338286.3340120"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/WHC.2019.8816098"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300550"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/2828993"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2462007"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290607.3312927"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174221"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376303"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3210240.3210315"},{"key":"e_1_2_1_72_1","unstructured":"CyberGlove Systems. 2020. CyberTouch. http:\/\/www.cyberglovesystems.com\/cybertouch.  CyberGlove Systems. 2020. CyberTouch. http:\/\/www.cyberglovesystems.com\/cybertouch."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301415"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/253671.253708"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/2785830.2785886"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10055-016-0296-6"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070481.2070497"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3027063.3053228"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/3159651"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/2669485.2669536"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3457141","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3457141","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:19Z","timestamp":1750191439000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3457141"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,27]]},"references-count":80,"journal-issue":{"issue":"EICS","published-print":{"date-parts":[[2021,5,27]]}},"alternative-id":["10.1145\/3457141"],"URL":"https:\/\/doi.org\/10.1145\/3457141","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,27]]},"assertion":[{"value":"2021-05-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}