{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T21:34:49Z","timestamp":1774992889917,"version":"3.50.1"},"reference-count":108,"publisher":"Association for Computing Machinery (ACM)","issue":"MHCI","license":[{"start":{"date-parts":[[2023,9,11]],"date-time":"2023-09-11T00:00:00Z","timestamp":1694390400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2023,9,11]]},"abstract":"<jats:p>Wearable devices allow quick and convenient interactions for controlling mobile computers. However, these interactions are often device-dependent, and users cannot control devices in a way they are familiar with if they do not wear the same wearable device. This paper proposes a new method, UnifiedSense, to enable device-dependent gestures even when the device that detects such gestures is missing by utilizing sensors on other wearable devices. UnifiedSense achieves this without explicit gesture training for different devices, by training its recognition model while users naturally perform gestures. The recognizer uses the gestures detected on the primary device (i.e., a device that reliably detects gestures) as labels for training samples and collects sensor data from all other available devices on the user. We conducted a technical evaluation with data collected from 15 participants with four types of wearable devices. It showed that UnifiedSense could correctly recognize 5 gestures (5 gestures \u00d7 5 configurations) with an accuracy of 90.9% (SD = 1.9%) without the primary device present.<\/jats:p>","DOI":"10.1145\/3604277","type":"journal-article","created":{"date-parts":[[2023,9,13]],"date-time":"2023-09-13T15:16:20Z","timestamp":1694618180000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["UnifiedSense: Enabling Without-Device Gesture Interactions Using Over-the-shoulder Training Between Redundant Wearable Sensors"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9744-8263","authenticated-orcid":false,"given":"Md Aashikur Rahman","family":"Azim","sequence":"first","affiliation":[{"name":"University of Virginia, Charlottesville, VA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1620-5268","authenticated-orcid":false,"given":"Adil","family":"Rahman","sequence":"additional","affiliation":[{"name":"University of Virginia, Charlottesville, VA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2004-4812","authenticated-orcid":false,"given":"Seongkook","family":"Heo","sequence":"additional","affiliation":[{"name":"University of Virginia, Charlottesville, VA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,9,13]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132272.3134136"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2011.2165707"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/506443.506566"},{"key":"e_1_2_2_4_1","volume-title":"Apple -- iOS 15 -- Continuity. https:\/\/www.macrumors.com\/guide\/universal-control\/. Online","author":"Apple Inc. 2021.","year":"2022","unstructured":"Apple Inc. 2021. Apple -- iOS 15 -- Continuity. https:\/\/www.macrumors.com\/guide\/universal-control\/. Online; accessed Aug 2022."},{"key":"e_1_2_2_5_1","volume-title":"Enabling mobile microinteractions","author":"Ashbrook Daniel L","unstructured":"Daniel L Ashbrook. 2010. Enabling mobile microinteractions. Georgia Institute of Technology."},{"key":"e_1_2_2_6_1","volume-title":"Over-The-Shoulder Training Between Redundant Wearable Sensors for Unified Gesture Interactions. In Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1--3.","author":"Rahman Azim Md Aashikur","year":"2022","unstructured":"Md Aashikur Rahman Azim, Adil Rahman, and Seongkook Heo. 2022. Over-The-Shoulder Training Between Redundant Wearable Sensors for Unified Gesture Interactions. In Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1--3."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300792"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2687922"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1013115.1013149"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2556955"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702451"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3170427.3188590"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.14236\/ewic\/HCI2010.49"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1613858.1613866"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357177"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2642918.2647396"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-020-00701-z"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.2007.4373768"},{"key":"e_1_2_2_20_1","volume-title":"Where the action is: the foundations of embodied interaction","author":"Dourish Paul","unstructured":"Paul Dourish. 2004. Where the action is: the foundations of embodied interaction. MIT press."},{"key":"e_1_2_2_21_1","volume-title":"PyTorch Manual: The developer's perspective. https:\/\/pytorch.org\/. Online","author":"Facebook's AI","year":"2022","unstructured":"Facebook's AI Research lab. 2022. PyTorch Manual: The developer's perspective. https:\/\/pytorch.org\/. Online; accessed Aug 2022."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/120782.120783"},{"key":"e_1_2_2_23_1","volume-title":"https:\/\/www.fitbit.com\/global\/us\/products\/smartwatches\/versa3. Online","author":"Fitbit Inc. 2022. Fitbit Versa.","year":"2022","unstructured":"Fitbit Inc. 2022. Fitbit Versa. https:\/\/www.fitbit.com\/global\/us\/products\/smartwatches\/versa3. Online; accessed Aug 2022."},{"key":"e_1_2_2_24_1","volume-title":"Unsupervised scalable representation learning for multivariate time series. Advances in neural information processing systems 32","author":"Franceschi Jean-Yves","year":"2019","unstructured":"Jean-Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. 2019. Unsupervised scalable representation learning for multivariate time series. Advances in neural information processing systems 32 (2019)."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2030112.2030154"},{"key":"e_1_2_2_26_1","doi-asserted-by":"crossref","unstructured":"Marcus Georgi Christoph Amma and Tanja Schultz. 2015. Recognizing Hand and Finger Gestures with IMU based Motion and EMG based Muscle Activity Sensing.. In Biosignals. Citeseer 99--108.","DOI":"10.5220\/0005276900990108"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197391.3205422"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2697076"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2984511.2984563"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126594.3126615"},{"key":"e_1_2_2_31_1","volume-title":"Android Wear: The developer's perspective. https:\/\/youtu.be\/G7tXr-w35UA. Online","author":"Google Inc.","year":"2014","unstructured":"Google Inc. 2014. Android Wear: The developer's perspective. https:\/\/youtu.be\/G7tXr-w35UA. Online; accessed Aug 2022."},{"key":"e_1_2_2_32_1","volume-title":"https:\/\/www.google.com\/glass\/start\/. Online","author":"Google Inc. 2022. Google Glass.","year":"2022","unstructured":"Google Inc. 2022. Google Glass. https:\/\/www.google.com\/glass\/start\/. Online; accessed Aug 2022."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3332165.3347947"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2750858.2807551"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753394"},{"key":"e_1_2_2_36_1","volume-title":"Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415","author":"Hendrycks Dan","year":"2016","unstructured":"Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 (2016)."},{"key":"e_1_2_2_37_1","unstructured":"Seongkook Heo Michelle Annett Benjamin J Lafreniere Tovi Grossman and George W Fitzmaurice. 2017. No Need to Stop What You're Doing: Exploring No-Handed Smartwatch Interaction.. In Graphics Interface. 107--114."},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/964696.964713"},{"key":"e_1_2_2_39_1","volume-title":"Long short-term memory. Neural computation 9, 8","author":"Hochreiter Sepp","year":"1997","unstructured":"Sepp Hochreiter and J\u00fcrgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780."},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702273"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432202"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858483"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300245"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300506"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2582051.2582066"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0254841"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2541831.2541875"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2642918.2647376"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380139"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.3390\/s19183827"},{"key":"e_1_2_2_51_1","volume-title":"Ijcai","volume":"14","author":"Ron","unstructured":"Ron Kohavi et al. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Ijcai, Vol. 14. Montreal, Canada, 1137--1145."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/IRDS.2002.1043893"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300568"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2984511.2984582"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341163.3347745"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3214275"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925953"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397308"},{"key":"e_1_2_2_59_1","volume-title":"Houston, Texas","author":"Liu Jiayang","year":"2008","unstructured":"Jiayang Liu, Zhen Wang, Lin Zhong, Jehan Wickramasuriya, and Venu Vasudevan. 2008. uWave: Accelerometer-based personalized gesture recognition. TR0630-08, Rice University and Motorola Labs, Houston, Texas (2008)."},{"key":"e_1_2_2_60_1","volume-title":"Proc. MobileHCI","volume":"10","author":"Loclair Christian","year":"2010","unstructured":"Christian Loclair, Sean Gustafson, and Patrick Baudisch. 2010. PinchWatch: a wearable device for one-handed microinteractions. In Proc. MobileHCI, Vol. 10. Citeseer."},{"key":"e_1_2_2_61_1","volume-title":"User Interface Beaming--Seamless Interaction with Smart Things Using Personal Wearable Computers. In 2014 11th International Conference on Wearable and Implantable Body Sensor Networks Workshops","author":"Mayer Simon","unstructured":"Simon Mayer and G\u00e1bor S\u00f6r\u00f6s. 2014. User Interface Beaming--Seamless Interaction with Smart Things Using Personal Wearable Computers. In 2014 11th International Conference on Wearable and Implantable Body Sensor Networks Workshops. IEEE, 46--49."},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126594.3126604"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126594.3126604"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025807"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2003.09.007"},{"key":"e_1_2_2_66_1","volume-title":"TaplD: Rapid Touch Interaction in Virtual Reality using Wearable Sensing. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR)","author":"Meier Manuel","unstructured":"Manuel Meier, Paul Streli, Andreas Fender, and Christian Holz. 2021. TaplD: Rapid Touch Interaction in Virtual Reality using Wearable Sensing. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, 519--528."},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/2802083.2802085"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1177\/1541931215591370"},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2466142"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/2742647.2742665"},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/1340961.1340963"},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2208695"},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/1029632.1029658"},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2466192"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","unstructured":"J. Rekimoto. 2001. GestureWrist and GesturePad: unobtrusive wearable interaction devices. 21--27. https:\/\/doi.org\/10.1109\/ISWC.2001.962092 ISSN: 1530-0811.","DOI":"10.1109\/ISWC.2001.962092"},{"key":"e_1_2_2_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/3161162"},{"key":"e_1_2_2_77_1","doi-asserted-by":"publisher","DOI":"10.1007\/978--3--319-07230--2_13"},{"key":"e_1_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025658"},{"key":"e_1_2_2_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/2493432.2493476"},{"key":"e_1_2_2_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357138"},{"key":"e_1_2_2_81_1","doi-asserted-by":"publisher","DOI":"10.1145\/1622176.1622208"},{"key":"e_1_2_2_82_1","doi-asserted-by":"publisher","DOI":"10.1145\/1622176.1622208"},{"key":"e_1_2_2_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866063"},{"key":"e_1_2_2_84_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2556984"},{"key":"e_1_2_2_85_1","doi-asserted-by":"publisher","DOI":"10.1145\/3279778.3279799"},{"key":"e_1_2_2_86_1","volume-title":"Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1","author":"Srivastava Nitish","year":"2014","unstructured":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929--1958."},{"key":"e_1_2_2_87_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.1997.629929"},{"key":"e_1_2_2_88_1","doi-asserted-by":"publisher","DOI":"10.1145\/2148131.2148195"},{"key":"e_1_2_2_89_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274783.3274854"},{"key":"e_1_2_2_90_1","doi-asserted-by":"publisher","DOI":"10.1145\/2957265.2961860"},{"key":"e_1_2_2_91_1","volume-title":"Attention is all you need. Advances in neural information processing systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_2_2_92_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376132"},{"key":"e_1_2_2_93_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dsp.2021.103091"},{"key":"e_1_2_2_94_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858466"},{"key":"e_1_2_2_95_1","unstructured":"William Antonelli and Jennifer Still. 2021. How to use Spotify on your Apple Watch to play music or control playback. https:\/\/www.businessinsider.in\/tech\/how-to\/how-to-use-spotify-on-your-apple-watch-to-play-music-orcontrol-playback\/articleshow\/82538864.cms. Online; accessed May 2023."},{"key":"e_1_2_2_96_1","doi-asserted-by":"publisher","DOI":"10.1145\/3379337.3415897"},{"key":"e_1_2_2_97_1","doi-asserted-by":"publisher","DOI":"10.1145\/3267242.3267251"},{"key":"e_1_2_2_98_1","doi-asserted-by":"publisher","DOI":"10.1145\/2699343.2699350"},{"key":"e_1_2_2_99_1","doi-asserted-by":"publisher","DOI":"10.1145\/3229434.3229449"},{"key":"e_1_2_2_100_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501904"},{"key":"e_1_2_2_101_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376836"},{"key":"e_1_2_2_102_1","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380137"},{"key":"e_1_2_2_103_1","doi-asserted-by":"publisher","DOI":"10.1145\/3332165.3347867"},{"key":"e_1_2_2_104_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467401"},{"key":"e_1_2_2_105_1","doi-asserted-by":"publisher","DOI":"10.1145\/2659766.2659773"},{"key":"e_1_2_2_106_1","doi-asserted-by":"publisher","DOI":"10.1145\/3123021.3123041"},{"key":"e_1_2_2_107_1","doi-asserted-by":"publisher","DOI":"10.1145\/2807442.2807480"},{"key":"e_1_2_2_108_1","volume-title":"https:\/\/ouraring.com\/. Online","author":"Health Ltd Oura","year":"2022","unstructured":"Oura \u00af Health Ltd. 2022. Oura Ring. https:\/\/ouraring.com\/. Online; accessed Aug 2022."}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3604277","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3604277","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:47:17Z","timestamp":1750178837000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3604277"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,11]]},"references-count":108,"journal-issue":{"issue":"MHCI","published-print":{"date-parts":[[2023,9,11]]}},"alternative-id":["10.1145\/3604277"],"URL":"https:\/\/doi.org\/10.1145\/3604277","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,11]]},"assertion":[{"value":"2023-09-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}