{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T16:55:18Z","timestamp":1783097718889,"version":"3.54.6"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2022,7,4]],"date-time":"2022-07-04T00:00:00Z","timestamp":1656892800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2022,7,4]]},"abstract":"<jats:p>Automatically recognizing a broad spectrum of human activities is key to realizing many compelling applications in health, personal assistance, human-computer interaction and smart environments. However, in real-world settings, approaches to human action perception have been largely constrained to detecting mobility states, e.g., walking, running, standing. In this work, we explore the use of inertial-acoustic sensing provided by off-the-shelf commodity smartwatches for detecting activities of daily living (ADLs). We conduct a semi-naturalistic study with a diverse set of 15 participants in their own homes and show that acoustic and inertial sensor data can be combined to recognize 23 activities such as writing, cooking, and cleaning with high accuracy. We further conduct a completely in-the-wild study with 5 participants to better evaluate the feasibility of our system in practical unconstrained scenarios. We comprehensively studied various baseline machine learning and deep learning models with three different fusion strategies, demonstrating the benefit of combining inertial and acoustic data for ADL recognition. Our analysis underscores the feasibility of high-performing recognition of daily activities using inertial-acoustic data from practical off-the-shelf wrist-worn devices while also uncovering challenges faced in unconstrained settings. We encourage researchers to use our public dataset to further push the boundary of ADL recognition in-the-wild.<\/jats:p>","DOI":"10.1145\/3534582","type":"journal-article","created":{"date-parts":[[2022,7,7]],"date-time":"2022-07-07T18:50:18Z","timestamp":1657219818000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":37,"title":["Leveraging Sound and Wrist Motion to Detect Activities of Daily Living with Commodity Smartwatches"],"prefix":"10.1145","volume":"6","author":[{"given":"Sarnab","family":"Bhattacharya","sequence":"first","affiliation":[{"name":"University of Texas at Austin, Austin, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rebecca","family":"Adaimi","sequence":"additional","affiliation":[{"name":"University of Texas at Austin, Austin, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edison","family":"Thomaz","sequence":"additional","affiliation":[{"name":"University of Texas at Austin, Austin, Texas, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,7,7]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448083"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/PerComWorkshops48775.2020.9156226"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351228"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448090"},{"key":"e_1_2_1_5_1","volume-title":"2017 IEEE 18th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM). 1--6.","author":"Arshad S.","unstructured":"S. Arshad, C. Feng, Y. Liu, Y. Hu, R. Yu, S. Zhou, and H. Li. 2017. Wi-chase: A WiFi based human activity recognition system for sensorless environments. In 2017 IEEE 18th International Symposium on A World of Wireless, Mobile and Multimedia Networks (WoWMoM). 1--6."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.3523476"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2015.04.005"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0098089"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341163.3347735"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/PERCOMW.2016.7457169"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jada.2010.10.008"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3380985"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302315"},{"key":"e_1_2_1_14_1","volume-title":"Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215)","author":"Clarkson B.","unstructured":"B. Clarkson and A. Pentland. 1998. Extracting context from environmental audio. In Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215). 154--155."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2030112.2030135"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2526667.2526685"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2638728.2641673"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.3390\/s17061230"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3090076"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.3390\/s140305687"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2021.3064252"},{"key":"e_1_2_1_22_1","volume-title":"2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 131--135","author":"Hershey S.","unstructured":"S. Hershey, S. Chaudhuri, D. P. W. Ellis, J. F. Gemmeke, A. Jansen, R. C. Moore, M. Plakal, D. Platt, R. A. Saurous, B. Seybold, M. Slaney, R. J. Weiss, and K. Wilson. 2017. CNN architectures for large-scale audio classification. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 131--135."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC44109.2020.9176412"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV45572.2020.9093298"},{"key":"e_1_2_1_25_1","volume-title":"2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 003707--003712","author":"Kim Hyunchoong","year":"2016","unstructured":"Hyunchoong Kim, Jonghoon Shin, Soohwan Kim, Yohan Ko, Kyoungwoo Lee, Hojung Cha, Seong-il Hahm, and TaeJun Kwon. 2016. Collaborative classification for daily activity recognition with a smartwatch. In 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 003707--003712."},{"key":"e_1_2_1_26_1","volume-title":"International Conference on Machine Learning. PMLR, 5637--5664","author":"Koh Pang Wei","year":"2021","unstructured":"Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. 2021. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning. PMLR, 5637--5664."},{"key":"e_1_2_1_27_1","doi-asserted-by":"crossref","unstructured":"Qiuqiang Kong Yin Cao Turab Iqbal Yuxuan Wang Wenwu Wang and Mark Plumbley. 2019. PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition.","DOI":"10.1109\/TASLP.2020.3030497"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242587.3242609"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300568"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2370216.2370248"},{"key":"e_1_2_1_31_1","volume-title":"22nd International Conference on Human-Computer Interaction with Mobile Devices and Services. 1--10.","author":"Liang Dawei","unstructured":"Dawei Liang, Wenting Song, and Edison Thomaz. 2020. Characterizing the Effect of Audio Degradation on Privacy Perception And Inference Performance in Audio-Based Human Activity Recognition. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services. 1--10."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3314404"},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Haojie Ma Wenzhong Li Xiao Zhang Songcheng Gao and Sanglu Lu. 2019. AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition.. In IJCAI. 3109--3115.","DOI":"10.24963\/ijcai.2019\/431"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/VS.1999.780265"},{"key":"e_1_2_1_35_1","volume-title":"2012 IEEE International Conference on Pervasive Computing and Communications Workshops. 510--512","author":"Maekawa T.","unstructured":"T. Maekawa, Y. Kishino, Y. Yanagisawa, and Y. Sakurai. 2012. WristSense: Wrist-worn sensor device with camera for daily activity recognition. In 2012 IEEE International Conference on Pervasive Computing and Communications Workshops. 510--512."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2005.1521728"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2557116"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.3390\/s16081341"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989370"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3123021.3123046"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.3390\/s16010115"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/11853565_8"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.2005.22"},{"key":"e_1_2_1_44_1","unstructured":"A. Physics. 2010. Nyquist-Shannon Sampling Theorem."},{"key":"e_1_2_1_45_1","volume-title":"2012 IEEE Conference on Computer Vision and Pattern Recognition. 2847--2854","author":"Pirsiavash H.","unstructured":"H. Pirsiavash and D. Ramanan. 2012. Detecting activities of daily living in first-person camera views. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. 2847--2854."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3161174"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2413097.2413148"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC44109.2020.9175949"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/PERCOMW.2015.7134104"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/b136479"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0227039"},{"key":"e_1_2_1_52_1","volume-title":"2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication. 509--514","author":"Stork J. A.","unstructured":"J. A. Stork, L. Spinello, J. Silva, and K. O. Arras. 2012. Audio-based human activity recognition using Non-Markovian Ensemble Voting. In 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication. 509--514."},{"key":"e_1_2_1_53_1","volume-title":"Activity Recognition in the Home Using Simple and Ubiquitous Sensors","author":"Tapia Emmanuel Munguia","unstructured":"Emmanuel Munguia Tapia, Stephen S. Intille, and Kent Larson. 2004. Activity Recognition in the Home Using Simple and Ubiquitous Sensors. In Pervasive Computing, Alois Ferscha and Friedemann Mattern (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 158--175."},{"key":"e_1_2_1_54_1","volume-title":"Activiome: A System for Annotating First-Person Photos and Multimodal Activity Sensor Data. In 2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)","author":"Thomaz Edison","year":"2020","unstructured":"Edison Thomaz. 2020. Activiome: A System for Annotating First-Person Photos and Multimodal Activity Sensor Data. In 2020 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, 1--6."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/MPRV.2017.3971131"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3161192"},{"key":"e_1_2_1_57_1","article-title":"Visualizing data using t-SNE","volume":"9","author":"der Maaten Laurens Van","year":"2008","unstructured":"Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).","journal-title":"Journal of machine learning research"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2006.197"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/BHI.2016.7455925"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3417313.3429383"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3323679.3326513"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2984511.2984515"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2992154.2992187"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3534582","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3534582","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:32:27Z","timestamp":1752467547000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3534582"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,4]]},"references-count":63,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,7,4]]}},"alternative-id":["10.1145\/3534582"],"URL":"https:\/\/doi.org\/10.1145\/3534582","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,4]]},"assertion":[{"value":"2022-07-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}