{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T16:26:57Z","timestamp":1783096017816,"version":"3.54.6"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,3,6]],"date-time":"2024-03-06T00:00:00Z","timestamp":1709683200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"the National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62122095, 62341201, 62072472, 62202256"],"award-info":[{"award-number":["62122095, 62341201, 62072472, 62202256"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"the Guoqiang Institute, Tsinghua University"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2024,3,6]]},"abstract":"<jats:p>Facial expression recognition (FER) is a crucial task for human-computer interaction and a multitude of multimedia applications that typically call for friendly, unobtrusive, ubiquitous, and even long-term monitoring. Achieving such a FER system meeting these multi-requirements faces critical challenges, mainly including the tiny irregular non-periodic deformation of emotion movements, high variability in facial positions and severe self-interference caused by users' own other behavior. In this work, we present UFace, a long-term, unobtrusive and reliable FER system for daily life using acoustic signals generated by a portable smartphone. We design an innovative network model with dual-stream input based on the attention mechanism, which can leverage distance-time profile features from various viewpoints to extract fine-grained emotion-related signal changes, thus enabling accurate identification of many kinds of expressions. Meanwhile, we propose effective mechanisms to deal with a series of interference issues during actual use. We implement UFace prototype with a daily-used smartphone and conduct extensive experiments in various real-world environments. The results demonstrate that UFace can successfully recognize 7 typical facial expressions with an average accuracy of 87.8% across 20 participants. Besides, the evaluation of different distances, angles, and interferences proves the great potential of the proposed system to be employed in practical scenarios.<\/jats:p>","DOI":"10.1145\/3643546","type":"journal-article","created":{"date-parts":[[2024,3,6]],"date-time":"2024-03-06T13:12:36Z","timestamp":1709730756000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["UFace"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9727-1434","authenticated-orcid":false,"given":"Shuning","family":"Wang","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, ChangSha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2178-5073","authenticated-orcid":false,"given":"Linghui","family":"Zhong","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, ChangSha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8481-2644","authenticated-orcid":false,"given":"Yongjian","family":"Fu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Central South University, ChangSha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6630-0976","authenticated-orcid":false,"given":"Lili","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2782-183X","authenticated-orcid":false,"given":"Ju","family":"Ren","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, BNRist, Tsinghua University, Beijing, China and Zhongguancun Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6717-461X","authenticated-orcid":false,"given":"Yaoxue","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, BNRist, Tsinghua University, Beijing, China and Zhongguancun Laboratory, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,3,6]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341163.3347747"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126594.3126649"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2993148.2993165"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM.2017.8057101"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3494977"},{"key":"e_1_2_1_6_1","first-page":"378","article-title":"WiFace: facial expression recognition using Wi-Fi signals","volume":"21","author":"Chen Yanjiao","year":"2020","unstructured":"Yanjiao Chen, Runmin Ou, Zhiyang Li, and Kaishun Wu. 2020. WiFace: facial expression recognition using Wi-Fi signals. IEEE Transactions on Mobile Computing 21, 1 (2020), 378--391.","journal-title":"IEEE Transactions on Mobile Computing"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM42981.2021.9488703"},{"key":"e_1_2_1_8_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3534597","article-title":"PPGface: Like What You Are Watching? Earphones Can","volume":"6","author":"Choi Seokmin","year":"2022","unstructured":"Seokmin Choi, Yang Gao, Yincheng Jin, Se jun Kim, Jiyang Li, Wenyao Xu, and Zhanpeng Jin. 2022. PPGface: Like What You Are Watching? Earphones Can\" Feel\" Your Facial Expressions. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 2 (2022), 1--32.","journal-title":"Feel\" Your Facial Expressions. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_9_1","volume-title":"Basic emotions. Handbook of cognition and emotion 98, 45-60","author":"Ekman Paul","year":"1999","unstructured":"Paul Ekman. 1999. Basic emotions. Handbook of cognition and emotion 98, 45-60 (1999), 16."},{"key":"e_1_2_1_10_1","volume-title":"Suppl S5","author":"Ekman Paul","year":"2004","unstructured":"Paul Ekman. 2004. Emotions revealed. Bmj 328, Suppl S5 (2004)."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3560905.3568530"},{"key":"e_1_2_1_12_1","first-page":"1","article-title":"SonicFace: Tracking Facial Expressions Using a Commodity Microphone Array","volume":"5","author":"Gao Yang","year":"2021","unstructured":"Yang Gao, Yincheng Jin, Seokmin Choi, Jiyang Li, Junjie Pan, Lin Shu, Chi Zhou, and Zhanpeng Jin. 2021. SonicFace: Tracking Facial Expressions Using a Commodity Microphone Array. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 4 (2021), 1--33.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_13_1","first-page":"1","article-title":"EchoWhisper: Exploring an Acoustic-based Silent Speech Interface for Smartphone Users","volume":"4","author":"Gao Yang","year":"2020","unstructured":"Yang Gao, Yincheng Jin, Jiyang Li, Seokmin Choi, and Zhanpeng Jin. 2020. EchoWhisper: Exploring an Acoustic-based Silent Speech Interface for Smartphone Users. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1--27.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2014.2313557"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_16_1","volume-title":"Eyemotion: Classifying facial expressions in VR using eye-tracking cameras. In 2019 IEEE winter conference on applications of computer vision (WACV)","author":"Hickson Steven","year":"2019","unstructured":"Steven Hickson, Nick Dufour, Avneesh Sud, Vivek Kwatra, and Irfan Essa. 2019. Eyemotion: Classifying facial expressions in VR using eye-tracking cameras. In 2019 IEEE winter conference on applications of computer vision (WACV). IEEE, 1626--1635."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2023.3243661"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3384419.3430780"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3569476"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.277"},{"key":"e_1_2_1_21_1","volume-title":"Ultragesture: Fine-grained gesture sensing and recognition","author":"Ling Kang","year":"2020","unstructured":"Kang Ling, Haipeng Dai, Yuntang Liu, Alex X Liu, Wei Wang, and Qing Gu. 2020. Ultragesture: Fine-grained gesture sensing and recognition. IEEE Transactions on Mobile Computing (2020)."},{"key":"e_1_2_1_22_1","first-page":"1","article-title":"BlinkListener: \"Listen\" to Your Eye Blink Using Your Smartphone","volume":"5","author":"Liu Jialin","year":"2021","unstructured":"Jialin Liu, Dong Li, Lei Wang, and Jie Xiong. 2021. BlinkListener: \"Listen\" to Your Eye Blink Using Your Smartphone. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1--27.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3210240.3210325"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025692"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858580"},{"key":"e_1_2_1_26_1","volume-title":"Ensemble learning. Ensemble machine learning: Methods and applications","author":"Polikar Robi","year":"2012","unstructured":"Robi Polikar. 2012. Ensemble learning. Ensemble machine learning: Methods and applications (2012), 1--34."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM.2018.8485978"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3314410"},{"key":"e_1_2_1_29_1","unstructured":"Daniel Ruby. 2023. Smartphone Usage Statistics 2023 (Facts & Data). https:\/\/www.demandsage.com\/smartphone-usage-statistics\/."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0077714"},{"key":"e_1_2_1_31_1","volume-title":"Fernando De la Torre, and Adam Rossi","author":"Ryan Andrew","year":"2009","unstructured":"Andrew Ryan, Jeffery F Cohn, Simon Lucey, Jason Saragih, Patrick Lucey, Fernando De la Torre, and Adam Rossi. 2009. Automated facial expression recognition system. In 43rd annual 2009 international Carnahan conference on security technology. IEEE, 172--177."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPSN54338.2022.00019"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.114"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447993.3448626"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3210240.3210312"},{"key":"e_1_2_1_36_1","volume-title":"Beamforming: A versatile approach to spatial filtering","author":"Van Veen Barry D","year":"1988","unstructured":"Barry D Van Veen and Kevin M Buckley. 1988. Beamforming: A versatile approach to spatial filtering. IEEE assp magazine 5, 2 (1988), 4--24."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478085"},{"key":"e_1_2_1_38_1","doi-asserted-by":"crossref","unstructured":"Guangjing Wang Qiben Yan Shane Patrarungrong Juexing Wang and Huacheng Zeng. 2023. FacER: Contrastive Attention based Expression Recognition via Smartphone Earpiece Speaker. (2023).","DOI":"10.1109\/INFOCOM53939.2023.10228985"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2956143"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1049\/cje.2018.09.020"},{"key":"e_1_2_1_41_1","first-page":"1","article-title":"Expression Complementary Disentanglement Network for Facial Expression Recognition","volume":"33","author":"Shanmin WANG, Hui SHUAI, Lei","year":"2023","unstructured":"Shanmin WANG, Hui SHUAI, Lei ZHU, and Qingshan LIU. 2023. Expression Complementary Disentanglement Network for Facial Expression Recognition. Chinese Journal of Electronics 33 (2023), 1--11.","journal-title":"Chinese Journal of Electronics"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3581791.3596839"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448105"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3307334.3326074"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00231"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.5555\/3162631.3162760"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3494990"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3494990"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2017.02.007"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3241539.3241575"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2020.3004555"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643546","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3643546","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T13:02:20Z","timestamp":1755867740000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643546"}},"subtitle":["Your Smartphone Can \"Hear\" Your Facial Expression!"],"short-title":[],"issued":{"date-parts":[[2024,3,6]]},"references-count":52,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3,6]]}},"alternative-id":["10.1145\/3643546"],"URL":"https:\/\/doi.org\/10.1145\/3643546","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,6]]},"assertion":[{"value":"2024-03-06","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}