{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T17:23:09Z","timestamp":1783099389062,"version":"3.54.6"},"reference-count":82,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2022,7,4]],"date-time":"2022-07-04T00:00:00Z","timestamp":1656892800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2022,7,4]]},"abstract":"<jats:p>Intelligent speech interfaces have been developing vastly to support the growing demands for convenient control and interaction with wearable\/earable and portable devices. To avoid privacy leakage during speech interactions and strengthen the resistance to ambient noise, silent speech interfaces have been widely explored to enable people's interaction with mobile\/wearable devices without audible sounds. However, most existing silent speech solutions require either restricted background illuminations or hand involvement to hold device or perform gestures. In this study, we propose a novel earphone-based, hand-free silent speech interaction approach, named EarCommand. Our technique discovers the relationship between the deformation of the ear canal and the movements of the articulator and takes advantage of this link to recognize different silent speech commands. Our system can achieve a WER (word error rate) of 10.02% for word-level recognition and 12.33% for sentence-level recognition, when tested in human subjects with 32 word-level commands and 25 sentence-level commands, which indicates the effectiveness of inferring silent speech commands. Moreover, EarCommand shows high reliability and robustness in a variety of configuration settings and environmental conditions. It is anticipated that EarCommand can serve as an efficient, intelligent speech interface for hand-free operation, which could significantly improve the quality and convenience of interactions.<\/jats:p>","DOI":"10.1145\/3534613","type":"journal-article","created":{"date-parts":[[2022,7,7]],"date-time":"2022-07-07T18:50:18Z","timestamp":1657219818000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["EarCommand"],"prefix":"10.1145","volume":"6","author":[{"given":"Yincheng","family":"Jin","sequence":"first","affiliation":[{"name":"University at Buffalo, State University of New York, Department of Computer Science and Engineering, Buffalo, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yang","family":"Gao","sequence":"additional","affiliation":[{"name":"Northwestern University, Department of Computer Science, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuhai","family":"Xu","sequence":"additional","affiliation":[{"name":"University of Washington, Information School, Seattle, Washington, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seokmin","family":"Choi","sequence":"additional","affiliation":[{"name":"University at Buffalo, State University of New York, Department of Computer Science and Engineering, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiyang","family":"Li","sequence":"additional","affiliation":[{"name":"University at Buffalo, State University of New York, Department of Computer Science and Engineering, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feng","family":"Liu","sequence":"additional","affiliation":[{"name":"University at Buffalo, State University of New York, Engineering Science, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhengxiong","family":"Li","sequence":"additional","affiliation":[{"name":"University of Colorado Denver, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhanpeng","family":"Jin","sequence":"additional","affiliation":[{"name":"University at Buffalo, State University of New York, Department of Computer Science and Engineering, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,7,7]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341163.3347747"},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the International Conference on Machine Learning (ICML'16)","author":"Amodei Dario","year":"2016","unstructured":"Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, et al. 2016. Deep speech 2: End-to-end speech recognition in English and Mandarin. In Proceedings of the International Conference on Machine Learning (ICML'16). 173--182."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2020-2946"},{"key":"e_1_2_1_4_1","volume-title":"Speech synthesis from neural decoding of spoken sentences. Nature 568, 7753","author":"Anumanchipalli Gopala K","year":"2019","unstructured":"Gopala K Anumanchipalli, Josh Chartier, and Edward F Chang. 2019. Speech synthesis from neural decoding of spoken sentences. Nature 568, 7753 (2019), 493--498."},{"key":"e_1_2_1_5_1","unstructured":"Apple. 2020. Powerbeats. https:\/\/www.beatsbydre.com\/earphones\/powerbeats."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM42981.2021.9488852"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2016.7472621"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3379337.3415879"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3427314"},{"key":"e_1_2_1_10_1","unstructured":"TMJ Relief Clinic. 2021. TMJ Relief Clinic - melbourne. https:\/\/tmjreliefclinic.com.au\/tmj-dysfunction-or-tmd\/."},{"key":"e_1_2_1_11_1","unstructured":"Syllable Counter. 2021. Syllable Counter. https:\/\/syllablecounter.net\/. [Online]."},{"key":"e_1_2_1_12_1","volume-title":"Apple Statistics","author":"Curry David","year":"2021","unstructured":"David Curry. 2021. Apple Statistics (2021). https:\/\/www.businessofapps.com\/data\/apple-statistics\/."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1063\/1.4792307"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIE.2013.2242656"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2006.1660033"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2009.08.002"},{"key":"e_1_2_1_17_1","volume-title":"https:\/\/www.amazon.com\/All-new-Echo-Buds\/dp\/B085WV7635. [Online","year":"2020","unstructured":"Echo-Buds. 2020. Amazon. https:\/\/www.amazon.com\/All-new-Echo-Buds\/dp\/B085WV7635. [Online; accessed 30-August-2020]."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2807442.2807499"},{"key":"e_1_2_1_19_1","doi-asserted-by":"crossref","unstructured":"Anthony S Fauci H Clifford Lane and Robert R Redfield. 2020. Covid-19---navigating the uncharted.","DOI":"10.1056\/NEJMe2002387"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.2105\/AJPH.2016.303527"},{"key":"e_1_2_1_21_1","first-page":"1","article-title":"SonicFace: Tracking Facial Expressions Using a Commodity Microphone Array","volume":"5","author":"Gao Yang","year":"2021","unstructured":"Yang Gao, Yincheng Jin, Seokmin Choi, Jiyang Li, Junjie Pan, Lin Shu, Chi Zhou, and Zhanpeng Jin. 2021. SonicFace: Tracking Facial Expressions Using a Commodity Microphone Array. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 4 (2021), 1--33.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_22_1","first-page":"1","article-title":"EchoWhisper: Exploring an Acoustic-based Silent Speech Interface for Smartphone Users","volume":"4","author":"Gao Yang","year":"2020","unstructured":"Yang Gao, Yincheng Jin, Jiyang Li, Seokmin Choi, and Zhanpeng Jin. 2020. EchoWhisper: Exploring an Acoustic-based Silent Speech Interface for Smartphone Users. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1--27.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_23_1","first-page":"1","article-title":"EarEcho: Using ear canal echo for wearable authentication","volume":"3","author":"Gao Yang","year":"2019","unstructured":"Yang Gao, Wei Wang, Vir V Phoha, Wei Sun, and Zhanpeng Jin. 2019. EarEcho: Using ear canal echo for wearable authentication. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--24.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_24_1","volume-title":"Global Industry Analysts","author":"Inc.","year":"2021","unstructured":"Inc. Global Industry Analysts. 2021. Earphones and Headphones --- Global Market Trajectory & Analysis. https:\/\/www.researchandmarkets.com\/reports\/5302827\/earphones-and-headphones-global-market."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.csl.2016.02.002"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2009.12.001"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3458709.3458985"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2012.02.001"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I\/O Systems and Assessment (O-COCOSDA). 58--62","author":"Bengu Wu Hao Zheng Xingyu Na","year":"2017","unstructured":"Xingyu Na Bengu Wu Hao Zheng Hui Bu, Jiayu Du. 2017. AIShell-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline. In Proceedings of the 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I\/O Systems and Assessment (O-COCOSDA). 58--62."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300506"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2017.2738568"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432216"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3494992"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3172944.3172977"},{"key":"e_1_2_1_36_1","volume-title":"Shyh Wei Teng, and Guojun Lu","author":"Karmakar Priyabrata","year":"2021","unstructured":"Priyabrata Karmakar, Shyh Wei Teng, and Guojun Lu. 2021. Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition. arXiv preprint arXiv:2102.07259 (2021)."},{"key":"e_1_2_1_37_1","volume-title":"Automatic Speech Recognition System for Tonal Languages: State-of-the-Art Survey. Archives of Computational Methods in Engineering 28, 3","author":"Kaur Jaspreet","year":"2021","unstructured":"Jaspreet Kaur, Amitoj Singh, and Virender Kadyan. 2021. Automatic Speech Recognition System for Tonal Languages: State-of-the-Art Survey. Archives of Computational Methods in Engineering 28, 3 (2021)."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274783.3275188"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the 22nd International Workshop on Mobile Computing Systems and Applications. 44--49","author":"Khanna Prerna","year":"2021","unstructured":"Prerna Khanna, Tanmay Srivastava, Shijia Pan, Shubham Jain, and Phuc Nguyen. 2021. JawSense: recognizing unvoiced sound using a low-cost ear-worn system. In Proceedings of the 22nd International Workshop on Mobile Computing Systems and Applications. 44--49."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098279.3098538"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3399715.3399852"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300376"},{"key":"e_1_2_1_43_1","doi-asserted-by":"crossref","unstructured":"Matias Laporte Preety Baglat Shkurta Gashi Martin Gjoreski Silvia Santini and Marc Langheinrich. 2021. Detecting Verbal and Non-Verbal Gestures Using Earables. In Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers. 165--170.","DOI":"10.1145\/3460418.3479322"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TUFFC.2011.1929"},{"key":"e_1_2_1_45_1","volume-title":"Proceedings of the International Conference on Applied Human Factors and Ergonomics. Springer, 241--250","author":"L\u00f3pez Gustavo","year":"2017","unstructured":"Gustavo L\u00f3pez, Luis Quesada, and Luis A Guerrero. 2017. Alexa vs. Siri vs. Cortana vs. Google Assistant: a comparison of speech-based natural user interfaces. In Proceedings of the International Conference on Applied Human Factors and Ergonomics. Springer, 241--250."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025692"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2017.2740000"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.2004.23"},{"key":"e_1_2_1_49_1","volume-title":"MOUSER ELECTRONICS","author":"Inc.","year":"2021","unstructured":"Inc. MOUSER ELECTRONICS. 2021. stmsteval-bcn00. https:\/\/www.mouser.com\/new\/stmicroelectronics\/stm-steval-bcn002v1b\/."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2003.1200069"},{"key":"e_1_2_1_51_1","unstructured":"NUHEARA. 2020. Active Noise Cancellation."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445565"},{"key":"e_1_2_1_53_1","volume-title":"Proceedings of the INTERSPEECH (Sep","author":"Park Daniel S.","year":"2019","unstructured":"Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, and Quoc V. Le. 2019. SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition. Proceedings of the INTERSPEECH (Sep 2019)."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1097\/01.HJ.0000285785.83490.6c"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.3390\/inventions3030048"},{"key":"e_1_2_1_56_1","volume-title":"https:\/\/www.apple.com\/airpods-pro\/. [Online","author":"Pro AirPods","year":"2020","unstructured":"AirPods Pro. 2020. Apple. https:\/\/www.apple.com\/airpods-pro\/. [Online; accessed 30-August-2020]."},{"key":"e_1_2_1_57_1","unstructured":"pytorch. 2021. pytorch. https:\/\/pytorch.org\/mobile\/android\/. [Online]."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445205"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2634317.2634322"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753573"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/348941.348990"},{"key":"e_1_2_1_62_1","unstructured":"Sony. 2020. WH-1000XM3 Wireless Noise-Cancelling Headphones Full Specifications and Features. https:\/\/www.sony.com\/electronics\/headband-headphones\/wh- 1000xm3\/specifications#features\/."},{"key":"e_1_2_1_63_1","unstructured":"Speech and Voice. 2020. Speech And Voice Recognition Market. https:\/\/www.marketsandmarkets.com\/Market-Reports\/speech-voice-recognition-market-202401714.html. [Online]."},{"key":"e_1_2_1_64_1","unstructured":"Speech and Voice. 2021. The best Siri commands for productivity information laughter and more. https:\/\/www.androidauthority.com\/best-siri-commands-1094484\/. [Online]."},{"key":"e_1_2_1_65_1","volume-title":"most downloaded apps","year":"2021","unstructured":"Statistics. 2021. most downloaded apps 2021. https:\/\/mashable.com\/article\/most-downloaded-apps-2021. [Online]."},{"key":"e_1_2_1_66_1","volume-title":"Oxford Dictionary of English","author":"Stevenson Angus","unstructured":"Angus Stevenson. 2010. Oxford Dictionary of English. Oxford University Press, USA."},{"key":"e_1_2_1_67_1","unstructured":"STMicroelectronics. 2021. STMicroelectronics App. https:\/\/github.com\/STMicroelectronics\/BlueSTSDK_Android\/."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242587.3242599"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397481.3450645"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432199"},{"key":"e_1_2_1_71_1","volume-title":"taxpayer expense","author":"S.","year":"2021","unstructured":"U.S. taxpayer expense. 2021. frequency names. https:\/\/www.ssa.gov\/oact\/babynames\/decades. [Online]."},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.386217"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1016\/0167-6393(93)90095-3"},{"key":"e_1_2_1_74_1","volume-title":"Proceedings of the Advances in Neural Information Processing Systems. 5998--6008","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems. 5998--6008."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478085"},{"key":"e_1_2_1_76_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3448098","article-title":"EarDynamic: An Ear Canal Deformation Based Continuous User Authentication Using In-Ear Wearables","volume":"5","author":"Wang Zi","year":"2021","unstructured":"Zi Wang, Sheng Tan, Linghan Zhang, Yili Ren, Zhi Wang, and Jie Yang. 2021. EarDynamic: An Ear Canal Deformation Based Continuous User Authentication Using In-Ear Wearables. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1 (2021), 1--27.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_77_1","unstructured":"Xomba.Com. 2021. frequent contacts. https:\/\/www.xomba.com\/best-contact-names\/. [Online]."},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376836"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/3081333.3081356"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/3494990"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432192"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST46873.2019.9047731"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3534613","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3534613","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:31:36Z","timestamp":1752467496000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3534613"}},"subtitle":["\"Hearing\" Your Silent Speech Commands In Ear"],"short-title":[],"issued":{"date-parts":[[2022,7,4]]},"references-count":82,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,7,4]]}},"alternative-id":["10.1145\/3534613"],"URL":"https:\/\/doi.org\/10.1145\/3534613","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,4]]},"assertion":[{"value":"2022-07-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}