{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,21]],"date-time":"2025-11-21T18:07:40Z","timestamp":1763748460928,"version":"3.41.0"},"reference-count":97,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2021,9,9]],"date-time":"2021-09-09T00:00:00Z","timestamp":1631145600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100004739","name":"Youth Innovation Promotion Association CAS","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004739","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Huawei"},{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2020AAA0105200"],"award-info":[{"award-number":["2020AAA0105200"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Beijing Natural Science Foundation","award":["No.JQ18011"],"award-info":[{"award-number":["No.JQ18011"]}]},{"name":"National Top-notch Youth Talents Program of China"},{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"crossref","award":["U1836211"],"award-info":[{"award-number":["U1836211"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Beijing Academy of Artificial Intelligence"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2021,9,9]]},"abstract":"<jats:p>Smart speakers, such as Google Home and Amazon Echo, have become popular. They execute user voice commands via their built-in functionalities together with various third-party voice-controlled applications, called skills. Malicious skills have brought significant threats to users in terms of security and privacy. As a countermeasure, only skills passing the strict vetting process can be released onto markets. However, malicious skills have been reported to exist on markets, indicating that the vetting process can be bypassed. This paper aims to demystify the vetting process of skills on main markets to discover weaknesses and protect markets better. To probe the vetting process, we carefully design numerous skills, perform the Turing test, a test for machine intelligence, to determine whether humans or machines perform vetting, and leverage natural language processing techniques to analyze their behaviors. Based on our comprehensive experiments, we gain a good understanding of the vetting process (e.g., machine or human testers and skill exploration strategies) and discover some weaknesses. In this paper, we design three types of attacks to verify our results and prove an attacker can embed sensitive behaviors in skills and bypass the strict vetting process. Accordingly, we also propose countermeasures to these attacks and weaknesses.<\/jats:p>","DOI":"10.1145\/3478101","type":"journal-article","created":{"date-parts":[[2021,9,14]],"date-time":"2021-09-14T22:48:23Z","timestamp":1631659703000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Demystifying the Vetting Process of Voice-controlled Skills on Markets"],"prefix":"10.1145","volume":"5","author":[{"given":"Dawei","family":"Wang","sequence":"first","affiliation":[{"name":"SKLOIS, Institute of Information Engineering, Chinese Academy of Sciences and School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China"}]},{"given":"Kai","family":"Chen","sequence":"additional","affiliation":[{"name":"SKLOIS, Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy of Sciences, and Beijing Academy of Artificial Intelligence, Beijing, China"}]},{"given":"Wei","family":"Wang","sequence":"additional","affiliation":[{"name":"Beijing Key Laboratory of Security and Privacy in Intelligent Transportation, Beijing Jiaotong University, Beijing, China"}]}],"member":"320","published-online":{"date-parts":[[2021,9,14]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Kevin RB Butler, and Joseph Wilson","author":"Abdullah Hadi","year":"2019","unstructured":"Hadi Abdullah , Washington Garcia , Christian Peeters , Patrick Traynor , Kevin RB Butler, and Joseph Wilson . 2019 . Practical hidden voice attacks against speech and speaker recognition systems. In NDSS. Hadi Abdullah, Washington Garcia, Christian Peeters, Patrick Traynor, Kevin RB Butler, and Joseph Wilson. 2019. Practical hidden voice attacks against speech and speaker recognition systems. In NDSS."},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","unstructured":"Hadi Abdullah Kevin Warren Vincent Bindschaedler Nicolas Papernot and Patrick Traynor. 2020. SoK: The Faults in our ASRs: An Overview of Attacks against Automatic Speech Recognition and Speaker Identification Systems. arXiv:2007.06622 [cs.CR]  Hadi Abdullah Kevin Warren Vincent Bindschaedler Nicolas Papernot and Patrick Traynor. 2020. SoK: The Faults in our ASRs: An Overview of Attacks against Automatic Speech Recognition and Speaker Identification Systems. arXiv:2007.06622 [cs.CR]","DOI":"10.1109\/SP40001.2021.00014"},{"key":"e_1_2_1_3_1","unstructured":"Amazon. 2018. Amazon Mechanical Turk. https:\/\/www.mturk.com\/.  Amazon. 2018. Amazon Mechanical Turk. https:\/\/www.mturk.com\/."},{"key":"e_1_2_1_4_1","unstructured":"Amazon. 2021. Alexa Skills Kit Command Line Interface (ASK CLI) Overview. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/smapi\/ask-cli-intro.html.  Amazon. 2021. Alexa Skills Kit Command Line Interface (ASK CLI) Overview. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/smapi\/ask-cli-intro.html."},{"key":"e_1_2_1_5_1","unstructured":"Amazon. 2021. Certification Requirements in Alexa. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/certification-requirements-for-custom-skills.html.  Amazon. 2021. Certification Requirements in Alexa. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/certification-requirements-for-custom-skills.html."},{"key":"e_1_2_1_6_1","unstructured":"Amazon. 2021. Create Intents Utterances and Slots. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/create-intents-utterances-and-slots.html.  Amazon. 2021. Create Intents Utterances and Slots. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/create-intents-utterances-and-slots.html."},{"key":"e_1_2_1_7_1","unstructured":"Amazon. 2021. Functional Testing for a Custom Skill. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/functional-testing-for-a-custom-skill.html.  Amazon. 2021. Functional Testing for a Custom Skill. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/functional-testing-for-a-custom-skill.html."},{"key":"e_1_2_1_8_1","unstructured":"Amazon. 2021. Manage the Skill Session and Session Attributes. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/manage-skill-session-and-session-attributes.html.  Amazon. 2021. Manage the Skill Session and Session Attributes. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/manage-skill-session-and-session-attributes.html."},{"key":"e_1_2_1_9_1","unstructured":"Amazon. 2021. Request and Response JSON Reference. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/request-and-response-json-reference.html.  Amazon. 2021. Request and Response JSON Reference. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/request-and-response-json-reference.html."},{"key":"e_1_2_1_10_1","unstructured":"Amazon. 2021. Standard Built-in Intents. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/standard-built-in-intents.html.  Amazon. 2021. Standard Built-in Intents. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/custom-skills\/standard-built-in-intents.html."},{"key":"e_1_2_1_11_1","unstructured":"Amazon. 2021. Test and Submit Your Skill for Certification. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/devconsole\/test-and-submit-your-skill.html.  Amazon. 2021. Test and Submit Your Skill for Certification. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/devconsole\/test-and-submit-your-skill.html."},{"key":"e_1_2_1_12_1","first-page":"23","article-title":"Drebin: Effective and explainable detection of android malware in your pocket","volume":"14","author":"Arp Daniel","year":"2014","unstructured":"Daniel Arp , Michael Spreitzenbarth , Malte Hubner , Hugo Gascon , Konrad Rieck , and CERT Siemens . 2014 . Drebin: Effective and explainable detection of android malware in your pocket .. In Ndss , Vol. 14. 23 -- 26 . Daniel Arp, Michael Spreitzenbarth, Malte Hubner, Hugo Gascon, Konrad Rieck, and CERT Siemens. 2014. Drebin: Effective and explainable detection of android malware in your pocket.. In Ndss, Vol. 14. 23--26.","journal-title":"Ndss"},{"key":"e_1_2_1_13_1","volume-title":"23rd USENIX Security Symposium (USENIX Security 14)","author":"Bhoraskar Ravi","year":"2014","unstructured":"Ravi Bhoraskar , Seungyeop Han , Jinseong Jeon , Tanzirul Azim , Shuo Chen , Jaeyeon Jung , Suman Nath , Rui Wang , and David Wetherall . 2014 . Brahmastra: Driving apps to test the security of third-party components . In 23rd USENIX Security Symposium (USENIX Security 14) . 1021--1036. Ravi Bhoraskar, Seungyeop Han, Jinseong Jeon, Tanzirul Azim, Shuo Chen, Jaeyeon Jung, Suman Nath, Rui Wang, and David Wetherall. 2014. Brahmastra: Driving apps to test the security of third-party components. In 23rd USENIX Security Symposium (USENIX Security 14). 1021--1036."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3212480.3212505"},{"key":"e_1_2_1_15_1","volume-title":"Smart Spies: Alexa and Google Home expose users to vishing and eavesdropping. https:\/\/srlabs.de\/bites\/smart-spies\/.","author":"Br\u00e4unlein Fabian","year":"2019","unstructured":"Fabian Br\u00e4unlein and Luise Frerichs . 2019 . Smart Spies: Alexa and Google Home expose users to vishing and eavesdropping. https:\/\/srlabs.de\/bites\/smart-spies\/. Fabian Br\u00e4unlein and Luise Frerichs. 2019. Smart Spies: Alexa and Google Home expose users to vishing and eavesdropping. https:\/\/srlabs.de\/bites\/smart-spies\/."},{"key":"e_1_2_1_16_1","unstructured":"BusinessWorldIT. 2019. 12 Best Artificial Intelligence Chatbots For Android. https:\/\/www.businessworldit.com\/ai\/best-ai-chatbots-for-android\/.  BusinessWorldIT. 2019. 12 Best Artificial Intelligence Chatbots For Android. https:\/\/www.businessworldit.com\/ai\/best-ai-chatbots-for-android\/."},{"volume-title":"25th {USENIX} Security Symposium ({USENIX} Security 16). 513--530.","author":"Carlini Nicholas","key":"e_1_2_1_17_1","unstructured":"Nicholas Carlini , Pratyush Mishra , Tavish Vaidya , Yuankai Zhang , Micah Sherr , Clay Shields , David Wagner , and Wenchao Zhou . 2016. Hidden voice commands . In 25th {USENIX} Security Symposium ({USENIX} Security 16). 513--530. Nicholas Carlini, Pratyush Mishra, Tavish Vaidya, Yuankai Zhang, Micah Sherr, Clay Shields, David Wagner, and Wenchao Zhou. 2016. Hidden voice commands. In 25th {USENIX} Security Symposium ({USENIX} Security 16). 513--530."},{"key":"e_1_2_1_18_1","unstructured":"Rollo Carpenter. 2011. jabberwacky - live chat bot. http:\/\/www.jabberwacky.com\/.  Rollo Carpenter. 2011. jabberwacky - live chat bot. http:\/\/www.jabberwacky.com\/."},{"key":"e_1_2_1_19_1","unstructured":"Rollo Carpenter. 2021. Cleverbot.com - a clever bot. https:\/\/www.cleverbot.com\/.  Rollo Carpenter. 2021. Cleverbot.com - a clever bot. https:\/\/www.cleverbot.com\/."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568286"},{"volume-title":"24th {USENIX} Security Symposium ({USENIX} Security 15). 659--674.","author":"Chen Kai","key":"e_1_2_1_21_1","unstructured":"Kai Chen , Peng Wang , Yeonjoon Lee , XiaoFeng Wang , Nan Zhang , Heqing Huang , Wei Zou , and Peng Liu . 2015. Finding unknown malice in 10 seconds: Mass vetting for new threats at the google-play scale . In 24th {USENIX} Security Symposium ({USENIX} Security 15). 659--674. Kai Chen, Peng Wang, Yeonjoon Lee, XiaoFeng Wang, Nan Zhang, Heqing Huang, Wei Zou, and Peng Liu. 2015. Finding unknown malice in 10 seconds: Mass vetting for new threats at the google-play scale. In 24th {USENIX} Security Symposium ({USENIX} Security 15). 659--674."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2016.29"},{"key":"e_1_2_1_23_1","volume-title":"29th USENIX Security Symposium (USENIX Security 20)","author":"Chen Yuxuan","year":"2020","unstructured":"Yuxuan Chen , Xuejing Yuan , Jiangshan Zhang , Yue Zhao , Shengzhi Zhang , Kai Chen , and XiaoFeng Wang . 2020 . Devil's Whisper: A General Approach for Physical Adversarial Attacks against Commercial Black-box Speech Recognition Devices . In 29th USENIX Security Symposium (USENIX Security 20) . Yuxuan Chen, Xuejing Yuan, Jiangshan Zhang, Yue Zhao, Shengzhi Zhang, Kai Chen, and XiaoFeng Wang. 2020. Devil's Whisper: A General Approach for Physical Adversarial Attacks against Commercial Black-box Speech Recognition Devices. In 29th USENIX Security Symposium (USENIX Security 20)."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3372297.3423339"},{"key":"e_1_2_1_25_1","unstructured":"Andrew Cohen. 2020. https:\/\/www.wearerockwater.com\/blog\/smart-speaker-boom-at-home-audio. https:\/\/www.wearerockwater.com\/blog\/smart-speaker-boom-at-home-audio.  Andrew Cohen. 2020. https:\/\/www.wearerockwater.com\/blog\/smart-speaker-boom-at-home-audio. https:\/\/www.wearerockwater.com\/blog\/smart-speaker-boom-at-home-audio."},{"key":"e_1_2_1_26_1","volume-title":"Proc. of ESORICS","volume":"13","author":"Crussell Jonathan","year":"2013","unstructured":"Jonathan Crussell , Clint Gibler , and Hao Chen . 2013 . Scalable semantics-based detection of similar android applications . In Proc. of ESORICS , Vol. 13 . Citeseer. Jonathan Crussell, Clint Gibler, and Hao Chen. 2013. Scalable semantics-based detection of similar android applications. In Proc. of ESORICS, Vol. 13. Citeseer."},{"key":"e_1_2_1_27_1","unstructured":"George Dvorsky. 2015. 8 Possible Alternatives To The Turing Test. https:\/\/io9.gizmodo.com\/8-possible-alternatives-to-the-turing-test-1697983985.  George Dvorsky. 2015. 8 Possible Alternatives To The Turing Test. https:\/\/io9.gizmodo.com\/8-possible-alternatives-to-the-turing-test-1697983985."},{"key":"e_1_2_1_28_1","unstructured":"dylanavalverde. 2018. A Brief History of Chatbots. https:\/\/pcc.cs.byu.edu\/2018\/03\/26\/a-brief-history-of-chatbots\/.  dylanavalverde. 2018. A Brief History of Chatbots. https:\/\/pcc.cs.byu.edu\/2018\/03\/26\/a-brief-history-of-chatbots\/."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2619091"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1653662.1653691"},{"key":"e_1_2_1_31_1","unstructured":"Linked Experiences. 2021. Jovo Framework. https:\/\/www.jovo.tech\/framework.  Linked Experiences. 2021. Jovo Framework. https:\/\/www.jovo.tech\/framework."},{"key":"e_1_2_1_32_1","unstructured":"Explosion. 2021. Industrial-Strength Natural Language Processing. https:\/\/spacy.io.  Explosion. 2021. Industrial-Strength Natural Language Processing. https:\/\/spacy.io."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635869"},{"key":"e_1_2_1_34_1","volume-title":"Automated synthesis of semantic malware signatures using maximum satisfiability. NDSS","author":"Feng Yu","year":"2017","unstructured":"Yu Feng , Osbert Bastani , Ruben Martins , Isil Dillig , and Saswat Anand . 2017. Automated synthesis of semantic malware signatures using maximum satisfiability. NDSS ( 2017 ). Yu Feng, Osbert Bastani, Ruben Martins, Isil Dillig, and Saswat Anand. 2017. Automated synthesis of semantic malware signatures using maximum satisfiability. NDSS (2017)."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985971"},{"key":"e_1_2_1_36_1","unstructured":"Google. 2020. App Actions built-in intents. https:\/\/developers.google.com\/assistant\/app\/reference\/built-in-intents.  Google. 2020. App Actions built-in intents. https:\/\/developers.google.com\/assistant\/app\/reference\/built-in-intents."},{"key":"e_1_2_1_37_1","unstructured":"Google. 2020. Conversation webhook format. https:\/\/developers.google.com\/assistant\/conversational\/df-asdk\/reference\/conversation-webhook-json.  Google. 2020. Conversation webhook format. https:\/\/developers.google.com\/assistant\/conversational\/df-asdk\/reference\/conversation-webhook-json."},{"key":"e_1_2_1_38_1","unstructured":"Google. 2020. Dialogflow webhook format. https:\/\/developers.google.com\/assistant\/conversational\/df-asdk\/reference\/dialogflow-webhook-json.  Google. 2020. Dialogflow webhook format. https:\/\/developers.google.com\/assistant\/conversational\/df-asdk\/reference\/dialogflow-webhook-json."},{"key":"e_1_2_1_39_1","unstructured":"Google. 2021. Custom intents. https:\/\/developers.google.com\/assistant\/app\/custom-intents.  Google. 2021. Custom intents. https:\/\/developers.google.com\/assistant\/app\/custom-intents."},{"key":"e_1_2_1_40_1","unstructured":"Google. 2021. Policies for Actions on Google. https:\/\/developers.google.com\/assistant\/console\/policies\/general-policies.  Google. 2021. Policies for Actions on Google. https:\/\/developers.google.com\/assistant\/console\/policies\/general-policies."},{"key":"e_1_2_1_41_1","volume-title":"SkillExplorer: Understanding the Behavior of Skills in Large Scale. In 29th USENIX Security Symposium (USENIX Security 20)","author":"Guo Zhixiu","year":"2020","unstructured":"Zhixiu Guo , Zijin Lin , Pan Li , and Kai Chen . 2020 . SkillExplorer: Understanding the Behavior of Skills in Large Scale. In 29th USENIX Security Symposium (USENIX Security 20) . USENIX Association, 2649--2666. https:\/\/www.usenix.org\/conference\/usenixsecurity20\/presentation\/guo Zhixiu Guo, Zijin Lin, Pan Li, and Kai Chen. 2020. SkillExplorer: Understanding the Behavior of Skills in Large Scale. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 2649--2666. https:\/\/www.usenix.org\/conference\/usenixsecurity20\/presentation\/guo"},{"key":"e_1_2_1_42_1","volume-title":"International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 62--81","author":"Hanna Steve","year":"2012","unstructured":"Steve Hanna , Ling Huang , Edward Wu , Saung Li , Charles Chen , and Dawn Song . 2012 . Juxtapp: A scalable system for detecting code reuse among android applications . In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 62--81 . Steve Hanna, Ling Huang, Edward Wu, Saung Li, Charles Chen, and Dawn Song. 2012. Juxtapp: A scalable system for detecting code reuse among android applications. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 62--81."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/SPW50608.2020.00029"},{"key":"e_1_2_1_44_1","unstructured":"Huggingface. 2021. NeuralCoref 4.0: Coreference Resolution in spaCy with Neural Networks. https:\/\/github.com\/huggingface\/neuralcoref.  Huggingface. 2021. NeuralCoref 4.0: Coreference Resolution in spaCy with Neural Networks. https:\/\/github.com\/huggingface\/neuralcoref."},{"key":"e_1_2_1_45_1","unstructured":"SimSimi Inc. 2019. SimSimi. www.simsimi.com.  SimSimi Inc. 2019. SimSimi. www.simsimi.com."},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the 20th Network and Distributed System Security Symposium (NDSS).","author":"Xuxian Jiang Yajin Zhou","year":"2013","unstructured":"Yajin Zhou Xuxian Jiang and Zhou Xuxian . 2013 . Detecting passive content leaks and pollution in android applications . In Proceedings of the 20th Network and Distributed System Security Symposium (NDSS). Yajin Zhou Xuxian Jiang and Zhou Xuxian. 2013. Detecting passive content leaks and pollution in android applications. In Proceedings of the 20th Network and Distributed System Security Symposium (NDSS)."},{"volume-title":"Psychological types","author":"Jung Carl Gustav","key":"e_1_2_1_47_1","unstructured":"Carl Gustav Jung . 2014. Psychological types . Routledge . Carl Gustav Jung. 2014. Psychological types. Routledge."},{"key":"e_1_2_1_48_1","volume-title":"Google Assistant Actions Total 4,253","author":"BRET KINSELLA.","year":"2019","unstructured":"BRET KINSELLA. 2019. Google Assistant Actions Total 4,253 in January 2019 . https:\/\/voicebot.ai\/2020\/01\/19\/google-assistant-actions-grew-quickly-in-several-languages-in-2019-match-alexa-growth-in-english\/. BRET KINSELLA. 2019. Google Assistant Actions Total 4,253 in January 2019. https:\/\/voicebot.ai\/2020\/01\/19\/google-assistant-actions-grew-quickly-in-several-languages-in-2019-match-alexa-growth-in-english\/."},{"key":"e_1_2_1_49_1","unstructured":"BRET KINSELLA. 2020. Alexa Skill Counts Surpass 80K in US Spain Adds the Most Skills New Skill Rate Falls Globally. https:\/\/voicebot.ai\/2021\/01\/14\/alexa-skill-counts-surpass-80k-in-us-spain-adds-the-most-skills-new-skill-introduction-rate-continues-to-fall-across-countries\/.  BRET KINSELLA. 2020. Alexa Skill Counts Surpass 80K in US Spain Adds the Most Skills New Skill Rate Falls Globally. https:\/\/voicebot.ai\/2021\/01\/14\/alexa-skill-counts-surpass-80k-in-us-spain-adds-the-most-skills-new-skill-introduction-rate-continues-to-fall-across-countries\/."},{"key":"e_1_2_1_50_1","volume-title":"27th USENIX Security Symposium (USENIX Security 18)","author":"Kumar Deepak","year":"2018","unstructured":"Deepak Kumar , Riccardo Paccagnella , Paul Murley , Eric Hennenfent , Joshua Mason , Adam Bates , and Michael Bailey . 2018 . Skill squatting attacks on Amazon Alexa . In 27th USENIX Security Symposium (USENIX Security 18) . 33--47. Deepak Kumar, Riccardo Paccagnella, Paul Murley, Eric Hennenfent, Joshua Mason, Adam Bates, and Michael Bailey. 2018. Skill squatting attacks on Amazon Alexa. In 27th USENIX Security Symposium (USENIX Security 18). 33--47."},{"key":"e_1_2_1_51_1","unstructured":"Eugenia Kuyda. 2019. replika. https:\/\/replika.ai\/.  Eugenia Kuyda. 2019. replika. https:\/\/replika.ai\/."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2016.52"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.38"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2382196.2382223"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635896"},{"key":"e_1_2_1_56_1","unstructured":"Taylor Martin. 2017. You can now use any Alexa skill without enabling it first. https:\/\/www.cnet.com\/home\/smart-home\/amazon-echo-you-can-now-use-any-alexa-skill-without-enabling-it-first\/.  Taylor Martin. 2017. You can now use any Alexa skill without enabling it first. https:\/\/www.cnet.com\/home\/smart-home\/amazon-echo-you-can-now-use-any-alexa-skill-without-enabling-it-first\/."},{"key":"e_1_2_1_57_1","unstructured":"MIT. 2021. Avoiding Plagiarism - Paraphrasing. https:\/\/integrity.mit.edu\/handbook\/academic-writing\/avoiding-plagiarism-paraphrasing.  MIT. 2021. Avoiding Plagiarism - Paraphrasing. https:\/\/integrity.mit.edu\/handbook\/academic-writing\/avoiding-plagiarism-paraphrasing."},{"key":"e_1_2_1_58_1","unstructured":"Pandorabots. 2020. Mitsuku. https:\/\/www.pandorabots.com\/mitsuku\/.  Pandorabots. 2020. Mitsuku. https:\/\/www.pandorabots.com\/mitsuku\/."},{"key":"e_1_2_1_59_1","unstructured":"Pandorabots. 2021. Bot A.L.I.C.E - Pandorabots. https:\/\/www.pandorabots.com\/pandora\/talk?botid=b8d616e35e36e881.  Pandorabots. 2021. Bot A.L.I.C.E - Pandorabots. https:\/\/www.pandorabots.com\/pandora\/talk?botid=b8d616e35e36e881."},{"key":"e_1_2_1_60_1","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni , Salim Roukos , Todd Ward , and Wei-Jing Zhu . 2002 . Bleu: a Method for Automatic Evaluation of Machine Translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics , Philadelphia, Pennsylvania, USA, 311--318. https:\/\/doi.org\/10.3115\/1073083.1073135 10.3115\/1073083.1073135 Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, 311--318. https:\/\/doi.org\/10.3115\/1073083.1073135"},{"key":"e_1_2_1_61_1","article-title":"The Inconsistency of \"Optimal","volume":"163","author":"Perkins Neil J.","year":"2006","unstructured":"Neil J. Perkins and Enrique F. Schisterman . 2006 . The Inconsistency of \"Optimal \" Cutpoints Obtained using Two Criteria based on the Receiver Operating Characteristic Curve. American Journal of Epidemiology 163 , 7 (01 2006), 670--675. https:\/\/doi.org\/10.1093\/aje\/kwj063 arXiv:https:\/\/academic.oup.com\/aje\/article-pdf\/163\/7\/670\/175717\/kwj063.pdf 10.1093\/aje Neil J. Perkins and Enrique F. Schisterman. 2006. The Inconsistency of \"Optimal\" Cutpoints Obtained using Two Criteria based on the Receiver Operating Characteristic Curve. American Journal of Epidemiology 163, 7 (01 2006), 670--675. https:\/\/doi.org\/10.1093\/aje\/kwj063 arXiv:https:\/\/academic.oup.com\/aje\/article-pdf\/163\/7\/670\/175717\/kwj063.pdf","journal-title":"American Journal of Epidemiology"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2435349.2435379"},{"key":"e_1_2_1_63_1","volume-title":"EuroSec","author":"Reina Alessandro","year":"2013","unstructured":"Alessandro Reina , Aristide Fattori , and Lorenzo Cavallaro . 2013 . A system call-centric analysis and stimulation technique to automatically reconstruct android malware behaviors . EuroSec , April (2013). Alessandro Reina, Aristide Fattori, and Lorenzo Cavallaro. 2013. A system call-centric analysis and stimulation technique to automatically reconstruct android malware behaviors. EuroSec, April (2013)."},{"key":"e_1_2_1_64_1","unstructured":"Get Riddles. 2018. Get Riddles. 1000+ Riddles With Answers for Kids and Adults. https:\/\/www.getriddles.com\/.  Get Riddles. 2018. Get Riddles. 1000+ Riddles With Answers for Kids and Adults. https:\/\/www.getriddles.com\/."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33018-6_30"},{"key":"e_1_2_1_66_1","volume-title":"where is my privacy? Exploring Accidental Triggers of Smart Speakers. ArXiv abs\/2008.00508","author":"Sch\u00f6nherr Lea","year":"2020","unstructured":"Lea Sch\u00f6nherr , Maximilian Golla , Thorsten Eisenhofer , Jan Wiele , D. Kolossa , and Thorsten Holz . 2020. Unacceptable , where is my privacy? Exploring Accidental Triggers of Smart Speakers. ArXiv abs\/2008.00508 ( 2020 ). Lea Sch\u00f6nherr, Maximilian Golla, Thorsten Eisenhofer, Jan Wiele, D. Kolossa, and Thorsten Holz. 2020. Unacceptable, where is my privacy? Exploring Accidental Triggers of Smart Speakers. ArXiv abs\/2008.00508 (2020)."},{"key":"e_1_2_1_67_1","unstructured":"Artificial Solutions. 2020. Elbot the Chatbot. www.elbot.com.  Artificial Solutions. 2020. Elbot the Chatbot. www.elbot.com."},{"key":"e_1_2_1_68_1","unstructured":"Robyn Speer Joshua Chin and Catherine Havasi. 2017. ConceptNet 5.5: An Open Multilingual Graph of General Knowledge. 4444--4451 pages. http:\/\/aaai.org\/ocs\/index.php\/AAAI\/AAAI17\/paper\/view\/14972  Robyn Speer Joshua Chin and Catherine Havasi. 2017. ConceptNet 5.5: An Open Multilingual Graph of General Knowledge. 4444--4451 pages. http:\/\/aaai.org\/ocs\/index.php\/AAAI\/AAAI17\/paper\/view\/14972"},{"volume-title":"29th {USENIX} Security Symposium ({USENIX} Security 20). 2631--2648.","author":"Sugawara Takeshi","key":"e_1_2_1_69_1","unstructured":"Takeshi Sugawara , Benjamin Cyr , Sara Rampazzi , Daniel Genkin , and Kevin Fu. 2020. Light commands: laser-based audio injection attacks on voice-controllable systems . In 29th {USENIX} Security Symposium ({USENIX} Security 20). 2631--2648. Takeshi Sugawara, Benjamin Cyr, Sara Rampazzi, Daniel Genkin, and Kevin Fu. 2020. Light commands: laser-based audio injection attacks on voice-controllable systems. In 29th {USENIX} Security Symposium ({USENIX} Security 20). 2631--2648."},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1109\/ITAP.2011.6006288"},{"key":"e_1_2_1_71_1","unstructured":"Purdue University. 2020. Paraphrasing \/\/ Purdue Writing Lab. https:\/\/owl.purdue.edu\/owl\/research_and_citation\/using_research\/quoting_paraphrasing_and_summarizing\/paraphrasing.html.  Purdue University. 2020. Paraphrasing \/\/ Purdue Writing Lab. https:\/\/owl.purdue.edu\/owl\/research_and_citation\/using_research\/quoting_paraphrasing_and_summarizing\/paraphrasing.html."},{"key":"e_1_2_1_72_1","volume-title":"9th USENIX Workshop on Offensive Technologies (WOOT 15)","author":"Vaidya Tavish","year":"2015","unstructured":"Tavish Vaidya , Yuankai Zhang , Micah Sherr , and Clay Shields . 2015 . Cocaine noodles: exploiting the gap between human and machine speech recognition . In 9th USENIX Workshop on Offensive Technologies (WOOT 15) . Tavish Vaidya, Yuankai Zhang, Micah Sherr, and Clay Shields. 2015. Cocaine noodles: exploiting the gap between human and machine speech recognition. In 9th USENIX Workshop on Offensive Technologies (WOOT 15)."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/2435349.2435378"},{"key":"e_1_2_1_74_1","unstructured":"Wikipedia. 2021. Determiner. https:\/\/en.wikipedia.org\/wiki\/Determiner.  Wikipedia. 2021. Determiner. https:\/\/en.wikipedia.org\/wiki\/Determiner."},{"key":"e_1_2_1_75_1","unstructured":"Wikipedia. 2021. Proper noun and specific noun. https:\/\/en.wikipedia.org\/wiki\/Proper_noun_and_common_noun.  Wikipedia. 2021. Proper noun and specific noun. https:\/\/en.wikipedia.org\/wiki\/Proper_noun_and_common_noun."},{"key":"e_1_2_1_76_1","unstructured":"Wikipedia. 2021. Stop words - Wikipedia. https:\/\/en.wikipedia.org\/wiki\/Stop_words.  Wikipedia. 2021. Stop words - Wikipedia. https:\/\/en.wikipedia.org\/wiki\/Stop_words."},{"key":"e_1_2_1_77_1","unstructured":"Wikipedia. 2021. Unicode - Wikipedia. https:\/\/en.wikipedia.org\/wiki\/Unicode.  Wikipedia. 2021. Unicode - Wikipedia. https:\/\/en.wikipedia.org\/wiki\/Unicode."},{"key":"e_1_2_1_78_1","unstructured":"Bruce Wilcox. 2017. Rose demo - Brillig Understanding. http:\/\/brilligunderstanding.com\/rosedemo.html.  Bruce Wilcox. 2017. Rose demo - Brillig Understanding. http:\/\/brilligunderstanding.com\/rosedemo.html."},{"key":"e_1_2_1_79_1","first-page":"21","article-title":"IntelliDroid: A Targeted Input Generator for the Dynamic Analysis of Android Malware","volume":"16","author":"Wong Michelle Y","year":"2016","unstructured":"Michelle Y Wong and David Lie . 2016 . IntelliDroid: A Targeted Input Generator for the Dynamic Analysis of Android Malware .. In NDSS , Vol. 16. 21 -- 24 . Michelle Y Wong and David Lie. 2016. IntelliDroid: A Targeted Input Generator for the Dynamic Analysis of Android Malware.. In NDSS, Vol. 16. 21--24.","journal-title":"NDSS"},{"key":"e_1_2_1_80_1","unstructured":"Chiachih Wu Yajin Zhou Kunal Patel Zhenkai Liang and Xuxian Jiang. 2014. AirBag: Boosting Smartphone Resistance to Malware Infection.. In NDSS.  Chiachih Wu Yajin Zhou Kunal Patel Zhenkai Liang and Xuxian Jiang. 2014. AirBag: Boosting Smartphone Resistance to Malware Infection.. In NDSS."},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1109\/AsiaJCIS.2012.18"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2015.60"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3134018"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2014.44"},{"key":"e_1_2_1_85_1","volume-title":"Droidscope: Seamlessly reconstructing the OS and dalvik semantic views for dynamic android malware analysis. In Presented as part of the 21st USENIX Security Symposium (USENIX Security 12). 569--584.","author":"Yan Lok Kwong","year":"2012","unstructured":"Lok Kwong Yan and Heng Yin . 2012 . Droidscope: Seamlessly reconstructing the OS and dalvik semantic views for dynamic android malware analysis. In Presented as part of the 21st USENIX Security Symposium (USENIX Security 12). 569--584. Lok Kwong Yan and Heng Yin. 2012. Droidscope: Seamlessly reconstructing the OS and dalvik semantic views for dynamic android malware analysis. In Presented as part of the 21st USENIX Security Symposium (USENIX Security 12). 569--584."},{"key":"e_1_2_1_86_1","doi-asserted-by":"crossref","unstructured":"Qiben Yan Kehai Liu Qin Zhou Hanqing Guo and Ning Zhang. 2020. SurfingAttack: Interactive Hidden Attack on Voice Assistants Using Ultrasonic Guided Waves. In NDSS.  Qiben Yan Kehai Liu Qin Zhou Hanqing Guo and Ning Zhang. 2020. SurfingAttack: Interactive Hidden Attack on Voice Assistants Using Ultrasonic Guided Waves. In NDSS.","DOI":"10.14722\/ndss.2020.24068"},{"key":"e_1_2_1_87_1","volume-title":"27th USENIX Security Symposium (USENIX Security 18)","author":"Yuan Xuejing","year":"2018","unstructured":"Xuejing Yuan , Yuxuan Chen , Yue Zhao , Yunhui Long , Xiaokang Liu , Kai Chen , Shengzhi Zhang , Heqing Huang , XiaoFeng Wang , and Carl A Gunter . 2018 . Commandersong: A systematic approach for practical adversarial voice recognition . In 27th USENIX Security Symposium (USENIX Security 18) . 49--64. Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, XiaoFeng Wang, and Carl A Gunter. 2018. Commandersong: A systematic approach for practical adversarial voice recognition. In 27th USENIX Security Symposium (USENIX Security 18). 49--64."},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1145\/2627393.2627395"},{"key":"e_1_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3134052"},{"key":"e_1_2_1_90_1","doi-asserted-by":"publisher","DOI":"10.1145\/2660267.2660359"},{"key":"e_1_2_1_91_1","volume-title":"Dangerous Skills: Understanding and Mitigating Security Risks of Voice-Controlled Third-Party Functions on Virtual Personal Assistant Systems. 2019 IEEE Symposium on Security and Privacy (SP)","author":"Zhang Nan","year":"2019","unstructured":"Nan Zhang , Xianghang Mi , Xuan Feng , XiaoFeng Wang , Yuan Tian , and Feng Qian . 2019 . Dangerous Skills: Understanding and Mitigating Security Risks of Voice-Controlled Third-Party Functions on Virtual Personal Assistant Systems. 2019 IEEE Symposium on Security and Privacy (SP) (2019), 1381--1396. Nan Zhang, Xianghang Mi, Xuan Feng, XiaoFeng Wang, Yuan Tian, and Feng Qian. 2019. Dangerous Skills: Understanding and Mitigating Security Risks of Voice-Controlled Third-Party Functions on Virtual Personal Assistant Systems. 2019 IEEE Symposium on Security and Privacy (SP) (2019), 1381--1396."},{"key":"e_1_2_1_92_1","doi-asserted-by":"crossref","unstructured":"Yangyong Zhang Lei Xu Abner Mendoza Guangliang Yang Phakpoom Chinprutthiwong and Guofei Gu. 2019. Life after Speech Recognition: Fuzzing Semantic Misinterpretation for Voice Assistant Applications.. In NDSS.  Yangyong Zhang Lei Xu Abner Mendoza Guangliang Yang Phakpoom Chinprutthiwong and Guofei Gu. 2019. Life after Speech Recognition: Fuzzing Semantic Misinterpretation for Voice Assistant Applications.. In NDSS.","DOI":"10.14722\/ndss.2019.23525"},{"key":"e_1_2_1_93_1","doi-asserted-by":"publisher","DOI":"10.1145\/2381934.2381950"},{"key":"e_1_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.1109\/TrustCom.2013.25"},{"key":"e_1_2_1_95_1","doi-asserted-by":"publisher","DOI":"10.1145\/2133601.2133640"},{"key":"e_1_2_1_96_1","first-page":"50","article-title":"Hey, you, get off of my market: detecting malicious apps in official and alternative android markets","volume":"25","author":"Zhou Yajin","year":"2012","unstructured":"Yajin Zhou , Zhi Wang , Wu Zhou , and Xuxian Jiang . 2012 . Hey, you, get off of my market: detecting malicious apps in official and alternative android markets .. In NDSS , Vol. 25. 50 -- 52 . Yajin Zhou, Zhi Wang, Wu Zhou, and Xuxian Jiang. 2012. Hey, you, get off of my market: detecting malicious apps in official and alternative android markets.. In NDSS, Vol. 25. 50--52.","journal-title":"NDSS"},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.1145\/2976749.2978304"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478101","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3478101","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:31:32Z","timestamp":1750188692000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478101"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,9]]},"references-count":97,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,9,9]]}},"alternative-id":["10.1145\/3478101"],"URL":"https:\/\/doi.org\/10.1145\/3478101","relation":{},"ISSN":["2474-9567"],"issn-type":[{"type":"electronic","value":"2474-9567"}],"subject":[],"published":{"date-parts":[[2021,9,9]]},"assertion":[{"value":"2021-09-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}