{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T11:20:50Z","timestamp":1768908050232,"version":"3.49.0"},"reference-count":71,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T00:00:00Z","timestamp":1667779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2022,11,7]]},"abstract":"<jats:p>As intelligent voice assistants become more widespread and the scope of their listening increases, they become attractive targets for attackers. In the future, a malicious actor could train voice assistants to listen to audio outside their purview, creating a threat to users' privacy and security. How can this misbehavior be detected? Due to the ambiguities of natural language, people may need to work in conjunction with algorithms to determine whether a given conversation should be heard. To investigate how accurately humans can perform this task, we developed a framework for people to conduct \"Test Drives\" of always-listening services: after submitting sample conversations, users receive instant feedback about whether these would have been captured. Leveraging a Wizard of Oz interface, we conducted a study with 200 participants to determine whether they could detect one of four types of attacks on three different services. We studied the behavior of individuals, as well as groups working collaboratively, and investigated the effects of task framing on performance. We found that individuals were able to successfully detect malicious apps at varying rates (7.5% to 75%), depending on the type of malicious attack, and that groups were highly successful when considered collectively. Our results suggest that the Test Drive framework can be an effective tool for studying user behaviors and concerns, as well as a potentially welcome addition to voice assistant app stores, where it could decrease privacy concerns surrounding always-listening services.<\/jats:p>","DOI":"10.1145\/3555613","type":"journal-article","created":{"date-parts":[[2022,11,11]],"date-time":"2022-11-11T22:58:54Z","timestamp":1668207534000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Can Humans Detect Malicious Always-Listening Assistants? A Framework for Crowdsourcing Test Drives"],"prefix":"10.1145","volume":"6","author":[{"given":"Nathan","family":"Malkin","sequence":"first","affiliation":[{"name":"University of California, Berkeley, Berkeley, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Wagner","sequence":"additional","affiliation":[{"name":"University of California, Berkeley, Berkeley, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Serge","family":"Egelman","sequence":"additional","affiliation":[{"name":"University of California, Berkeley, USA &amp; International Computer Science Institute, Berkeley, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,11,11]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Fifteenth Symposium on Usable Privacy and Security (SOUPS 2019","author":"Abdi Noura","year":"2019","unstructured":"Noura Abdi , Kopo M. Ramokapane , and Jose M. Such . 2019. More than Smart Speakers : Security and Privacy Perceptions of Smart Home Personal Assistants . In Fifteenth Symposium on Usable Privacy and Security (SOUPS 2019 ). USENIX Association. https:\/\/www.usenix.org\/conference\/soups 2019 \/presentation\/abdi Noura Abdi, Kopo M. Ramokapane, and Jose M. Such. 2019. More than Smart Speakers : Security and Privacy Perceptions of Smart Home Personal Assistants. In Fifteenth Symposium on Usable Privacy and Security (SOUPS 2019). USENIX Association. https:\/\/www.usenix.org\/conference\/soups2019\/presentation\/abdi"},{"key":"e_1_2_2_2_1","volume-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.","volume":"5","author":"Adaimi Rebecca","year":"2021","unstructured":"Rebecca Adaimi , Howard Yong , and Edison Thomaz . 2021 . Ok Google, What Am I Doing? Acoustic Activity Recognition Bounded by Conversational Assistant Interactions . Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. , Vol. 5 , 1, Article 2 (March 2021). https:\/\/doi.org\/10.1145\/3448090 10.1145\/3448090 Rebecca Adaimi, Howard Yong, and Edison Thomaz. 2021. Ok Google, What Am I Doing? Acoustic Activity Recognition Bounded by Conversational Assistant Interactions. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. , Vol. 5, 1, Article 2 (March 2021). https:\/\/doi.org\/10.1145\/3448090"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2462456.2464460"},{"key":"e_1_2_2_4_1","unstructured":"Amazon. [n. d.]. Alexa Simulator. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/devconsole\/alexa-simulator.html  Amazon. [n. d.]. Alexa Simulator. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/devconsole\/alexa-simulator.html"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3311956"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196709.3196734"},{"key":"e_1_2_2_7_1","volume-title":"Amazon Halo Will Charge a Subscription Fee to Monitor the Tone of Your Voice. Ars Technica (Aug","author":"Axon Samuel","year":"2020","unstructured":"Samuel Axon . 2020. Amazon Halo Will Charge a Subscription Fee to Monitor the Tone of Your Voice. Ars Technica (Aug . 2020 ). https:\/\/arstechnica.com\/gadgets\/2020\/08\/amazon-halo-will-charge-a-subscription-fee-to-monitor-the-tone-of-your-voice\/ Samuel Axon. 2020. Amazon Halo Will Charge a Subscription Fee to Monitor the Tone of Your Voice. Ars Technica (Aug. 2020). https:\/\/arstechnica.com\/gadgets\/2020\/08\/amazon-halo-will-charge-a-subscription-fee-to-monitor-the-tone-of-your-voice\/"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1257\/jep.12.3.151"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675206"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2656211"},{"key":"e_1_2_2_11_1","volume-title":"PowerCut and Obfuscator: An Exploration of the Design Space for Privacy-Preserving Interventions for Smart Speakers. In Seventeenth Symposium on Usable Privacy and Security (SOUPS 2021","author":"Chandrasekaran Varun","year":"2021","unstructured":"Varun Chandrasekaran , Suman Banerjee , Bilge Mutlu , and Kassem Fawaz . 2021 . PowerCut and Obfuscator: An Exploration of the Design Space for Privacy-Preserving Interventions for Smart Speakers. In Seventeenth Symposium on Usable Privacy and Security (SOUPS 2021 ). USENIX Association, 535--552. https:\/\/www.usenix.org\/conference\/soups 2021\/presentation\/chandrasekaran Varun Chandrasekaran, Suman Banerjee, Bilge Mutlu, and Kassem Fawaz. 2021. PowerCut and Obfuscator: An Exploration of the Design Space for Privacy-Preserving Interventions for Smart Speakers. In Seventeenth Symposium on Usable Privacy and Security (SOUPS 2021). USENIX Association, 535--552. https:\/\/www.usenix.org\/conference\/soups2021\/presentation\/chandrasekaran"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376304"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3372297.3423339"},{"key":"e_1_2_2_14_1","volume-title":"Again. ZDNet (Oct.","author":"Cimpanu Catalin","year":"2019","unstructured":"Catalin Cimpanu . 2019. Alexa and Google Home Devices Leveraged to Phish and Eavesdrop on Users , Again. ZDNet (Oct. 2019 ). https:\/\/www.zdnet.com\/article\/alexa-and-google-home-devices-leveraged-to-phish-and-eavesdrop-on-users-again\/ Catalin Cimpanu. 2019. Alexa and Google Home Devices Leveraged to Phish and Eavesdrop on Users, Again. ZDNet (Oct. 2019). https:\/\/www.zdnet.com\/article\/alexa-and-google-home-devices-leveraged-to-phish-and-eavesdrop-on-users-again\/"},{"key":"e_1_2_2_15_1","volume-title":"Academics Smuggle 234 Policy-Violating Skills on the Alexa Skills Store. ZDNet (July","author":"Cimpanu Catalin","year":"2020","unstructured":"Catalin Cimpanu . 2020. Academics Smuggle 234 Policy-Violating Skills on the Alexa Skills Store. ZDNet (July 2020 ). https:\/\/www.zdnet.com\/article\/academics-smuggle-234-policy-violating-skills-on-the-alexa-skills-store\/ Catalin Cimpanu. 2020. Academics Smuggle 234 Policy-Violating Skills on the Alexa Skills Store. ZDNet (July 2020). https:\/\/www.zdnet.com\/article\/academics-smuggle-234-policy-violating-skills-on-the-alexa-skills-store\/"},{"key":"e_1_2_2_16_1","volume-title":"Proceedings of the 1st International Conference on Intelligent User Interfaces (IUI '93)","author":"Nils","year":"1993","unstructured":"Nils Dahlb\"ack, Arne J\u00f6nsson , and Lars Ahrenberg . 1993 . Wizard of Oz Studies: Why and How . In Proceedings of the 1st International Conference on Intelligent User Interfaces (IUI '93) . Association for Computing Machinery, New York, NY, USA, 193--200. https:\/\/doi.org\/10.1145\/169891.169968 10.1145\/169891.169968 Nils Dahlb\"ack, Arne J\u00f6nsson, and Lars Ahrenberg. 1993. Wizard of Oz Studies: Why and How. In Proceedings of the 1st International Conference on Intelligent User Interfaces (IUI '93). Association for Computing Machinery, New York, NY, USA, 193--200. https:\/\/doi.org\/10.1145\/169891.169968"},{"key":"e_1_2_2_17_1","volume-title":"Newly Released Amazon Patent Shows Just How Much Creepier Alexa Can Get. Science Alert (May","author":"Dockrill Peter","year":"2019","unstructured":"Peter Dockrill . 2019. Newly Released Amazon Patent Shows Just How Much Creepier Alexa Can Get. Science Alert (May 2019 ). https:\/\/www.sciencealert.com\/creepy-new-amazon-patent-would-mean-alexa-records-everything-you-say-from-now-on Peter Dockrill. 2019. Newly Released Amazon Patent Shows Just How Much Creepier Alexa Can Get. Science Alert (May 2019). https:\/\/www.sciencealert.com\/creepy-new-amazon-patent-would-mean-alexa-records-everything-you-say-from-now-on"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.2478\/popets-2020-0072"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2839509.2844619"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302262"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2501604.2501617"},{"key":"e_1_2_2_22_1","volume-title":"An Empirical Study of Vulnerability Rewards Programs. In 22nd USENIX Security Symposium (USENIX Security 13)","author":"Finifter Matthew","year":"2013","unstructured":"Matthew Finifter , Devdatta Akhawe , and David Wagner . 2013 . An Empirical Study of Vulnerability Rewards Programs. In 22nd USENIX Security Symposium (USENIX Security 13) . USENIX Association, Washington, D.C., 273--288. https:\/\/www.usenix.org\/conference\/usenixsecurity13\/technical-sessions\/presentation\/finifter Matthew Finifter, Devdatta Akhawe, and David Wagner. 2013. An Empirical Study of Vulnerability Rewards Programs. In 22nd USENIX Security Symposium (USENIX Security 13). USENIX Association, Washington, D.C., 273--288. https:\/\/www.usenix.org\/conference\/usenixsecurity13\/technical-sessions\/presentation\/finifter"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2110013119"},{"key":"e_1_2_2_24_1","volume-title":"29th USENIX Security Symposium (USENIX Security 20)","author":"Guo Zhixiu","year":"2020","unstructured":"Zhixiu Guo , Zijin Lin , Pan Li , and Kai Chen . 2020 . SkillExplorer : Understanding the Behavior of Skills in Large Scale . In 29th USENIX Security Symposium (USENIX Security 20) . USENIX Association, 2649--2666. https:\/\/www.usenix.org\/conference\/usenixsecurity20\/presentation\/guo Zhixiu Guo, Zijin Lin, Pan Li, and Kai Chen. 2020. SkillExplorer : Understanding the Behavior of Skills in Large Scale. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 2649--2666. https:\/\/www.usenix.org\/conference\/usenixsecurity20\/presentation\/guo"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2019.02.010"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376529"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/800045.801609"},{"key":"e_1_2_2_28_1","volume-title":"Researchers Find Amazon Alexa Can Be Hacked to Record Users. eWeek (April","author":"Kerner Sean Michael","year":"2018","unstructured":"Sean Michael Kerner . 2018. Researchers Find Amazon Alexa Can Be Hacked to Record Users. eWeek (April 2018 ). https:\/\/www.eweek.com\/security\/researchers-find-amazon-alexa-can-be-hacked-to-record-users Sean Michael Kerner. 2018. Researchers Find Amazon Alexa Can Be Hacked to Record Users. eWeek (April 2018). https:\/\/www.eweek.com\/security\/researchers-find-amazon-alexa-can-be-hacked-to-record-users"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1878101.1878112"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979321"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2810103.2813689"},{"key":"e_1_2_2_32_1","volume-title":"Skill Squatting Attacks on Amazon Alexa. In 27th USENIX Security Symposium (USENIX Security 18)","author":"Kumar Deepak","year":"2018","unstructured":"Deepak Kumar , Riccardo Paccagnella , Paul Murley , Eric Hennenfent , Joshua Mason , Adam Bates , and Michael Bailey . 2018 . Skill Squatting Attacks on Amazon Alexa. In 27th USENIX Security Symposium (USENIX Security 18) . USENIX Association, 33--47. https:\/\/www.usenix.org\/conference\/usenixsecurity18\/presentation\/kumar Deepak Kumar, Riccardo Paccagnella, Paul Murley, Eric Hennenfent, Joshua Mason, Adam Bates, and Michael Bailey. 2018. Skill Squatting Attacks on Amazon Alexa. In 27th USENIX Security Symposium (USENIX Security 18). USENIX Association, 33--47. https:\/\/www.usenix.org\/conference\/usenixsecurity18\/presentation\/kumar"},{"key":"e_1_2_2_33_1","volume-title":"Concerns and Privacy-seeking Behaviors with Smart Speakers. Proc. ACM Hum.-Comput. Interact.","volume":"2","author":"Lau Josephine","year":"2018","unstructured":"Josephine Lau , Benjamin Zimmerman , and Florian Schaub . 2018 . Alexa, Are You Listening ?: Privacy Perceptions , Concerns and Privacy-seeking Behaviors with Smart Speakers. Proc. ACM Hum.-Comput. Interact. , Vol. 2 , CSCW (Nov. 2018), 102:1--102:31. https:\/\/doi.org\/10.1145\/3274371 10.1145\/3274371 Josephine Lau, Benjamin Zimmerman, and Florian Schaub. 2018. Alexa, Are You Listening ?: Privacy Perceptions, Concerns and Privacy-seeking Behaviors with Smart Speakers. Proc. ACM Hum.-Comput. Interact. , Vol. 2, CSCW (Nov. 2018), 102:1--102:31. https:\/\/doi.org\/10.1145\/3274371"},{"key":"e_1_2_2_34_1","volume-title":"Turning an Echo Into a Spy Device Only Took Some Clever Coding. Wired (April","author":"Newman Lily Hay","year":"2018","unstructured":"Lily Hay Newman . 2018. Turning an Echo Into a Spy Device Only Took Some Clever Coding. Wired (April 2018 ). https:\/\/www.wired.com\/story\/amazon-echo-alexa-skill-spying\/ Lily Hay Newman. 2018. Turning an Echo Into a Spy Device Only Took Some Clever Coding. Wired (April 2018). https:\/\/www.wired.com\/story\/amazon-echo-alexa-skill-spying\/"},{"key":"e_1_2_2_35_1","volume-title":"An Alexa Bug Could Have Exposed Your Voice History to Hackers. Wired (Aug","author":"Newman Lily Hay","year":"2020","unstructured":"Lily Hay Newman . 2020. An Alexa Bug Could Have Exposed Your Voice History to Hackers. Wired (Aug . 2020 ). https:\/\/www.wired.com\/story\/amazon-alexa-bug-exposed-voice-history-hackers\/ Lily Hay Newman. 2020. An Alexa Bug Could Have Exposed Your Voice History to Hackers. Wired (Aug. 2020). https:\/\/www.wired.com\/story\/amazon-alexa-bug-exposed-voice-history-hackers\/"},{"key":"e_1_2_2_36_1","first-page":"2021","volume-title":"Proceedings on Privacy Enhancing Technologies","volume":"2021","author":"Liu Yuchen","year":"2021","unstructured":"Yuchen Liu , Ziyu Xiang , Eun Ji Seong , Apu Kapadia , and Donald S. Williamson . 2021 . Defending Against Microphone-Based Attacks with Personalized Noise. Proceedings on Privacy Enhancing Technologies , Vol. 2021 , 2 ( April 2021 ), 130--150. https:\/\/doi.org\/10.2478\/popets- 2021 - 0021 10.2478\/popets-2021-0021 Yuchen Liu, Ziyu Xiang, Eun Ji Seong, Apu Kapadia, and Donald S. Williamson. 2021. Defending Against Microphone-Based Attacks with Personalized Noise. Proceedings on Privacy Enhancing Technologies, Vol. 2021, 2 (April 2021), 130--150. https:\/\/doi.org\/10.2478\/popets-2021-0021"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1093\/cybsec\/tyx008"},{"key":"e_1_2_2_38_1","volume-title":"Marshini Chetty, and Nick Feamster.","author":"Major David J.","year":"2019","unstructured":"David J. Major , Danny Yuxing Huang , Marshini Chetty, and Nick Feamster. 2019 . Alexa, Who Am I Speaking To ? Understanding Users ' Ability to Identify Third-Party Apps on Amazon Alexa . (Oct. 2019). http:\/\/arxiv.org\/abs\/1910.14112 David J. Major, Danny Yuxing Huang, Marshini Chetty, and Nick Feamster. 2019. Alexa, Who Am I Speaking To ? Understanding Users ' Ability to Identify Third-Party Apps on Amazon Alexa. (Oct. 2019). http:\/\/arxiv.org\/abs\/1910.14112"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.2478\/popets-2019-0068"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368860.3368867"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/169059.169215"},{"key":"e_1_2_2_42_1","volume-title":"Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17)","author":"McGregor Moira","unstructured":"Moira McGregor and John C. Tang . 2017. More to Meetings : Challenges in Using Speech-Based Technology to Support Meetings . In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17) . ACM, New York, NY, USA, 2208--2220. https:\/\/doi.org\/10.1145\/2998181.2998335 10.1145\/2998181.2998335 Moira McGregor and John C. Tang. 2017. More to Meetings : Challenges in Using Speech-Based Technology to Support Meetings. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17). ACM, New York, NY, USA, 2208--2220. https:\/\/doi.org\/10.1145\/2998181.2998335"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702532"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.2478\/popets-2020-0026"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3321705.3329842"},{"key":"e_1_2_2_46_1","volume-title":"You Can Now Buy an Amazon Echo Add-on That Stops Alexa from Listening","author":"Newman Jared","year":"2020","unstructured":"Jared Newman . 2020. You Can Now Buy an Amazon Echo Add-on That Stops Alexa from Listening . Fast Company (July 2020 ). https:\/\/www.fastcompany.com\/90532150\/you-can-now-buy-an-amazon-echo-add-on-that-stops-alexa-from-listening Jared Newman. 2020. You Can Now Buy an Amazon Echo Add-on That Stops Alexa from Listening. Fast Company (July 2020). https:\/\/www.fastcompany.com\/90532150\/you-can-now-buy-an-amazon-echo-add-on-that-stops-alexa-from-listening"},{"key":"e_1_2_2_47_1","volume-title":"Large Effect: Measuring the Impact of User Reviews on Android App Security & Privacy. In 2019 IEEE Symposium on Security and Privacy (SP ). 555--569","author":"Nguyen D. C.","year":"2019","unstructured":"D. C. Nguyen , E. Derr , M. Backes , and S. Bugiel . 2019. Short Text , Large Effect: Measuring the Impact of User Reviews on Android App Security & Privacy. In 2019 IEEE Symposium on Security and Privacy (SP ). 555--569 . https:\/\/doi.org\/10.1109\/SP. 2019 .00012 10.1109\/SP.2019.00012 D. C. Nguyen, E. Derr, M. Backes, and S. Bugiel. 2019. Short Text, Large Effect: Measuring the Impact of User Reviews on Android App Security & Privacy. In 2019 IEEE Symposium on Security and Privacy (SP ). 555--569. https:\/\/doi.org\/10.1109\/SP.2019.00012"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1515\/popets-2018-0030"},{"key":"e_1_2_2_49_1","volume-title":"Tell Me about the Smart Speaker Market","author":"Paxton Mike","year":"2021","unstructured":"Mike Paxton . 2021. Alexa , Tell Me about the Smart Speaker Market in 2021 . https:\/\/www.spglobal.com\/marketintelligence\/en\/news-insights\/blog\/alexa-tell-me-about-the-smart-speaker-market-in-2021 Mike Paxton. 2021. Alexa, Tell Me about the Smart Speaker Market in 2021. https:\/\/www.spglobal.com\/marketintelligence\/en\/news-insights\/blog\/alexa-tell-me-about-the-smart-speaker-market-in-2021"},{"key":"e_1_2_2_50_1","volume-title":"Particularly Younger Users. Techcrunch (April","author":"Perez Sarah","year":"2020","unstructured":"Sarah Perez . 2020. COVID-19 Quarantine Boosts Smart Speaker Usage among U. S. Adults , Particularly Younger Users. Techcrunch (April 2020 ). https:\/\/techcrunch.com\/2020\/04\/30\/covid-19-quarantine-boosts-smart-speaker-usage-among-u-s-adults-particularly-younger-users\/ Sarah Perez. 2020. COVID-19 Quarantine Boosts Smart Speaker Usage among U. S. Adults, Particularly Younger Users. Techcrunch (April 2020). https:\/\/techcrunch.com\/2020\/04\/30\/covid-19-quarantine-boosts-smart-speaker-usage-among-u-s-adults-particularly-younger-users\/"},{"key":"e_1_2_2_51_1","volume-title":"Security Researchers Expose New Alexa and Google Home Vulnerability. The Verge (Oct","author":"Porter Jon","year":"2019","unstructured":"Jon Porter . 2019. Security Researchers Expose New Alexa and Google Home Vulnerability. The Verge (Oct . 2019 ). https:\/\/www.theverge.com\/2019\/10\/21\/20924886\/alexa-google-home-security-vulnerability-srlabs-phishing-eavesdropping Jon Porter. 2019. Security Researchers Expose New Alexa and Google Home Vulnerability. The Verge (Oct. 2019). https:\/\/www.theverge.com\/2019\/10\/21\/20924886\/alexa-google-home-security-vulnerability-srlabs-phishing-eavesdropping"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1515\/popets-2018-0021"},{"key":"e_1_2_2_53_1","first-page":"1","article-title":"Wizard of Oz Studies in HRI : A Systematic Review and New Reporting Guidelines","volume":"1","author":"Riek Laurel D.","year":"2012","unstructured":"Laurel D. Riek . 2012 . Wizard of Oz Studies in HRI : A Systematic Review and New Reporting Guidelines . J. Hum.-Robot Interact. , Vol. 1 , 1 (July 2012), 119--136. https:\/\/doi.org\/10.5898\/JHRI.1.1.Riek 10.5898\/JHRI.1.1.Riek Laurel D. Riek. 2012. Wizard of Oz Studies in HRI : A Systematic Review and New Reporting Guidelines. J. Hum.-Robot Interact. , Vol. 1, 1 (July 2012), 119--136. https:\/\/doi.org\/10.5898\/JHRI.1.1.Riek","journal-title":"J. Hum.-Robot Interact."},{"key":"e_1_2_2_54_1","volume-title":"The Curious Case of Machine Learning In Malware Detection. (May","author":"Saad Sherif","year":"2019","unstructured":"Sherif Saad , William Briguglio , and Haytham Elmiligi . 2019. The Curious Case of Machine Learning In Malware Detection. (May 2019 ). http:\/\/arxiv.org\/abs\/1905.07573 Sherif Saad, William Briguglio, and Haytham Elmiligi. 2019. The Curious Case of Machine Learning In Malware Detection. (May 2019). http:\/\/arxiv.org\/abs\/1905.07573"},{"key":"e_1_2_2_55_1","volume-title":"Where Is My Privacy? Exploring Accidental Triggers of Smart Speakers. arXiv:2008.00508 [cs] (Aug","author":"Sch\u00f6nherr Lea","year":"2020","unstructured":"Lea Sch\u00f6nherr , Maximilian Golla , Thorsten Eisenhofer , Jan Wiele , Dorothea Kolossa , and Thorsten Holz . 2020. Unacceptable , Where Is My Privacy? Exploring Accidental Triggers of Smart Speakers. arXiv:2008.00508 [cs] (Aug . 2020 ). arxiv: 2008.00508 [cs] http:\/\/arxiv.org\/abs\/2008.00508 Lea Sch\u00f6nherr, Maximilian Golla, Thorsten Eisenhofer, Jan Wiele, Dorothea Kolossa, and Thorsten Holz. 2020. Unacceptable, Where Is My Privacy? Exploring Accidental Triggers of Smart Speakers. arXiv:2008.00508 [cs] (Aug. 2020). arxiv: 2008.00508 [cs] http:\/\/arxiv.org\/abs\/2008.00508"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380179"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998208"},{"key":"e_1_2_2_58_1","volume-title":"Amazon Digital Assistant Patents Reveal Plans for Mass Snooping.","author":"Simpson John M.","year":"2017","unstructured":"John M. Simpson . 2017. Home Assistant Adopter Beware : Google , Amazon Digital Assistant Patents Reveal Plans for Mass Snooping. ( 2017 ). https:\/\/www.consumerwatchdog.org\/privacy-technology\/home-assistant-adopter-beware-google-amazon-digital-assistant-patents-reveal John M. Simpson. 2017. Home Assistant Adopter Beware : Google, Amazon Digital Assistant Patents Reveal Plans for Mass Snooping. (2017). https:\/\/www.consumerwatchdog.org\/privacy-technology\/home-assistant-adopter-beware-google-amazon-digital-assistant-patents-reveal"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3369807"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106290"},{"key":"e_1_2_2_61_1","volume-title":"Making Privacy Graspable : Can We Nudge Users to Use Privacy Enhancing Techniques ? arXiv:1911.07701 [cs] (Nov","author":"Tiefenau Christian","year":"2019","unstructured":"Christian Tiefenau , Maximilian H\"aring, Eva Gerlitz , and Emanuel von Zezschwitz . 2019. Making Privacy Graspable : Can We Nudge Users to Use Privacy Enhancing Techniques ? arXiv:1911.07701 [cs] (Nov . 2019 ). arxiv: 1911.07701 [cs] http:\/\/arxiv.org\/abs\/1911.07701 Christian Tiefenau, Maximilian H\"aring, Eva Gerlitz, and Emanuel von Zezschwitz. 2019. Making Privacy Graspable : Can We Nudge Users to Use Privacy Enhancing Techniques ? arXiv:1911.07701 [cs] (Nov. 2019). arxiv: 1911.07701 [cs] http:\/\/arxiv.org\/abs\/1911.07701"},{"key":"e_1_2_2_62_1","volume-title":"9th USENIX Workshop on Offensive Technologies (WOOT 15)","author":"Vaidya Tavish","year":"2015","unstructured":"Tavish Vaidya , Yuankai Zhang , Micah Sherr , and Clay Shields . 2015 . Cocaine Noodles: Exploiting the Gap between Human and Machine Speech Recognition . In 9th USENIX Workshop on Offensive Technologies (WOOT 15) . USENIX Association, Washington, D.C. https:\/\/www.usenix.org\/conference\/woot15\/workshop-program\/presentation\/vaidya Tavish Vaidya, Yuankai Zhang, Micah Sherr, and Clay Shields. 2015. Cocaine Noodles: Exploiting the Gap between Human and Machine Speech Recognition. In 9th USENIX Workshop on Offensive Technologies (WOOT 15). USENIX Association, Washington, D.C. https:\/\/www.usenix.org\/conference\/woot15\/workshop-program\/presentation\/vaidya"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445536"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376877"},{"key":"e_1_2_2_65_1","volume-title":"An Empirical Study of Bug Bounty Programs. In 2020 IEEE 2nd International Workshop on Intelligent Bug Fixing (IBF ). 35--44","author":"Walshe T.","year":"2020","unstructured":"T. Walshe and A. Simpson . 2020 . An Empirical Study of Bug Bounty Programs. In 2020 IEEE 2nd International Workshop on Intelligent Bug Fixing (IBF ). 35--44 . https:\/\/doi.org\/10.1109\/IBF50092. 2020 .9034828 10.1109\/IBF50092.2020.9034828 T. Walshe and A. Simpson. 2020. An Empirical Study of Bug Bounty Programs. In 2020 IEEE 2nd International Workshop on Intelligent Bug Fixing (IBF ). 35--44. https:\/\/doi.org\/10.1109\/IBF50092.2020.9034828"},{"key":"#cr-split#-e_1_2_2_66_1.1","doi-asserted-by":"crossref","unstructured":"R. Wang Z. Wang B. Tang L. Zhao and L. Wang. 2019. SmartPI : Understanding Permission Implications of Android Apps from User Reviews. IEEE Transactions on Mobile Computing (2019) 1--1. https:\/\/doi.org\/10.1109\/TMC.2019.2934441 10.1109\/TMC.2019.2934441","DOI":"10.1109\/TMC.2019.2934441"},{"key":"#cr-split#-e_1_2_2_66_1.2","doi-asserted-by":"crossref","unstructured":"R. Wang Z. Wang B. Tang L. Zhao and L. Wang. 2019. SmartPI : Understanding Permission Implications of Android Apps from User Reviews. IEEE Transactions on Mobile Computing (2019) 1--1. https:\/\/doi.org\/10.1109\/TMC.2019.2934441","DOI":"10.1109\/TMC.2019.2934441"},{"key":"e_1_2_2_67_1","volume-title":"CHI 2020 (2020","author":"Wei Jing","year":"2020","unstructured":"Jing Wei , Tilman Dingler , Enying Gong , Brian Oldenburg , and Vassilis Kostakos . 2020 . Proactive Smart Speakers for Chronic Disease Management: Challenges and Opportunities. \"Mapping Grand Challenges for the Conversational User Interface Community\" workshop , CHI 2020 (2020 ). http:\/\/www.speechinteraction.org\/CHI2020\/papers\/Proactive%20Smart%20Speakers%20for%20Chronic%20Disease%20Management(2).pdf Jing Wei, Tilman Dingler, Enying Gong, Brian Oldenburg, and Vassilis Kostakos. 2020. Proactive Smart Speakers for Chronic Disease Management: Challenges and Opportunities. \"Mapping Grand Challenges for the Conversational User Interface Community\" workshop, CHI 2020 (2020). http:\/\/www.speechinteraction.org\/CHI2020\/papers\/Proactive%20Smart%20Speakers%20for%20Chronic%20Disease%20Management(2).pdf"},{"key":"e_1_2_2_68_1","volume-title":"Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21)","author":"Wei Jing","unstructured":"Jing Wei , Tilman Dingler , and Vassilis Kostakos . 2021. Developing the Proactive Speaker Prototype Based on Google Home . In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21) . Association for Computing Machinery , New York, NY, USA , Article 292. https:\/\/doi.org\/10.1145\/3411763.3451642 10.1145\/3411763.3451642 Jing Wei, Tilman Dingler, and Vassilis Kostakos. 2021. Developing the Proactive Speaker Prototype Based on Google Home. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21). Association for Computing Machinery, New York, NY, USA, Article 292. https:\/\/doi.org\/10.1145\/3411763.3451642"},{"key":"e_1_2_2_69_1","volume-title":"Comput. Surveys","volume":"54","author":"Zhang-Kennedy Leah","year":"2021","unstructured":"Leah Zhang-Kennedy and Sonia Chiasson . 2021 . A Systematic Review of Multimedia Tools for Cybersecurity Awareness and Education . Comput. Surveys , Vol. 54 , 1, Article 12 (Jan. 2021). https:\/\/doi.org\/10.1145\/3427920 10.1145\/3427920 Leah Zhang-Kennedy and Sonia Chiasson. 2021. A Systematic Review of Multimedia Tools for Cybersecurity Awareness and Education. Comput. Surveys, Vol. 54, 1, Article 12 (Jan. 2021). https:\/\/doi.org\/10.1145\/3427920"},{"key":"e_1_2_2_70_1","volume-title":"Breaking Glass. CNET News (Dec.","author":"Zhou Marrian","year":"2018","unstructured":"Marrian Zhou . 2018. Amazon's Alexa Guard Can Alert You If an Echo Detects Smoke Alarm , Breaking Glass. CNET News (Dec. 2018 ). https:\/\/www.cnet.com\/news\/amazons-alexa-guard-can-alert-you-if-an-echo-detects-smoke-alarm-breaking-glass\/ Marrian Zhou. 2018. Amazon's Alexa Guard Can Alert You If an Echo Detects Smoke Alarm, Breaking Glass. CNET News (Dec. 2018). https:\/\/www.cnet.com\/news\/amazons-alexa-guard-can-alert-you-if-an-echo-detects-smoke-alarm-breaking-glass\/"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555613","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3555613","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:03Z","timestamp":1750182543000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555613"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,7]]},"references-count":71,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2022,11,7]]}},"alternative-id":["10.1145\/3555613"],"URL":"https:\/\/doi.org\/10.1145\/3555613","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,7]]},"assertion":[{"value":"2022-11-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}