{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,10]],"date-time":"2026-01-10T03:33:17Z","timestamp":1768015997137,"version":"3.49.0"},"reference-count":65,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,8,22]],"date-time":"2024-08-22T00:00:00Z","timestamp":1724284800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100006374","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2239605, 2228616, 2114920"],"award-info":[{"award-number":["2239605, 2228616, 2114920"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2024,8,22]]},"abstract":"<jats:p>The Amazon Alexa marketplace has grown rapidly in recent years due to third-party developers creating large amounts of content and publishing directly to a skills store. Despite the growth of the Amazon Alexa skills store, there have been several reported security and usability concerns, which may not be identified during the vetting phase. However, user reviews can offer valuable insights into the security &amp; privacy, quality, and usability of the skills. To better understand the effects of these problematic skills on end-users, we introduce ReviewTracker, a tool capable of discerning and classifying semantically negative user reviews to identify likely malicious, policy violating, or malfunctioning behavior on Alexa skills. ReviewTracker employs a pre-trained FastText classifier to identify different undesired skill behaviors. We collected over 700,000 user reviews spanning 6 years with more than 200,000 negative sentiment reviews. ReviewTracker was able to identify 17,820 reviews reporting violations related to Alexa policy requirements across 2,813 skills, and 131,855 reviews highlighting different types of user frustrations associated with 9,294 skills. In addition, we developed a dynamic skill testing framework using ChatGPT to conduct two distinct types of tests on Alexa skills: one using a software-based simulation for interaction to explore the actual behaviors of skills and another through actual voice commands to understand the potential factors causing discrepancies between intended skill functionalities and user experiences. Based on the number of the undesired skill behavior reviews, we tested the top identified problematic skills and detected more than 228 skills violating at least one policy requirement. Our results demonstrate that user reviews could serve as a valuable means to identify undesired skill behaviors.<\/jats:p>","DOI":"10.1145\/3678517","type":"journal-article","created":{"date-parts":[[2024,9,9]],"date-time":"2024-09-09T14:36:21Z","timestamp":1725892581000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["End-Users Know Best: Identifying Undesired Behavior of Alexa Skills Through User Review Analysis"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7603-0765","authenticated-orcid":false,"given":"Mohammed","family":"Aldeen","sequence":"first","affiliation":[{"name":"Clemson University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3233-4078","authenticated-orcid":false,"given":"Jeffrey","family":"Young","sequence":"additional","affiliation":[{"name":"Clemson University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5264-7573","authenticated-orcid":false,"given":"Song","family":"Liao","sequence":"additional","affiliation":[{"name":"Clemson University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-6300-9559","authenticated-orcid":false,"given":"Tsu-Yao","family":"Chang","sequence":"additional","affiliation":[{"name":"Clemson University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-4819-5406","authenticated-orcid":false,"given":"Long","family":"Cheng","sequence":"additional","affiliation":[{"name":"Clemson University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5224-9970","authenticated-orcid":false,"given":"Haipeng","family":"Cai","sequence":"additional","affiliation":[{"name":"Washington State University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9082-3208","authenticated-orcid":false,"given":"Xiapu","family":"Luo","sequence":"additional","affiliation":[{"name":"Hong Kong Polytechnic University, Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8710-247X","authenticated-orcid":false,"given":"Hongxin","family":"Hu","sequence":"additional","affiliation":[{"name":"University at Buffalo, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,9,9]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Health insurance portability and accountability act of","author":"Act Accountability","year":"1996","unstructured":"Accountability Act. 1996. Health insurance portability and accountability act of 1996. Public law 104 (1996), 191."},{"key":"e_1_2_1_2_1","unstructured":"LOVO AI. Year. Getting Started with Genny API. https:\/\/lovo.ai\/post\/getting-started-with-genny-api [Accessed: 23-Jan-2024]."},{"key":"e_1_2_1_3_1","volume-title":"Human Annotators: A Comprehensive Analysis of ChatGPT for Text Annotation. In 2023 International Conference on Machine Learning and Applications (ICMLA). IEEE, 602--609","author":"Aldeen Mohammed","year":"2023","unstructured":"Mohammed Aldeen, Joshua Luo, Ashley Lian, Venus Zheng, Allen Hong, Preethika Yetukuri, and Long Cheng. 2023. ChatGPT vs. Human Annotators: A Comprehensive Analysis of ChatGPT for Text Annotation. In 2023 International Conference on Machine Learning and Applications (ICMLA). IEEE, 602--609."},{"key":"e_1_2_1_4_1","unstructured":"Alexa Simulator. [n.d.]. https:\/\/developer.amazon.com\/en-US\/docs\/alexa\/devconsole\/alexa-simulator.html. [Accessed: 30-Jan-2024]."},{"key":"e_1_2_1_5_1","unstructured":"Alexa Skills Privacy Requirements. [n. d.]. https:\/\/developer.amazon.com\/fr-FR\/docs\/alexa\/custom-skills\/policy-requirements-for-an-alexa-skill.html. [Accessed: 22-May-2023]."},{"key":"e_1_2_1_6_1","unstructured":"Alexa Skills Security Requirements. [n. d.]. https:\/\/developer.amazon.com\/fr-FR\/docs\/alexa\/custom-skills\/security-testing-for-an-alexa-skill.html. [Accessed: 22-May-2023]."},{"key":"e_1_2_1_7_1","volume-title":"Different App Stores: A Comparative Study. In 2017 IEEE\/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft). 79--90","author":"Ali M.","unstructured":"M. Ali, M. E. Joorabchi, and A. Mesbah. 2017. Same App, Different App Stores: A Comparative Study. In 2017 IEEE\/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft). 79--90."},{"key":"e_1_2_1_8_1","volume-title":"Iftekhar Ahmed, and Amin Alipour.","author":"Atefi Soodeh","year":"2020","unstructured":"Soodeh Atefi, Andrew Truelove, Matheus Rheinschmitt, Eduardo Santana de Almeida, Iftekhar Ahmed, and Amin Alipour. 2020. Examining user reviews of conversational systems: a case study of Alexa skills. CoRR abs\/2003.00919 (2020). arXiv:2003.00919 https:\/\/arxiv.org\/abs\/2003.00919"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/365024.365304"},{"key":"e_1_2_1_10_1","first-page":"993","article-title":"Latent dirichlet allocation","author":"Blei David M","year":"2003","unstructured":"David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993--1022.","journal-title":"Journal of machine Learning research 3"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"e_1_2_1_12_1","volume-title":"Dangerous Skills Got Certified: Measuring the Trustworthiness of Skill Certification in Voice Personal Assistant Platforms. In ACM SIGSAC Conference on Computer and Communications Security (CCS).","author":"Cheng Long","year":"2020","unstructured":"Long Cheng, Christin Wilson, Song Liao, Jeffrey Young, Daniel Dong, and Hongxin Hu. 2020. Dangerous Skills Got Certified: Measuring the Trustworthiness of Skill Certification in Voice Personal Assistant Platforms. In ACM SIGSAC Conference on Computer and Communications Security (CCS)."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300705"},{"key":"e_1_2_1_14_1","volume-title":"A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1","author":"Cohen Jacob","year":"1960","unstructured":"Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37--46."},{"key":"e_1_2_1_15_1","unstructured":"Federal Trade Commission et al. 1998. Children's online privacy protection act of 1998 (COPPA)."},{"key":"e_1_2_1_16_1","volume-title":"Unsupervised cross-lingual representation learning for speech recognition. arXiv preprint arXiv:2006.13979","author":"Conneau Alexis","year":"2020","unstructured":"Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli. 2020. Unsupervised cross-lingual representation learning for speech recognition. arXiv preprint arXiv:2006.13979 (2020)."},{"key":"e_1_2_1_17_1","volume-title":"Jacson Rodrigues Barbosa, and Ricardo Marcondes Marcacini","author":"Alves de Lima Vitor Mesaque","year":"2023","unstructured":"Vitor Mesaque Alves de Lima, Jacson Rodrigues Barbosa, and Ricardo Marcondes Marcacini. 2023. Learning Risk Factors from App Reviews: A Large Language Model Approach for Risk Matrix Construction. (2023)."},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. 499--510","author":"Sorbo Andrea Di","unstructured":"Andrea Di Sorbo, Sebastiano Panichella, Carol V. Alexandru, Junji Shimagaki, Corrado A. Visaggio, Gerardo Canfora, and Harald C. Gall. 2016. What Would Users Change in My App? Summarizing App Reviews for Recommending Software Changes. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. 499--510."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3638067.3638081"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.21659\/rupkatha.v15n4.10"},{"key":"e_1_2_1_21_1","volume-title":"Jose Such, and Guillermo Suarez-Tangil.","author":"Edu Jide","year":"2021","unstructured":"Jide Edu, Xavi Ferrer Aran, Jose Such, and Guillermo Suarez-Tangil. 2021. SkillVet: Automated Traceability Analysis of Amazon Alexa Skills. IEEE Transactions on Dependable and Secure Computing (2021)."},{"key":"e_1_2_1_22_1","unstructured":"Anna Fleck. 2023. Alexa What's America's Favorite Smart Speaker? Statista. https:\/\/www.statista.com\/chart\/23943\/share-of-us-adults-who-own-smart-speakers\/ [Accessed: 01-May-2024]."},{"key":"e_1_2_1_23_1","volume-title":"Consumer Attitudes Towards Privacy and Security in Home Assistants. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems.","author":"Fruchter Nathaniel","year":"2018","unstructured":"Nathaniel Fruchter and Ilaria Liccardi. 2018. Consumer Attitudes Towards Privacy and Security in Home Assistants. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems."},{"key":"e_1_2_1_24_1","volume-title":"The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD. 1276--1284","author":"Fu Bin","unstructured":"Bin Fu, Jialiu Lin, Lei Li, Christos Faloutsos, Jason I. Hong, and Norman M. Sadeh. 2013. Why people hate your app: making sense of user feedback in a mobile app store. In The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD. 1276--1284."},{"key":"e_1_2_1_25_1","volume-title":"Online App Review Analysis for Identifying Emerging Issues. In 2018 IEEE\/ACM 40th International Conference on Software Engineering (ICSE). 48--58","author":"Gao C.","unstructured":"C. Gao, J. Zeng, M. R. Lyu, and I. King. 2018. Online App Review Analysis for Identifying Emerging Issues. In 2018 IEEE\/ACM 40th International Conference on Software Engineering (ICSE). 48--58."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/SmartWorld.2018.00094"},{"key":"e_1_2_1_27_1","volume-title":"irogue: Identifying rogue behavior from app reviews. arXiv preprint arXiv:2303.10795","author":"Garg Vaibhav","year":"2023","unstructured":"Vaibhav Garg, Hui Guo, Nirav Ajmeri, Saikath Bhattacharya, and Munindar P Singh. 2023. irogue: Identifying rogue behavior from app reviews. arXiv preprint arXiv:2303.10795 (2023)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-73450-7_72"},{"key":"e_1_2_1_29_1","volume-title":"29th {USENIX} Security Symposium ({USENIX} Security 20). 2649--2666.","author":"Guo Zhixiu","unstructured":"Zhixiu Guo, Zijin Lin, Pan Li, and Kai Chen. 2020. SkillExplorer: Understanding the Behavior of Skills in Large Scale. In 29th {USENIX} Security Symposium ({USENIX} Security 20). 2649--2666."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP46214.2022.9833729"},{"key":"e_1_2_1_31_1","unstructured":"Drew Harwell. 2018. Amazon's Alexa and Google Home show accent bias with Chinese and Spanish hardest to understand. (2018). https:\/\/www.scmp.com\/magazines\/post-magazine\/long-reads\/article\/2156455\/amazons-alexa-and-google-home-show-accent-bias"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.digbus.2020.100001"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/312624.312649"},{"key":"e_1_2_1_34_1","unstructured":"Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings convolutional neural networks and incremental parsing. (2017). To appear."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00089"},{"key":"e_1_2_1_36_1","volume-title":"Shyam Sundar Sharma, and Rajkumar Kaushik","author":"Jaiswal Rajesh Kumar","year":"2023","unstructured":"Rajesh Kumar Jaiswal, Shyam Sundar Sharma, and Rajkumar Kaushik. 2023. ETHICS IN AI AND MACHINE LEARNING. Journal of Nonlinear Analysis and Optimization (2023). https:\/\/api.semanticscholar.org\/CorpusID:266320296"},{"key":"e_1_2_1_37_1","volume-title":"The menlo report: Ethical principles guiding information and communication technology research. Available at SSRN 2445102","author":"Kenneally Erin","year":"2012","unstructured":"Erin Kenneally and David Dittrich. 2012. The menlo report: Ethical principles guiding information and communication technology research. Available at SSRN 2445102 (2012)."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1915768117"},{"key":"e_1_2_1_39_1","volume-title":"CLARIN-Emo: Training Emotion Recognition Models Using Human Annotation and ChatGPT. In International Conference on Computational Science. Springer, 365--379","author":"Koptyra Bart\u0142omiej","year":"2023","unstructured":"Bart\u0142omiej Koptyra, Anh Ngo, \u0141ukasz Radli\u0144ski, and Jan Koco\u0144. 2023. CLARIN-Emo: Training Emotion Recognition Models Using Human Annotation and ChatGPT. In International Conference on Computational Science. Springer, 365--379."},{"key":"e_1_2_1_40_1","volume-title":"Skill Squatting Attacks on Amazon Alexa. In 27th USENIX Security Symposium (USENIX Security). 33--47","author":"Kumar Deepak","year":"2018","unstructured":"Deepak Kumar, Riccardo Paccagnella, Paul Murley, Eric Hennenfent, Joshua Mason, Adam Bates, and Michael Bailey. 2018. Skill Squatting Attacks on Amazon Alexa. In 27th USENIX Security Symposium (USENIX Security). 33--47."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3539609"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2021.23111"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556957"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589334.3645409"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858288"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-5010"},{"key":"e_1_2_1_47_1","volume-title":"Using Foundation Models to Detect Policy Violations with Minimal Supervision. arXiv preprint arXiv:2306.06234","author":"Mittal Sid","year":"2023","unstructured":"Sid Mittal, Vineet Gupta, Frederick Liu, and Mukund Sundararajan. 2023. Using Foundation Models to Detect Policy Violations with Minimal Supervision. arXiv preprint arXiv:2306.06234 (2023)."},{"key":"e_1_2_1_48_1","volume-title":"Large Effect: Measuring the Impact of User Reviews on Android App Security Privacy. In 2019 IEEE Symposium on Security and Privacy (SP). 555--569","author":"Nguyen D. C.","unstructured":"D. C. Nguyen, E. Derr, M. Backes, and S. Bugiel. 2019. Short Text, Large Effect: Measuring the Impact of User Reviews on Android App Security Privacy. In 2019 IEEE Symposium on Security and Privacy (SP). 555--569."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.3390\/socsci12030148"},{"key":"e_1_2_1_50_1","unstructured":"Selenium WebDriver. [n. d.]. https:\/\/www.selenium.dev."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432233"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432233"},{"key":"e_1_2_1_53_1","volume-title":"Social Media Data-Our Ethical Conundrum. A Quarterly bulletin of the IEEE Computer Society Technical Committee on Database Engineering","author":"Singh Lisa","year":"2020","unstructured":"Lisa Singh, Agoritsa Polyzou, Yanchen Wang, Jason Farr, and Carole Roan Gresenz. 2020. Social Media Data-Our Ethical Conundrum. A Quarterly bulletin of the IEEE Computer Society Technical Committee on Database Engineering (2020)."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106290"},{"key":"e_1_2_1_55_1","unstructured":"Venture Beat. [n. d.]. https:\/\/venturebeat.com\/2019\/09\/25\/the-alexa-skills-store-now-has-more-than-100000-voice-apps\/."},{"key":"e_1_2_1_56_1","volume-title":"32nd USENIX Security Symposium (USENIX Security 23)","author":"Vetrivel Swaathi","year":"2023","unstructured":"Swaathi Vetrivel, Veerle Van Harten, Carlos H Ga\u00f1\u00e1n, Michel Van Eeten, and Simon Parkin. 2023. Examining consumer reviews to understand security and privacy issues in the market of smart home devices. In 32nd USENIX Security Symposium (USENIX Security 23). 1523--1540."},{"key":"e_1_2_1_57_1","volume-title":"Proceedings of the 30th IEEE\/ACM International Conference on Automated Software Engineering. 749--459","author":"Vu Phong Minh","year":"2015","unstructured":"Phong Minh Vu, Tam The Nguyen, Hung Viet Pham, and Tung Thanh Nguyen. 2015. Mining User Opinions in Mobile App Reviews: A Keyword-Based Approach. In Proceedings of the 30th IEEE\/ACM International Conference on Automated Software Engineering. 749--459."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3507657.3528558"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478101"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2019.2934441"},{"key":"e_1_2_1_61_1","unstructured":"Jeffrey Young Song Liao Long Cheng Hongxin Hu and Huixing Deng. 2021. SkillDetective: Automated Policy-Violation Detection of Voice Assistant Applications in the Wild."},{"key":"e_1_2_1_62_1","volume-title":"Localizing Function Errors in Mobile Apps with User Reviews. In 2018 48th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN). 418--429","author":"Yu L.","unstructured":"L. Yu, J. Chen, H. Zhou, X. Luo, and K. Liu. 2018. Localizing Function Errors in Mobile Apps with User Reviews. In 2018 48th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN). 418--429."},{"key":"e_1_2_1_63_1","volume-title":"Understanding and Mitigating the Security Risks of Voice-Controlled Third-Party Skills on Amazon Alexa and Google Home. In IEEE Symposium on Security and Privacy (SP).","author":"Zhang Nan","year":"2019","unstructured":"Nan Zhang, Xianghang Mi, Xuan Feng, XiaoFeng Wang, Yuan Tian, and Feng Qian. 2019. Understanding and Mitigating the Security Risks of Voice-Controlled Third-Party Skills on Amazon Alexa and Google Home. In IEEE Symposium on Security and Privacy (SP)."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/QRS57517.2022.00115"},{"key":"e_1_2_1_65_1","volume-title":"Why does chatgpt fall short in answering questions faithfully? arXiv preprint arXiv:2304.10513","author":"Zheng Shen","year":"2023","unstructured":"Shen Zheng, Jie Huang, and Kevin Chen-Chuan Chang. 2023. Why does chatgpt fall short in answering questions faithfully? arXiv preprint arXiv:2304.10513 (2023)."}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678517","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3678517","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T14:44:06Z","timestamp":1755787446000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678517"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,22]]},"references-count":65,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,8,22]]}},"alternative-id":["10.1145\/3678517"],"URL":"https:\/\/doi.org\/10.1145\/3678517","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,22]]},"assertion":[{"value":"2024-09-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}