{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T06:02:52Z","timestamp":1775023372339,"version":"3.50.1"},"reference-count":95,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T00:00:00Z","timestamp":1764633600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["NAIRR240418"],"award-info":[{"award-number":["NAIRR240418"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2025,12,2]]},"abstract":"<jats:p>Recent advances in large language models (LLMs) have enabled the creation of highly effective chatbots. However, the compute costs of widely deploying LLMs have raised questions about profitability. Companies have proposed exploring ad-based revenue streams for monetizing LLMs, which could serve as the new de facto platform for advertising. This paper investigates the implications of personalizing LLM advertisements to individual users via a between-subjects experiment with 179 participants. We developed a chatbot that embeds personalized product advertisements within LLM responses, inspired by similar forays by AI companies. The evaluation of our benchmarks showed that ad injection only slightly impacted LLM performance, particularly response desirability. Results revealed that participants struggled to detect ads, and even preferred LLM responses with hidden advertisements. Rather than clicking on our advertising disclosure, participants tried changing their advertising settings using natural language queries. We created an advertising dataset and an open-source LLM, Phi-4-Ads, fine-tuned to serve ads and flexibly adapt to user preferences.<\/jats:p>","DOI":"10.1145\/3770640","type":"journal-article","created":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T19:42:32Z","timestamp":1764704552000},"page":"1-47","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Ads that Talk Back: Implications and Perceptions of Injecting Personalized Advertising into LLM Chatbots"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6140-922X","authenticated-orcid":false,"given":"Brian Jay","family":"Tang","sequence":"first","affiliation":[{"name":"Computer Science and Engineering, University of Michigan, Ann Arbor, Michigan, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6568-2792","authenticated-orcid":false,"given":"Kaiwen","family":"Sun","sequence":"additional","affiliation":[{"name":"Informatics, Indiana University Bloomington, Bloomington, Indiana, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-8339-9297","authenticated-orcid":false,"given":"Noah T.","family":"Curran","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, University of Michigan, Ann Arbor, Michigan, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1039-7155","authenticated-orcid":false,"given":"Florian","family":"Schaub","sequence":"additional","affiliation":[{"name":"School of Information, University of Michigan, Ann Arbor, Michigan, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0086-8777","authenticated-orcid":false,"given":"Kang G.","family":"Shin","sequence":"additional","affiliation":[{"name":"Department of EECS, The University of Michigan Ann Arbor, Ann Arbor, Michigan, USA"}]}],"member":"320","published-online":{"date-parts":[[2025,12,2]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Topics API developer guide - Chrome Developers. https:\/\/developer.chrome.com\/docs\/privacy-sandbox\/topics [Online","year":"2023","unstructured":"1979. Topics API developer guide - Chrome Developers. https:\/\/developer.chrome.com\/docs\/privacy-sandbox\/topics [Online; accessed 7. Apr. 2023]."},{"key":"e_1_2_1_2_1","volume-title":"Disclosures 101 for Social Media Influencers. https:\/\/www.ftc.gov\/business-guidance\/resources\/disclosures-101-social-media-influencers [Online","year":"2024","unstructured":"2023. Disclosures 101 for Social Media Influencers. https:\/\/www.ftc.gov\/business-guidance\/resources\/disclosures-101-social-media-influencers [Online; accessed 12. Sep. 2024]."},{"key":"e_1_2_1_3_1","unstructured":"2023. How Amazon continues to improve the customer reviews experience with generative AI. https:\/\/www.aboutamazon.com\/news\/amazon-ai\/amazon-improves-customer-reviews-with-generative ai"},{"key":"e_1_2_1_4_1","unstructured":"2023. Reinventing search with a new AI-powered Microsoft Bing and Edge your copilot for the web. https:\/\/blogs.microsoft.com\/blog\/2023\/02\/07\/reinventing-search-with-a-new-ai-powered-microsoft-bing-and-edge-your-copilot-for-the-web\/"},{"key":"e_1_2_1_5_1","volume-title":"Microsoft Privacy Statement - Microsoft privacy. https:\/\/privacy.microsoft.com\/en-us\/privacystatement [Online","year":"2024","unstructured":"2024. Microsoft Privacy Statement - Microsoft privacy. https:\/\/privacy.microsoft.com\/en-us\/privacystatement [Online; accessed 9. Sep. 2024]."},{"key":"e_1_2_1_6_1","volume-title":"Is Destroying Productivity. https:\/\/hbr.org\/2025\/09\/ai-generated-workslop-is-destroying-productivity [Online","author":"Generated","year":"2025","unstructured":"2025. AI-Generated \u201cWorkslop\u201d Is Destroying Productivity. https:\/\/hbr.org\/2025\/09\/ai-generated-workslop-is-destroying-productivity [Online; accessed 30. Sep. 2025]."},{"key":"e_1_2_1_7_1","volume-title":"https:\/\/openai.com\/index\/chatgpt [Online","author":"Introducing","year":"2025","unstructured":"2025. Introducing ChatGPT. https:\/\/openai.com\/index\/chatgpt [Online; accessed 3. Oct. 2025]."},{"key":"e_1_2_1_8_1","volume-title":"Privacy and human behavior in the age of information. Science 347, 6221","author":"Acquisti Alessandro","year":"2015","unstructured":"Alessandro Acquisti, Laura Brandimarte, and George Loewenstein. 2015. Privacy and human behavior in the age of information. Science 347, 6221 (2015), 509\u2013514."},{"key":"e_1_2_1_9_1","volume-title":"Microsoft rolls out new chat ads API as part of AI blitz. Marketing Dive (May","author":"Adams Peter","year":"2023","unstructured":"Peter Adams. 2023. Microsoft rolls out new chat ads API as part of AI blitz. Marketing Dive (May 2023). https:\/\/www.marketingdive.com\/news\/Microsoft-Bing-chat-ads-api-generative-AI\/649660"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1177\/1464884918754829"},{"key":"e_1_2_1_11_1","volume-title":"The dark side of language models: Exploring the potential of llms in multimedia disinformation generation and dissemination. Machine Learning with Applications","author":"Barman Dipto","year":"2024","unstructured":"Dipto Barman, Ziyi Guo, and Owen Conlan. 2024. The dark side of language models: Exploring the potential of llms in multimedia disinformation generation and dissemination. Machine Learning with Applications (2024), 100545."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-008-0001-3"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445922"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1080\/00913367.2017.1339368"},{"key":"e_1_2_1_15_1","volume-title":"Language models are few-shot learners. arXiv preprint arXiv:2005.14165","author":"Brown Tom B","year":"2020","unstructured":"Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)."},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Erik Brynjolfsson Avinash Collis Asad Liaqat Daley Kutzman Haritz Garro Daniel Deisenroth and Nils Wernerfelt. 2024. The consumer welfare effects of online ads: Evidence from a 9-year experiment. Technical Report. National Bureau of Economic Research.","DOI":"10.3386\/w32846"},{"key":"e_1_2_1_17_1","volume-title":"Conversational marketing: How the world's fastest growing companies use chatbots to generate leads 24\/7\/365 (and how you can too)","author":"Cancel David","unstructured":"David Cancel and Dave Gerhardt. 2019. Conversational marketing: How the world's fastest growing companies use chatbots to generate leads 24\/7\/365 (and how you can too). John Wiley & Sons."},{"key":"e_1_2_1_18_1","volume-title":"Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al.","author":"Chen Mark","year":"2021","unstructured":"Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021)."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.techfore.2011.08.011"},{"key":"e_1_2_1_20_1","first-page":"1","article-title":"Palm: Scaling language modeling with pathways","volume":"24","author":"Chowdhery Aakanksha","year":"2023","unstructured":"Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research 24, 240 (2023), 1\u2013113.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_21_1","volume-title":"Training Verifiers to Solve Math Word Problems. arXiv preprint arXiv:2110.14168","author":"Cobbe Karl","year":"2021","unstructured":"Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training Verifiers to Solve Math Word Problems. arXiv preprint arXiv:2110.14168 (2021)."},{"key":"e_1_2_1_22_1","unstructured":"A Datta M Tschantz and A Datta. 2017. Discrimination and opacity in online behavioral advertising."},{"key":"e_1_2_1_23_1","volume-title":"Michael Carl Tschantz, and Anupam Datta","author":"Datta Amit","year":"2014","unstructured":"Amit Datta, Michael Carl Tschantz, and Anupam Datta. 2014. Automated experiments on ad privacy settings: A tale of opacity, choice, and discrimination. arXiv preprint arXiv:1408.6491 (2014)."},{"key":"e_1_2_1_24_1","volume-title":"DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. arXiv preprint arXiv:1903.00161","author":"Dua Dheeru","year":"2019","unstructured":"Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. arXiv preprint arXiv:1903.00161 (2019)."},{"key":"e_1_2_1_25_1","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1080\/00913367.2020.1765909","article-title":"A meta-analysis of the effects of disclosing sponsored content","volume":"49","author":"Eisend Martin","year":"2020","unstructured":"Martin Eisend, Eva A Van Reijmersdal, Sophie C Boerman, and Farid Tarrahi. 2020. A meta-analysis of the effects of disclosing sponsored content. Journal of Advertising 49, 3 (2020), 344\u2013366.","journal-title":"Journal of Advertising"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.comcom.2016.12.016"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1080\/02650487.2018.1474998"},{"key":"e_1_2_1_28_1","volume-title":"Online advertisements with llms: Opportunities and challenges. arXiv preprint arXiv:2311.07601","author":"Feizi Soheil","year":"2023","unstructured":"Soheil Feizi, MohammadTaghi Hajiaghayi, Keivan Rezaei, and Suho Shin. 2023. Online advertisements with llms: Opportunities and challenges. arXiv preprint arXiv:2311.07601 (2023)."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1177\/160940690600500107"},{"key":"e_1_2_1_30_1","volume-title":"Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination. arXiv preprint arXiv:2406.08818","author":"Fleisig Eve","year":"2024","unstructured":"Eve Fleisig, Genevieve Smith, Madeline Bossi, Ishita Rustagi, Xavier Yin, and Dan Klein. 2024. Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination. arXiv preprint arXiv:2406.08818 (2024)."},{"key":"e_1_2_1_31_1","volume-title":"The impact of generative ai and chatgpt on creating digital advertising campaigns. Cybernetics and Systems","author":"Go\u0142\u0105b-Andrzejak Edyta","year":"2023","unstructured":"Edyta Go\u0142\u0105b-Andrzejak. 2023. The impact of generative ai and chatgpt on creating digital advertising campaigns. Cybernetics and Systems (2023), 1\u201315."},{"key":"e_1_2_1_32_1","volume-title":"\u201c authentic","author":"Haimson Oliver L","year":"2016","unstructured":"Oliver L Haimson and Anna Lauren Hoffmann. 2016. Constructing and enforcing\u201c authentic\u201d identity online: Facebook, real names, and non-normative identities. First Monday (2016)."},{"key":"e_1_2_1_33_1","volume-title":"Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300","author":"Hendrycks Dan","year":"2020","unstructured":"Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 (2020)."},{"key":"e_1_2_1_34_1","volume-title":"Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874","author":"Hendrycks Dan","year":"2021","unstructured":"Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874 (2021)."},{"key":"e_1_2_1_35_1","first-page":"3","article-title":"Lora: Low-rank adaptation of large language models","volume":"1","author":"Hu Edward J","year":"2022","unstructured":"Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models. ICLR 1, 2 (2022), 3.","journal-title":"ICLR"},{"key":"e_1_2_1_36_1","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1080\/15252019.2023.2262456","article-title":"Investigating the Effects of Perceived Autonomy in Chatbot Advertising","volume":"23","author":"Hu Xiaohan","year":"2023","unstructured":"Xiaohan Hu, Xiaoyu Xu, and Chen Chen. 2023. Investigating the Effects of Perceived Autonomy in Chatbot Advertising. Journal of Interactive Advertising 23, 4 (2023), 323\u2013338.","journal-title":"Journal of Interactive Advertising"},{"key":"e_1_2_1_37_1","volume-title":"Anka Reuel, Max Lamparth, Kevin Feng, Lama Ahmad, Prajna Soni, Alia El Kattan, Merlin Stein, et al.","author":"Ibrahim Lujain","year":"2025","unstructured":"Lujain Ibrahim, Katherine M Collins, Sunnie SY Kim, Anka Reuel, Max Lamparth, Kevin Feng, Lama Ahmad, Prajna Soni, Alia El Kattan, Merlin Stein, et al. 2025. Measuring and mitigating overreliance is necessary for building human-compatible AI. arXiv preprint arXiv:2509.08010 (2025)."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2022.102921"},{"key":"e_1_2_1_39_1","volume-title":"Survey of hallucination in natural language generation. Comput. Surveys","author":"Ji Ziwei","year":"2022","unstructured":"Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2022. Survey of hallucination in natural language generation. Comput. Surveys (2022)."},{"key":"e_1_2_1_40_1","volume-title":"Proceedings of naacL-HLT","volume":"1","author":"Ming-Wei Chang Jacob Devlin","year":"2019","unstructured":"Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, Vol. 1. 2."},{"key":"e_1_2_1_41_1","volume-title":"Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust. In The 2024 ACM Conference on Fairness, Accountability, and Transparency. 822\u2013835","author":"Kim Sunnie SY","year":"2024","unstructured":"Sunnie SY Kim, Q Vera Liao, Mihaela Vorvoreanu, Stephanie Ballard, and Jennifer Wortman Vaughan. 2024. \u201c I'm Not Sure, But\u2026\u201d: Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust. In The 2024 ACM Conference on Fairness, Accountability, and Transparency. 822\u2013835."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1080\/00913367.2022.2043795"},{"key":"e_1_2_1_43_1","volume-title":"Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa.","author":"Kojima Takeshi","year":"2022","unstructured":"Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems 35 (2022), 22199\u201322213."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijresmar.2021.11.002"},{"key":"e_1_2_1_45_1","doi-asserted-by":"crossref","unstructured":"Nir Kshetri Yogesh K Dwivedi Thomas H Davenport and Niki Panteli. 2023. Generative artificial intelligence in marketing: Applications opportunities challenges and research agenda. 102716 pages.","DOI":"10.1016\/j.ijinfomgt.2023.102716"},{"key":"e_1_2_1_46_1","volume-title":"Transforming Search and Advertising with Generative AI. Microsoft advertising (Sept","author":"Kya Sainsbury-Carter","year":"2023","unstructured":"Sainsbury-Carter Kya. 2023. Transforming Search and Advertising with Generative AI. Microsoft advertising (Sept. 2023). https:\/\/about.ads.microsoft.com\/en\/blog\/post\/september-2023\/transforming-search-and-advertising-with-generative-ai"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1509\/jmr.11.0503"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2381966.2381970"},{"key":"e_1_2_1_49_1","unstructured":"Patrick Lewis Ethan Perez Aleksandra Piktus Fabio Petroni Vladimir Karpukhin Naman Goyal Heinrich K\u00fcttler Mike Lewis Wen-tau Yih Tim Rockt\u00e4schel et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33 (2020) 9459\u20139474."},{"key":"e_1_2_1_50_1","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1177\/1077699018782203","article-title":"The effect of preference stability and extremity on personalized advertising","volume":"96","author":"Li Cong","year":"2019","unstructured":"Cong Li, Jiangmeng Liu, and Cheng Hong. 2019. The effect of preference stability and extremity on personalized advertising. Journalism & mass communication quarterly 96, 2 (2019), 406\u2013427.","journal-title":"Journalism & mass communication quarterly"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","unstructured":"Q. Liao and J. Vaughan. 2023. AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap. ArXiv abs\/2306.01941 (2023). https:\/\/doi.org\/10.48550\/arXiv.2306.01941","DOI":"10.48550\/arXiv.2306.01941"},{"key":"e_1_2_1_52_1","volume-title":"Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958","author":"Lin Stephanie","year":"2021","unstructured":"Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 (2021)."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11747-016-0495-4"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359183"},{"key":"e_1_2_1_55_1","volume-title":"On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661","author":"Maynez Joshua","year":"2020","unstructured":"Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661 (2020)."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359174"},{"key":"e_1_2_1_57_1","volume-title":"chatbot-ui. https:\/\/github.com\/mckaywrigley\/chatbot-ui [Online","year":"2024","unstructured":"mckaywrigley. 2024. chatbot-ui. https:\/\/github.com\/mckaywrigley\/chatbot-ui [Online; accessed 14. Aug. 2024]."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.3390\/app13179765"},{"key":"e_1_2_1_59_1","volume-title":"Microsoft's new Bing AI chatbot is already insulting and gaslighting users","author":"Morris Chris","year":"2023","unstructured":"Chris Morris. 2023. Microsoft's new Bing AI chatbot is already insulting and gaslighting users. Fast Company (Feb. 2023). https:\/\/www.fastcompany.com\/90850277\/bing-new-chatgpt-ai-chatbot-insulting-gaslighting-users"},{"key":"e_1_2_1_60_1","volume-title":"Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02707","author":"Mukherjee Subhabrata","year":"2023","unstructured":"Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02707 (2023)."},{"key":"e_1_2_1_61_1","volume-title":"OpenAI explores advertising as it steps up revenue drive. Financial Times (Dec","author":"Murgia Madhumita","year":"2024","unstructured":"Madhumita Murgia, George Hammond, and Cristina Criddle. 2024. OpenAI explores advertising as it steps up revenue drive. Financial Times (Dec. 2024). https:\/\/www.ft.com\/content\/9350d075-1658-4d3c-8bc9-b9b3dfc29b26"},{"key":"e_1_2_1_62_1","volume-title":"Lawyers submitted bogus case law created by ChatGPT. A judge fined them $5,000. AP News (June","author":"Neumeister Larry","year":"2023","unstructured":"Larry Neumeister. 2023. Lawyers submitted bogus case law created by ChatGPT. A judge fined them $5,000. AP News (June 2023). https:\/\/apnews.com\/article\/artificial-intelligence-chatgpt-fake-case-lawyers-d6ae9fa79d0542db9e1455397aef381c"},{"key":"e_1_2_1_63_1","volume-title":"Evolving My AI with Sponsored Links powered by Microsoft Advertising. Snapchat Newsroom (Sept","author":"Newsroom Snapchat","year":"2023","unstructured":"Snapchat Newsroom. 2023. Evolving My AI with Sponsored Links powered by Microsoft Advertising. Snapchat Newsroom (Sept. 2023). https:\/\/newsroom.snap.com\/evolving-my-ai-with-sponsored-links-powered-by-microsoft-advertising"},{"key":"e_1_2_1_64_1","doi-asserted-by":"crossref","unstructured":"Elena Osadchaya Ben Marder Jennifer A Yule Amy Yau Laura Lavertu Nikolaos Stylos Sebastian Oliver Rob Angell et al. 2024. To ChatGPT or not to ChatGPT: Navigating the paradoxes of generative AI in the advertising industry. Business Horizons (2024).","DOI":"10.1016\/j.bushor.2024.05.002"},{"key":"e_1_2_1_65_1","unstructured":"Trishla Ostwal. 2024. Interactivity and Proprietary Data: Marketers Welcome Ads in Chatbots. https:\/\/www.adweek.com\/media\/why-marketers-welcome-ads-in-chatbots\/"},{"key":"e_1_2_1_66_1","unstructured":"Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L Wainwright Pamela Mishkin Chong Zhang Sandhini Agarwal Katarina Slama Alex Ray et al. 2022. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155 (2022)."},{"key":"e_1_2_1_67_1","volume-title":"Microsoft's Bing chatbot is getting more ads. Verge (March","author":"Peters Jay","year":"2023","unstructured":"Jay Peters. 2023. Microsoft's Bing chatbot is getting more ads. Verge (March 2023). https:\/\/www.theverge.com\/2023\/3\/29\/23662476\/microsoft-bing-chatbot-ads-revenue-sharing"},{"key":"e_1_2_1_68_1","volume-title":"Exploring User Perceptions of Discrimination in Online Targeted Advertising. In 26th USENIX Security Symposium (USENIX Security 17)","author":"Plane Angelisa C.","year":"2017","unstructured":"Angelisa C. Plane, Elissa M. Redmiles, Michelle L. Mazurek, and Michael Carl Tschantz. 2017. Exploring User Perceptions of Discrimination in Online Targeted Advertising. In 26th USENIX Security Symposium (USENIX Security 17). USENIX Association, Vancouver, BC, 935\u2013951. https:\/\/www.usenix.org\/conference\/usenixsecurity17\/technical-sessions\/presentation\/plane"},{"key":"e_1_2_1_69_1","unstructured":"A Radford. 2018. Improving language understanding by generative pre-training. (2018)."},{"key":"e_1_2_1_70_1","unstructured":"Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei Ilya Sutskever et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1 8 (2019) 9."},{"key":"e_1_2_1_71_1","volume-title":"Vice President Trishul Chilimbi, and Distinguished Scientist","author":"Rajiv Mehta Vice President","year":"2024","unstructured":"Vice President Rajiv Mehta, Vice President Trishul Chilimbi, and Distinguished Scientist. 2024. \u2018Amazon Rufus\u2019 AI experience comes to the Amazon Shopping app. US About Amazon (Feb. 2024). https:\/\/www.aboutamazon.com\/news\/retail\/amazon-rufus"},{"key":"e_1_2_1_72_1","volume-title":"Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman.","author":"Rein David","year":"2023","unstructured":"David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022 (2023)."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.nlp.2024.100056"},{"key":"e_1_2_1_74_1","volume-title":"The Coding Manual for Qualitative Researchers. The coding manual for qualitative researchers","author":"Salda\u00f1a Johnny","year":"2021","unstructured":"Johnny Salda\u00f1a. 2021. The Coding Manual for Qualitative Researchers. The coding manual for qualitative researchers (2021), 1\u2013440."},{"key":"e_1_2_1_75_1","volume-title":"Yi Tay, Sebastian Ruder, Denny Zhou, et al.","author":"Shi Freda","year":"2022","unstructured":"Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. 2022. Language models are multilingual chain-of-thought reasoners. arXiv preprint arXiv:2210.03057 (2022)."},{"key":"e_1_2_1_76_1","volume-title":"Laissez-Faire Harms: Algorithmic Biases in Generative Language Models. arXiv preprint arXiv:2404.07475","author":"Shieh Evan","year":"2024","unstructured":"Evan Shieh, Faye-Marie Vassel, Cassidy Sugimoto, and Thema Monroe-White. 2024. Laissez-Faire Harms: Algorithmic Biases in Generative Language Models. arXiv preprint arXiv:2404.07475 (2024)."},{"key":"e_1_2_1_77_1","volume-title":"The persuasive effects of political microtargeting in the age of generative artificial intelligence. PNAS nexus 3, 2","author":"Simchon Almog","year":"2024","unstructured":"Almog Simchon, Matthew Edwards, and Stephan Lewandowsky. 2024. The persuasive effects of political microtargeting in the age of generative artificial intelligence. PNAS nexus 3, 2 (2024), pgae035."},{"key":"e_1_2_1_78_1","volume-title":"Conference on fairness, accountability and transparency. PMLR, 5\u201319","author":"Speicher Till","year":"2018","unstructured":"Till Speicher, Muhammad Ali, Giridhari Venkatadri, Filipe Nunes Ribeiro, George Arvanitakis, Fabr\u00edcio Benevenuto, Krishna P Gummadi, Patrick Loiseau, and Alan Mislove. 2018. Potential for discrimination in online targeted advertising. In Conference on fairness, accountability and transparency. PMLR, 5\u201319."},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2216261120"},{"key":"e_1_2_1_80_1","volume-title":"Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971","author":"Touvron Hugo","year":"2023","unstructured":"Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth\u00e9e Lacroix, Baptiste Rozi\u00e8re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)."},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1145\/2335356.2335362"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2019.04.009"},{"key":"e_1_2_1_83_1","volume-title":"Attention is all you need. Advances in neural information processing systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_2_1_84_1","doi-asserted-by":"crossref","unstructured":"Jun Wang Weinan Zhang Shuai Yuan et al. 2017. Display advertising with real-time bidding (RTB) and behavioural targeting. Foundations and Trends\u00ae in Information Retrieval 11 4\u20135 (2017) 297\u2013435.","DOI":"10.1561\/1500000049"},{"key":"e_1_2_1_85_1","volume-title":"Denny Zhou, et al.","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824\u201324837."},{"key":"e_1_2_1_86_1","doi-asserted-by":"crossref","first-page":"299","DOI":"10.3390\/info15060299","article-title":"The Era of Artificial Intelligence Deception","volume":"15","author":"Williamson Steven M","year":"2024","unstructured":"Steven M Williamson and Victor Prybutok. 2024. The Era of Artificial Intelligence Deception: Unraveling the Complexities of False Realities and Emerging Threats of Misinformation. Information 15, 6 (2024), 299.","journal-title":"Information"},{"key":"e_1_2_1_87_1","volume-title":"Large language models as optimizers","author":"Yang Chengrun","year":"2023","unstructured":"Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023. Large language models as optimizers. 2023. Accessed on 1 (2023)."},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1145\/2501040.2501980"},{"key":"e_1_2_1_89_1","first-page":"120735","article-title":"A synthetic dataset for personal attribute inference","volume":"37","author":"Yukhymenko Hanna","year":"2024","unstructured":"Hanna Yukhymenko, Robin Staab, Mark Vero, and Martin Vechev. 2024. A synthetic dataset for personal attribute inference. Advances in Neural Information Processing Systems 37 (2024), 120735\u2013120779.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_90_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581388"},{"key":"e_1_2_1_91_1","volume-title":"LLM-Driven Agents for Influencer Selection in Digital Advertising Campaigns. arXiv preprint arXiv:2403.15105","author":"Zhang Xiaoqing","year":"2024","unstructured":"Xiaoqing Zhang, Xiuying Chen, Yuhan Liu, Jianzhou Wang, Zhenxing Hu, and Rui Yan. 2024. LLM-Driven Agents for Influencer Selection in Digital Advertising Campaigns. arXiv preprint arXiv:2403.15105 (2024)."},{"key":"e_1_2_1_92_1","volume-title":"Wildchat: 1m chatGPT interaction logs in the wild. arXiv preprint arXiv:2405.01470","author":"Zhao Wenting","year":"2024","unstructured":"Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. 2024. Wildchat: 1m chatGPT interaction logs in the wild. arXiv preprint arXiv:2405.01470 (2024)."},{"key":"e_1_2_1_93_1","unstructured":"Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu Yonghao Zhuang Zi Lin Zhuohan Li Dacheng Li Eric Xing et al. 2024. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems 36 (2024)."},{"key":"e_1_2_1_94_1","volume-title":"Large language models for information retrieval: A survey. arXiv preprint arXiv:2308.07107","author":"Zhu Yutao","year":"2023","unstructured":"Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Haonan Chen, Zhicheng Dou, and Ji-Rong Wen. 2023. Large language models for information retrieval: A survey. arXiv preprint arXiv:2308.07107 (2023)."},{"key":"e_1_2_1_95_1","doi-asserted-by":"publisher","DOI":"10.1057\/jit.2015.5"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3770640","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3770640","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T19:52:29Z","timestamp":1764705149000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3770640"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,2]]},"references-count":95,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12,2]]}},"alternative-id":["10.1145\/3770640"],"URL":"https:\/\/doi.org\/10.1145\/3770640","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,2]]},"assertion":[{"value":"2025-12-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}