{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T09:50:43Z","timestamp":1763459443632,"version":"3.45.0"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2017,11,18]],"date-time":"2017-11-18T00:00:00Z","timestamp":1510963200000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000140","name":"U.S. Department of Transportation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000140","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CNS-1320175"],"award-info":[{"award-number":["CNS-1320175"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"publisher","award":["W911NF-15-1-0227, W911NF-14-1-0387, W911NF-16-1-0377"],"award-info":[{"award-number":["W911NF-15-1-0227, W911NF-14-1-0387, W911NF-16-1-0377"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Model. Perform. Eval. Comput. Syst."],"published-print":{"date-parts":[[2017,3,31]]},"abstract":"<jats:p>With the vast number of items, Web pages, and news from which to choose, online services and customers both benefit tremendously from personalized recommender systems. Such systems additionally provide great opportunities for targeted advertisements by displaying ads alongside genuine recommendations. We consider a biased recommendation system in which such ads are displayed without any tags (disguised as genuine recommendations), rendering them indistinguishable to a single user. We ask whether it is possible for a small subset of collaborating users to detect such bias. We propose an algorithm that can detect this type of bias through statistical analysis on the collaborating users\u2019 feedback. The algorithm requires only binary information indicating whether a user was satisfied with each of the recommended item or not. This makes the algorithm widely appealing to real-world issues such as identification of search engine bias and pharmaceutical lobbying. We prove that the proposed algorithm detects the bias with high probability for a broad class of recommendation systems when a sufficient number of users provides feedback on a sufficient number of recommendations. We provide extensive simulations with real datasets and practical recommender systems, which confirm the trade-offs in the theoretical guarantees.<\/jats:p>","DOI":"10.1145\/2988543","type":"journal-article","created":{"date-parts":[[2016,11,18]],"date-time":"2016-11-18T10:40:54Z","timestamp":1479465654000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Detecting Sponsored Recommendations"],"prefix":"10.1145","volume":"2","author":[{"given":"Subhashini","family":"Krishnasamy","sequence":"first","affiliation":[{"name":"University of Texas at Austin, Austin, TX"}]},{"given":"Rajat","family":"Sen","sequence":"additional","affiliation":[{"name":"University of Texas at Austin, Austin, TX"}]},{"given":"Sanjay","family":"Shakkottai","sequence":"additional","affiliation":[{"name":"University of Texas at Austin, Austin, TX"}]},{"given":"Sewoong","family":"Oh","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL"}]}],"member":"320","published-online":{"date-parts":[[2016,11,18]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Retrieved","author":"American Association of Advertising Agencies.","year":"2009","unstructured":"American Association of Advertising Agencies. 2009. Self-Regulatory Principles for Online Behavioral Advertising. Retrieved October 15, 2016, from http:\/\/www.aboutads.info\/resource\/download\/seven-principles-07-01-09.pdf."},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","unstructured":"John Abraham. 2002. The pharmaceutical industry as a political player. Lancet 360 9344 1498--1502.","DOI":"10.1016\/S0140-6736(02)11477-2"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-006-0036-4"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/376284.375668"},{"key":"e_1_2_1_5_1","volume-title":"Wall Street Journal. Retrieved","author":"Albergotti Reed","year":"2014","unstructured":"Reed Albergotti. 2014. Facebook to clean up news feeds. Wall Street Journal. Retrieved October 15, 2016, from http:\/\/www.wsj.com\/articles\/facebook-to-clean-up-news-feeds-1416002403."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611972719.28"},{"volume-title":"Research and Advanced Technology for Digital Libraries","author":"Beel Joeran","key":"e_1_2_1_7_1","unstructured":"Joeran Beel, Stefan Langer, and Marcel Genzmehr. 2013. Sponsored vs. organic (research paper) recommendations and the impact of labeling. In Research and Advanced Technology for Digital Libraries. Springer, 391--395."},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the KDD Cup and Workshop","volume":"2007","author":"Bennett James","year":"2007","unstructured":"James Bennett and Stan Lanning. 2007. The Netflix prize. In Proceedings of the KDD Cup and Workshop, Vol. 2007. 35."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1756-2171.2011.00143.x"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMhpr042734"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1214\/ss\/1042727940"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1150402.1150465"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1541880.1541882"},{"key":"e_1_2_1_14_1","volume-title":"Bing Blogs. Retrieved","author":"Crook Aidan","year":"2011","unstructured":"Aidan Crook and Sanaz Ahari. 2011. Making search yours. Bing Blogs. Retrieved October 15, 2016, from http:\/\/blogs.bing.com\/search\/2011\/02\/10\/making-search-yours."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242651"},{"key":"e_1_2_1_16_1","first-page":"9500","article-title":"The UK drug industry: Responsible, ethical, and professional","volume":"366","author":"Editorial Lancet","year":"2005","unstructured":"Lancet Editorial. 2005. The UK drug industry: Responsible, ethical, and professional? Lancet 366, 9500, 1828.","journal-title":"Lancet"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-008-9177-3"},{"volume-title":"Retrieved","year":"2014","key":"e_1_2_1_18_1","unstructured":"eMarketer. 2014. Native Advertising Roundup. Retrieved October 15, 2016, from https:\/\/www.emarketer.com\/public_media\/docs\/eMarketer_Native_Advertising_Roundup.pdf."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1460-2466.2006.00336.x"},{"key":"e_1_2_1_20_1","article-title":"How Google could end democracy","author":"Epstein Robert","year":"2014","unstructured":"Robert Epstein. 2014. How Google could end democracy. U.S. News and World Report. Retrieved October 15, 2016, from http:\/\/www.usnews.com\/opinion\/articles\/2014\/06\/09\/how-googles-search-rankings-could-manipulate-elections-and-end-democracy.","journal-title":"U.S. News and World Report. Retrieved"},{"key":"e_1_2_1_21_1","volume-title":"Retrieved","author":"Federal Trade Commission","year":"1983","unstructured":"Federal Trade Commission. 1983. FTC Policy Statement on Deception. Retrieved October 15, 2016, from https:\/\/www.ftc.gov\/public-statements\/1983\/10\/ftc-policy-statement-deception."},{"key":"e_1_2_1_22_1","volume-title":"Retrieved","author":"Federal Trade Commission","year":"2013","unstructured":"Federal Trade Commission. 2013. .com Disclosures: How to Make Effective Disclosures in Digital Advertising. Retrieved October 15, 2016, from https:\/\/www.ftc.gov\/system\/files\/documents\/plain-language\/bus41-dot-com-disclosures-information-about-online-advertising.pdf."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.0040150"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1198\/016214505000001131"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.1090.1054"},{"key":"e_1_2_1_26_1","volume-title":"Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients","author":"Goldacre Ben","year":"2014","unstructured":"Ben Goldacre. 2014. Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients. Macmillan."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1080\/01972243.2010.511560"},{"volume-title":"CCI charges Google with rigging search results","author":"Gupta Deepali","key":"e_1_2_1_28_1","unstructured":"Deepali Gupta. 2015. CCI charges Google with rigging search results; Flipkart, Facebook corroborate complaints. Economic Times. Available at http:\/\/economictimes.indiatimes.com."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.5555\/646111.679466"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-8655(03)00003-5"},{"volume-title":"Retrieved","year":"2014","key":"e_1_2_1_31_1","unstructured":"Hexagram. 2014. State of Native Advertising 2014. Retrieved October 15, 2016, from http:\/\/stateofnativeadvertising.hexagram.com\/."},{"key":"e_1_2_1_32_1","volume-title":"Google Official Blog. Retrieved","author":"Horling Bryan","year":"2009","unstructured":"Bryan Horling and Matthew Kulick. 2009. Personalized search for everyone. Google Official Blog. Retrieved October 15, 2016, from https:\/\/googleblog.blogspot.com\/2009\/12\/personalized-search-for-everyone.html."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.1060.0619"},{"key":"e_1_2_1_34_1","unstructured":"Interactive Advertising Bureau. 2010. Major Marketing\/Media Trade Groups Launch Program to Give Consumers Enhanced Control Over Collection and Use of Web Viewing Data for Online Behavioral Advertising. Available at https:\/\/www.iab.com."},{"key":"e_1_2_1_35_1","volume-title":"The Native Advertising Playbook. Retrieved","author":"Interactive Advertising Bureau","year":"2013","unstructured":"Interactive Advertising Bureau. 2013. IAB. The Native Advertising Playbook. Retrieved October 15, 2016, from http:\/\/www.iab.net\/media\/file\/IAB-Native-Advertising-Playbook2.pdf."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1080\/01972240050133634"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2010.2046205"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2745844.2745885"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/948109.948144"},{"key":"e_1_2_1_40_1","volume-title":"NSOM: A Real-Time Network-Based Intrusion Detection System Using Self-Organizing Maps. Technical Report. Department of Applied Science","author":"Labib Khaled","year":"2002","unstructured":"Khaled Labib and Rao Vemuri. 2002. NSOM: A Real-Time Network-Based Intrusion Detection System Using Self-Organizing Maps. Technical Report. Department of Applied Science, University of California, Davis."},{"key":"e_1_2_1_41_1","volume-title":"Vohra","author":"Lahaie S\u00e9bastien","year":"2007","unstructured":"S\u00e9bastien Lahaie, David M. Pennock, Amin Saberi, and Rakesh V. Vohra. 2007. Sponsored search auctions. In Algorithmic Game Theory, N. Nisan, T. Roughgarden, E. Tardos, and V. Vazirani (Eds.). Stanford University, Stanford, CA, 699--716."},{"key":"e_1_2_1_42_1","first-page":"2","article-title":"The neurontin legacy\u2014marketing through misinformation and manipulation","volume":"360","author":"Seth Landefeld C.","year":"2009","unstructured":"C. Seth Landefeld and Michael A. Steinman. 2009. The neurontin legacy\u2014marketing through misinformation and manipulation. New England Journal of Medicine 360, 2, 103.","journal-title":"New England Journal of Medicine"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.amjmed.2003.10.037"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611972733.3"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772758"},{"key":"e_1_2_1_46_1","unstructured":"Zhouchen Lin Minming Chen and Yi Ma. 2010. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.5055."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775102"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2507157.2507163"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.5555\/1076315"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/1278366.1278372"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2003.1260802"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/335191.335437"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/192844.192905"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/1134707.1134736"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1162\/089976601750264965"},{"key":"e_1_2_1_56_1","volume-title":"Global Issues. Retrieved","author":"Shah Anup","year":"2010","unstructured":"Anup Shah. 2010. Pharmaceutical corporations and medical research. Global Issues. Retrieved October 15, 2016, from http:\/\/globalissues.org\/article\/52\/pharmaceutical-corporations-and-medical-research."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2002.807277"},{"key":"e_1_2_1_58_1","volume-title":"Stanford Encyclopedia of Philosophy. Retrieved","author":"Tavani Herman","year":"2016","unstructured":"Herman Tavani. 2014. Search engines and ethics. In Stanford Encyclopedia of Philosophy. Retrieved October 15, 2016, from http:\/\/plato.stanford.edu\/entries\/ethics-search."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.1975897"},{"key":"e_1_2_1_60_1","first-page":"880","article-title":"System and method for implementing advertising in an online social network","volume":"12","author":"Turpin A. M.","year":"2008","unstructured":"A. M. Turpin and J. B. Katz. 2008. System and method for implementing advertising in an online social network. US Patent App.12\/011,880.","journal-title":"US Patent App."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1287\/mksc.1090.0552"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.5555\/846227.848602"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1007\/s101150200013"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1002\/bult.1720320211"}],"container-title":["ACM Transactions on Modeling and Performance Evaluation of Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2988543","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2988543","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2988543","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T09:46:08Z","timestamp":1763459168000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2988543"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,11,18]]},"references-count":64,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2017,3,31]]}},"alternative-id":["10.1145\/2988543"],"URL":"https:\/\/doi.org\/10.1145\/2988543","relation":{},"ISSN":["2376-3639","2376-3647"],"issn-type":[{"type":"print","value":"2376-3639"},{"type":"electronic","value":"2376-3647"}],"subject":[],"published":{"date-parts":[[2016,11,18]]},"assertion":[{"value":"2015-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-08-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-11-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}