{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T07:46:55Z","timestamp":1767944815382,"version":"3.49.0"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,2,11]],"date-time":"2021-02-11T00:00:00Z","timestamp":1613001600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,2,11]],"date-time":"2021-02-11T00:00:00Z","timestamp":1613001600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Encouraging people to vaccinate is a challenging endeavor, but one which has tremendous public health benefits. Doing so requires overcoming barriers of awareness, availability, and (sometimes) vaccine hesitancy. Here we focus on nudging people to vaccinate through online advertising. We conducted a pre-registered online ads campaign encouraging people to vaccinate against three diseases: influenza, human papillomavirus, and herpes zoster. Ads were shown to ~69,000 people and were compared to similar ads shown to 8.6 million people. Outcome measures were clicks on ads and future searches for relevant terms. We find that ads have two main effects: First, a congruence effect whereby ads increase the likelihood of clicks and future searches by up to 116% in people who express an interest in the disease or the vaccine. Most commercial vaccine advertising is aimed entirely at this population. Second, we observed a priming effect, where ads shown to people who were searching for terms unrelated to the vaccine could be encouraged to click on them (odds ratios of 7.5\u201333.0) and, more often, search for the vaccine later (hazard ratios of 6.9\u2013157.3). We provide analysis for optimizing vaccine advertising campaign budgets to balance the two populations. These findings demonstrate that digital advertising campaigns should consider not just advertising to direct keywords or to individuals that look exactly like existing customers, but consider tangential keywords that draw a wider target population who are likely earlier in their conversion funnel, thus increasing the number of people who vaccinate and maximizing vaccines uptake.<\/jats:p>","DOI":"10.1038\/s41746-021-00395-7","type":"journal-article","created":{"date-parts":[[2021,2,11]],"date-time":"2021-02-11T11:03:58Z","timestamp":1613041438000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Vaccine advertising: preach to the converted or to the unaware?"],"prefix":"10.1038","volume":"4","author":[{"given":"Masha","family":"Krupenkin","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2380-4584","authenticated-orcid":false,"given":"Elad","family":"Yom-Tov","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Rothschild","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,2,11]]},"reference":[{"key":"395_CR1","unstructured":"Organization, W. H. Immunization coverage. https:\/\/www.who.int\/en\/news-room\/fact-sheets\/detail\/immunization-coverage (2019)."},{"key":"395_CR2","unstructured":"Centers for Disease Control and Prevention. Vaccination coverage among adults in the united states, national health interview survey, 2017. https:\/\/www.cdc.gov\/vaccines\/imz-managers\/coverage\/adultvaxview\/pubs-resources\/NHIS-2017.html (2017)."},{"key":"395_CR3","doi-asserted-by":"crossref","unstructured":"Garcia-Molina, H., Koutrika, G. & Parameswaran, A. Information seeking: Convergence of search, recommendations and advertising. Communications of the ACM (2011).","DOI":"10.1145\/2018396.2018423"},{"key":"395_CR4","unstructured":"Statista: Global no.1 business data platform. https:\/\/www.statista.com\/outlook\/219\/109\/search-advertising\/united-states#market-revenue (2019)."},{"key":"395_CR5","unstructured":"Statista: Global no.1 business data platform. https:\/\/www.statista.com\/outlook\/219\/100\/search-advertising\/worldwide#market-revenue (2019)."},{"key":"395_CR6","doi-asserted-by":"publisher","first-page":"e306","DOI":"10.2196\/jmir.6563","volume":"18","author":"E Yom-Tov","year":"2016","unstructured":"Yom-Tov, E., Muennig, P. & El-Sayed, A. M. Web-based antismoking advertising to promote smoking cessation: A randomized controlled trial. J. Med. Internet Res. 18, e306 (2016).","journal-title":"J. Med. Internet Res."},{"key":"395_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-017-0008-y","volume":"1","author":"E Yom-Tov","year":"2018","unstructured":"Yom-Tov, E., Shembekar, J., Barclay, S. & Muennig, P. The effectiveness of public health advertisements to promote health: A randomized-controlled trial on 794,000 participants. NPJ Digital Med. 1, 1\u20136 (2018).","journal-title":"NPJ Digital Med."},{"key":"395_CR8","doi-asserted-by":"publisher","first-page":"e6","DOI":"10.2196\/mental.8212","volume":"5","author":"E Yom-Tov","year":"2018","unstructured":"Yom-Tov, E., Brunstein-Klomek, A., Mandel, O., Hadas, A. & Fennig, S. Inducing behavioral change in seekers of pro-anorexia content using internet advertisements: Randomized controlled trial. JMIR Mental Health 5, e6 (2018).","journal-title":"JMIR Mental Health"},{"key":"395_CR9","first-page":"235","volume":"12","author":"RA Lewis","year":"2014","unstructured":"Lewis, R. A. & Reiley, D. H. Online ads and offline sales: measuring the effect of retail advertising via a controlled experiment on Yahoo! Quantitative Market. Economics 12, 235\u2013266 (2014).","journal-title":"Economics"},{"key":"395_CR10","doi-asserted-by":"publisher","first-page":"896","DOI":"10.1016\/j.sapharm.2014.01.003","volume":"10","author":"JE Simmering","year":"2014","unstructured":"Simmering, J. E., Polgreen, L. A. & Polgreen, P. M. Web search query volume as a measure of pharmaceutical utilization and changes in prescribing patterns. Res. Social Administrative Pharmacy 10, 896\u2013903 (2014).","journal-title":"Res. Social Administrative Pharmacy"},{"key":"395_CR11","doi-asserted-by":"publisher","first-page":"448","DOI":"10.1016\/j.amepre.2010.12.007","volume":"40","author":"JW Ayers","year":"2011","unstructured":"Ayers, J. W., Ribisl, K. M. & Brownstein, J. S. Tracking the rise in popularity of electronic nicotine delivery systems (electronic cigarettes) using search query surveillance. Am. J. Preventive Med. 40, 448\u2013453 (2011).","journal-title":"Am. J. Preventive Med."},{"key":"395_CR12","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1007\/s10018-017-0196-3","volume":"20","author":"T Liu","year":"2018","unstructured":"Liu, T., He, G. & Lau, A. Avoidance behavior against air pollution: evidence from online search indices for anti-pm 2.5 masks and air filters in Chinese cities. Environ. Economic Policy Studies 20, 325\u2013363 (2018).","journal-title":"Economic Policy Studies"},{"key":"395_CR13","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pntd.0001258","volume":"5","author":"BM Althouse","year":"2011","unstructured":"Althouse, B. M., Ng, Y. Y. & Cummings, D. A. Prediction of dengue incidence using search query surveillance. PLoS Negl Trop Dis. 5, e1258 (2011).","journal-title":"PLoS Negl Trop Dis."},{"key":"395_CR14","doi-asserted-by":"publisher","unstructured":"Lampos, V. et al. Tracking covid-19 using online search. NPJ Digital Medicine. https:\/\/doi.org\/10.1038\/s41746-021-00384-w (2020).","DOI":"10.1038\/s41746-021-00384-w"},{"key":"395_CR15","unstructured":"Sarigul, S. & Rui, H. Nowcasting obesity in the us using Google search volume data. Tech. Rep. (2014)."},{"key":"395_CR16","doi-asserted-by":"publisher","first-page":"e45921","DOI":"10.1371\/journal.pone.0045921","volume":"7","author":"Y Ofran","year":"2012","unstructured":"Ofran, Y., Paltiel, O., Pelleg, D., Rowe, J. M. & Yom-Tov, E. Patterns of information-seeking for cancer on the internet: An analysis of real-world data. PLoS ONE 7, e45921 (2012).","journal-title":"PLoS ONE"},{"key":"395_CR17","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1111\/j.1475-4932.2012.00809.x","volume":"88","author":"H Choi","year":"2012","unstructured":"Choi, H. & Varian, H. Predicting the present with Google trends. Economic Record 88, 2\u20139 (2012).","journal-title":"Economic Record"},{"key":"395_CR18","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1016\/j.autcon.2016.06.009","volume":"70","author":"S Kim","year":"2016","unstructured":"Kim, S. et al. Forecasting short-term air passenger demand using big data from search engine queries. Automation Construction 70, 98\u2013108 (2016).","journal-title":"Automation Construction"},{"key":"395_CR19","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1145\/1096000.1096010","volume":"48","author":"M Ettredge","year":"2005","unstructured":"Ettredge, M., Gerdes, J. & Karuga, G. Using web-based search data to predict macroeconomic statistics. Commun. ACM 48, 87\u201392 (2005).","journal-title":"Commun. ACM"},{"key":"395_CR20","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1093\/pan\/mpv002","volume":"23","author":"A Street","year":"2015","unstructured":"Street, A., Murray, T. A., Blitzer, J. & Patel, R. S. Estimating voter registration deadline effects with web search data. Political Analysis 23, 225\u2013241 (2015).","journal-title":"Political Analysis"},{"key":"395_CR21","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1140\/epjds\/s13688-016-0094-0","volume":"5","author":"L Kristoufek","year":"2016","unstructured":"Kristoufek, L., Moat, H. S. & Preis, T. Estimating suicide occurrence statistics using Google trends. EPJ Data Sci. 5, 32 (2016).","journal-title":"EPJ Data Sci."},{"key":"395_CR22","doi-asserted-by":"publisher","first-page":"5955","DOI":"10.1016\/j.vaccine.2018.08.060","volume":"36","author":"S Mohanty","year":"2018","unstructured":"Mohanty, S., Leader, A. E., Gibeau, E. & Johnson, C. Using Facebook to reach adolescents for human papillomavirus (HPV) vaccination. Vaccine 36, 5955\u20135961 (2018).","journal-title":"Vaccine"},{"key":"395_CR23","doi-asserted-by":"publisher","first-page":"512","DOI":"10.1016\/j.vaccine.2019.10.066","volume":"38","author":"AM Jamison","year":"2020","unstructured":"Jamison, A. M. et al. Vaccine-related advertising in the Facebook ad archive. Vaccine 38, 512\u2013520 (2020).","journal-title":"Vaccine"},{"key":"395_CR24","doi-asserted-by":"publisher","first-page":"480","DOI":"10.1509\/jmr.13.0099","volume":"51","author":"K Jerath","year":"2014","unstructured":"Jerath, K., Ma, L. & Park, Y.-H. Consumer click behavior at a search engine: the role of keyword popularity. J. Marketing Res. 51, 480\u2013486 (2014).","journal-title":"J. Marketing Res."},{"key":"395_CR25","doi-asserted-by":"publisher","first-page":"508","DOI":"10.1016\/j.ijresmar.2014.12.005","volume":"33","author":"M Joo","year":"2016","unstructured":"Joo, M., Wilbur, K. C. & Zhu, Y. Effects of TV advertising on keyword search. Int. J. Res. Marketing 33, 508\u2013523 (2016).","journal-title":"Int. J. Res. Marketing"},{"key":"395_CR26","doi-asserted-by":"publisher","first-page":"299","DOI":"10.2753\/MIS0742-1222300411","volume":"30","author":"X Lu","year":"2014","unstructured":"Lu, X. & Zhao, X. Differential effects of keyword selection in search engine advertising on direct and indirect sales. J. Management Inf. Syst. 30, 299\u2013326 (2014).","journal-title":"J. Management Inf. Syst."},{"key":"395_CR27","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1287\/mksc.17.3.214","volume":"17","author":"PA Naik","year":"1998","unstructured":"Naik, P. A., Mantrala, M. K. & Sawyer, A. G. Planning media schedules in the presence of dynamic advertising quality. Marketing Sci. 17, 214\u2013235 (1998).","journal-title":"Marketing Sci."},{"key":"395_CR28","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1145\/1314215.1314224","volume":"51","author":"MT Keane","year":"2008","unstructured":"Keane, M. T., O\u2019Brien, M. & Smyth, B. Are people biased in their use of search engines? Commun. ACM 51, 49\u201352 (2008).","journal-title":"Commun. ACM"},{"key":"395_CR29","doi-asserted-by":"publisher","first-page":"155","DOI":"10.3982\/ECTA12423","volume":"83","author":"T Blake","year":"2015","unstructured":"Blake, T., Nosko, C. & Tadelis, S. Consumer heterogeneity and paid search effectiveness: A large-scale field experiment. Econometrica 83, 155\u2013174 (2015).","journal-title":"Econometrica"},{"key":"395_CR30","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1287\/mksc.1090.0552","volume":"29","author":"S Yang","year":"2010","unstructured":"Yang, S. & Ghose, A. Analyzing the relationship between organic and sponsored search advertising: Positive, negative, or zero interdependence? Marketing Sci. 29, 602\u2013623 (2010).","journal-title":"Marketing Sci."},{"key":"395_CR31","unstructured":"Chan, D., Kumar, D., Ma, S. & Koehler, J. Impact of ranking of organic search results on the incrementality of search ads. http:\/\/static.googleusercontent.com\/media\/research.google.com\/en\/\/pubs\/archive\/37731.pdf (2012)."},{"key":"395_CR32","doi-asserted-by":"crossref","unstructured":"Danescu-Niculescu-Mizil, C., Broder, A. Z., Gabrilovich, E., Josifovski, V. & Pang, B. Competing for users\u2019 attention: On the interplay between organic and sponsored search results. In Proc. 19th Int. Conference on World Wide Web, 291\u2013300 (2010).","DOI":"10.1145\/1772690.1772721"},{"key":"395_CR33","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1007\/s11747-017-0551-8","volume":"46","author":"D Chandrasekaran","year":"2018","unstructured":"Chandrasekaran, D., Srinivasan, R. & Sihi, D. Effects of offline ad content on online brand search: Insights from super bowl advertising. J. Acad. Marketing Sci. 46, 403\u2013430 (2018).","journal-title":"J. Acad. Marketing Sci."},{"key":"395_CR34","doi-asserted-by":"crossref","unstructured":"Youngmann, B., Yom-Tov, E., Gilad-Bachrach, R. & Karmon, D. The automated copywriter: Algorithmic rephrasing of health-related advertisements to improve their performance. In Proc. The Web Conference 2020, 1366\u20131377 (2020).","DOI":"10.1145\/3366423.3380211"},{"key":"395_CR35","doi-asserted-by":"publisher","first-page":"849","DOI":"10.1016\/j.jbusres.2011.01.006","volume":"65","author":"BA Huhmann","year":"2012","unstructured":"Huhmann, B. A., Franke, G. R. & Mothersbaugh, D. L. Print advertising: executional factors and the RPB grid. J. Business Res. 65, 849\u2013854 (2012).","journal-title":"J. Business Res."},{"key":"395_CR36","first-page":"423","volume":"16","author":"MP MJ","year":"1995","unstructured":"MJ, M. P. et al. Evaluation of the completion of influenza vaccination. Atencion Primaria 16, 423\u2013427 (1995).","journal-title":"Atencion Primaria"},{"key":"395_CR37","doi-asserted-by":"publisher","first-page":"e0220230","DOI":"10.1371\/journal.pone.0220230","volume":"14","author":"H Bricout","year":"2019","unstructured":"Bricout, H. et al. Determinants of shingles vaccine acceptance in the United Kingdom. PLoS ONE 14, e0220230 (2019).","journal-title":"PLoS ONE"},{"key":"395_CR38","doi-asserted-by":"publisher","first-page":"1763","DOI":"10.4161\/hv.24657","volume":"9","author":"E Dub\u00e9","year":"2013","unstructured":"Dub\u00e9, E. et al. Vaccine hesitancy: An overview. Hum Vaccin Immunother. 9, 1763\u20131773 (2013).","journal-title":"Hum Vaccin Immunother."},{"key":"395_CR39","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1177\/1090198117709318","volume":"45","author":"KR McBride","year":"2018","unstructured":"McBride, K. R. & Singh, S. Predictors of adults\u2019 knowledge and awareness of HPV, HPV-associated cancers, and the HPV vaccine: Implications for health education. Health Educ. Behav. 45, 68\u201376 (2018).","journal-title":"Health Educ. Behav."},{"key":"395_CR40","doi-asserted-by":"publisher","first-page":"3104","DOI":"10.1016\/j.vaccine.2017.04.028","volume":"35","author":"P-J Lu","year":"2017","unstructured":"Lu, P.-J et al. Awareness among adults of vaccine-preventable diseases and recommended vaccinations, United States, 2015. Vaccine 35, 3104\u20133115 (2017).","journal-title":"Vaccine"},{"key":"395_CR41","doi-asserted-by":"publisher","first-page":"375","DOI":"10.1509\/jmr.13.0277","volume":"52","author":"PR Hoban","year":"2015","unstructured":"Hoban, P. R. & Bucklin, R. E. Effects of internet display advertising in the purchase funnel: Model-based insights from a randomized field experiment. J. Marketing Res 52, 375\u2013393 (2015).","journal-title":"J. Marketing Res"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00395-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00395-7","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00395-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,3]],"date-time":"2022-12-03T19:13:41Z","timestamp":1670094821000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-021-00395-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,11]]},"references-count":41,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["395"],"URL":"https:\/\/doi.org\/10.1038\/s41746-021-00395-7","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,11]]},"assertion":[{"value":"26 August 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 January 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 February 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"E.Y.T. and D.R. are employees of Microsoft, owner of Bing. M.K. declare that she has no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"23"}}