{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T02:31:04Z","timestamp":1785983464604,"version":"3.56.0"},"reference-count":45,"publisher":"SAGE Publications","issue":"6","license":[{"start":{"date-parts":[[2017,12,1]],"date-time":"2017-12-01T00:00:00Z","timestamp":1512086400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Marketing Research"],"published-print":{"date-parts":[[2017,12]]},"abstract":"<jats:p> To measure the effects of advertising, marketers must know how consumers would behave had they not seen the ads. The authors develop a methodology they call \u201cghost ads,\u201d which facilitates this comparison by identifying the control group counterparts of the exposed consumers in a randomized experiment. The authors show that, relative to public service announcement and intent-to-treat A\/B tests, ghost ads can reduce the cost of experimentation, improve measurement precision, deliver the relevant strategic baseline, and work with modern ad platforms that optimize ad delivery in real time. The authors also describe a variant, \u201cpredicted ghost ad\u201d methodology, which is compatible with online display advertising platforms; their implementation records more than 100 million predicted ghost ads per day. The authors demonstrate the methodology with an online retailer's display retargeting campaign. They show novel evidence that retargeting can work: the ads lifted website visits by 17.2% and purchases by 10.5%. Compared with intent-to-treat and public service announcement experiments, advertisers can measure ad lift just as precisely while spending at least an order of magnitude less. <\/jats:p>","DOI":"10.1509\/jmr.15.0297","type":"journal-article","created":{"date-parts":[[2017,3,7]],"date-time":"2017-03-07T16:19:14Z","timestamp":1488903554000},"page":"867-884","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":148,"title":["Ghost Ads: Improving the Economics of Measuring Online Ad Effectiveness"],"prefix":"10.1177","volume":"54","author":[{"given":"Garrett A.","family":"Johnson","sequence":"first","affiliation":[{"name":"Visiting Assistant Professor of Marketing, Kellogg School of Management, Northwestern University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Randall A.","family":"Lewis","sequence":"additional","affiliation":[{"name":"Economic Research Scientist, Netflix"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elmar I.","family":"Nubbemeyer","sequence":"additional","affiliation":[{"name":"Product Manager, Netflix"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2017,12,1]]},"reference":[{"key":"bibr1-jmr.15.0297","unstructured":"AdRoll (2014), \u201cState of the Industry: A Close Look at Retargeting and the Programmatic Marketer,\u201d technical report, AdRoll."},{"key":"bibr2-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1145\/2566486.2567967"},{"key":"bibr3-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1145\/2229012.2229027"},{"key":"bibr4-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1287\/mksc.2016.0982"},{"key":"bibr5-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1509\/jmr.13.0503"},{"key":"bibr6-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.3982\/ECTA12423"},{"key":"bibr7-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1287\/mksc.2015.0930"},{"key":"bibr8-jmr.15.0297","unstructured":"CandelaJoaquin Q., BaileyMichael, and DominowskaEwa (2014), \u201cMachine Learning and the Facebook Ads Auction,\u201d presentation in Joint Session for EC, NBER and Decentralization on CS and Economics, EC14, Palo Alto, CA."},{"key":"bibr9-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2882984"},{"key":"bibr10-jmr.15.0297","unstructured":"Econsultancy (2014), \u201cDisplay Retargeting Buyer's Guide,\u201d technical report, Econsultancy."},{"key":"bibr11-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1991.10474996"},{"key":"bibr12-jmr.15.0297","unstructured":"Facebook (2016), \u201cHow We're Making Ad Measurement More Insightful,\u201d Facebook for Business News."},{"key":"bibr13-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-10841-9_20"},{"key":"bibr14-jmr.15.0297","volume-title":"\u201cBest Practices for Conducting Online Ad Effectiveness Research,\u201d technical report","author":"Gluck M.","year":"2011"},{"key":"bibr15-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1287\/mksc.1100.0583"},{"key":"bibr16-jmr.15.0297","unstructured":"Google (2015), \u201cWhere Ads Might Appear in the Display Network,\u201d Google Support."},{"key":"bibr17-jmr.15.0297","doi-asserted-by":"crossref","unstructured":"GordonBrett, ZettelmeyerFlorian, BhargavaNeha, and ChapskyDan (2017), \u201cA Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook,\u201d white paper.","DOI":"10.2139\/ssrn.3033144"},{"key":"bibr18-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1509\/jmr.13.0277"},{"key":"bibr19-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.2501\/S0021849907070353"},{"key":"bibr20-jmr.15.0297","unstructured":"HunterAnne, JacobsenMeredith, TalensRichard, and WindersTony (2010), \u201cWhen Money Moves to Digital, Where Should It Go?\u201d comScore white paper, https:\/\/www.comscore.com\/Insights\/Presentations-and-Whitepapers\/2010\/When-Money-Moves-to-Digital-Where-Should-It-Go."},{"key":"bibr21-jmr.15.0297","unstructured":"IAB (2014), \u201cIAB Internet Advertising Revenue Report 2013,\u201d http:\/\/www.iab.net\/AdRevenueReport."},{"key":"bibr22-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.2307\/2951620"},{"key":"bibr23-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781139025751"},{"key":"bibr24-jmr.15.0297","unstructured":"JohnsonGarrett A., LewisRandall A., and NubbemeyerElmar I. (2017), \u201cThe Online Display Ad Effectiveness Funnel and Carryover: Lessons from 432 Field Experiments,\u201d working paper, https:\/\/ssrn.com\/abstract=2701578."},{"key":"bibr25-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1287\/mksc.2016.0998"},{"key":"bibr26-jmr.15.0297","doi-asserted-by":"crossref","unstructured":"KalyanamKirthi, McAteerJohn, MarekJonathan, HodgesJames A., and LinLifeng (2015), \u201cCross Channel Effects of Search Engine Advertising on Brick and Mortar Retail Sales: Insights from Multiple Large Scale Field Experiments on Google.com,\u201d working paper, https:\/\/ssrn.com\/abstract=268411.","DOI":"10.2139\/ssrn.2684110"},{"key":"bibr27-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1509\/jmr.11.0503"},{"key":"bibr28-jmr.15.0297","volume-title":"\u201cAn Evaluation of Methods Used to Assess the Effectiveness of Advertising on the Internet,\u201d technical report","author":"Lavrakas Paul J.","year":"2010"},{"key":"bibr29-jmr.15.0297","volume-title":"Economic Analysis of the Digital Economy","author":"Lewis Randall","year":"2015"},{"key":"bibr30-jmr.15.0297","unstructured":"LewisRandall A. (2014), \u201cWorn-Out or Just Getting Started? The Impact of Frequency in Online Display Advertising,\u201d working paper, Google."},{"key":"bibr31-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1007\/s11129-015-9155-0"},{"key":"bibr32-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1093\/qje\/qjv023"},{"key":"bibr33-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1007\/s11129-014-9146-6"},{"key":"bibr34-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1177\/002224379503200201"},{"key":"bibr35-jmr.15.0297","volume-title":"Identification for Prediction and Decision","author":"Manski Charles F.","year":"2007"},{"key":"bibr36-jmr.15.0297","doi-asserted-by":"crossref","unstructured":"MoriguchiTakeshi, XiongGuiyang, and LuoXueming (2016), \u201cRetargeting Ads for Shopping Cart Recovery: Evidence from Online Field Experiments,\u201d working paper, https:\/\/ssrn.com\/abstract=2847631.","DOI":"10.2139\/ssrn.2847631"},{"key":"bibr37-jmr.15.0297","unstructured":"PriceWaterhouseCoopers (2011), \u201cMeasuring the Effectiveness of Online Advertising,\u201d technical report, IAB France and SRI."},{"key":"bibr38-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1007\/s11129-015-9159-9"},{"key":"bibr39-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1509\/jmr.14.0274"},{"key":"bibr40-jmr.15.0297","doi-asserted-by":"crossref","unstructured":"SahniNavdeep S., and NairHarikesh (2016), \u201cDoes Advertising Serve as a Signal? Evidence from Field Experiments in Mobile Search,\u201d working paper, https:\/\/ssrn.com\/abstract=2721468.","DOI":"10.2139\/ssrn.2721468"},{"key":"bibr41-jmr.15.0297","doi-asserted-by":"crossref","unstructured":"SahniNavdeep S., NarayananS., and KalyanamK. (2016), \u201cAn Experimental Investigation of the Effects of Retargeted Advertising: The Role of Frequency and Timing,\u201d working paper, https:\/\/ssrn.com\/abstract=2852484.","DOI":"10.2139\/ssrn.2852484"},{"key":"bibr42-jmr.15.0297","unstructured":"ShrivastavaA. (2015), \u201cUnderstanding the Impact of Twitter Ads Through Conversion Lift Reports,\u201d Twitter blog."},{"key":"bibr43-jmr.15.0297","doi-asserted-by":"publisher","DOI":"10.1111\/j.1465-7295.2008.00161.x"},{"key":"bibr44-jmr.15.0297","unstructured":"TuchmanA.E. (2016), \u201cAdvertising and Demand for Addictive Goods: The Effects of e-Cigarette Advertising,\u201d working paper, Northwestern University."},{"key":"bibr45-jmr.15.0297","unstructured":"YildizT. and NarayananS. (2013), \u201cStar Digital: Assessing the Effectiveness of Display Advertising,\u201d Harvard Business Review Case Study."}],"container-title":["Journal of Marketing Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1509\/jmr.15.0297","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1509\/jmr.15.0297","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1509\/jmr.15.0297","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,11]],"date-time":"2025-03-11T07:17:46Z","timestamp":1741677466000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1509\/jmr.15.0297"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,12]]},"references-count":45,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2017,12]]}},"alternative-id":["10.1509\/jmr.15.0297"],"URL":"https:\/\/doi.org\/10.1509\/jmr.15.0297","relation":{},"ISSN":["0022-2437","1547-7193"],"issn-type":[{"value":"0022-2437","type":"print"},{"value":"1547-7193","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,12]]}}}