{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T15:10:14Z","timestamp":1788966614796,"version":"build-2803163510"},"reference-count":35,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2019,9,30]],"date-time":"2019-09-30T00:00:00Z","timestamp":1569801600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100008982","name":"National Science Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100008982","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,12,1]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>Experimentation platforms are essential to large modern technology companies, as they are used to carry out many randomized experiments daily. The classic assumption of no interference among users, under which the outcome for one user does not depend on the treatment assigned to other users, is rarely tenable on such platforms. Here, we introduce an experimental design strategy for testing whether this assumption holds. Our approach is in the spirit of the Durbin\u2013Wu\u2013Hausman test for endogeneity in econometrics, where multiple estimators return the same estimate if and only if the null hypothesis holds. The design that we introduce makes no assumptions on the interference model between units, nor on the network among the units, and has a sharp bound on the variance and an implied analytical bound on the Type I error rate. We discuss how to apply the proposed design strategy to large experimentation platforms, and we illustrate it in the context of an experiment on the LinkedIn platform.<\/jats:p>","DOI":"10.1093\/biomet\/asz047","type":"journal-article","created":{"date-parts":[[2019,9,23]],"date-time":"2019-09-23T19:28:54Z","timestamp":1569266934000},"page":"929-940","source":"Crossref","is-referenced-by-count":22,"title":["Testing for arbitrary interference on experimentation platforms"],"prefix":"10.1093","volume":"106","author":[{"given":"J","family":"Pouget-Abadie","sequence":"first","affiliation":[{"name":"Google Research, 111 8th Avenue, New York, New York, U.S.A"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"G","family":"Saint-Jacques","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, 100 Main Street, Cambridge, Massachusetts 02139, U.S.A"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M","family":"Saveski","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, 100 Main Street, Cambridge, Massachusetts 02139, U.S.A"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"W","family":"Duan","sequence":"additional","affiliation":[{"name":"LinkedIn, 1000 W. Maude Avenue, Sunnyvale, California, U.S.A"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"S","family":"Ghosh","sequence":"additional","affiliation":[{"name":"LinkedIn, 1000 W. Maude Avenue, Sunnyvale, California, U.S.A"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Y","family":"Xu","sequence":"additional","affiliation":[{"name":"LinkedIn, 1000 W. Maude Avenue, Sunnyvale, California, U.S.A"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"E M","family":"Airoldi","sequence":"additional","affiliation":[{"name":"Fox School of Business, Temple University, 1810 Liacouras Walk, Philadelphia, Pennsylvania, U.S.A"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2019,9,30]]},"reference":[{"key":"2020092920044670800_B1","doi-asserted-by":"crossref","first-page":"1623","DOI":"10.1287\/mnsc.1110.1421","article-title":"Creating social contagion through viral product design: A randomized trial of peer influence in networks","volume":"57","author":"Aral","year":"2011","journal-title":"Manag. Sci."},{"key":"2020092920044670800_B2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1177\/0049124112437535","article-title":"A general method for detecting interference between units in randomized experiments","volume":"41","author":"Aronow","year":"2012","journal-title":"Sociol. Meth. Res."},{"key":"2020092920044670800_B3","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1080\/01621459.2016.1241178","article-title":"Exact $p$-values for network interference","volume":"113","author":"Athey","year":"2018","journal-title":"J. Am. Statist. Assoc."},{"key":"2020092920044670800_B4","first-page":"615","article-title":"Network bucket testing","author":"Backstrom","year":"2011","journal-title":"Proc. 20th Int. Conf. on World Wide Web"},{"key":"2020092920044670800_B5","doi-asserted-by":"crossref","DOI":"10.1596\/1813-9450-6824","author":"Baird","year":"2014","journal-title":"Designing Experiments to Measure Spillover Effects"},{"key":"2020092920044670800_B6","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1145\/2566486.2567967","article-title":"Designing and deploying online field experiments","author":"Bakshy","year":"2014","journal-title":"Proc. 23rd Int. Conf. on World Wide Web"},{"key":"2020092920044670800_B7","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1093\/biomet\/asy036","article-title":"Model-assisted design of experiments in the presence of network-correlated outcomes","volume":"105","author":"Basse","year":"2018","journal-title":"Biometrika"},{"key":"2020092920044670800_B8","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1080\/01621459.2016.1194845","article-title":"Estimation of monotone treatment effects in network experiments","volume":"112","author":"Choi","year":"2017","journal-title":"J. Am. Statist. Assoc."},{"key":"2020092920044670800_B9","doi-asserted-by":"crossref","first-page":"1620","DOI":"10.1093\/jnci\/83.22.1620","article-title":"Community intervention trial for smoking cessation (COMMIT): Summary of design and intervention","volume":"83","year":"1991","journal-title":"J. Nat. Cancer Inst."},{"key":"2020092920044670800_B10","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1093\/oxfordjournals.aje.a112592","article-title":"Symposium on CHD prevention trials: Design issues in testing life style intervention randomization by group: A formal analysis","volume":"108","author":"Cornfield","year":"1978","journal-title":"Am. J. Epidemiol."},{"key":"2020092920044670800_B11","doi-asserted-by":"crossref","first-page":"792","DOI":"10.1111\/j.0006-341X.1999.00792.x","article-title":"Efficiency of estimating vaccine efficacy for susceptibility and infectiousness: Randomization by individual versus household","volume":"55","author":"Datta","year":"1999","journal-title":"Biometrics"},{"key":"2020092920044670800_B12","doi-asserted-by":"crossref","first-page":"416","DOI":"10.2105\/AJPH.94.3.416","article-title":"Pitfalls of and controversies in cluster randomization trials","volume":"94","author":"Donner","year":"2004","journal-title":"Am. J. Public Health"},{"key":"2020092920044670800_B13","first-page":"23","article-title":"Design and analysis of experiments in networks: Reducing bias from interference","volume":"5","author":"Eckles","year":"2017","journal-title":"J. Causal Inf."},{"key":"2020092920044670800_B14","doi-asserted-by":"crossref","first-page":"7316","DOI":"10.1073\/pnas.1511201113","article-title":"Estimating peer effects in networks with peer encouragement designs","volume":"113","author":"Eckles","year":"2016","journal-title":"Proc. Nat. Acad. Sci."},{"key":"2020092920044670800_B15","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1145\/2736277.2741081","article-title":"Network A\/B testing: From sampling to estimation","author":"Gui","year":"2015","journal-title":"Proc. 24th Int. Conf. on World Wide Web"},{"key":"2020092920044670800_B16","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1198\/016214506000000447","article-title":"Evaluating kindergarten retention policy: A case study of causal inference for multilevel observational data","volume":"101","author":"Hong","year":"2006","journal-title":"J. Am. Statist. Assoc."},{"key":"2020092920044670800_B17","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1198\/016214508000000292","article-title":"Toward causal inference with interference","volume":"103","author":"Hudgens","year":"2008","journal-title":"J. Am. Statist. Assoc."},{"key":"2020092920044670800_B18","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781139025751","author":"Imbens","year":"2015","journal-title":"Causal Inference in Statistics, Social, and Biomedical Sciences"},{"key":"2020092920044670800_B19","article-title":"A systematic investigation of classical causal inference strategies under mis-specification due to network interference","author":"Karwa","year":"2018"},{"key":"2020092920044670800_B20","first-page":"1029","article-title":"Framework and algorithms for network bucket testing","author":"Katzir","year":"2012","journal-title":"Proc. 21st Int. Conf. on World Wide Web"},{"key":"2020092920044670800_B21","doi-asserted-by":"crossref","first-page":"1168","DOI":"10.1145\/2487575.2488217","article-title":"Online controlled experiments at large scale","author":"Kohavi","year":"2013","journal-title":"Proc. 19th Int. Conf. on Knowledge Discovery and Data Mining"},{"key":"2020092920044670800_B22","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-662-44415-3_4","article-title":"Balanced $k$-means for clustering","author":"Malinen","year":"2014","journal-title":"Structural, Syntactic, and Statistical Pattern Recognition. S+SSPR 2014"},{"key":"2020092920044670800_B23","doi-asserted-by":"crossref","first-page":"S1","DOI":"10.1111\/j.1368-423X.2012.00368.x","article-title":"Identification of treatment response with social interactions","volume":"16","author":"Manski","year":"2013","journal-title":"Economet. J."},{"key":"2020092920044670800_B24","doi-asserted-by":"crossref","first-page":"1106","DOI":"10.1145\/2487575.2487696","article-title":"Restreaming graph partitioning: Simple versatile algorithms for advanced balancing","author":"Nishimura","year":"2013","journal-title":"Proc. 19th Int. Conf. on Knowledge Discovery and Data Mining"},{"key":"2020092920044670800_B25","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1198\/016214506000001112","article-title":"Interference between units in randomized experiments","volume":"102","author":"Rosenbaum","year":"2007","journal-title":"J. Am. Statist. Assoc."},{"key":"2020092920044670800_B26","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1037\/h0037350","article-title":"Estimating causal effects of treatments in randomized and nonrandomized studies","volume":"66","author":"Rubin","year":"1974","journal-title":"J. Educ. Psychol."},{"key":"2020092920044670800_B27","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/0378-3758(90)90077-8","article-title":"Formal mode of statistical inference for causal effects","volume":"25","author":"Rubin","year":"1990","journal-title":"J. Statist. Plan. Infer."},{"key":"2020092920044670800_B28","doi-asserted-by":"crossref","first-page":"1027","DOI":"10.1145\/3097983.3098192","article-title":"Detecting network effects: Randomizing over randomized experiments","author":"Saveski","year":"2017","journal-title":"Proc. 23rd Int. Conf. on Knowledge Discovery and Data Mining"},{"key":"2020092920044670800_B29","first-page":"e012996","article-title":"Exploiting social influence to magnify population-level behavior change in maternal and child health: Study protocol for a randomized controlled trial of network targeting algorithms in rural Honduras","volume":"7","author":"Shakya","year":"2017","journal-title":"Br. Med. J. Open"},{"key":"2020092920044670800_B30","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1111\/j.1540-5907.2012.00592.x","article-title":"Detecting spillover effects: Design and analysis of multilevel experiments","volume":"56","author":"Sinclair","year":"2012","journal-title":"Am. J. Polit. Sci."},{"key":"2020092920044670800_B31","first-page":"17","article-title":"Overlapping experiment infrastructure: More, better, faster experimentation","author":"Tang","year":"2010","journal-title":"Proc. 16th Int. Conf. on Knowledge Discovery and Data Mining"},{"key":"2020092920044670800_B32","first-page":"55","article-title":"On causal inference in the presence of interference","volume":"21","author":"Tchetgen","year":"2012","journal-title":"Statist. Meth.: Med. Res."},{"key":"2020092920044670800_B33","first-page":"1489","article-title":"Estimation of causal peer influence effects","volume":"28","author":"Toulis","year":"2013","journal-title":"Proc. Mach. Learn. Res."},{"key":"2020092920044670800_B34","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1145\/2487575.2487695","article-title":"Graph cluster randomization: Network exposure to multiple universes","author":"Ugander","year":"2013","journal-title":"Proc. 19th Int. Conf. on Knowledge Discovery and Data Mining"},{"key":"2020092920044670800_B35","first-page":"2227","article-title":"From infrastructure to culture: A\/B testing challenges in large scale social networks","author":"Xu","year":"2015","journal-title":"Proc. 21st Int. Conf. on Knowledge Discovery and Data Mining"}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/biomet\/article-pdf\/106\/4\/929\/30646748\/asz047.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/biomet\/article-pdf\/106\/4\/929\/30646748\/asz047.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,20]],"date-time":"2023-09-20T17:39:03Z","timestamp":1695231543000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/106\/4\/929\/5578431"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,30]]},"references-count":35,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2019,9,30]]},"published-print":{"date-parts":[[2019,12,1]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asz047","relation":{},"ISSN":["0006-3444","1464-3510"],"issn-type":[{"value":"0006-3444","type":"print"},{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,12]]},"published":{"date-parts":[[2019,9,30]]}}}