{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T03:50:14Z","timestamp":1772077814711,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":35,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,8,20]],"date-time":"2020-08-20T00:00:00Z","timestamp":1597881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,8,23]]},"DOI":"10.1145\/3394486.3403306","type":"proceedings-article","created":{"date-parts":[[2020,8,20]],"date-time":"2020-08-20T23:03:53Z","timestamp":1597964633000},"page":"2562-2570","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Lumos"],"prefix":"10.1145","author":[{"given":"Jamie","family":"Pool","sequence":"first","affiliation":[{"name":"Microsoft, Redmond, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ebrahim","family":"Beyrami","sequence":"additional","affiliation":[{"name":"Microsoft, Redmond, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vishak","family":"Gopal","sequence":"additional","affiliation":[{"name":"Microsoft, Redmond, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ashkan","family":"Aazami","sequence":"additional","affiliation":[{"name":"Microsoft, Redmond, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jayant","family":"Gupchup","sequence":"additional","affiliation":[{"name":"Microsoft, Redmond, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jeff","family":"Rowland","sequence":"additional","affiliation":[{"name":"Microsoft, Redmond, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Binlong","family":"Li","sequence":"additional","affiliation":[{"name":"Microsoft, Redmond, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pritesh","family":"Kanani","sequence":"additional","affiliation":[{"name":"Microsoft, Redmond, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ross","family":"Cutler","sequence":"additional","affiliation":[{"name":"Microsoft, Redmond, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Johannes","family":"Gehrke","sequence":"additional","affiliation":[{"name":"Microsoft, Redmond, WA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,8,20]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/170035.170072"},{"key":"e_1_3_2_1_2_1","volume-title":"An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate behavioral research","author":"Austin Peter C","year":"2011","unstructured":"Peter C Austin . 2011. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate behavioral research , Vol. 46 , 3 ( 2011 ), 399--424. Peter C Austin. 2011. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate behavioral research, Vol. 46, 3 (2011), 399--424."},{"key":"e_1_3_2_1_3_1","series-title":"Series A","volume-title":"Controlling bias in observational studies: A review. Sankhy\u0101: The Indian Journal of Statistics","author":"Cochran William G","year":"1973","unstructured":"William G Cochran and Donald B Rubin . 1973. Controlling bias in observational studies: A review. Sankhy\u0101: The Indian Journal of Statistics , Series A ( 1973 ), 417--446. William G Cochran and Donald B Rubin. 1973. Controlling bias in observational studies: A review. Sankhy\u0101: The Indian Journal of Statistics, Series A (1973), 417--446."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1557019.1557139"},{"key":"e_1_3_2_1_5_1","volume-title":"Propensity score-matching methods for nonexperimental causal studies. Review of Economics and statistics","author":"Dehejia Rajeev H","year":"2002","unstructured":"Rajeev H Dehejia and Sadek Wahba . 2002. Propensity score-matching methods for nonexperimental causal studies. Review of Economics and statistics , Vol. 84 , 1 ( 2002 ), 151--161. Rajeev H Dehejia and Sadek Wahba. 2002. Propensity score-matching methods for nonexperimental causal studies. Review of Economics and statistics, Vol. 84, 1 (2002), 151--161."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219919"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2433396.2433413"},{"key":"e_1_3_2_1_8_1","volume-title":"The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results","author":"Ellis Paul D","unstructured":"Paul D Ellis . 2010. The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results . Cambridge University Press . Paul D Ellis. 2010. The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results. Cambridge University Press."},{"key":"e_1_3_2_1_9_1","first-page":"1","article-title":"Neural Architecture Search: A Survey","volume":"20","author":"Elsken Thomas","year":"2019","unstructured":"Thomas Elsken , Jan Hendrik Metzen , and Frank Hutter . 2019 . Neural Architecture Search: A Survey . Journal of Machine Learning Research , Vol. 20 , 55 (2019), 1 -- 21 . Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural Architecture Search: A Survey. Journal of Machine Learning Research, Vol. 20, 55 (2019), 1--21.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDMW.2010.33"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/QoMEX.2017.7965648"},{"key":"e_1_3_2_1_12_1","volume-title":"Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political analysis","author":"Ho Daniel E","year":"2007","unstructured":"Daniel E Ho , Kosuke Imai , Gary King , and Elizabeth A Stuart . 2007. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political analysis , Vol. 15 , 3 ( 2007 ), 199--236. Daniel E Ho, Kosuke Imai, Gary King, and Elizabeth A Stuart. 2007. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political analysis, Vol. 15, 3 (2007), 199--236."},{"key":"e_1_3_2_1_13_1","volume-title":"Causal inference in statistics, social, and biomedical sciences","author":"Imbens Guido W","unstructured":"Guido W Imbens and Donald B Rubin . 2015. Causal inference in statistics, social, and biomedical sciences . Cambridge University Press . Guido W Imbens and Donald B Rubin. 2015. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press."},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330648"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339653"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1281192.1281295"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219905"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884795"},{"key":"e_1_3_2_1_19_1","unstructured":"LinkedIn. 2018. Luminol. https:\/\/github.com\/linkedin\/luminol LinkedIn. 2018. Luminol. https:\/\/github.com\/linkedin\/luminol"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0138-9"},{"key":"e_1_3_2_1_21_1","first-page":"I","article-title":"A Unified Approach to Interpreting Model Predictions","volume":"30","author":"Lundberg Scott M","year":"2017","unstructured":"Scott M Lundberg and Su-In Lee . 2017 . A Unified Approach to Interpreting Model Predictions . In Advances in Neural Information Processing Systems 30 , I . Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 4765--4774. http:\/\/papers.nips.cc\/paper\/7062-a-unified-approach-to-interpreting-model-predictions.pdf Scott M Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 4765--4774. http:\/\/papers.nips.cc\/paper\/7062-a-unified-approach-to-interpreting-model-predictions.pdf","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_22_1","unstructured":"Microsoft. 2020. Lumos. https:\/\/github.com\/microsoft\/MS-Lumos Microsoft. 2020. Lumos. https:\/\/github.com\/microsoft\/MS-Lumos"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908812.2908918"},{"key":"e_1_3_2_1_24_1","volume-title":"Propensity score analysis: Fundamentals and developments","author":"Pan Wei","unstructured":"Wei Pan and Haiyan Bai . 2015. Propensity score analysis: Fundamentals and developments . Guilford Publications . Wei Pan and Haiyan Bai. 2015. Propensity score analysis: Fundamentals and developments. Guilford Publications."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2016.7840800"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330680"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/70.1.41"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1080\/00031305.1985.10479383","article-title":"Constructing a control group using multivariate matched sampling methods that incorporate the propensity score","volume":"39","author":"Rosenbaum Paul R","year":"1985","unstructured":"Paul R Rosenbaum and Donald B Rubin . 1985 . Constructing a control group using multivariate matched sampling methods that incorporate the propensity score . The American Statistician , Vol. 39 , 1 (1985), 33 -- 38 . Paul R Rosenbaum and Donald B Rubin. 1985. Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. The American Statistician, Vol. 39, 1 (1985), 33--38.","journal-title":"The American Statistician"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0037350"},{"key":"e_1_3_2_1_31_1","unstructured":"Ando Saabas. 2015. Tree Interpreter. https:\/\/github.com\/andosa\/treeinterpreter Ando Saabas. 2015. Tree Interpreter. https:\/\/github.com\/andosa\/treeinterpreter"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098039"},{"key":"e_1_3_2_1_33_1","volume-title":"6th {USENIX} Workshop on Hot Topics in Cloud Computing (HotCloud 14).","author":"Vallis Owen","unstructured":"Owen Vallis , Jordan Hochenbaum , and Arun Kejariwal . 2014. A novel technique for long-term anomaly detection in the cloud . In 6th {USENIX} Workshop on Hot Topics in Cloud Computing (HotCloud 14). Owen Vallis, Jordan Hochenbaum, and Arun Kejariwal. 2014. A novel technique for long-term anomaly detection in the cloud. In 6th {USENIX} Workshop on Hot Topics in Cloud Computing (HotCloud 14)."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/347090.347112"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219860"}],"event":{"name":"KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Virtual Event CA USA","acronym":"KDD '20","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394486.3403306","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3394486.3403306","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:01:48Z","timestamp":1750197708000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3394486.3403306"}},"subtitle":["A Library for Diagnosing Metric Regressions in Web-Scale Applications"],"short-title":[],"issued":{"date-parts":[[2020,8,20]]},"references-count":35,"alternative-id":["10.1145\/3394486.3403306","10.1145\/3394486"],"URL":"https:\/\/doi.org\/10.1145\/3394486.3403306","relation":{},"subject":[],"published":{"date-parts":[[2020,8,20]]},"assertion":[{"value":"2020-08-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}