{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T04:24:22Z","timestamp":1780374262794,"version":"3.54.1"},"publisher-location":"New York, NY, USA","reference-count":18,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,8,14]],"date-time":"2021-08-14T00:00:00Z","timestamp":1628899200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,8,14]]},"DOI":"10.1145\/3447548.3467165","type":"proceedings-article","created":{"date-parts":[[2021,8,12]],"date-time":"2021-08-12T06:12:09Z","timestamp":1628748729000},"page":"3522-3530","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Contextual Bandit Applications in a Customer Support Bot"],"prefix":"10.1145","author":[{"given":"Sandra","family":"Sajeev","sequence":"first","affiliation":[{"name":"Microsoft Azure AI, Redmond, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jade","family":"Huang","sequence":"additional","affiliation":[{"name":"Microsoft Azure AI, Redmond, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nikos","family":"Karampatziakis","sequence":"additional","affiliation":[{"name":"Microsoft Azure AI, Redmond, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew","family":"Hall","sequence":"additional","affiliation":[{"name":"Microsoft Azure AI, Redmond, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"Kochman","sequence":"additional","affiliation":[{"name":"Microsoft Azure AI, Redmond, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weizhu","family":"Chen","sequence":"additional","affiliation":[{"name":"Microsoft Azure AI, Redmond, WA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,8,14]]},"reference":[{"key":"e_1_3_2_2_2_1","unstructured":"Alekh Agarwal Sarah Bird Markus Cozowicz Luong Hoang John Langford Stephen Lee Jiaji Li Dan Melamed Gal Oshri Oswaldo Ribas etal 2016. Making contextual decisions with low technical debt. arXiv preprint arXiv:1606.03966(2016).  Alekh Agarwal Sarah Bird Markus Cozowicz Luong Hoang John Langford Stephen Lee Jiaji Li Dan Melamed Gal Oshri Oswaldo Ribas et al. 2016. Making contextual decisions with low technical debt. arXiv preprint arXiv:1606.03966(2016)."},{"key":"e_1_3_2_2_3_1","volume-title":"Proceedings of The 30th International Conference on Machine Learning. 127--135","author":"Agrawal Shipra","year":"2013","unstructured":"Shipra Agrawal and Navin Goyal . 2013 . Thompson Sampling for Contextual Bandits with Linear Payoffs . In Proceedings of The 30th International Conference on Machine Learning. 127--135 . Shipra Agrawal and Navin Goyal. 2013. Thompson Sampling for Contextual Bandits with Linear Payoffs. In Proceedings of The 30th International Conference on Machine Learning. 127--135."},{"key":"e_1_3_2_2_4_1","volume-title":"ONNX: Open Neural Network Exchange. https:\/\/github.com\/onnx\/onnx.","author":"Bai Junjie","year":"2019","unstructured":"Junjie Bai , Fang Lu , Ke Zhang , 2019 . ONNX: Open Neural Network Exchange. https:\/\/github.com\/onnx\/onnx. Junjie Bai, Fang Lu, Ke Zhang, et al. 2019. ONNX: Open Neural Network Exchange. https:\/\/github.com\/onnx\/onnx."},{"key":"e_1_3_2_2_5_1","volume-title":"Advances in Neural Information Processing Systems 24","volume":"24","author":"Chapelle Olivier","year":"2011","unstructured":"Olivier Chapelle and Lihong Li . 2011 . An Empirical Evaluation of Thompson Sampling . In Advances in Neural Information Processing Systems 24 , Vol. 24 . 2249--2257. Olivier Chapelle and Lihong Li. 2011. An Empirical Evaluation of Thompson Sampling. In Advances in Neural Information Processing Systems 24, Vol. 24. 2249--2257."},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3290999"},{"key":"e_1_3_2_2_7_1","volume-title":"Residual Loss Prediction: Reinforcement Learning With No Incremental Feedback. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=HJNMYceCW","author":"John Langford Hal Daum\u00e9 III","year":"2018","unstructured":"Hal Daum\u00e9 III , John Langford , and Amr Sharaf . 2018 . Residual Loss Prediction: Reinforcement Learning With No Incremental Feedback. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=HJNMYceCW Hal Daum\u00e9 III, John Langford, and Amr Sharaf. 2018. Residual Loss Prediction: Reinforcement Learning With No Incremental Feedback. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=HJNMYceCW"},{"key":"e_1_3_2_2_8_1","volume-title":"Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring. In ICLR 2020 : Eighth International Conference on Learning Representations.","author":"Humeau Samuel","year":"2020","unstructured":"Samuel Humeau , Kurt Shuster , Marie-Anne Lachaux , and Jason Weston . 2020 . Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring. In ICLR 2020 : Eighth International Conference on Learning Representations. Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2020. Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring. In ICLR 2020 : Eighth International Conference on Learning Representations."},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/360"},{"key":"e_1_3_2_2_10_1","unstructured":"Nikos Karampatziakis Sebastian Kochman Jade Huang Paul Mineiro Kathy Osborne and Weizhu Chen. 2019. Lessons from Contextual Bandit Learning in a Customer Support Bot. arXiv:1905.02219 [cs.LG]  Nikos Karampatziakis Sebastian Kochman Jade Huang Paul Mineiro Kathy Osborne and Weizhu Chen. 2019. Lessons from Contextual Bandit Learning in a Customer Support Bot. arXiv:1905.02219 [cs.LG]"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/2981562.2981665"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772758"},{"key":"e_1_3_2_2_14_1","volume-title":"PyTorch: An Imperative Style","author":"Paszke Adam","unstructured":"Adam Paszke , Sam Gross , Francisco Massa , Adam Lerer , James Bradbury , Gregory Chanan , Trevor Killeen , Zeming Lin , Natalia Gimelshein , Luca Antiga , Alban Desmaison , Andreas Kopf , Edward Yang , Zachary DeVito , Martin Raison , Alykhan Tejani , Sasank Chilamkurthy , Benoit Steiner , Lu Fang , Junjie Bai , and Soumith Chintala . 2019. PyTorch: An Imperative Style , High-Performance Deep Learning Library . In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alche-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 8024--8035. http:\/\/papers.neurips.cc\/paper\/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alche-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 8024--8035. http:\/\/papers.neurips.cc\/paper\/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf"},{"key":"e_1_3_2_2_15_1","unstructured":"Carlos Riquelme George Tucker and Jasper Snoek. 2018. Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling. arXiv:1802.09127 [stat.ML]  Carlos Riquelme George Tucker and Jasper Snoek. 2018. Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling. arXiv:1802.09127 [stat.ML]"},{"key":"e_1_3_2_2_16_1","volume-title":"Abbas Kazerouni, Ian Osband, and Zheng Wen.","author":"Russo Daniel","year":"2020","unstructured":"Daniel Russo , Benjamin Van Roy , Abbas Kazerouni, Ian Osband, and Zheng Wen. 2020 . A Tutorial on Thompson Sampling . arXiv:1707.02038 [cs.LG] Daniel Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, and Zheng Wen. 2020. A Tutorial on Thompson Sampling. arXiv:1707.02038 [cs.LG]"},{"key":"e_1_3_2_2_17_1","unstructured":"Tobias Schnabel Adith Swaminathan Ashudeep Singh Navin Chandak and Thorsten Joachims. 2016. Recommendations as treatments: Debiasing learning and evaluation. arXiv preprint arXiv:1602.05352(2016).  Tobias Schnabel Adith Swaminathan Ashudeep Singh Navin Chandak and Thorsten Joachims. 2016. Recommendations as treatments: Debiasing learning and evaluation. arXiv preprint arXiv:1602.05352(2016)."},{"key":"e_1_3_2_2_18_1","volume-title":"Advances in Neural Information Processing Systems(advances in neural information processing systems ed.)","author":"Swaminathan Adith","unstructured":"Adith Swaminathan and Thorsten Joachims . 2015. The Self-Normalized Estimator for Counterfactual Learning . In Advances in Neural Information Processing Systems(advances in neural information processing systems ed.) , Vol. 28 . Curran Associates, Inc. , 3231--3239. https:\/\/www.microsoft.com\/en-us\/research\/publication\/self-normalized-estimator-counterfactual-learning\/ Adith Swaminathan and Thorsten Joachims. 2015. The Self-Normalized Estimator for Counterfactual Learning. In Advances in Neural Information Processing Systems(advances in neural information processing systems ed.), Vol. 28. Curran Associates, Inc., 3231--3239. https:\/\/www.microsoft.com\/en-us\/research\/publication\/self-normalized-estimator-counterfactual-learning\/"},{"key":"e_1_3_2_2_19_1","unstructured":"Adith Swaminathan Akshay Krishnamurthy Alekh Agarwal Miroslav Dudik John Langford Damien Jose and Imed Zitouni. 2017. Off-policy evaluation for slate recommendation. arXiv:1605.04812 [cs.LG]  Adith Swaminathan Akshay Krishnamurthy Alekh Agarwal Miroslav Dudik John Langford Damien Jose and Imed Zitouni. 2017. Off-policy evaluation for slate recommendation. arXiv:1605.04812 [cs.LG]"},{"key":"e_1_3_2_2_20_1","first-page":"3","article-title":"ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLES","volume":"25","author":"WILLIAM R","year":"1933","unstructured":"WILLIAM R THOMPSON. 1933 . ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLES . Biometrika 25 , 3 -- 4 (12 1933), 285--294. https:\/\/doi.org\/10.1093\/biomet\/25.3--4.285 arXiv: https:\/\/academic.oup.com\/biomet\/article-pdf\/25\/3--4\/285\/513725\/25--3--4--285.pdf 10.1093\/biomet WILLIAM R THOMPSON. 1933. ON THE LIKELIHOOD THAT ONE UNKNOWN PROBABILITY EXCEEDS ANOTHER IN VIEW OF THE EVIDENCE OF TWO SAMPLES. Biometrika 25, 3--4 (12 1933), 285--294. https:\/\/doi.org\/10.1093\/biomet\/25.3--4.285 arXiv: https:\/\/academic.oup.com\/biomet\/article-pdf\/25\/3--4\/285\/513725\/25--3--4--285.pdf","journal-title":"Biometrika"}],"event":{"name":"KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Virtual Event Singapore","acronym":"KDD '21","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447548.3467165","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447548.3467165","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:27Z","timestamp":1750191507000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447548.3467165"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,14]]},"references-count":18,"alternative-id":["10.1145\/3447548.3467165","10.1145\/3447548"],"URL":"https:\/\/doi.org\/10.1145\/3447548.3467165","relation":{},"subject":[],"published":{"date-parts":[[2021,8,14]]},"assertion":[{"value":"2021-08-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}