{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T16:37:46Z","timestamp":1776530266601,"version":"3.51.2"},"publisher-location":"New York, New York, USA","reference-count":43,"publisher":"ACM Press","license":[{"start":{"date-parts":[[2018,1,1]],"date-time":"2018-01-01T00:00:00Z","timestamp":1514764800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Tsinghua University Initiative Scienti c Research Grant"},{"name":"National Natural Science Foundation of China Grant","award":["61561146398"],"award-info":[{"award-number":["61561146398"]}]},{"name":"Alibaba Innovative Research program"},{"name":"China Youth 1000-talent program"},{"name":"ERC Advanced Grant","award":["321171"],"award-info":[{"award-number":["321171"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018]]},"DOI":"10.1145\/3178876.3186039","type":"proceedings-article","created":{"date-parts":[[2018,4,13]],"date-time":"2018-04-13T15:53:48Z","timestamp":1523634828000},"page":"1339-1348","source":"Crossref","is-referenced-by-count":40,"title":["Reinforcement Mechanism Design for e-commerce"],"prefix":"10.1145","author":[{"given":"Qingpeng","family":"Cai","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aris","family":"Filos-Ratsikas","sequence":"additional","affiliation":[{"name":"University of Oxford, Oxford, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pingzhong","family":"Tang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yiwei","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of California, Berkeley, Berkeley, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","reference":[{"key":"key-10.1145\/3178876.3186039-1","doi-asserted-by":"crossref","unstructured":"Sander Adam, Lucian Busoniu, and Robert Babuska . 2012. Experience replay for real-time reinforcement learning control. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 42, 2 (2012), 201--212.","DOI":"10.1109\/TSMCC.2011.2106494"},{"key":"key-10.1145\/3178876.3186039-2","doi-asserted-by":"crossref","unstructured":"Rajeev Agrawal . 1995. Sample mean based index policies by O (log n) regret for the multi-armed bandit problem. Advances in Applied Probability Vol. 27, 4 (1995), 1054--1078.","DOI":"10.2307\/1427934"},{"key":"key-10.1145\/3178876.3186039-3","unstructured":"Shipra Agrawal and Navin Goyal . 2013. Thompson sampling for contextual bandits with linear payoffs International Conference on Machine Learning. 127--135."},{"key":"key-10.1145\/3178876.3186039-4","unstructured":"Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer . 2002 a. Finite-time analysis of the multiarmed bandit problem. Machine learning, Vol. 47, 2--3 (2002), 235--256."},{"key":"key-10.1145\/3178876.3186039-5","unstructured":"Peter Auer, Nicolo Cesa-Bianchi, Yoav Freund, and Robert E Schapire . 1995. Gambling in a rigged casino: The adversarial multi-armed bandit problem Foundations of Computer Science, 1995. Proceedings., 36th Annual Symposium on. IEEE, 322--331."},{"key":"key-10.1145\/3178876.3186039-6","doi-asserted-by":"crossref","unstructured":"Peter Auer, Nicolo Cesa-Bianchi, Yoav Freund, and Robert E Schapire . 2002 b. The nonstochastic multiarmed bandit problem. SIAM journal on computing Vol. 32, 1 (2002), 48--77.","DOI":"10.1137\/S0097539701398375"},{"key":"key-10.1145\/3178876.3186039-7","unstructured":"Shalabh Bhatnagar, Richard S Sutton, Mohammad Ghavamzadeh, and Mark Lee . 2007. Incremental Natural Actor-Critic Algorithms.. In NIPS. 105--112."},{"key":"key-10.1145\/3178876.3186039-8","unstructured":"Djallel Bouneffouf, Amel Bouzeghoub, and Alda Lopes Ganccarski . 2012. A contextual-bandit algorithm for mobile context-aware recommender system International Conference on Neural Information Processing. Springer, 324--331."},{"key":"key-10.1145\/3178876.3186039-9","unstructured":"Giuseppe Burtini, Jason Loeppky, and Ramon Lawrence . 2015. A survey of online experiment design with the stochastic multi-armed bandit. arXiv preprint arXiv:1510.00757 (2015)."},{"key":"key-10.1145\/3178876.3186039-10","unstructured":"Qingpeng Cai, Aris Filos-Ratsikas, Chang Liu, and Pingzhong Tang . 2016. Mechanism Design for Personalized Recommender Systems Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 159--166."},{"key":"key-10.1145\/3178876.3186039-11","unstructured":"Qingpeng Cai, Aris Filos-Ratsikas, Pingzhong Tang, and Yiwei Zhang . 2018. Reinforcement Mechanism Design for Fraudulent Behaviour in e-Commerce. (2018)."},{"key":"key-10.1145\/3178876.3186039-12","doi-asserted-by":"crossref","unstructured":"Shuchi Chawla, Jason Hartline, and Denis Nekipelov . 2016. A\/B testing of auctions. In Proceedings of the 2016 ACM Conference on Economics and Computation. ACM, 19--20.","DOI":"10.1145\/2940716.2940757"},{"key":"key-10.1145\/3178876.3186039-13","unstructured":"Constantinos Daskalakis and Vasilis Syrgkanis . 2016. Learning in auctions: Regret is hard, envy is easy Foundations of Computer Science (FOCS), 2016 IEEE 57th Annual Symposium on. IEEE, 219--228."},{"key":"key-10.1145\/3178876.3186039-14","unstructured":"Peter Dayan and CJCH Watkins . 1992. Q-learning. Machine learning, Vol. 8, 3 (1992), 279--292."},{"key":"key-10.1145\/3178876.3186039-15","unstructured":"Thomas Degris, Patrick M Pilarski, and Richard S Sutton . 2012. Model-free reinforcement learning with continuous action in practice American Control Conference (ACC), 2012. IEEE, 2177--2182."},{"key":"key-10.1145\/3178876.3186039-16","unstructured":"Dylan J Foster, Zhiyuan Li, Thodoris Lykouris, Karthik Sridharan, and Eva Tardos . 2016. Learning in games: Robustness of fast convergence. Advances in Neural Information Processing Systems. 4734--4742."},{"key":"key-10.1145\/3178876.3186039-17","unstructured":"Yoav Freund and Robert E Schapire . 1995. A desicion-theoretic generalization of on-line learning and an application to boosting European conference on computational learning theory. Springer, 23--37."},{"key":"key-10.1145\/3178876.3186039-18","doi-asserted-by":"crossref","unstructured":"Sergiu Hart . 2005. Adaptive heuristics. Econometrica, Vol. 73, 5 (2005), 1401--1430.","DOI":"10.1111\/j.1468-0262.2005.00625.x"},{"key":"key-10.1145\/3178876.3186039-19","doi-asserted-by":"crossref","unstructured":"Sergiu Hart and Andreu Mas-Colell . 2000. A simple adaptive procedure leading to correlated equilibrium. Econometrica, Vol. 68, 5 (2000), 1127--1150.","DOI":"10.1111\/1468-0262.00153"},{"key":"key-10.1145\/3178876.3186039-20","doi-asserted-by":"crossref","unstructured":"Sergiu Hart and Andreu Mas-Colell . 2001. A general class of adaptive strategies. Journal of Economic Theory Vol. 98, 1 (2001), 26--54.","DOI":"10.1006\/jeth.2000.2746"},{"key":"key-10.1145\/3178876.3186039-21","unstructured":"Jason Hartline, Vasilis Syrgkanis, and Eva Tardos . 2015. No-regret learning in Bayesian games. In Advances in Neural Information Processing Systems. 3061--3069."},{"key":"key-10.1145\/3178876.3186039-22","unstructured":"Nicolas Heess, Jonathan J Hunt, Timothy P Lillicrap, and David Silver . 2015. Memory-based control with recurrent neural networks. arXiv preprint arXiv:1512.04455 (2015)."},{"key":"key-10.1145\/3178876.3186039-23","unstructured":"Sergey Ioffe and Christian Szegedy . 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)."},{"key":"key-10.1145\/3178876.3186039-24","unstructured":"Diederik Kingma and Jimmy Ba . 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"key-10.1145\/3178876.3186039-25","unstructured":"Andreas Krause and Cheng S Ong . 2011. Contextual gaussian process bandit optimization. Advances in Neural Information Processing Systems. 2447--2455."},{"key":"key-10.1145\/3178876.3186039-26","unstructured":"Lihong Li, Wei Chu, John Langford, and Robert E Schapire . 2010. A contextual-bandit approach to personalized news article recommendation Proceedings of the 19th international conference on World wide web. ACM, 661--670."},{"key":"key-10.1145\/3178876.3186039-27","unstructured":"Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra . 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)."},{"key":"key-10.1145\/3178876.3186039-28","unstructured":"Thodoris Lykouris, Vasilis Syrgkanis, and Éva Tardos . 2016. Learning and efficiency in games with dynamic population Proceedings of the Twenty-Seventh Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics, 120--129."},{"key":"key-10.1145\/3178876.3186039-29","doi-asserted-by":"crossref","unstructured":"Eric S Maskin . 2008. Mechanism design: How to implement social goals. The American Economic Review Vol. 98, 3 (2008), 567--576.","DOI":"10.1257\/aer.98.3.567"},{"key":"key-10.1145\/3178876.3186039-30","unstructured":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller . 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)."},{"key":"key-10.1145\/3178876.3186039-31","doi-asserted-by":"crossref","unstructured":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et almbox. . 2015. Human-level control through deep reinforcement learning. Nature, Vol. 518, 7540 (2015), 529--533.","DOI":"10.1038\/nature14236"},{"key":"key-10.1145\/3178876.3186039-32","doi-asserted-by":"crossref","unstructured":"Denis Nekipelov, Vasilis Syrgkanis, and Eva Tardos . 2015. Econometrics for learning agents. In Proceedings of the Sixteenth ACM Conference on Economics and Computation. ACM, 1--18.","DOI":"10.1145\/2764468.2764522"},{"key":"key-10.1145\/3178876.3186039-33","unstructured":"A Blum PI, M Blum, M Kearns, T Sandholm, and MT Hajiaghayi . [n. d.]. Machine Learning, Game Theory, and Mechanism Design for a Networked World. (. [n. d.])."},{"key":"key-10.1145\/3178876.3186039-34","doi-asserted-by":"crossref","unstructured":"Francesco Ricci, Lior Rokach, and Bracha Shapira . 2011. Introduction to recommender systems handbook. Springer.","DOI":"10.1007\/978-0-387-85820-3"},{"key":"key-10.1145\/3178876.3186039-35","doi-asserted-by":"crossref","unstructured":"Ariel Rubinstein . 1998. Modeling bounded rationality. MIT press.","DOI":"10.7551\/mitpress\/4702.001.0001"},{"key":"key-10.1145\/3178876.3186039-36","unstructured":"Weiran Shen, Binghui Peng, Hanpeng Liu, Michael Zhang, Ruohan Qian, Yan Hong, Zhi Guo, Zongyao Ding, Pengjun Lu, and Pingzhong Tang . 2017. Reinforcement mechanism design, with applications to dynamic pricing in sponsored search auctions. arXiv preprint arXiv:1711.10279 (2017)."},{"key":"key-10.1145\/3178876.3186039-37","unstructured":"David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Riedmiller . 2014. Deterministic policy gradient algorithms. In International Conference on Machine Learning (ICML."},{"key":"key-10.1145\/3178876.3186039-38","doi-asserted-by":"crossref","unstructured":"Richard S Sutton . 1988. Learning to predict by the methods of temporal differences. Machine learning, Vol. 3, 1 (1988), 9--44.","DOI":"10.1007\/BF00115009"},{"key":"key-10.1145\/3178876.3186039-39","unstructured":"Richard S Sutton, David A McAllester, Satinder P Singh, Yishay Mansour, et almbox. . 1999. Policy gradient methods for reinforcement learning with function approximation. NIPS, Vol. Vol. 99. 1057--1063."},{"key":"key-10.1145\/3178876.3186039-40","doi-asserted-by":"crossref","unstructured":"Pingzhong Tang . 2017. Reinforcement Mechanism Design. In Early Carrer Highlights at Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI. 5146--5150.","DOI":"10.24963\/ijcai.2017\/739"},{"key":"key-10.1145\/3178876.3186039-41","unstructured":"Eva Tardos . 2017. Learning and Efficiency of Outcomes in Games. (2017). Seminar Slides."},{"key":"key-10.1145\/3178876.3186039-42","doi-asserted-by":"crossref","unstructured":"Hado Van Hasselt, Arthur Guez, and David Silver . 2016. Deep Reinforcement Learning with Double Q-Learning. AAAI. 2094--2100.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"key-10.1145\/3178876.3186039-43","unstructured":"Christopher John Cornish Hellaby Watkins . 1989. Learning from delayed rewards. Ph.D. Dissertation. bibinfoschoolKing's College, Cambridge."}],"event":{"name":"the 2018 World Wide Web Conference","location":"Lyon, France","acronym":"WWW '18","number":"2018","sponsor":["SIGWEB, ACM Special Interest Group on Hypertext, Hypermedia, and Web","IW3C2, International World Wide Web Conference Committee"],"start":{"date-parts":[[2018,4,23]]},"end":{"date-parts":[[2018,4,27]]}},"container-title":["Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3178876.3186039","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/dl.acm.org\/ft_gateway.cfm?id=3186039&ftid=1957358&dwn=1","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:27:00Z","timestamp":1750213620000},"score":1,"resource":{"primary":{"URL":"http:\/\/dl.acm.org\/citation.cfm?doid=3178876.3186039"}},"subtitle":[],"proceedings-subject":"World Wide Web","short-title":[],"issued":{"date-parts":[[2018]]},"references-count":43,"URL":"https:\/\/doi.org\/10.1145\/3178876.3186039","relation":{},"subject":[],"published":{"date-parts":[[2018]]}}}