{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T10:07:46Z","timestamp":1775815666750,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":30,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,8,14]],"date-time":"2021-08-14T00:00:00Z","timestamp":1628899200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,8,14]]},"DOI":"10.1145\/3447548.3467086","type":"proceedings-article","created":{"date-parts":[[2021,8,12]],"date-time":"2021-08-12T06:12:05Z","timestamp":1628748725000},"page":"2890-2898","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["Real Negatives Matter: Continuous Training with Real Negatives for Delayed Feedback Modeling"],"prefix":"10.1145","author":[{"given":"Siyu","family":"Gu","sequence":"first","affiliation":[{"name":"Alibaba Group, Beijing, China"}]},{"given":"Xiang-Rong","family":"Sheng","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}]},{"given":"Ying","family":"Fan","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}]},{"given":"Guorui","family":"Zhou","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}]},{"given":"Xiaoqiang","family":"Zhu","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}]}],"member":"320","published-online":{"date-parts":[[2021,8,14]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.5555\/2567709.2567766"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623634"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1257\/jep.23.3.37"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2005.10.010"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1198\/tech.2003.s768"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1287\/mksc.1100.0583"},{"key":"e_1_3_2_1_7_1","volume-title":"The Elements of Statistical Learning: Data Mining, Inference, and Prediction","author":"Hastie Trevor","unstructured":"Trevor Hastie , Jerome H. Friedman , and Robert Tibshirani . 2001. The Elements of Statistical Learning: Data Mining, Inference, and Prediction . Springer . Trevor Hastie, Jerome H. Friedman, and Robert Tibshirani. 2001. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2648584.2648589"},{"key":"e_1_3_2_1_9_1","volume-title":"Proceedings of the 32nd International Conference on Machine Learning","volume":"37","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy . 2015 . Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift . In Proceedings of the 32nd International Conference on Machine Learning , Vol. 37 . 448--456. Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In Proceedings of the 32nd International Conference on Machine Learning, Vol. 37. 448--456."},{"key":"e_1_3_2_1_10_1","volume-title":"Proceedings of the 30th International Conference on Machine Learning","volume":"28","author":"Joulani Pooria","year":"2013","unstructured":"Pooria Joulani , Andr\u00e1 s Gy\u00f6 rgy, and Csaba Szepesv\u00e1 ri. 2013 . Online Learning under Delayed Feedback . In Proceedings of the 30th International Conference on Machine Learning , Vol. 28 . 1453--1461. Pooria Joulani, Andr\u00e1 s Gy\u00f6 rgy, and Csaba Szepesv\u00e1 ri. 2013. Online Learning under Delayed Feedback. In Proceedings of the 30th International Conference on Machine Learning, Vol. 28. 1453--1461."},{"key":"e_1_3_2_1_11_1","volume-title":"Proceedings of the 3rd International Conference on Learning Representations .","author":"Diederik","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization . In Proceedings of the 3rd International Conference on Learning Representations . Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of the 3rd International Conference on Learning Representations ."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3347002"},{"key":"e_1_3_2_1_13_1","volume-title":"Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 768--776","author":"Orten Burkay","year":"2012","unstructured":"Kuang-chih Lee, Burkay Orten , Ali Dasdan , and Wentong Li . 2012 . Estimating Conversion Rate in Display Advertising from Past Performance Data . In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 768--776 . Kuang-chih Lee, Burkay Orten, Ali Dasdan, and Wentong Li. 2012. Estimating Conversion Rate in Display Advertising from Past Performance Data. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 768--776."},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210104"},{"key":"e_1_3_2_1_15_1","volume-title":"Proceedings of the 30th International Conference on Machine Learning","volume":"30","author":"Maas Andrew L","year":"2013","unstructured":"Andrew L Maas , Awni Y Hannun , and Andrew Y Ng . 2013 . Rectifier nonlinearities improve neural network acoustic models . In Proceedings of the 30th International Conference on Machine Learning , Vol. 30 . 3. Andrew L Maas, Awni Y Hannun, and Andrew Y Ng. 2013. Rectifier nonlinearities improve neural network acoustic models. In Proceedings of the 30th International Conference on Machine Learning, Vol. 30. 3."},{"key":"e_1_3_2_1_16_1","volume-title":"Proceedings of the 14th International Conference on Artificial Intelligence and Statistics","volume":"15","author":"McMahan H. Brendan","year":"2011","unstructured":"H. Brendan McMahan . 2011 . Follow-the-Regularized-Leader and Mirror Descent: Equivalence Theorems and L1 Regularization . In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics , Vol. 15 . 525--533. H. Brendan McMahan. 2011. Follow-the-Regularized-Leader and Mirror Descent: Equivalence Theorems and L1 Regularization. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, Vol. 15. 525--533."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2020408.2020436"},{"key":"e_1_3_2_1_18_1","volume-title":"Aggregated Anonymous Feedback. In Proceedings of the 35th International Conference on Machine Learning","volume":"80","author":"Pike-Burke Ciara","year":"2018","unstructured":"Ciara Pike-Burke , Shipra Agrawal , Csaba Szepesv\u00e1ri , and Steffen Gr\u00fcnewalder . 2018 . Bandits with Delayed , Aggregated Anonymous Feedback. In Proceedings of the 35th International Conference on Machine Learning , Vol. 80 . 4102--4110. Ciara Pike-Burke, Shipra Agrawal, Csaba Szepesv\u00e1ri, and Steffen Gr\u00fcnewalder. 2018. Bandits with Delayed, Aggregated Anonymous Feedback. In Proceedings of the 35th International Conference on Machine Learning, Vol. 80. 4102--4110."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2124295.2124333"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3481941"},{"key":"e_1_3_2_1_21_1","volume-title":"Paul von B\u00fc nau, and Motoaki Kawanabe","author":"Sugiyama Masashi","year":"2007","unstructured":"Masashi Sugiyama , Shinichi Nakajima , Hisashi Kashima , Paul von B\u00fc nau, and Motoaki Kawanabe . 2007 . Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation. In Advances in Neural Information Processing Systems 20. 1433--1440. Masashi Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul von B\u00fc nau, and Motoaki Kawanabe. 2007. Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation. In Advances in Neural Information Processing Systems 20. 1433--1440."},{"key":"e_1_3_2_1_22_1","volume-title":"Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, Gal Elidan, Kristian Kersting, and Alexander T. Ihler (Eds.).","author":"Vernade Claire","year":"2017","unstructured":"Claire Vernade , Olivier Capp\u00e9 , and Vianney Perchet . 2017 . Stochastic Bandit Models for Delayed Conversions . In Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, Gal Elidan, Kristian Kersting, and Alexander T. Ihler (Eds.). Claire Vernade, Olivier Capp\u00e9, and Vianney Perchet. 2017. Stochastic Bandit Models for Delayed Conversions. In Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, Gal Elidan, Kristian Kersting, and Alexander T. Ihler (Eds.)."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1561\/9781680833119"},{"key":"e_1_3_2_1_24_1","volume-title":"Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling. CoRR","author":"Yang Jia-Qi","year":"2020","unstructured":"Jia-Qi Yang , Xiang Li , Shuguang Han , Tao Zhuang , De-Chuan Zhan , Xiaoyi Zeng , and Bin Tong . 2020. Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling. CoRR , Vol. abs\/ 2012 .03245 ( 2020 ). Jia-Qi Yang, Xiang Li, Shuguang Han, Tao Zhuang, De-Chuan Zhan, Xiaoyi Zeng, and Bin Tong. 2020. Capturing Delayed Feedback in Conversion Rate Prediction via Elapsed-Time Sampling. CoRR, Vol. abs\/2012.03245 (2020)."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380032"},{"key":"e_1_3_2_1_26_1","volume-title":"A Nonparametric Delayed Feedback Model for Conversion Rate Prediction. CoRR","author":"Yoshikawa Yuya","year":"2018","unstructured":"Yuya Yoshikawa and Yusaku Imai . 2018. A Nonparametric Delayed Feedback Model for Conversion Rate Prediction. CoRR , Vol. abs\/ 1802 .00255 ( 2018 ). Yuya Yoshikawa and Yusaku Imai. 2018. A Nonparametric Delayed Feedback Model for Conversion Rate Prediction. CoRR, Vol. abs\/1802.00255 (2018)."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623633"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939713"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33015941"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219823"}],"event":{"name":"KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Virtual Event Singapore","acronym":"KDD '21","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447548.3467086","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447548.3467086","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:25:11Z","timestamp":1750195511000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447548.3467086"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,14]]},"references-count":30,"alternative-id":["10.1145\/3447548.3467086","10.1145\/3447548"],"URL":"https:\/\/doi.org\/10.1145\/3447548.3467086","relation":{},"subject":[],"published":{"date-parts":[[2021,8,14]]},"assertion":[{"value":"2021-08-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}