{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,28]],"date-time":"2026-07-28T15:12:01Z","timestamp":1785251521203,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":33,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,9,13]],"date-time":"2021-09-13T00:00:00Z","timestamp":1631491200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,9,13]]},"DOI":"10.1145\/3460231.3474235","type":"proceedings-article","created":{"date-parts":[[2021,9,13]],"date-time":"2021-09-13T21:45:02Z","timestamp":1631569502000},"page":"96-101","source":"Crossref","is-referenced-by-count":2,"title":["Online Evaluation Methods for the Causal Effect of Recommendations"],"prefix":"10.1145","author":[{"given":"Masahiro","family":"Sato","sequence":"first","affiliation":[{"name":"independent researcher, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,9,13]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1509\/jmkr.45.1.77"},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240360"},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2094072.2094078"},{"key":"e_1_3_2_2_4_1","unstructured":"Jiawei Chen Hande Dong Xiang Wang Fuli Feng Meng Wang and Xiangnan He. 2020. Bias and debias in recommender system: A survey and future directions. arXiv preprint arXiv:2010.03240(2020).  Jiawei Chen Hande Dong Xiang Wang Fuli Feng Meng Wang and Xiangnan He. 2020. Bias and debias in recommender system: A survey and future directions. arXiv preprint arXiv:2010.03240(2020)."},{"key":"e_1_3_2_2_5_1","volume-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft","author":"Cosley Dan","year":"2003","unstructured":"Dan Cosley , Shyong\u00a0 K. Lam , Istvan Albert , Joseph\u00a0 A. Konstan , and John Riedl . 2003 . Is Seeing Believing? How Recommender System Interfaces Affect Users\u2019 Opinions . In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft . Lauderdale, Florida, USA) (CHI \u201903). Association for Computing Machinery, New York, NY, USA, 585\u2013592. https:\/\/doi.org\/10.1145\/642611.642713 Dan Cosley, Shyong\u00a0K. Lam, Istvan Albert, Joseph\u00a0A. Konstan, and John Riedl. 2003. Is Seeing Believing? How Recommender System Interfaces Affect Users\u2019 Opinions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI \u201903). Association for Computing Machinery, New York, NY, USA, 585\u2013592. https:\/\/doi.org\/10.1145\/642611.642713"},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1454008.1454054"},{"key":"e_1_3_2_2_7_1","article-title":"The MovieLens Datasets","volume":"5","author":"Harper Maxwell","year":"2015","unstructured":"F.\u00a0 Maxwell Harper and Joseph\u00a0 A. Konstan . 2015 . The MovieLens Datasets : History and Context. ACM Trans. Interact. Intell. Syst. 5 , 4, Article 19 (Dec. 2015), 19\u00a0pages. https:\/\/doi.org\/10.1145\/2827872 F.\u00a0Maxwell Harper and Joseph\u00a0A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Trans. Interact. Intell. Syst. 5, 4, Article 19 (Dec. 2015), 19\u00a0pages. https:\/\/doi.org\/10.1145\/2827872","journal-title":"History and Context. ACM Trans. Interact. Intell. Syst."},{"key":"e_1_3_2_2_8_1","unstructured":"MA Hern\u00e1n and JM Robins. 2020. Causal inference: What if. Boca Raton: Chapman & Hill\/CRC(2020).  MA Hern\u00e1n and JM Robins. 2020. Causal inference: What if. Boca Raton: Chapman & Hill\/CRC(2020)."},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063618"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1986.10478354"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3347008"},{"key":"e_1_3_2_2_12_1","volume-title":"Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction","author":"Imbens W.","unstructured":"Guido\u00a0 W. Imbens and Donald\u00a0 B. Rubin . 2015. Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction . Cambridge University Press , New York, NY, USA . Guido\u00a0W. Imbens and Donald\u00a0B. Rubin. 2015. Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press, New York, NY, USA."},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3370082"},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775067"},{"key":"e_1_3_2_2_15_1","volume-title":"Evaluating Retrieval Performance Using Clickthrough Data","author":"Joachims Thorsten","unstructured":"Thorsten Joachims . 2003. Evaluating Retrieval Performance Using Clickthrough Data . In Text Mining, Theoretical Aspects and Applications, J\u00fcrgen Franke, Gholamreza Nakhaeizadeh, and Ingrid Renz (Eds.). Physica-Verlag , 79\u201396. Thorsten Joachims. 2003. Evaluating Retrieval Performance Using Clickthrough Data. In Text Mining, Theoretical Aspects and Applications, J\u00fcrgen Franke, Gholamreza Nakhaeizadeh, and Ingrid Renz (Eds.). Physica-Verlag, 79\u201396."},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.1903"},{"key":"e_1_3_2_2_17_1","volume-title":"A Comprehensive Survey of Neighborhood-Based Recommendation Methods","author":"Ning Xia","unstructured":"Xia Ning , Christian Desrosiers , and George Karypis . 2015. A Comprehensive Survey of Neighborhood-Based Recommendation Methods . Springer US , Boston, MA , 37\u201376. Xia Ning, Christian Desrosiers, and George Karypis. 2015. A Comprehensive Survey of Neighborhood-Based Recommendation Methods. Springer US, Boston, MA, 37\u201376."},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2433396.2433429"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458092"},{"key":"e_1_3_2_2_20_1","volume-title":"Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (Montreal","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle , Christoph Freudenthaler , Zeno Gantner , and Lars Schmidt-Thieme . 2009 . BPR: Bayesian Personalized Ranking from Implicit Feedback . In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (Montreal , Quebec, Canada) (UAI \u201909). AUAI Press, Arlington, Virginia, USA, 452\u2013461. Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (Montreal, Quebec, Canada) (UAI \u201909). AUAI Press, Arlington, Virginia, USA, 452\u2013461."},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/70.1.41"},{"key":"e_1_3_2_2_22_1","volume-title":"Estimating causal effects of treatments in randomized and nonrandomized studies.Journal of educational Psychology 66, 5","author":"Rubin B","year":"1974","unstructured":"Donald\u00a0 B Rubin . 1974. Estimating causal effects of treatments in randomized and nonrandomized studies.Journal of educational Psychology 66, 5 ( 1974 ), 688. Donald\u00a0B Rubin. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies.Journal of educational Psychology 66, 5 (1974), 688."},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3336191.3371783"},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2930238.2930259"},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3347018"},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341105.3375760"},{"key":"e_1_3_2_2_27_1","volume-title":"Unbiased Learning for the Causal Effect of Recommendation. In Fourteenth ACM Conference on Recommender Systems","author":"Sato Masahiro","year":"2020","unstructured":"Masahiro Sato , Sho Takemori , Janmajay Singh , and Tomoko Ohkuma . 2020 . Unbiased Learning for the Causal Effect of Recommendation. In Fourteenth ACM Conference on Recommender Systems ( Virtual Event, Brazil) (RecSys \u201920). Association for Computing Machinery, New York, NY, USA, 378\u2013387. https:\/\/doi.org\/10.1145\/3383313.3412261 Masahiro Sato, Sho Takemori, Janmajay Singh, and Tomoko Ohkuma. 2020. Unbiased Learning for the Causal Effect of Recommendation. In Fourteenth ACM Conference on Recommender Systems (Virtual Event, Brazil) (RecSys \u201920). Association for Computing Machinery, New York, NY, USA, 378\u2013387. https:\/\/doi.org\/10.1145\/3383313.3412261"},{"key":"e_1_3_2_2_28_1","volume-title":"Advances in Information Retrieval(ECIR \u201921)","author":"Sato Masahiro","unstructured":"Masahiro Sato , Sho Takemori , Janmajay Singh , and Qian Zhang . 2021. Causality-Aware Neighborhood Methods for Recommender Systems . In Advances in Information Retrieval(ECIR \u201921) . Springer International Publishing , 603\u2013618. https:\/\/doi.org\/10.1007\/978-3-030-72113-8_40 Masahiro Sato, Sho Takemori, Janmajay Singh, and Qian Zhang. 2021. Causality-Aware Neighborhood Methods for Recommender Systems. In Advances in Information Retrieval(ECIR \u201921). Springer International Publishing, 603\u2013618. https:\/\/doi.org\/10.1007\/978-3-030-72113-8_40"},{"key":"e_1_3_2_2_29_1","volume-title":"Proceedings of the 33rd International Conference on International Conference on Machine Learning -","volume":"48","author":"Schnabel Tobias","year":"2016","unstructured":"Tobias Schnabel , Adith Swaminathan , Ashudeep Singh , Navin Chandak , and Thorsten Joachims . 2016 . Recommendations as Treatments: Debiasing Learning and Evaluation . In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 (New York, NY, USA) (ICML\u201916). JMLR.org, 1670\u20131679. Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as Treatments: Debiasing Learning and Evaluation. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 (New York, NY, USA) (ICML\u201916). JMLR.org, 1670\u20131679."},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767838"},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661829.2661952"},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2764468.2764488"},{"key":"e_1_3_2_2_33_1","volume-title":"Causal Inference for Recommender Systems. In Fourteenth ACM Conference on Recommender Systems","author":"Wang Yixin","year":"2020","unstructured":"Yixin Wang , Dawen Liang , Laurent Charlin , and David\u00a0 M. Blei . 2020 . Causal Inference for Recommender Systems. In Fourteenth ACM Conference on Recommender Systems ( Virtual Event, Brazil) (RecSys \u201920). Association for Computing Machinery, New York, NY, USA, 426\u2013431. https:\/\/doi.org\/10.1145\/3383313.3412225 Yixin Wang, Dawen Liang, Laurent Charlin, and David\u00a0M. Blei. 2020. Causal Inference for Recommender Systems. In Fourteenth ACM Conference on Recommender Systems(Virtual Event, Brazil) (RecSys \u201920). Association for Computing Machinery, New York, NY, USA, 426\u2013431. https:\/\/doi.org\/10.1145\/3383313.3412225"}],"event":{"name":"RecSys '21: Fifteenth ACM Conference on Recommender Systems","location":"Amsterdam Netherlands","acronym":"RecSys '21","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGAI ACM Special Interest Group on Artificial Intelligence","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data","SIGIR ACM Special Interest Group on Information Retrieval","SIGCHI ACM Special Interest Group on Computer-Human Interaction","SIGecom Special Interest Group on Economics and Computation"]},"container-title":["Fifteenth ACM Conference on Recommender Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460231.3474235","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3460231.3474235","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:12:17Z","timestamp":1750191137000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460231.3474235"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,13]]},"references-count":33,"alternative-id":["10.1145\/3460231.3474235","10.1145\/3460231"],"URL":"https:\/\/doi.org\/10.1145\/3460231.3474235","relation":{},"subject":[],"published":{"date-parts":[[2021,9,13]]}}}