{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,1]],"date-time":"2025-12-01T02:54:39Z","timestamp":1764557679128,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":13,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T00:00:00Z","timestamp":1691107200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Hoffman?Yee Research Grants Program and the Stanford Institute for Human-Centered Artificial Intelligence"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,8,6]]},"DOI":"10.1145\/3580305.3599222","type":"proceedings-article","created":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T18:10:58Z","timestamp":1691172658000},"page":"5851-5852","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["EvalRS 2023: Well-Rounded Recommender Systems for Real-World Deployments"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0776-361X","authenticated-orcid":false,"given":"Federico","family":"Bianchi","sequence":"first","affiliation":[{"name":"Stanford University, Palo Alto, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1859-3944","authenticated-orcid":false,"given":"Patrick John","family":"Chia","sequence":"additional","affiliation":[{"name":"Coveo, Montreal, PQ, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8634-6122","authenticated-orcid":false,"given":"Jacopo","family":"Tagliabue","sequence":"additional","affiliation":[{"name":"New York University & Bauplan, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-0359-4130","authenticated-orcid":false,"given":"Ciro","family":"Greco","sequence":"additional","affiliation":[{"name":"Bauplan, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9745-5014","authenticated-orcid":false,"given":"Gabriel S P","family":"Moreira","sequence":"additional","affiliation":[{"name":"Nvidia, S\u00e3o Paulo, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-2122-300X","authenticated-orcid":false,"given":"Davide","family":"Eynard","sequence":"additional","affiliation":[{"name":"mozilla.ai, London, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-6816-9278","authenticated-orcid":false,"given":"Fahd","family":"Husain","sequence":"additional","affiliation":[{"name":"mozilla.ai, Toronto, ON, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5206-3909","authenticated-orcid":false,"given":"Claudio","family":"Pomo","sequence":"additional","affiliation":[{"name":"Politecnico di Bari, Bari, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,8,4]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Tommaso Di Noia, Dietmar Jannach, and Claudio Pomo.","author":"Anelli Vito Walter","year":"2022","unstructured":"Vito Walter Anelli , Alejandro Bellog'i n , Tommaso Di Noia, Dietmar Jannach, and Claudio Pomo. 2022 . Top-N Recommendation Algorithms: A Quest for the State-of-the-Art. In UMAP. ACM , 121--131. Vito Walter Anelli, Alejandro Bellog'i n, Tommaso Di Noia, Dietmar Jannach, and Claudio Pomo. 2022. Top-N Recommendation Algorithms: A Quest for the State-of-the-Art. In UMAP. ACM, 121--131."},{"volume-title":"NAACL-HLT (Industry Papers)","author":"Bianchi Federico","key":"e_1_3_2_1_2_1","unstructured":"Federico Bianchi , Jacopo Tagliabue , and Bingqing Yu. 2021. Query2Prod2Vec: Grounded Word Embeddings for eCommerce . In NAACL-HLT (Industry Papers) . Association for Computational Linguistics , 154--162. Federico Bianchi, Jacopo Tagliabue, and Bingqing Yu. 2021. Query2Prod2Vec: Grounded Word Embeddings for eCommerce. In NAACL-HLT (Industry Papers). Association for Computational Linguistics, 154--162."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Patrick John Chia Jacopo Tagliabue Federico Bianchi Chloe He and Brian Ko. 2022. Beyond NDCG: Behavioral Testing of Recommender Systems with RecList. In WWW (Companion Volume). ACM 99--104. Patrick John Chia Jacopo Tagliabue Federico Bianchi Chloe He and Brian Ko. 2022. Beyond NDCG: Behavioral Testing of Recommender Systems with RecList. In WWW (Companion Volume). ACM 99--104.","DOI":"10.1145\/3487553.3524215"},{"volume-title":"Algorithmic recommendations, anyone? Nature Machine Intelligence","year":"2023","key":"e_1_3_2_1_4_1","unstructured":"Editors. 2023. Algorithmic recommendations, anyone? Nature Machine Intelligence ( 2023 ). https:\/\/doi.org\/10.1038\/s42256-023-00631--7 10.1038\/s42256-023-00631--7 Editors. 2023. Algorithmic recommendations, anyone? Nature Machine Intelligence (2023). https:\/\/doi.org\/10.1038\/s42256-023-00631--7"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Karl Higley Even Oldridge Ronay Ak Sara Rabhi and Gabriel de Souza Pereira Moreira. 2022. Building and Deploying a Multi-Stage Recommender System with Merlin. In RecSys. ACM 632--635. Karl Higley Even Oldridge Ronay Ak Sara Rabhi and Gabriel de Souza Pereira Moreira. 2022. Building and Deploying a Multi-Stage Recommender System with Merlin. In RecSys. ACM 632--635.","DOI":"10.1145\/3523227.3551468"},{"key":"e_1_3_2_1_6_1","volume-title":"Gabriel de Souza Pereira Moreira, and Even Oldridge","author":"Jannach Dietmar","year":"2020","unstructured":"Dietmar Jannach , Gabriel de Souza Pereira Moreira, and Even Oldridge . 2020 . Why Are Deep Learning Models Not Consistently Winning Recommender Systems Competitions Yet?: A Position Paper. In RecSys Challenge. ACM , 44--49. Dietmar Jannach, Gabriel de Souza Pereira Moreira, and Even Oldridge. 2020. Why Are Deep Learning Models Not Consistently Winning Recommender Systems Competitions Yet?: A Position Paper. In RecSys Challenge. ACM, 44--49."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v42i3.18139"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"crossref","unstructured":"Pigi Kouki Ilias Fountalis Nikolaos Vasiloglou Xiquan Cui Edo Liberty and Khalifeh Al Jadda. 2020. From the lab to production: A case study of session-based recommendations in the home-improvement domain. In RecSys. ACM 140--149. Pigi Kouki Ilias Fountalis Nikolaos Vasiloglou Xiquan Cui Edo Liberty and Khalifeh Al Jadda. 2020. From the lab to production: A case study of session-based recommendations in the home-improvement domain. In RecSys. ACM 140--149.","DOI":"10.1145\/3383313.3412235"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Ahmed Rashed Shayan Jawed Lars Schmidt-Thieme and Andre Hintsches. 2020. MultiRec: A Multi-Relational Approach for Unique Item Recommendation in Auction Systems. In RecSys. ACM 230--239. Ahmed Rashed Shayan Jawed Lars Schmidt-Thieme and Andre Hintsches. 2020. MultiRec: A Multi-Relational Approach for Unique Item Recommendation in Auction Systems. In RecSys. ACM 230--239.","DOI":"10.1145\/3383313.3412242"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Zhu Sun Di Yu Hui Fang Jie Yang Xinghua Qu Jie Zhang and Cong Geng. 2020. Are We Evaluating Rigorously? Benchmarking Recommendation for Reproducible Evaluation and Fair Comparison. In RecSys. ACM 23--32. Zhu Sun Di Yu Hui Fang Jie Yang Xinghua Qu Jie Zhang and Cong Geng. 2020. Are We Evaluating Rigorously? Benchmarking Recommendation for Reproducible Evaluation and Fair Comparison. In RecSys. ACM 23--32.","DOI":"10.1145\/3383313.3412489"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"crossref","unstructured":"Jacopo Tagliabue. 2021. You Do Not Need a Bigger Boat: Recommendations at Reasonable Scale in a (Mostly) Serverless and Open Stack. In RecSys. ACM 598--600. Jacopo Tagliabue. 2021. You Do Not Need a Bigger Boat: Recommendations at Reasonable Scale in a (Mostly) Serverless and Open Stack. In RecSys. ACM 598--600.","DOI":"10.1145\/3460231.3474604"},{"key":"e_1_3_2_1_12_1","volume-title":"Gabriel de Souza Moreira, and Patrick John Chia","author":"Tagliabue Jacopo","year":"2023","unstructured":"Jacopo Tagliabue , Federico Bianchi , Tobias Schnabel , Giuseppe Attanasio , Ciro Greco , Gabriel de Souza Moreira, and Patrick John Chia . 2023 . A challenge for rounded evaluation of recommender systems. Nature Machine Intelligence ( 2023), 1--2. Jacopo Tagliabue, Federico Bianchi, Tobias Schnabel, Giuseppe Attanasio, Ciro Greco, Gabriel de Souza Moreira, and Patrick John Chia. 2023. A challenge for rounded evaluation of recommender systems. Nature Machine Intelligence (2023), 1--2."},{"key":"e_1_3_2_1_13_1","volume-title":"CIKM Workshops (CEUR Workshop Proceedings","author":"Tagliabue Jacopo","year":"2022","unstructured":"Jacopo Tagliabue , Federico Bianchi , Tobias Schnabel , Giuseppe Attanasio , Ciro Greco , Gabriel de Souza P. Moreira , and Patrick John Chia . 2022 . EvalRS: a rounded evaluation of recommender systems . In CIKM Workshops (CEUR Workshop Proceedings , Vol. 3318). CEUR-WS.org. Jacopo Tagliabue, Federico Bianchi, Tobias Schnabel, Giuseppe Attanasio, Ciro Greco, Gabriel de Souza P. Moreira, and Patrick John Chia. 2022. EvalRS: a rounded evaluation of recommender systems. In CIKM Workshops (CEUR Workshop Proceedings, Vol. 3318). CEUR-WS.org."}],"event":{"name":"KDD '23: The 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"],"location":"Long Beach CA USA","acronym":"KDD '23"},"container-title":["Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580305.3599222","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3580305.3599222","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:51:15Z","timestamp":1750182675000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580305.3599222"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,4]]},"references-count":13,"alternative-id":["10.1145\/3580305.3599222","10.1145\/3580305"],"URL":"https:\/\/doi.org\/10.1145\/3580305.3599222","relation":{},"subject":[],"published":{"date-parts":[[2023,8,4]]},"assertion":[{"value":"2023-08-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}