{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T02:34:27Z","timestamp":1785378867129,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":40,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T00:00:00Z","timestamp":1603065600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation","award":["17-51278"],"award-info":[{"award-number":["17-51278"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,10,19]]},"DOI":"10.1145\/3340531.3412778","type":"proceedings-article","created":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T05:31:06Z","timestamp":1603085466000},"page":"2999-3006","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":67,"title":["LensKit for Python: Next-Generation Software for Recommender Systems Experiments"],"prefix":"10.1145","author":[{"given":"Michael D.","family":"Ekstrand","sequence":"first","affiliation":[{"name":"Boise State University, Boise, ID, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,10,19]]},"reference":[{"key":"e_1_3_2_2_1_1","unstructured":"Mart'in Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Ian Goodfellow Andrew Harp Geoffrey Irving Michael Isard Yangqing Jia Rafal Jozefowicz Lukasz Kaiser Manjunath Kudlur Josh Levenberg Dandelion Man\u00e9 Rajat Monga Sherry Moore Derek Murray Chris Olah Mike Schuster Jonathon Shlens Benoit Steiner Ilya Sutskever Kunal Talwar Paul Tucker Vincent Vanhoucke Vijay Vasudevan Fernanda Vi\u00e9gas Oriol Vinyals Pete Warden Martin Wattenberg Martin Wicke Yuan Yu and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https:\/\/www.tensorflow.org\/ Software available from tensorflow.org.  Mart'in Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Ian Goodfellow Andrew Harp Geoffrey Irving Michael Isard Yangqing Jia Rafal Jozefowicz Lukasz Kaiser Manjunath Kudlur Josh Levenberg Dandelion Man\u00e9 Rajat Monga Sherry Moore Derek Murray Chris Olah Mike Schuster Jonathon Shlens Benoit Steiner Ilya Sutskever Kunal Talwar Paul Tucker Vincent Vanhoucke Vijay Vasudevan Fernanda Vi\u00e9gas Oriol Vinyals Pete Warden Martin Wattenberg Martin Wicke Yuan Yu and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https:\/\/www.tensorflow.org\/ Software available from tensorflow.org."},{"key":"e_1_3_2_2_2_1","unstructured":"Christian Ricardo Balseca N\u00ed\u00f1ez and Daniel Andr\u00e9s Fern\u00e1ndez Pe\u00f1afiel. 2019. Estudio del estado del arte de la ciencia de datos aplicada a la neuroeconom\u00eda. Master's thesis. Universidad Polit\u00e9cnica Salesiana. http:\/\/hdl.handle.net\/123456789\/17605  Christian Ricardo Balseca N\u00ed\u00f1ez and Daniel Andr\u00e9s Fern\u00e1ndez Pe\u00f1afiel. 2019. Estudio del estado del arte de la ciencia de datos aplicada a la neuroeconom\u00eda. Master's thesis. Universidad Polit\u00e9cnica Salesiana. http:\/\/hdl.handle.net\/123456789\/17605"},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.5555\/2567709.2567766"},{"key":"e_1_3_2_2_5_1","volume-title":"API design for machine learning software: experiences from the scikit-learn project. CoRR","author":"Buitinck Lars","year":"2013","unstructured":"Lars Buitinck , Gilles Louppe , Mathieu Blondel , Fabian Pedregosa , Andreas Mueller , Olivier Grisel , Vlad Niculae , Peter Prettenhofer , Alexandre Gramfort , Jaques Grobler , Robert Layton , Jake VanderPlas , Arnaud Joly , Brian Holt , and Ga\u00eb l Varoquaux . 2013. API design for machine learning software: experiences from the scikit-learn project. CoRR , Vol. abs\/ 1309 .0238 ( 2013 ). http:\/\/arxiv.org\/abs\/1309.0238 Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Ga\u00eb l Varoquaux. 2013. API design for machine learning software: experiences from the scikit-learn project. CoRR, Vol. abs\/1309.0238 (2013). http:\/\/arxiv.org\/abs\/1309.0238"},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/963770.963776"},{"key":"e_1_3_2_2_7_1","unstructured":"Michael Ekstrand. 2016. Testing Recommenders. https:\/\/buildingrecommenders.wordpress.com\/2016\/02\/04\/testing-recommenders\/  Michael Ekstrand. 2016. Testing Recommenders. https:\/\/buildingrecommenders.wordpress.com\/2016\/02\/04\/testing-recommenders\/"},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043932.2043958"},{"key":"e_1_3_2_2_9_1","volume-title":"Foundations and Trends\u00ae in Human-Computer Interaction","volume":"4","author":"Ekstrand Michael","year":"2010","unstructured":"Michael Ekstrand , John Riedl , and Joseph A Konstan . 2010 . Collaborative Filtering Recommender Systems . Foundations and Trends\u00ae in Human-Computer Interaction , Vol. 4 , 2 (2010), 81--173. https:\/\/doi.org\/10.1561\/1100000009 10.1561\/1100000009 Michael Ekstrand, John Riedl, and Joseph A Konstan. 2010. Collaborative Filtering Recommender Systems. Foundations and Trends\u00ae in Human-Computer Interaction, Vol. 4, 2 (2010), 81--173. https:\/\/doi.org\/10.1561\/1100000009"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"crossref","unstructured":"Michael D Ekstrand and Daniel Kluver. 2019. Exploring Author Gender in Book Rating and Recommendation. https:\/\/md.ekstrandom.net\/pubs\/bag-extended  Michael D Ekstrand and Daniel Kluver. 2019. Exploring Author Gender in Book Rating and Recommendation. https:\/\/md.ekstrandom.net\/pubs\/bag-extended","DOI":"10.1145\/3240323.3240373"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.5381\/jot.2016.15.1.a1"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240373"},{"key":"e_1_3_2_2_13_1","volume-title":"Netflix Update: Try This at Home","author":"Funk Simon","year":"2006","unstructured":"Simon Funk . 2006 . Netflix Update: Try This at Home . http:\/\/sifter.org\/simon\/journal\/20061211.html Simon Funk. 2006. Netflix Update: Try This at Home. http:\/\/sifter.org\/simon\/journal\/20061211.html"},{"key":"e_1_3_2_2_14_1","volume-title":"Scalable Recommendation with Poisson Factorization. arXiv:1311.1704 [cs, stat] (Nov","author":"Gopalan Prem","year":"2013","unstructured":"Prem Gopalan , Jake M Hofman , and David M Blei . 2013. Scalable Recommendation with Poisson Factorization. arXiv:1311.1704 [cs, stat] (Nov . 2013 ). http:\/\/arxiv.org\/abs\/1311.1704 Prem Gopalan, Jake M Hofman, and David M Blei. 2013. Scalable Recommendation with Poisson Factorization. arXiv:1311.1704 [cs, stat] (Nov. 2013). http:\/\/arxiv.org\/abs\/1311.1704"},{"key":"e_1_3_2_2_15_1","article-title":"A Survey of Accuracy Evaluation Metrics of Recommendation Tasks","volume":"10","author":"Gunawardana Asela","year":"2009","unstructured":"Asela Gunawardana and Guy Shani . 2009 . A Survey of Accuracy Evaluation Metrics of Recommendation Tasks . Journal of Machine Learning Research , Vol. 10 ( Dec. 2009), 2935--2962. http:\/\/jmlr.org\/papers\/v10\/gunawardana09a.html Asela Gunawardana and Guy Shani. 2009. A Survey of Accuracy Evaluation Metrics of Recommendation Tasks. Journal of Machine Learning Research , Vol. 10 (Dec. 2009), 2935--2962. http:\/\/jmlr.org\/papers\/v10\/gunawardana09a.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_2_16_1","first-page":"4","article-title":"The MovieLens Datasets: History and Context","volume":"5","author":"Maxwell Harper F.","year":"2015","unstructured":"F. Maxwell Harper and Joseph A. Konstan . 2015 . The MovieLens Datasets: History and Context . ACM Transactions on Interactive Intelligent Systems , Vol. 5 , 4 (Dec. 2015), 19:1--19:19. https:\/\/doi.org\/10.1145\/2827872 10.1145\/2827872 F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems, Vol. 5, 4 (Dec. 2015), 19:1--19:19. https:\/\/doi.org\/10.1145\/2827872","journal-title":"ACM Transactions on Interactive Intelligent Systems"},{"key":"#cr-split#-e_1_3_2_2_17_1.1","unstructured":"Tim Head MechCoder Gilles Louppe Iaroslav Shcherbatyi fcharras Z\u00e9 Vin\u00edcius cmmalone Christopher Schr\u00f6der nel215 Nuno Campos Todd Young Stefano Cereda Thomas Fan rene rex Kejia (KJ) Shi Justus Schwabedal carlosdanielcsantos Hvass-Labs Mikhail Pak SoManyUsernamesTaken Fred Callaway Lo\u00efc Est\u00e8ve Lilian Besson Mehdi Cherti Karlson Pfannschmidt Fabian Linzberger Christophe Cauet Anna Gut Andreas Mueller and Alexander Fabisch. 2018. scikit-optimize\/scikit-optimize: v0.5.2. https:\/\/doi.org\/10.5281\/zenodo.1207017 10.5281\/zenodo.1207017"},{"key":"#cr-split#-e_1_3_2_2_17_1.2","unstructured":"Tim Head MechCoder Gilles Louppe Iaroslav Shcherbatyi fcharras Z\u00e9 Vin\u00edcius cmmalone Christopher Schr\u00f6der nel215 Nuno Campos Todd Young Stefano Cereda Thomas Fan rene rex Kejia (KJ) Shi Justus Schwabedal carlosdanielcsantos Hvass-Labs Mikhail Pak SoManyUsernamesTaken Fred Callaway Lo\u00efc Est\u00e8ve Lilian Besson Mehdi Cherti Karlson Pfannschmidt Fabian Linzberger Christophe Cauet Anna Gut Andreas Mueller and Alexander Fabisch. 2018. scikit-optimize\/scikit-optimize: v0.5.2. https:\/\/doi.org\/10.5281\/zenodo.1207017"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1020443909834"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.22"},{"key":"e_1_3_2_2_20_1","unstructured":"Nicolas Hug. 2017. Surprise a Python Library for Recommender Systems. http:\/\/surpriselib.com  Nicolas Hug. 2017. Surprise a Python Library for Recommender Systems. http:\/\/surpriselib.com"},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2728171"},{"key":"e_1_3_2_2_22_1","volume-title":"Vera, Max Hora, xliiv, Peter Rowlands, Wojciech Baranowski, Sanidhya Mangal, and Clemens Wolff.","author":"Kuprieiev Ruslan","year":"2020","unstructured":"Ruslan Kuprieiev , Dmitry Petrov , Ram\u00f3n Valles , Pawe? Redzy'ski, Casper da Costa-Luis , Alexander Schepanovski , Ivan Shcheklein , Saugat Pachhai , Jorge Orpinel , F\u00e1bio Santos , Aman Sharma , Zhanibek, Dani Hodovic , Earl, Andrew Grigorev , Nabanita Dash , George Vyshnya , maykulkarni , Vera, Max Hora, xliiv, Peter Rowlands, Wojciech Baranowski, Sanidhya Mangal, and Clemens Wolff. 2020 . DVC : Data Version Control - Git for Data & Models . https:\/\/doi.org\/10.5281\/zenodo.3813759 10.5281\/zenodo.3813759 Ruslan Kuprieiev, Dmitry Petrov, Ram\u00f3n Valles, Pawe? Redzy'ski, Casper da Costa-Luis, Alexander Schepanovski, Ivan Shcheklein, Saugat Pachhai, Jorge Orpinel, F\u00e1bio Santos, Aman Sharma, Zhanibek, Dani Hodovic, Earl, Andrew Grigorev, Nabanita Dash, George Vyshnya, maykulkarni, Vera, Max Hora, xliiv, Peter Rowlands, Wojciech Baranowski, Sanidhya Mangal, and Clemens Wolff. 2020. DVC: Data Version Control - Git for Data & Models. https:\/\/doi.org\/10.5281\/zenodo.3813759"},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2833157.2833162"},{"key":"e_1_3_2_2_24_1","unstructured":"Wes McKinney. 2018. Python for Data Analysis: Data Wrangling with pandas NumPy and IPython. O'Reilly.  Wes McKinney. 2018. Python for Data Analysis: Data Wrangling with pandas NumPy and IPython. O'Reilly."},{"key":"e_1_3_2_2_25_1","volume-title":"What is the Value of Rating Obscure Items? An Analysis of the Effect of Less-Popular Items on Recommendation Quality. Master's thesis","author":"Narayan A","year":"2061","unstructured":"A Narayan . 2019. What is the Value of Rating Obscure Items? An Analysis of the Effect of Less-Popular Items on Recommendation Quality. Master's thesis . University of Minnesota . https:\/\/hdl.handle.net\/11299\/ 2061 48 A Narayan. 2019. What is the Value of Rating Obscure Items? An Analysis of the Effect of Less-Popular Items on Recommendation Quality. Master's thesis. University of Minnesota. https:\/\/hdl.handle.net\/11299\/206148"},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJBIDM.2020.104738"},{"key":"e_1_3_2_2_27_1","volume-title":"A Guide to NumPy","author":"Oliphant Travis E.","unstructured":"Travis E. Oliphant . 2006. A Guide to NumPy . Trelgol Publishing . Travis E. Oliphant. 2006. A Guide to NumPy. Trelgol Publishing."},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2007.58"},{"key":"e_1_3_2_2_29_1","unstructured":"PBS Digital Studios. 2019. Let's Make a Movie Recommendation System (LAB) #16. https:\/\/www.pbs.org\/video\/3036002175\/  PBS Digital Studios. 2019. Let's Make a Movie Recommendation System (LAB) #16. https:\/\/www.pbs.org\/video\/3036002175\/"},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1864708.1864726"},{"key":"e_1_3_2_2_32_1","volume-title":"BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proc. UAI '09","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle , Christoph Freudenthaler , Zeno Gantner , and Lars Schmidt-Thieme . 2009 . BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proc. UAI '09 . AUAI Press , 452--461. https:\/\/doi.org\/10.1145\/1795114.1795167 10.1145\/1795114.1795167 Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In Proc. UAI '09. AUAI Press, 452--461. https:\/\/doi.org\/10.1145\/1795114.1795167"},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/371920.372071"},{"key":"e_1_3_2_2_34_1","volume-title":"RecSys '17 Poster Proceedings. https:\/\/arxiv.org\/abs\/1706","author":"Sepulveda Gabriel","year":"2017","unstructured":"Gabriel Sepulveda , Vicente Dominguez , and Denis Parra . 2017 . pyRecLab: A Software Library for Quick Prototyping of Recommender Systems . In RecSys '17 Poster Proceedings. https:\/\/arxiv.org\/abs\/1706 .06291 Gabriel Sepulveda, Vicente Dominguez, and Denis Parra. 2017. pyRecLab: A Software Library for Quick Prototyping of Recommender Systems. In RecSys '17 Poster Proceedings. https:\/\/arxiv.org\/abs\/1706.06291"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043932.2043987"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3343413.3378004"},{"key":"e_1_3_2_2_37_1","volume-title":"Practical Data Science with Python 3","author":"Varga Ervin","unstructured":"Ervin Varga . 2019. Recommender Systems . In Practical Data Science with Python 3 . Apress , 317--339. https:\/\/doi.org\/10.1007\/978--1--4842--4859--1 10.1007\/978--1--4842--4859--1 Ervin Varga. 2019. Recommender Systems. In Practical Data Science with Python 3. Apress, 317--339. https:\/\/doi.org\/10.1007\/978--1--4842--4859--1"},{"key":"e_1_3_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICTAI.2019.00253"},{"key":"e_1_3_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-68880-8_32"},{"key":"e_1_3_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1060745.1060754"}],"event":{"name":"CIKM '20: The 29th ACM International Conference on Information and Knowledge Management","location":"Virtual Event Ireland","acronym":"CIKM '20","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 29th ACM International Conference on Information & Knowledge Management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3340531.3412778","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3340531.3412778","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:56Z","timestamp":1750197776000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3340531.3412778"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,19]]},"references-count":40,"alternative-id":["10.1145\/3340531.3412778","10.1145\/3340531"],"URL":"https:\/\/doi.org\/10.1145\/3340531.3412778","relation":{},"subject":[],"published":{"date-parts":[[2020,10,19]]},"assertion":[{"value":"2020-10-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}