{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T18:23:52Z","timestamp":1771007032811,"version":"3.50.1"},"reference-count":89,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,2,23]],"date-time":"2023-02-23T00:00:00Z","timestamp":1677110400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Internet Technol."],"published-print":{"date-parts":[[2023,2,28]]},"abstract":"<jats:p>Collaborative Filtering (CF) recommendation algorithms are a popular solution to the information overload problem, aiding users in the item selection process. Relevant research has long focused on refining and improving these models to produce better (more effective) recommendations, and has converged on a methodology to predict their effectiveness on target datasets by evaluating them on random samples of the latter. However, predicting the efficiency of the solutions\u2014especially with regard to their time- and resource-hungry training phase, whose requirements dwarf those of the prediction\/recommendation phase\u2014has received little to no attention in the literature. This article addresses this gap for a number of representative and highly popular CF models, including algorithms based on matrix factorization, k-nearest neighbors, co-clustering, and slope one schemes. To this end, we first study the computational complexity of the training phase of said CF models and derive time and space complexity equations. Then, using characteristics of the input and the aforementioned equations, we contribute a methodology for predicting the processing time and memory usage of their training phase. Our contributions further include an adaptive sampling strategy, to address the tradeoff between resource usage costs and prediction accuracy, and a framework that quantifies both the efficiency and effectiveness of CF. Finally, a systematic experimental evaluation demonstrates that our method outperforms state-of-the-art regression schemes by a considerable margin, with an overhead that is a small fraction of the overall requirements of CF training.<\/jats:p>","DOI":"10.1145\/3554979","type":"journal-article","created":{"date-parts":[[2022,8,12]],"date-time":"2022-08-12T11:32:26Z","timestamp":1660303946000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["White Box: On the Prediction of Collaborative Filtering Recommendation Systems\u2019 Performance"],"prefix":"10.1145","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3780-0429","authenticated-orcid":false,"given":"Iulia","family":"Paun","sequence":"first","affiliation":[{"name":"School of Computing Science, University of Glasgow, Scotland, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4186-1088","authenticated-orcid":false,"given":"Yashar","family":"Moshfeghi","sequence":"additional","affiliation":[{"name":"NeuraSearch Laboratory, University of Strathclyde, Scotland, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8676-0948","authenticated-orcid":false,"given":"Nikos","family":"Ntarmos","sequence":"additional","affiliation":[{"name":"School of Computing Science, University of Glasgow, Scotland, UK"}]}],"member":"320","published-online":{"date-parts":[[2023,2,23]]},"reference":[{"issue":"1","key":"e_1_3_2_2_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2151163.2151166","article-title":"Impact of data characteristics on recommender systems performance","volume":"3","author":"Adomavicius Gediminas","year":"2012","unstructured":"Gediminas Adomavicius and Jingjing Zhang. 2012. Impact of data characteristics on recommender systems performance. ACM Transactions on Management Information Systems (TMIS) 3, 1 (2012), 1\u201317.","journal-title":"ACM Transactions on Management Information Systems (TMIS)"},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","unstructured":"Charu C. Aggarwal 2016. Recommender Systems. Vol. 1. Springer Berlin.","DOI":"10.1007\/978-3-319-29659-3_1"},{"key":"e_1_3_2_4_2","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511804090","volume-title":"Computational Complexity: A Modern Approach","author":"Arora Sanjeev","year":"2009","unstructured":"Sanjeev Arora and Boaz Barak. 2009. Computational Complexity: A Modern Approach. Cambridge University Press, Cambridge, UK."},{"issue":"3","key":"e_1_3_2_5_2","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1007\/BF00167586","article-title":"Sample size: How much is enough?","volume":"17","author":"Austin Homer W.","year":"1983","unstructured":"Homer W. Austin. 1983. Sample size: How much is enough? Quality and Quantity 17, 3 (1983), 239\u2013245.","journal-title":"Quality and Quantity"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1007\/978-1-4302-5990-9_4","volume-title":"Efficient Learning Machines","author":"Awad Mariette","year":"2015","unstructured":"Mariette Awad and Rahul Khanna. 2015. Support vector regression. In Efficient Learning Machines. Springer, Berlin, 67\u201380."},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","first-page":"1111","DOI":"10.1109\/CEC.2004.1330986","volume-title":"Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No. 04TH8753)","volume":"1","author":"Bartz-Beielstein Thomas","year":"2004","unstructured":"Thomas Bartz-Beielstein and Sandor Markon. 2004. Tuning search algorithms for real-world applications: A regression tree based approach. In Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No. 04TH8753), Vol. 1. IEEE, 1111\u20131118."},{"key":"e_1_3_2_8_2","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.knosys.2016.08.013","article-title":"Recommender systems for product bundling","volume":"111","author":"Beladev Moran","year":"2016","unstructured":"Moran Beladev, Lior Rokach, and Bracha Shapira. 2016. Recommender systems for product bundling. Knowledge-based Systems 111 (2016), 193\u2013206.","journal-title":"Knowledge-based Systems"},{"key":"e_1_3_2_9_2","first-page":"382","volume-title":"European Conference on Information Retrieval","author":"Bellog\u00edn Alejandro","year":"2010","unstructured":"Alejandro Bellog\u00edn and Pablo Castells. 2010. A performance prediction approach to enhance collaborative filtering performance. In European Conference on Information Retrieval. Springer, Berlin, 382\u2013393."},{"issue":"6","key":"e_1_3_2_10_2","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1007\/s10791-017-9312-z","article-title":"Statistical biases in information retrieval metrics for recommender systems","volume":"20","author":"Bellog\u00edn Alejandro","year":"2017","unstructured":"Alejandro Bellog\u00edn, Pablo Castells, and Iv\u00e1n Cantador. 2017. Statistical biases in information retrieval metrics for recommender systems. Information Retrieval Journal 20, 6 (2017), 606\u2013634.","journal-title":"Information Retrieval Journal"},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","DOI":"10.1137\/1.9781611971484","volume-title":"Numerical Methods for Least Squares Problems","author":"Bjorck Ake","year":"1996","unstructured":"Ake Bjorck. 1996. Numerical Methods for Least Squares Problems. Vol. 51. SIAM, Philadelphia, PA."},{"issue":"8","key":"e_1_3_2_12_2","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1145\/209937.209946","article-title":"High-level optimization via automated statistical modeling","volume":"30","author":"Brewer Eric A.","year":"1995","unstructured":"Eric A. Brewer. 1995. High-level optimization via automated statistical modeling. ACM SIGPLAN Notices 30, 8 (1995), 80\u201391.","journal-title":"ACM SIGPLAN Notices"},{"key":"e_1_3_2_13_2","doi-asserted-by":"crossref","DOI":"10.1201\/b10905","volume-title":"Handbook of Markov Chain Monte Carlo","author":"Brooks Steve","year":"2011","unstructured":"Steve Brooks, Andrew Gelman, Galin Jones, and Xiao-Li Meng. 2011. Handbook of Markov Chain Monte Carlo. CRC Press, Florida."},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","unstructured":"Dinh-Mao Bui YongIk Yoon Eui-Nam Huh SungIk Jun and Sungyoung Lee. 2017. Energy efficiency for cloud computing system based on predictive optimization. J. Parallel and Distrib. Comput. 102 (2017) 103\u2013114.","DOI":"10.1016\/j.jpdc.2016.11.011"},{"issue":"1","key":"e_1_3_2_15_2","doi-asserted-by":"crossref","first-page":"17","DOI":"10.3390\/a9010017","article-title":"Co-clustering under the maximum norm","volume":"9","author":"Bulteau Laurent","year":"2016","unstructured":"Laurent Bulteau, Vincent Froese, Sepp Hartung, and Rolf Niedermeier. 2016. Co-clustering under the maximum norm. Algorithms 9, 1 (2016), 17.","journal-title":"Algorithms"},{"issue":"1","key":"e_1_3_2_16_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1921591.1921593","article-title":"Comparison of collaborative filtering algorithms: Limitations of current techniques and proposals for scalable, high-performance recommender systems","volume":"5","author":"Cacheda Fidel","year":"2011","unstructured":"Fidel Cacheda, V\u00edctor Carneiro, Diego Fern\u00e1ndez, and Vreixo Formoso. 2011. Comparison of collaborative filtering algorithms: Limitations of current techniques and proposals for scalable, high-performance recommender systems. ACM Transactions on the Web (TWEB) 5, 1 (2011), 1\u201333.","journal-title":"ACM Transactions on the Web (TWEB)"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/1921591.1921593"},{"key":"e_1_3_2_18_2","first-page":"415","volume-title":"The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval","author":"Ca\u00f1amares Roc\u00edo","year":"2018","unstructured":"Roc\u00edo Ca\u00f1amares and Pablo Castells. 2018. Should I follow the crowd? A probabilistic analysis of the effectiveness of popularity in recommender systems. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM, 415\u2013424."},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1145\/3383313.3412259","volume-title":"14th ACM Conference on Recommender Systems","author":"Ca\u00f1amares Roc\u00edo","year":"2020","unstructured":"Roc\u00edo Ca\u00f1amares and Pablo Castells. 2020. On target item sampling in offline recommender system evaluation. In 14th ACM Conference on Recommender Systems. ACM, 259\u2013268."},{"issue":"4","key":"e_1_3_2_20_2","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1007\/s10791-020-09371-3","article-title":"Offline evaluation options for recommender systems","volume":"23","author":"Ca\u00f1amares Roc\u00edo","year":"2020","unstructured":"Roc\u00edo Ca\u00f1amares, Pablo Castells, and Alistair Moffat. 2020. Offline evaluation options for recommender systems. Information Retrieval Journal 23, 4 (2020), 387\u2013410.","journal-title":"Information Retrieval Journal"},{"key":"e_1_3_2_21_2","doi-asserted-by":"crossref","DOI":"10.1201\/b14884","volume-title":"Bayesian Methods for Data Analysis","author":"Carlin Bradley P.","year":"2008","unstructured":"Bradley P. Carlin and Thomas A. Louis. 2008. Bayesian Methods for Data Analysis. CRC Press, Florida."},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","unstructured":"Surajit Chaudhuri Rajeev Motwani and Vivek Narasayya. 1998. Random sampling for histogram construction: How much is enough? ACM SIGMOD Record 27 2 (1998) 436\u2013447.","DOI":"10.1145\/276305.276343"},{"key":"e_1_3_2_23_2","first-page":"45\u20131\u201345\u201313","volume-title":"Handbook of Linear Algebra","author":"Cline Alan Kaylor","year":"2007","unstructured":"Alan Kaylor Cline and Inderjit S. Dhillon. 2007. Computation of the singular value decomposition. In Handbook of Linear Algebra, Leslie Hogben (Ed.). Chapman & Hall\/CRC, Boca Raton, FL, Chapter 45, 45\u20131\u201345\u201313."},{"key":"e_1_3_2_24_2","volume-title":"Introduction to Algorithms (3rd ed.)","author":"Cormen Thomas H.","year":"2009","unstructured":"Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algorithms (3rd ed.). MIT Press, Cambridge, MA."},{"key":"e_1_3_2_25_2","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1145\/3397271.3401046","volume-title":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Deldjoo Yashar","year":"2020","unstructured":"Yashar Deldjoo, Tommaso Di Noia, Eugenio Di Sciascio, and Felice Antonio Merra. 2020. How dataset characteristics affect the robustness of collaborative recommendation models. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 951\u2013960."},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/978-0-387-85820-3_4","volume-title":"Recommender Systems Handbook","author":"Desrosiers Christian","year":"2011","unstructured":"Christian Desrosiers and George Karypis. 2011. A comprehensive survey of neighborhood-based recommendation methods. In Recommender Systems Handbook. Springer, Boston, MA, 107\u2013144."},{"key":"e_1_3_2_27_2","first-page":"1","volume-title":"Proceedings of the 1st Workshop on Machine Learning and Systems (EuroMLSys\u201921)","author":"Dice Dave","year":"2021","unstructured":"Dave Dice and Alex Kogan. 2021. Optimizing inference performance of transformers on CPUs. In Proceedings of the 1st Workshop on Machine Learning and Systems (EuroMLSys\u201921). ACM, 1\u20138."},{"key":"e_1_3_2_28_2","doi-asserted-by":"crossref","DOI":"10.1002\/9781118625590","volume-title":"Applied Regression Analysis","author":"Draper Norman R.","year":"1998","unstructured":"Norman R. Draper and Harry Smith. 1998. Applied Regression Analysis. Vol. 326. John Wiley & Sons, Hoboken, NJ."},{"key":"e_1_3_2_29_2","unstructured":"Eugene Fink. 1998. How to solve it automatically: Selection among problem solving methods. In Proceedings of the Fourth International Conference on Artificial Intelligence Planning Systems . Association for the Advancement of Artificial Intelligence (AAAI) Press 128\u2013136. Pittsburgh USA."},{"issue":"1","key":"e_1_3_2_30_2","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1006\/jcss.1997.1504","article-title":"A decision-theoretic generalization of on-line learning and an application to boosting","volume":"55","author":"Freund Yoav","year":"1997","unstructured":"Yoav Freund and Robert E. Schapire. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55, 1 (1997), 119\u2013139.","journal-title":"Journal of Computer and System Sciences"},{"key":"e_1_3_2_31_2","volume-title":"The Elements of Statistical Learning","author":"Friedman Jerome","year":"2001","unstructured":"Jerome Friedman, Trevor Hastie, and Robert Tibshirani. 2001. The Elements of Statistical Learning. Vol. 1 (10). Springer Series in Statistics, New York."},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","DOI":"10.1201\/b16018","volume-title":"Bayesian Data Analysis","author":"Gelman Andrew","year":"2013","unstructured":"Andrew Gelman, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin. 2013. In Bayesian Data Analysis. CRC Press, Florida."},{"key":"e_1_3_2_33_2","unstructured":"Thomas George and Srujana Merugu. 2005. A scalable collaborative filtering framework based on co-clustering. In Fifth IEEE International Conference on Data Mining (ICDM\u201905) Houston USA. IEEE 4\u2013pp."},{"issue":"3","key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1145\/581751.581753","article-title":"Fast incremental maintenance of approximate histograms","volume":"27","author":"Gibbons Phillip B.","year":"2002","unstructured":"Phillip B. Gibbons, Yossi Matias, and Viswanath Poosala. 2002. Fast incremental maintenance of approximate histograms. ACM Transactions on Database Systems (TODS) 27, 3 (2002), 261\u2013298.","journal-title":"ACM Transactions on Database Systems (TODS)"},{"issue":"121","key":"e_1_3_2_35_2","doi-asserted-by":"crossref","first-page":"124","DOI":"10.2307\/2223319","article-title":"Measurement of inequality of incomes","volume":"31","author":"Gini Corrado","year":"1921","unstructured":"Corrado Gini. 1921. Measurement of inequality of incomes. Economic Journal 31, 121 (1921), 124\u2013126.","journal-title":"Economic Journal"},{"key":"e_1_3_2_36_2","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1007\/978-1-4899-7637-6_8","volume-title":"Recommender Systems Handbook","author":"Gunawardana Asela","year":"2015","unstructured":"Asela Gunawardana and Guy Shani. 2015. Evaluating recommender systems. In Recommender Systems Handbook. Springer, Boston, MA, 265\u2013308."},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1145\/130283.130335","volume-title":"Proceedings of the 1992 ACM SIGMOD International Conference on Management of Data","author":"Haas Peter J.","year":"1992","unstructured":"Peter J. Haas and Arun N. Swami. 1992. Sequential sampling procedures for query size estimation. In Proceedings of the 1992 ACM SIGMOD International Conference on Management of Data. ACM, New York, NY, 341\u2013350."},{"issue":"4","key":"e_1_3_2_38_2","first-page":"1","article-title":"The Movielens datasets: History and context","volume":"5","author":"Harper F. Maxwell","year":"2015","unstructured":"F. Maxwell Harper and Joseph A. Konstan. 2015. The Movielens datasets: History and context. ACM Transactions on Interactive Intelligent Systems (TIIS) 5, 4 (2015), 1\u201319.","journal-title":"ACM Transactions on Interactive Intelligent Systems (TIIS)"},{"key":"e_1_3_2_39_2","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1145\/312624.312682","volume-title":"Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Herlocker Jonathan L.","year":"1999","unstructured":"Jonathan L. Herlocker, Joseph A. Konstan, Al Borchers, and John Riedl. 1999. An algorithmic framework for performing collaborative filtering. In Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 230\u2013237."},{"issue":"1","key":"e_1_3_2_40_2","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1145\/963770.963772","article-title":"Evaluating collaborative filtering recommender systems","volume":"22","author":"Herlocker Jonathan L.","year":"2004","unstructured":"Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen, and John T. Riedl. 2004. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems (TOIS) 22, 1 (2004), 5\u201353.","journal-title":"ACM Transactions on Information Systems (TOIS)"},{"issue":"2","key":"e_1_3_2_41_2","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1145\/119995.115837","article-title":"Error-constrained COUNT query evaluation in relational databases","volume":"20","author":"Hou Wen-Chi","year":"1991","unstructured":"Wen-Chi Hou, Gultekin Ozsoyoglu, and Erdogan Dogdu. 1991. Error-constrained COUNT query evaluation in relational databases. ACM SIGMOD Record 20, 2 (1991), 278\u2013287.","journal-title":"ACM SIGMOD Record"},{"key":"e_1_3_2_42_2","first-page":"62","volume-title":"European Conference on Planning","author":"Howe Adele E.","year":"1999","unstructured":"Adele E. Howe, Eric Dahlman, Christopher Hansen, Michael Scheetz, and Anneliese Von Mayrhauser. 1999. Exploiting competitive planner performance. In European Conference on Planning. Springer, Berlin, 62\u201372."},{"key":"e_1_3_2_43_2","first-page":"263","volume-title":"2008 8th IEEE International Conference on Data Mining","author":"Hu Yifan","year":"2008","unstructured":"Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In 2008 8th IEEE International Conference on Data Mining. IEEE, 263\u2013272."},{"issue":"9","key":"e_1_3_2_44_2","doi-asserted-by":"crossref","first-page":"1518","DOI":"10.1109\/TC.2017.2693186","article-title":"Energy-efficient resource utilization for heterogeneous embedded computing systems","volume":"66","author":"Huang Jing","year":"2017","unstructured":"Jing Huang, Renfa Li, Jiyao An, Derrick Ntalasha, Fan Yang, and Keqin Li. 2017. Energy-efficient resource utilization for heterogeneous embedded computing systems. IEEE Trans. Comput. 66, 9 (2017), 1518\u20131531.","journal-title":"IEEE Trans. Comput."},{"issue":"5","key":"e_1_3_2_45_2","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1109\/MIS.2007.4338497","article-title":"A comparison of collaborative-filtering recommendation algorithms for e-commerce","volume":"22","author":"Huang Zan","year":"2007","unstructured":"Zan Huang, Daniel Zeng, and Hsinchun Chen. 2007. A comparison of collaborative-filtering recommendation algorithms for e-commerce. IEEE Intelligent Systems 22, 5 (2007), 68\u201378.","journal-title":"IEEE Intelligent Systems"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.21105\/joss.02174"},{"issue":"1","key":"e_1_3_2_47_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1644873.1644874","article-title":"Factor in the neighbors: Scalable and accurate collaborative filtering","volume":"4","author":"Koren Yehuda","year":"2010","unstructured":"Yehuda Koren. 2010. Factor in the neighbors: Scalable and accurate collaborative filtering. ACM Transactions on Knowledge Discovery from Data (TKDD) 4, 1 (2010), 1\u201324.","journal-title":"ACM Transactions on Knowledge Discovery from Data (TKDD)"},{"key":"e_1_3_2_48_2","first-page":"147","volume-title":"International Workshop on Knowledge Discovery on the Web","author":"Lam Shyong K.","year":"2006","unstructured":"Shyong K. Lam, Adam LaPitz, George Karypis, John Riedl, et\u00a0al. 2006. Towards a scalable kNN CF algorithm: Exploring effective applications of clustering. In International Workshop on Knowledge Discovery on the Web. Springer, Berlin, 147\u2013166."},{"issue":"1","key":"e_1_3_2_49_2","first-page":"1","article-title":"A survey on advancing the DBMS query optimizer: Cardinality estimation, cost model, and plan enumeration","volume":"6","author":"Lan Hai","year":"2021","unstructured":"Hai Lan, Zhifeng Bao, and Yuwei Peng. 2021. A survey on advancing the DBMS query optimizer: Cardinality estimation, cost model, and plan enumeration. Data Science and Engineering 6, 1 (2021), 1\u201316.","journal-title":"Data Science and Engineering"},{"key":"e_1_3_2_50_2","first-page":"1","volume-title":"7th ICML Workshop on Automated Machine Learning (AutoML\u201920)","author":"LeDell Erin","year":"2020","unstructured":"Erin LeDell and Sebastien Poirier. 2020. H2O AutoML: Scalable automatic machine learning. In 7th ICML Workshop on Automated Machine Learning (AutoML\u201920). ICML, 1\u201316. https:\/\/www.automl.org\/wp-content\/uploads\/2020\/07\/AutoML_2020_paper_61.pdf"},{"issue":"9","key":"e_1_3_2_51_2","doi-asserted-by":"crossref","first-page":"1606","DOI":"10.14778\/3461535.3461549","article-title":"Towards cost-optimal query processing in the cloud","volume":"14","author":"Leis Viktor","year":"2021","unstructured":"Viktor Leis and Maximilian Kuschewski. 2021. Towards cost-optimal query processing in the cloud. Proceedings of the VLDB Endowment 14, 9 (2021), 1606\u20131612.","journal-title":"Proceedings of the VLDB Endowment"},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1137\/1.9781611972757.43","volume-title":"Proceedings of the 2005 SIAM International Conference on Data Mining","author":"Lemire Daniel","year":"2005","unstructured":"Daniel Lemire and Anna Maclachlan. 2005. Slope one predictors for online rating-based collaborative filtering. In Proceedings of the 2005 SIAM International Conference on Data Mining. SIAM, 471\u2013475."},{"key":"e_1_3_2_53_2","first-page":"556","volume-title":"International Conference on Principles and Practice of Constraint Programming","author":"Leyton-Brown Kevin","year":"2002","unstructured":"Kevin Leyton-Brown, Eugene Nudelman, and Yoav Shoham. 2002. Learning the empirical hardness of optimization problems: The case of combinatorial auctions. In International Conference on Principles and Practice of Constraint Programming. Springer, 556\u2013572."},{"issue":"11","key":"e_1_3_2_54_2","first-page":"1","article-title":"Robust estimation of resource consumption for SQL queries using statistical techniques","volume":"5","author":"Li Jiexing","year":"2012","unstructured":"Jiexing Li, Arnd Christian K\u00f6nig, Vivek Narasayya, and Surajit Chaudhuri. 2012. Robust estimation of resource consumption for SQL queries using statistical techniques. Proceedings of the VLDB Endowment 5, 11 (2012), 1\u201312.","journal-title":"Proceedings of the VLDB Endowment"},{"issue":"3","key":"e_1_3_2_55_2","doi-asserted-by":"crossref","first-page":"45","DOI":"10.2753\/MIS0742-1222230303","article-title":"Personalized content recommendation and user satisfaction: Theoretical synthesis and empirical findings","volume":"23","author":"Liang Ting-Peng","year":"2006","unstructured":"Ting-Peng Liang, Hung-Jen Lai, and Yi-Cheng Ku. 2006. Personalized content recommendation and user satisfaction: Theoretical synthesis and empirical findings. Journal of Management Information Systems 23, 3 (2006), 45\u201370.","journal-title":"Journal of Management Information Systems"},{"issue":"1","key":"e_1_3_2_56_2","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1006\/jcss.1995.1050","article-title":"Query size estimation by adaptive sampling","volume":"51","author":"Lipton Richard J.","year":"1995","unstructured":"Richard J. Lipton and Jeffrey F. Naughton. 1995. Query size estimation by adaptive sampling. J. Comput. System Sci. 51, 1 (1995), 18\u201325.","journal-title":"J. Comput. System Sci."},{"key":"e_1_3_2_57_2","doi-asserted-by":"crossref","first-page":"462","DOI":"10.1145\/3298689.3347041","volume-title":"Proceedings of the 13th ACM Conference on Recommender Systems","author":"Ludewig Malte","year":"2019","unstructured":"Malte Ludewig, Noemi Mauro, Sara Latifi, and Dietmar Jannach. 2019. Performance comparison of neural and non-neural approaches to session-based recommendation. In Proceedings of the 13th ACM Conference on Recommender Systems. ACM, 462\u2013466."},{"issue":"2","key":"e_1_3_2_58_2","doi-asserted-by":"crossref","first-page":"1273","DOI":"10.1109\/TII.2014.2308433","article-title":"An efficient non-negative matrix-factorization-based approach to collaborative filtering for recommender systems","volume":"10","author":"Luo Xin","year":"2014","unstructured":"Xin Luo, Mengchu Zhou, Yunni Xia, and Qingsheng Zhu. 2014. An efficient non-negative matrix-factorization-based approach to collaborative filtering for recommender systems. IEEE Transactions on Industrial Informatics 10, 2 (2014), 1273\u20131284.","journal-title":"IEEE Transactions on Industrial Informatics"},{"key":"e_1_3_2_59_2","doi-asserted-by":"crossref","first-page":"7776","DOI":"10.1109\/ACCESS.2017.2696365","article-title":"Machine learning with big data: Challenges and approaches","volume":"5","author":"L\u2019heureux Alexandra","year":"2017","unstructured":"Alexandra L\u2019heureux, Katarina Grolinger, Hany F. Elyamany, and Miriam A. M. Capretz. 2017. Machine learning with big data: Challenges and approaches. IEEE Access 5 (2017), 7776\u20137797.","journal-title":"IEEE Access"},{"issue":"2","key":"e_1_3_2_60_2","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1145\/304181.304204","article-title":"Random sampling techniques for space efficient online computation of order statistics of large datasets","volume":"28","author":"Manku Gurmeet Singh","year":"1999","unstructured":"Gurmeet Singh Manku, Sridhar Rajagopalan, and Bruce G. Lindsay. 1999. Random sampling techniques for space efficient online computation of order statistics of large datasets. ACM SIGMOD Record 28, 2 (1999), 251\u2013262.","journal-title":"ACM SIGMOD Record"},{"key":"e_1_3_2_61_2","doi-asserted-by":"crossref","DOI":"10.1002\/9781118594841","volume-title":"Statistical Intervals: A Guide for Practitioners and Researchers","author":"Meeker William Q.","year":"2017","unstructured":"William Q. Meeker, Gerald J. Hahn, and Luis A. Escobar. 2017. Statistical Intervals: A Guide for Practitioners and Researchers. Vol. 541. John Wiley & Sons, Hoboken, NJ."},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1017\/9781107185920.017"},{"key":"e_1_3_2_63_2","volume-title":"Machine Learning: A Probabilistic Perspective","author":"Murphy Kevin P.","year":"2012","unstructured":"Kevin P. Murphy. 2012. Machine Learning: A Probabilistic Perspective. MIT Press, London, UK."},{"key":"e_1_3_2_64_2","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1145\/3383313.3411452","volume-title":"Proceedings of the 14th ACM Conference on Recommender Systems (RecSys\u201920)","author":"Paun Iulia","year":"2020","unstructured":"Iulia Paun. 2020. Efficiency-effectiveness trade-offs in recommendation systems. In Proceedings of the 14th ACM Conference on Recommender Systems (RecSys\u201920). ACM, 770\u2013775."},{"key":"e_1_3_2_65_2","first-page":"1","volume-title":"Proceedings of the 1st Workshop on Machine Learning and Systems (EuroMLSys\u201921)","author":"Paun Iulia","year":"2021","unstructured":"Iulia Paun, Yashar Moshfeghi, and Nikos Ntarmos. 2021. Are we there yet? Estimating training time for recommendation systems. In Proceedings of the 1st Workshop on Machine Learning and Systems (EuroMLSys\u201921). ACM, 1\u20139."},{"key":"e_1_3_2_66_2","first-page":"1","volume-title":"Proceedings of the 3rd ACM Symposium on Cloud Computing","author":"Reiss Charles","year":"2012","unstructured":"Charles Reiss, Alexey Tumanov, Gregory R. Ganger, Randy H. Katz, and Michael A. Kozuch. 2012. Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In Proceedings of the 3rd ACM Symposium on Cloud Computing. ACM, 1\u201313."},{"key":"e_1_3_2_67_2","volume-title":"Recommender Systems Handbook (1st ed.)","author":"Ricci Francesco","year":"2010","unstructured":"Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor. 2010. Recommender Systems Handbook (1st ed.). Springer, Boston, MA."},{"key":"e_1_3_2_68_2","first-page":"46","volume-title":"International Workshop on Engineering Stochastic Local Search Algorithms","author":"Ridge Enda","year":"2007","unstructured":"Enda Ridge and Daniel Kudenko. 2007. Tuning the performance of the MMAS heuristic. In International Workshop on Engineering Stochastic Local Search Algorithms. Springer, Berlin, 46\u201360."},{"key":"e_1_3_2_69_2","first-page":"36","volume-title":"ICAPS 2007 Workshop AI Planning and Learning","author":"Roberts Mark","year":"2007","unstructured":"Mark Roberts, Adele Howe, and Landon Flom. 2007. Learned models of performance for many planners. In ICAPS 2007 Workshop AI Planning and Learning. ICAPS, 36\u201340."},{"issue":"1","key":"e_1_3_2_70_2","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1007\/s10844-018-0529-0","article-title":"Engagement in proactive recommendations","volume":"54","author":"Rook Laurens","year":"2020","unstructured":"Laurens Rook, Adem Sabic, and Markus Zanker. 2020. Engagement in proactive recommendations. Journal of Intelligent Information Systems 54, 1 (2020), 79\u2013100.","journal-title":"Journal of Intelligent Information Systems"},{"key":"e_1_3_2_71_2","article-title":"Stochastic gradient descent 0.23.0 documentation","year":"2020","unstructured":"Scikit-learn.org. 2020. Stochastic gradient descent 0.23.0 documentation. https:\/\/scikit-learn.org\/stable\/modules\/sgd.html #complexity.","journal-title":"https:\/\/scikit-learn.org\/stable\/modules\/sgd.html #complexity"},{"key":"e_1_3_2_72_2","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1016\/S0927-0507(03)10006-0","article-title":"Monte Carlo sampling methods","volume":"10","author":"Shapiro Alexander","year":"2003","unstructured":"Alexander Shapiro. 2003. Monte Carlo sampling methods. Handbooks in Operations Research and Management Science 10 (2003), 353\u2013425.","journal-title":"Handbooks in Operations Research and Management Science"},{"key":"e_1_3_2_73_2","first-page":"210","volume-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems","author":"Shardanand Upendra","year":"1995","unstructured":"Upendra Shardanand and Pattie Maes. 1995. Social information filtering: Algorithms for automating \u201cword of mouth.\u201d In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 210\u2013217."},{"issue":"2","key":"e_1_3_2_74_2","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1016\/j.neunet.2006.01.012","article-title":"Machine learning approaches for estimation of prediction interval for the model output","volume":"19","author":"Shrestha Durga L.","year":"2006","unstructured":"Durga L. Shrestha and Dimitri P. Solomatine. 2006. Machine learning approaches for estimation of prediction interval for the model output. Neural Networks 19, 2 (2006), 225\u2013235.","journal-title":"Neural Networks"},{"issue":"3","key":"e_1_3_2_75_2","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/MIC.2017.72","article-title":"Two decades of recommender systems at Amazon.com","volume":"21","author":"Smith Brent","year":"2017","unstructured":"Brent Smith and Greg Linden. 2017. Two decades of recommender systems at Amazon.com. IEEE Internet Computing 21, 3 (2017), 12\u201318.","journal-title":"IEEE Internet Computing"},{"key":"e_1_3_2_76_2","doi-asserted-by":"crossref","DOI":"10.1137\/1.9781611973228","volume-title":"Uncertainty Quantification: Theory, Implementation, and Applications","author":"Smith Ralph C.","year":"2013","unstructured":"Ralph C. Smith. 2013. Uncertainty Quantification: Theory, Implementation, and Applications. Vol. 12. SIAM, Philadelphia, PA."},{"key":"e_1_3_2_77_2","first-page":"1","article-title":"Energy and policy considerations for deep learning in NLP","volume":"1906","author":"Strubell Emma","year":"2019","unstructured":"Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. CoRR abs\/1906.02243 (2019), 1\u20136. arxiv:1906.02243http:\/\/arxiv.org\/abs\/1906.02243","journal-title":"CoRR"},{"key":"e_1_3_2_78_2","first-page":"949","volume-title":"2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS\u201918)","author":"Sun Xiaoyang","year":"2018","unstructured":"Xiaoyang Sun, Chunming Hu, Renyu Yang, Peter Garraghan, Tianyu Wo, Jie Xu, Jianyong Zhu, and Chao Li. 2018. Rose: Cluster resource scheduling via speculative over-subscription. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS\u201918). IEEE, 949\u2013960."},{"key":"e_1_3_2_79_2","first-page":"1826","volume-title":"2011 8th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD\u201911)","volume":"3","author":"Sun Zilei","year":"2011","unstructured":"Zilei Sun, Nianlong Luo, and Wei Kuang. 2011. One real-time personalized recommendation systems based on slope one algorithm. In 2011 8th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD\u201911), Vol. 3. IEEE, 1826\u20131830."},{"issue":"3","key":"e_1_3_2_80_2","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1198\/106186006X142681","article-title":"Monte Carlo integration with acceptance-rejection","volume":"15","author":"Tan Zhiqiang","year":"2006","unstructured":"Zhiqiang Tan. 2006. Monte Carlo integration with acceptance-rejection. Journal of Computational and Graphical Statistics 15, 3 (2006), 735\u2013752.","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"e_1_3_2_81_2","first-page":"2993","volume-title":"Conference on Learning Theory","author":"Tosh Christopher","year":"2019","unstructured":"Christopher Tosh and Sanjoy Dasgupta. 2019. The relative complexity of maximum likelihood estimation, map estimation, and sampling. In Conference on Learning Theory. PMLR, 2993\u20133035."},{"key":"e_1_3_2_82_2","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1007\/978-981-15-5309-7_8","volume-title":"Data Science and Security","author":"Vairale Vaishali S.","year":"2021","unstructured":"Vaishali S. Vairale and Samiksha Shukla. 2021. Recommendation of food items for thyroid patients using content-based KNN method. In Data Science and Security. Springer, Berlin, 71\u201377."},{"key":"e_1_3_2_83_2","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.1145\/3035918.3064029","volume-title":"Proceedings of the 2017 ACM International Conference on Management of Data","author":"Aken Dana Van","year":"2017","unstructured":"Dana Van Aken, Andrew Pavlo, Geoffrey J. Gordon, and Bohan Zhang. 2017. Automatic database management system tuning through large-scale machine learning. In Proceedings of the 2017 ACM International Conference on Management of Data. ACM, 1009\u20131024."},{"key":"e_1_3_2_84_2","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/j.procs.2014.05.382","article-title":"A CUDA-enabled parallel implementation of collaborative filtering","volume":"30","author":"Wang Zhongya","year":"2014","unstructured":"Zhongya Wang, Ying Liu, and Pengshan Ma. 2014. A CUDA-enabled parallel implementation of collaborative filtering. Procedia Computer Science 30 (2014), 66\u201374.","journal-title":"Procedia Computer Science"},{"key":"e_1_3_2_85_2","first-page":"2753","volume-title":"2013 46th Hawaii International Conference on System Sciences","author":"Wu Ling-Ling","year":"2013","unstructured":"Ling-Ling Wu, Yuh-Jzer Joung, and Jonglin Lee. 2013. Recommendation systems and consumer satisfaction online: Moderating effects of consumer product awareness. In 2013 46th Hawaii International Conference on System Sciences. IEEE, 2753\u20132762."},{"key":"e_1_3_2_86_2","first-page":"664","volume-title":"Proceedings of the 11th ACM International Conference on Web Search and Data Mining","author":"Yang Longqi","year":"2018","unstructured":"Longqi Yang, Eugene Bagdasaryan, Joshua Gruenstein, Cheng-Kang Hsieh, and Deborah Estrin. 2018. Openrec: A modular framework for extensible and adaptable recommendation algorithms. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. ACM, 664\u2013672."},{"issue":"4","key":"e_1_3_2_87_2","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1002\/bdm.2118","article-title":"Making sense of recommendations","volume":"32","author":"Yeomans Michael","year":"2019","unstructured":"Michael Yeomans, Anuj Shah, Sendhil Mullainathan, and Jon Kleinberg. 2019. Making sense of recommendations. Journal of Behavioral Decision Making 32, 4 (2019), 403\u2013414.","journal-title":"Journal of Behavioral Decision Making"},{"key":"e_1_3_2_88_2","first-page":"1","volume-title":"12th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud\u201920)","author":"Yeung Gingfung","year":"2020","unstructured":"Gingfung Yeung, Damian Borowiec, Adrian Friday, Richard Harper, and Peter Garraghan. 2020. Towards GPU utilization prediction for cloud deep learning. In 12th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud\u201920). USENIX, Online, 1\u20139."},{"key":"e_1_3_2_89_2","article-title":"Goodbooks-10k: a new dataset for book recommendations","author":"Zajac Zygmunt","year":"2017","unstructured":"Zygmunt Zajac. 2017. Goodbooks-10k: a new dataset for book recommendations. http:\/\/fastml.com\/goodbooks-10k.","journal-title":"http:\/\/fastml.com\/goodbooks-10k"},{"issue":"5","key":"e_1_3_2_90_2","doi-asserted-by":"crossref","first-page":"1165","DOI":"10.1007\/s13042-018-0795-8","article-title":"Efficient collaborative filtering recommendations with multi-channel feature vectors","volume":"10","author":"Zhang Heng-Ru","year":"2019","unstructured":"Heng-Ru Zhang, Fan Min, Zhi-Heng Zhang, and Song Wang. 2019. Efficient collaborative filtering recommendations with multi-channel feature vectors. International Journal of Machine Learning and Cybernetics 10, 5 (2019), 1165\u20131172.","journal-title":"International Journal of Machine Learning and Cybernetics"}],"container-title":["ACM Transactions on Internet Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3554979","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3554979","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T18:43:57Z","timestamp":1750272237000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3554979"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,23]]},"references-count":89,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,2,28]]}},"alternative-id":["10.1145\/3554979"],"URL":"https:\/\/doi.org\/10.1145\/3554979","relation":{},"ISSN":["1533-5399","1557-6051"],"issn-type":[{"value":"1533-5399","type":"print"},{"value":"1557-6051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,23]]},"assertion":[{"value":"2021-09-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-07-21","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}