{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T05:22:57Z","timestamp":1772083377303,"version":"3.50.1"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2012,11,1]],"date-time":"2012-11-01T00:00:00Z","timestamp":1351728000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000145","name":"Division of Information and Intelligent Systems","doi-asserted-by":"publisher","award":["IIS-0546443"],"award-info":[{"award-number":["IIS-0546443"]}],"id":[{"id":"10.13039\/100000145","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2012,11]]},"abstract":"<jats:p>The article explores stability as a new measure of recommender systems performance. Stability is defined to measure the extent to which a recommendation algorithm provides predictions that are consistent with each other. Specifically, for a stable algorithm, adding some of the algorithm\u2019s own predictions to the algorithm\u2019s training data (for example, if these predictions were confirmed as accurate by users) would not invalidate or change the other predictions. While stability is an interesting theoretical property that can provide additional understanding about recommendation algorithms, we believe stability to be a desired practical property for recommender systems designers as well, because unstable recommendations can potentially decrease users\u2019 trust in recommender systems and, as a result, reduce users\u2019 acceptance of recommendations. In this article, we also provide an extensive empirical evaluation of stability for six popular recommendation algorithms on four real-world datasets. Our results suggest that stability performance of individual recommendation algorithms is consistent across a variety of datasets and settings. In particular, we find that model-based recommendation algorithms consistently demonstrate higher stability than neighborhood-based collaborative filtering techniques. In addition, we perform a comprehensive empirical analysis of many important factors (e.g., the sparsity of original rating data, normalization of input data, the number of new incoming ratings, the distribution of incoming ratings, the distribution of evaluation data, etc.) and report the impact they have on recommendation stability.<\/jats:p>","DOI":"10.1145\/2382438.2382442","type":"journal-article","created":{"date-parts":[[2012,11,29]],"date-time":"2012-11-29T15:02:27Z","timestamp":1354201347000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":61,"title":["Stability of Recommendation Algorithms"],"prefix":"10.1145","volume":"30","author":[{"given":"Gediminas","family":"Adomavicius","sequence":"first","affiliation":[{"name":"University of Minnesota"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingjing","family":"Zhang","sequence":"additional","affiliation":[{"name":"Indiana University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2012,11]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2005.99"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1864708.1864722"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/245108.245124"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 7--14","author":"Bell R. M."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1345448.1345465"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2007.90"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the KDD-Cup and Workshop. www.netflixprize.com.","author":"Bennett J."},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 15th International Conference on Machine Learning (ICML\u201998)","author":"Billsus D."},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence.","author":"Breese J. S."},{"key":"e_1_2_1_10_1","unstructured":"Burden R. L. and Faires D. 2004. Numerical Analysis. Thomson Brooks\/Cole. Burden R. L. and Faires D. 2004. Numerical Analysis . Thomson Brooks\/Cole."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1520-6793(199912)16:8<677::AID-MAR4>3.0.CO;2-T"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1454008.1454054"},{"key":"e_1_2_1_13_1","unstructured":"Duda R. O. Hart P. E. and Stork D. G. 2000. Pattern Classification. Wiley. Duda R. O. Hart P. E. and Stork D. G. 2000. Pattern Classification . Wiley."},{"key":"e_1_2_1_14_1","unstructured":"Funk S. 2006. Netflix update: Try this at home. http:\/\/sifter.org\/~simon\/journal\/20061211.html. Funk S. 2006. Netflix update: Try this at home. http:\/\/sifter.org\/~simon\/journal\/20061211.html."},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Garfinkel R. Gopal R. Pathak B. Venkatesan R. and Yin F. 2006. Empirical analysis of the business value of recommender systems. http:\/\/ssrn.com\/abstract=958770. Garfinkel R. Gopal R. Pathak B. Venkatesan R. and Yin F. 2006. Empirical analysis of the business value of recommender systems. http:\/\/ssrn.com\/abstract=958770.","DOI":"10.2139\/ssrn.958770"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1207\/153276603768344870"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1011419012209"},{"key":"e_1_2_1_18_1","unstructured":"Grouplens. 2006. Movielens Data Sets. http:\/\/www.grouplens.org\/. Grouplens. 2006. Movielens Data Sets. http:\/\/www.grouplens.org\/."},{"key":"e_1_2_1_19_1","doi-asserted-by":"crossref","unstructured":"Hastie T. Tibshirani R. and Friedman J. 2009. The Elements of Statistical Learning: Data Mining Inference and Prediction. Springer. Hastie T. Tibshirani R. and Friedman J. 2009. The Elements of Statistical Learning: Data Mining Inference and Prediction. Springer.","DOI":"10.1007\/978-0-387-84858-7"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/312624.312682"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/963770.963772"},{"key":"e_1_2_1_22_1","doi-asserted-by":"crossref","volume-title":"Accuracy and Stability of Numerical Algorithms","author":"Higham N. J.","DOI":"10.1137\/1.9780898718027"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/2017317.2017325"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1644873.1644874"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2009.263"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/245108.245126"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/988672.988726"},{"key":"e_1_2_1_28_1","unstructured":"Legwinski T. 2010. Recommendation explanations increase sales & customer confidence. http:\/\/blog.strands.com\/2010\/03\/17\/recommendations-increase-sales-customer-confidence\/. Legwinski T. 2010. Recommendation explanations increase sales & customer confidence. http:\/\/blog.strands.com\/2010\/03\/17\/recommendations-increase-sales-customer-confidence\/."},{"key":"e_1_2_1_29_1","volume-title":"The General Problem of the Stability of Motion","author":"Lyapunov A. M."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1297231.1297235"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the International Conference on Trust Management. Springer, 221--235","author":"Massa P."},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of the 21st Conference on Artificial Intelligence (AAAI\u201906)","author":"Mobasher B."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/11891321_6"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1278366.1278372"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1040830.1040870"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0218213006003053"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:AIRE.0000036256.39422.25"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/192844.192905"},{"key":"e_1_2_1_39_1","unstructured":"Rigby C. 2011. Internet Retailing webinars. Review: Boosting email marketing revenue with personalised recommendations by Silverpop Baynote and MusicRoom.com. http:\/\/www.internetretailing.net\/2011\/10\/internet-retailing-webinars-review-boosting-email-marketing-revenue-with-personalised-recommendations-by-silverpop-baynote-and-musicroom-com\/. Rigby C. 2011. Internet Retailing webinars. Review: Boosting email marketing revenue with personalised recommendations by Silverpop Baynote and MusicRoom.com. http:\/\/www.internetretailing.net\/2011\/10\/internet-retailing-webinars-review-boosting-email-marketing-revenue-with-personalised-recommendations-by-silverpop-baynote-and-musicroom-com\/."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/289444.289509"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/371920.372071"},{"key":"e_1_2_1_42_1","unstructured":"Shani G. and Gunawardana A. 2011. Evaluating recommender systems. In Recommender Systems Handbook: A Complete Guide for Research Scientists and Practitioners P. Kantor F. Ricci L. Rokach and B. Shapira Eds. Springer. Shani G. and Gunawardana A. 2011. Evaluating recommender systems. In Recommender Systems Handbook: A Complete Guide for Research Scientists and Practitioners P. Kantor F. Ricci L. Rokach and B. Shapira Eds. Springer."},{"key":"e_1_2_1_43_1","unstructured":"Simitses G. and Hodges D. 2005. Fundamentals of Structural Stability. Butterworth-Heinemann Burlington MA. Simitses G. and Hodges D. 2005. Fundamentals of Structural Stability . Butterworth-Heinemann Burlington MA."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1080\/09528139408953794"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00993473"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1348\/014466604X17092"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.17705\/1jais.00065"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2382438.2382442","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2382438.2382442","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T09:34:49Z","timestamp":1750239289000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2382438.2382442"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,11]]},"references-count":47,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2012,11]]}},"alternative-id":["10.1145\/2382438.2382442"],"URL":"https:\/\/doi.org\/10.1145\/2382438.2382442","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,11]]},"assertion":[{"value":"2011-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-11-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}