{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T16:03:39Z","timestamp":1770998619731,"version":"3.50.1"},"reference-count":44,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2021,5,29]],"date-time":"2021-05-29T00:00:00Z","timestamp":1622246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2021,10,31]]},"abstract":"<jats:p>Recommender systems have been upgraded, tested, and applied in many, often incomparable ways. In attempts to diligently understand user behavior in certain environments, those systems have been frequently utilized in domains like e-commerce, e-learning, and tourism. Their increasing need and popularity have allowed the existence of numerous research paths on major issues like data sparsity, cold start, malicious noise, and natural noise, which immensely limit their performance. It is typical that the quality of the data that fuel those systems should be extremely reliable. Inconsistent user information in datasets can alter the performance of recommenders, albeit running advanced personalizing algorithms. The consequences of this can be costly as such systems are employed in abundant online businesses. Successfully managing these inconsistencies results in more personalized user experiences. In this article, the previous works conducted on natural noise management in recommender datasets are thoroughly analyzed. We adequately explore the ways in which the proposed methods measure improved performances and touch on the different natural noise management techniques and the attributes of the solutions. Additionally, we test the evaluation methods employed to assess the approaches and discuss several key gaps and other improvements the field should realize in the future. Our work considers the likelihood of a modern research branch on natural noise management and recommender assessment.<\/jats:p>","DOI":"10.1145\/3447780","type":"journal-article","created":{"date-parts":[[2021,5,30]],"date-time":"2021-05-30T01:59:11Z","timestamp":1622339951000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Critique on Natural Noise in Recommender Systems"],"prefix":"10.1145","volume":"15","author":[{"given":"Wissam Al","family":"Jurdi","sequence":"first","affiliation":[{"name":"Univ. Bourgogne Franche-Comt\u00e9, FEMTO-ST Institute and Lebanese University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jacques Bou","family":"Abdo","sequence":"additional","affiliation":[{"name":"University of Nebraska at Kearney, NE, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jacques","family":"Demerjian","sequence":"additional","affiliation":[{"name":"Lebanese University, Fanar, Lebanon"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Abdallah","family":"Makhoul","sequence":"additional","affiliation":[{"name":"Univ. Bourgogne Franche-Comt\u00e9, FEMTO-ST Institute"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,5,29]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-29659-3"},{"key":"e_1_2_1_2_1","volume-title":"Introduction to Recommender Systems Handbook","author":"Ricci Francesco","unstructured":"Francesco Ricci , Lior Rokach , and Bracha Shapira . 2011. Introduction to Recommender Systems Handbook . Springer US , Boston, MA , 1\u201335. DOI:http:\/\/dx.doi.org\/10.1007\/978-0-387-85820-3_1 Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011. Introduction to Recommender Systems Handbook. Springer US, Boston, MA, 1\u201335. DOI:http:\/\/dx.doi.org\/10.1007\/978-0-387-85820-3_1"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-012-9364-9"},{"key":"e_1_2_1_4_1","volume-title":"I Like It Not: Evaluating user ratings noise in recommender systems","author":"Amatriain Xavier","unstructured":"Xavier Amatriain , Josep M. Pujol , and Nuria Oliver . 2009. I Like It ... I Like It Not: Evaluating user ratings noise in recommender systems . In User Modeling, Adaptation, and Personalization. Geert-Jan Houben, Gord McCalla, Fabio Pianesi, and Massimo Zancanaro (Eds.). Springer , Berlin , 247\u2013258. Xavier Amatriain, Josep M. Pujol, and Nuria Oliver. 2009. I Like It... I Like It Not: Evaluating user ratings noise in recommender systems. In User Modeling, Adaptation, and Personalization. Geert-Jan Houben, Gord McCalla, Fabio Pianesi, and Massimo Zancanaro (Eds.). Springer, Berlin, 247\u2013258."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1639714.1639744"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1111449.1111477"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/963770.963772"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the Information and Knowledge Engineering.","author":"Jurdi Wissam Al","year":"2018","unstructured":"Wissam Al Jurdi , Miriam El Khoury Badran , Chady Abou Jaoude , Jacques Bou Abdo , Jacques Demerjian , and Abdallah Makhoul . 2018 . Serendipity-aware noise detection system for recommender systems . In Proceedings of the Information and Knowledge Engineering. Wissam Al Jurdi, Miriam El Khoury Badran, Chady Abou Jaoude, Jacques Bou Abdo, Jacques Demerjian, and Abdallah Makhoul. 2018. Serendipity-aware noise detection system for recommender systems. In Proceedings of the Information and Knowledge Engineering."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2014.12.011"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11280-012-0161-9"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/223904.223929"},{"key":"e_1_2_1_12_1","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/j.eswa.2017.10.060","article-title":"A fuzzy approach for natural noise management in group recommender systems","volume":"94","author":"Gallardo Jorge Castro","year":"2017","unstructured":"Jorge Castro Gallardo , Raciel Yera Toledo , and Luis Martinez . 2017 . A fuzzy approach for natural noise management in group recommender systems . Expert Systems with Applications 94 (2017), 237 - 249 . DOI:http:\/\/dx.doi.org\/10.1016\/j.eswa.2017.10.060 Jorge Castro Gallardo, Raciel Yera Toledo, and Luis Martinez. 2017. A fuzzy approach for natural noise management in group recommender systems. Expert Systems with Applications 94 (2017), 237-249. DOI:http:\/\/dx.doi.org\/10.1016\/j.eswa.2017.10.060","journal-title":"Expert Systems with Applications"},{"key":"e_1_2_1_13_1","volume-title":"Recommender systems survey. Knowledge-Based Systems 46 (Jul","author":"Bobadilla J.","year":"2013","unstructured":"J. Bobadilla , F. Ortega , A. Hernando , and A. Guti\u00e9Rrez . 2013. Recommender systems survey. Knowledge-Based Systems 46 (Jul . 2013 ), 109\u2013132. DOI:http:\/\/dx.doi.org\/10.1016\/j.knosys.2013.03.012 J. Bobadilla, F. Ortega, A. Hernando, and A. Guti\u00e9Rrez. 2013. Recommender systems survey. Knowledge-Based Systems 46 (Jul. 2013), 109\u2013132. DOI:http:\/\/dx.doi.org\/10.1016\/j.knosys.2013.03.012"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49001-4_1"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2019.01.001"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2365952.2365974"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2348283.2348469"},{"key":"e_1_2_1_18_1","volume-title":"de Vries","author":"Bellog\u00edn Alejandro","year":"2014","unstructured":"Alejandro Bellog\u00edn , Alan Said , and Arjen P . de Vries . 2014 . The magic barrier of recommender systems \u2013 no magic, just ratings. In User Modeling, Adaptation, and Personalization. Vania Dimitrova, Tsvi Kuflik, David Chin, Francesco Ricci, Peter Dolog, and Geert-Jan Houben (Eds.). Springer International Publishing , Cham, 25\u201336. Alejandro Bellog\u00edn, Alan Said, and Arjen P. de Vries. 2014. The magic barrier of recommender systems \u2013 no magic, just ratings. In User Modeling, Adaptation, and Personalization. Vania Dimitrova, Tsvi Kuflik, David Chin, Francesco Ricci, Peter Dolog, and Geert-Jan Houben (Eds.). Springer International Publishing, Cham, 25\u201336."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-31454-4_20"},{"key":"e_1_2_1_20_1","volume-title":"Coherence and inconsistencies in rating behavior: Estimating the magic barrier of recommender systems. User Modeling and User-Adapted Interaction 28, 2","author":"Said Alan","year":"2018","unstructured":"Alan Said and Alejandro Bellog\u00edn . 2018. Coherence and inconsistencies in rating behavior: Estimating the magic barrier of recommender systems. User Modeling and User-Adapted Interaction 28, 2 ( 2018 ). DOI:http:\/\/dx.doi.org\/10.1007\/s11257-018-9202-0 Alan Said and Alejandro Bellog\u00edn. 2018. Coherence and inconsistencies in rating behavior: Estimating the magic barrier of recommender systems. User Modeling and User-Adapted Interaction 28, 2 (2018). DOI:http:\/\/dx.doi.org\/10.1007\/s11257-018-9202-0"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-39937-9_18"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10844-016-0406-7"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5220\/0005041401540161"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/IFSA-NAFIPS.2013.6608515"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2016.09.020"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-013-1563-0"},{"key":"e_1_2_1_27_1","volume-title":"C (Mar.","author":"Yera Raciel","year":"2016","unstructured":"Raciel Yera , Jorge Castro , and Luis Mart\u00ednez . 2016. A fuzzy model for managing natural noise in recommender systems. Applied Soft Computing 40 , C (Mar. 2016 ), 187\u2013198. DOI:http:\/\/dx.doi.org\/10.1016\/j.asoc.2015.10.060 Raciel Yera, Jorge Castro, and Luis Mart\u00ednez. 2016. A fuzzy model for managing natural noise in recommender systems. Applied Soft Computing 40, C (Mar. 2016), 187\u2013198. DOI:http:\/\/dx.doi.org\/10.1016\/j.asoc.2015.10.060"},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 8th International Conference on Multimedia Communications, Services and Security. 46\u201358","author":"Latha R.","unstructured":"R. Latha and R. Nadarajan . 2015. Ranking based approach for noise handling in recommender systems . In Proceedings of the 8th International Conference on Multimedia Communications, Services and Security. 46\u201358 . DOI:http:\/\/dx.doi.org\/10.1007\/978-3-319-26404-2_4 R. Latha and R. Nadarajan. 2015. Ranking based approach for noise handling in recommender systems. In Proceedings of the 8th International Conference on Multimedia Communications, Services and Security. 46\u201358. DOI:http:\/\/dx.doi.org\/10.1007\/978-3-319-26404-2_4"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.2991\/ijcis.2017.10.1.52"},{"key":"e_1_2_1_30_1","volume-title":"C (Nov.","author":"Choudhary Priyankar","year":"2017","unstructured":"Priyankar Choudhary , Vibhor Kant , and Pragya Dwivedi . 2017. Handling natural noise in multi criteria recommender system utilizing effective similarity measure and particle swarm optimization. Procedia Computer Science 115 , C (Nov. 2017 ), 853\u2013862. DOI:http:\/\/dx.doi.org\/10.1016\/j.procs.2017.09.168 Priyankar Choudhary, Vibhor Kant, and Pragya Dwivedi. 2017. Handling natural noise in multi criteria recommender system utilizing effective similarity measure and particle swarm optimization. Procedia Computer Science 115, C (Nov. 2017), 853\u2013862. DOI:http:\/\/dx.doi.org\/10.1016\/j.procs.2017.09.168"},{"key":"e_1_2_1_31_1","volume-title":"Recommender Systems Handbook","author":"Ricci Francesco","unstructured":"Francesco Ricci , Lior Rokach , and Bracha Shapira . 2010. Recommender Systems Handbook . Vol. 1- 35 . Springer , 1\u201335. DOI:http:\/\/dx.doi.org\/10.1007\/978-0-387-85820-3_1 Francesco Ricci, Lior Rokach, and Bracha Shapira. 2010. Recommender Systems Handbook. Vol. 1-35. Springer, 1\u201335. DOI:http:\/\/dx.doi.org\/10.1007\/978-0-387-85820-3_1"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/963770.963775"},{"key":"e_1_2_1_33_1","volume-title":"C (Jul.","author":"Patra Bidyut Kr.","year":"2015","unstructured":"Bidyut Kr. Patra , Raimo Launonen , Ville Ollikainen , and Sukumar Nandi . 2015. A new similarity measure using Bhattacharyya coefficient for collaborative filtering in sparse data. Knowledge-Based Systems 82 , C (Jul. 2015 ), 163\u2013177. DOI:http:\/\/dx.doi.org\/10.1016\/j.knosys.2015.03.001 Bidyut Kr. Patra, Raimo Launonen, Ville Ollikainen, and Sukumar Nandi. 2015. A new similarity measure using Bhattacharyya coefficient for collaborative filtering in sparse data. Knowledge-Based Systems 82, C (Jul. 2015), 163\u2013177. DOI:http:\/\/dx.doi.org\/10.1016\/j.knosys.2015.03.001"},{"key":"e_1_2_1_34_1","volume-title":"Jorge Castro Gallardo, and Luis Martinez","author":"Toledo Raciel Yera","year":"2020","unstructured":"Raciel Yera Toledo , Jorge Castro Gallardo, and Luis Martinez . 2020 . Natural Noise Management in Recommender Systems Using Fuzzy Tools. Springer , 1\u201324. DOI:http:\/\/dx.doi.org\/10.1007\/978-3-030-23760-8_1 Raciel Yera Toledo, Jorge Castro Gallardo, and Luis Martinez. 2020. Natural Noise Management in Recommender Systems Using Fuzzy Tools. Springer, 1\u201324. DOI:http:\/\/dx.doi.org\/10.1007\/978-3-030-23760-8_1"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-012-1119-8"},{"key":"e_1_2_1_36_1","volume-title":"Integrating multiple experts for correction process in interactive recommendation systems","author":"Pham Xuan Hau","unstructured":"Xuan Hau Pham , Jason J. Jung , and Ngoc-Thanh Nguyen . 2012. Integrating multiple experts for correction process in interactive recommendation systems . In Computational Collective Intelligence. Technologies and Applications. Ngoc-Thanh Nguyen, Kiem Hoang, and Piotr Jedrzejowicz (Eds.). Springer , Berlin , 31\u201340. Xuan Hau Pham, Jason J. Jung, and Ngoc-Thanh Nguyen. 2012. Integrating multiple experts for correction process in interactive recommendation systems. In Computational Collective Intelligence. Technologies and Applications. Ngoc-Thanh Nguyen, Kiem Hoang, and Piotr Jedrzejowicz (Eds.). Springer, Berlin, 31\u201340."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0130968"},{"key":"e_1_2_1_38_1","volume-title":"Retrieved","author":"Beel Joeran","year":"2019","unstructured":"Joeran Beel . 2019 . \u201c And the winner is... Movielens\u201d \u2014 On the popularity of recommender-system datasets . Retrieved July 19, 2020 from https:\/\/www.scss.tcd.ie\/joeran.beel\/blog\/2019\/08\/03\/and-the-winner-is-movielens-on-the-popularity-of-recommender-system-datasets\/ Joeran Beel. 2019. \u201cAnd the winner is... Movielens\u201d \u2014 On the popularity of recommender-system datasets. Retrieved July 19, 2020 from https:\/\/www.scss.tcd.ie\/joeran.beel\/blog\/2019\/08\/03\/and-the-winner-is-movielens-on-the-popularity-of-recommender-system-datasets\/"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/MENACOMM.2018.8371003"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-017-4481-8"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.5220\/0007409507390745"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611975673.84"},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of the 2015 International Conference on Signal Processing and Communication Engineering Systems. DOI:http:\/\/dx.doi.org\/10","author":"Kishan Kalitkar S.K.","year":"2015","unstructured":"S.K. , Kishan Kalitkar , G. Subba , and Rao. 2015 . A review on noise reduction methods for brain MRI images . In Proceedings of the 2015 International Conference on Signal Processing and Communication Engineering Systems. DOI:http:\/\/dx.doi.org\/10 .1109\/SPACES.2015.7058284 S.K., Kishan Kalitkar, G. Subba, and Rao. 2015. A review on noise reduction methods for brain MRI images. In Proceedings of the 2015 International Conference on Signal Processing and Communication Engineering Systems. DOI:http:\/\/dx.doi.org\/10.1109\/SPACES.2015.7058284"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2013.07.007"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447780","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447780","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:41:10Z","timestamp":1750200070000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447780"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,29]]},"references-count":44,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2021,10,31]]}},"alternative-id":["10.1145\/3447780"],"URL":"https:\/\/doi.org\/10.1145\/3447780","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,29]]},"assertion":[{"value":"2020-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-05-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}