{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T08:39:43Z","timestamp":1773391183743,"version":"3.50.1"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"3","funder":[{"DOI":"10.13039\/501100001824","name":"Czech Science Foundation","doi-asserted-by":"crossref","award":["25-16785S"],"award-info":[{"award-number":["25-16785S"]}],"id":[{"id":"10.13039\/501100001824","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100007397","name":"Charles University","doi-asserted-by":"crossref","award":["SVV-260698\/2023"],"award-info":[{"award-number":["SVV-260698\/2023"]}],"id":[{"id":"10.13039\/100007397","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100007543","name":"Charles University Grant Agency","doi-asserted-by":"crossref","award":["188322"],"award-info":[{"award-number":["188322"]}],"id":[{"id":"10.13039\/100007543","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Recomm. Syst."],"published-print":{"date-parts":[[2026,9,30]]},"abstract":"<jats:p>\n                    Recommender systems (RS) rely on interaction data between users and items to generate effective results. Originally, RS aimed solely at predicting items\u2019 relevance, but additional (beyond-relevance) quality criteria gained increased attention over time. Objectives such as diversity, novelty, fairness, or serendipity are nowadays at the center of RS research and also among the core components in production systems. Naturally, to properly steer toward such objectives, the system has to gain an understanding of how the users perceive these objectives, to what extent they require them in the recommendations, and how they evaluate the sufficiency of the results w.r.t. these objectives. However, so far, there is no publicly available dataset that would capture all the necessary knowledge. This results in a\n                    <jats:italic toggle=\"yes\">half-blind<\/jats:italic>\n                    algorithmic design and evaluation, where the importance of individual objectives or the metrics for their evaluation cannot be validated from the users\u2019 perspective.\n                  <\/jats:p>\n                  <jats:p>\n                    To address this issue, we present SM-RS 2.0, an expansion of the original single- and multi-objective recommendations dataset. The dataset links the self-declared propensity toward individual objectives with impressions, item selections, and explicit evaluation of individual quality criteria. Together with the dataset, we also distribute an evaluation framework containing six rather unique tasks that are rarely available to conduct on existing RS datasets. These include impression-aware click prediction, predicting propensity toward individual objectives, construction of proportional recommendations, and predicting the user-perceived fulfillment of individual objectives as well as their overall satisfaction. The dataset is available at\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/osf.io\/wsakx\">https:\/\/osf.io\/wsakx<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1145\/3754459","type":"journal-article","created":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T11:37:22Z","timestamp":1753357042000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["SM-RS 2.0: User-perceived Qualities of Single- and Multi-Objective Recommender Systems"],"prefix":"10.1145","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1423-628X","authenticated-orcid":false,"given":"Patrik","family":"Dokoupil","sequence":"first","affiliation":[{"name":"Faculty of Mathematics and Physics, Dept. of Software Engineering, Charles University","place":["Prague, Czech Republic"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8082-4509","authenticated-orcid":false,"given":"Ladislav","family":"Peska","sequence":"additional","affiliation":[{"name":"Faculty of Mathematics and Physics, Dept. of Software Engineering, Charles University","place":["Prague, Czech Republic"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,3,2]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3450613.3456821"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/2559952"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3627043.3659558"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209219.3209251"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102363"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273513"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291025"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-7637-6_26"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3564284"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313469"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/2645710.2645774"},{"key":"e_1_3_2_13_2","first-page":"15","volume-title":"Proceedings of the 11th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems","author":"Dokoupil Patrik","year":"2024","unstructured":"Patrik Dokoupil, Ludovico Boratto, and Ladislav Peska. 2024. Comparing user interfaces for customizing multi-objective recommender systems. In Proceedings of the 11th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems (Bari, Italy). CEUR-WS.org, Aachen, Germany, 15\u201328. Retrieved from https:\/\/ceur-ws.org\/Vol-3815\/paper2.pdf"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3627043.3659555"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3604915.3610640"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3604915.3608848"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3626772.3657863"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","unstructured":"Simen Eide David S. Leslie and Arnoldo Frigessi. 2022. Dynamic slate recommendation with gated recurrent units and Thompson sampling. Data Min. Knowl. Discov. 36 5 (Sept. 2022) 1756\u20131786. 10.1007\/s10618-022-00849-w","DOI":"10.1007\/s10618-022-00849-w"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460231.3474607"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3412778"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2645710.2645737"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/1276958.1277117"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3346967"},{"key":"e_1_3_2_24_2","unstructured":"Kilol Gupta Mukund Yelahanka Raghuprasad and Pankhuri Kumar. 2018. A Hybrid Variational Autoencoder for Collaborative Filtering. arxiv:1808.01006. Retrieved from https:\/\/arxiv.org\/abs\/1808.01006"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/2827872"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1020443909834"},{"key":"e_1_3_2_27_2","volume-title":"4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings.","author":"Hidasi Bal\u00e1zs","year":"2016","unstructured":"Bal\u00e1zs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings.Yoshua Bengio and Yann LeCun (Eds.), arXiv:1511.06939. Retrieved from http:\/\/arxiv.org\/abs\/1511.06939"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/2245276.2232109"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.22"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.22"},{"key":"e_1_3_2_31_2","volume-title":"Algorithmic vs. Perceived Fairness in Music Recommender Systems: An Investigation of Popularity Bias from a User Perspective","author":"Ingesson Eveline","year":"2022","unstructured":"Eveline Ingesson. 2022. Algorithmic vs. Perceived Fairness in Music Recommender Systems: An Investigation of Popularity Bias from a User Perspective. Ph. D. Dissertation. Jonkoping University. Retrieved from http:\/\/hj.diva-portal.org\/smash\/record.jsf?pid=diva2%3A1665181&dswid=6212."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.3389\/fdata.2023.1157899"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-022-09351-w"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2017.03.055"},{"key":"e_1_3_2_35_2","volume-title":"Proceedings of the Workshop on Recommender Systems Evaluation: Dimensions and Design (Workshop Programme of the 8th ACM Conference on Recommender Systems)","author":"Kaminskas Marius","year":"2014","unstructured":"Marius Kaminskas and Derek Bridge. 2014. Measuring surprise in recommender systems. In Proceedings of the Workshop on Recommender Systems Evaluation: Dimensions and Design (Workshop Programme of the 8th ACM Conference on Recommender Systems). Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:14983894"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2022.3181164"},{"key":"e_1_3_2_37_2","unstructured":"Kihwan Kim. 2021. An Empirical Analysis on Transparent Algorithmic Exploration in Recommender Systems. arxiv:2108.00151. Retrieved from https:\/\/arxiv.org\/abs\/2108.00151"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/2365952.2365966"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463019"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3132926"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3626772.3657802"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3627673.3679629"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186150"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","unstructured":"Fernando Benjam\u00edn P\u00e9rez Maurera Maurizio Ferrari Dacrema Pablo Castells and Paolo Cremonesi. 2025. Impression-aware recommender systems. ACM Trans. Recomm. Syst. 3 4 Article 48 (April 2025) 46 pages. 10.1145\/3712292","DOI":"10.1145\/3712292"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3523227.3551483"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3412774"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531787"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3512956"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3109859.3109896"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3539597.3570380"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159700"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3488560.3498471"},{"key":"e_1_3_2_53_2","volume-title":"Proceedings of the 9th International Workshop on News Recommendation and Analytics, September 25, 2021, Amsterdam, Netherlands","author":"Starke A.D.","year":"2021","unstructured":"A.D. Starke, Sebastian Larsen, and Christoph Trattner. 2021. Predicting feature-based similarity in the news domain using human judgments. In Proceedings of the 9th International Workshop on News Recommendation and Analytics, September 25, 2021, Amsterdam, Netherlands. CEUR-WS."},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240372"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313710"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3616855.3635833"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-019-09245-4"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-020-09281-5"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/2645710.2645743"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/2043932.2043955"},{"key":"e_1_3_2_61_2","unstructured":"Max Welz and Andreas Alfons. 2025. When Respondents Don\u2019t Care Anymore: Identifying the Onset of Careless Responding. arxiv:2303.07167. Retrieved from https:\/\/arxiv.org\/abs\/2303.07167"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-016-9178-6"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.331"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3109859.3109887"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240351"},{"key":"e_1_3_2_66_2","volume-title":"Goodbooks-10k: A New Dataset for Book Recommendations","author":"Zajac Zygmunt","year":"2017","unstructured":"Zygmunt Zajac. 2017. Goodbooks-10k: A New Dataset for Book Recommendations. Retrieved August 5, 2025 from https:\/\/fastml.com\/goodbooks-10k-a-new-dataset-for-book-recommendations\/"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.5334\/tismir.37"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3218994"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/TETCI.2023.3251352"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/1060745.1060754"}],"container-title":["ACM Transactions on Recommender Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3754459","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T06:18:13Z","timestamp":1773382693000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3754459"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,2]]},"references-count":69,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,9,30]]}},"alternative-id":["10.1145\/3754459"],"URL":"https:\/\/doi.org\/10.1145\/3754459","relation":{},"ISSN":["2770-6699"],"issn-type":[{"value":"2770-6699","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,2]]},"assertion":[{"value":"2024-12-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-07","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}