{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T01:43:24Z","timestamp":1769651004842,"version":"3.49.0"},"reference-count":72,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"name":"European Union\u2019s Horizon 2020 Research and Innovation programme","award":["856632"],"award-info":[{"award-number":["856632"]}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"crossref","award":["EP\/X525716\/1"],"award-info":[{"award-number":["EP\/X525716\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2026,2,28]]},"abstract":"<jats:p>The use of recommender systems to assist in the provision of financial asset and portfolio recommendations to investors is increasing, spanning a wide range of algorithms and techniques. Several strategies have been devised for the evaluation of financial asset recommendations, with the two most prominent perspectives measuring, respectively, (a) the money customers could obtain if they followed the recommendations (profitability-based evaluation) and (b) the ability of models to predict future customer investments (transaction-based evaluation). If customers are effective investors, we would expect these two perspectives to be positively correlated. In this article, we explore the actual relationship between these two families of metrics. Theoretically, we prove that these perspectives are independent. Furthermore, we perform experiments over a large-scale financial recommendation dataset with real customer investment transactions. Surprisingly, we find that transaction and profitability-based metrics are, in fact, negatively correlated. Moreover, algorithms that actively learn from past customer transactions might lose money in the mid-term. A thorough analysis of model performance and customer transaction patterns over time shows that this is due to customers failing to consistently beat the market with their investments, with time appearing as an important confounding variable\u2014since the point of time where recommendations are provided and the investment horizon largely affect the customer\u2019s investment performance.<\/jats:p>","DOI":"10.1145\/3780097","type":"journal-article","created":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T14:38:27Z","timestamp":1765377507000},"page":"1-56","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Investors Are (Not) Always Right: A Comparison of Transaction-Based and Profitability-Based Metrics for Financial Asset Recommendations"],"prefix":"10.1145","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7829-5174","authenticated-orcid":false,"given":"Javier","family":"Sanz-Cruzado","sequence":"first","affiliation":[{"name":"University of Glasgow, Glasgow, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2751-2087","authenticated-orcid":false,"given":"Richard","family":"McCreadie","sequence":"additional","affiliation":[{"name":"University of Glasgow, Glasgow, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-4986-0501","authenticated-orcid":false,"given":"Nikolaos","family":"Droukas","sequence":"additional","affiliation":[{"name":"National Bank of Greece, Athens, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3143-279X","authenticated-orcid":false,"given":"Craig","family":"Macdonald","sequence":"additional","affiliation":[{"name":"University of Glasgow, Glasgow, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4701-3223","authenticated-orcid":false,"given":"Iadh","family":"Ounis","sequence":"additional","affiliation":[{"name":"University of Glasgow, Glasgow, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,1,28]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.5555\/645920.672836"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics11234019"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.121404"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383313.3412206"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2019.05.037"},{"key":"e_1_3_2_7_2","unstructured":"Chris Burges. 2010. From RankNet to LambdaRank to LambdaMART: An Overview. Microsoft Research Technical Report MSR-TR-2010-82. Microsoft."},{"issue":"32","key":"e_1_3_2_8_2","first-page":"180","article-title":"Knowledge-based recommender systems","volume":"69","author":"Burke Robin D.","year":"2000","unstructured":"Robin D. Burke. 2000. Knowledge-based recommender systems. Encyclopedia of Library and Information Systems 69, Supplement 32 (2000), 180\u2013200.","journal-title":"Encyclopedia of Library and Information Systems"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-72079-9_12"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080836"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210014"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-020-09371-3"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCIT.2006.339989"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/1864708.1864721"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3309547"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2021.108119"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3604915.3608832"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jjimei.2022.100130"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2012.04.008"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-0716-2197-4_15"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401063"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052569"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2021.3079496"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2011.10.009"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.3389\/fdata.2023.1157899"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12599-018-0521-9"},{"key":"e_1_3_2_28_2","volume-title":"Proceedings of the 30th Conference on Neural Information Processing Systems (NeurIPS \u201917)","author":"Ke Guolin","year":"2017","unstructured":"Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. LightGBM: A highly efficient gradient boosting decision tree. In Proceedings of the 30th Conference on Neural Information Processing Systems (NeurIPS \u201917). Curran Associates, Inc."},{"key":"e_1_3_2_29_2","unstructured":"Richard Kibble Margaret Doyle and Alexandra Dobra-Kiel. 2020. The Future of Retail Banking: The Hyper-Personalization Imperative. Technical Report. Deloitte."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.2197\/ipsjjip.30.361"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2639968.2640064"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3677052.3698662"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1561\/1500000016"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341105.3375767"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1540-6261.1952.tb01525.x"},{"key":"e_1_3_2_36_2","first-page":"267","volume-title":"Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI \u201907)","author":"Marlin Benjamin M.","year":"2007","unstructured":"Benjamin M. Marlin, Richard S. Zemel, Sam Roweis, and Malcolm Slaney. 2007. Collaborative filtering and the missing at random assumption. In Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI \u201907). AUAI Press, 267\u2013275."},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04428-1_14"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-94590-9_10"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-45068-6_1"},{"key":"e_1_3_2_40_2","volume-title":"Technical Analysis of the Financial Markets: A Comprehensive Guide to Trading Methods and Applications","author":"Murphy John J.","year":"1999","unstructured":"John J. Murphy. 1999. Technical Analysis of the Financial Markets: A Comprehensive Guide to Trading Methods and Applications. Penguin Publishing Group, London."},{"key":"e_1_3_2_41_2","first-page":"35","volume-title":"Proceedings of the 1st International Workshop on Personalization and Recommender Systems in Financial Services (FinRec \u201915). CEUR Workshop Proceedings","author":"Musto Cataldo","year":"2015","unstructured":"Cataldo Musto and Giovanni Semeraro. 2015. Case-based recommender systems for personalized finance advisory. In Proceedings of the 1st International Workshop on Personalization and Recommender Systems in Financial Services (FinRec \u201915). CEUR Workshop Proceedings, 35\u201336."},{"key":"e_1_3_2_42_2","volume-title":"Poster Proceedings of the 8th ACM Conference on Recommender Systems (RecSys \u201914)","author":"Musto Cataldo","year":"2014","unstructured":"Cataldo Musto, Giovanni Semeraro, Pasquale Lops, Marco de Gemmis, and Georgios Lekkas. 2014. Financial product recommendation through case-based reasoning and diversification techniques. In Poster Proceedings of the 8th ACM Conference on Recommender Systems (RecSys \u201914). CEUR Workshop Proceedings."},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2015.06.001"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-0716-2197-4_2"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2012.09.012"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2024.123522"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0957-4174(99)00041-X"},{"key":"e_1_3_2_48_2","first-page":"452","volume-title":"Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI \u201909).","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI \u201909). AUAI Press, 452\u2013461."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383313.3412488"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-0716-2197-4_1"},{"key":"e_1_3_2_51_2","volume-title":"Proceedings of the IJCAI-2024 Workshop on Recommender Systems in Finance (Fin-RecSys \u201924), Co-Located with the 33rd International Joint Conference on Artificial Intelligence (IJCAI \u201924)","author":"Sanz-Cruzado Javier","year":"2024","unstructured":"Javier Sanz-Cruzado, Nikolaos Droukas, and Richard McCreadie. 2024. FAR-Trans: An investment dataset for financial asset recommendation. In Proceedings of the IJCAI-2024 Workshop on Recommender Systems in Finance (Fin-RecSys \u201924), Co-Located with the 33rd International Joint Conference on Artificial Intelligence (IJCAI \u201924)."},{"key":"e_1_3_2_52_2","volume-title":"Proceedings of the 3rd International Workshop on Personalization and Recommender Systems in Financial Services (FinRec \u201922), Co-Located with the 16th ACM Conference on Recommender Systems (RecSys \u201922)","author":"Sanz-Cruzado Javier","year":"2022","unstructured":"Javier Sanz-Cruzado, Richard McCreadie, Nikolaos Droukas, Craig Macdonald, and Iadh Ounis. 2022. On transaction-based metrics as proxy for profitability of financial asset recommendations. In Proceedings of the 3rd International Workshop on Personalization and Recommender Systems in Financial Services (FinRec \u201922), Co-Located with the 16th ACM Conference on Recommender Systems (RecSys \u201922)."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-0716-2197-4_24"},{"key":"e_1_3_2_54_2","first-page":"1670","volume-title":"Proceedings of the 33rd International Conference on International Conference on Machine Learning (ICML \u201916)","author":"Schnabel Tobias","year":"2016","unstructured":"Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as treatments: Debiasing learning and evaluation. In Proceedings of the 33rd International Conference on International Conference on Machine Learning (ICML \u201916). JMLR, 1670\u20131679."},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-94590-9"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2017.02.097"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/2043932.2043957"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00779-018-1121-x"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240386"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3592035"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00354-024-00241-w"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591850"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13278-013-0141-9"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2018.02.011"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1111\/jofi.13128"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1007\/S10489-024-05377-2"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/TrustCom\/BigDataSE.2018.00253"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1155\/2016\/5725143"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3556536"},{"key":"e_1_3_2_70_2","unstructured":"Yu Zhang Harshdeep Ahluwalia Allison Ying Michael Rabinovich and Aidan Geysen. 2022. Rational Rebalancing: An Analytical Approach to Multiasset Portfolio Rebalancing Decisions and Insights. Technical Report. Vanguard."},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/2792838.2800181"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2019.113006"},{"key":"e_1_3_2_73_2","first-page":"3","volume-title":"Proceedings of the 2nd International Workshop on Personalization and Recommender Systems in Financial Services (FinRec \u201916).","author":"Zibriczky D\u00e1vid","year":"2016","unstructured":"D\u00e1vid Zibriczky. 2016. Recommender systems meet finance: A literature review. In Proceedings of the 2nd International Workshop on Personalization and Recommender Systems in Financial Services (FinRec \u201916). CEUR Workshop Proceedings, 3\u201310."}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3780097","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,28]],"date-time":"2026-01-28T13:53:38Z","timestamp":1769608418000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3780097"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,28]]},"references-count":72,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,2,28]]}},"alternative-id":["10.1145\/3780097"],"URL":"https:\/\/doi.org\/10.1145\/3780097","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,28]]},"assertion":[{"value":"2025-01-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-02","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}