{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,5]],"date-time":"2026-04-05T20:31:47Z","timestamp":1775421107239,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,6,25]],"date-time":"2023-06-25T00:00:00Z","timestamp":1687651200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Saving and investment behaviour is crucial for all individuals to guarantee their welfare during work-life and retirement. We introduce a deep reinforcement learning model in which agents learn optimal portfolio allocation and saving strategies suitable for their heterogeneous profiles. The environment is calibrated with occupation- and age-dependent income dynamics. The research focuses on heterogeneous income trajectories dependent on agents\u2019 profiles and incorporates the parameterisation of agents\u2019 behaviours. The model provides a new flexible methodology to estimate lifetime consumption and investment choices for individuals with heterogeneous profiles.<\/jats:p>","DOI":"10.3390\/e25070977","type":"journal-article","created":{"date-parts":[[2023,6,26]],"date-time":"2023-06-26T05:39:13Z","timestamp":1687757953000},"page":"977","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Heterogeneous Retirement Savings Strategy Selection with Reinforcement Learning"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7870-6316","authenticated-orcid":false,"given":"Fatih","family":"Ozhamaratli","sequence":"first","affiliation":[{"name":"Department of Computer Science, University College London, London WC1E 6BT, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4588-667X","authenticated-orcid":false,"given":"Paolo","family":"Barucca","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University College London, London WC1E 6BT, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,25]]},"reference":[{"key":"ref_1","unstructured":"OECD (2022, June 01). Pension Markets in Focus 2020. Available online: www.oecd.org\/finance\/pensionmarketsinfocus.htm."},{"key":"ref_2","unstructured":"ONS (2022, May 01). Occupational Pension Schemes in the UK, Available online: https:\/\/www.ons.gov.uk\/peoplepopulationandcommunity\/personalandhouseholdfinances\/pensionssavingsandinvestments\/datasets\/occupationalpensionschemessurvey."},{"key":"ref_3","first-page":"S51","article-title":"What impact has the COVID-19 pandemic had on underpensioned groups?","volume":"34","author":"Wilkinson","year":"2021","journal-title":"Pensions Policy Inst."},{"key":"ref_4","unstructured":"Abraham, K., Haltiwanger, J., Sandusky, K., and Spletzer, J. (2017). Measuring and Accounting for Innovation in the 21st Century, Springer."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1140\/epjds\/s13688-022-00317-x","article-title":"A generative model for age and income distribution","volume":"11","author":"Ozhamaratli","year":"2022","journal-title":"EPJ Data Sci."},{"key":"ref_6","first-page":"55","article-title":"The \u201cLife Cycle\u201d Hypothesis of Saving: Aggregate Implications and Tests","volume":"53","author":"Ando","year":"1963","journal-title":"Am. Econ. Rev."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"239","DOI":"10.2307\/1926559","article-title":"Lifetime Portfolio Selection By Dynamic Stochastic Programming","volume":"51","author":"Samuelson","year":"1969","journal-title":"Rev. Econ. Stat."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"247","DOI":"10.2307\/1926560","article-title":"Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case","volume":"51","author":"Merton","year":"1969","journal-title":"Rev. Econ. Stat."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1016\/0022-0531(71)90038-X","article-title":"Optimum consumption and portfolio rules in a continuous-time model","volume":"3","author":"Merton","year":"1971","journal-title":"J. Econ. Theory"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1093\/rfs\/hhi017","article-title":"Consumption and Portfolio Choice over the Life Cycle","volume":"18","author":"Cocco","year":"2005","journal-title":"Rev. Financ. Stud."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.jmoneco.2014.11.008","article-title":"Life-cycle portfolio choice with liquid and illiquid financial assets","volume":"71","author":"Campanale","year":"2015","journal-title":"J. Monet. Econ."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"937","DOI":"10.2307\/1913778","article-title":"Substitution, Risk Aversion, and the Temporal Behavior of Consumption and Asset Returns: A Theoretical Framework","volume":"57","author":"Epstein","year":"1989","journal-title":"Econometrica"},{"key":"ref_13","first-page":"1893","article-title":"On the Asset Allocation of a Default Pension Fund","volume":"4","author":"Dahlquist","year":"2016","journal-title":"Ssrn Electron. J."},{"key":"ref_14","unstructured":"Zheng, S., Trott, A., Srinivasa, S., Naik, N., Gruesbeck, M., Parkes, D.C., and Socher, R. (2020). The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gomes, F.J., and Michaelides, A. (2002). Life-Cycle Asset Allocation: A Model with Borrowing Constraints, Uninsurable Labor Income Risk and Stock-Market Participation Costs. Ssrn Electron. J.","DOI":"10.2139\/ssrn.299388"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1257\/aer.20130456","article-title":"Systemic risk and stability in financial networks","volume":"105","author":"Acemoglu","year":"2015","journal-title":"Am. Econ. Rev."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1257\/jep.27.1.173","article-title":"Thirty years of prospect theory in economics: A review and assessment","volume":"27","author":"Barberis","year":"2013","journal-title":"J. Econ. Perspect."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"122","DOI":"10.2307\/1913738","article-title":"Risk Aversion in the Small and in the Large","volume":"32","author":"Pratt","year":"1964","journal-title":"Econometrica"},{"key":"ref_19","first-page":"7","article-title":"Actor-critic algorithms","volume":"12","author":"Konda","year":"1999","journal-title":"Adv. Neural Inf. Process. Syst. 1008\u20131014"},{"key":"ref_20","unstructured":"Hill, A., Raffin, A., Ernestus, M., Gleave, A., Kanervisto, A., Traore, R., Dhariwal, P., Hesse, C., Klimov, O., and Nichol, A. (2022, June 15). Stable Baselines. Available online: https:\/\/github.com\/hill-a\/stable-baselines."},{"key":"ref_21","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv."},{"key":"ref_22","unstructured":"Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. (2015). High-Dimensional Continuous Control Using Generalized Advantage Estimation. arXiv."},{"key":"ref_23","unstructured":"Huang, S., Dossa, R.F.J., Raffin, A., Kanervisto, A., and Wang, W. (2022, June 15). The 37 Implementation Details of Proximal Policy Optimization. Available online: https:\/\/iclr-blog-track.github.io\/2022\/03\/25\/ppo-implementation-details\/."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1080\/09540099108946587","article-title":"Function Optimization using Connectionist Reinforcement Learning Algorithms","volume":"3","author":"Williams","year":"1991","journal-title":"Connect. Sci."},{"key":"ref_25","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_26","unstructured":"Terry, J.K., Black, B., Grammel, N., Jayakumar, M., Hari, A., Sulivan, R., Santos, L., Perez, R., Horsch, C., and Dieffendahl, C. (2020). PettingZoo: Gym for Multi-Agent Reinforcement Learning. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1186\/1748-5908-7-37","article-title":"Validation of the theoretical domains framework for use in behaviour change and implementation research","volume":"7","author":"Cane","year":"2012","journal-title":"Implement. Sci."},{"key":"ref_29","unstructured":"BLS (2019). 2019 Annual Averages\u2014Household Data\u2014Tables from Employment and Earnings."},{"key":"ref_30","unstructured":"Department of Health and Human Services (2019). Annual Update of the HHS Poverty Guidelines, Federal Register, No. 22, 1 February 2019; Notices."},{"key":"ref_31","unstructured":"SSA (2017). Actuarial Life Table\u2014SSA."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/7\/977\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:00:23Z","timestamp":1760126423000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/7\/977"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,25]]},"references-count":31,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,7]]}},"alternative-id":["e25070977"],"URL":"https:\/\/doi.org\/10.3390\/e25070977","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,25]]}}}