{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,7]],"date-time":"2025-11-07T19:30:41Z","timestamp":1762543841613,"version":"build-2065373602"},"reference-count":34,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2023,5,31]],"date-time":"2023-05-31T00:00:00Z","timestamp":1685491200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ESRC","award":["ES\/K005154\/1"],"award-info":[{"award-number":["ES\/K005154\/1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>We construct a New Keynesian (NK) behavioural macroeconomic model with bounded-rationality (BR) and heterogeneous agents. We solve and simulate the model using a third-order approximation for a given policy and evaluate its properties using this solution. The model is inhabited by fully rational (RE) and BR agents. The latter are anticipated utility learners, given their beliefs of aggregate states, and they use simple heuristic rules to forecast aggregate variables exogenous to their micro-environment. In the most general form of the model, RE and BR agents learn from their forecasting errors by observing and comparing them with each other, making the composition of the two types endogenous. This reinforcement learning is then at the core of the heterogeneous expectations model and leads to the striking result that increasing the volatility of exogenous shocks, by assisting the learning process, increases the proportion of RE agents and is welfare-increasing.<\/jats:p>","DOI":"10.3390\/a16060280","type":"journal-article","created":{"date-parts":[[2023,6,1]],"date-time":"2023-06-01T02:12:45Z","timestamp":1685585565000},"page":"280","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Reinforcement Learning in a New Keynesian Model"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2467-3202","authenticated-orcid":false,"given":"Szabolcs","family":"De\u00e1k","sequence":"first","affiliation":[{"name":"Department of Economics, University of Exeter, Exeter EX4 4PU, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul","family":"Levine","sequence":"additional","affiliation":[{"name":"School of Economics, University of Surrey, Guildford GU2 7XH, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joseph","family":"Pearlman","sequence":"additional","affiliation":[{"name":"Department of Economics, City University London, London EC1R 0JD, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bo","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Economics, Swansea University, Swansea SA2 8PP, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1007\/s00199-010-0539-9","article-title":"Monetary Policy and Heterogeneous Agents","volume":"47","author":"Branch","year":"2011","journal-title":"Econ. Theory"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1007\/s00199-010-0543-0","article-title":"Animal spirits and monetary policy","volume":"47","year":"2011","journal-title":"Econ. Theory"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1016\/j.jebo.2012.02.013","article-title":"Booms and Busts in Economic Activity: A Behavioral Explanation","volume":"83","year":"2012","journal-title":"J. Econ. Behav. Organ."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"De Grauwe, P. (2012). Lectures on Behavioral Macroeconomics, Princeton University Press.","DOI":"10.2307\/j.ctvcmxp1g"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2307\/1912017","article-title":"Macroeconomics and Reality","volume":"48","author":"Sims","year":"1980","journal-title":"Econometrica"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1146\/annurev.economics.050708.142927","article-title":"Learning and Macroeconomics","volume":"1","author":"Evans","year":"2009","journal-title":"Annu. Rev. Econ."},{"key":"ref_7","unstructured":"Eusepi, S., and Preston, B. (2016). Federal Reserve Bank of New York Satff Reports, Federal Reserve Bank of New York. No. 782."},{"key":"ref_8","unstructured":"Branch, W.A., and McGough, B. (2018). Handbook of Computational Economics 4, Elsevier."},{"key":"ref_9","first-page":"58","article-title":"Behavioural New Keynesian Models","volume":"59","author":"CalvertJump","year":"2019","journal-title":"J. Macroecon."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1016\/j.jedc.2016.06.001","article-title":"Agent based-stock flow consistent macroeconomics: Towards a benchmark model","volume":"69","author":"Caiani","year":"2016","journal-title":"J. Econ. Dyn. Control"},{"key":"ref_11","unstructured":"Tesfatsion, L.S., and Judd, K.L. (2006). Handbook of Computational Economics, North-Holand."},{"key":"ref_12","unstructured":"Levine, P. (2020). Oxford Research Encyclopedia of Economics and Finance, Oxford University Press."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.2307\/2171879","article-title":"A Rational Route to Randomness","volume":"65","author":"Brock","year":"1997","journal-title":"Econometrica"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"482","DOI":"10.1177\/00076503221101888","article-title":"The irrationality of rationality in market economics: A paradox of incentives perspective","volume":"62","author":"Tamvada","year":"2023","journal-title":"Bus. Soc."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Sirota, M., Juanchich, M., and Holford, D.L. (2023). Rationally irrational: When people do not correct their reasoning errors even if they could. J. Exp. Psychol. Gen., advance online publication.","DOI":"10.31234\/osf.io\/5sjwz"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Jacobs, D., Kalai, E., and Kamien, M. (1998). Frontiers of Research in Economic Theory, Cambridge University Press.","DOI":"10.1017\/CCOL0521632226"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1224","DOI":"10.1016\/j.jet.2010.11.003","article-title":"Internal Rationality, Imperfect Market Knowledge and Asset Prices","volume":"146","author":"Adam","year":"2011","journal-title":"J. Econ. Theory"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1111\/j.1468-2354.2008.00477.x","article-title":"Anticipated utility and rational expectations as approximations of bayesian decision making","volume":"49","author":"Cogley","year":"2008","journal-title":"Int. Econ. Rev."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1492","DOI":"10.1016\/j.jedc.2010.03.012","article-title":"Dynamic predictor election in a new keynesian model with heterogeneous agents","volume":"34","author":"Branch","year":"2010","journal-title":"J. Econ. Dyn. Control"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1016\/j.jedc.2012.11.001","article-title":"Heterogeneous Expectations in Monetary DSGE Models","volume":"37","author":"Massaro","year":"2013","journal-title":"J. Econ. Dyn. Control"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1080\/07350015.2017.1321548","article-title":"Behavioral heterogeneity in us inflation","volume":"37","author":"Cornea","year":"2017","journal-title":"J. Bus. Econ. Stat."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1016\/j.jebo.2019.07.014","article-title":"Learning, Heterogeneity, and Complexity in the New Keynesian model","volume":"166","author":"CalvertJump","year":"2019","journal-title":"J. Econ. Behav. Organ."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2065","DOI":"10.1016\/j.jmoneco.2006.11.007","article-title":"Expectations, learning and macroeconomic persistence","volume":"54","author":"Milani","year":"2007","journal-title":"J. Monet. Econ."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Anufriev, M., Hommes, C., and Makarewicz, T. (2015). Simple Forecasting Heuristics that Make Us Smart: Evidence from Different Market Experiments, Economics Discipline Group, UTS Business School, University of Technology.","DOI":"10.2139\/ssrn.2634445"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Hommes, C. (2013). Behavioral Rationality and Heterogeneous Expectations in Complex Economic Systems, Cambridge University Press.","DOI":"10.1017\/CBO9781139094276"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/S0022-1996(02)00056-9","article-title":"Closing small open economy models","volume":"61","author":"Uribe","year":"2003","journal-title":"J. Int. Econ."},{"key":"ref_27","first-page":"297","article-title":"Monopolistic competition and optimal product diversity","volume":"67","author":"Dixit","year":"1977","journal-title":"Am. Econ. Rev."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1016\/0304-3932(83)90060-0","article-title":"Staggered Prices in a Utility-Maximising Framework","volume":"12","author":"Calvo","year":"1983","journal-title":"J. Monet. Econ."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2320","DOI":"10.1257\/aer.104.8.2320","article-title":"Man-Bites-Dog Business Cycles","volume":"104","author":"Nimark","year":"2014","journal-title":"Am. Econ. Rev."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Deak, S., Levine, P., Pearlman, J., and Yang, B. (2017). Internal Rationality, Learning and Imperfect Information, School of Economics, University of Surrey. Discussion Papers 08\/17.","DOI":"10.2139\/ssrn.3091876"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1257\/aer.97.3.586","article-title":"Shocks and Frictions in US business cycles: A Bayesian DSGE approach","volume":"97","author":"Smets","year":"2007","journal-title":"Am. Econ. Rev."},{"key":"ref_32","unstructured":"Deak, S., Mirza, A., Levine, P., and Pearlman, J. (2019). Designing Robust Policies using Optimal Pooling, School of Economics, University of Surrey. Discussion Papers 12\/19."},{"key":"ref_33","unstructured":"Deak, S., Mirza, A., Levine, P., and Pham, S. (2023). Negotiating the Wilderness of Bounded Rationality through Robust Policy, School of Economics, University of Surrey. Discussion Papers 02\/23."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"31630","DOI":"10.1007\/s11356-018-3062-3","article-title":"Markov-switching vector autoregressive neural networks and sensitivity analysis of environment, economic growth and petrol prices","volume":"25","author":"Bildirici","year":"2018","journal-title":"Environ. Sci. Pollut. Res."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/6\/280\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:46:09Z","timestamp":1760125569000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/6\/280"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,31]]},"references-count":34,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["a16060280"],"URL":"https:\/\/doi.org\/10.3390\/a16060280","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2023,5,31]]}}}