{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T04:25:00Z","timestamp":1777523100128,"version":"3.51.4"},"reference-count":61,"publisher":"SAGE Publications","issue":"5-6","license":[{"start":{"date-parts":[[2025,9,8]],"date-time":"2025-09-08T00:00:00Z","timestamp":1757289600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,9,8]],"date-time":"2025-09-08T00:00:00Z","timestamp":1757289600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"crossref","award":["FA9550-24-1-0305"],"award-info":[{"award-number":["FA9550-24-1-0305"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Research Corporation for Science Advancement & Frederick Gardner Cottrell Foundation","award":["29087"],"award-info":[{"award-number":["29087"]}]},{"name":"Canada Research Chair Dynamics of Cognition","award":["FD507106"],"award-info":[{"award-number":["FD507106"]}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["RGPIN-2020-05577"],"award-info":[{"award-number":["RGPIN-2020-05577"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Adaptive Behavior"],"published-print":{"date-parts":[[2025,12]]},"abstract":"<jats:p>Experimental data suggest that episodic memory is involved in sequential value-based decision-making. By contrast, standard computational models of decision-making assume that prior reward outcomes are integrated into subjective values rather than remembered discretely. Previous work developed a minimal computational framework for sequential value-based decision-making that is based on noisy sampling of episodic memories, rather than calculating value. We called these agents \u201cImperfect Memory Programs\u201d (IMPs) and showed how their single free parameter optimizes the trade-off between the magnitude of error and the complexity of imperfect recall. Here, we develop biologically plausible approximations to the IMPs with lossy agents (LIMPs) that maintain only 1 bit of reward memory for binary outcomes but fail to encode rewards with some probability. Both IMPs and LIMPs perform similarly to or better than a simple agent with perfect memory in multiple classic decision-making tasks and generate phenomenology that resembles value-based computations. We find that allowing different encoding probabilities for rewards and omissions improves performance further and allows to trade-off matching versus maximizing behavior, as well as flexible versus stable performance. Together, these results suggest that episodic agents can approximate value-based agents through capitalizing on realistic encoding and\/or sampling noise. This suggests that mnemonic errors (1) can improve, rather than impair decision-making and (2) provide a plausible alternative explanation for some behavioral correlates of \u201cvalue\u201d.<\/jats:p>","DOI":"10.1177\/10597123251372839","type":"journal-article","created":{"date-parts":[[2025,9,8]],"date-time":"2025-09-08T17:09:40Z","timestamp":1757351380000},"page":"333-345","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":1,"title":["Noisy Memory Generates Value in Changing Environments"],"prefix":"10.1177","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3505-3319","authenticated-orcid":false,"given":"Jorge","family":"Ram\u00edrez-Ruiz","sequence":"first","affiliation":[{"name":"Universit\u00e9 de Montr\u00e9al"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"R. Becket","family":"Ebitz","sequence":"additional","affiliation":[{"name":"Universit\u00e9 de Montr\u00e9al"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2025,9,8]]},"reference":[{"key":"e_1_3_3_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2006.03.036"},{"key":"e_1_3_3_3_1","doi-asserted-by":"publisher","DOI":"10.1002\/bdm.443"},{"key":"e_1_3_3_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jmp.2008.05.006"},{"key":"e_1_3_3_5_1","doi-asserted-by":"publisher","DOI":"10.1038\/ncomms15958"},{"key":"e_1_3_3_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/0031-9384(69)90075-4"},{"key":"e_1_3_3_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00422-013-0571-5"},{"key":"e_1_3_3_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2020.09.075"},{"key":"e_1_3_3_9_1","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.69748"},{"key":"e_1_3_3_10_1","doi-asserted-by":"publisher","DOI":"10.1101\/lm.053595.122"},{"key":"e_1_3_3_11_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1460-9568.2011.07980.x"},{"key":"e_1_3_3_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2016.08.031"},{"key":"e_1_3_3_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00052-7"},{"key":"e_1_3_3_14_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature04766"},{"key":"e_1_3_3_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2018.01.011"},{"key":"e_1_3_3_16_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1007475"},{"key":"e_1_3_3_17_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1912330117"},{"key":"e_1_3_3_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/bdm.602"},{"key":"e_1_3_3_19_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.83.1.37"},{"key":"e_1_3_3_20_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0140525X00009444"},{"key":"e_1_3_3_21_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-020-00971-z"},{"key":"e_1_3_3_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cobeha.2021.02.018"},{"key":"e_1_3_3_23_1","doi-asserted-by":"publisher","DOI":"10.1038\/nn.2342"},{"key":"e_1_3_3_24_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0706111104"},{"key":"e_1_3_3_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11065-020-09472-2"},{"key":"e_1_3_3_26_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-psych-122414-033625"},{"key":"e_1_3_3_27_1","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.1980.0090"},{"key":"e_1_3_3_28_1","doi-asserted-by":"publisher","DOI":"10.1101\/2025.05.06.652482"},{"key":"e_1_3_3_29_1","doi-asserted-by":"publisher","DOI":"10.1006\/jmps.2001.1388"},{"key":"e_1_3_3_30_1","doi-asserted-by":"crossref","unstructured":"Jurewicz K. Sleezer B. J. Mehta P. S. Hayden B. Y. Ebitz R. B. (2022). Irrational choices via a curvilinear representational geometry for value. bioRxiv (pp. 2022\u201303). Publisher: Cold Spring Harbor Laboratory.","DOI":"10.1101\/2022.03.31.486635"},{"key":"e_1_3_3_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cognition.2020.104239"},{"key":"e_1_3_3_32_1","unstructured":"Kirtland A. Ivanov A. Allen C. Littman M. Konidaris G. (2025). Memory as state abstraction over trajectories."},{"key":"e_1_3_3_33_1","doi-asserted-by":"publisher","DOI":"10.1901\/jeab.2005.110-04"},{"key":"e_1_3_3_34_1","doi-asserted-by":"crossref","unstructured":"Laurie V.-J. Shourkeshti A. Chen C. S. Herman A. B. Grissom N. M. Ebitz R. B. (2024). Persistent decision-making in mice monkeys and humans. bioRxiv (pp. 2024\u201305). Publisher: Cold Spring Harbor Laboratory.","DOI":"10.1101\/2024.05.07.592970"},{"key":"e_1_3_3_35_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-017-0067"},{"key":"e_1_3_3_36_1","volume-title":"Advances in neural information processing systems","author":"Lengyel M.","year":"2007","unstructured":"Lengyel M., Dayan P. (2007). Hippocampal contributions to control: The third way. In Advances in neural information processing systems (Vol. 20). Curran Associates, Inc."},{"key":"e_1_3_3_37_1","doi-asserted-by":"publisher","DOI":"10.1038\/sj.npp.1300137"},{"key":"e_1_3_3_38_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-15146-7"},{"key":"e_1_3_3_39_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41593-018-0232-z"},{"key":"e_1_3_3_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0022-5193(87)80209-6"},{"key":"e_1_3_3_41_1","doi-asserted-by":"crossref","unstructured":"Nicholas J. Daw N. D. Shohamy D. (2022). Uncertainty alters the balance between incremental learning and episodic memory. eLife 11 Article e81679. Publisher: eLife Sciences Publications Ltd. https:\/\/doi.org\/10.7554\/eLife.81679","DOI":"10.7554\/eLife.81679"},{"key":"e_1_3_3_42_1","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.5498-10.2012"},{"key":"e_1_3_3_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2022.04.005"},{"key":"e_1_3_3_44_1","first-page":"16948","volume-title":"Advances in neural information processing systems","author":"Patel N.","year":"2020","unstructured":"Patel N., Acerbi L., Pouget A. (2020). Dynamic allocation of limited memory resources in reinforcement learning. In Advances in neural information processing systems (Vol. 33, pp. 16948\u201316960). Curran Associates, Inc."},{"key":"e_1_3_3_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2009.07.048"},{"key":"e_1_3_3_46_1","doi-asserted-by":"crossref","unstructured":"Pisupati S. Chartarifsky-Lynn L. Khanal A. Churchland A. K. (2021). Lapses in perceptual decisions reflect exploration. Elife 10 Article e55490. Publisher: eLife Sciences Publications Ltd. https:\/\/doi.org\/10.7554\/eLife.55490","DOI":"10.7554\/eLife.55490"},{"key":"e_1_3_3_47_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0039413"},{"key":"e_1_3_3_48_1","unstructured":"Ramani D. (2019). A short survey on memory based reinforcement learning. arXiv:1904.06736 [cs]."},{"key":"e_1_3_3_49_1","first-page":"301","volume-title":"From animals to animats 17. SAB 2024. Lecture notes in computer science, volume 14993","author":"Ram\u00edrez-Ruiz J.","year":"2025","unstructured":"Ram\u00edrez-Ruiz J., Ebitz R. B. (2025). \u201cValue\u201d emerges from imperfect memory. In Brock O., Krichmar J. (Eds.), From animals to animats 17. SAB 2024. Lecture notes in computer science, volume 14993 (pp. 301\u2013313). Springer."},{"key":"e_1_3_3_50_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2014856118"},{"key":"e_1_3_3_51_1","first-page":"64","volume-title":"Classical conditioning II: Current research and theory","author":"Rescorla R. A.","year":"1972","unstructured":"Rescorla R. A., Wagner A. R. (1972). A theory of Pavlovian conditioning: Variations on the effectiveness of reinforcement and non-reinforcement. In Black A. H., Prokasy W. F. (Eds.), Classical conditioning II: Current research and theory (pp. 64\u201399). Appleton-Century-Crofts."},{"key":"e_1_3_3_52_1","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729586"},{"key":"e_1_3_3_53_1","doi-asserted-by":"crossref","unstructured":"Rosenbaum G. M. Grassie H. L. Hartley C. A. (2022). Valence biases in reinforcement learning shift across adolescence and modulate subsequent memory. eLife 11 Article e64620. https:\/\/doi.org\/10.7554\/eLife.64620","DOI":"10.7554\/eLife.64620"},{"key":"e_1_3_3_54_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0003795"},{"key":"e_1_3_3_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2015.11.002"},{"key":"e_1_3_3_56_1","doi-asserted-by":"crossref","unstructured":"Shourkeshti A. Marrocco G. Jurewicz K. Moore T. Ebitz R. B. (2023). Pupil size predicts the onset of exploration in brain and behavior. bioRxiv. Publisher: Cold Spring Harbor Laboratory Preprints.","DOI":"10.1101\/2023.05.24.541981"},{"key":"e_1_3_3_57_1","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.5159-05.2006"},{"key":"e_1_3_3_58_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1094765"},{"key":"e_1_3_3_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.1998.712192"},{"key":"e_1_3_3_60_1","doi-asserted-by":"crossref","unstructured":"Wilson R. C. Collins A. G. (2019). Ten simple rules for the computational modeling of behavioral data. eLife 8 Article e49547. Publisher: eLife Sciences Publications Ltd. https:\/\/doi.org\/10.7554\/eLife.49547","DOI":"10.7554\/eLife.49547"},{"key":"e_1_3_3_61_1","doi-asserted-by":"publisher","DOI":"10.1007\/s004269900003"},{"key":"e_1_3_3_62_1","doi-asserted-by":"crossref","unstructured":"Zid M. Laurie V.-J. Ram\u00edrez-Ruiz J. Lavigne-Champagne A. Shourkeshti A. Harrell D. Herman A. B. Ebitz R. B. (2025). Humans forage for reward in reinforcement learning tasks. Pages: 2024.07.08.602539 Section: New Results.","DOI":"10.1101\/2024.07.08.602539"}],"container-title":["Adaptive Behavior"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10597123251372839","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/10597123251372839","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10597123251372839","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T16:19:40Z","timestamp":1777393180000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/10597123251372839"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,8]]},"references-count":61,"journal-issue":{"issue":"5-6","published-print":{"date-parts":[[2025,12]]}},"alternative-id":["10.1177\/10597123251372839"],"URL":"https:\/\/doi.org\/10.1177\/10597123251372839","relation":{},"ISSN":["1059-7123","1741-2633"],"issn-type":[{"value":"1059-7123","type":"print"},{"value":"1741-2633","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,8]]}}}