{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,27]],"date-time":"2026-01-27T22:10:03Z","timestamp":1769551803205,"version":"3.49.0"},"reference-count":109,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T00:00:00Z","timestamp":1730851200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Neurosci."],"abstract":"<jats:p>Deep Reinforcement Learning is a branch of artificial intelligence that uses artificial neural networks to model reward-based learning as it occurs in biological agents. Here we modify a Deep Reinforcement Learning approach by imposing a suppressive effect on the connections between neurons in the artificial network\u2014simulating the effect of dendritic spine loss as observed in major depressive disorder (MDD). Surprisingly, this simulated spine loss is sufficient to induce a variety of MDD-like behaviors in the artificially intelligent agent, including anhedonia, increased temporal discounting, avoidance, and an altered exploration\/exploitation balance. Furthermore, simulating alternative and longstanding reward-processing-centric conceptions of MDD (dysfunction of the dopamine system, altered reward discounting, context-dependent learning rates, increased exploration) does not produce the same range of MDD-like behaviors. These results support a conceptual model of MDD as a reduction of brain connectivity (and thus information-processing capacity) rather than an imbalance in monoamines\u2014though the computational model suggests a possible explanation for the dysfunction of dopamine systems in MDD. Reversing the spine-loss effect in our computational MDD model can lead to rescue of rewarding behavior under some conditions. This supports the search for treatments that increase plasticity and synaptogenesis, and the model suggests some implications for their effective administration.<\/jats:p>","DOI":"10.3389\/fncom.2024.1466364","type":"journal-article","created":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T04:43:52Z","timestamp":1730868232000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Simulated synapse loss induces depression-like behaviors in deep reinforcement learning"],"prefix":"10.3389","volume":"18","author":[{"given":"Eric","family":"Chalmers","sequence":"first","affiliation":[]},{"given":"Santina","family":"Duarte","sequence":"additional","affiliation":[]},{"given":"Xena","family":"Al-Hejji","sequence":"additional","affiliation":[]},{"given":"Daniel","family":"Devoe","sequence":"additional","affiliation":[]},{"given":"Aaron","family":"Gruber","sequence":"additional","affiliation":[]},{"given":"Robert J.","family":"McDonald","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,11,6]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"929","DOI":"10.1016\/j.tips.2021.08.003","article-title":"Neuroplasticity as a convergent mechanism of ketamine and classical psychedelics","volume":"42","author":"Aleksandrova","year":"2021","journal-title":"Trends Pharmacol. Sci."},{"key":"ref2","volume-title":"Diagnostic and statistical manual of mental disorders","year":"2022"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"1176","DOI":"10.1001\/jamapsychiatry.2019.2102","article-title":"Delay discounting as a Transdiagnostic process in psychiatric disorders: a meta-analysis","volume":"76","author":"Amlung","year":"2019","journal-title":"JAMA Psychiatry"},{"key":"ref4","author":"Andriushchenko","year":"2023"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1002\/da.22576","article-title":"Spared internal but impaired external reward prediction error signals in major depressive disorder during reinforcement learning","volume":"34","author":"Bakic","year":"2017","journal-title":"Depress. Anxiety"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1016\/j.bbr.2009.09.029","article-title":"Juvenile peer play experience and the development of the orbitofrontal and medial prefrontal cortices","volume":"207","author":"Bell","year":"2010","journal-title":"Behav. Brain Res."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"1036","DOI":"10.1093\/ijnp\/pyx056","article-title":"Dopamine system dysregulation in major depressive disorders","volume":"20","author":"Belujon","year":"2017","journal-title":"Int. J. Neuropsychopharmacol."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"803","DOI":"10.1016\/j.molmed.2020.03.010","article-title":"Adult hippocampal neurogenesis in major depressive disorder and Alzheimer\u2019s disease","volume":"26","author":"Berger","year":"2020","journal-title":"Trends Mol. Med."},{"key":"ref9","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1016\/s0006-3223(99)00230-9","article-title":"Antidepressant effects of ketamine in depressed patients","volume":"47","author":"Berman","year":"2000","journal-title":"Biol. Psychiatry"},{"key":"ref10","first-page":"e266","article-title":"Stress-induced anhedonia is associated with hypertrophy of medium spiny neurons of the nucleus accumbens","volume-title":"Transl. Psychiatry","author":"Bessa","year":"2013"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"563","DOI":"10.1016\/j.cognition.2013.08.018","article-title":"The influence of depression symptoms on exploratory decision-making","volume":"129","author":"Blanco","year":"2013","journal-title":"Cognition"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"603","DOI":"10.1016\/j.neuron.2020.06.014","article-title":"Deep reinforcement learning and its neuroscientific implications","volume":"107","author":"Botvinick","year":"2020","journal-title":"Neuron"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1080\/09540091.2015.1031081","article-title":"A reinforcement learning model of joy, distress, hope and fear","volume":"27","author":"Broekens","year":"2015","journal-title":"Connect. Sci."},{"key":"ref14","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1017\/S0033291720001956","article-title":"Challenging the negative learning bias hypothesis of depression: reversal learning in a naturalistic psychiatric sample","volume":"52","author":"Brolsma","year":"2022","journal-title":"Psychol. Med."},{"key":"ref15","doi-asserted-by":"publisher","first-page":"161","DOI":"10.3389\/fphar.2013.00161","article-title":"Antidepressant effects of ketamine: mechanisms underlying fast-acting novel antidepressants","volume":"4","author":"Browne","year":"2013","journal-title":"Front. Pharmacol."},{"key":"ref16","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1038\/s41386-022-01389-z","article-title":"Towards an understanding of psychedelic-induced neuroplasticity","volume":"48","author":"Calder","year":"2023","journal-title":"Neuropsychopharmacology"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1016\/j.biopsych.2021.05.008","article-title":"Brain-derived neurotrophic factor signaling in depression and antidepressant action","volume":"90","author":"Castr\u00e9n","year":"2021","journal-title":"Biol. Psychiatry, Ronald Duman\u2019s Legacy"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"44374","DOI":"10.1038\/srep44374","article-title":"Valuation in major depression is intact and stable in a non-learning environment","volume":"7","author":"Chung","year":"2017","journal-title":"Sci. Rep."},{"key":"ref19","doi-asserted-by":"publisher","first-page":"6770","DOI":"10.1038\/s41598-019-43245-z","article-title":"Dopamine blockade impairs the exploration-exploitation trade-off in rats","volume":"9","author":"Cinotti","year":"2019","journal-title":"Sci. Rep."},{"key":"ref20","volume-title":"Model-based reinforcement learning as cognitive search: Neurocomputational theories","author":"Daw","year":"2012"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"1704","DOI":"10.1038\/nn1560","article-title":"Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control","volume":"8","author":"Daw","year":"2005","journal-title":"Nat. Neurosci."},{"key":"ref22","doi-asserted-by":"publisher","first-page":"6945","DOI":"10.1038\/s41467-021-27268-7","article-title":"Chronic nicotine increases midbrain dopamine neuron activity and biases individual strategies towards reduced exploration in mice","volume":"12","author":"Dongelmans","year":"2021","journal-title":"Nat. Commun."},{"key":"ref23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00401-015-1449-5","article-title":"Analyzing dendritic spine pathology in Alzheimer\u2019s disease: problems and opportunities","volume":"130","author":"Dorostkar","year":"2015","journal-title":"Acta Neuropathol. (Berl.)"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1177\/2045125316638008","article-title":"Antidepressive, anxiolytic, and antiaddictive effects of ayahuasca, psilocybin and lysergic acid diethylamide (LSD): a systematic review of clinical trials published in the last 25 years","volume":"6","author":"dos Santos","year":"2016","journal-title":"Ther. Adv. Psychopharmacol."},{"key":"ref25","doi-asserted-by":"publisher","first-page":"238","DOI":"10.1038\/nm.4050","article-title":"Synaptic plasticity and depression: new insights from stress and rapid-acting antidepressants","volume":"22","author":"Duman","year":"2016","journal-title":"Nat. Med."},{"key":"ref26","doi-asserted-by":"publisher","first-page":"387","DOI":"10.3390\/neurolint13030038","article-title":"Selective serotonin reuptake inhibitors and adverse effects: a narrative review","volume":"13","author":"Edinoff","year":"2021","journal-title":"Neurol. Int."},{"key":"ref27","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1016\/j.biopsych.2010.01.027","article-title":"Reward and punishment processing in depression","volume":"68","author":"Eshel","year":"2010","journal-title":"Biol. Psychiatry"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1016\/S0165-0173(02)00158-3","article-title":"Dendritic spine pathology: cause or consequence of neurological disorders?","volume":"39","author":"Fiala","year":"2002","journal-title":"Brain Res. Rev."},{"key":"ref29","doi-asserted-by":"publisher","first-page":"626128","DOI":"10.3389\/fncel.2021.626128","article-title":"Molecular mechanisms underlying synaptic and axon degeneration in Parkinson\u2019s disease","volume":"15","author":"Gcwensa","year":"2021","journal-title":"Front. Cell. Neurosci."},{"key":"ref30","doi-asserted-by":"publisher","first-page":"937133","DOI":"10.3389\/fnmol.2022.937133","article-title":"Neuronal cell death mechanisms in Alzheimer\u2019s disease: an insight","volume":"15","author":"Goel","year":"2022","journal-title":"Front. Mol. Neurosci."},{"key":"ref31","doi-asserted-by":"publisher","first-page":"17102","DOI":"10.1523\/JNEUROSCI.4623-10.2010","article-title":"More is less: a disinhibited prefrontal cortex impairs cognitive flexibility","volume":"30","author":"Gruber","year":"2010","journal-title":"J. Neurosci."},{"key":"ref32","doi-asserted-by":"publisher","first-page":"536","DOI":"10.1016\/j.jad.2019.09.039","article-title":"Impairments in episodic future thinking for positive events and anticipatory pleasure in major depression","volume":"260","author":"Hallford","year":"2020","journal-title":"J. Affect. Disord."},{"key":"ref33","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1016\/s0166-4328(99)00114-x","article-title":"Changes in adult brain and behavior caused by neonatal limbic damage: implications for the etiology of schizophrenia","volume":"107","author":"Hanlon","year":"2000","journal-title":"Behav. Brain Res."},{"key":"ref34","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1192\/bjp.bp.108.051193","article-title":"Why do antidepressants take so long to work? A cognitive neuropsychological model of antidepressant drug action","volume":"195","author":"Harmer","year":"","journal-title":"Br. J. Psychiatry"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"1178","DOI":"10.1176\/appi.ajp.2009.09020149","article-title":"Effect of acute antidepressant administration on negative affective Bias in depressed patients","volume":"166","author":"Harmer","year":"","journal-title":"Am. J. Psychiatry"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"174222","DOI":"10.3389\/fncir.2016.00023","article-title":"Why neurons have thousands of synapses, a theory of sequence memory in neocortex","volume":"10","author":"Hawkins","year":"2016","journal-title":"Front. Neural Circuits"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"2715","DOI":"10.2147\/NDT.S170989","article-title":"Neuronal connectivity in major depressive disorder: a systematic review","volume":"14","author":"Helm","year":"2018","journal-title":"Neuropsychiatr. Dis. Treat."},{"key":"ref38","doi-asserted-by":"publisher","first-page":"7","DOI":"10.3389\/fnint.2013.00067","article-title":"Learning from negative feedback in patients with major depressive disorder is attenuated by SSRI antidepressants","volume":"7","author":"Herzallah","year":"2013","journal-title":"Front. Integr. Neurosci."},{"key":"ref39","doi-asserted-by":"publisher","first-page":"E6871","DOI":"10.1073\/pnas.1803274115","article-title":"Redundancy in synaptic connections enables neurons to learn optimally","volume":"115","author":"Hiratani","year":"2018","journal-title":"Proc. Natl. Acad. Sci."},{"key":"ref40","doi-asserted-by":"publisher","first-page":"1529","DOI":"10.1038\/s41467-019-09562-7","article-title":"Lower synaptic density is associated with depression severity and network alterations","volume":"10","author":"Holmes","year":"2019","journal-title":"Nat. Commun."},{"key":"ref41","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1186\/2045-5380-3-12","article-title":"Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis","volume":"3","author":"Huys","year":"2013","journal-title":"Biol. Mood Anxiety Disord."},{"key":"ref42","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1016\/j.neunet.2023.10.005","article-title":"Elucidating multifinal and equifinal pathways to developmental disorders by constructing real-world neurorobotic models","volume":"169","author":"Idei","year":"2024","journal-title":"Neural Netw."},{"key":"ref43","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1016\/j.drugalcdep.2013.11.014","article-title":"Delay discounting by depressed and non-depressed adolescent smokers and non-smokers","volume":"135","author":"Imhoff","year":"2014","journal-title":"Drug Alcohol Depend."},{"key":"ref44","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1016\/j.biopsych.2019.08.022","article-title":"Approach-avoidance conflict in major depressive disorder: congruent neural findings in humans and nonhuman Primates","volume":"87","author":"Ironside","year":"2020","journal-title":"Biol. Psychiatry"},{"key":"ref45","doi-asserted-by":"publisher","first-page":"3149","DOI":"10.1038\/s41380-023-02095-y","article-title":"A leaky umbrella has little value: evidence clearly indicates the serotonin system is implicated in depression","volume":"28","author":"Jauhar","year":"2023","journal-title":"Mol. Psychiatry"},{"key":"ref46","doi-asserted-by":"publisher","first-page":"4272","DOI":"10.1038\/s41380-023-02285-8","article-title":"Effects of escitalopram on synaptic density in the healthy human brain: a randomized controlled trial","volume":"28","author":"Johansen","year":"2023","journal-title":"Mol. Psychiatry"},{"key":"ref47","doi-asserted-by":"publisher","first-page":"e1010776","DOI":"10.1371\/journal.pcbi.1010776","article-title":"Robust and brain-like working memory through short-term synaptic plasticity","volume":"18","author":"Kozachkov","year":"2022","journal-title":"PLoS Comput. Biol."},{"key":"ref48","doi-asserted-by":"publisher","first-page":"774","DOI":"10.1016\/j.neuron.2019.02.005","article-title":"Ketamine: a paradigm shift for depression research and treatment","volume":"101","author":"Krystal","year":"2019","journal-title":"Neuron"},{"key":"ref49","doi-asserted-by":"publisher","first-page":"e2305772120","DOI":"10.1073\/pnas.2305772120","article-title":"Ketamine and the neurobiology of depression: toward next-generation rapid-acting antidepressant treatments","volume":"120","author":"Krystal","year":"2023","journal-title":"Proc. Natl. Acad. Sci."},{"key":"ref50","doi-asserted-by":"publisher","first-page":"1088","DOI":"10.1016\/j.jbtep.2012.05.007","article-title":"Effects of depression on reward-based decision making and variability of action in probabilistic learning","volume":"43","author":"Kunisato","year":"2012","journal-title":"J. Behav. Ther. Exp. Psychiatry"},{"key":"ref51","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1016\/j.neunet.2019.10.014","article-title":"A review on neural network models of schizophrenia and autism spectrum disorder","volume":"122","author":"Lanillos","year":"2020","journal-title":"Neural Netw."},{"key":"ref52","doi-asserted-by":"publisher","first-page":"856","DOI":"10.1016\/j.bpsc.2021.02.001","article-title":"Artificial intelligence for mental health care: clinical applications, barriers, facilitators, and artificial wisdom","volume":"6","author":"Lee","year":"2021","journal-title":"Biol. Psychiatry Cogn. Neurosci. Neuroimaging"},{"key":"ref53","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1016\/j.jbtep.2010.02.003","article-title":"Delay discounting and future-directed thinking in anhedonic individuals","volume":"41","author":"Lempert","year":"2010","journal-title":"J. Behav. Ther. Exp. Psychiatry"},{"key":"ref54","doi-asserted-by":"publisher","first-page":"e40316","DOI":"10.1371\/journal.pone.0040316","article-title":"Victims\u2019 time discounting 2.5 years after the Wenchuan earthquake: An ERP study","volume":"7","author":"Li","year":"2012","journal-title":"PLoS One"},{"key":"ref55","doi-asserted-by":"publisher","first-page":"959","DOI":"10.1126\/science.1190287","article-title":"mTOR-dependent synapse formation underlies the rapid antidepressant effects of NMDA antagonists","volume":"329","author":"Li","year":"2010","journal-title":"Science"},{"key":"ref56","doi-asserted-by":"publisher","first-page":"754","DOI":"10.1016\/j.biopsych.2010.12.015","article-title":"Glutamate N-methyl-D-aspartate receptor antagonists rapidly reverse behavioral and synaptic deficits caused by chronic stress exposure","volume":"69","author":"Li","year":"2011","journal-title":"Biol. Psychiatry"},{"key":"ref57","doi-asserted-by":"publisher","first-page":"11","DOI":"10.3389\/fncel.2017.00305","article-title":"From serotonin to neuroplasticity: evolvement of theories for major depressive disorder","volume":"11","author":"Liu","year":"2017","journal-title":"Front. Cell. Neurosci."},{"key":"ref58","doi-asserted-by":"publisher","first-page":"664535","DOI":"10.3389\/fcell.2021.664535","article-title":"A selective review of the excitatory-inhibitory imbalance in schizophrenia: underlying biology, genetics, microcircuits, and symptoms","volume":"9","author":"Liu","year":"2021","journal-title":"Front. Cell Dev. Biol."},{"key":"ref59","doi-asserted-by":"publisher","first-page":"3170","DOI":"10.1016\/j.celrep.2018.05.022","article-title":"Psychedelics promote structural and functional neural plasticity","volume":"23","author":"Ly","year":"2018","journal-title":"Cell Rep."},{"key":"ref60","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1016\/j.cognition.2012.06.011","article-title":"Depressive symptoms enhance loss-minimization, but attenuate gain-maximization in history-dependent decision-making","volume":"125","author":"Maddox","year":"2012","journal-title":"Cognition"},{"key":"ref61","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1038\/nn.2723","article-title":"From reinforcement learning models to psychiatric and neurological disorders","volume":"14","author":"Maia","year":"2011","journal-title":"Nat. Neurosci."},{"key":"ref62","doi-asserted-by":"publisher","first-page":"a005751","DOI":"10.1101\/cshperspect.a005751","article-title":"Synapses and memory storage","volume":"4","author":"Mayford","year":"2012","journal-title":"Cold Spring Harb. Perspect. Biol."},{"key":"ref63","doi-asserted-by":"publisher","first-page":"17","DOI":"10.3389\/fnins.2023.1200842","article-title":"Accounting for multiscale processing in adaptive real-world decision-making via the hippocampus","volume":"17","author":"Mehrotra","year":"2023","journal-title":"Front. Neurosci."},{"key":"ref64","doi-asserted-by":"publisher","first-page":"12350","DOI":"10.1038\/s41598-021-91726-x","article-title":"Dendritic spines are lost in clusters in Alzheimer\u2019s disease","volume":"11","author":"Mijalkov","year":"2021","journal-title":"Sci. Rep."},{"key":"ref65","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref66","doi-asserted-by":"publisher","first-page":"eaat8078","DOI":"10.1126\/science.aat8078","article-title":"Sustained rescue of prefrontal circuit dysfunction by antidepressant-induced spine formation","volume":"364","author":"Moda-Sava","year":"2019","journal-title":"Science"},{"key":"ref67","doi-asserted-by":"publisher","first-page":"3243","DOI":"10.1038\/s41380-022-01661-0","article-title":"The serotonin theory of depression: a systematic umbrella review of the evidence","volume":"28","author":"Moncrieff","year":"2023","journal-title":"Mol. Psychiatry"},{"key":"ref68","doi-asserted-by":"publisher","first-page":"1936","DOI":"10.1523\/JNEUROSCI.16-05-01936.1996","article-title":"A framework for mesencephalic dopamine systems based on predictive Hebbian learning","volume":"16","author":"Montague","year":"1996","journal-title":"J. Neurosci."},{"key":"ref69","doi-asserted-by":"publisher","first-page":"462","DOI":"10.1177\/09637214231194962","article-title":"Leveraging decision science to characterize depression","volume":"32","author":"Mukherjee","year":"2023","journal-title":"Curr. Dir. Psychol. Sci."},{"key":"ref70","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1176\/appi.ajp.2013.13030392","article-title":"Antidepressant efficacy of ketamine in treatment-resistant major depression: a two-site randomized controlled trial","volume":"170","author":"Murrough","year":"2013","journal-title":"Am. J. Psychiatry"},{"key":"ref71","year":""},{"key":"ref72","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1038\/s42256-019-0025-4","article-title":"Reinforcement learning in artificial and biological systems","volume":"1","author":"Neftci","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref73","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1016\/j.neunet.2021.05.030","article-title":"The asymmetric learning rates of murine exploratory behavior in sparse reward environments","volume":"143","author":"Ohta","year":"2021","journal-title":"Neural Netw."},{"key":"ref74","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.brat.2014.03.005","article-title":"An examination of avoidance in major depression in comparison to social anxiety disorder","volume":"56","author":"Ottenbreit","year":"2014","journal-title":"Behav. Res. Ther."},{"key":"ref75","first-page":"30S","article-title":"A review of psychostimulants for adults with depression","volume":"32","author":"Pary","year":"2015","journal-title":"Fed. Pract."},{"key":"ref76","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1016\/j.tips.2008.01.002","article-title":"The neuropharmacology of impulsive behaviour","volume":"29","author":"Pattij","year":"2008","journal-title":"Trends Pharmacol. Sci."},{"key":"ref77","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1001\/jamapsychiatry.2022.0051","article-title":"Reinforcement learning in patients with mood and anxiety disorders vs control individuals: a systematic review and meta-analysis","volume":"79","author":"Pike","year":"2022","journal-title":"JAMA Psychiatry"},{"key":"ref78","doi-asserted-by":"publisher","first-page":"1825","DOI":"10.1017\/S0033291713002584","article-title":"Temporal discounting in major depressive disorder","volume":"44","author":"Pulcu","year":"2014","journal-title":"Psychol. Med."},{"key":"ref79","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2016\/8056370","article-title":"Dendritic spines in depression: what we learned from animal models","volume":"2016","author":"Qiao","year":"2016","journal-title":"Neural Plast."},{"key":"ref80","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1016\/j.obhdp.2004.01.002","article-title":"Time discounting over the lifespan","volume":"94","author":"Read","year":"2004","journal-title":"Organ. Behav. Hum. Decis. Process."},{"key":"ref81","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1016\/j.euroneuro.2021.08.002","article-title":"Differential reinforcement learning responses to positive and negative information in unmedicated individuals with depression","volume":"53","author":"Reinen","year":"2021","journal-title":"Eur. Neuropsychopharmacol."},{"key":"ref82","doi-asserted-by":"publisher","first-page":"1079","DOI":"10.1177\/0269881120919957","article-title":"Efficacy of psychedelic treatments on depressive symptoms: a meta-analysis","volume":"34","author":"Romeo","year":"2020","journal-title":"J. Psychopharmacol. Oxf. Engl."},{"key":"ref83","doi-asserted-by":"publisher","first-page":"1147","DOI":"10.1093\/brain\/awx025","article-title":"Neural mechanisms of reinforcement learning in unmedicated patients with major depressive disorder","volume":"140","author":"Rothkirch","year":"2017","journal-title":"Brain"},{"key":"ref84","doi-asserted-by":"publisher","first-page":"1220","DOI":"10.1177\/0269881116679368","article-title":"Psychedelics in the treatment of unipolar mood disorders: a systematic review","volume":"30","author":"Rucker","year":"2016","journal-title":"J. Psychopharmacol. (Oxf.)"},{"key":"ref85","doi-asserted-by":"publisher","first-page":"36","DOI":"10.3389\/fnsyn.2020.00036","article-title":"Dendritic spine plasticity: function and mechanisms","volume":"12","author":"Runge","year":"2020","journal-title":"Front. Synaptic Neurosci."},{"key":"ref86","doi-asserted-by":"publisher","first-page":"13798","DOI":"10.1038\/s41598-018-31730-w","article-title":"Major depression impairs the use of reward values for decision-making","volume":"8","author":"Rupprechter","year":"2018","journal-title":"Sci. Rep."},{"key":"ref87","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1016\/j.neubiorev.2018.10.008","article-title":"Using rodents to model abnormal sensitivity to feedback in depression","volume":"95","author":"Rygula","year":"2018","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref88","doi-asserted-by":"publisher","first-page":"604","DOI":"10.1038\/s41586-020-03051-4","article-title":"Mastering Atari, go, chess and shogi by planning with a learned model","volume":"588","author":"Schrittwieser","year":"2020","journal-title":"Nature"},{"key":"ref89","doi-asserted-by":"publisher","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","article-title":"A neural substrate of prediction and reward","volume":"275","author":"Schultz","year":"1997","journal-title":"Science"},{"key":"ref90","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1162\/0899766053011546","article-title":"A computational model of the functional role of the ventral-striatal D2 receptor in the expression of previously acquired behaviors","volume":"17","author":"Smith","year":"2005","journal-title":"Neural Comput."},{"key":"ref91","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1016\/j.jad.2009.01.034","article-title":"fMRI of alterations in reward selection, anticipation, and feedback in major depressive disorder","volume":"118","author":"Smoski","year":"2009","journal-title":"J. Affect. Disord."},{"key":"ref92","doi-asserted-by":"publisher","first-page":"3336","DOI":"10.1038\/s41380-020-00896-z","article-title":"Distinct non-inflammatory signature of microglia in post-mortem brain tissue of patients with major depressive disorder","volume":"26","author":"Snijders","year":"2021","journal-title":"Mol. Psychiatry"},{"key":"ref93","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1016\/j.neurobiolaging.2018.04.006","article-title":"Age affects reinforcement learning through dopamine-based learning imbalance and high decision noise\u2014not through parkinsonian mechanisms","volume":"68","author":"Sojitra","year":"2018","journal-title":"Neurobiol. Aging"},{"key":"ref94","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1111\/jnc.13962","article-title":"Serotonin 5-HT7 receptor increases the density of dendritic spines and facilitates synaptogenesis in forebrain neurons","volume":"141","author":"Speranza","year":"2017","journal-title":"J. Neurochem."},{"key":"ref95","doi-asserted-by":"publisher","first-page":"114222","DOI":"10.1016\/j.bbr.2022.114222","article-title":"Atypical play experiences in the juvenile period has an impact on the development of the medial prefrontal cortex in both male and female rats","volume":"439","author":"Stark","year":"2023","journal-title":"Behav. Brain Res."},{"key":"ref96","volume-title":"Reinforcement learning: An introduction","author":"Sutton","year":"2018"},{"key":"ref97","doi-asserted-by":"publisher","first-page":"409","DOI":"10.1001\/archpsyc.59.5.409","article-title":"Probing brain reward system function in major depressive disorder: altered response to Dextroamphetamine","volume":"59","author":"Tremblay","year":"2002","journal-title":"Arch. Gen. Psychiatry"},{"key":"ref98","doi-asserted-by":"publisher","first-page":"748370","DOI":"10.3389\/fninf.2021.748370","article-title":"Modeling neurodegeneration in silico with deep learning","volume":"15","author":"Tuladhar","year":"2021","journal-title":"Front. Neuroinform."},{"key":"ref99","doi-asserted-by":"publisher","first-page":"4696","DOI":"10.1017\/S0033291722001593","article-title":"Contextual influence of reinforcement learning performance of depression: evidence for a negativity bias?","volume":"53","author":"Vandendriessche","year":"2023","journal-title":"Psychol. Med."},{"key":"ref100","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1038\/s41586-019-1724-z","article-title":"Grandmaster level in StarCraft II using multi-agent reinforcement learning","volume":"575","author":"Vinyals","year":"2019","journal-title":"Nature"},{"key":"ref101","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1186\/s12974-022-02492-0","article-title":"Microglia in depression: an overview of microglia in the pathogenesis and treatment of depression","volume":"19","author":"Wang","year":"2022","journal-title":"J. Neuroinflammation"},{"key":"ref102","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1007\/BF03395661","article-title":"Temporal discounting of hypothetical monetary rewards by adolescents, adults, and older adults","volume":"59","author":"Whelan","year":"2009","journal-title":"Psychol. Rec."},{"key":"ref103","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1016\/j.neuroscience.2016.04.021","article-title":"Opposing effects of acute and chronic d-amphetamine on decision-making in rats","volume":"345","author":"Wong","year":"2017","journal-title":"Neuroscience"},{"key":"ref104","volume-title":"The ICD-10 classification of mental and behavioural disorders: diagnostic criteria for research","year":"1993"},{"key":"ref105","doi-asserted-by":"publisher","first-page":"3","DOI":"10.3109\/03009734.2012.724118","article-title":"Acute administration of ketamine in rats increases hippocampal BDNF and mTOR levels during forced swimming test","volume":"118","author":"Yang","year":"2013","journal-title":"Ups. J. Med. Sci."},{"key":"ref106","doi-asserted-by":"publisher","first-page":"637","DOI":"10.1016\/j.tins.2015.08.001","article-title":"Depression as a microglial disease","volume":"38","author":"Yirmiya","year":"2015","journal-title":"Trends Neurosci."},{"key":"ref107","doi-asserted-by":"publisher","first-page":"856","DOI":"10.1001\/archpsyc.63.8.856","article-title":"A randomized trial of an N-methyl-D-aspartate antagonist in treatment-resistant major depression","volume":"63","author":"Zarate","year":"2006","journal-title":"Arch. Gen. Psychiatry"},{"key":"ref108","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1016\/j.eurpsy.2013.10.005","article-title":"Ketamine-induced antidepressant effects are associated with AMPA receptors-mediated upregulation of mTOR and BDNF in rat hippocampus and prefrontal cortex","volume":"29","author":"Zhou","year":"2014","journal-title":"Eur. Psychiatry"},{"key":"ref109","doi-asserted-by":"publisher","first-page":"93","DOI":"10.3389\/fnbeh.2019.00093","article-title":"Treadmill exercise reverses depression model-induced alteration of dendritic spines in the brain areas of mood circuit","volume":"13","author":"Zhuang","year":"2019","journal-title":"Front. Behav. Neurosci."}],"container-title":["Frontiers in Computational Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1466364\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T04:43:58Z","timestamp":1730868238000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1466364\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,6]]},"references-count":109,"alternative-id":["10.3389\/fncom.2024.1466364"],"URL":"https:\/\/doi.org\/10.3389\/fncom.2024.1466364","relation":{},"ISSN":["1662-5188"],"issn-type":[{"value":"1662-5188","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,6]]},"article-number":"1466364"}}