{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T22:50:18Z","timestamp":1791327018059,"version":"4.3.1"},"reference-count":63,"publisher":"Institute of Mathematical Statistics","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Ann. Appl. Stat."],"published-print":{"date-parts":[[2025,12,1]]},"abstract":"Major depressive disorder (MDD), a leading cause of disability and mortality, is associated with reward-processing abnormalities and concentration difficulties. Motivated by the probabilistic reward task from the Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) study, we propose a novel framework that integrates the reinforcement learning (RL) model and drift-diffusion model (DDM) to jointly analyze reward-based decision making with response times. To account for emerging evidence suggesting that decision making may alternate between multiple interleaved strategies, we model latent state switching using a hidden Markov model (HMM). In the engaged state, decisions follow an RL-DDM, simultaneously capturing reward processing, decision dynamics and temporal structure. In contrast, in the lapsed state, decision making is modeled using a simplified DDM, where specific parameters are fixed to approximate random guessing with equal probability. The proposed method is implemented using a computationally efficient generalized expectation-maximization (EM) algorithm with forward-backward procedures. Through extensive numerical studies, we demonstrate that our proposed method outperforms competing approaches across various reward-generating distributions, under both strategy-switching and nonswitching scenarios, as well as in the presence of input perturbations. When applied to the EMBARC study, our framework reveals that MDD patients exhibit lower overall engagement than healthy controls and experience longer responses when they do engage. Additionally, we show that neuroimaging measures of brain activities are associated with decision-making characteristics in the engaged state but not in the lapsed state, providing evidence of a brain-behavior association specific to the engaged state.<\/jats:p>","DOI":"10.1214\/25-aoas2112","type":"journal-article","created":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T15:55:15Z","timestamp":1764950115000},"page":"3372-3393","source":"Crossref","is-referenced-by-count":0,"title":["Joint modeling for learning decision-making dynamics in behavioral experiments"],"prefix":"10.1214","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1879-5887","authenticated-orcid":false,"given":"Yuan","family":"Bian","sequence":"first","affiliation":[{"name":"Department of Biostatistics, Columbia University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2001-9879","authenticated-orcid":false,"given":"Xingche","family":"Guo","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of Connecticut"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1510-3315","authenticated-orcid":false,"given":"Yuanjia","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Psychiatry, Columbia University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"108","reference":[{"key":"21","doi-asserted-by":"crossref","unstructured":"Goulden, N.<\/small>, Khusnulina, A.<\/small>, Davis, N. J.<\/small>, Bracewell, R. M.<\/small>, Bokde, A. L.<\/small>, McNulty, J. P.<\/small> and Mullins, P. G.<\/small> (2014). The salience network is responsible for switching between the default mode network and the central executive network: Replication from DCM. NeuroImage<\/i> 99<\/b> 180\u2013190.","DOI":"10.1016\/j.neuroimage.2014.05.052"},{"key":"49","unstructured":"SEELEY, W. W.<\/small>, MENON, V.<\/small>, SCHATZBERG, A. F.<\/small>, KELLER, J.<\/small>, GLOVER, G. H.<\/small>, KENNA, H.<\/small>, REISS, A. L.<\/small> and GREICIUS, M. D.<\/small> (2007). Dissociable intrinsic connectivity networks for salience processing and executive control. J. Neurosci.<\/i> 27<\/b> 2349\u20132356."},{"key":"10","doi-asserted-by":"publisher","unstructured":"COLE, M. W.<\/small>, REYNOLDS, J. R.<\/small>, POWER, J. D.<\/small>, REPOVS, G.<\/small>, ANTICEVIC, A.<\/small> and BRAVER, T. S.<\/small> (2013). Multi-task connectivity reveals flexible hubs for adaptive task control. Nat. Neurosci.<\/i> 16<\/b> 1348\u20131355.","DOI":"10.1038\/nn.3470"},{"key":"5","doi-asserted-by":"crossref","unstructured":"BENJAMINI, Y.<\/small> and HOCHBERG, Y.<\/small> (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B, Methodol.<\/i> 57<\/b> 289\u2013300.","DOI":"10.1111\/j.2517-6161.1995.tb02031.x"},{"key":"12","doi-asserted-by":"crossref","unstructured":"DEMPSTER, A. P.<\/small>, LAIRD, N. M.<\/small> and RUBIN, D. B.<\/small> (1977). Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. Ser. B, Methodol.<\/i> 39<\/b> 1\u201338. With discussion.","DOI":"10.1111\/j.2517-6161.1977.tb01600.x"},{"key":"1","doi-asserted-by":"crossref","unstructured":"ABBEEL, P.<\/small> and NG, A. Y.<\/small> (2004). Apprenticeship learning via inverse reinforcement learning. In The Twenty-First International Conference on Machine Learning<\/i>.","DOI":"10.1145\/1015330.1015430"},{"key":"2","doi-asserted-by":"crossref","unstructured":"ASHWOOD, Z. C.<\/small>, ROY, N. A.<\/small>, STONE, I. R.<\/small>, URAI, A. E.<\/small>, CHURCHLAND, A. K.<\/small>, POUGET, A.<\/small> and PILLOW, J. W.<\/small> (2022). Mice alternate between discrete strategies during perceptual decision-making. Nat. Neurosci.<\/i> 25<\/b> 201\u2013212.","DOI":"10.1038\/s41593-021-01007-z"},{"key":"3","doi-asserted-by":"crossref","unstructured":"BADDELEY, A. D.<\/small> (1968). A 3 min reasoning test based on grammatical transformation. Psychon. Sci.<\/i> 10<\/b> 341\u2013342.","DOI":"10.3758\/BF03331551"},{"key":"4","doi-asserted-by":"publisher","unstructured":"BAUM, L. E.<\/small>, PETRIE, T.<\/small>, SOULES, G.<\/small> and WEISS, N.<\/small> (1970). A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Statist.<\/i> 41<\/b> 164\u2013171.","DOI":"10.1214\/aoms\/1177697196"},{"key":"6","doi-asserted-by":"crossref","unstructured":"BIAN, Y.<\/small>, GUO, X.<\/small> and WANG, Y.<\/small> (2025). Supplement to \u201cJoint modeling for learning decision-making dynamics in behavioral experiments.\u201d https:\/\/doi.org\/10.1214\/25-AOAS2112SUPP<\/a>","DOI":"10.1214\/25-AOAS2112SUPP"},{"key":"7","doi-asserted-by":"publisher","unstructured":"BLURTON, S. P.<\/small>, KESSELMEIER, M.<\/small> and GONDAN, M.<\/small> (2012). Fast and accurate calculations for cumulative first-passage time distributions in Wiener diffusion models. J. Math. Psych.<\/i> 56<\/b> 470\u2013475.","DOI":"10.1016\/j.jmp.2012.09.002"},{"key":"8","doi-asserted-by":"crossref","unstructured":"BOTVINICK, M. M.<\/small>, BRAVER, T. S.<\/small>, BARCH, D. M.<\/small>, CARTER, C. S.<\/small> and COHEN, J. D.<\/small> (2001). Conflict monitoring and cognitive control. Psychol. Rev.<\/i> 108<\/b> 624\u2013652.","DOI":"10.1037\/0033-295X.108.3.624"},{"key":"9","doi-asserted-by":"crossref","unstructured":"CALHOUN, A. J.<\/small>, PILLOW, J. W.<\/small> and MURTHY, M.<\/small> (2019). Unsupervised identification of the internal states that shape natural behavior. Nat. Neurosci.<\/i> 22<\/b> 2040\u20132049.","DOI":"10.1038\/s41593-019-0533-x"},{"key":"11","doi-asserted-by":"crossref","unstructured":"CUIJPERS, P.<\/small> and SCHOEVERS, R. A.<\/small> (2004). Increased mortality in depressive disorders: A review. Curr. Psychiatry Rep.<\/i> 6<\/b> 430\u2013437.","DOI":"10.1007\/s11920-004-0007-y"},{"key":"13","unstructured":"DILLON, D. G.<\/small>, BELLEAU, E. L.<\/small>, ORIGLIO, J.<\/small>, MCKEE, M.<\/small>, JAHAN, A.<\/small>, MEYER, A.<\/small>, SOUTHER, M. K.<\/small>, BRUNNER, D.<\/small>, KUHN, M.<\/small> et al. (2024). Using drift diffusion and RL models to disentangle effects of depression on decision-making vs. learning in the probabilistic reward task. Comput. Psychiatry<\/i> 8<\/b> 46\u201369."},{"key":"14","doi-asserted-by":"crossref","unstructured":"DOSENBACH, N. U.<\/small>, FAIR, D. A.<\/small>, COHEN, A. L.<\/small>, SCHLAGGAR, B. L.<\/small> and PETERSEN, S. E.<\/small> (2008). A dual-networks architecture of top-down control. Trends Cogn. Sci.<\/i> 12<\/b> 99\u2013105.","DOI":"10.1016\/j.tics.2008.01.001"},{"key":"15","doi-asserted-by":"crossref","unstructured":"FONTANESI, L.<\/small>, GLUTH, S.<\/small>, SPEKTOR, M. S.<\/small> and RIESKAMP, J.<\/small> (2019). A reinforcement learning diffusion decision model for value-based decisions. Psychon. Bull. Rev.<\/i> 26<\/b> 1099\u20131121.","DOI":"10.3758\/s13423-018-1554-2"},{"key":"16","doi-asserted-by":"crossref","unstructured":"FONTANESI, L.<\/small>, PALMINTERI, S.<\/small> and LEBRETON, M.<\/small> (2019). Decomposing the effects of context valence and feedback information on speed and accuracy during reinforcement learning: A meta-analytical approach using diffusion decision modeling. Cogn. Affect. Behav. Neurosci.<\/i> 19<\/b> 490\u2013502.","DOI":"10.3758\/s13415-019-00723-1"},{"key":"17","unstructured":"FONZO, G. A.<\/small>, ETKIN, A.<\/small>, ZHANG, Y.<\/small>, WU, W.<\/small>, COOPER, C.<\/small>, CHIN-FATT, C.<\/small>, JHA, M. K.<\/small>, TROMBELLO, J.<\/small>, DECKERSBACH, T.<\/small> et al. (2019). Brain regulation of emotional conflict predicts antidepressant treatment response for depression. Nat. Hum. Behav.<\/i> 3<\/b> 1319\u20131331."},{"key":"18","doi-asserted-by":"crossref","unstructured":"FRANK, M. J.<\/small>, DOLL, B. B.<\/small>, OAS-TERPSTRA, J.<\/small> and MORENO, F.<\/small> (2009). Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat. Neurosci.<\/i> 12<\/b> 1062\u20131068.","DOI":"10.1038\/nn.2342"},{"key":"19","doi-asserted-by":"crossref","unstructured":"FRENSCH, P. A.<\/small> and R\u00dcNGER, D.<\/small> (2003). Implicit learning. Curr. Dir. Psychol. Sci.<\/i> 12<\/b> 13\u201318.","DOI":"10.1111\/1467-8721.01213"},{"key":"20","doi-asserted-by":"crossref","unstructured":"FRIEDRICH, M. J.<\/small> (2017). Depression is the leading cause of disability around the world. JAMA<\/i> 317<\/b> 1517.","DOI":"10.1001\/jama.2017.3826"},{"key":"22","doi-asserted-by":"publisher","unstructured":"GUO, X.<\/small>, ZENG, D.<\/small> and WANG, Y.<\/small> (2024). A semiparametric inverse reinforcement learning approach to characterize decision making for mental disorders. J. Amer. Statist. Assoc.<\/i> 119<\/b> 27\u201338.","DOI":"10.1080\/01621459.2023.2261184"},{"key":"23","doi-asserted-by":"publisher","unstructured":"GUO, X.<\/small>, ZENG, D.<\/small> and WANG, Y.<\/small> (2025). HMM for discovering decision-making dynamics using reinforcement learning experiments. Biostatistics<\/i> 26<\/b> Paper No. kxae033, 16.","DOI":"10.1093\/biostatistics\/kxae033"},{"key":"24","doi-asserted-by":"crossref","unstructured":"HOLMES, A. J.<\/small>, BOGDAN, R.<\/small> and PIZZAGALLI, D. A.<\/small> (2010). Serotonin transporter genotype and action monitoring dysfunction: A possible substrate underlying increased vulnerability to depression. Neuropsychopharmacology<\/i> 35<\/b> 1186\u20131197.","DOI":"10.1038\/npp.2009.223"},{"key":"25","doi-asserted-by":"publisher","unstructured":"HORROCKS, J.<\/small> and THOMPSON, M. E.<\/small> (2004). Modeling event times with multiple outcomes using the Wiener process with drift. Lifetime Data Anal.<\/i> 10<\/b> 29\u201349.","DOI":"10.1023\/B:LIDA.0000019254.29153.1a"},{"key":"26","doi-asserted-by":"crossref","unstructured":"HUYS, Q. J.<\/small>, PIZZAGALLI, D. A.<\/small>, BOGDAN, R.<\/small> and DAYAN, P.<\/small> (2013). Mapping anhedonia onto reinforcement learning: A behavioural meta-analysis. Biol. Mood Anxiety Disord.<\/i> 3<\/b> 1\u201316.","DOI":"10.1186\/2045-5380-3-12"},{"key":"27","doi-asserted-by":"crossref","unstructured":"IIGAYA, K.<\/small>, FONSECA, M. S.<\/small>, MURAKAMI, M.<\/small>, MAINEN, Z. F.<\/small> and DAYAN, P.<\/small> (2018). An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals. Nat. Commun.<\/i> 9<\/b> 2477.","DOI":"10.1038\/s41467-018-04840-2"},{"key":"28","doi-asserted-by":"crossref","unstructured":"KRAJBICH, I.<\/small>, BARTLING, B.<\/small>, HARE, T.<\/small> and FEHR, E.<\/small> (2015). Rethinking fast and slow based on a critique of reaction-time reverse inference. Nat. Commun.<\/i> 6<\/b> 7455.","DOI":"10.1038\/ncomms8455"},{"key":"29","unstructured":"LAWLOR, V. M.<\/small>, WEBB, C. A.<\/small>, WIECKI, T. V.<\/small>, FRANK, M. J.<\/small>, TRIVEDI, M.<\/small>, PIZZAGALLI, D. A.<\/small> and DILLON, D. G.<\/small> (2020). Dissecting the impact of depression on decision-making. Psychol. Med.<\/i> 50<\/b> 1613\u20131622."},{"key":"30","doi-asserted-by":"crossref","unstructured":"LEECH, R.<\/small> and SHARP, D. J.<\/small> (2014). The role of the posterior cingulate cortex in cognition and disease. Brain<\/i> 137<\/b> 12\u201332.","DOI":"10.1093\/brain\/awt162"},{"key":"31","doi-asserted-by":"publisher","unstructured":"LI, J.-J.<\/small>, SHI, C.<\/small>, LI, L.<\/small> and COLLINS, A. G. E.<\/small> (2024). Dynamic noise estimation: A generalized method for modeling noise fluctuations in decision-making. J. Math. Psych.<\/i> 119<\/b> Paper No. 102842, 11.","DOI":"10.1016\/j.jmp.2024.102842"},{"key":"32","doi-asserted-by":"publisher","unstructured":"MARLEY, A. A. J.<\/small> and COLONIUS, H.<\/small> (1992). The \u201chorse race\u201d random utility model for choice probabilities and reaction times, and its competing risks interpretation. J. Math. Psych.<\/i> 36<\/b> 1\u201320.","DOI":"10.1016\/0022-2496(92)90050-H"},{"key":"33","doi-asserted-by":"crossref","unstructured":"MENON, V.<\/small> and UDDIN, L. Q.<\/small> (2010). Saliency, switching, attention and control: A network model of insula function. Brain Struct. Funct.<\/i> 214<\/b> 655\u2013667.","DOI":"10.1007\/s00429-010-0262-0"},{"key":"34","doi-asserted-by":"crossref","unstructured":"MILETI\u0106, S.<\/small>, BOAG, R. J.<\/small> and FORSTMANN, B. U.<\/small> (2020). Mutual benefits: Combining reinforcement learning with sequential sampling models. Neuropsychologia<\/i> 136<\/b> 107261.","DOI":"10.1016\/j.neuropsychologia.2019.107261"},{"key":"35","doi-asserted-by":"publisher","unstructured":"NAVARRO, D. J.<\/small> and FUSS, I. G.<\/small> (2009). Fast and accurate calculations for first-passage times in Wiener diffusion models. J. Math. Psych.<\/i> 53<\/b> 222\u2013230.","DOI":"10.1016\/j.jmp.2009.02.003"},{"key":"36","doi-asserted-by":"crossref","unstructured":"NELSON, J. C.<\/small> and CHARNEY, D. S.<\/small> (1981). The symptoms of major depressive illness. Amer. J. Psychiatry<\/i> 138<\/b> 1\u201313.","DOI":"10.1176\/ajp.138.1.1"},{"key":"37","unstructured":"NG, A. Y.<\/small> and RUSSELL, S.<\/small> (2000). Algorithms for inverse reinforcement learning. In The Seventeenth International Conference on Machine Learning<\/i>."},{"key":"38","doi-asserted-by":"crossref","unstructured":"O\u2019DOHERTY, J. P.<\/small>, DAYAN, P.<\/small>, FRISTON, K.<\/small>, CRITCHLEY, H.<\/small> and DOLAN, R. J.<\/small> (2003). Temporal difference models and reward-related learning in the human brain. Neuron<\/i> 38<\/b> 329\u2013337.","DOI":"10.1016\/S0896-6273(03)00169-7"},{"key":"39","doi-asserted-by":"crossref","unstructured":"PEDERSEN, M. L.<\/small>, FRANK, M. J.<\/small> and BIELE, G.<\/small> (2017). The drift diffusion model as the choice rule in reinforcement learning. Psychon. Bull. Rev.<\/i> 24<\/b> 1234\u20131251.","DOI":"10.3758\/s13423-016-1199-y"},{"key":"40","doi-asserted-by":"crossref","unstructured":"PIZZAGALLI, D. A.<\/small>, JAHN, A. L.<\/small> and O\u2019SHEA, J. P.<\/small> (2005). Toward an objective characterization of an anhedonic phenotype: A signal-detection approach. Biol. Psychiatry<\/i> 57<\/b> 319\u2013327.","DOI":"10.1016\/j.biopsych.2004.11.026"},{"key":"41","doi-asserted-by":"crossref","unstructured":"RATCLIFF, R.<\/small> (1978). A theory of memory retrieval. Psychol. Rev.<\/i> 8<\/b> 59\u2013108.","DOI":"10.1037\/0033-295X.85.2.59"},{"key":"42","doi-asserted-by":"crossref","unstructured":"RATCLIFF, R.<\/small> and CHILDERS, R.<\/small> (2015). Individual differences and fitting methods for the two-choice diffusion model of decision making. Decision<\/i> 2<\/b> 237\u2013279.","DOI":"10.1037\/dec0000030"},{"key":"43","doi-asserted-by":"crossref","unstructured":"RATCLIFF, R.<\/small> and MCKOON, G.<\/small> (2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Comput.<\/i> 20<\/b> 873\u2013922.","DOI":"10.1162\/neco.2008.12-06-420"},{"key":"44","unstructured":"RESCORLA, R. A.<\/small> and WAGNER, A. R.<\/small> (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Classical Conditioning II<\/i>: Current Research and Theory<\/i> (A. H. Black and W. F. Prokasy, eds.) 64\u201392. Appleton-Century-Crofts, New York."},{"key":"45","doi-asserted-by":"crossref","unstructured":"RICE, F.<\/small>, RIGLIN, L.<\/small>, LOMAX, T.<\/small>, SOUTER, E.<\/small>, POTTER, R.<\/small>, SMITH, D.<\/small>, THAPAR, A. K.<\/small> and THAPAR, A.<\/small> (2019). Adolescent and adult differences in major depression symptom profiles. J. Affective Disorders<\/i> 243<\/b> 175\u2013181.","DOI":"10.1016\/j.jad.2018.09.015"},{"key":"46","doi-asserted-by":"crossref","unstructured":"ROCK, P. L.<\/small>, ROISER, J. P.<\/small>, RIEDEL, W. J.<\/small> and BLACKWELL, A. D.<\/small> (2014). Cognitive impairment in depression: A systematic review and meta-analysis. Psychol. Med.<\/i> 44<\/b> 2029\u20132040.","DOI":"10.1017\/S0033291713002535"},{"key":"47","unstructured":"ROSS, S.<\/small> and BAGNELL, D.<\/small> (2010). Efficient reductions for imitation learning. In The Thirteenth International Conference on Artificial Intelligence and Statistics<\/i>."},{"key":"48","doi-asserted-by":"crossref","unstructured":"SCHULTZ, W.<\/small>, DAYAN, P.<\/small> and MONTAGUE, P. R.<\/small> (1997). A neural substrate of prediction and reward. Science<\/i> 275<\/b> 1593\u20131599.","DOI":"10.1126\/science.275.5306.1593"},{"key":"50","doi-asserted-by":"crossref","unstructured":"SHAHAR, N.<\/small>, HAUSER, T. U.<\/small>, MOUTOUSSIS, M.<\/small>, MORAN, R.<\/small>, KERAMATI, M.<\/small>, CONSORTIUM, N.<\/small> and DOLAN, R. J.<\/small> (2019). Improving the reliability of model-based decision-making estimates in the two-stage decision task with reaction-times and drift-diffusion modeling. PLoS Comput. Biol.<\/i> 15<\/b> e1006803.","DOI":"10.1371\/journal.pcbi.1006803"},{"key":"51","doi-asserted-by":"publisher","unstructured":"SMITH, P. L.<\/small> (2000). Stochastic dynamic models of response time and accuracy: A foundational primer. J. Math. Psych.<\/i> 44<\/b> 408\u2013463.","DOI":"10.1006\/jmps.1999.1260"},{"key":"53","unstructured":"THORNE, D. R.<\/small>, GENSER, S. G.<\/small>, SING, H. C.<\/small> and HEGGE, F. W.<\/small> (1985). The Walter Reed performance assessment battery. Neurobehav. Toxicol. Teratol.<\/i> 7<\/b> 415\u2013418."},{"key":"54","doi-asserted-by":"crossref","unstructured":"TORABI, F.<\/small>, WARNELL, G.<\/small> and STONE, P.<\/small> (2018). Behavioral cloning from observation. In The Twenty-Seventh International Joint Conference on Artificial Intelligence<\/i>.","DOI":"10.24963\/ijcai.2018\/687"},{"key":"55","unstructured":"TRIVEDI, M. H.<\/small>, MCGRATH, P. J.<\/small>, FAVA, M.<\/small>, PARSEY, R. V.<\/small>, KURIAN, B. T.<\/small>, PHILLIPS, M. L.<\/small>, OQUENDO, M. A.<\/small>, BRUDER, G.<\/small>, PIZZAGALLI, D.<\/small> et al. (2016). Establishing moderators and biosignatures of antidepressant response in clinical care (EMBARC): Rationale and design. J. Psychiatr. Res.<\/i> 78<\/b> 11\u201323."},{"key":"56","doi-asserted-by":"publisher","unstructured":"TUERLINCKX, F.<\/small> and DE BOECK, P.<\/small> (2005). Two interpretations of the discrimination parameter. Psychometrika<\/i> 70<\/b> 629\u2013650.","DOI":"10.1007\/s11336-000-0810-3"},{"key":"57","doi-asserted-by":"crossref","unstructured":"UDDIN, L. Q.<\/small> (2015). Salience processing and insular cortical function and dysfunction. Nat. Rev. Neurosci.<\/i> 16<\/b> 55\u201361.","DOI":"10.1038\/nrn3857"},{"key":"58","doi-asserted-by":"crossref","unstructured":"WAELTI, P.<\/small>, DICKINSON, A.<\/small> and SCHULTZ, W.<\/small> (2001). Dopamine responses comply with basic assumptions of formal learning theory. Nature<\/i> 412<\/b> 43\u201348.","DOI":"10.1038\/35083500"},{"key":"59","doi-asserted-by":"publisher","unstructured":"WHITE, C. N.<\/small>, RATCLIFF, R.<\/small>, VASEY, M. W.<\/small> and MCKOON, G.<\/small> (2010). Using diffusion models to understand clinical disorders. J. Math. Psych.<\/i> 54<\/b> 39\u201352.","DOI":"10.1016\/j.jmp.2010.01.004"},{"key":"60","unstructured":"WHITEFORD, H. A.<\/small>, DEGENHARDT, L.<\/small>, REHM, J.<\/small>, BAXTER, A. J.<\/small>, FERRARI, A. J.<\/small>, ERSKINE, H. E.<\/small>, CHARLSON, F. J.<\/small>, NORMAN, R. E.<\/small>, FLAXMAN, A. D.<\/small> et al. (2013). Global burden of disease attributable to mental and substance use disorders: Findings from the Global Burden of Disease study 2010. Lancet<\/i> 382<\/b> 1575\u20131586."},{"key":"61","doi-asserted-by":"crossref","unstructured":"WICKELGREN, W. A.<\/small> (1977). Speed-accuracy tradeoff and information processing dynamics. Acta Psychol.<\/i> 41<\/b> 67\u201385.","DOI":"10.1016\/0001-6918(77)90012-9"},{"key":"62","unstructured":"WORLD HEALTH ORGANIZATION<\/small> (2023). Depressive disorder (depression). Accessed on Jan 5, 2025."},{"key":"63","doi-asserted-by":"crossref","unstructured":"WORTHY, D. A.<\/small>, HAWTHORNE, M. J.<\/small> and OTTO, A. R.<\/small> (2013). Heterogeneity of strategy use in the Iowa gambling task: A comparison of win-stay\/lose-shift and reinforcement learning models. Psychon. Bull. Rev.<\/i> 20<\/b> 364\u2013371.","DOI":"10.3758\/s13423-012-0324-9"},{"key":"52","unstructured":"SUTTON, R. S.<\/small> and BARTO, A. G.<\/small> (2018). Reinforcement Learning<\/i>: An Introduction<\/i>, 2nd ed. Adaptive Computation and Machine Learning<\/i>. MIT Press, Cambridge, MA."}],"container-title":["The Annals of Applied Statistics"],"original-title":[],"deposited":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T21:55:03Z","timestamp":1791323703000},"score":1,"resource":{"primary":{"URL":"https:\/\/projecteuclid.org\/journals\/annals-of-applied-statistics\/volume-19\/issue-4\/Joint-modeling-for-learning-decision-making-dynamics-in-behavioral-experiments\/10.1214\/25-AOAS2112.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,1]]},"references-count":63,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,12,1]]}},"URL":"https:\/\/doi.org\/10.1214\/25-aoas2112","relation":{},"ISSN":["1932-6157","1941-7330"],"issn-type":[{"value":"1932-6157","type":"print"},{"value":"1941-7330","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,1]]}}}