{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T20:24:59Z","timestamp":1783023899809,"version":"3.54.6"},"reference-count":46,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T00:00:00Z","timestamp":1780963200000},"content-version":"vor","delay-in-days":159,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100021783","name":"Courtois Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100021783","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004489","name":"Mitacs","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004489","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,7,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Videogames provide a promising framework to understand brain activity in a rich, engaging, and active environment, in contrast to mostly passive tasks currently dominating the field, such as image viewing. Analyzing videogames neuroimaging data is, however, challenging, and relies on time-intensive manual annotations of game events, based on somewhat arbitrary rules. Here, we introduce an innovative approach using Artificial Neural networks (ANN) and brain encoding techniques to generate activation maps associated with videogame behavior using functional magnetic resonance imaging (fMRI). As individual behavior is highly variable across subjects in complex environments, we hypothesized that ANNs need to account for subject-specific behavior to capture brain dynamics properly. In this study, we used data collected while subjects played Shinobi III: Return of the Ninja Master (an action-platformer released by Sega in 1993), an action-platformer videogame. Using imitation learning, we trained an ANN to play the game while closely replicating the unique gameplay style of individual participants. We found that hidden layers of our imitation learning model successfully encoded task-relevant neural representations, and predicted individual brain dynamics with higher accuracy than models trained on other subjects\u2019 gameplay. Individual-specific models also outperformed several baselines to predict brain activity, such as pixel inputs, or button presses. The highest correlations between layer activations and brain signals were observed in biologically plausible brain areas, that is, somatosensory, attention, and visual networks. This work thus demonstrates that subject-specific imitation models can be trained from scratch and improve brain encoding in an active naturalistic task. Our framework builds on a commercial videogame of unprecedented complexity in the fMRI brain encoding literature. We used a flexible game emulator that supports a broad range of commercial videogames, opening new naturalistic interactive environments for cognitive neuroscience.<\/jats:p>","DOI":"10.1162\/imag.a.1286","type":"journal-article","created":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T20:01:43Z","timestamp":1781035303000},"update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":0,"title":["Behavioral imitation with artificial neural networks leads to personalized models of brain dynamics during videogame play"],"prefix":"10.1162","volume":"4","author":[{"given":"Anirudha","family":"Kemtur","sequence":"first","affiliation":[{"name":"Computer Science Department, Universit\u00e9 de Montr\u00e9al, Montr\u00e9al, Canada"},{"name":"Mila \u2013 Quebec AI Institute, Montr\u00e9al, Canada"},{"name":"Centre de Recherche de l\u2019Institut Universitaire de G\u00e9riatrie de Montr\u00e9al, Montr\u00e9al, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Francois","family":"Paugam","sequence":"additional","affiliation":[{"name":"Computer Science Department, Universit\u00e9 de Montr\u00e9al, Montr\u00e9al, Canada"},{"name":"Mila \u2013 Quebec AI Institute, Montr\u00e9al, Canada"},{"name":"Centre de Recherche de l\u2019Institut Universitaire de G\u00e9riatrie de Montr\u00e9al, Montr\u00e9al, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Basile","family":"Pinsard","sequence":"additional","affiliation":[{"name":"Centre de Recherche de l\u2019Institut Universitaire de G\u00e9riatrie de Montr\u00e9al, Montr\u00e9al, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yann","family":"Harel","sequence":"additional","affiliation":[{"name":"Centre de Recherche de l\u2019Institut Universitaire de G\u00e9riatrie de Montr\u00e9al, Montr\u00e9al, Canada"},{"name":"Psychology Department, Universit\u00e9 de Montr\u00e9al, Montr\u00e9al, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pravish","family":"Sainath","sequence":"additional","affiliation":[{"name":"Computer Science Department, Universit\u00e9 de Montr\u00e9al, Montr\u00e9al, Canada"},{"name":"Mila \u2013 Quebec AI Institute, Montr\u00e9al, Canada"},{"name":"Centre de Recherche de l\u2019Institut Universitaire de G\u00e9riatrie de Montr\u00e9al, Montr\u00e9al, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maximilien Le","family":"Clei","sequence":"additional","affiliation":[{"name":"Centre de Recherche de l\u2019Institut Universitaire de G\u00e9riatrie de Montr\u00e9al, Montr\u00e9al, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julie","family":"Boyle","sequence":"additional","affiliation":[{"name":"Centre de Recherche de l\u2019Institut Universitaire de G\u00e9riatrie de Montr\u00e9al, Montr\u00e9al, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karim","family":"Jerbi","sequence":"additional","affiliation":[{"name":"Computer Science Department, Universit\u00e9 de Montr\u00e9al, Montr\u00e9al, Canada"},{"name":"Mila \u2013 Quebec AI Institute, Montr\u00e9al, Canada"},{"name":"Centre de Recherche de l\u2019Institut Universitaire de G\u00e9riatrie de Montr\u00e9al, Montr\u00e9al, Canada"},{"name":"Psychology Department, Universit\u00e9 de Montr\u00e9al, Montr\u00e9al, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lune","family":"Bellec","sequence":"additional","affiliation":[{"name":"Computer Science Department, Universit\u00e9 de Montr\u00e9al, Montr\u00e9al, Canada"},{"name":"Mila \u2013 Quebec AI Institute, Montr\u00e9al, Canada"},{"name":"Centre de Recherche de l\u2019Institut Universitaire de G\u00e9riatrie de Montr\u00e9al, Montr\u00e9al, Canada"},{"name":"Psychology Department, Universit\u00e9 de Montr\u00e9al, Montr\u00e9al, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2026,7,2]]},"reference":[{"key":"2026070215560599300_IMAG.a.1286-b1","doi-asserted-by":"crossref","first-page":"158","DOI":"10.3389\/fpsyg.2020.00158","article-title":"Investigating the \u201cFlow\u201d experience: Key conceptual and operational issues","volume":"11","author":"Abuhamdeh","year":"2020","journal-title":"Frontiers in Psychology"},{"issue":"1","key":"2026070215560599300_IMAG.a.1286-b2","doi-asserted-by":"crossref","first-page":"170181","DOI":"10.1038\/sdata.2017.181","article-title":"An open resource for transdiagnostic research in pediatric mental health and learning disorders","volume":"4","author":"Alexander","year":"2017","journal-title":"Scientific Data"},{"issue":"1","key":"2026070215560599300_IMAG.a.1286-b3","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1038\/s41593-021-00962-x","article-title":"A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence","volume":"25","author":"Allen","year":"2022","journal-title":"Nature Neuroscience"},{"issue":"12","key":"2026070215560599300_IMAG.a.1286-b4","doi-asserted-by":"crossref","first-page":"3983","DOI":"10.1162\/jocn_a_00033","article-title":"Brain regions engaged by part-and whole-task performance in a video game: A model-based test of the decomposition hypothesis","volume":"23","author":"Anderson","year":"2011","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2026070215560599300_IMAG.a.1286-b5","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1016\/j.visres.2011.08.007","article-title":"Neural bases of selective attention in action video game players","volume":"61","author":"Bavelier","year":"2012","journal-title":"Vision Research"},{"issue":"1","key":"2026070215560599300_IMAG.a.1286-b6","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.neuron.2019.09.031","article-title":"Enhancing attentional control: Lessons from action video games","volume":"104","author":"Bavelier","year":"2019","journal-title":"Neuron"},{"key":"2026070215560599300_IMAG.a.1286-b7","article-title":"Bridging the gap between perception and action: The case for neuroimaging, AI and video games","author":"Bellec","year":"2019","journal-title":"PsyArXiv"},{"key":"2026070215560599300_IMAG.a.1286-b8","article-title":"The Courtois project on neuronal modelling. In","author":"Boyle","year":"2020","journal-title":"26th Annual Meeting of the Organization for Human Brain Mapping"},{"key":"2026070215560599300_IMAG.a.1286-b9","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1038\/s42003-022-03036-1","article-title":"Brains and algorithms partially converge in natural language processing","volume":"5","author":"Caucheteux","year":"2022","journal-title":"Communications Biology"},{"key":"2026070215560599300_IMAG.a.1286-b10","doi-asserted-by":"crossref","DOI":"10.1109\/CEC45853.2021.9505001","article-title":"Behavioral cloning in atari games using a combined variational autoencoder and predictor model. In","author":"Chen","year":"2021","journal-title":"2021 IEEE Congress on Evolutionary Computation (CEC)"},{"issue":"4","key":"2026070215560599300_IMAG.a.1286-b11","doi-asserted-by":"crossref","first-page":"724","DOI":"10.1016\/j.neuron.2020.11.021","article-title":"Using deep reinforcement learning to reveal how the brain encodes abstract state-space representations in high-dimensional environments","volume":"109","author":"Cross","year":"2021","journal-title":"Neuron"},{"issue":"13","key":"2026070215560599300_IMAG.a.1286-b12","doi-asserted-by":"crossref","first-page":"4529","DOI":"10.3390\/app10134529","article-title":"A survey of planning and learning in games","volume":"10","author":"Duarte","year":"2020","journal-title":"Applied Sciences"},{"key":"2026070215560599300_IMAG.a.1286-b13","doi-asserted-by":"crossref","first-page":"116860","DOI":"10.1016\/j.neuroimage.2020.116860","article-title":"Towards clinical applications of movie fMRI","volume":"217","author":"Eickhoff","year":"2020","journal-title":"NeuroImage"},{"issue":"1","key":"2026070215560599300_IMAG.a.1286-b14","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1038\/s41592-018-0235-4","article-title":"fMRIPrep: A robust preprocessing pipeline for functional MRI","volume":"16","author":"Esteban","year":"2019","journal-title":"Nature Methods"},{"key":"2026070215560599300_IMAG.a.1286-b15","doi-asserted-by":"crossref","DOI":"10.1162\/imag_a_00525","article-title":"Alignment of auditory artificial networks with massive individual fMRI brain data leads to generalisable improvements in brain encoding and downstream tasks","author":"Freteault","year":"2025","journal-title":"Imaging Neuroscience"},{"issue":"4","key":"2026070215560599300_IMAG.a.1286-b16","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1006\/nimg.1998.0419","article-title":"Deconvolution of impulse response in event-related BOLD fMRI1","volume":"9","author":"Glover","year":"1999","journal-title":"NeuroImage"},{"issue":"2","key":"2026070215560599300_IMAG.a.1286-b17","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1016\/j.neuron.2018.03.035","article-title":"Functional brain networks are dominated by stable group and individual factors, not cognitive or daily variation","volume":"98","author":"Gratton","year":"2018","journal-title":"Neuron"},{"issue":"11","key":"2026070215560599300_IMAG.a.1286-b18","doi-asserted-by":"crossref","first-page":"e0290158","DOI":"10.1371\/journal.pone.0290158","article-title":"Open design of a reproducible videogame controller for MRI and MEG","volume":"18","author":"Harel","year":"2023","journal-title":"PLoS One"},{"key":"2026070215560599300_IMAG.a.1286-b19","doi-asserted-by":"crossref","DOI":"10.1162\/IMAG.a.1256","article-title":"Gamer in the scanner: Event-related analysis of fMRI activity during retro videogame play guided by automated annotations of game content","volume":"4","author":"Harel","year":"2026","journal-title":"Imaging Neuroscience"},{"issue":"4","key":"2026070215560599300_IMAG.a.1286-b20","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1038\/nn.4247","article-title":"Explicit information for category-orthogonal object properties increases along the ventral stream","volume":"19","author":"Hong","year":"2016","journal-title":"Nature Neuroscience"},{"issue":"1","key":"2026070215560599300_IMAG.a.1286-b21","doi-asserted-by":"crossref","first-page":"10678","DOI":"10.1038\/s41598-019-47176-7","article-title":"Neurocognitive signatures of naturalistic reading of scientific texts: A fixation-related fMRI study","volume":"9","author":"Hsu","year":"2019","journal-title":"Scientific Reports"},{"key":"2026070215560599300_IMAG.a.1286-b22","doi-asserted-by":"crossref","first-page":"558","DOI":"10.1109\/CoG47356.2020.9231600","article-title":"Benchmarking end-to-end behavioural cloning on video games","volume-title":"2020 IEEE conference on games (CoG)","author":"Kanervisto","year":"2020"},{"issue":"3","key":"2026070215560599300_IMAG.a.1286-b23","doi-asserted-by":"crossref","first-page":"630","DOI":"10.1016\/j.neuron.2018.03.044","article-title":"A task-optimized neural network replicates human auditory behavior, predicts brain responses, and reveals a cortical processing hierarchy","volume":"98","author":"Kell","year":"2018","journal-title":"Neuron"},{"key":"2026070215560599300_IMAG.a.1286-b24","doi-asserted-by":"crossref","first-page":"e10520","DOI":"10.7717\/peerj.10520","article-title":"Peripheral-physiological and neural correlates of the flow experience while playing video games: A comprehensive review","volume":"8","author":"Khoshnoud","year":"2020","journal-title":"PeerJ"},{"key":"2026070215560599300_IMAG.a.1286-b25","doi-asserted-by":"crossref","first-page":"819834","DOI":"10.3389\/fnhum.2022.819834","article-title":"Brain\u2013heart interaction and the experience of flow while playing a video game","volume":"16","author":"Khoshnoud","year":"2022","journal-title":"Frontiers in Human Neuroscience"},{"issue":"1","key":"2026070215560599300_IMAG.a.1286-b26","doi-asserted-by":"crossref","first-page":"eabd6127","DOI":"10.1126\/sciadv.abd6127","article-title":"Visual number sense in untrained deep neural networks","volume":"7","author":"Kim","year":"2021","journal-title":"Science Advances"},{"key":"2026070215560599300_IMAG.a.1286-b27","unstructured":"Kingma, D. P., & Ba, J. (2017). Adam: A method for stochastic optimization. https:\/\/arxiv.org\/abs\/1412.6980"},{"issue":"12","key":"2026070215560599300_IMAG.a.1286-b28","doi-asserted-by":"crossref","first-page":"948","DOI":"10.1002\/hbm.20234","article-title":"Toward brain correlates of natural behavior: fMRI during violent video games","volume":"27","author":"Mathiak","year":"2006","journal-title":"Human Brain Mapping"},{"issue":"1","key":"2026070215560599300_IMAG.a.1286-b29","doi-asserted-by":"crossref","first-page":"4638","DOI":"10.1038\/s41598-020-61409-0","article-title":"Emergence of visual center-periphery spatial organization in deep convolutional neural networks","volume":"10","author":"Mohsenzadeh","year":"2020","journal-title":"Scientific Reports"},{"issue":"2","key":"2026070215560599300_IMAG.a.1286-b30","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1016\/j.neuroimage.2010.07.073","article-title":"Encoding and decoding in fMRI","volume":"56","author":"Naselaris","year":"2011","journal-title":"NeuroImage"},{"key":"2026070215560599300_IMAG.a.1286-b31","unstructured":"Nichol, A., Pfau, V., Hesse, C., Klimov, O., & Schulman, J. (2018). Gotta learn fast: A new benchmark for generalization in RL. arXiv preprint arXiv:1804.03720. https:\/\/doi.org\/10.48550\/arXiv.1804.03720"},{"key":"2026070215560599300_IMAG.a.1286-b32","doi-asserted-by":"crossref","first-page":"248","DOI":"10.3389\/fnhum.2017.00248","article-title":"Neural basis of video gaming: A systematic review","volume":"11","author":"Palaus","year":"2017","journal-title":"Frontiers in Human Neuroscience"},{"issue":"1","key":"2026070215560599300_IMAG.a.1286-b33","doi-asserted-by":"crossref","first-page":"195","DOI":"10.3758\/s13428-018-01193-y","article-title":"PsychoPy2: Experiments in behavior made easy","volume":"51","author":"Peirce","year":"2019","journal-title":"Behavior Research Methods"},{"key":"2026070215560599300_IMAG.a.1286-b34","doi-asserted-by":"crossref","first-page":"536","DOI":"10.1016\/j.neuroimage.2014.10.044","article-title":"Recent progress and outstanding issues in motion correction in resting state fMRI","volume":"105","author":"Power","year":"2015","journal-title":"NeuroImage"},{"key":"2026070215560599300_IMAG.a.1286-b35","article-title":"Brain-score: Which artificial neural network for object recognition is most brain-like?","volume":"407007","author":"Schrimpf","year":"2018","journal-title":"bioRxiv"},{"issue":"1","key":"2026070215560599300_IMAG.a.1286-b36","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1016\/j.neuroimage.2012.06.033","article-title":"Improving diffusion MRI using simultaneous multi-slice echo planar imaging","volume":"63","author":"Setsompop","year":"2012","journal-title":"NeuroImage"},{"key":"2026070215560599300_IMAG.a.1286-b37","article-title":"Toward human-AI alignment in large-scale multi-player games","author":"Sharma","year":"2024","journal-title":"arXiv"},{"key":"2026070215560599300_IMAG.a.1286-b38","article-title":"A disciplined approach to neural network hyper-parameters: Part 1 - learning rate, batch size, momentum, and weight decay","author":"Smith","year":"2018"},{"issue":"8","key":"2026070215560599300_IMAG.a.1286-b39","doi-asserted-by":"crossref","first-page":"699","DOI":"10.1016\/j.tics.2019.05.004","article-title":"Naturalistic stimuli in neuroscience: Critically acclaimed","volume":"23","author":"Sonkusare","year":"2019","journal-title":"Trends in Cognitive Sciences"},{"key":"2026070215560599300_IMAG.a.1286-b40","article-title":"Deep image prior. In","author":"Ulyanov","year":"2018","journal-title":"Proceedings of the IEEE conference on computer vision and pattern recognition"},{"key":"2026070215560599300_IMAG.a.1286-b41","doi-asserted-by":"crossref","first-page":"3","DOI":"10.12688\/mniopenres.12767.2","article-title":"MIST: A multi-resolution parcellation of functional brain networks","volume":"1","author":"Urchs","year":"2019","journal-title":"MNI Open Research"},{"key":"2026070215560599300_IMAG.a.1286-b42","doi-asserted-by":"crossref","first-page":"100600","DOI":"10.1016\/j.dcn.2018.10.004","article-title":"Movies in the magnet: Naturalistic paradigms in developmental functional neuroimaging","volume":"36","author":"Vanderwal","year":"2019","journal-title":"Developmental Cognitive Neuroscience"},{"issue":"48","key":"2026070215560599300_IMAG.a.1286-b43","doi-asserted-by":"crossref","first-page":"15769","DOI":"10.1523\/JNEUROSCI.3454-15.2015","article-title":"Encoding voxels with deep learning","volume":"35","author":"Wang","year":"2015","journal-title":"Journal of Neuroscience"},{"issue":"1","key":"2026070215560599300_IMAG.a.1286-b44","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1207\/S1532785XMEP0801_4","article-title":"Does playing violent video games induce aggression? Empirical evidence of a functional magnetic resonance imaging study","volume":"8","author":"Weber","year":"2006","journal-title":"Media Psychology"},{"issue":"3","key":"2026070215560599300_IMAG.a.1286-b45","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1038\/nn.4244","article-title":"Using goal-driven deep learning models to understand sensory cortex","volume":"19","author":"Yamins","year":"2016","journal-title":"Nature Neuroscience"},{"issue":"4","key":"2026070215560599300_IMAG.a.1286-b46","doi-asserted-by":"crossref","first-page":"1538","DOI":"10.1214\/009053605000000255","article-title":"Boosting with early stopping: Convergence and consistency","volume":"33","author":"Zhang","year":"2005","journal-title":"The Annals of Statistics"}],"container-title":["Imaging Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/imag\/article-pdf\/doi\/10.1162\/IMAG.a.1286\/2605405\/imag.a.1286.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/imag\/article-pdf\/doi\/10.1162\/IMAG.a.1286\/2605405\/imag.a.1286.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T19:56:22Z","timestamp":1783022182000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/imag\/article\/doi\/10.1162\/IMAG.a.1286\/137102\/Behavioral-imitation-with-artificial-neural"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":46,"URL":"https:\/\/doi.org\/10.1162\/imag.a.1286","relation":{},"ISSN":["2837-6056"],"issn-type":[{"value":"2837-6056","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026]]},"published":{"date-parts":[[2026]]},"article-number":"IMAG.a.1286"}}