{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T03:26:56Z","timestamp":1772249216350,"version":"3.50.1"},"reference-count":52,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,12,22]],"date-time":"2023-12-22T00:00:00Z","timestamp":1703203200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neuroinform."],"abstract":"<jats:p>\n                    Goal-driven deep learning increasingly supplements classical modeling approaches in computational neuroscience. The strength of deep neural networks as models of the brain lies in their ability to autonomously learn the connectivity required to solve complex and ecologically valid tasks, obviating the need for hand-engineered or hypothesis-driven connectivity patterns. Consequently, goal-driven models can generate hypotheses about the neurocomputations underlying cortical processing that are grounded in macro- and mesoscopic anatomical properties of the network's biological counterpart. Whereas, goal-driven modeling is already becoming prevalent in the neuroscience of perception, its application to the sensorimotor domain is currently hampered by the complexity of the methods required to train models comprising the closed sensation-action loop. This paper describes\n                    <jats:italic>AngoraPy<\/jats:italic>\n                    , a Python library that mitigates this obstacle by providing researchers with the tools necessary to train complex recurrent convolutional neural networks that model the human sensorimotor system. To make the technical details of this toolkit more approachable, an illustrative example that trains a recurrent toy model on in-hand object manipulation accompanies the theoretical remarks. An extensive benchmark on various classical, 3D robotic, and anthropomorphic control tasks demonstrates AngoraPy's general applicability to a wide range of tasks. Together with its ability to adaptively handle custom architectures, the flexibility of this toolkit demonstrates its power for goal-driven sensorimotor modeling.\n                  <\/jats:p>","DOI":"10.3389\/fninf.2023.1223687","type":"journal-article","created":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T23:39:21Z","timestamp":1703201961000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["AngoraPy: A Python toolkit for modeling anthropomorphic goal-driven sensorimotor systems"],"prefix":"10.3389","volume":"17","author":[{"given":"Tonio","family":"Weidler","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rainer","family":"Goebel","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mario","family":"Senden","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,12,22]]},"reference":[{"key":"B1","first-page":"265","article-title":"\u201cTensorFlow: a system for large-scale machine learning,\u201d","volume-title":"12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16)","author":"Abadi","year":"2016"},{"key":"B2","unstructured":"BradburyJ.\n            FrostigR.\n            HawkinsP.\n            JohnsonM. J.\n            LearyC.\n            MaclaurinD.\n          JAX: Composable Transformations of Python+NumPy Programs2018"},{"key":"B3","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1016\/j.tics.2011.12.010","article-title":"The variable nature of cognitive control: a dual mechanisms framework","volume":"16","author":"Braver","year":"2012","journal-title":"Trends Cogn. Sci"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1606.01540","article-title":"OpenAI Gym","author":"Brockman","year":"2016","journal-title":"arXiv [Preprint"},{"key":"B5","doi-asserted-by":"publisher","first-page":"1733","DOI":"10.1152\/jn.01265.2006","article-title":"A model of V4 shape selectivity and invariance","volume":"98","author":"Cadieu","year":"2007","journal-title":"J. Neurophysiol"},{"key":"B6","doi-asserted-by":"publisher","first-page":"10577","DOI":"10.1523\/JNEUROSCI.3726-05.2005","article-title":"Do we know what the early visual system does?","volume":"25","author":"Carandini","year":"2005","journal-title":"J. Neurosci"},{"key":"B7","unstructured":"CholletF.\n          Keras2015"},{"key":"B8","first-page":"834","article-title":"\u201cImproving stochastic policy gradients in continuous control with deep reinforcement learning using the beta distribution,\u201d","author":"Chou","year":"2017","journal-title":"Proceedings of the 34th International Conference on Machine Learning, Vol. 70"},{"key":"B9","doi-asserted-by":"publisher","first-page":"12465","DOI":"10.1523\/JNEUROSCI.0902-15.2015","article-title":"Apparent and actual trajectory control depend on the behavioral context in upper limb motor tasks","volume":"35","author":"Cluff","year":"2015","journal-title":"J. Neurosci"},{"key":"B10","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1093\/acprof:oso\/9780199544318.003.0004","article-title":"\u201cOn the nature of emergent reality,\u201d","author":"Ellis","year":"2008","journal-title":"The Re-Emergence of Emergence: The Emergentist Hypothesis from Science to Religion"},{"key":"B11","article-title":"\u201cImplementation matters in deep RL: a case study on PPO and TRPO,\u201d","author":"Engstrom","year":"2020","journal-title":"Eighth International Conference on Learning Representations"},{"key":"B12","doi-asserted-by":"publisher","first-page":"3144","DOI":"10.1093\/cercor\/bhu110","article-title":"Observed, executed, and imagined action representations can be decoded from ventral and dorsal areas","volume":"25","author":"Filimon","year":"2015","journal-title":"Cereb. Cortex"},{"key":"B13","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1016\/j.neuron.2011.10.006","article-title":"Computational mechanisms of sensorimotor control","volume":"72","author":"Franklin","year":"2011","journal-title":"Neuron"},{"key":"B14","doi-asserted-by":"publisher","first-page":"519","DOI":"10.1038\/s41583-018-0045-9","article-title":"Decision-making in sensorimotor control","volume":"19","author":"Gallivan","year":"2018","journal-title":"Nat. Rev. Neurosci"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1991","DOI":"10.1523\/JNEUROSCI.0541-12.2013","article-title":"Where one hand meets the other: limb-specific and action-dependent movement plans decoded from preparatory signals in single human frontoparietal brain areas","volume":"33","author":"Gallivan","year":"2013","journal-title":"J. Neurosci"},{"key":"B16","unstructured":"GuadarramaS.\n            KorattikaraA.\n            RamirezO.\n            CastroP.\n            HollyE.\n            FishmanS.\n          TF-Agents: A Library for Reinforcement Learning in TensorFlow2018"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2111.03062","article-title":"Generalization in dexterous manipulation via geometry-aware multi-task learning","author":"Huang","year":"2021","journal-title":"arXiv [Preprint"},{"key":"B18","article-title":"\u201cA closer look at deep policy gradients,\u201d","author":"Ilyas","year":"2020","journal-title":"Eighth International Conference on Learning Representations"},{"key":"B19","doi-asserted-by":"publisher","first-page":"630","DOI":"10.1016\/j.neuron.2018.03.044","article-title":"A task-optimized neural network replicates human auditory behavior, predicts brain responses, and reveals a cortical processing hierarchy","volume":"98","author":"Kell","year":"2018","journal-title":"Neuron"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1412.6980","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"arXiv [Preprint]"},{"key":"B21","doi-asserted-by":"publisher","first-page":"1148","DOI":"10.1038\/s41593-018-0210-5","article-title":"Cognitive computational neuroscience","volume":"21","author":"Kriegeskorte","year":"2018","journal-title":"Nat. Neurosci"},{"key":"B22","doi-asserted-by":"publisher","first-page":"8","DOI":"10.3389\/neuro.06.004.2008","article-title":"Representational similarity analysis - connecting the branches of systems neuroscience","volume":"2","author":"Kriegeskorte","year":"2008","journal-title":"Front. Syst. Neurosci"},{"key":"B23","doi-asserted-by":"publisher","first-page":"408385","DOI":"10.1101\/408385","article-title":"CORnet: modeling the neural mechanisms of core object recognition","volume":"2018","author":"Kubilius","year":"2018","journal-title":"bioRxiv [Preprint"},{"key":"B24","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1101\/2022.03.14.484195","article-title":"Dissecting neural computations of the human auditory pathway using deep neural networks for speech","author":"Li","year":"2022","journal-title":"bioRxiv [Preprint"},{"key":"B26","doi-asserted-by":"publisher","first-page":"70","DOI":"10.3389\/fncom.2015.00070","article-title":"Major remaining gaps in models of sensorimotor systems","volume":"9","author":"Loeb","year":"2015","journal-title":"Front. Comput. Neurosci"},{"key":"B27","volume-title":"Vision: A Computational Investigation into the Human Representation and Processing of Visual Information","author":"Marr","year":"1982"},{"key":"B28","doi-asserted-by":"publisher","first-page":"32124","DOI":"10.1073\/pnas.2005087117","article-title":"A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping","volume":"117","author":"Michaels","year":"2020","journal-title":"Proc. Natl. Acad. Sci. U.S.A"},{"key":"B29","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1146\/annurev.neuro.24.1.167","article-title":"An integrative theory of prefrontal cortex function","volume":"24","author":"Miller","year":"2001","journal-title":"Annu. Rev. Neurosci"},{"key":"B30","doi-asserted-by":"publisher","first-page":"810","DOI":"10.3389\/fnhum.2014.00810","article-title":"Activity of right premotor-parietal regions dependent upon imagined force level: an fMRI study","volume":"8","author":"Mizuguchi","year":"2014","journal-title":"Front. Hum. Neurosci"},{"key":"B31","first-page":"1928","article-title":"\u201cAsynchronous methods for deep reinforcement learning,\u201d","volume-title":"Proceedings of The 33rd International Conference on Machine Learning","author":"Mnih","year":"2016"},{"key":"B32","unstructured":"\u201cRay: a distributed framework for emerging AI applications,\u201d561577\n            MoritzP.\n            NishiharaR.\n            WangS.\n            TumanovA.\n            LiawR.\n            LiangE.\n          13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18)2018"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1910.07113","article-title":"Solving Rubik's cube with a robot hand","author":"OpenAI","year":"2019","journal-title":"arXiv [Preprint"},{"key":"B34","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1177\/0278364919887447","article-title":"Learning dexterous in-hand manipulation","volume":"39","author":"OpenAI","year":"2020","journal-title":"Int. J. Robot. Res"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1912.01703","article-title":"PyTorch: an imperative style, high-performance deep learning library","author":"Paszke","year":"2019","journal-title":"arXiv [Preprint]"},{"key":"B36","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1016\/0025-5564(78)90018-4","article-title":"Optimization of muscle-force sequencing in human locomotion","volume":"38","author":"Pedotti","year":"1978","journal-title":"Math. Biosci"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2018.XIV.008","article-title":"Asymmetric actor critic for image-based robot learning","author":"Pinto","year":"2018","journal-title":"Robot. Sci. Syst. Found"},{"key":"B38","first-page":"1","article-title":"Stable-baselines3: reliable reinforcement learning implementations","volume":"22","author":"Raffin","year":"2021","journal-title":"J. Mach. Learn. Res"},{"key":"B39","doi-asserted-by":"publisher","author":"Schrimpf","year":"2020","DOI":"10.1101\/407007"},{"key":"B40","first-page":"1889","article-title":"\u201cTrust region policy optimization,\u201d","volume-title":"Proceedings of the 32nd International Conference on Machine Learning","author":"Schulman","year":"2015"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1707.06347","article-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017","journal-title":"arXiv [Preprint"},{"key":"B42","unstructured":"2021"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2012.6386109","article-title":"\u201cMuJoCo: a physics engine for model-based control,\u201d","author":"Todorov","year":"2012","journal-title":"IEEE\/RSJ International Conference on Intelligent Robots and Systems"},{"key":"B44","author":"Towers","year":"2023","journal-title":"Gymnasium"},{"key":"B45","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1109\/TNSRE.2011.2162851","article-title":"Mammalian muscle model for predicting force and energetics during physiological behaviors","volume":"20","author":"Tsianos","year":"2012","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1611.01224","article-title":"Sample efficient actor-critic with experience replay","author":"Wang","year":"2017","journal-title":"arXiv [Preprint"},{"key":"B47","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","article-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","volume":"8","author":"Williams","year":"1992","journal-title":"Mach. Learn"},{"key":"B48","doi-asserted-by":"publisher","first-page":"490","DOI":"10.1162\/neco.1990.2.4.490","article-title":"An efficient gradient-based algorithm for on-line training of recurrent network trajectories","volume":"2","author":"Williams","year":"1990","journal-title":"Neural Comput"},{"key":"B49","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1080\/09540099108946587","article-title":"Function optimization using connectionist reinforcement learning algorithms","volume":"3","author":"Williams","year":"1991","journal-title":"Connect. Sci"},{"key":"B50","doi-asserted-by":"publisher","first-page":"739","DOI":"10.1038\/nrn3112","article-title":"Principles of sensorimotor learning","volume":"12","author":"Wolpert","year":"2011","journal-title":"Nat. Rev. Neurosci"},{"key":"B51","doi-asserted-by":"publisher","first-page":"356","DOI":"10.1038\/nn.4244","article-title":"Using goal-driven deep learning models to understand sensory cortex","volume":"19","author":"Yamins","year":"2016","journal-title":"Nat. Neurosci"},{"key":"B52","doi-asserted-by":"publisher","first-page":"8619","DOI":"10.1073\/pnas.1403112111","article-title":"Performance-optimized hierarchical models predict neural responses in higher visual cortex","volume":"111","author":"Yamins","year":"2014","journal-title":"Proc. Natl. Acad. Sci. U.S.A"}],"container-title":["Frontiers in Neuroinformatics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2023.1223687\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T23:39:39Z","timestamp":1703201979000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2023.1223687\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,22]]},"references-count":52,"alternative-id":["10.3389\/fninf.2023.1223687"],"URL":"https:\/\/doi.org\/10.3389\/fninf.2023.1223687","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.10.05.560998","asserted-by":"object"}]},"ISSN":["1662-5196"],"issn-type":[{"value":"1662-5196","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,22]]},"article-number":"1223687"}}