{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"bioRxiv"}],"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T03:51:08Z","timestamp":1791258668682,"version":"4.1.0"},"posted":{"date-parts":[[2026,10,1]]},"group-title":"Neuroscience","reference-count":67,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T00:00:00Z","timestamp":1790812800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                <jats:p>Understanding how the brain supports flexible, goal-directed behavior is a central aim of neuroscience. Yet such behavior is highly variable and unfolds over long timescales, making it challenging to understand its organization. Advances in behavioral tracking quantify movements but do not reveal the objectives that guide them. Motivated by this, we develop inverse reinforcement learning (IRL) into a general framework for inferring these objectives from naturalistic behavior. Our approach represents an animal\u2019s objectives as internal rewards, and infers these rewards as well as the policies they induce from observed behavior. It applies across diverse behavioral settings, from freely moving behaviors to structured tasks. It can infer objectives that switch over time or gradually change with experience. In mice capturing crickets, it identified a small set of objectives that the animals switched between over the course of a trial. In mice learning a cued labyrinth, it inferred how the animals\u2019 internal rewards changed as learning progressed. Across both datasets, the models predicted held-out actions and generated extended behavior that reproduced key features of animal performance. By transforming long, variable behavioral sequences into time-resolved estimates of the animal\u2019s objectives, our approach provides a quantitative basis for studying how the brain organizes natural behavior.<\/jats:p>","DOI":"10.64898\/2026.09.25.754554","type":"posted-content","created":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T02:25:21Z","timestamp":1790907921000},"source":"Crossref","is-referenced-by-count":0,"title":["Inferring the evolving objectives that organize animal behavior"],"prefix":"10.64898","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9892-5807","authenticated-orcid":false,"given":"Aditi","family":"Jha","sequence":"first","affiliation":[{"name":"Department of Statistics, Stanford University"},{"name":"Wu Tsai Neuroscience Institute, Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Keyue Sky","family":"Shi","sequence":"additional","affiliation":[{"name":"Department of Biology, Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kara A.","family":"Fulton","sequence":"additional","affiliation":[{"name":"Department of Neurobiology, Harvard Medical School"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hanwen Lily","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Neurobiology, Harvard Medical School"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher D.","family":"Harvey","sequence":"additional","affiliation":[{"name":"Department of Neurobiology, Harvard Medical School"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sandeep Robert","family":"Datta","sequence":"additional","affiliation":[{"name":"Department of Neurobiology, Harvard Medical School"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lisa M.","family":"Giocomo","sequence":"additional","affiliation":[{"name":"Department of Neurobiology, Stanford University School of Medicine"},{"name":"Howard Hughes Medical Institute, Stanford University School of Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Scott W.","family":"Linderman","sequence":"additional","affiliation":[{"name":"Department of Statistics, Stanford University"},{"name":"Wu Tsai Neuroscience Institute, Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","reference":[{"key":"2026100520051228000_2026.09.25.754554v1.1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2016.12.041"},{"key":"2026100520051228000_2026.09.25.754554v1.2","doi-asserted-by":"publisher","DOI":"10.1037\/bne0000471"},{"key":"2026100520051228000_2026.09.25.754554v1.3","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2019.09.038"},{"key":"2026100520051228000_2026.09.25.754554v1.4","doi-asserted-by":"publisher","DOI":"10.1017\/S0952523800010269"},{"key":"2026100520051228000_2026.09.25.754554v1.5","doi-asserted-by":"publisher","DOI":"10.1038\/341052a0"},{"key":"2026100520051228000_2026.09.25.754554v1.6","doi-asserted-by":"publisher","DOI":"10.1038\/32891"},{"key":"2026100520051228000_2026.09.25.754554v1.7","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.neuro.29.051605.113038"},{"key":"2026100520051228000_2026.09.25.754554v1.8","doi-asserted-by":"publisher","DOI":"10.1038\/nn.3410"},{"key":"2026100520051228000_2026.09.25.754554v1.9","doi-asserted-by":"publisher","DOI":"10.1126\/science.1233912"},{"key":"2026100520051228000_2026.09.25.754554v1.10","doi-asserted-by":"publisher","DOI":"10.7554\/elife.63711"},{"key":"2026100520051228000_2026.09.25.754554v1.11","doi-asserted-by":"publisher","DOI":"10.1038\/s41593-018-0209-y"},{"key":"2026100520051228000_2026.09.25.754554v1.12","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.47994"},{"key":"2026100520051228000_2026.09.25.754554v1.13","doi-asserted-by":"crossref","first-page":"109730","DOI":"10.1016\/j.celrep.2021.109730","article-title":"Anipose: a toolkit for robust markerless 3D pose estimation","volume":"36","year":"2021","journal-title":"Cell Reports"},{"key":"2026100520051228000_2026.09.25.754554v1.14","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1038\/s41592-021-01106-6","article-title":"Geometric deep learning enables 3D kinematic profiling across species and environments","volume":"18","year":"2021","journal-title":"Nature Methods"},{"key":"2026100520051228000_2026.09.25.754554v1.15","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-022-01426-1\/"},{"key":"2026100520051228000_2026.09.25.754554v1.16","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1038\/nmeth.1328","article-title":"High-throughput ethomics in large groups of Drosophila","volume":"6","year":"2009","journal-title":"Nature Methods"},{"key":"2026100520051228000_2026.09.25.754554v1.17","doi-asserted-by":"crossref","first-page":"20140672","DOI":"10.1098\/rsif.2014.0672","article-title":"Mapping the stereotyped behaviour of freely moving fruit flies","volume":"11","year":"2014","journal-title":"Journal of the Royal Society Interface"},{"key":"2026100520051228000_2026.09.25.754554v1.18","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2015.11.031"},{"key":"2026100520051228000_2026.09.25.754554v1.19","doi-asserted-by":"crossref","first-page":"e63720","DOI":"10.7554\/eLife.63720","article-title":"The Mouse Action Recognition System (MARS) software pipeline for automated analysis of social behaviors in mice","volume":"10","year":"2021","journal-title":"eLife"},{"key":"2026100520051228000_2026.09.25.754554v1.20","doi-asserted-by":"crossref","first-page":"5188","DOI":"10.1038\/s41467-021-25420-x","article-title":"B-SOiD, an open-source unsupervised algorithm for identification and fast prediction of behaviors","volume":"12","year":"2021","journal-title":"Nature Communications"},{"key":"2026100520051228000_2026.09.25.754554v1.21","doi-asserted-by":"crossref","first-page":"1329","DOI":"10.1038\/s41592-024-02318-2","article-title":"Keypoint-MoSeq: parsing behavior by linking point tracking to pose dynamics","volume":"21","year":"2024","journal-title":"Nature Methods"},{"key":"2026100520051228000_2026.09.25.754554v1.22","doi-asserted-by":"publisher","DOI":"10.1016\/j.conb.2022.102549"},{"key":"2026100520051228000_2026.09.25.754554v1.23","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1607601113"},{"key":"2026100520051228000_2026.09.25.754554v1.24","doi-asserted-by":"publisher","DOI":"10.1016\/j.cognition.2009.07.005"},{"key":"2026100520051228000_2026.09.25.754554v1.25","unstructured":"Ng, A. Y. & Russell, S. J. Algorithms for inverse reinforcement learning. Proceedings of the Seventeenth International Conference on Machine Learning, 663\u2013670 (Morgan Kaufmann, 2000)."},{"key":"2026100520051228000_2026.09.25.754554v1.26","doi-asserted-by":"crossref","unstructured":"Abbeel, P. & Ng, A. Y. Apprenticeship learning via inverse reinforcement learning. Proceedings of the Twenty-First International Conference on Machine Learning, 1 (Association for Computing Machinery, 2004).","DOI":"10.1145\/1015330.1015430"},{"key":"2026100520051228000_2026.09.25.754554v1.27","unstructured":"Ziebart, B. D. , Maas, A. , Bagnell, J. A. & Dey, A. K. Maximum entropy inverse reinforcement learning. Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, 1433\u20131438 (AAAI Press, 2008)."},{"key":"2026100520051228000_2026.09.25.754554v1.28","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2018. 06.008."},{"key":"2026100520051228000_2026.09.25.754554v1.29","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-022-05611-2"},{"key":"2026100520051228000_2026.09.25.754554v1.30","doi-asserted-by":"crossref","unstructured":"Chericoni, A. et al. Neural basis of compositional control. Nature (2026).","DOI":"10.1038\/s41586-026-10896-8"},{"key":"2026100520051228000_2026.09.25.754554v1.31","doi-asserted-by":"crossref","unstructured":"Cheng, Y. et al. Asymmetric prefrontal representations for leader\u2013follower dynamics. Nature (2026).","DOI":"10.1038\/s41586-026-10900-1"},{"key":"2026100520051228000_2026.09.25.754554v1.32","doi-asserted-by":"crossref","first-page":"2261","DOI":"10.1038\/s41593-026-02377-y","article-title":"Representations of the intrinsic value of information in mouse orbitofrontal cortex","volume":"29","year":"2026","journal-title":"Nature Neuroscience"},{"key":"2026100520051228000_2026.09.25.754554v1.33","doi-asserted-by":"crossref","unstructured":"Lee, S. H. , Chung, C. , Oh, M.-h. & Ahn, W.-Y. Inverse reinforcement learning reveals action-oriented value signals in naturalistic decision making. bioRxiv (2026). Preprint.","DOI":"10.64898\/2026.06.24.733779"},{"key":"2026100520051228000_2026.09.25.754554v1.34","doi-asserted-by":"crossref","first-page":"e02447","DOI":"10.1002\/ecs2.2447","article-title":"Can AI predict animal movements? Filling gaps in animal trajectories using inverse reinforcement learning","volume":"9","year":"2018","journal-title":"Ecosphere"},{"key":"2026100520051228000_2026.09.25.754554v1.35","doi-asserted-by":"crossref","first-page":"e1006122","DOI":"10.1371\/journal.pcbi.1006122","article-title":"Identification of animal behavioral strategies by inverse reinforcement learning","volume":"14","year":"2018","journal-title":"PLOS Computational Biology"},{"key":"2026100520051228000_2026.09.25.754554v1.36","doi-asserted-by":"crossref","unstructured":"Ashwood, Z. C. , Jha, A. & Pillow, J. W. Dynamic inverse reinforcement learning for characterizing animal behavior. Advances in Neural Information Processing Systems, Vol. 35 (Curran Associates, Inc., 2022).","DOI":"10.52202\/068431-2151"},{"key":"2026100520051228000_2026.09.25.754554v1.37","doi-asserted-by":"crossref","unstructured":"Jha, A. , Geadah, V. & Pillow, J. W. Modeling complex animal behavior with latent state inverse reinforcement learning. bioRxiv (2024). Preprint.","DOI":"10.1101\/2024.11.13.623515"},{"key":"2026100520051228000_2026.09.25.754554v1.38","unstructured":"Zhu, H. et al. Multi-intention inverse Q-learning for interpretable behavior representation. Transactions on Machine Learning Research (2024). URL https:\/\/openreview.net\/forum?id=hrKHkmLUFk."},{"key":"2026100520051228000_2026.09.25.754554v1.39","unstructured":"Ke, J. , Wu, F. , Wang, J. , Markowitz, J. & Wu, A. Inverse reinforcement learning with switching rewards and history dependency for characterizing animal behaviors. Proceedings of the 42nd International Conference on Machine Learning, Vol. 267 of Proceedings of Machine Learning Research, 29608\u201329625 (PMLR, 2025). URL https:\/\/proceedings.mlr.press\/v267\/ke25b.html."},{"key":"2026100520051228000_2026.09.25.754554v1.40","unstructured":"Wulfmeier, M. , Ondruska, P. & Posner, I. Maximum entropy deep inverse reinforcement learning. arXiv (2015). Preprint."},{"key":"2026100520051228000_2026.09.25.754554v1.41","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2025.11.021"},{"key":"2026100520051228000_2026.09.25.754554v1.42","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.66175"},{"key":"2026100520051228000_2026.09.25.754554v1.43","doi-asserted-by":"publisher","DOI":"10.1038\/s41593-019-0533-x"},{"key":"2026100520051228000_2026.09.25.754554v1.44","doi-asserted-by":"publisher","DOI":"10.1016\/J.NEURON.2022.04.030"},{"key":"2026100520051228000_2026.09.25.754554v1.45","unstructured":"Fu, J. , Luo, K. & Levine, S. Learning robust rewards with adversarial inverse reinforcement learning. International Conference on Learning Representations (2018). URL https:\/\/openreview.net\/forum?id=rkHywl-A-."},{"key":"2026100520051228000_2026.09.25.754554v1.46","unstructured":"Ross, S. , Gordon, G. & Bagnell, D. A reduction of imitation learning and structured prediction to noregret online learning. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Vol. 15 of Proceedings of Machine Learning Research, 627\u2013635 (PMLR, 2011). URL https:\/\/proceedings.mlr.press\/v15\/ross11a.html."},{"key":"2026100520051228000_2026.09.25.754554v1.47","unstructured":"Im, D. J. , Kwak, I. & Branson, K. Evaluation metrics for behaviour modeling. arXiv (2020). Preprint."},{"key":"2026100520051228000_2026.09.25.754554v1.48","doi-asserted-by":"publisher","DOI":"10.1007\/s00422-013-0562-6"},{"key":"2026100520051228000_2026.09.25.754554v1.49","doi-asserted-by":"crossref","first-page":"e1006518","DOI":"10.1371\/journal.pcbi.1006518","article-title":"Modeling sensory-motor decisions in natural behavior","volume":"14","year":"2018","journal-title":"PLOS Computational Biology"},{"key":"2026100520051228000_2026.09.25.754554v1.50","first-page":"7898","volume-title":"Advances in Neural Information Processing Systems","volume":"33","year":"2020"},{"key":"2026100520051228000_2026.09.25.754554v1.51","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.76635"},{"key":"2026100520051228000_2026.09.25.754554v1.52","first-page":"7065","volume-title":"Advances in Neural Information Processing Systems","volume":"36","year":"2023"},{"key":"2026100520051228000_2026.09.25.754554v1.53","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1912336117"},{"key":"2026100520051228000_2026.09.25.754554v1.54","doi-asserted-by":"publisher","DOI":"10.1038\/s41593-019-0561-6"},{"key":"2026100520051228000_2026.09.25.754554v1.55","doi-asserted-by":"publisher","DOI":"10.1016\/j.cell.2019.04.027"},{"key":"2026100520051228000_2026.09.25.754554v1.56","doi-asserted-by":"publisher","DOI":"10.1038\/s41593-019-0502-4"},{"key":"2026100520051228000_2026.09.25.754554v1.57","doi-asserted-by":"publisher","DOI":"10.1126\/science.aav7893"},{"key":"2026100520051228000_2026.09.25.754554v1.58","unstructured":"Reddy, S. , Dragan, A. & Levine, S. Where do you think you\u2019re going?: inferring beliefs about dynamics from behavior. Advances in Neural Information Processing Systems, Vol. 31 (Curran Associates, Inc., 2018). URL https:\/\/proceedings.neurips.cc\/paper\/2018\/hash\/6f2268bd1d3d3ebaabb04d6b5d099425-Abstract.html."},{"key":"2026100520051228000_2026.09.25.754554v1.59","first-page":"12362","volume-title":"Advances in Neural Information Processing Systems","volume":"34","year":"2021"},{"key":"2026100520051228000_2026.09.25.754554v1.60","doi-asserted-by":"crossref","first-page":"104525","DOI":"10.1016\/j.artint.2026.104525","article-title":"Partial identifiability and misspecification in inverse reinforcement learning","volume":"356","year":"2026","journal-title":"Artificial Intelligence"},{"key":"2026100520051228000_2026.09.25.754554v1.61","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(98)00023-X"},{"key":"2026100520051228000_2026.09.25.754554v1.62","unstructured":"Ziebart, B. D. , Bagnell, J. A. & Dey, A. K. Modeling interaction via the principle of maximum causal entropy. Proceedings of the 27th International Conference on Machine Learning, 1255\u20131262 (2010). URL https:\/\/icml.cc\/Conferences\/2010\/papers\/28.pdf."},{"key":"2026100520051228000_2026.09.25.754554v1.63","doi-asserted-by":"crossref","unstructured":"Bloem, M. & Bambos, N. Infinite time horizon maximum causal entropy inverse reinforcement learning. 53rd IEEE Conference on Decision and Control, 4911\u20134916 (IEEE, 2014).","DOI":"10.1109\/CDC.2014.7040156"},{"key":"2026100520051228000_2026.09.25.754554v1.64","unstructured":"Levine, S. Reinforcement learning and control as probabilistic inference: tutorial and review. arXiv (2018). Preprint."},{"key":"2026100520051228000_2026.09.25.754554v1.65","unstructured":"Haarnoja, T. , Tang, H. , Abbeel, P. & Levine, S. Reinforcement learning with deep energy-based policies. Proceedings of the 34th International Conference on Machine Learning, Vol. 70 of Proceedings of Machine Learning Research, 1352\u20131361 (PMLR, 2017). URL https:\/\/proceedings.mlr.press\/v70\/haarnoja17a.html."},{"key":"2026100520051228000_2026.09.25.754554v1.66","unstructured":"Finn, C. , Levine, S. & Abbeel, P. Guided cost learning: deep inverse optimal control via policy optimization. (eds Balcan, M. F. & Weinberger, K. Q. ) Proceedings of the 33rd International Conference on Machine Learning, Vol. 48 of Proceedings of Machine Learning Research, 49\u201358 (PMLR, 2016). URL https:\/\/proceedings.mlr.press\/v48\/finn16.html."},{"key":"2026100520051228000_2026.09.25.754554v1.67","first-page":"10122","volume-title":"Advances in Neural Information Processing Systems","volume":"35","year":"2022"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.09.25.754554","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T03:05:23Z","timestamp":1791255923000},"score":1,"resource":{"primary":{"URL":"http:\/\/biorxiv.org\/lookup\/doi\/10.64898\/2026.09.25.754554"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,1]]},"references-count":67,"URL":"https:\/\/doi.org\/10.64898\/2026.09.25.754554","relation":{},"subject":[],"published":{"date-parts":[[2026,10,1]]},"subtype":"preprint"}}