{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T03:12:27Z","timestamp":1792120347042,"version":"4.4.0"},"reference-count":327,"publisher":"Emerald","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,1,4]]},"abstract":"<jats:p>Sequential decision making, commonly formalized as Markov Decision Process (MDP) optimization, is an important challenge in artificial intelligence. Two key approaches to this problem are reinforcement learning (RL) and planning. This survey is an integration of both fields, better known as model-based reinforcement learning. Model-based RL has two main steps. First, we systematically cover approaches to dynamics model learning, including challenges like dealing with stochasticity, uncertainty, partial observability, and temporal abstraction. Second, we present a systematic categorization of planning-learning integration, including aspects like: where to start planning, what budgets to allocate to planning and real data collection, how to plan, and how to integrate planning in the learning and acting loop. After these two sections, we also discuss implicit model-based RL as an end-to-end alternative for model learning and planning, and we cover the potential benefits of model-based RL. Along the way, the survey also draws connections to several related RL fields, like hierarchical RL and transfer learning. Altogether, the survey presents a broad conceptual overview of the combination of planning and learning for MDP optimization.<\/jats:p>","DOI":"10.1561\/2200000086","type":"journal-article","created":{"date-parts":[[2023,1,4]],"date-time":"2023-01-04T07:59:00Z","timestamp":1672819140000},"page":"1-118","source":"Crossref","is-referenced-by-count":462,"title":["Model-based Reinforcement Learning: A Survey"],"prefix":"10.1108","volume":"16","author":[{"given":"Thomas M.","family":"Moerland","sequence":"first","affiliation":[{"name":"LIACS, Leiden University ,","place":["The Netherlands"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joost","family":"Broekens","sequence":"additional","affiliation":[{"name":"LIACS, Leiden University ,","place":["The Netherlands"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aske","family":"Plaat","sequence":"additional","affiliation":[{"name":"LIACS, Leiden University ,","place":["The Netherlands"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Catholijn M.","family":"Jonker","sequence":"additional","affiliation":[{"name":"LIACS, Leiden University ,","place":["The Netherlands"]},{"name":"Interactive Intelligence, TU Delft ,","place":["The Netherlands"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"140","published-online":{"date-parts":[[2023,1,4]]},"reference":[{"key":"2026033012254233700_ref001","first-page":"1","article-title":"Learning first-order Markov models for control","volume-title":"Advances in neural information processing systems.","author":"Abbeel","year":"2005"},{"key":"2026033012254233700_ref002","volume-title":"Variational option discovery algorithms","author":"Achiam","year":"2018"},{"key":"2026033012254233700_ref003","volume-title":"Surprise-based intrinsic motivation for deep reinforcement learning","author":"Achiam","year":"2017"},{"issue":"8","key":"2026033012254233700_ref004","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1038\/s42256-019-0070-z","article-title":"Solving the Rubik\u2019s cube with deep reinforcement learning and search","volume":"1","author":"Agostinelli","year":"2019","journal-title":"Nature Machine Intelligence."},{"key":"2026033012254233700_ref005","first-page":"5074","article-title":"Learning to poke by poking: Experiential learning of intuitive physics","volume-title":"Advances in Neural Information Processing Systems.","author":"Agrawal","year":"2016"},{"key":"2026033012254233700_ref006","first-page":"1184","article-title":"Posterior sampling for reinforcement learning: worst-case regret bounds","volume-title":"Advances in Neural Information Processing Systems.","author":"Agrawal","year":"2017"},{"key":"2026033012254233700_ref007","volume-title":"Concrete problems in AI safety","author":"Amodei","year":"2016"},{"key":"2026033012254233700_ref008","first-page":"8766","article-title":"Unsupervised state representation learning in atari","volume-title":"Advances in Neural Information Processing Systems.","author":"Anand","year":"2019"},{"key":"2026033012254233700_ref009","volume-title":"Policy Gradient Search: Online Planning and Expert Iteration without Search Trees","author":"Anthony","year":"2019"},{"key":"2026033012254233700_ref010","first-page":"5360","article-title":"Thinking fast and slow with deep learning and tree search","volume-title":"Advances in Neural Information Processing Systems.","author":"Anthony","year":"2017"},{"key":"2026033012254233700_ref011","volume-title":"Towards a Simple Approach to Multi-step Model-based Reinforcement Learning","author":"Asadi","year":"2018"},{"key":"2026033012254233700_ref012","first-page":"19","volume-title":"A Bayesian sampling approach to exploration in reinforcement learning","author":"Asmuth","year":"2009"},{"issue":"2","key":"2026033012254233700_ref013","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/0005-1098(71)90059-8","article-title":"System identification\u2014a survey","volume":"7","author":"\u00c5str\u00f6m","year":"1971","journal-title":"Automatica."},{"issue":"5","key":"2026033012254233700_ref014","doi-asserted-by":"crossref","first-page":"1216","DOI":"10.1016\/j.automatica.2013.02.003","article-title":"Provably safe and robust learning-based model predictive control","volume":"49","author":"Aswani","year":"2013","journal-title":"Automatica."},{"key":"2026033012254233700_ref015","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1007\/978-94-017-2053-3_3","volume-title":"Lazy learning.","author":"Atkeson","year":"1997"},{"key":"2026033012254233700_ref016","first-page":"3557","volume-title":"A comparison of direct and model-based reinforcement learning","author":"Atkeson","year":"1997"},{"key":"2026033012254233700_ref017","first-page":"397","article-title":"Using confidence bounds for exploitation-exploration trade-offs","volume":"3","author":"Auer","year":"2002","journal-title":"Journal of Machine Learning Research."},{"key":"2026033012254233700_ref018","first-page":"7178","article-title":"End-to-end differentiable physics for learning and control","volume-title":"Advances in Neural Information Processing Systems.","author":"de Avila Belbute-Peres","year":"2018"},{"key":"2026033012254233700_ref019","first-page":"263","volume-title":"Minimax regret bounds for reinforcement learning","author":"Azar","year":"2017"},{"key":"2026033012254233700_ref020","volume-title":"Stochastic variational video prediction","author":"Babaeizadeh","year":"2017"},{"key":"2026033012254233700_ref021","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v31i1.10916","volume-title":"The option-critic architecture","author":"Bacon","year":"2017"},{"key":"2026033012254233700_ref022","first-page":"1615","article-title":"Autonomous helicopter control using reinforcement learning policy search methods","author":"Bagnell","year":"2001"},{"issue":"3","key":"2026033012254233700_ref023","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1109\/TAMD.2009.2037513","article-title":"R-iac: Robust intrinsically motivated exploration and active learning","volume":"1","author":"Baranes","year":"2009","journal-title":"IEEE Transactions on Autonomous Mental Development."},{"issue":"1","key":"2026033012254233700_ref024","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.robot.2012.05.008","article-title":"Active learning of inverse models with intrinsically motivated goal exploration in robots","volume":"61","author":"Baranes","year":"2013","journal-title":"Robotics and Autonomous Systems."},{"key":"2026033012254233700_ref025","first-page":"4055","article-title":"Successor features for transfer in reinforcement learning","volume-title":"Advances in neural information processing systems.","author":"Barreto","year":"2017"},{"issue":"1-2","key":"2026033012254233700_ref026","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/0004-3702(94)00011-O","article-title":"Learning to act using real-time dynamic programming","volume":"72","author":"Barto","year":"1995","journal-title":"Artificial intelligence."},{"issue":"1-2","key":"2026033012254233700_ref027","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1022140919877","article-title":"Recent advances in hierarchical reinforcement learning","volume":"13","author":"Barto","year":"2003","journal-title":"Discrete event dynamic systems."},{"key":"2026033012254233700_ref028","first-page":"4502","article-title":"Interaction networks for learning about objects, relations and physics","volume-title":"Advances in neural information processing systems.","author":"Battaglia"},{"key":"2026033012254233700_ref029","volume-title":"Relational inductive biases, deep learning, and graph networks","author":"Battaglia","year":"2018"},{"key":"2026033012254233700_ref030","volume-title":"TDLeaf (lambda): Combining temporal difference learning with game-tree search","author":"Baxter","year":"1999"},{"key":"2026033012254233700_ref031","volume-title":"Amrl: Aggregated memory for reinforcement learning","author":"Beck","year":"2020"},{"key":"2026033012254233700_ref032","first-page":"1471","article-title":"Unifying count-based exploration and intrinsic motivation","volume-title":"Advances in Neural Information Processing Systems.","author":"Bellemare","year":"2016"},{"key":"2026033012254233700_ref033","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1613\/jair.3912","article-title":"The arcade learning environment: An evaluation platform for general agents","volume":"47","author":"Bellemare","year":"2013","journal-title":"Journal of Artificial Intelligence Research."},{"issue":"6","key":"2026033012254233700_ref034","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1090\/S0002-9904-1954-09848-8","article-title":"The theory of dynamic programming","volume":"60","author":"Bellman","year":"1954","journal-title":"Bulletin of the American Mathematical Society."},{"issue":"3731","key":"2026033012254233700_ref035","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1126\/science.153.3731.34","article-title":"Dynamic programming","volume":"153","author":"Bellman","year":"1966","journal-title":"Science."},{"key":"2026033012254233700_ref036","first-page":"41","volume-title":"Curriculum learning","author":"Bengio","year":"2009"},{"key":"2026033012254233700_ref037","first-page":"908","article-title":"Safe model-based reinforcement learning with stability guarantees","volume-title":"Advances in neural information processing systems.","author":"Berkenkamp","year":"2017"},{"key":"2026033012254233700_ref038","volume-title":"Dynamic programming and optimal control.","author":"Bertsekas","year":"1995"},{"key":"2026033012254233700_ref039","volume-title":"Pattern recognition and machine learning.","author":"Bishop","year":"2006"},{"key":"2026033012254233700_ref040","volume-title":"Model-free episodic control","author":"Blundell","year":"2016"},{"key":"2026033012254233700_ref041","first-page":"229","volume-title":"Efficient reinforcement learning: Model-based acrobot control","author":"Boone","year":"1997"},{"key":"2026033012254233700_ref042","first-page":"213","article-title":"R-max-a general polynomial time algorithm for near-optimal reinforcement learning","volume":"3","author":"Brafman","year":"2002","journal-title":"Journal of Machine Learning Research."},{"key":"2026033012254233700_ref043","volume-title":"OpenAI Gym","author":"Brockman","year":"2016"},{"issue":"1","key":"2026033012254233700_ref044","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TCIAIG.2012.2186810","article-title":"A survey of monte carlo tree search methods","volume":"4","author":"Browne","year":"2012","journal-title":"IEEE Transactions on Computational Intelligence and AI in games."},{"key":"2026033012254233700_ref045","first-page":"316","volume-title":"Pac-inspired option discovery in lifelong reinforcement learning","author":"Brunskill","year":"2014"},{"key":"2026033012254233700_ref046","first-page":"8224","article-title":"Sample-efficient reinforcement learning with stochastic ensemble value expansion","volume-title":"Advances in Neural Information Processing Systems.","author":"Buckman","year":"2018"},{"key":"2026033012254233700_ref047","volume-title":"Learning and querying fast generative models for reinforcement learning","author":"Buesing","year":"2018"},{"issue":"1","key":"2026033012254233700_ref048","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1007379606734","article-title":"Multitask learning","volume":"28","author":"Caruana","year":"1997","journal-title":"Machine learning."},{"key":"2026033012254233700_ref049","article-title":"Using Linear Programming for Bayesian Exploration in Markov Decision Processes","volume-title":"IJCAI.","author":"Castro","year":"2007"},{"key":"2026033012254233700_ref050","volume-title":"A compositional object-based approach to learning physical dynamics","author":"Chang","year":"2016"},{"key":"2026033012254233700_ref051","first-page":"1281","article-title":"Intrinsically motivated reinforcement learning","volume-title":"Advances in neural information processing systems.","author":"Chentanez","year":"2005"},{"key":"2026033012254233700_ref052","volume-title":"Recurrent environment simulators","author":"Chiappa","year":"2017"},{"key":"2026033012254233700_ref053","volume-title":"Contingency-aware exploration in reinforcement learning","author":"Choi","year":"2018"},{"key":"2026033012254233700_ref054","first-page":"183","article-title":"Reinforcement learning with perceptual aliasing: The perceptual distinctions approach","volume-title":"AAAI.","author":"Chrisman","year":"1992"},{"key":"2026033012254233700_ref055","volume-title":"Transfer from simulation to real world through learning deep inverse dynamics model","author":"Christiano","year":"2016"},{"key":"2026033012254233700_ref056","first-page":"4754","article-title":"Deep reinforcement learning in a handful of trials using probabilistic dynamics models","volume-title":"Advances in Neural Information Processing Systems.","author":"Chua","year":"2018"},{"key":"2026033012254233700_ref057","first-page":"617","volume-title":"Model-Based Reinforcement Learning via Meta-Policy Optimization","author":"Clavera","year":"2018"},{"key":"2026033012254233700_ref058","volume-title":"Efficient model-based deep reinforcement learning with variational state tabulation","author":"Corneil","year":"2018"},{"key":"2026033012254233700_ref059","first-page":"72","volume-title":"Efficient selectivity and backup operators in Monte-Carlo tree search","author":"Coulom","year":"2006"},{"key":"2026033012254233700_ref060","volume-title":"The Nature of Explanation","author":"Craik","year":"1943"},{"key":"2026033012254233700_ref061","first-page":"217","volume-title":"Dealing with non-stationary environments using context detection","author":"Da Silva","year":"2006"},{"issue":"2-3","key":"2026033012254233700_ref062","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1007\/s10994-016-5580-x","article-title":"Probabilistic inference for determining options in reinforcement learning","volume":"104","author":"Daniel","year":"2016","journal-title":"Machine Learning."},{"key":"2026033012254233700_ref063","first-page":"2818","volume-title":"Sample complexity of episodic fixed-horizon reinforcement learning","author":"Dann","year":"2015"},{"issue":"4","key":"2026033012254233700_ref064","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1162\/neco.1993.5.4.613","article-title":"Improving generalization for temporal difference learning: The successor representation","volume":"5","author":"Dayan","year":"1993","journal-title":"Neural Computation."},{"key":"2026033012254233700_ref065","first-page":"271","article-title":"Feudal reinforcement learning","volume-title":"Advances in neural information processing systems.","author":"Dayan","year":"1993"},{"key":"2026033012254233700_ref066","first-page":"150","volume-title":"Model based Bayesian exploration","author":"Dearden","year":"1999"},{"key":"2026033012254233700_ref067","first-page":"761","article-title":"Bayesian Q-learning","volume-title":"AAAI\/IAAI.","author":"Dearden","year":"1998"},{"key":"2026033012254233700_ref068","first-page":"13","article-title":"A differentiable physics engine for deep learning in robotics","volume-title":"Frontiers in neurorobotics.","author":"Degrave","year":"2019"},{"key":"2026033012254233700_ref069","first-page":"465","volume-title":"PILCO: A model-based and data-efficient approach to policy search","author":"Deisenroth","year":"2011"},{"key":"2026033012254233700_ref070","volume-title":"Learning and policy search in stochastic dynamical systems with bayesian neural networks","author":"Depeweg","year":"2016"},{"issue":"2","key":"2026033012254233700_ref071","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.strusafe.2008.06.020","article-title":"Aleatory or epistemic? Does it matter?","volume":"31","author":"Der Kiureghian","year":"2009","journal-title":"Structural Safety."},{"key":"2026033012254233700_ref072","doi-asserted-by":"crossref","DOI":"10.1109\/TNNLS.2019.2891792","article-title":"Feature control as intrinsic motivation for hierarchical reinforcement learning","volume-title":"IEEE transactions on neural networks and learning systems.","author":"Dilokthanakul","year":"2019"},{"key":"2026033012254233700_ref073","first-page":"240","volume-title":"An object-oriented representation for efficient reinforcement learning","author":"Diuk","year":"2008"},{"issue":"6","key":"2026033012254233700_ref074","doi-asserted-by":"crossref","first-page":"1075","DOI":"10.1016\/j.conb.2012.08.003","article-title":"The ubiquity of model-based reinforcement learning","volume":"22","author":"Doll","year":"2012","journal-title":"Current opinion in neurobiology."},{"issue":"6","key":"2026033012254233700_ref075","doi-asserted-by":"crossref","first-page":"1347","DOI":"10.1162\/089976602753712972","article-title":"Multiple model-based reinforcement learning","volume":"14","author":"Doya","year":"2002","journal-title":"Neural computation."},{"key":"2026033012254233700_ref076","volume-title":"Optimal Learning: Computational procedures for Bayes-adaptive Markov decision processes","author":"Duff","year":"2002"},{"key":"2026033012254233700_ref077","volume-title":"Go-explore: a new approach for hard-exploration problems","author":"Ecoffet","year":"2019"},{"key":"2026033012254233700_ref078","volume-title":"Forward-backward reinforcement learning","author":"Edwards","year":"2018"},{"key":"2026033012254233700_ref079","first-page":"1386","volume-title":"Beyond the One-Step Greedy Approach in Reinforcement Learning","author":"Efroni","year":"2018"},{"key":"2026033012254233700_ref080","volume-title":"Multi-Step Greedy and Approximate Real Time Dynamic Programming","author":"Efroni","year":"2019"},{"key":"2026033012254233700_ref081","volume-title":"Tight Regret Bounds for Model-Based Reinforcement Learning with Greedy Policies","author":"Efroni","year":"2019"},{"key":"2026033012254233700_ref082","first-page":"493","article-title":"Hierarchical recurrent neural networks for long-term dependencies","volume-title":"Advances in neural information processing systems.","author":"El Hihi","year":"1996"},{"issue":"4","key":"2026033012254233700_ref083","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1111\/j.2044-8295.1984.tb01915.x","article-title":"Heuristic and analytic processes in reasoning","volume":"75","author":"Evans","year":"1984","journal-title":"British Journal of Psychology."},{"key":"2026033012254233700_ref084","first-page":"32","article-title":"Search on the Replay Buffer: Bridging Planning and Reinforcement Learning","volume-title":"Advances in Neural Information Processing Systems.","author":"Eysenbach","year":"2019"},{"key":"2026033012254233700_ref085","volume-title":"Diversity is All You Need: Learning Skills without a Reward Function","author":"Eysenbach","year":"2019"},{"key":"2026033012254233700_ref086","first-page":"1","volume-title":"Value-gradient learning","author":"Fairbank","year":"2012"},{"key":"2026033012254233700_ref087","article-title":"Treeqn and atreec: Differentiable tree planning for deep reinforcement learning","author":"Farquhar","year":"2018"},{"key":"2026033012254233700_ref088","first-page":"64","article-title":"Unsupervised learning for physical interaction through video prediction","volume-title":"Advances in neural information processing systems.","author":"Finn","year":"2016"},{"key":"2026033012254233700_ref089","volume-title":"Stochastic neural networks for hierarchical reinforcement learning","author":"Florensa","year":"2017"},{"key":"2026033012254233700_ref090","first-page":"1514","volume-title":"Automatic Goal Generation for Reinforcement Learning Agents","author":"Florensa","year":"2018"},{"key":"2026033012254233700_ref091","volume-title":"Generalization of reinforcement learners with working and episodic memory","author":"Fortunato","year":"2019"},{"key":"2026033012254233700_ref092","volume-title":"Multi-level discovery of deep options","author":"Fox","year":"2017"},{"key":"2026033012254233700_ref093","volume-title":"Principled option learning in Markov decision processes","author":"Fox","year":"2016"},{"key":"2026033012254233700_ref094","first-page":"3601","article-title":"A disentangled recognition and nonlinear dynamics model for unsupervised learning","volume-title":"Advances in Neural Information Processing Systems.","author":"Fraccaro","year":"2017"},{"key":"2026033012254233700_ref095","volume-title":"Learning visual predictive models of physics for playing billiards","author":"Fragkiadaki","year":"2015"},{"key":"2026033012254233700_ref096","first-page":"3582","volume-title":"Combined reinforcement learning via abstract representations","author":"Fran\u00e7ois-Lavet","year":"2019"},{"key":"2026033012254233700_ref097","volume-title":"Meta learning shared hierarchies","author":"Frans","year":"2018"},{"key":"2026033012254233700_ref098","first-page":"61","volume-title":"Uncertainty analysis for non-identifiable dynamical systems: Profile likelihoods, bootstrapping and more","author":"Fr\u00f6hlich","year":"2014"},{"key":"2026033012254233700_ref099","volume-title":"Improving PILCO with Bayesian neural network dynamics models","author":"Gal","year":"2016"},{"key":"2026033012254233700_ref100","volume-title":"Generative temporal models with memory","author":"Gemici","year":"2017"},{"key":"2026033012254233700_ref101","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1146\/annurev-psych-122414-033625","article-title":"Reinforcement learning and episodic memory in humans and animals: an integrative framework","volume":"68","author":"Gershman","year":"2017","journal-title":"Annual review of psychology."},{"key":"2026033012254233700_ref102","volume-title":"Tech. rep.","author":"Ghahramani","year":"1996"},{"key":"2026033012254233700_ref103","first-page":"431","article-title":"Learning nonlinear dynamical systems using an EM algorithm","volume-title":"Advances in neural information processing systems.","author":"Ghahramani","year":"1999"},{"issue":"5-6","key":"2026033012254233700_ref104","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1561\/2200000049","article-title":"Bayesian reinforcement learning: A survey","volume":"8","author":"Ghavamzadeh","year":"2015","journal-title":"Foundations and Trends\u00ae in Machine Learning."},{"key":"2026033012254233700_ref105","volume-title":"Learning Actionable Representations with Goal-Conditioned Policies","author":"Ghosh","year":"2018"},{"key":"2026033012254233700_ref106","first-page":"346","volume-title":"Subgoal discovery for hierarchical reinforcement learning using learned policies","author":"Goel","year":"2003"},{"key":"2026033012254233700_ref107","volume-title":"Deep learning.","author":"Goodfellow","year":"2016"},{"key":"2026033012254233700_ref108","first-page":"861","volume-title":"Thompson sampling for learning parameterized markov decision processes","author":"Gopalan","year":"2015"},{"key":"2026033012254233700_ref109","volume-title":"Neural turing machines","author":"Graves","year":"2014"},{"key":"2026033012254233700_ref110","volume-title":"Variational intrinsic control","author":"Gregor","year":"2016"},{"key":"2026033012254233700_ref111","article-title":"The Value Equivalence Principle for Model-Based Reinforcement Learning","volume-title":"Advances in Neural Information Processing Systems.","author":"Grimm","year":"2020"},{"key":"2026033012254233700_ref112","first-page":"2829","volume-title":"Continuous deep q-learning with model-based acceleration","author":"Gu","year":"2016"},{"key":"2026033012254233700_ref113","first-page":"1003","volume-title":"Generalizing plans to new environments in relational MDPs","author":"Guestrin","year":"2003"},{"key":"2026033012254233700_ref114","first-page":"2464","volume-title":"An Investigation of Model-Free Planning","author":"Guez"},{"key":"2026033012254233700_ref115","first-page":"1025","article-title":"Efficient Bayes-adaptive reinforcement learning using sample-based search","volume-title":"Advances in neural information processing systems.","author":"Guez","year":"2012"},{"key":"2026033012254233700_ref116","volume-title":"Learning to search with MCTSnets","author":"Guez","year":"2018"},{"key":"2026033012254233700_ref117","first-page":"3338","article-title":"Deep learning for real-time Atari game play using offline Monte-Carlo tree search planning","volume-title":"Advances in neural information processing systems.","author":"Guo","year":"2014"},{"key":"2026033012254233700_ref118","first-page":"2450","article-title":"Recurrent world models facilitate policy evolution","volume-title":"Advances in Neural Information Processing Systems.","author":"Ha","year":"2018"},{"key":"2026033012254233700_ref119","volume-title":"Dream to Control: Learning Behaviors by Latent Imagination","author":"Hafner","year":"2019"},{"key":"2026033012254233700_ref120","first-page":"2555","volume-title":"Learning latent dynamics for planning from pixels","author":"Hafner","year":"2019"},{"key":"2026033012254233700_ref121","volume-title":"Active imitation learning of hierarchical policies","author":"Hamidi","year":"2015"},{"key":"2026033012254233700_ref122","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.cobeha.2018.12.011","article-title":"Analogues of mental simulation and imagination in deep learning","volume":"29","author":"Hamrick","year":"2019","journal-title":"Current Opinion in Behavioral Sciences."},{"key":"2026033012254233700_ref123","volume-title":"Metacontrol for adaptive imagination-based optimization","author":"Hamrick","year":"2017"},{"key":"2026033012254233700_ref124","volume-title":"Combining q-learning and search with amortized value estimates","author":"Hamrick","year":"2020"},{"key":"2026033012254233700_ref125","first-page":"14322","article-title":"When to use parametric models in reinforcement learning?","volume-title":"Advances in Neural Information Processing Systems.","author":"van Hasselt","year":"2019"},{"key":"2026033012254233700_ref126","volume-title":"Learning an Embedding Space for Transferable Robot Skills","author":"Hausman","year":"2018"},{"key":"2026033012254233700_ref127","volume-title":"Learning and transfer of modulated locomotor controllers","author":"Heess","year":"2016"},{"key":"2026033012254233700_ref128","first-page":"2944","article-title":"Learning continuous control policies by stochastic value gradients","volume-title":"Advances in Neural Information Processing Systems.","author":"Heess","year":"2015"},{"key":"2026033012254233700_ref129","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v32i1.11694","volume-title":"Deep reinforcement learning that matters","author":"Henderson","year":"2018"},{"key":"2026033012254233700_ref130","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1007\/978-1-4899-7687-1_363","article-title":"Hierarchical reinforcement learning","volume-title":"Encyclopedia of Machine Learning and Data Mining:","author":"Hengst","year":"2017"},{"key":"2026033012254233700_ref131","first-page":"1","volume-title":"Intrinsically motivated model learning for a developing curious agent","author":"Hester","year":"2012"},{"key":"2026033012254233700_ref132","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1007\/978-3-642-27645-3_4","volume-title":"Reinforcement learning.","author":"Hester","year":"2012"},{"issue":"3","key":"2026033012254233700_ref133","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1007\/s10994-012-5322-7","article-title":"TEXPLORE: real-time sample-efficient reinforcement learning for robots","volume":"90","author":"Hester","year":"2013","journal-title":"Machine learning."},{"issue":"8","key":"2026033012254233700_ref134","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural computation."},{"key":"2026033012254233700_ref135","volume-title":"The effect of planning shape on dyna-style planning in high-dimensional state spaces","author":"Holland","year":"2018"},{"key":"2026033012254233700_ref136","first-page":"1109","article-title":"Vime: Variational information maximizing exploration","volume-title":"Advances in Neural Information Processing Systems.","author":"Houthooft","year":"2016"},{"key":"2026033012254233700_ref137","volume-title":"Generalizable episodic memory for deep reinforcement learning","author":"Hu","year":"2021"},{"key":"2026033012254233700_ref138","volume-title":"Reinforcement learning with unsupervised auxiliary tasks","author":"Jaderberg","year":"2016"},{"issue":"4","key":"2026033012254233700_ref139","article-title":"Near-optimal Regret Bounds for Reinforcement Learning","volume":"11","author":"Jaksch","year":"2010","journal-title":"Journal of Machine Learning Research."},{"key":"2026033012254233700_ref140","first-page":"12519","article-title":"When to trust your model: Model-based policy optimization","volume-title":"Advances in Neural Information Processing Systems.","author":"Janner","year":"2019"},{"key":"2026033012254233700_ref141","first-page":"26","volume-title":"Learning in non-stationary partially observable Markov decision processes","author":"Jaulmes","year":"2005"},{"key":"2026033012254233700_ref142","volume-title":"Time-agnostic prediction: Predicting predictable video frames","author":"Jayaraman","year":"2018"},{"key":"2026033012254233700_ref143","first-page":"2289","volume-title":"Feedback-Based Tree Search for Reinforcement Learning","author":"Jiang","year":"2018"},{"key":"2026033012254233700_ref144","first-page":"4868","article-title":"Is Q-learning provably efficient?","author":"Jin","year":"2018"},{"key":"2026033012254233700_ref145","first-page":"95","volume-title":"Model-based function approximation in reinforcement learning","author":"Jong","year":"2007"},{"issue":"3","key":"2026033012254233700_ref146","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1007\/s10514-015-9459-7","article-title":"Learning state representations with robotic priors","volume":"39","author":"Jonschkowski","year":"2015","journal-title":"Autonomous Robots."},{"issue":"3","key":"2026033012254233700_ref147","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1207\/s15516709cog1603_1","article-title":"Forward models: Supervised learning with a distal teacher","volume":"16","author":"Jordan","year":"1992","journal-title":"Cognitive science."},{"key":"2026033012254233700_ref148","volume-title":"Thinking, fast and slow.","author":"Kahneman","year":"2011"},{"key":"2026033012254233700_ref149","volume-title":"Approximately optimal approximate reinforcement learning","author":"Kakade","year":"2002"},{"key":"2026033012254233700_ref150","volume-title":"Variance reduction methods for sublinear reinforcement learning","author":"Kakade","year":"2018"},{"key":"2026033012254233700_ref151","volume-title":"On the sample complexity of reinforcement learning","author":"Kakade","year":"2003"},{"key":"2026033012254233700_ref152","first-page":"1771","volume-title":"Video pixel networks","author":"Kalchbrenner","year":"2017"},{"key":"2026033012254233700_ref153","first-page":"195","volume-title":"Uncertainty-driven imagination for continuous deep reinforcement learning","author":"Kalweit","year":"2017"},{"key":"2026033012254233700_ref154","volume-title":"Data-efficient reinforcement learning with probabilistic model predictive control","author":"Kamthe","year":"2017"},{"key":"2026033012254233700_ref155","first-page":"1809","volume-title":"Schema networks: Zero-shot transfer with a generative causal model of intuitive physics","author":"Kansky","year":"2017"},{"key":"2026033012254233700_ref156","volume-title":"Deep variational bayes filters: Unsupervised learning of state space models from raw data","author":"Karl","year":"2016"},{"key":"2026033012254233700_ref157","volume-title":"Learning Dynamics Model in Reinforcement Learning by Incorporating the Long Term Future","author":"Ke","year":"2019"},{"issue":"5","key":"2026033012254233700_ref158","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1002055","article-title":"Speed\/accuracy tradeoff between the habitual and the goal-directed processes","volume":"7","author":"Keramati","year":"2011","journal-title":"PLoS computational biology."},{"issue":"5","key":"2026033012254233700_ref159","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1109\/TRO.2011.2159412","article-title":"Learning stable nonlinear dynamical systems with gaussian mixture models","volume":"27","author":"Khansari-Zadeh","year":"2011","journal-title":"IEEE Transactions on Robotics."},{"key":"2026033012254233700_ref160","volume-title":"Contrastive Learning of Structured World Models","author":"Kipf","year":"2020"},{"issue":"11","key":"2026033012254233700_ref161","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1177\/0278364913495721","article-title":"Reinforcement learning in robotics: A survey","volume":"32","author":"Kober","year":"2013","journal-title":"The International Journal of Robotics Research."},{"key":"2026033012254233700_ref162","first-page":"282","volume-title":"ECML.","author":"Kocsis","year":"2006"},{"key":"2026033012254233700_ref163","first-page":"513","volume-title":"Near-Bayesian exploration in polynomial time","author":"Kolter","year":"2009"},{"key":"2026033012254233700_ref164","first-page":"895","article-title":"Building Portable Options: Skill Transfer in Reinforcement Learning","volume":"7","author":"Konidaris","year":"2007","journal-title":"IJCAI."},{"issue":"3","key":"2026033012254233700_ref165","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1177\/0278364911428653","article-title":"Robot learning from demonstration by constructing skill trees","volume":"31","author":"Konidaris","year":"2012","journal-title":"The International Journal of Robotics Research."},{"key":"2026033012254233700_ref166","volume-title":"A framework for transfer in reinforcement learning","author":"Konidaris","year":"2006"},{"issue":"2-3","key":"2026033012254233700_ref167","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/0004-3702(90)90054-4","article-title":"Real-time heuristic search","volume":"42","author":"Korf","year":"1990","journal-title":"Artificial intelligence."},{"key":"2026033012254233700_ref168","volume-title":"Deep Kalman Filters","author":"Krishnan","year":"2015"},{"key":"2026033012254233700_ref169","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume-title":"Advances in neural information processing systems.","author":"Krizhevsky","year":"2012"},{"key":"2026033012254233700_ref170","first-page":"3675","article-title":"Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation","volume-title":"Advances in neural information processing systems.","author":"Kulkarni","year":"2016"},{"key":"2026033012254233700_ref171","volume-title":"Robotics: Science and systems.","author":"Kurniawati","year":"2008"},{"key":"2026033012254233700_ref172","first-page":"8733","article-title":"Learning plannable representations with causal infogan","volume-title":"Advances in Neural Information Processing Systems.","author":"Kurutach","year":"2018"},{"key":"2026033012254233700_ref173","volume-title":"Giraffe: Using deep reinforcement learning to play chess","author":"Lai","year":"2015"},{"issue":"1","key":"2026033012254233700_ref174","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/0196-8858(85)90002-8","article-title":"Asymptotically efficient adaptive allocation rules","volume":"6","author":"Lai","year":"1985","journal-title":"Advances in applied mathematics."},{"key":"2026033012254233700_ref175","volume-title":"Option discovery in hierarchical reinforcement learning using spatio-temporal clustering","author":"Lakshminarayanan","year":"2016"},{"key":"2026033012254233700_ref176","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1007\/978-3-642-27645-3_2","volume-title":"Reinforcement learning.","author":"Lange","year":"2012"},{"key":"2026033012254233700_ref177","volume-title":"Rapidly-exploring random trees: A new tool for path planning","author":"LaValle","year":"1998"},{"key":"2026033012254233700_ref178","first-page":"487","volume-title":"Curiosity Driven Exploration of Learned Disentangled Goal Spaces","author":"Laversanne-Finot","year":"2018"},{"key":"2026033012254233700_ref179","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/978-3-642-27645-3_5","volume-title":"Reinforcement Learning.","author":"Lazaric","year":"2012"},{"key":"2026033012254233700_ref180","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1016\/j.neunet.2018.07.006","article-title":"State representation learning for control: An overview","volume":"108","author":"Lesort","year":"2018","journal-title":"Neural Networks."},{"key":"2026033012254233700_ref181","first-page":"1071","article-title":"Learning neural network policies with guided policy search under unknown dynamics","volume-title":"Advances in Neural Information Processing Systems.","author":"Levine","year":"2014"},{"key":"2026033012254233700_ref182","first-page":"1","volume-title":"Guided policy search","author":"Levine","year":"2013"},{"key":"2026033012254233700_ref183","volume-title":"Hierarchical Reinforcement Learning with Hindsight","author":"Levy","year":"2019"},{"issue":"3-4","key":"2026033012254233700_ref184","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1023\/A:1022628806385","article-title":"Self-improving reactive agents based on reinforcement learning, planning and teaching","volume":"8","author":"Lin","year":"1992","journal-title":"Machine learning."},{"key":"2026033012254233700_ref185","volume-title":"Tech. rep.","author":"Lin","year":"1993"},{"key":"2026033012254233700_ref186","volume-title":"Memory approaches to reinforcement learning in non-Markovian domains.","author":"Lin","year":"1992"},{"key":"2026033012254233700_ref187","doi-asserted-by":"crossref","DOI":"10.24963\/ijcai.2018\/337","volume-title":"Episodic memory deep Q-networks","author":"Lin","year":"2018"},{"key":"2026033012254233700_ref188","article-title":"System identification","volume-title":"Wiley Encyclopedia of Electrical and Electronics Engineering.","author":"Ljung","year":"2001"},{"key":"2026033012254233700_ref189","first-page":"206","article-title":"Exploration in model-based reinforcement learning by empirically estimating learning progress","volume-title":"Advances in neural information processing systems.","author":"Lopes","year":"2012"},{"key":"2026033012254233700_ref190","volume-title":"Plan online, learn offline: Efficient learning and exploration via model-based control","author":"Lowrey","year":"2018"},{"key":"2026033012254233700_ref191","volume-title":"Now I Remember! Episodic Memory For Reinforcement Learning","author":"Loynd","year":"2018"},{"key":"2026033012254233700_ref192","volume-title":"Adaptive Online Planning for Continual Lifelong Learning","author":"Lu","year":"2019"},{"key":"2026033012254233700_ref193","first-page":"2295","volume-title":"A laplacian framework for option discovery in reinforcement learning","author":"Machado","year":"2017"},{"key":"2026033012254233700_ref194","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1613\/jair.5699","article-title":"Revisiting the arcade learning environment: Evaluation protocols and open problems for general agents","volume":"61","author":"Machado","year":"2018","journal-title":"Journal of Artificial Intelligence Research."},{"issue":"4","key":"2026033012254233700_ref195","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1561\/2200000003","article-title":"Learning Representation and Control in Markov Decision Pro cesses: New Frontiers","volume":"1","author":"Mahadevan","year":"2009","journal-title":"Foundations and Trends\u00ae in Machine Learning."},{"key":"2026033012254233700_ref196","first-page":"127","volume-title":"Scaling up approximate value iteration with options: Better policies with fewer iterations","author":"Mann","year":"2014"},{"key":"2026033012254233700_ref197","first-page":"71","volume-title":"Dynamic abstraction in reinforcement learning via clustering","author":"Mannor","year":"2004"},{"key":"2026033012254233700_ref198","volume-title":"Teacher-student curriculum learning","author":"Matiisen","year":"2017"},{"key":"2026033012254233700_ref199","volume-title":"Reinforcement learning with selective perception and hidden state","author":"McCallum","year":"1997"},{"key":"2026033012254233700_ref200","volume-title":"Automatic discovery of subgoals in reinforcement learning using diverse density","author":"McGovern","year":"2001"},{"key":"2026033012254233700_ref201","first-page":"295","volume-title":"Q-cut\u2014dynamic discovery of sub-goals in reinforcement learning","author":"Menache","year":"2002"},{"key":"2026033012254233700_ref202","first-page":"2459","volume-title":"Prediction and control with temporal segment models","author":"Mishra","year":"2017"},{"issue":"7540","key":"2026033012254233700_ref203","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature."},{"key":"2026033012254233700_ref204","volume-title":"Efficient exploration with double uncertain value networks","author":"Moerland","year":"2017"},{"key":"2026033012254233700_ref205","volume-title":"Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning","author":"Moerland","year":"2017"},{"issue":"2","key":"2026033012254233700_ref206","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1007\/s10994-017-5666-0","article-title":"Emotion in reinforcement learning agents and robots: a survey","volume":"107","author":"Moerland","year":"2018","journal-title":"Machine Learning."},{"key":"2026033012254233700_ref207","volume-title":"A0C: Alpha zero in continuous action space","author":"Moerland","year":"2018"},{"key":"2026033012254233700_ref208","volume-title":"Think Too Fast Nor Too Slow: The Computational Tradeoff Between Planning And Reinforcement Learning","author":"Moerland","year":"2020"},{"key":"2026033012254233700_ref209","volume-title":"A Framework for Reinforcement Learning and Planning","author":"Moerland","year":"2020"},{"issue":"9","key":"2026033012254233700_ref210","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1038\/s41562-017-0180-8","article-title":"The successor representation in human reinforcement learning","volume":"1","author":"Momennejad","year":"2017","journal-title":"Nature Human Behaviour."},{"issue":"1","key":"2026033012254233700_ref211","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1023\/A:1022635613229","article-title":"Prioritized sweeping: Reinforcement learning with less data and less time","volume":"13","author":"Moore","year":"1993","journal-title":"Machine learning."},{"key":"2026033012254233700_ref212","first-page":"999","volume-title":"Predicting time series with support vector machines","author":"M\u00fcller","year":"1997"},{"key":"2026033012254233700_ref213","first-page":"560","article-title":"Error bounds for approximate policy iteration","volume":"3","author":"Munos","year":"2003","journal-title":"ICML."},{"key":"2026033012254233700_ref214","first-page":"3303","article-title":"Data-efficient hierarchical reinforcement learning","volume-title":"Advances in Neural Information Processing Systems.","author":"Nachum","year":"2018"},{"key":"2026033012254233700_ref215","volume-title":"Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning","author":"Nagabandi","year":"2018"},{"key":"2026033012254233700_ref216","volume-title":"Deep online learning via meta-learning: Continual adaptation for model-based RL","author":"Nagabandi","year":"2018"},{"key":"2026033012254233700_ref217","first-page":"7559","volume-title":"Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning","author":"Nagabandi","year":"2018"},{"issue":"1","key":"2026033012254233700_ref218","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/72.80202","article-title":"Identification and control of dynamical systems using neural networks","volume":"1","author":"Narendra","year":"1990","journal-title":"IEEE Transactions on neural networks."},{"key":"2026033012254233700_ref219","first-page":"9816","article-title":"Adaptive skip intervals: Temporal abstraction for recurrent dynamical models","volume-title":"Advances in Neural Information Processing Systems.","author":"Neitz","year":"2018"},{"issue":"4","key":"2026033012254233700_ref220","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1007\/s10339-011-0404-1","article-title":"Model learning for robot control: a survey","volume":"12","author":"Nguyen-Tuong","year":"2011","journal-title":"Cognitive processing."},{"issue":"1","key":"2026033012254233700_ref221","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1007\/s10994-010-5202-y","article-title":"Dimension reduction and its application to model-based exploration in continuous spaces","volume":"81","author":"Nouri","year":"2010","journal-title":"Machine Learning."},{"key":"2026033012254233700_ref222","first-page":"2863","article-title":"Action-conditional video prediction using deep networks in atari games","volume-title":"Advances in neural information processing systems.","author":"Oh","year":"2015"},{"key":"2026033012254233700_ref223","first-page":"6118","article-title":"Value prediction network","volume-title":"Advances in Neural Information Processing Systems.","author":"Oh","year":"2017"},{"key":"2026033012254233700_ref224","first-page":"4026","article-title":"Deep exploration via bootstrapped DQN","volume-title":"Advances in Neural Information Processing Systems.","author":"Osband","year":"2016"},{"key":"2026033012254233700_ref225","volume-title":"Behaviour Suite for Reinforcement Learning","author":"Osband","year":"2019"},{"key":"2026033012254233700_ref226","volume-title":"(More) efficient reinforcement learning via posterior sampling","author":"Osband","year":"2013"},{"key":"2026033012254233700_ref227","volume-title":"On lower bounds for regret in reinforcement learning","author":"Osband","year":"2016"},{"key":"2026033012254233700_ref228","first-page":"2701","article-title":"Why is posterior sampling better than optimism for reinforcement learning?","author":"Osband","year":"2017"},{"issue":"13","key":"2026033012254233700_ref229","doi-asserted-by":"crossref","first-page":"1547","DOI":"10.1177\/0278364916645661","article-title":"Robust constrained learning-based NMPC enabling reliable mobile robot path tracking","volume":"35","author":"Ostafew","year":"2016","journal-title":"The International Journal of Robotics Research."},{"key":"2026033012254233700_ref230","first-page":"2721","volume-title":"Count-based exploration with neural density models","author":"Ostrovski","year":"2017"},{"issue":"2","key":"2026033012254233700_ref231","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1109\/TEVC.2006.890271","article-title":"Intrinsic motivation systems for autonomous mental development","volume":"11","author":"Oudeyer","year":"2007","journal-title":"IEEE transactions on evolutionary computation."},{"key":"2026033012254233700_ref232","first-page":"29","volume-title":"How can we define intrinsic motivation","author":"Oudeyer","year":"2008"},{"issue":"2","key":"2026033012254233700_ref233","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1109\/72.279189","article-title":"Application of the recurrent multilayer perceptron in modeling complex process dynamics","volume":"5","author":"Parlos","year":"1994","journal-title":"IEEE Transactions on Neural Networks."},{"key":"2026033012254233700_ref234","first-page":"752","volume-title":"An analysis of linear models, linear value-function approximation, and feature selection for reinforcement learning","author":"Parr","year":"2008"},{"key":"2026033012254233700_ref235","volume-title":"Learning model-based planning from scratch","author":"Pascanu","year":"2017"},{"key":"2026033012254233700_ref236","first-page":"16","volume-title":"Curiosity-driven exploration by self-supervised prediction","author":"Pathak","year":"2017"},{"key":"2026033012254233700_ref237","volume-title":"Unsu-pervised learning of goal spaces for intrinsically motivated goal exploration","author":"P\u00e9r\u00e9","year":"2018"},{"key":"2026033012254233700_ref238","first-page":"307","volume-title":"Learning Policies with External Memory","author":"Peshkin","year":"1999"},{"key":"2026033012254233700_ref239","volume-title":"Model-Based Deep Reinforcement Learning for High-Dimensional Problems, a Survey","author":"Plaat","year":"2020"},{"key":"2026033012254233700_ref240","volume-title":"Parameter space noise for exploration","author":"Plappert","year":"2017"},{"issue":"2","key":"2026033012254233700_ref241","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1007\/s10846-017-0468-y","article-title":"Survey of model-based reinforcement learning: Applications on robotics","volume":"86","author":"Polydoros","year":"2017","journal-title":"Journal of Intelligent & Robotic Systems."},{"key":"2026033012254233700_ref242","volume-title":"Temporal Difference Models: Model-Free Deep RL for Model-Based Control","author":"Pong","year":"2018"},{"key":"2026033012254233700_ref243","first-page":"2827","volume-title":"Neural Episodic Control","author":"Pritzel","year":"2017"},{"key":"2026033012254233700_ref244","volume-title":"Markov Decision Processes.: Discrete Stochastic Dynamic Programming.","author":"Puterman","year":"2014"},{"key":"2026033012254233700_ref245","first-page":"5690","article-title":"Imagination-augmented agents for deep reinforcement learning","volume-title":"Advances in neural information processing systems.","author":"Racani\u00e8re"},{"key":"2026033012254233700_ref246","volume-title":"A short survey on memory based reinforcement learning","author":"Ramani","year":"2019"},{"key":"2026033012254233700_ref247","first-page":"10424","article-title":"Learning abstract options","volume-title":"Advances in Neural Information Processing Systems.","author":"Riemer","year":"2018"},{"key":"2026033012254233700_ref248","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1613\/jair.3987","article-title":"A survey of multi-objective sequential decision-making","volume":"48","author":"Roijers","year":"2013","journal-title":"Journal of Artificial Intelligence Research."},{"issue":"1","key":"2026033012254233700_ref249","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/978-3-031-01576-2","article-title":"Multi-objective decision making","volume":"11","author":"Roijers","year":"2017","journal-title":"Synthesis Lectures on Artificial Intelligence and Machine Learning."},{"key":"2026033012254233700_ref250","volume-title":"On-line Q-learning using connectionist systems.","author":"Rummery","year":"1994"},{"key":"2026033012254233700_ref251","volume-title":"Artificial intelligence: a modern approach.","author":"Russell","year":"2016"},{"issue":"6","key":"2026033012254233700_ref252","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1147\/rd.116.0601","article-title":"Some studies in machine learning using the game of checkers. II - Recent progress","volume":"11","author":"Samuel","year":"1967","journal-title":"IBM Journal of research and development."},{"key":"2026033012254233700_ref253","volume-title":"Disentangling Controllable and Uncontrollable Factors of Variation by Interacting with the World","author":"Sawada","year":"2018"},{"key":"2026033012254233700_ref254","first-page":"1312","volume-title":"Universal value function approximators","author":"Schaul","year":"2015"},{"key":"2026033012254233700_ref255","volume-title":"Prioritized Experience Replay","author":"Schaul","year":"2016"},{"key":"2026033012254233700_ref256","first-page":"1458","volume-title":"Curious model-building control systems","author":"Schmidhuber","year":"1991"},{"key":"2026033012254233700_ref257","volume-title":"Mastering atari, go, chess and shogi by planning with a learned model","author":"Schrittwieser","year":"2019"},{"key":"2026033012254233700_ref258","volume-title":"Planning to Explore via Self-Supervised World Models","author":"Sekar","year":"2020"},{"issue":"5","key":"2026033012254233700_ref259","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1177\/1059712314543837","article-title":"Learning by appraising: an emotion-based approach to intrinsic reward design","volume":"22","author":"Sequeira","year":"2014","journal-title":"Adaptive Behavior."},{"key":"2026033012254233700_ref260","first-page":"1134","volume-title":"Time-contrastive networks: Self-supervised learning from video","author":"Sermanet","year":"2018"},{"key":"2026033012254233700_ref261","volume-title":"Dynamics-Aware Unsupervised Discovery of Skills","author":"Sharma","year":"2019"},{"key":"2026033012254233700_ref262","volume-title":"Hierarchical and interpretable skill acquisition in multi-task reinforcement learning","author":"Shu","year":"2017"},{"key":"2026033012254233700_ref263","first-page":"5779","volume-title":"Model-Based Active Exploration","author":"Shyam","year":"2019"},{"issue":"12","key":"2026033012254233700_ref264","doi-asserted-by":"crossref","first-page":"1115","DOI":"10.1016\/j.robot.2011.07.006","article-title":"On-line regression algorithms for learning mechanical models of robots: a survey","volume":"59","author":"Sigaud","year":"2011","journal-title":"Robotics and Autonomous Systems."},{"key":"2026033012254233700_ref265","first-page":"3191","volume-title":"The predictron: End-to-end learning and planning","author":"Silver","year":"2017"},{"issue":"6419","key":"2026033012254233700_ref266","doi-asserted-by":"crossref","first-page":"1140","DOI":"10.1126\/science.aar6404","article-title":"A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play","volume":"362","author":"Silver","year":"2018","journal-title":"Science."},{"issue":"7676","key":"2026033012254233700_ref267","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature."},{"key":"2026033012254233700_ref268","first-page":"968","volume-title":"Sample-based learning and search with permanent and transient memories","author":"Silver","year":"2008"},{"key":"2026033012254233700_ref269","first-page":"2164","article-title":"Monte-Carlo planning in large POMDPs","volume-title":"Advances in neural information processing systems.","author":"Silver","year":"2010"},{"key":"2026033012254233700_ref270","first-page":"816","volume-title":"Identifying useful subgoals in reinforcement learning by local graph partitioning","author":"\u015eim\u015fek","year":"2005"},{"key":"2026033012254233700_ref271","first-page":"361","article-title":"Reinforcement learning with soft state aggregation","volume-title":"Advances in neural information processing systems.","author":"Singh","year":"1995"},{"key":"2026033012254233700_ref272","first-page":"2399","volume-title":"A point-based POMDP algorithm for robot planning","author":"Spaan","year":"2004"},{"issue":"1","key":"2026033012254233700_ref273","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1111\/j.1467-7687.2007.00569.x","article-title":"Core knowledge","volume":"10","author":"Spelke","year":"2007","journal-title":"Developmental science."},{"key":"2026033012254233700_ref274","volume-title":"Universal planning networks","author":"Srinivas","year":"2018"},{"key":"2026033012254233700_ref275","volume-title":"Incentivizing exploration in reinforcement learning with deep predictive models","author":"Stadie","year":"2015"},{"key":"2026033012254233700_ref276","volume-title":"Pac reinforcement learning bounds for rtdp and rand-rtdp","author":"Strehl","year":"2006"},{"key":"2026033012254233700_ref277","first-page":"216","volume-title":"Integrated architectures for learning, planning, and reacting based on approximating dynamic programming","author":"Sutton","year":"1990"},{"issue":"4","key":"2026033012254233700_ref278","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1145\/122344.122377","article-title":"Dyna, an integrated architecture for learning, planning, and reacting","volume":"2","author":"Sutton","year":"1991","journal-title":"ACM Sigart Bulletin."},{"key":"2026033012254233700_ref279","volume-title":"Reinforcement learning: An introduction.","author":"Sutton","year":"2018"},{"issue":"1-2","key":"2026033012254233700_ref280","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/S0004-3702(99)00052-1","article-title":"Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning","volume":"112","author":"Sutton","year":"1999","journal-title":"Artificial intelligence."},{"key":"2026033012254233700_ref281","volume-title":"Dyna-style planning with linear function approximation and prioritized sweeping","author":"Sutton","year":"2012"},{"key":"2026033012254233700_ref282","first-page":"528","volume-title":"Dyna-style Planning with Linear Function Approximation and Prioritized Sweeping","author":"Sutton","year":"2008"},{"key":"2026033012254233700_ref283","article-title":"Model-based reinforcement learning with nearly tight exploration complexity bounds","volume-title":"ICML.","author":"Szita","year":"2010"},{"key":"2026033012254233700_ref284","first-page":"780","article-title":"Model Regularization for Stable Sample Rollouts.","volume-title":"UAI.","author":"Talvitie","year":"2014"},{"key":"2026033012254233700_ref285","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v31i1.10850","volume-title":"Self-correcting models for model-based reinforcement learning","author":"Talvitie","year":"2017"},{"key":"2026033012254233700_ref286","first-page":"2154","article-title":"Value iteration networks","volume-title":"Advances in Neural Information Processing Systems.","author":"Tamar","year":"2016"},{"key":"2026033012254233700_ref287","first-page":"1633","article-title":"Transfer learning for reinforcement learning domains: A survey","volume":"10","author":"Taylor","year":"2009","journal-title":"Journal of Machine Learning Research."},{"key":"2026033012254233700_ref288","first-page":"1068","volume-title":"Advances in Neural Information Processing Systems 9.","author":"Tesauro","year":"1997"},{"key":"2026033012254233700_ref289","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v31i1.10744","volume-title":"A deep hierarchical approach to lifelong learning in minecraft","author":"Tessler","year":"2017"},{"key":"2026033012254233700_ref290","volume-title":"Disentangling the independently controllable factors of variation by interacting with the world","author":"Thomas","year":"2018"},{"issue":"3\/4","key":"2026033012254233700_ref291","doi-asserted-by":"crossref","first-page":"285","DOI":"10.2307\/2332286","article-title":"On the likelihood that one unknown probability exceeds another in view of the evidence of two samples","volume":"25","author":"Thompson","year":"1933","journal-title":"Biometrika."},{"key":"2026033012254233700_ref292","first-page":"385","article-title":"Finding structure in reinforcement learning","volume-title":"Advances in neural information processing systems.","author":"Thrun","year":"1995"},{"key":"2026033012254233700_ref293","volume-title":"Efficient exploration in reinforcement learning","author":"Thrun","year":"1992"},{"key":"2026033012254233700_ref327","first-page":"23","volume-title":"Domain randomization for transferring deep neural networks from simulation to the real world","author":"Tobin","year":"2017"},{"key":"2026033012254233700_ref294","first-page":"300","volume-title":"A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems","author":"Todorov","year":"2005"},{"issue":"4","key":"2026033012254233700_ref295","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1037\/h0061626","article-title":"Cognitive maps in rats and men","volume":"55","author":"Tolman","year":"1948","journal-title":"Psychological review."},{"key":"2026033012254233700_ref296","first-page":"3928","volume-title":"Stable reinforcement learning with autoencoders for tactile and visual data","author":"Van Hoof","year":"2016"},{"key":"2026033012254233700_ref297","first-page":"33","article-title":"The LoCA Regret: A Consistent Metric to Evaluate Model-Based Behavior in Reinforcement Learning","volume-title":"Advances in Neural Information Processing Systems.","author":"Van Seijen","year":"2020"},{"key":"2026033012254233700_ref298","volume-title":"Relational neural expectation maximization: Unsupervised discovery of objects and their interactions","author":"Van Steenkiste","year":"2018"},{"key":"2026033012254233700_ref299","first-page":"2314","volume-title":"A deeper look at planning as learning from replay","author":"Vanseijen","year":"2015"},{"key":"2026033012254233700_ref300","first-page":"1937","article-title":"Bootstrapping from game tree search","volume-title":"Advances in neural information processing systems.","author":"Veness","year":"2009"},{"key":"2026033012254233700_ref301","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v29i1.9590","volume-title":"Improving multi-step prediction of learned time series models","author":"Venkatraman","year":"2015"},{"key":"2026033012254233700_ref302","first-page":"3540","volume-title":"Feudal networks for hierarchical reinforcement learning","author":"Vezhnevets","year":"2017"},{"key":"2026033012254233700_ref303","volume-title":"Contrastive explanations for reinforcement learning in terms of expected consequences","author":"van der Waa","year":"2018"},{"key":"2026033012254233700_ref304","volume-title":"From pixels to torques: Policy learning with deep dynamical models","author":"Wahlstr\u00f6m","year":"2015"},{"key":"2026033012254233700_ref305","first-page":"1441","article-title":"Gaussian process dynamical models","volume-title":"Advances in neural information processing systems.","author":"Wang","year":"2006"},{"key":"2026033012254233700_ref306","article-title":"Benchmarking Model-Based Reinforcement Learning","volume-title":"CoRR.","author":"Wang","year":"2019"},{"issue":"3-4","key":"2026033012254233700_ref307","first-page":"279","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Machine learning."},{"issue":"2","key":"2026033012254233700_ref308","first-page":"123","article-title":"The development and generalization of\u201c contingency awareness\u201c in early infancy: Some hypotheses","volume":"12","author":"Watson","year":"1966","journal-title":"Merrill-Palmer Quarterly of Behavior and Development."},{"key":"2026033012254233700_ref309","first-page":"2746","article-title":"Embed to control: A locally linear latent dynamics model for control from raw images","volume-title":"Advances in neural information processing systems.","author":"Watter","year":"2015"},{"key":"2026033012254233700_ref310","volume-title":"Cobra: Data-efficient model-based rl through unsupervised object discovery and curiosity-driven exploration","author":"Watters","year":"2019"},{"key":"2026033012254233700_ref311","first-page":"260","volume-title":"Neural networks for control and system identification","author":"Werbos","year":"1989"},{"key":"2026033012254233700_ref312","first-page":"223","volume-title":"Efficient model-based exploration","author":"Wiering","year":"1998"},{"key":"2026033012254233700_ref313","first-page":"1","volume-title":"Model-based multi-objective reinforcement learning","author":"Wiering","year":"2014"},{"key":"2026033012254233700_ref314","volume-title":"Value targets in off-policy AlphaZero: a new greedy backup","author":"Willemsen","year":"2020"},{"issue":"3-4","key":"2026033012254233700_ref315","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1023\/A:1022672621406","article-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","volume":"8","author":"Williams","year":"1992","journal-title":"Machine learning."},{"key":"2026033012254233700_ref316","first-page":"1015","volume-title":"Multi-task reinforcement learning: a hierarchical Bayesian approach","author":"Wilson","year":"2007"},{"issue":"5232","key":"2026033012254233700_ref317","doi-asserted-by":"crossref","first-page":"1880","DOI":"10.1126\/science.7569931","article-title":"An internal model for sensorimotor integration","volume":"269","author":"Wolpert","year":"1995","journal-title":"Science."},{"key":"2026033012254233700_ref318","first-page":"127","article-title":"Galileo: Perceiving physical object properties by integrating a physics engine with deep learning","volume":"28","author":"Wu","year":"2015","journal-title":"Advances in neural information processing systems."},{"key":"2026033012254233700_ref319","volume-title":"Unsupervised discovery of parts, structure, and dynamics","author":"Xu","year":"2019"},{"key":"2026033012254233700_ref320","first-page":"311","volume-title":"Model-Based Multi-objective Reinforcement Learning with Unknown Weights","author":"Yamaguchi","year":"2019"},{"key":"2026033012254233700_ref321","volume-title":"Seqgan: Sequence generative adversarial nets with policy gradient","author":"Yu","year":"2017"},{"key":"2026033012254233700_ref322","first-page":"7304","volume-title":"Tighter problem-dependent regret bounds in reinforcement learning without domain knowledge using value function bounds","author":"Zanette","year":"2019"},{"key":"2026033012254233700_ref323","first-page":"12611","volume-title":"World model as a graph: Learning latent landmarks for planning","author":"Zhang","year":"2021"},{"key":"2026033012254233700_ref324","first-page":"7444","volume-title":"SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning","author":"Zhang","year":"2019"},{"key":"2026033012254233700_ref325","volume-title":"Transfer Learning in Deep Reinforcement Learning: A Survey","author":"Zhu","year":"2020"},{"key":"2026033012254233700_ref326","volume-title":"Latent normalizing flows for discrete sequences","author":"Ziegler","year":"2019"}],"container-title":["Foundations and Trends\u00ae in Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/ftmal\/article-pdf\/16\/1\/1\/11135021\/2200000086en.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/www.emerald.com\/ftmal\/article-pdf\/16\/1\/1\/11135021\/2200000086en.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T18:10:51Z","timestamp":1777486251000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.emerald.com\/ftmal\/article\/16\/1\/1\/1331296\/Model-based-Reinforcement-Learning-A-Survey"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,4]]},"references-count":327,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,1,4]]}},"URL":"https:\/\/doi.org\/10.1561\/2200000086","relation":{},"ISSN":["1935-8237","1935-8245"],"issn-type":[{"value":"1935-8237","type":"print"},{"value":"1935-8245","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,4]]}}}