{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,11]],"date-time":"2026-10-11T05:07:36Z","timestamp":1791695256984,"version":"4.3.4"},"reference-count":65,"publisher":"Springer Science and Business Media LLC","issue":"8059","license":[{"start":{"date-parts":[[2025,4,2]],"date-time":"2025-04-02T00:00:00Z","timestamp":1743552000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,4,2]],"date-time":"2025-04-02T00:00:00Z","timestamp":1743552000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Nature"],"published-print":{"date-parts":[[2025,4,17]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Developing a general algorithm that learns to solve tasks across a wide range of applications has been a fundamental challenge in artificial intelligence. Although current reinforcement-learning algorithms can be readily applied to tasks similar to what they have been developed for, configuring them for new application domains requires substantial human expertise and experimentation\n                    <jats:sup>1,2<\/jats:sup>\n                    . Here we present the third generation of Dreamer, a general algorithm that outperforms specialized methods across over 150 diverse tasks, with a single configuration. Dreamer learns a model of the environment and improves its behaviour by imagining future scenarios. Robustness techniques based on normalization, balancing and transformations enable stable learning across domains. Applied out of the box, Dreamer is, to our knowledge, the first algorithm to collect diamonds in\n                    <jats:italic>Minecraft<\/jats:italic>\n                    from scratch without human data or curricula. This achievement has been posed as a substantial challenge in artificial intelligence that requires exploring farsighted strategies from pixels and sparse rewards in an open world\n                    <jats:sup>3<\/jats:sup>\n                    . Our work allows solving challenging control problems without extensive experimentation, making reinforcement learning broadly applicable.\n                  <\/jats:p>","DOI":"10.1038\/s41586-025-08744-2","type":"journal-article","created":{"date-parts":[[2025,4,4]],"date-time":"2025-04-04T14:01:38Z","timestamp":1743775298000},"page":"647-653","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":199,"title":["Mastering diverse control tasks through world models"],"prefix":"10.1038","volume":"640","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9534-7271","authenticated-orcid":false,"given":"Danijar","family":"Hafner","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jurgis","family":"Pasukonis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jimmy","family":"Ba","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Timothy","family":"Lillicrap","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,4,2]]},"reference":[{"key":"8744_CR1","unstructured":"Andrychowicz, M. et al. What matters for on-policy deep actor\u2013critic methods? A large-scale study. In Proc. International Conference on Learning Representations (ICLR, 2021)."},{"key":"8744_CR2","unstructured":"Huang, S., Dossa, R. F. J., Raffin, A., Kanervisto, A. & Wang, W. The 37 implementation details of proximal policy optimization. The ICLR Blog Track https:\/\/iclr-blog-track.github.io\/2022\/03\/25\/ppo-implementation-details\/ (2023)."},{"key":"8744_CR3","unstructured":"Kanervisto, A. et al. MineRL Diamond 2021 competition: overview, results, and lessons learned. In Proc. NeurIPS Competitions and Demonstrations Track 13\u201328 (PMLR, 2021)."},{"key":"8744_CR4","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","volume":"529","author":"D Silver","year":"2016","unstructured":"Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484\u2013489 (2016).","journal-title":"Nature"},{"key":"8744_CR5","unstructured":"Brockman, G. OpenAI Five intro. https:\/\/blog.gregbrockman.com\/openai-five-benchmark-intro (2018)."},{"key":"8744_CR6","unstructured":"Ouyang, L. et al. Training language models to follow instructions with human feedback. In Proc. 36th International Conference on Neural Information Processing Systems 27730\u201327744 (Association for Computing Machinery, 2022)."},{"key":"8744_CR7","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A. & Klimov, O. Proximal policy optimization algorithms. Preprint at https:\/\/arxiv.org\/abs\/1707.06347 (2017)."},{"key":"8744_CR8","unstructured":"Lillicrap, T. P. et al. Continuous control with deep reinforcement learning. In Proc. International Conference on Learning Representations (ICLR, 2016)."},{"key":"8744_CR9","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih, V. et al. Human-level control through deep reinforcement learning. Nature 518, 529\u2013533 (2015).","journal-title":"Nature"},{"key":"8744_CR10","doi-asserted-by":"publisher","first-page":"604","DOI":"10.1038\/s41586-020-03051-4","volume":"588","author":"J Schrittwieser","year":"2020","unstructured":"Schrittwieser, J. et al. Mastering Atari, Go, Chess and Shogi by planning with a learned model. Nature 588, 604\u2013609 (2020).","journal-title":"Nature"},{"key":"8744_CR11","unstructured":"Jaderberg, M. et al. Reinforcement learning with unsupervised auxiliary tasks. In Proc. International Conference on Learning Representations (ICLR, 2017)."},{"key":"8744_CR12","unstructured":"Anand, A. et al. Unsupervised state representation learning in Atari. In Proc. 33rd International Conference on Neural Information Processing Systems 8769\u20138782 (Association for Computing Machinery, 2019)."},{"key":"8744_CR13","unstructured":"Driess, D., Schubert, I., Florence, P., Li, Y. & Toussaint, M. Reinforcement learning with neural radiance fields. In Proc. 36th International Conference on Neural Information Processing Systems 16931\u201316945 (Association for Computing Machinery, 2022)."},{"key":"8744_CR14","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1038\/nature24270","volume":"550","author":"D Silver","year":"2017","unstructured":"Silver, D. et al. Mastering the game of Go without human knowledge. Nature 550, 354\u2013359 (2017).","journal-title":"Nature"},{"key":"8744_CR15","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1145\/122344.122377","volume":"2","author":"RS Sutton","year":"1991","unstructured":"Sutton, R. S. Dyna, an integrated architecture for learning, planning, and reacting. ACM SIGART Bull. 2, 160\u2013163 (1991).","journal-title":"ACM SIGART Bull."},{"key":"8744_CR16","doi-asserted-by":"crossref","unstructured":"Finn, C. & Levine, S. Deep visual foresight for planning robot motion. In Proc. International Conference on Robotics and Automation 2786\u20132793 (IEEE, 2017).","DOI":"10.1109\/ICRA.2017.7989324"},{"key":"8744_CR17","unstructured":"Ha, D. & Schmidhuber, J. Recurrent world models facilitate policy evolution. In Proc. 32nd International Conference on Neural Information Processing Systems 2455\u20132467 (Association for Computing Machinery, 2018)."},{"key":"8744_CR18","unstructured":"Kaiser, L. et al. Model-based reinforcement learning for Atari. In Proc. International Conference on Learning Representations (ICLR, 2020)."},{"key":"8744_CR19","unstructured":"Guss, W. H. et al. The MineRL competition on sample efficient reinforcement learning using human priors. Preprint at https:\/\/arxiv.org\/abs\/1904.10079 (2019)."},{"key":"8744_CR20","unstructured":"Kanitscheider, I. et al. Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft. Preprint at https:\/\/arxiv.org\/abs\/2106.14876 (2021)."},{"key":"8744_CR21","unstructured":"Baker, B. et al. Video pretraining (VPT): learning to act by watching unlabeled online videos. In Proc. 36th International Conference on Neural Information Processing Systems 24639\u201324654 (Association for Computing Machinery, 2022)."},{"key":"8744_CR22","unstructured":"Hafner, D., Lillicrap, T., Ba, J. & Norouzi, M. Dream to control: learning behaviors by latent imagination. In Proc. International Conference on Learning Representations (ICLR, 2020)."},{"key":"8744_CR23","unstructured":"Hafner, D., Lillicrap, T., Norouzi, M. & Ba, J. Mastering Atari with discrete world models. In Proc. International Conference on Learning Representations (ICLR, 2021)."},{"key":"8744_CR24","unstructured":"Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. In Proc. International Conference on Learning Representations (ICLR, 2014)."},{"key":"8744_CR25","unstructured":"Hafner, D. et al. Learning latent dynamics for planning from pixels. In Proc. 36th International Conference on Machine Learning 2555\u20132565 (PMLR, 2019)."},{"key":"8744_CR26","unstructured":"Kingma, D. P. et al. Improved variational inference with inverse autoregressive flow. In Proc. Advances in Neural Information Processing Systems 29 (NeurIPS, 2016)."},{"key":"8744_CR27","unstructured":"Child, R. Very deep VAEs generalize autoregressive models and can outperform them on images. In Proc. International Conference on Learning Representations (ICLR, 2021)."},{"key":"8744_CR28","unstructured":"Bellemare, M. G., Dabney, W. & Munos, R. A distributional perspective on reinforcement learning. In Proc. 34th International Conference on Machine Learning 449\u2013458 (PMLR, 2017)."},{"key":"8744_CR29","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","volume":"8","author":"RJ Williams","year":"1992","unstructured":"Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 229\u2013256 (1992).","journal-title":"Mach. Learn."},{"key":"8744_CR30","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P. & Levine, S. Soft actor\u2013critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proc. 35th International Conference on Machine Learning (PMLR, 2018)."},{"key":"8744_CR31","unstructured":"Abdolmaleki, A. et al. Maximum a posteriori policy optimisation. In Proc. International Conference on Learning Representations (ICLR, 2018)."},{"key":"8744_CR32","doi-asserted-by":"publisher","first-page":"027001","DOI":"10.1088\/0957-0233\/24\/2\/027001","volume":"24","author":"JBW Webber","year":"2012","unstructured":"Webber, J. B. W. A bi-symmetric log transformation for wide-range data. Meas. Sci. Technol. 24, 027001 (2012).","journal-title":"Meas. Sci. Technol."},{"key":"8744_CR33","unstructured":"Kapturowski, S., Ostrovski, G., Quan, J., Munos, R. & Dabney, W. Recurrent experience replay in distributed reinforcement learning. In Proc. International Conference on Learning Representations (ICLR, 2018)."},{"key":"8744_CR34","unstructured":"Cobbe, K. W., Hilton, J., Klimov, O. & Schulman, J. Phasic policy gradient. In Proc. 38th International Conference on Machine Learning 2020\u20132027 (PMLR, 2021)."},{"key":"8744_CR35","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1613\/jair.3912","volume":"47","author":"MG Bellemare","year":"2013","unstructured":"Bellemare, M. G., Naddaf, Y., Veness, J. & Bowling, M. The arcade learning environment: an evaluation platform for general agents. J. Artif. Intell. Res. 47, 253\u2013279 (2013).","journal-title":"J. Artif. Intell. Res."},{"key":"8744_CR36","doi-asserted-by":"crossref","unstructured":"Hessel, M. et al. Rainbow: combining improvements in deep reinforcement learning. In Proc. Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence 3215\u20133222 (Association for Computing Machinery, 2018).","DOI":"10.1609\/aaai.v32i1.11796"},{"key":"8744_CR37","unstructured":"Dabney, W., Ostrovski, G., Silver, D. & Munos, R. Implicit quantile networks for distributional reinforcement learning. In Proc. 35th International Conference on Machine Learning 1096\u20131105 (PMLR, 2018)."},{"key":"8744_CR38","unstructured":"Cobbe, K., Hesse, C., Hilton, J. & Schulman, J. Leveraging procedural generation to benchmark reinforcement learning. In Proc. 37th International Conference on Machine Learning 2048\u20132056 (Association for Computing Machinery, 2020)."},{"key":"8744_CR39","unstructured":"Beattie, C. et al. DeepMind Lab. Preprint at https:\/\/arxiv.org\/abs\/1612.03801 (2016)."},{"key":"8744_CR40","first-page":"25476","volume":"34","author":"W Ye","year":"2021","unstructured":"Ye, W., Liu, S., Kurutach, T., Abbeel, P. & Gao, Y. Mastering Atari games with limited data. Adv. Neural Inf. Process. Syst. 34, 25476\u201325488 (2021).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"8744_CR41","unstructured":"Micheli, V., Alonso, E. & Fleuret, F. Transformers are sample efficient world models. In Proc. International Conference on Learning Representations (ICLR, 2023)."},{"key":"8744_CR42","unstructured":"Tassa, Y. et al. DeepMind Control Suite. Preprint at https:\/\/arxiv.org\/abs\/1801.00690 (2018)."},{"key":"8744_CR43","unstructured":"Hansen, N., Su, H. & Wang, X. TD-MPC2: scalable, robust world models for continuous control. In Proc. International Conference on Learning Representations (ICLR, 2024)."},{"key":"8744_CR44","unstructured":"Yarats, D., Fergus, R., Lazaric, A. & Pinto, L. Mastering visual continuous control: Improved data-augmented reinforcement learning. In Proc. International Conference on Learning Representations (ICLR, 2022)."},{"key":"8744_CR45","unstructured":"Osband, I. et al. Behaviour suite for reinforcement learning. In Proc. International Conference on Learning Representations (ICLR, 2020)."},{"key":"8744_CR46","doi-asserted-by":"crossref","unstructured":"Dizon-Paradis, O., Wormald, S., Capecci, D., Bhandarkar, A. & Woodard, D. Investigating the practicality of existing reinforcement learning algorithms: a performance comparison. Preprint at https:\/\/www.techrxiv.org\/doi\/full\/10.36227\/techrxiv.23739099.v1 (2023).","DOI":"10.36227\/techrxiv.23739099"},{"key":"8744_CR47","unstructured":"Wang, G. et al. Voyager: an open-ended embodied agent with large language models. Preprint at https:\/\/arxiv.org\/abs\/2305.16291 (2023)."},{"key":"8744_CR48","unstructured":"Hafner, D. Benchmarking the spectrum of agent capabilities. In Proc. International Conference on Learning Representations (ICLR, 2022)."},{"key":"8744_CR49","unstructured":"Reed, S. et al. A generalist agent. Preprint at https:\/\/arxiv.org\/abs\/2205.06175 (2022)."},{"key":"8744_CR50","unstructured":"Samsami, M. R., Zholus, A., Rajendran, J. & Chandar, S. Mastering memory tasks with world models. In Proc. International Conference on Learning Representations (ICLR, 2024)."},{"key":"8744_CR51","unstructured":"Hoffman, M. et al. Acme: a research framework for distributed reinforcement learning. Preprint at https:\/\/arxiv.org\/abs\/2006.00979 (2020)."},{"key":"8744_CR52","unstructured":"Espeholt, L. et al. IMPALA: scalable distributed deep-RL with importance weighted actor-learner architectures. In Proc. 35th International Conference on Machine Learning 1407\u20131416 (PMLR, 2018)."},{"key":"8744_CR53","unstructured":"Robine, J., H\u00f6ftmann, M., Uelwer, T. & Harmeling, S. Transformer-based world models are happy with 100k interactions. In Proc. International Conference on Learning Representations (ICLR, 2023)."},{"key":"8744_CR54","unstructured":"Osband, I., Blundell, C., Pritzel, A. & Van Roy, B. Deep exploration via bootstrapped DQN. In Proc. Advances in Neural Information Processing Systems 29 (NeurIPS, 2016)."},{"key":"8744_CR55","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1613\/jair.5699","volume":"61","author":"MC Machado","year":"2018","unstructured":"Machado, M. C. et al. Revisiting the arcade learning environment: evaluation protocols and open problems for general agents. J. Artif. Intell. Res. 61, 523\u2013562 (2018).","journal-title":"J. Artif. Intell. Res."},{"key":"8744_CR56","doi-asserted-by":"crossref","unstructured":"Hessel, M. et al. Multi-task deep reinforcement learning with PopArt. In Proc. AAAI Conference on Artificial Intelligence Vol. 33, 3796\u20133803 (Association for the Advancement of Artificial Intelligence, 2019).","DOI":"10.1609\/aaai.v33i01.33013796"},{"key":"8744_CR57","doi-asserted-by":"crossref","unstructured":"Cho, K. et al. Learning phrase representations using RNN encoder\u2013decoder for statistical machine translation. In Proc. 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (eds Moschitti, A. et al.) 1724\u20131734 (Association for Computational Linguistics, 2014).","DOI":"10.3115\/v1\/D14-1179"},{"key":"8744_CR58","unstructured":"Van Keirsbilck, M., Keller, A. & Yang, X. Rethinking full connectivity in recurrent neural networks. Preprint at https:\/\/arxiv.org\/abs\/1905.12340 (2019)."},{"key":"8744_CR59","unstructured":"Gruslys, A. et al. The reactor: a fast and sample-efficient actor\u2013critic agent for reinforcement learning. In Proc. International Conference on Learning Representations (ICLR, 2018)."},{"key":"8744_CR60","unstructured":"Brock, A., De, S., Smith, S. L. & Simonyan, K. High-performance large-scale image recognition without normalization. In Proc. 38th International Conference on Machine Learning 1059\u20131071 (PMLR, 2021)."},{"key":"8744_CR61","unstructured":"Ziyin, L., Wang, Z. T. & Ueda, M. LaProp: Separating momentum and adaptivity in Adam. Preprint at https:\/\/arxiv.org\/abs\/2002.04839 (2020)."},{"key":"8744_CR62","unstructured":"Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. International Conference on Learning Representations (ICLR, 2017)."},{"key":"8744_CR63","unstructured":"Schmitt, S., Hessel, M. & Simonyan, K. Off-policy actor\u2013critic with shared experience replay. In Proc. 37th International Conference on Machine Learning 8545\u20138554 (PMLR, 2020)."},{"key":"8744_CR64","unstructured":"Schaul, T., Quan, J., Antonoglou, I. & Silver, D. Prioritized experience replay. In Proc. International Conference on Learning Representations (ICLR, 2018)."},{"key":"8744_CR65","unstructured":"Sharma, S., Srinivas, A. & Ravindran, B. Learning to repeat: fine grained action repetition for deep reinforcement learning. In Proc. International Conference on Learning Representations (ICLR, 2017)."}],"container-title":["Nature"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41586-025-08744-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41586-025-08744-2","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41586-025-08744-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,16]],"date-time":"2025-04-16T09:25:48Z","timestamp":1744795548000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41586-025-08744-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,2]]},"references-count":65,"journal-issue":{"issue":"8059","published-print":{"date-parts":[[2025,4,17]]}},"alternative-id":["8744"],"URL":"https:\/\/doi.org\/10.1038\/s41586-025-08744-2","relation":{},"ISSN":["0028-0836","1476-4687"],"issn-type":[{"value":"0028-0836","type":"print"},{"value":"1476-4687","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,2]]},"assertion":[{"value":"5 April 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 February 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 April 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}