{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T05:35:18Z","timestamp":1782970518864,"version":"3.54.5"},"reference-count":66,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,4,30]],"date-time":"2025-04-30T00:00:00Z","timestamp":1745971200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>Understanding the world in terms of objects and the possible interactions with them is an important cognitive ability. However, current world models adopted in reinforcement learning typically lack this structure and represent the world state in a global latent vector. To address this, we propose FOCUS, a model-based agent that learns an object-centric world model. This novel representation also enables the design of an object-centric exploration mechanism, which encourages the agent to interact with objects and discover useful interactions. We benchmark FOCUS in several robotic manipulation settings, where we found that our method can be used to improve manipulation skills. The object-centric world model leads to more accurate predictions of the objects in the scene and it enables more efficient learning. The object-centric exploration strategy fosters interactions with the objects in the environment, such as reaching, moving, and rotating them, and it allows fast adaptation of the agent to sparse reward reinforcement learning tasks. Using a Franka Emika robot arm, we also showcase how FOCUS proves useful in real-world applications. Website: <jats:ext-link><jats:monospace>focus-manipulation.github.io<\/jats:monospace><\/jats:ext-link>.<\/jats:p>","DOI":"10.3389\/fnbot.2025.1585386","type":"journal-article","created":{"date-parts":[[2025,4,30]],"date-time":"2025-04-30T05:38:33Z","timestamp":1745991513000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["FOCUS: object-centric world models for robotic manipulation"],"prefix":"10.3389","volume":"19","author":[{"given":"Stefano","family":"Ferraro","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pietro","family":"Mazzaglia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tim","family":"Verbelen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bart","family":"Dhoedt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,4,30]]},"reference":[{"key":"B1","first-page":"29304","article-title":"Deep reinforcement learning at the edge of the statistical precipice","volume":"34","author":"Agarwal","year":"2021","journal-title":"Adv Neural Inf Process Syst"},{"key":"B2","article-title":"Concrete problems in AI safety","author":"Amodei","year":"2016","journal-title":"arXiv [Preprint]."},{"key":"B3","article-title":"Large-scale study of curiosity-driven learning","author":"Burda","year":"","journal-title":"arXiv [Preprint]."},{"key":"B4","article-title":"Exploration by random network distillation","author":"Burda","year":"","journal-title":"arXiv [Preprint]."},{"key":"B5","article-title":"Monet: unsupervised scene decomposition and representation","author":"Burgess","year":"2019","journal-title":"arXiv [Preprint]."},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.06558","article-title":"Segment and track anything","author":"Cheng","year":"2023","journal-title":"arXiv [Preprint]"},{"key":"B7","unstructured":"Clark\n              J.\n            \n            \n              Amodei\n              D.\n            \n          \n          Faulty Reward Functions in the Wild\n          \n          2016"},{"key":"B8","first-page":"4644","article-title":"\u201cUnsupervised image representation learning with deep latent particles,\u201d","volume-title":"Proceedings of the 39th International Conference on Machine Learning","author":"Daniel","year":"2022"},{"key":"B9","first-page":"4956","article-title":"\u201cDreamerpro: reconstruction-free model-based reinforcement learning with prototypical representations,\u201d","author":"Deng","year":"2022","journal-title":"International Conference on Machine Learning"},{"key":"B10","article-title":"Generalization and robustness implications in object-centric learning","author":"Dittadi","year":"2021","journal-title":"arXiv [Preprint]."},{"key":"B11","first-page":"240","article-title":"\u201cAn object-oriented representation for efficient reinforcement learning,\u201d","author":"Diuk","year":"2008","journal-title":"Proceedings of the 25th International Conference on Machine Learning"},{"key":"B12","doi-asserted-by":"publisher","first-page":"7382","DOI":"10.3390\/s22197382","article-title":"Computational optimization of image-based reinforcement learning for robotics","volume":"22","author":"Ferraro","year":"","journal-title":"Sensors"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-28719-0_3","article-title":"Disentangling shape and pose for object-centric deep active inference models","author":"Ferraro","year":"","journal-title":"arXiv [Preprint]."},{"key":"B14","doi-asserted-by":"publisher","first-page":"20220077","DOI":"10.1098\/rsfs.2022.0077","article-title":"Symmetry and complexity in object-centric deep active inference models","volume":"13","author":"Ferraro","year":"2023","journal-title":"Interf. Focus"},{"key":"B15","first-page":"1587","article-title":"\u201cAddressing function approximation error in actor-critic methods,\u201d","author":"Fujimoto","year":"2018","journal-title":"International Conference on Machine Learning"},{"key":"B16","unstructured":"Goodfellow\n              I.\n            \n            \n              Bengio\n              Y.\n            \n            \n              Courville\n              A.\n            \n          \n          Cambridge, MA\n          MIT Press\n          Deep Learning\n          \n          2016"},{"key":"B17","article-title":"Multi-object representation learning with iterative variational inference","author":"Greff","year":"2020","journal-title":"arXiv [Preprint]."},{"key":"B18","article-title":"Maniskill2: a unified benchmark for generalizable manipulation skills","author":"Gu","year":"2023","journal-title":"arXiv [Preprint]."},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.5281\/ZENODO.1207631","article-title":"World models","author":"Ha","year":"2018","journal-title":"Zenodo"},{"key":"B20","article-title":"Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor","author":"Haarnoja","year":"2018","journal-title":"arXiv [Preprint]."},{"key":"B21","first-page":"2555","article-title":"\u201cLearning latent dynamics for planning from pixels,\u201d","author":"Hafner","year":"2019","journal-title":"ICML"},{"key":"B22","article-title":"\u201cMastering atari with discrete world models,\u201d","author":"Hafner","year":"2021","journal-title":"ICLR"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2301.04104","article-title":"Mastering diverse domains through world models","author":"Hafner","year":"2023","journal-title":"arXiv [Preprint]."},{"key":"B24","article-title":"Temporal difference learning for model predictive control","author":"Hansen","year":"2022","journal-title":"arXiv [Preprint]."},{"key":"B25","article-title":"Entity-centric reinforcement learning for object manipulation from pixels","author":"Haramati","year":"2024","journal-title":"arXiv [Preprint]."},{"key":"B26","doi-asserted-by":"publisher","first-page":"81","DOI":"10.3389\/fncir.2017.00081","article-title":"A theory of how columns in the neocortex enable learning the structure of the world","volume":"11","author":"Hawkins","year":"2017","journal-title":"Front. Neural Circuits"},{"key":"B27","article-title":"When to trust your model: model-based policy optimization","author":"Janner","year":"2021","journal-title":"arXiv [Preprint]."},{"key":"B28","article-title":"\u201cReasoning about physical interactions with object-centric models,\u201d","author":"Janner","year":"2019","journal-title":"International Conference on Learning Representations"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.48550\/Arxiv.1806.10293","article-title":"Qt-Opt: Scalable deep reinforcement learning for vision-based robotic manipulation","author":"Kalashnikov","year":"2018","journal-title":"arXiv [Preprint]."},{"key":"B30","article-title":"Contrastive learning of structured world models","author":"Kipf","year":"2020","journal-title":"arXiv [Preprint]."},{"key":"B31","article-title":"Segment anything","author":"Kirillov","year":"2023","journal-title":"arXiv [Preprint]."},{"key":"B32","unstructured":"Krakovna\n              V.\n            \n          \n          \n          2020"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2110.06192","article-title":"Beyond pick-and-place: Tackling robotic stacking of diverse shapes","author":"Lee","year":"2021","journal-title":"arXiv [Preprint]."},{"key":"B34","first-page":"1334","article-title":"End-to-end training of deep visuomotor policies","volume":"17","author":"Levine","year":"2016","journal-title":"J. Mach. Learn. Res"},{"key":"B35","first-page":"18459","article-title":"\u201cBehavior from the void: Unsupervised active pre-training,\u201d","author":"Liu","year":"2021"},{"key":"B36","article-title":"Object-centric learning with slot attention","author":"Locatello","year":"2020","journal-title":"arXiv [Preprint]."},{"key":"B37","article-title":"\u201cAW-Opt: learning robotic skills with imitation andreinforcement at scale,\u201d","author":"Lu","year":"2021","journal-title":"5th Annual Conference on Robot Learning (CoRL"},{"key":"B38","article-title":"Curiosity-driven exploration via latent Bayesian surprise","author":"Mazzaglia","year":"2022","journal-title":"arXiv [Preprint]."},{"key":"B39","doi-asserted-by":"crossref","DOI":"10.1109\/ICRA57147.2024.10611170","article-title":"\u201cInformation-driven affordance discovery for efficient robotic manipulation,\u201d","volume-title":"2024 IEEE International Conference on Robotics and Automation (ICRA)","author":"Mazzaglia","year":"2024"},{"key":"B40","article-title":"Task-agnostic exploration via policy gradient of a non-parametric state entropy estimate","author":"Mutti","year":"2021","journal-title":"arXiv [Preprint]."},{"key":"B41","article-title":"\u201cInteraction-based disentanglement of entities for object-centric world models,\u201d","author":"Nakano","year":"2023","journal-title":"The Eleventh International Conference on Learning Representations"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1910.07113","article-title":"Solving rubik's cube with a robot hand","author":"Akkaya","year":"2019","journal-title":"arXiv [Preprint]."},{"key":"B43","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1109\/TEVC.2006.890271","article-title":"Intrinsic motivation systems for autonomous mental development","volume":"11","author":"Oudeyer","year":"2007","journal-title":"IEEE Trans. Evol. Comp"},{"key":"B44","doi-asserted-by":"publisher","first-page":"651432","DOI":"10.3389\/fnbot.2021.651432","article-title":"Generative models for active vision","volume":"15","author":"Parr","year":"2021","journal-title":"Front. Neurorobot"},{"key":"B45","article-title":"Curiosity-driven exploration by self-supervised prediction","author":"Pathak","year":"2017","journal-title":"arXiv [Preprint]."},{"key":"B46","article-title":"Self-supervised exploration via disagreement","author":"Pathak","year":"2019","journal-title":"arXiv [Preprint]."},{"key":"B47","article-title":"Data-efficient deep reinforcement learning for dexterous manipulation","author":"Popov","year":"2017","journal-title":"arXiv [Preprint]."},{"key":"B48","article-title":"Touch-based curiosity for sparse-reward tasks","author":"Rajeswar","year":"2021","journal-title":"arXiv [Preprint]."},{"key":"B49","author":"Rajeswar","year":"2023","journal-title":"Mastering the Unsupervised Reinforcement Learning Benchmark From Pixels"},{"key":"B50","unstructured":"\u201cCurious exploration via structured world models yields zero-shot object manipulation,\u201d\n          \n          \n            \n              Sancaktar\n              C.\n            \n            \n              Blaes\n              S.\n            \n            \n              Martius\n              G.\n            \n          \n          Advances in Neural Information Processing Systems\n          \n          2022"},{"key":"B51","first-page":"1458","article-title":"\u201cCurious model-building control systems,\u201d","volume-title":"Proceedings 1991 IEEE International Joint Conference on Neural Networks","author":"Schmidhuber","year":"1991"},{"key":"B52","doi-asserted-by":"publisher","first-page":"604","DOI":"10.1038\/s41586-020-03051-4","article-title":"Mastering atari, go, chess and shogi by planning with a learned model","volume":"588","author":"Schrittwieser","year":"2020","journal-title":"Nature"},{"key":"B53","article-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017","journal-title":"arXiv [Preprint]"},{"key":"B54","article-title":"\u201cPlanning to explore via self-supervised world models,\u201d","author":"Sekar","year":"2020","journal-title":"ICML"},{"key":"B55","article-title":"State entropy maximization with random encoders for efficient exploration","author":"Seo","year":"2021","journal-title":"arXiv [Preprint]."},{"key":"B56","article-title":"Masked world models for visual control","author":"Seo","year":"2022","journal-title":"arXiv [Preprint]."},{"key":"B57","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1080\/01966324.2003.10737616","article-title":"Nearest neighbor estimates of entropy","volume":"23","author":"Singh","year":"2003","journal-title":"Am. J. Mathem. Managem. Sci"},{"key":"B58","doi-asserted-by":"publisher","first-page":"12816","DOI":"10.1111\/desc.12816","article-title":"Self' generated variability in object images predicts vocabulary growth","volume":"22","author":"Slone","year":"2019","journal-title":"Dev. Sci"},{"key":"B59","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1016\/j.tics.2018.02.004","article-title":"The developing infant creates a curriculum for statistical learning","volume":"22","author":"Smith","year":"2018","journal-title":"Trends in Cogn. Sci"},{"key":"B60","doi-asserted-by":"publisher","first-page":"6962","DOI":"10.1109\/TKDE.2024.3399746","article-title":"Scene-driven multimodal knowledge graph construction for embodied ai","volume":"36","author":"Song","year":"2024","journal-title":"IEEE Trans. Knowl. Data Eng"},{"key":"B61","article-title":"Daydreamer: world models for physical robot learning","author":"Wu","year":"2022","journal-title":"arXiv [Preprint]."},{"key":"B62","article-title":"Track anything: segment anything meets videos","author":"Yang","year":"2023","journal-title":"arXiv [Preprint]."},{"key":"B63","article-title":"An investigation into pre-training object-centric representations for reinforcement learning","author":"Yoon","year":"2023","journal-title":"arXiv [Preprint]."},{"key":"B64","unstructured":"\u201cMeta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,\u201d\n          \n          \n            \n              Yu\n              T.\n            \n            \n              Quillen\n              D.\n            \n            \n              He\n              Z.\n            \n            \n              Julian\n              R.\n            \n            \n              Hausman\n              K.\n            \n            \n              Finn\n              C.\n            \n          \n          Conference on Robot Learning (CoRL)\n          \n          2019"},{"key":"B65","article-title":"Fast segment anything","author":"Zhao","year":"2023","journal-title":"arXiv [Preprint]."},{"key":"B66","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2009.12293","article-title":"robosuite: a modular simulation framework and benchmark for robot learning","author":"Zhu","year":"2020","journal-title":"arXiv [Preprint]"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2025.1585386\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,30]],"date-time":"2025-04-30T05:39:01Z","timestamp":1745991541000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2025.1585386\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,30]]},"references-count":66,"alternative-id":["10.3389\/fnbot.2025.1585386"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2025.1585386","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,30]]},"article-number":"1585386"}}