{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,20]],"date-time":"2026-04-20T19:08:37Z","timestamp":1776712117852,"version":"3.51.2"},"reference-count":32,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2024,3,31]],"date-time":"2024-03-31T00:00:00Z","timestamp":1711843200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Union\u2019s NextGenerationEU\/PRTR","award":["PDC2022-133597-C41"],"award-info":[{"award-number":["PDC2022-133597-C41"]}]},{"name":"European Union\u2019s NextGenerationEU\/PRTR","award":["PID2022-137344OB-C31"],"award-info":[{"award-number":["PID2022-137344OB-C31"]}]},{"name":"Spanish Ministry of Science and Innovation","award":["PDC2022-133597-C41"],"award-info":[{"award-number":["PDC2022-133597-C41"]}]},{"name":"Spanish Ministry of Science and Innovation","award":["PID2022-137344OB-C31"],"award-info":[{"award-number":["PID2022-137344OB-C31"]}]},{"name":"FEDER Project","award":["PDC2022-133597-C41"],"award-info":[{"award-number":["PDC2022-133597-C41"]}]},{"name":"FEDER Project","award":["PID2022-137344OB-C31"],"award-info":[{"award-number":["PID2022-137344OB-C31"]}]},{"name":"Universidad de la Rep\u00fablica","award":["PDC2022-133597-C41"],"award-info":[{"award-number":["PDC2022-133597-C41"]}]},{"name":"Universidad de la Rep\u00fablica","award":["PID2022-137344OB-C31"],"award-info":[{"award-number":["PID2022-137344OB-C31"]}]},{"name":"MCIN\/AEI\/10.13039\/501100011033\/FEDER, UE","award":["PDC2022-133597-C41"],"award-info":[{"award-number":["PDC2022-133597-C41"]}]},{"name":"MCIN\/AEI\/10.13039\/501100011033\/FEDER, UE","award":["PID2022-137344OB-C31"],"award-info":[{"award-number":["PID2022-137344OB-C31"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>We introduce both conceptual and empirical findings arising from the amalgamation of a robotics cognitive architecture with an embedded physics simulator, aligning with the principles outlined in the intuitive physics literature. The employed robotic cognitive architecture, named CORTEX, leverages a highly efficient distributed working memory known as deep state representation. This working memory inherently encompasses a fundamental ontology, state persistency, geometric and logical relationships among elements, and tools for reading, updating, and reasoning about its contents. Our primary objective is to investigate the hypothesis that the integration of a physics simulator into the architecture streamlines the implementation of various functionalities that would otherwise necessitate extensive coding and debugging efforts. Furthermore, we categorize these enhanced functionalities into broad types based on the nature of the problems they address. These include addressing challenges related to occlusion, model-based perception, self-calibration, scene structural stability, and human activity interpretation. To demonstrate the outcomes of our experiments, we employ CoppeliaSim as the embedded simulator and both a Kinova Gen3 robotic arm and the Open-Manipulator-P as the real-world scenarios. Synchronization is maintained between the simulator and the stream of real events. Depending on the ongoing task, numerous queries are computed, and the results are projected into the working memory. Participating agents can then leverage this information to enhance overall performance.<\/jats:p>","DOI":"10.3390\/s24072249","type":"journal-article","created":{"date-parts":[[2024,3,31]],"date-time":"2024-03-31T13:32:56Z","timestamp":1711891976000},"page":"2249","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Enhancing Robotic Perception through Synchronized Simulation and Physical Common-Sense Reasoning"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-4328-7877","authenticated-orcid":false,"given":"Guillermo","family":"Trinidad Barnech","sequence":"first","affiliation":[{"name":"Facultad de Ingenier\u00eda, Universidad de la Rep\u00fablica, Montevideo 11200, Uruguay"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0373-6200","authenticated-orcid":false,"given":"Gonzalo","family":"Tejera","sequence":"additional","affiliation":[{"name":"Facultad de Ingenier\u00eda, Universidad de la Rep\u00fablica, Montevideo 11200, Uruguay"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5662-5937","authenticated-orcid":false,"given":"Juan","family":"Valle-Lisboa","sequence":"additional","affiliation":[{"name":"Facultad de Ciencias, Universidad de la Rep\u00fablica, Montevideo 11200, Uruguay"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3615-8833","authenticated-orcid":false,"given":"Pedro","family":"N\u00fa\u00f1ez","sequence":"additional","affiliation":[{"name":"Computer and Communication Technology Department, Universidad de Extremadura, 10005 C\u00e1ceres, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pilar","family":"Bachiller","sequence":"additional","affiliation":[{"name":"Computer and Communication Technology Department, Universidad de Extremadura, 10005 C\u00e1ceres, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pablo","family":"Bustos","sequence":"additional","affiliation":[{"name":"Computer and Communication Technology Department, Universidad de Extremadura, 10005 C\u00e1ceres, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1016\/S1364-6613(02)01913-7","article-title":"Conscious thought as simulation of behaviour and perception","volume":"6","author":"Hesslow","year":"2002","journal-title":"Trends Cogn. Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1016\/j.tics.2017.06.002","article-title":"Intuitive Physics: Current Research and Controversies","volume":"21","author":"Kubricht","year":"2017","journal-title":"Trends Cogn. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1016\/j.tics.2017.05.012","article-title":"Mind games: Game engines as an architecture for intuitive physics","volume":"21","author":"Tomer","year":"2017","journal-title":"Trends Cogn. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"18327","DOI":"10.1073\/pnas.1306572110","article-title":"Simulation as an engine of physical scene understanding","volume":"110","author":"Battaglia","year":"2013","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neucom.2004.12.005","article-title":"Internal simulation of perception: A minimal neuro-robotic model","volume":"68","author":"Ziemke","year":"2005","journal-title":"Neurocomputing"},{"key":"ref_6","unstructured":"Arras, K., Baerveldt, A.-J., Balkenius, C., Burgard, W., and Siegwart, R. (2001). Exploring internal simulation of perception in mobile robots, In 2001 Fourth European Workshop on Advanced Mobile Robotics\u2014Proceedings, Lund University Cognitive Studies."},{"key":"ref_7","unstructured":"Wintermute, S. (2019, January 6\u20139). Integrating Action and Reasoning through Simulation. Proceedings of the 2nd Conference on Artificial General Intelligence, Arlington, VA, USA."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.artint.2015.12.003","article-title":"The scope and limits of simulation in automated reasoning","volume":"233","author":"Davis","year":"2016","journal-title":"Artif. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Mania, P., Kenfack, F.K., Neumann, M., and Beetz, M. (October, January 27). Imagination-enabled robot perception. Proceedings of the 2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic.","DOI":"10.1109\/IROS51168.2021.9636359"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kenghagho, F., Neumann, M., Mania, P., Tan, T., Siddiky, F., Weller, R., Zachmann, G., and Beetz, M. (2022, January 28\u201330). NaivPhys4RP-Towards Human-like Robot Perception \u201cPhysical Reasoning based on Embodied Probabilistic Simulation\u201d. Proceedings of the 2022 IEEE-RAS 21st International Conference On Humanoid Robots (Humanoids), Ginowan, Japan.","DOI":"10.1109\/Humanoids53995.2022.10000153"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Sallami, Y., Lemaignan, S., Clodic, A., and Alami, R. (2019, January 3\u20138). Simulation-based physics reasoning for consistent scene estimation in an HRI context. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Venetian Macao, Macau.","DOI":"10.1109\/IROS40897.2019.8968106"},{"key":"ref_12","unstructured":"Barnech, G.T., Tejera, G., Valle-Lisboa, J., N\u00fa\u00f1ez, P., Bachiller, P., and Bustos, P. (2022). ROBOT2022: Fifth Iberian Robotics Conference, Zaragoza, Spain, 23\u201325 November 2022, Springer."},{"key":"ref_13","unstructured":"Bustos, P., Manso-Arg\u00fcelles, L., Bandera, A., Bandera, J.P., Garc\u00eda-Varea, I., and Mart\u00ednez-G\u00f3mez, J. (2016, January 8\u20139). CORTEX: A new Cognitive Architecture for Social Robots. Proceedings of the EUCognition Meeting\u2014Cognitive Robot Architectures, Viena, Austria."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.cogsys.2019.01.003","article-title":"The cortex cognitive robotics architecture: Use cases","volume":"55","author":"Bustos","year":"2019","journal-title":"Cogn. Syst. Res."},{"key":"ref_15","first-page":"13","article-title":"A standard model of the mind: Toward a common computational framework across artificial intelligence, cognitive science, neuroscience, and robotics","volume":"38","author":"Laird","year":"2017","journal-title":"AI Mag."},{"key":"ref_16","unstructured":"Garcia, J.C. (2021). G: A Low-Latency, Shared-Graph for Robotics Cognitive Architectures. [Master\u2019s Thesis, University of Extremadura]."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"N\u00fa\u00f1ez, P., Garc\u00eda, J.C., Bustos, P., and Bsahciller, P. (2022, January 29\u201330). Towards the design of efficient and versatile cognitive robotic architecture based on distributed, low-latency working memory. Proceedings of the International Conference in Advanced Robotics and Competitions, Santa Maria da Feira, Portugal.","DOI":"10.1109\/ICARSC55462.2022.9784798"},{"key":"ref_18","unstructured":"Bergasa, L.M., Oca\u00f1a, M., Barea, R., L\u00f3pez-Guill\u00e9n, E., and Revenga, P. Advances in Physical Agents II, Springer International Publishing."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Beetz, M., Be\u00dfler, D., Haidu, A., Pomarlan, M., Bozcuo\u011flu, A.K., and Bartels, G. (2018, January 21\u201325). Know rob 2.0\u2014A 2nd generation knowledge processing framework for cognition-enabled robotic agents. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460964"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Olson, E. (2011, January 9\u201313). AprilTag: A robust and flexible visual fiducial system. Proceedings of the IEEE International Conference on Robotics and Automation, Shanghai, China.","DOI":"10.1109\/ICRA.2011.5979561"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Rohmer, E., Singh, S.P.N., and Freese, M. (2013, January 3\u20137). CoppeliaSim (formerly V-REP): A Versatile and Scalable Robot Simulation Framework. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6696520"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Mosenlechner, L., and Beetz, M. (2013, January 6\u201310). Fast temporal projection using accurate physics-based geometric reasoning. Proceedings of the IEEE International Conference on Robotics and Automation, Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6630817"},{"key":"ref_23","unstructured":"Zhang, F., Bazarevsky, V., Vakunov, A., Tkachenka, A., Sung, G., Chang, C.L., and Grundmann, M. (2020). Mediapipe hands: On-device real-time hand tracking. arXiv."},{"key":"ref_24","first-page":"8985","article-title":"Modeling expectation violation in intuitive physics with coarse probabilistic object representations","volume":"32","author":"Smith","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_25","unstructured":"Ghallab, M., Howe, A., Knoblock, C., Mcdermott, D., Ram, A., Veloso, M., Weld, D., and Wilkins, D. (2024, February 01). PDDL\u2014The Planning Domain Definition Language. Available online: https:\/\/www.researchgate.net\/profile\/Craig-Knoblock\/publication\/2278933_PDDL_-_The_Planning_Domain_Definition_Language\/links\/0912f50c0c99385e19000000\/PDDL-The-Planning-Domain-Definition-Language.pdf."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1613\/jair.1705","article-title":"The fast downward planning system","volume":"26","author":"Helmert","year":"2006","journal-title":"J. Artif. Intell. Res."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/NECO_a_00912","article-title":"Active inference: A process theory","volume":"29","author":"Friston","year":"2017","journal-title":"Neural Comput."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"780","DOI":"10.1080\/01691864.2023.2225232","article-title":"World models and predictive coding for cognitive and developmental robotics: Frontiers and challenges","volume":"37","author":"Taniguchi","year":"2023","journal-title":"Adv. Robot."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1080\/02643294.2022.2083950","article-title":"Partial mental simulation explains fallacies in physical reasoning","volume":"38","author":"Bass","year":"2022","journal-title":"Cogn. Neuropsychol."},{"key":"ref_30","unstructured":"Ellis, K., Wong, C., Nye, M., Sable-Meyer, M., Cary, L., Morales, L., Hewitt, L., Solar-Lezama, A., and Tenenbaum, J.B. (2020). DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"929267","DOI":"10.3389\/frobt.2022.929267","article-title":"Two ways to make your robot proactive: Reasoning about human intentions or reasoning about possible futures","volume":"9","author":"Buyukgoz","year":"2022","journal-title":"Front. Robot. AI"},{"key":"ref_32","unstructured":"Shekhar, S., Favier, A., Alami, R., and Croitoru, M. (2003, January 15\u201317). A Knowledge Rich Task Planning Framework for Human-Robot Collaboration. Proceedings of the International Conference On Innovative Techniques And Applications Of Artificial Intelligence, Cambridge, UK."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/7\/2249\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:21:57Z","timestamp":1760106117000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/7\/2249"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,31]]},"references-count":32,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["s24072249"],"URL":"https:\/\/doi.org\/10.3390\/s24072249","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,31]]}}}