{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T09:43:50Z","timestamp":1774950230119,"version":"3.50.1"},"reference-count":62,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2016,2,19]],"date-time":"2016-02-19T00:00:00Z","timestamp":1455840000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"EU's Seventh Framework Programme","award":["FP7-ICT-270108"],"award-info":[{"award-number":["FP7-ICT-270108"]}]},{"DOI":"10.13039\/100004412","name":"Human Frontier Science Program","doi-asserted-by":"publisher","award":["RGY0088\/2014"],"award-info":[{"award-number":["RGY0088\/2014"]}],"id":[{"id":"10.13039\/100004412","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We present an information-theoretic method permitting one to find structure in a problem space (here, in a spatial navigation domain) and cluster it in ways that are convenient to solve different classes of control problems, which include planning a path to a goal from a known or an unknown location, achieving multiple goals and exploring a novel environment. Our generative nonparametric approach, called the generative embedded Chinese restaurant process (geCRP), extends the family of Chinese restaurant process (CRP) models by introducing a parameterizable notion of distance (or kernel) between the states to be clustered together. By using different kernels, such as the the conditional probability or joint probability of two states, the same geCRP method clusters the environment in ways that are more sensitive to different control-related information, such as goal, sub-goal and path information. We perform a series of simulations in three scenarios\u2014an open space, a grid world with four rooms and a maze having the same structure as the Hanoi Tower\u2014in order to illustrate the characteristics of the different clusters (obtained using different kernels) and their relative benefits for solving planning and control problems.<\/jats:p>","DOI":"10.3390\/e18020061","type":"journal-article","created":{"date-parts":[[2016,2,19]],"date-time":"2016-02-19T11:29:39Z","timestamp":1455881379000},"page":"61","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Nonparametric Problem-Space Clustering: Learning Efficient Codes for Cognitive Control Tasks"],"prefix":"10.3390","volume":"18","author":[{"given":"Domenico","family":"Maisto","sequence":"first","affiliation":[{"name":"Institute for High Performance Computing and Networking, National Research Council, Via Pietro Castellino 111, 80131 Naples, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4248-5360","authenticated-orcid":false,"given":"Francesco","family":"Donnarumma","sequence":"additional","affiliation":[{"name":"Institute of Cognitive Sciences and Technologies, National Research Council, Via S. Martino della Battaglia 44, 00185 Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6813-8282","authenticated-orcid":false,"given":"Giovanni","family":"Pezzulo","sequence":"additional","affiliation":[{"name":"Institute of Cognitive Sciences and Technologies, National Research Council, Via S. Martino della Battaglia 44, 00185 Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2016,2,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1038\/381607a0","article-title":"Emergence of simple-cell receptive field properties by learning a sparse code for natural images","volume":"381","author":"Olshausen","year":"1996","journal-title":"Nature"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.1146\/annurev.neuro.24.1.1193","article-title":"Natural image statistics and neural representation","volume":"24","author":"Simoncelli","year":"2001","journal-title":"Annu. Rev. Neurosci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: A unified brain theory?","volume":"11","author":"Friston","year":"2010","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.cobeha.2015.08.009","article-title":"Reinforcement learning, efficient coding, and the statistics of natural tasks","volume":"5","author":"Botvinick","year":"2015","journal-title":"Curr. Opin. Behav. Sci."},{"key":"ref_5","unstructured":"Russell, S., and Norvig, P. (2009). Artificial Intelligence: A Modern Approach, Prentice Hall. [3rd ed.]."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sutton, R., and Barto, A. (1998). Reinforcement Learning: An Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Van Dijk, S.G., and Polani, D. (2011, January 11\u201315). Grounding sub-goals in information transitions. Proceedings of the 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), Paris, France.","DOI":"10.1109\/ADPRL.2011.5967384"},{"key":"ref_9","unstructured":"Van Dijk, S.G., Polani, D., and Nehaniv, C.L. (2011). Advances in Artificial Life. Darwin Meets von Neumann, Springer."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Van Dijk, S., and Polani, D. (2013). Informational Constraints-Driven Organization in Goal-Directed Behavior. Adv. Complex Syst., 16.","DOI":"10.1142\/S0219525913500161"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Maisto, D., Donnarumma, F., and Pezzulo, G. (2015). Divide et impera: Subgoaling reduces the complexity of probabilistic inference and problem solving. J. R. Soc. Interface, 12.","DOI":"10.1098\/rsif.2014.1335"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Solway, A., Diuk, C., Cordova, N., Yee, D., Barto, A.G., Niv, Y., and Botvinick, M.M. (2014). Optimal behavioral hierarchy. PLoS Comput. Biol., 10.","DOI":"10.1371\/journal.pcbi.1003779"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1038\/nature12160","article-title":"The importance of mixed selectivity in complex cognitive tasks","volume":"497","author":"Rigotti","year":"2013","journal-title":"Nature"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"656","DOI":"10.1016\/j.neuron.2012.02.023","article-title":"Encoding goals but not abstract magnitude in the primate prefrontal cortex","volume":"74","author":"Genovesio","year":"2012","journal-title":"Neuron"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1007\/s00426-009-0237-z","article-title":"Thinking as the Control of Imagination: A Conceptual Framework for Goal-Directed Systems","volume":"73","author":"Pezzulo","year":"2009","journal-title":"Psychol. Res. PRPF"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.pneurobio.2015.09.001","article-title":"Active Inference, homeostatic regulation and adaptive behavioural control","volume":"134","author":"Pezzulo","year":"2015","journal-title":"Prog. Neurobiol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1162\/jocn_a_00886","article-title":"Prefrontal goal-codes emerge as latent states in probabilistic value learning","volume":"28","author":"Stoianov","year":"2015","journal-title":"J. Cogn. Neurosci."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Verschure, P.F.M.J., Pennartz, C.M.A., and Pezzulo, G. (2014). The why, what, where, when and how of goal-directed choice: Neuronal and computational principles. Philos. Trans. R. Soc. B, 369.","DOI":"10.1098\/rstb.2013.0483"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Picard, J. (2006). Combinatorial Stochastic Processes, Springer.","DOI":"10.1007\/b11601500"},{"key":"ref_20","first-page":"2461","article-title":"Distance dependent Chinese restaurant processes","volume":"12","author":"Blei","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_21","unstructured":"Murphy, K.P. (2012). Machine Learning: A Probabilistic Perspective, MIT Press."},{"key":"ref_22","unstructured":"Therrien, C.W. (1989). Decision Estimation and Classification: An Introduction to Pattern Recognition and Related Topics, Wiley."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1214\/aos\/1176342360","article-title":"A Bayesian analysis of some nonparametric problems","volume":"1","author":"Ferguson","year":"1973","journal-title":"Ann. Stat."},{"key":"ref_24","unstructured":"Dahl, D.B. (August, January 30). Distance-based probability distribution for set partitions with applications to Bayesian nonparametrics. JSM Proceedings, Section on Bayesian Statistical Science, Washington, DC, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ahmed, A., and Xing, E. (2008, January 24\u201326). Dynamic Non-Parametric Mixture Models and the Recurrent Chinese Restaurant Process: With Applications to Evolutionary Clustering. Proceedings of the 2008 SIAM International Conference on Data Mining, Atlanta, GA, USA.","DOI":"10.1137\/1.9781611972788.20"},{"key":"ref_26","unstructured":"Zhu, X., Ghahramani, Z., and Lafferty, J. (2005). Time-Sensitive Dirichlet Process Mixture Models, Carnegie Mellon University. Technical Report CMU-CALD-05-104."},{"key":"ref_27","first-page":"881","article-title":"Infinite mixtures of Gaussian process experts","volume":"2","author":"Rasmussen","year":"2002","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_28","unstructured":"Haussler, D. (1999). Convolution Kernels on Discrete Structures, University of California at Santa Cruz. Technical Report UCSC-CRL-99-10."},{"key":"ref_29","unstructured":"Jaakkola, T., and Haussler, D. (1999). Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Shawe-Taylor, J., and Cristianini, N. (2004). Kernel Methods for Pattern Analysis, Cambridge university Press.","DOI":"10.1017\/CBO9780511809682"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Brodersen, K.H., Schofield, T.M., Leff, A.P., Ong, C.S., Lomakina, E.I., Buhmann, J.M., and Stephan, K.E. (2011). Generative embedding for model-based classification of fMRI data. PLoS Comput. Biol., 7.","DOI":"10.1371\/journal.pcbi.1002079"},{"key":"ref_32","unstructured":"Li, M., and Vit\u00e1nyi, P.M. (2009). An Introduction to Kolmogorov Complexity and Its Applications, Springer."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0019-9958(64)90223-2","article-title":"A formal theory of inductive inference. Part I","volume":"7","author":"Solomonoff","year":"1964","journal-title":"Inf. Control"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1016\/S0019-9958(64)90131-7","article-title":"A formal theory of inductive inference. Part II","volume":"7","author":"Solomonoff","year":"1964","journal-title":"Inf. Control"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1109\/TIT.1978.1055913","article-title":"Complexity-based induction systems: Comparisons and convergence theorems","volume":"24","author":"Solomonoff","year":"1978","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_36","unstructured":"Hutter, M. (2005). Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability, Springer."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1070\/RM1970v025n06ABEH001269","article-title":"The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms","volume":"25","author":"Zvonkin","year":"1970","journal-title":"Russ. Math. Surv."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1350016","DOI":"10.1142\/S021952591303001X","article-title":"Informational constraints-driven organization in goal-directed behavior","volume":"16","author":"Polani","year":"2013","journal-title":"Adv. Complex Syst."},{"key":"ref_39","unstructured":"Newell, A., and Simon, H.A. (1972). Human Problem Solving, Prentice Hall."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Sch\u00f6lkopf, B., Tsuda, K., and Vert, J.P. (2004). Kernel Methods in Computational Biology, MIT Press.","DOI":"10.7551\/mitpress\/4057.001.0001"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1109\/72.896793","article-title":"Nonlinear kernel-based statistical pattern analysis","volume":"12","author":"Ruiz","year":"2001","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1023\/A:1008202821328","article-title":"Differential evolution\u2014A simple and efficient heuristic for global optimization over continuous spaces","volume":"11","author":"Storn","year":"1997","journal-title":"J. Glob. Optim."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"721","DOI":"10.1109\/TPAMI.1984.4767596","article-title":"Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images","volume":"PAMI-6","author":"Geman","year":"1984","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_44","unstructured":"Robert, C., and Casella, G. (2013). Monte Carlo Statistical Methods, Springer."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1080\/10618600.2000.10474879","article-title":"Markov chain sampling methods for Dirichlet process mixture models","volume":"9","author":"Neal","year":"2000","journal-title":"J. Comput. Graph. Stat."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1037\/0033-295X.98.3.409","article-title":"The adaptive nature of human categorization","volume":"98","author":"Anderson","year":"1991","journal-title":"Psychol. Rev."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.socnet.2004.11.008","article-title":"Centrality and network flow","volume":"27","author":"Borgatti","year":"2005","journal-title":"Soc. Netw."},{"key":"ref_48","unstructured":"Tishby, N., and Polani, D. (2011). Perception-Action Cycle, Springer."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Barrat, A., Barthelemy, M., and Vespignani, A. (2008). Dynamical Processes on Complex Networks, Cambridge University Press.","DOI":"10.1017\/CBO9780511791383"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1022140919877","article-title":"Recent advances in hierarchical reinforcement learning","volume":"13","author":"Barto","year":"2003","journal-title":"Discret. Event Dyn. Syst."},{"key":"ref_51","unstructured":"Nilsson, N.J. (1971). Problem-Solving Methods in Artificial Intelligence, McGraw-Hill."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1016\/j.tics.2008.02.009","article-title":"Hierarchical models of behavior and prefrontal function","volume":"12","author":"Botvinick","year":"2008","journal-title":"Trends Cogn. Sci."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Kiebel, S.J., Daunizeau, J., and Friston, K.J. (2008). A hierarchy of time-scales and the brain. PLoS Comput. Biol., 4.","DOI":"10.1371\/journal.pcbi.1000209"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1126\/science.1135935","article-title":"Schemas and memory consolidation","volume":"316","author":"Tse","year":"2007","journal-title":"Science"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1016\/j.tics.2014.06.011","article-title":"Internally generated sequences in learning and executing goal-directed behavior","volume":"18","author":"Pezzulo","year":"2014","journal-title":"Trends Cogn. Sci."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1037\/0033-295X.102.3.419","article-title":"Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory","volume":"102","author":"McClelland","year":"1995","journal-title":"Psychol. Rev."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Collins, A., and Koechlin, E. (2012). Reasoning, learning, and creativity: Frontal lobe function and human decision-making. PLoS Biol., 10.","DOI":"10.1371\/journal.pbio.1001293"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"1481","DOI":"10.1126\/science.1252254","article-title":"Foundations of human reasoning in the prefrontal cortex","volume":"344","author":"Donoso","year":"2014","journal-title":"Science"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"486","DOI":"10.1038\/nn.3331","article-title":"Neural representations of events arise from temporal community structure","volume":"16","author":"Schapiro","year":"2013","journal-title":"Nat. Neurosci."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/j.tics.2010.01.004","article-title":"The multiple-demand (MD) system of the primate brain: Mental programs for intelligent behaviour","volume":"14","author":"Duncan","year":"2010","journal-title":"Trends Cogn. Sci."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Passingham, R.E., and Wise, S.P. (2012). The Neurobiology of the Prefrontal Cortex: Anatomy, Evolution, and the Origin of Insight, Oxford University Press.","DOI":"10.1093\/acprof:osobl\/9780199552917.001.0001"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"1550017","DOI":"10.1142\/S0129065715500173","article-title":"A Programmer-Interpreter Neural Network Architecture for Prefrontal Cognitive Control","volume":"25","author":"Donnarumma","year":"2015","journal-title":"Int. J. Neural Syst."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/18\/2\/61\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:19:25Z","timestamp":1760210365000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/18\/2\/61"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,2,19]]},"references-count":62,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2016,2]]}},"alternative-id":["e18020061"],"URL":"https:\/\/doi.org\/10.3390\/e18020061","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,2,19]]}}}