{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,1,18]],"date-time":"2024-01-18T02:26:12Z","timestamp":1705544772605},"reference-count":156,"publisher":"MIT Press","issue":"2","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,1,20]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We present a unified computational theory of an agent's perception and memory. In our model, both perception and memory are realized by different operational modes of the oscillating interactions between a symbolic index layer and a subsymbolic representation layer. The two layers form a bilayer tensor network (BTN). The index layer encodes indices for concepts, predicates, and episodic instances. The representation layer broadcasts information and reflects the cognitive brain state; it is our model of what authors have called the \u201cmental canvas\u201d or the \u201cglobal workspace.\u201d As a bridge between perceptual input and the index layer, the representation layer enables the grounding of indices by their subsymbolic embeddings, which are implemented as connection weights linking both layers. The propagation of activation to earlier perceptual processing layers in the brain can lead to embodiments of indices. Perception and memories first create subsymbolic representations, which are subsequently decoded semantically to produce sequences of activated indices that form symbolic triple statements. The brain is a sampling engine: only activated indices are communicated to the remaining parts of the brain. Triple statements are dynamically embedded in the representation layer and embodied in earlier processing layers: the brain speaks to itself. Although memory appears to be about the past, its main purpose is to support the agent in the present and the future. Recent episodic memory provides the agent with a sense of the here and now. Remote episodic memory retrieves relevant past experiences to provide information about possible future scenarios. This aids the agent in decision making. \u201cFuture\u201d episodic memory, based on expected future events, guides planning and action. Semantic memory retrieves specific information, which is not delivered by current perception, and defines priors for future observations. We argue that it is important for the agent to encode individual entities, not just classes and attributes. Perception is learning: episodic memories are constantly being formed, and we demonstrate that a form of self-supervised learning can acquire new concepts and refine existing ones. We test our model on a standard benchmark data set, which we expanded to contain richer representations for attributes, classes, and individuals. Our key hypothesis is that obtaining a better understanding of perception and memory is a crucial prerequisite to comprehending human-level intelligence.<\/jats:p>","DOI":"10.1162\/neco_a_01552","type":"journal-article","created":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T13:59:18Z","timestamp":1669298358000},"page":"156-227","update-policy":"http:\/\/dx.doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":1,"title":["The Tensor Brain: A Unified Theory of Perception, Memory, and Semantic Decoding"],"prefix":"10.1162","volume":"35","author":[{"given":"Volker","family":"Tresp","sequence":"first","affiliation":[{"name":"LMU Munich and Siemens Munich, Germany volker.tresp@lmu.de"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sahand","family":"Sharifzadeh","sequence":"additional","affiliation":[{"name":"LMU Munich, Germany sahand.sharifzadeh@gmail.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hang","family":"Li","sequence":"additional","affiliation":[{"name":"LMU Munich and Siemens Munich, Germany hang.li@campus.lmu.de"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dario","family":"Konopatzki","sequence":"additional","affiliation":[{"name":"LMU Munich, Germany dk@dkonopatzki.de"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yunpu","family":"Ma","sequence":"additional","affiliation":[{"name":"LMU Munich and Siemens Munich, Germany cognitive.yunpu@gmail.com"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2023,1,20]]},"reference":[{"issue":"82","key":"2023012618411738400_B1","first-page":"1","article-title":"Pykeen 1.0: A Python library for training and evaluating knowledge graph embeddings","volume":"22","author":"Ali","year":"2021","journal-title":"Journal of Machine Learning Research"},{"key":"2023012618411738400_B2","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2018.00636","article-title":"Bottom-up and top-down attention for image captioning and visual question answering","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Anderson","year":"2018"},{"key":"2023012618411738400_B3","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/11442.003.0040","article-title":"Online and off-line memory states in the human brain","volume-title":"The cognitive neurosciences","author":"Awh","year":"2020"},{"key":"2023012618411738400_B4","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:oso\/9780195102659.001.1","volume-title":"In the theater of consciousness: The workspace of the mind","author":"Baars","year":"1997"},{"issue":"4","key":"2023012618411738400_B5","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1016\/0166-2236(88)90145-2","article-title":"Cognitive psychology and human memory","volume":"11","author":"Baddeley","year":"1988","journal-title":"Trends in Neurosciences"},{"issue":"5044","key":"2023012618411738400_B6","doi-asserted-by":"publisher","first-page":"556","DOI":"10.1126\/science.1736359","article-title":"Working memory","volume":"255","author":"Baddeley","year":"1992","journal-title":"Science"},{"key":"2023012618411738400_B7","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1016\/S0079-7421(08)60452-1","article-title":"Working memory","volume":"8","author":"Baddeley","year":"1974","journal-title":"Psychology of Learning and Motivation"},{"key":"2023012618411738400_B8","first-page":"53","article-title":"Improving visual relationship detection using semantic modeling of scene descriptions","volume":"1","author":"Baier","year":"2017","journal-title":"ISWC"},{"key":"2023012618411738400_B9","first-page":"5214","article-title":"Improving information extraction from images with learned semantic models","volume-title":"Proceedings of the 27th International Joint Conference on Artificial Intelligence","author":"Baier","year":"2018"},{"issue":"1","key":"2023012618411738400_B10","doi-asserted-by":"publisher","first-page":"617","DOI":"10.1146\/annurev.psych.59.103006.093639","article-title":"Grounded cognition","volume":"59","author":"Barsalou","year":"2008","journal-title":"Annual Review of Psychology"},{"issue":"3","key":"2023012618411738400_B11","doi-asserted-by":"publisher","DOI":"10.1038\/nn.4502","article-title":"Network neuroscience","volume":"20","author":"Bassett","year":"2017","journal-title":"Nature Neuroscience"},{"issue":"6","key":"2023012618411738400_B12","doi-asserted-by":"publisher","first-page":"722","DOI":"10.1002\/hipo.20095","article-title":"A computational principle for hippocampal learning and neurogenesis","volume":"15","author":"Becker","year":"2005","journal-title":"Hippocampus"},{"issue":"11","key":"2023012618411738400_B13","doi-asserted-by":"publisher","first-page":"527","DOI":"10.1016\/j.tics.2011.10.001","article-title":"The neurobiology of semantic memory","volume":"15","author":"Binder","year":"2011","journal-title":"Trends in Cognitive Sciences"},{"key":"2023012618411738400_B14","author":"Bloice","year":"2017","journal-title":"Augmentor: An image augmentation library for machine learning"},{"key":"2023012618411738400_B15","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2012.00063","article-title":"Consciousness and the prefrontal parietal network: Insights from attention, working memory, and chunking","volume":"3","author":"Bor","year":"2012","journal-title":"Frontiers in Psychology"},{"key":"2023012618411738400_B16","article-title":"Translating embeddings for modeling multi-relational data","volume-title":"Advances in neural information processing systems, 26","author":"Bordes","year":"2013"},{"key":"2023012618411738400_B17","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Advances in neural information processing systems"},{"key":"2023012618411738400_B18","article-title":"AI and the limits of language","author":"Browning","year":"2022","journal-title":"Noema Magazine"},{"key":"2023012618411738400_B19","doi-asserted-by":"crossref","first-page":"357","DOI":"10.7551\/mitpress\/11442.003.0041","article-title":"How working memory works","volume-title":"The cognitive neurosciences","author":"Buschman","year":"2020","edition":"6th"},{"issue":"5","key":"2023012618411738400_B20","doi-asserted-by":"publisher","first-page":"889","DOI":"10.1162\/neco.1995.7.5.889","article-title":"The Helmholtz machine","volume":"7","author":"Dayan","year":"1995","journal-title":"Neural Computation"},{"issue":"27","key":"2023012618411738400_B21","doi-asserted-by":"publisher","first-page":"6539","DOI":"10.1523\/JNEUROSCI.3267-16.2017","article-title":"The hierarchical cortical organization of human speech processing","volume":"37","author":"de Heer","year":"2017","journal-title":"Journal of Neuroscience"},{"key":"2023012618411738400_B22","volume-title":"Consciousness and the brain: Deciphering how the brain codes our thoughts","author":"Dehaene","year":"2014"},{"key":"2023012618411738400_B23","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v32i1.11573","article-title":"Convolutional 2D knowledge graph embeddings","volume-title":"Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence","author":"Dettmers","year":"2018"},{"issue":"11","key":"2023012618411738400_B24","article-title":"Memory states influence value-based decisions","volume":"145","author":"Duncan","year":"2016","journal-title":"Journal of Experimental Psychology: General"},{"issue":"11","key":"2023012618411738400_B25","doi-asserted-by":"publisher","DOI":"10.1038\/nrn3827","article-title":"Time cells in the hippocampus","volume":"15","author":"Eichenbaum","year":"2014","journal-title":"Nature Reviews Neuroscience"},{"issue":"7","key":"2023012618411738400_B26","doi-asserted-by":"publisher","first-page":"1597","DOI":"10.1016\/j.neubiorev.2011.07.006","article-title":"Towards a functional organization of episodic memory in the medial temporal lobe","volume":"36","author":"Eichenbaum","year":"2012","journal-title":"Neuroscience and Biobehavioral Reviews"},{"issue":"10","key":"2023012618411738400_B27","doi-asserted-by":"publisher","first-page":"454","DOI":"10.1016\/j.tics.2003.08.012","article-title":"In two minds: Dual-process accounts of reasoning","volume":"7","author":"Evans","year":"2003","journal-title":"Trends in Cognitive Sciences"},{"issue":"2","key":"2023012618411738400_B28","first-page":"129","article-title":"Cognitive linguistics","volume":"3","author":"Evans","year":"2012","journal-title":"Cognitive Science"},{"key":"2023012618411738400_B29","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1007\/978-90-481-8847-5_10","article-title":"Wordnet","volume-title":"Theory and applications of ontology: Computer applications","author":"Fellbaum","year":"2010"},{"key":"2023012618411738400_B30","volume-title":"The language of thought","author":"Fodor","year":"1975"},{"issue":"2","key":"2023012618411738400_B31","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1038\/nrn1607","article-title":"The organization of recent and remote memories","volume":"6","author":"Frankland","year":"2005","journal-title":"Nature Reviews Neuroscience"},{"issue":"2","key":"2023012618411738400_B32","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: A unified brain theory?","volume":"11","author":"Friston","year":"2010","journal-title":"Nature Reviews Neuroscience"},{"issue":"4","key":"2023012618411738400_B33","doi-asserted-by":"publisher","first-page":"302","DOI":"10.1002\/hbm.460030405","article-title":"Characterising the complexity of neuronal interactions","volume":"3","author":"Friston","year":"1995","journal-title":"Human Brain Mapping"},{"key":"2023012618411738400_B34","volume-title":"The geometry of meaning: Semantics based on conceptual spaces","author":"G\u00e4rdenfors","year":"2016"},{"key":"2023012618411738400_B35","volume-title":"Cognitive neuroscience: The biology of the mind","author":"Gazzaniga","year":"2013","edition":"4th"},{"key":"2023012618411738400_B36","volume-title":"Learning and memory: From brain to behavior","author":"Gluck","year":"2013"},{"issue":"5439","key":"2023012618411738400_B37","doi-asserted-by":"publisher","first-page":"548","DOI":"10.1126\/science.286.5439.548","article-title":"Neurogenesis in the neocortex of adult primates","volume":"286","author":"Gould","year":"1999","journal-title":"Science"},{"key":"2023012618411738400_B38","author":"Goyal","year":"2021","journal-title":"Coordination among neural modules through a shared global workspace"},{"key":"2023012618411738400_B39","author":"Graves","year":"2014","journal-title":"Neural Turing machines"},{"issue":"5","key":"2023012618411738400_B40","doi-asserted-by":"publisher","first-page":"748","DOI":"10.1017\/S1355617710000676","article-title":"Interdependence of episodic and semantic memory: Evidence from neuropsychology","volume":"16","author":"Greenberg","year":"2010","journal-title":"Journal of the International Neuropsychological society"},{"key":"2023012618411738400_B41","article-title":"Bayesian models of cognition","volume-title":"The Cambridge handbook of computational psychology","author":"Griffiths","year":"2008"},{"key":"2023012618411738400_B42","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-28027-6","volume-title":"Tensor spaces and numerical tensor calculus","author":"Hackbusch","year":"2012"},{"issue":"6","key":"2023012618411738400_B43","doi-asserted-by":"publisher","first-page":"803","DOI":"10.1017\/S0140525X98001769","article-title":"Processing capacity defined by relational complexity: Implications for comparative, developmental, and cognitive psychology","volume":"21","author":"Halford","year":"1998","journal-title":"Behavioral and Brain Sciences"},{"key":"2023012618411738400_B44","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/10054.001.0001","volume-title":"Categorizing cognition: Toward conceptual coherence in the foundations of psychology","author":"Halford","year":"2014"},{"key":"2023012618411738400_B45","article-title":"Explainable subgraph reasoning for forecasting on temporal knowledge graphs","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Han","year":"2020"},{"issue":"1\u20133","key":"2023012618411738400_B46","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1016\/0167-2789(90)90087-6","article-title":"The symbol grounding problem","volume":"42","author":"Harnad","year":"1990","journal-title":"Physica D: Nonlinear Phenomena"},{"key":"2023012618411738400_B47","first-page":"1026","article-title":"Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"He","year":"2015"},{"key":"2023012618411738400_B48","article-title":"Automatic acquisition of hyponyms from large text corpora","author":"Hearst","year":"1992","journal-title":"COLING 1992 Volume 2: The Proceedings of the 14th International Conference on Computational Linguistics"},{"issue":"1","key":"2023012618411738400_B49","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1111\/j.1756-8765.2008.01010.x","article-title":"A Bayesian account of reconstructive memory","volume":"1","author":"Hemmer","year":"2009","journal-title":"Topics in Cognitive Science"},{"issue":"3","key":"2023012618411738400_B50","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1162\/neco.1990.2.3.355","article-title":"The bootstrap Widrow-Hoff rule as a cluster formation algorithm","volume":"2","author":"Hinton","year":"1990","journal-title":"Neural Computation"},{"issue":"2","key":"2023012618411738400_B51","doi-asserted-by":"publisher","first-page":"96","DOI":"10.3758\/BF03202365","article-title":"Minerva 2: A simulation model of human memory","volume":"16","author":"Hintzman","year":"1984","journal-title":"Behavior Research Methods, Instruments, and Computers"},{"issue":"8","key":"2023012618411738400_B52","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Computation"},{"issue":"5","key":"2023012618411738400_B53","doi-asserted-by":"publisher","first-page":"849","DOI":"10.1017\/S0140525X01000103","article-title":"The theory of event coding (TEC): A framework for perception and action planning","volume":"24","author":"Hommel","year":"2001","journal-title":"Behavioral and Brain Sciences"},{"key":"2023012618411738400_B54","first-page":"5901","article-title":"Learning by abstraction: The neural state machine","volume-title":"Advances in neural information processing systems, 32","author":"Hudson","year":"2019"},{"issue":"2","key":"2023012618411738400_B55","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.96.2.208","article-title":"Different ways to cue a coherent memory system: A theory for episodic, semantic, and procedural tasks","volume":"96","author":"Humphreys","year":"1989","journal-title":"Psychological Review"},{"key":"2023012618411738400_B56","doi-asserted-by":"publisher","first-page":"453","DOI":"10.1038\/nature17637","article-title":"Natural speech reveals the semantic maps that tile human cerebral cortex","volume":"532","author":"Huth","year":"2016","journal-title":"Nature"},{"key":"2023012618411738400_B57","first-page":"3668","article-title":"Image retrieval using scene graphs","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Johnson","year":"2015"},{"key":"2023012618411738400_B58","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.psych.59.103006.093615","article-title":"The mind and brain of short-term memory","volume":"59","author":"Jonides","year":"2008","journal-title":"Annual Review of Psychology"},{"key":"2023012618411738400_B59","author":"Jung","year":"2020","journal-title":"Augmentor: An image augmentation library for machine learning"},{"key":"2023012618411738400_B60","volume-title":"Thinking, fast and slow","author":"Kahneman","year":"2011"},{"key":"2023012618411738400_B61","volume-title":"Sparse distributed memory","author":"Kanerva","year":"1988"},{"issue":"7","key":"2023012618411738400_B62","doi-asserted-by":"publisher","first-page":"805","DOI":"10.1016\/j.cortex.2011.04.006","article-title":"Conceptual representations in mind and brain: Theoretical developments, current evidence and future directions","volume":"48","author":"Kiefer","year":"2012","journal-title":"Cortex"},{"key":"2023012618411738400_B63","author":"Kingma","year":"2014","journal-title":"Adam: A method for stochastic optimization"},{"issue":"9","key":"2023012618411738400_B64","doi-asserted-by":"publisher","first-page":"438","DOI":"10.1101\/lm.038687.115","article-title":"Entorhinal\u2013hippocampal neuronal circuits bridge temporally discontiguous events","volume":"22","author":"Kitamura","year":"2015","journal-title":"Learning and Memory"},{"issue":"6","key":"2023012618411738400_B65","doi-asserted-by":"publisher","first-page":"1317","DOI":"10.1016\/j.neuron.2015.08.036","article-title":"Entorhinal cortical ocean cells encode specific contexts and drive contextspecific fear memory","volume":"87","author":"Kitamura","year":"2015","journal-title":"Neuron"},{"issue":"12","key":"2023012618411738400_B66","doi-asserted-by":"publisher","first-page":"712","DOI":"10.1016\/j.tins.2004.10.007","article-title":"The Bayesian brain: The role of uncertainty in neural coding and computation","volume":"27","author":"Knill","year":"2004","journal-title":"Trends in Neurosciences"},{"issue":"3","key":"2023012618411738400_B67","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1038\/scientificamericanmind0514-26","article-title":"Keep it in mind","volume":"25","author":"Koch","year":"2014","journal-title":"Scientific American Mind"},{"issue":"5","key":"2023012618411738400_B68","doi-asserted-by":"crossref","DOI":"10.1038\/nrn.2016.22","article-title":"Neural correlates of consciousness: Progress and problems","volume":"17","author":"Koch","year":"2016","journal-title":"Nature Reviews Neuroscience"},{"key":"2023012618411738400_B69","author":"Koner","year":"2020","journal-title":"Relation transformer network"},{"issue":"5","key":"2023012618411738400_B70","doi-asserted-by":"crossref","first-page":"3161","DOI":"10.1152\/jn.00275.2004","article-title":"Bayesian integration in force estimation","volume":"92","author":"K\u00f6rding","year":"2004","journal-title":"Journal of Neurophysiology"},{"issue":"9","key":"2023012618411738400_B71","doi-asserted-by":"publisher","first-page":"1148","DOI":"10.1038\/s41593-018-0210-5","article-title":"Cognitive computational neuroscience","volume":"21","author":"Kriegeskorte","year":"2018","journal-title":"Nature Neuroscience"},{"issue":"1","key":"2023012618411738400_B72","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1007\/s11263-016-0981-7","article-title":"Visual genome: Connecting language and vision using crowdsourced dense image annotations","volume":"123","author":"Krishna","year":"2017","journal-title":"International Journal of Computer Vision"},{"key":"2023012618411738400_B73","author":"Kumar","year":"2015","journal-title":"Ask me anything: Dynamic memory networks for natural language processing"},{"key":"2023012618411738400_B74","doi-asserted-by":"crossref","unstructured":"Kumaran, D., Hassabis, D., & McClelland, J. L. (2016). What learning systems do intelligent agents need?Trends in Cognitive Sciences, 20(7), 512\u2013534. 10.1016\/j.tics.2016.05.004","DOI":"10.1016\/j.tics.2016.05.004"},{"issue":"4","key":"2023012618411738400_B75","first-page":"631","article-title":"Review of philosophy in the flesh: The embodied mind & its challenge to Western thought","volume":"25","author":"Lakoff","year":"1999","journal-title":"Computational Linguistics"},{"key":"2023012618411738400_B76","article-title":"Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks","volume-title":"Workshop on Challenges in Representation Learning","author":"Lee","year":"2013"},{"key":"2023012618411738400_B77","doi-asserted-by":"crossref","first-page":"141","DOI":"10.7551\/mitpress\/12593.003.0013","article-title":"Functional architecture of the cerebral cortex","volume-title":"The neocortex","author":"Leopold","year":"2019"},{"key":"2023012618411738400_B78","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-46448-0_51","article-title":"Visual relationship detection with language priors","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Lu","year":"2016"},{"key":"2023012618411738400_B79","author":"Luo","year":"2019","journal-title":"Context-aware zero-shot recognition"},{"key":"2023012618411738400_B80","article-title":"Holistic representations for memorization and inference","volume-title":"Proceedings of the Conference on Uncertainty in Artificial Intelligence","author":"Ma","year":"2018"},{"issue":"C","key":"2023012618411738400_B81","article-title":"Embedding models for episodic knowledge graphs","volume":"59","author":"Ma","year":"2018","journal-title":"Journal of Web Semantics"},{"issue":"3","key":"2023012618411738400_B82","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1023\/A:1009748302351","article-title":"Discovery of frequent episodes in event sequences","volume":"1","author":"Mannila","year":"1997","journal-title":"Data Mining and Knowledge Discovery"},{"issue":"42","key":"2023012618411738400_B83","doi-asserted-by":"publisher","first-page":"25966","DOI":"10.1073\/pnas.1910416117","article-title":"Placing language in an integrated understanding system: Next steps toward human level performance in neural language models","volume":"117","author":"McClelland","year":"2020","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"3","key":"2023012618411738400_B84","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.102.3.419","article-title":"Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory","volume":"102","author":"McClelland","year":"1995","journal-title":"Psychological Review"},{"issue":"3","key":"2023012618411738400_B85","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1111\/j.1755-2567.1970.tb00434.x","article-title":"Universal grammar","volume":"36","author":"Montague","year":"1970","journal-title":"Theoria"},{"key":"2023012618411738400_B86","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1146\/annurev-psych-113011-143733","article-title":"Episodic memory and beyond: The hippocampus and neocortex in transformation","volume":"67","author":"Moscovitch","year":"2016","journal-title":"Annual Review of Psychology"},{"issue":"2","key":"2023012618411738400_B87","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1016\/S0959-4388(97)80010-4","article-title":"Memory consolidation, retrograde amnesia and the hippocampal complex","volume":"7","author":"Nadel","year":"1997","journal-title":"Current Opinion in Neurobiology"},{"key":"2023012618411738400_B88","first-page":"1179","article-title":"Reducing the rank in relational factorization models by including observable patterns","volume-title":"Advances in neural information processing systems 27","author":"Nickel","year":"2014"},{"key":"2023012618411738400_B89","article-title":"A review of relational machine learning for knowledge graphs","volume-title":"Proceedings of the IEEE","author":"Nickel","year":"2015"},{"key":"2023012618411738400_B90","author":"Nickel","year":"2015","journal-title":"Holographic embeddings of knowledge graphs"},{"key":"2023012618411738400_B91","first-page":"809","article-title":"A three-way model for collective learning on multi-relational data","volume-title":"Proceedings of the 28th International Conference on International Conference on Machine Learning","author":"Nickel","year":"2011"},{"key":"2023012618411738400_B92","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1145\/2187836.2187874","article-title":"Factorizing YAGO: Scalable machine learning for linked data","volume-title":"Proceedings of the 21st International Conference on World Wide Web","author":"Nickel","year":"2012"},{"issue":"4","key":"2023012618411738400_B93","article-title":"Defining n-ary relations on the semantic web","volume":"12","author":"Noy","year":"2006","journal-title":"W3C Working Group Note"},{"key":"2023012618411738400_B94","volume-title":"The meaning of meaning: A study of the influence of thought and of the science of symbolism","author":"Ogden","year":"1923"},{"issue":"2","key":"2023012618411738400_B95","doi-asserted-by":"publisher","DOI":"10.1037\/a0038692","article-title":"Sources of interference in item and associative recognition memory","volume":"122","author":"Osth","year":"2015","journal-title":"Psychological Review"},{"issue":"7","key":"2023012618411738400_B96","doi-asserted-by":"publisher","first-page":"1125","DOI":"10.1016\/0306-4522(80)90191-8","article-title":"Tensorial approach to the geometry of brain function: Cerebellar coordination via a metric tensor","volume":"5","author":"Pellionisz","year":"1980","journal-title":"Neuroscience"},{"key":"2023012618411738400_B97","first-page":"15","article-title":"A common framework for distributed representation schemes for compositional structure","volume-title":"Connectionist systems for knowledge representation and deduction","author":"Plate","year":"1997"},{"issue":"48","key":"2023012618411738400_B98","doi-asserted-by":"publisher","first-page":"30039","DOI":"10.1073\/pnas.1907369117","article-title":"Theoretical issues in deep networks","volume":"117","author":"Poggio","year":"2020","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"1","key":"2023012618411738400_B99","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1016\/0004-3702(90)90005-K","article-title":"Recursive distributed representations","volume":"46","author":"Pollack","year":"1990","journal-title":"Artificial Intelligence"},{"issue":"46","key":"2023012618411738400_B100","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2109380118","article-title":"Modular origins of high-amplitude cofluctuations in fine-scale functional connectivity dynamics","volume":"118","author":"Pope","year":"2021","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"1","key":"2023012618411738400_B101","doi-asserted-by":"publisher","first-page":"200","DOI":"10.1515\/tnsci-2019-0035","article-title":"Processing of action and sound verbs in context: An FMRI study","volume":"10","author":"Popp","year":"2019","journal-title":"Translational Neuroscience"},{"issue":"8","key":"2023012618411738400_B102","doi-asserted-by":"publisher","DOI":"10.1038\/nrn3251","article-title":"Concept cells: The building blocks of declarative memory functions","volume":"13","author":"Quiroga","year":"2012","journal-title":"Nature Reviews Neuroscience"},{"issue":"7045","key":"2023012618411738400_B103","doi-asserted-by":"publisher","first-page":"1102","DOI":"10.1038\/nature03687","article-title":"Invariant visual representation by single neurons in the human brain","volume":"435","author":"Quiroga","year":"2005","journal-title":"Nature"},{"issue":"1","key":"2023012618411738400_B104","doi-asserted-by":"publisher","DOI":"10.1038\/nrn.2016.150","article-title":"The neural and computational bases of semantic cognition","volume":"18","author":"Ralph","year":"2017","journal-title":"Nature Reviews Neuroscience"},{"issue":"1","key":"2023012618411738400_B105","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1038\/4580","article-title":"Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects","volume":"2","author":"Rao","year":"1999","journal-title":"Nature Neuroscience"},{"key":"2023012618411738400_B106","first-page":"91","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume-title":"Advances in neural information processing systems, 28","author":"Ren","year":"2015"},{"key":"2023012618411738400_B107","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:oso\/9780198784852.001.0001","volume-title":"Cerebral cortex: Principles of operation","author":"Rolls","year":"2016"},{"issue":"8","key":"2023012618411738400_B108","doi-asserted-by":"publisher","first-page":"1844","DOI":"10.1162\/jocn.2009.21310","article-title":"The function of words: Distinct neural correlates for words denoting differently manipulable objects","volume":"22","author":"Rueschemeyer","year":"2010","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2023012618411738400_B109","article-title":"You can teach an old dog new tricks! On training knowledge graph embeddings","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Ruffinelli","year":"2019"},{"issue":"3","key":"2023012618411738400_B110","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"International Journal of Computer Vision"},{"key":"2023012618411738400_B111","article-title":"Dynamic routing between capsules","volume-title":"Advances in neural information processing systems","author":"Sabour","year":"2017"},{"issue":"7","key":"2023012618411738400_B112","doi-asserted-by":"publisher","first-page":"439","DOI":"10.1038\/s41583-022-00587-4","article-title":"Theories of consciousness","volume":"23","author":"Seth","year":"2022","journal-title":"Nature Reviews Neuroscience"},{"key":"2023012618411738400_B113","doi-asserted-by":"crossref","DOI":"10.4324\/9780203850664","volume-title":"Embodied cognition","author":"Shapiro","year":"2010"},{"key":"2023012618411738400_B114","article-title":"Classification by attention: Scene graph classification with prior knowledge","volume-title":"Proceedings of the National Conference on Artificial Intelligence","author":"Sharifzadeh","year":"2020"},{"key":"2023012618411738400_B115","author":"Sharifzadeh","year":"2019","journal-title":"Improving visual relation detection using depth maps"},{"key":"2023012618411738400_B116","author":"Simonyan","year":"2014","journal-title":"Very deep convolutional networks for largescale image recognition"},{"issue":"1","key":"2023012618411738400_B117","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1111\/j.1749-6632.2001.tb05712.x","article-title":"Consciousness and the binding problem","volume":"929","author":"Singer","year":"2001","journal-title":"Annals of the New York Academy of Sciences"},{"issue":"1","key":"2023012618411738400_B118","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1016\/0004-3702(90)90007-M","article-title":"Tensor product variable binding and the representation of symbolic structures in connectionist systems","volume":"46","author":"Smolensky","year":"1990","journal-title":"Artificial Intelligence"},{"key":"2023012618411738400_B119","article-title":"Reasoning with neural tensor networks for knowledge base completion","volume-title":"Advances in neural information processing systems","author":"Socher","year":"2013"},{"key":"2023012618411738400_B120","first-page":"1714","article-title":"A Bayesian analysis of dynamics in free recall","volume-title":"Advances in neural information processing systems","author":"Socher","year":"2009"},{"issue":"2","key":"2023012618411738400_B121","doi-asserted-by":"publisher","DOI":"10.31887\/DCNS.2018.20.2\/osporns","article-title":"Graph theory methods: Applications in brain networks","volume":"20","author":"Sporns","year":"2018","journal-title":"Dialogues in Clinical Neuroscience"},{"key":"2023012618411738400_B122","volume-title":"Memory and brain","author":"Squire","year":"1987"},{"issue":"2","key":"2023012618411738400_B123","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1016\/0959-4388(95)80023-9","article-title":"Retrograde amnesia and memory consolidation: A neurobiological perspective","volume":"5","author":"Squire","year":"1995","journal-title":"Current Opinion in Neurobiology"},{"key":"2023012618411738400_B124","first-page":"237","article-title":"Word association spaces for predicting semantic similarity effects in episodic memory","volume-title":"Experimental cognitive psychology and its applications: Festschrift in honor of Lyle Bourne, Walter Kintsch, and Thomas Landauer","author":"Steyvers","year":"2004"},{"issue":"7063","key":"2023012618411738400_B125","doi-asserted-by":"publisher","first-page":"1272","DOI":"10.1038\/nature04286","article-title":"Sleep-dependent memory consolidation","volume":"437","author":"Stickgold","year":"2005","journal-title":"Nature"},{"key":"2023012618411738400_B126","first-page":"2440","article-title":"End-to-end memory networks","volume-title":"Advances in neural information processing systems, 28","author":"Sukhbaatar","year":"2015"},{"key":"2023012618411738400_B127","first-page":"1073","article-title":"Learning in reactive sequential decision tasks: The clarion model","volume-title":"Proceedings of International Conference on Neural Networks","author":"Sun","year":"1996"},{"key":"2023012618411738400_B128","article-title":"Detectron2","author":"Wu","year":"2019"},{"issue":"7","key":"2023012618411738400_B129","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1016\/j.tics.2006.05.009","article-title":"Theory-based Bayesian models of inductive learning and reasoning","volume":"10","author":"Tenenbaum","year":"2006","journal-title":"Trends in Cognitive Sciences"},{"issue":"2","key":"2023012618411738400_B130","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1037\/0735-7044.100.2.147","article-title":"The hippocampal memory indexing theory","volume":"100","author":"Teyler","year":"1986","journal-title":"Behavioral Neuroscience"},{"issue":"12","key":"2023012618411738400_B131","doi-asserted-by":"publisher","first-page":"1158","DOI":"10.1002\/hipo.20350","article-title":"The hippocampal indexing theory and episodic memory","volume":"17","author":"Teyler","year":"2007","journal-title":"Hippocampus"},{"issue":"1","key":"2023012618411738400_B132","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"Journal of the Royal Statistical Society: Series B (Methodological)"},{"issue":"4","key":"2023012618411738400_B133","doi-asserted-by":"crossref","DOI":"10.1037\/h0061626","article-title":"Cognitive maps in rats and men","volume":"55","author":"Tolman","year":"1948","journal-title":"Psychological Review"},{"key":"2023012618411738400_B134","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1016\/j.neuropsychologia.2016.07.004","article-title":"Brain connections of words, perceptions and actions: A neurobiological model of spatiotemporal semantic activation in the human cortex","volume":"98","author":"Tomasello","year":"2017","journal-title":"Neuropsychologia"},{"issue":"8","key":"2023012618411738400_B135","doi-asserted-by":"publisher","DOI":"10.1038\/s41583-018-0031-2","article-title":"The role of engram cells in the systems consolidation of memory","volume":"19","author":"Tonegawa","year":"2018","journal-title":"Nature Reviews Neuroscience"},{"key":"2023012618411738400_B136","author":"Tresp","year":"2015","journal-title":"Learning with memory embeddings"},{"key":"2023012618411738400_B137","article-title":"The tensor memory hypothesis","author":"Tresp","year":"2016","journal-title":"NIPS Workshop on Representation Learning"},{"key":"2023012618411738400_B138","article-title":"Tensor memories","author":"Tresp","year":"2017","journal-title":"Proceeedings of the Hippocamp Conference on Cognitive Computational Neuroscience"},{"key":"2023012618411738400_B139","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1007\/978-3-319-58068-5_13","article-title":"Embedding learning for declarative memories","volume-title":"European Semantic Web Conference","author":"Tresp","year":"2017"},{"key":"2023012618411738400_B140","article-title":"A model for perception and memory","author":"Tresp","year":"2019","journal-title":"Proceedings of the Conference on Cognitive Computational Neuroscience"},{"key":"2023012618411738400_B141","author":"Tresp","year":"2020","journal-title":"The tensor brain: Semantic decoding for perception and memory"},{"key":"2023012618411738400_B142","first-page":"2071","article-title":"Complex embeddings for simple link prediction","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Trouillon","year":"2016"},{"key":"2023012618411738400_B143","article-title":"Episodic and semantic memory 1","volume-title":"Organization of memory","author":"Tulving","year":"1972"},{"key":"2023012618411738400_B144","volume-title":"Elements of episodic memory","author":"Tulving","year":"1985"},{"issue":"1","key":"2023012618411738400_B145","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1146\/annurev.psych.53.100901.135114","article-title":"Episodic memory: From mind to brain","volume":"53","author":"Tulving","year":"2002","journal-title":"Annual Review of Psychology"},{"issue":"12","key":"2023012618411738400_B146","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1016\/j.tics.2013.09.012","article-title":"Network hubs in the human brain","volume":"17","author":"van den Heuvel","year":"2013","journal-title":"Trends in Cognitive Sciences"},{"issue":"11","key":"2023012618411738400_B147","article-title":"Visualizing data using t-SNE","volume":"9","author":"Van der Maaten","year":"2008","journal-title":"Journal of Machine Learning Research"},{"key":"2023012618411738400_B148","first-page":"5998","article-title":"Attention is all you need","volume-title":"Advances in neural information processing systems, 30","author":"Vaswani","year":"2017"},{"key":"2023012618411738400_B149","author":"Weston","year":"2014","journal-title":"Memory networks"},{"issue":"5","key":"2023012618411738400_B150","doi-asserted-by":"crossref","first-page":"1249","DOI":"10.1016\/j.cell.2020.10.024","article-title":"The Tolman-Eichenbaum machine: Unifying space and relational memory through generalization in the hippocampal formation","volume":"183","author":"Whittington","year":"2020","journal-title":"Cell"},{"key":"2023012618411738400_B150a","author":"Wu","year":"2019","journal-title":"Detectron2"},{"key":"2023012618411738400_B151","author":"Yang","year":"2014","journal-title":"Embedding entities and relations for learning and inference in knowledge bases"},{"key":"2023012618411738400_B152","first-page":"670","article-title":"Graph R-CNN for scene graph generation","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Yang","year":"2018"},{"key":"2023012618411738400_B153","volume-title":"The cognitive neuroscience of semantic memory","author":"Yee","year":"2014"},{"key":"2023012618411738400_B154","first-page":"5831","article-title":"Neural motifs: Scene graph parsing with global context","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Zellers","year":"2018"},{"key":"2023012618411738400_B155","first-page":"5532","article-title":"Visual translation embedding network for visual relation detection","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Zhang","year":"2017"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/2\/156\/2067726\/neco_a_01552.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/2\/156\/2067726\/neco_a_01552.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,1]],"date-time":"2023-12-01T17:32:47Z","timestamp":1701451967000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/35\/2\/156\/113809\/The-Tensor-Brain-A-Unified-Theory-of-Perception"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,20]]},"references-count":156,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,1,20]]},"published-print":{"date-parts":[[2023,1,20]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01552","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,2]]},"published":{"date-parts":[[2023,1,20]]}}}