{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:32:33Z","timestamp":1760243553512,"version":"build-2065373602"},"reference-count":71,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2013,8,6]],"date-time":"2013-08-06T00:00:00Z","timestamp":1375747200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>This article is an overview of the SP theory of intelligence, which aims to simplify and integrate concepts across artificial intelligence, mainstream computing and human perception and cognition, with information compression as a unifying theme. It is conceived of as a brain-like system that receives \"New\" information and stores some or all of it in compressed form as \"Old\" information; and it is realised in the form of a computer model, a first version of the SP machine. The matching and unification of patterns and the concept of multiple alignment are central ideas. Using heuristic techniques, the system builds multiple alignments that are \"good\" in terms of information compression. For each multiple alignment, probabilities may be calculated for associated inferences. Unsupervised learning is done by deriving new structures from partial matches between patterns and via heuristic search for sets of structures that are \"good\" in terms of information compression. These are normally ones that people judge to be \"natural\", in accordance with the \"DONSVIC\" principle\u2014the discovery of natural structures via information compression. The SP theory provides an interpretation for concepts and phenomena in several other areas, including \"computing\", aspects of mathematics and logic, the representation of knowledge, natural language processing, pattern recognition, several kinds of reasoning, information storage and retrieval, planning and problem solving, information compression, neuroscience and human perception and cognition. Examples include the parsing and production of language with discontinuous dependencies in syntax, pattern recognition at multiple levels of abstraction and its integration with part-whole relations, nonmonotonic reasoning and reasoning with default values, reasoning in Bayesian networks, including \"explaining away\", causal diagnosis, and the solving of a geometric analogy problem.<\/jats:p>","DOI":"10.3390\/info4030283","type":"journal-article","created":{"date-parts":[[2013,8,6]],"date-time":"2013-08-06T12:33:17Z","timestamp":1375792397000},"page":"283-341","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["The SP Theory of Intelligence: An Overview"],"prefix":"10.3390","volume":"4","author":[{"given":"J Gerard","family":"Wolff","sequence":"first","affiliation":[{"name":"CognitionResearch.org, Menai Bridge, UK"}]}],"member":"1968","published-online":{"date-parts":[[2013,8,6]]},"reference":[{"key":"ref_1","unstructured":"Apart from the period between early 2006 and late 2012, when I was working on other things."},{"key":"ref_2","unstructured":"See www.cognitionresearch.org\/sp.htm#PUBS."},{"key":"ref_3","unstructured":"Wolff, J.G. (2006). Unifying Computing and Cognition: the SP Theory and Its Applications, CognitionResearch.org.uk."},{"key":"ref_4","unstructured":"Some of the text and figures in this article come from the book, with permission. Details of other permissions are given at appropriate points in the article."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1037\/h0054663","article-title":"Some informational aspects of visual perception","volume":"61","author":"Attneave","year":"1954","journal-title":"Psychol. Rev."},{"key":"ref_6","unstructured":"Barlow, H.B. (1959). The Mechanisation of Thought Processes, Her Majesty\u2019s Stationery Office."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Leibovic, K.N. (1969). Information Processes in the Nervous System, Springer.","DOI":"10.1007\/978-3-662-25549-0"},{"key":"ref_8","unstructured":"Also relevant and still of interest is Zipf\u2019s [68] Human Behaviour and the Principle of Least Effort. Incidentally, Barlow later suggested that \u201c... the [original] idea was right in drawing attention to the importance of redundancy in sensory messages ... but it was wrong in emphasizing the main technical use for redundancy, which is compressive coding.\u201d (p. 242 in [69]). As we shall see, the SP theory is closer to Barlow\u2019s original thinking than what he said later."},{"key":"ref_9","unstructured":"This focus on compression of information in binocular vision is distinct from the more usual interest in the way that slight differences between the two images enables us to see the scene in depth."},{"key":"ref_10","unstructured":"Levy, Y., Schlesinger, I.M., and Braine, M.D.S. (1988). Categories and Processes in Language Acquisition, Lawrence Erlbaum."},{"key":"ref_11","unstructured":"See www.cognitionresearch.org\/lang learn.html."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0019-9958(64)90223-2","article-title":"A formal theory of inductive inference. Part I","volume":"7","author":"Solomonoff","year":"1964","journal-title":"Inf. Control"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1016\/S0019-9958(64)90131-7","article-title":"A formal theory of inductive inference. Part II","volume":"7","author":"Solomonoff","year":"1964","journal-title":"Inf. Control"},{"key":"ref_14","unstructured":"Li, M., and Vit\u00e4nyi, P. (2009). An Introduction to Kolmogorov Complexity and Its Applications, Springer."},{"key":"ref_15","unstructured":"Chase, W.G. (1973). Visual Information Processing, Academic Press."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Laird, J.E. (2012). The Soar Cognitive Architecture, MIT Press.","DOI":"10.7551\/mitpress\/7688.001.0001"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1036","DOI":"10.1037\/0033-295X.111.4.1036","article-title":"An integrated theory of the mind","volume":"111","author":"Anderson","year":"2004","journal-title":"Psychol. Rev."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Schmidhuber, J., Th\u00f6risson, K.R., and Looks, M. (2011). Artificial General Intelligence: 4th International Conference, AGI 2011, Mountain View, CA, USA, August 3\u20136, 2011, Proceedings, Springer. Volume 6830, Lecture Notes in Artificial Intelligence.","DOI":"10.1007\/978-3-642-22887-2"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1007\/s11023-011-9235-1","article-title":"Significance of models of computation, from Turing model to natural computation","volume":"21","year":"2011","journal-title":"Minds Mach."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Steunebrink, B.R., and Schmidhuber, J. A family of G\u00f6del machine implementations. In [18]. Available online: www.idsia.ch\/juergen\/agi2011bas.pdf.","DOI":"10.1007\/978-3-642-22887-2_29"},{"key":"ref_21","unstructured":"Hutter, M. (2005). Universal Artificial Intelligence: Sequential Decisions based on Algorithmic Probability, Springer."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1093\/comjnl\/33.6.518","article-title":"Simplicity and power\u2014some unifying ideas in computing","volume":"33","author":"Wolff","year":"1990","journal-title":"Comput. J."},{"key":"ref_23","unstructured":"Of course, people can and do learn with assistance from teachers and others. However, unsupervised learning has been a focus of interest in developing the SP theory, since it is clear that much of our learning is done without assistance and because unsupervised learning raises some interesting issues and yields some useful insights, as outlined in Section 5.2."},{"key":"ref_24","unstructured":"The source code for the models, with associated documents and files, may be downloaded via links under the heading \u201cSOURCE CODE\u201d at the bottom of the page on\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t          http:\/\/bit.ly\/WtXa3g\n\t\t\t\t\t\t\t\t\t\t\t\t          (accessed on 5 August 2013)\n\t\t\t\t\t\t\t\t\t\t\t\t        ."},{"key":"ref_25","unstructured":"As in ordinary search engines and, indeed, in the brains of people and other animals, high levels of parallelism are needed to achieve speedy processing with large data sets (see also Section 4.3 and Section 5.1.5. )."},{"key":"ref_26","unstructured":"Wolff, J.G. The SP theory of intelligence: Benefits and applications. Available online: http:\/\/bit.ly\/12YmQJW."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wolff, J.G. Application of the SP theory of intelligence to the understanding of natural vision and the development of computer vision. Available online: http:\/\/bit.ly\/Xj3nDY.","DOI":"10.1186\/2193-1801-3-552"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Prince, S.J.D. (2012). Computer Vision: Models, Learning, and Inference, Cambridge University Press.","DOI":"10.1017\/CBO9780511996504"},{"key":"ref_29","unstructured":"Whether multiple alignments are shown with patterns in rows or in columns depends largely on what fits best on the page."},{"key":"ref_30","unstructured":"Cover, T.M., and Thomas, J.A. (1991). Elements of Information Theory, Wiley."},{"key":"ref_31","unstructured":"Although this scheme is slightly less efficient than the well-known Huffman scheme, it has been adopted, because, unlike the Huffman scheme, it does not produce anomalous results when probabilities are derived from code sizes, as described in BK (Section 3.7)."},{"key":"ref_32","unstructured":"See, for example, \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t          \u201cSequence alignment\u201d, Wikipedia\n\t\t\t\t\t\t\t\t\t\t\t\t\t          Available online: en.wikipedia.org\/wiki\/Sequence alignment\n\t\t\t\t\t\t\t\t\t\t\t\t\t          ((accessed on 8 May 2013)"},{"key":"ref_33","unstructured":"Sankoff, D., and Kruskall, J.B. (1983). Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparisons, Addison-Wesley."},{"key":"ref_34","unstructured":"WinMerge, latest stable version 2.14.0; Open Source differencing and merging tool for Windows. Available online: http:\/\/winmerge.org."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1016\/j.datak.2006.04.003","article-title":"Towards an intelligent database system founded on the SP theory of computing and cognition","volume":"60","author":"Wolff","year":"2007","journal-title":"Data Knowl. Eng."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1109\/4235.585892","article-title":"Ant colony system: A cooperative learning approach to the traveling salesman problem","volume":"1","author":"Dorigo","year":"1997","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_37","unstructured":"Thus \u201ccomputing as compression\u201d does not imply that all redundancy is bad and should be removed. Redundancy in information is often useful in, for example, understanding speech in noisy conditions (cf., Section 4.2.2) or in backup copies for data."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/0271-5309(82)90035-0","article-title":"Language acquisition, data compression and generalization","volume":"2","author":"Wolff","year":"1982","journal-title":"Lang. Commun."},{"key":"ref_39","unstructured":"In this and other examples in this subsection, we shall assume that letters are analogues of low-level perceptual features in speech, such as formant ratios or formant transitions."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1111\/j.2044-8295.1977.tb01563.x","article-title":"The discovery of segments in natural language","volume":"68","author":"Wolff","year":"1977","journal-title":"Br. J. Psychol."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1177\/002383098002300303","article-title":"Language acquisition and the discovery of phrase structure","volume":"23","author":"Wolff","year":"1980","journal-title":"Lang. Speech"},{"key":"ref_42","unstructured":"The Chomskian doctrine that children are born with a knowledge of \u201cuniversal grammar\u201d fails to account for the specifics of syntactic forms in different languages, and it depends on the still-unproven idea that there is something of substance that is shared by all the world\u2019s languages."},{"key":"ref_43","unstructured":"Relevant evidence comes from cases where children learn to understand language even though they have little or no ability to speak [70,71]\u2014so that there is little or nothing for anyone to correct."},{"key":"ref_44","unstructured":"If an error is not rare, it is likely to acquire the status of a dialect or idiolect variation and cease to be regarded as an error."},{"key":"ref_45","unstructured":"Such as: learning in the kinds of artificial neural network that are popular in computer science; Hebb\u2019s [66] concept of learning; Pavlovian learning; and Skinnerian learning."},{"key":"ref_46","first-page":"230","article-title":"On computable numbers, with an application to the Entscheidungsproblem","volume":"42","author":"Turing","year":"1936","journal-title":"Proc. Lond. Math. Soc."},{"key":"ref_47","first-page":"544","article-title":"On computable numbers, with an application to the Entscheidungsproblem: a correction","volume":"43","author":"Turing","year":"1937","journal-title":"Proc. Lond. Math. Soc."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"197","DOI":"10.2307\/2371809","article-title":"Formal reductions of the general combinatorial decision problem","volume":"65","author":"Post","year":"1943","journal-title":"Am. J. Math."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1038\/scientificamerican0788-80","article-title":"Randomness in arithmetic","volume":"259","author":"Chaitin","year":"1988","journal-title":"Sci. Am."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Ma, Z. (2006). Soft Computing in Ontologies and Semantic Web, Springer-Verlag.","DOI":"10.1007\/3-540-33473-4"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1016\/j.dss.2005.02.005","article-title":"Medical diagnosis as pattern recognition in a framework of information compression by multiple alignment, unification and search","volume":"42","author":"Wolff","year":"2006","journal-title":"Decis. Support Syst."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/0824-7935.00041","article-title":"Introduction to special issue on context in natural language processing","volume":"13","author":"Iwanska","year":"1997","journal-title":"Comput. Intell."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Chomsky, N. (1957). Syntactic Structures, Mouton.","DOI":"10.1515\/9783112316009"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1016\/0004-3702(80)90003-X","article-title":"Definite clause grammars for language analysis\u2014a survey of the formalism and a comparison with augmented transition networks","volume":"13","author":"Pereira","year":"1980","journal-title":"Artif. Intell."},{"key":"ref_55","unstructured":"In this figure, the sentence, \u201cit is wash ed\u201d, could have been represented more elegantly as, \u201ci t i s w a s h e d\u201d, as in previous examples. The form shown here has been adopted, because it helps to stop multiple alignments growing too large. Likewise, with Figure 14."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1016\/j.tics.2007.09.009","article-title":"The role of context in object recognition","volume":"11","author":"Oliva","year":"2007","journal-title":"Trends Cogn. Sci."},{"key":"ref_57","unstructured":"Although the term \u201cheterarchy\u201d is not widely used, it can be useful as a means of referring to hierarchies in which, as in the example in the text, a given node may appear in two or more higherlevel nodes that are not themselves hierarchically related. In the SP framework, there may be heterarchies in both class-inclusion structures and part-whole structures. However, to avoid the clumsy expression \u201chierarchy or heterarchy\u201d, the term \u201chierarchy\u201d is used in most parts of this article as a shorthand for both concepts."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s10462-007-9058-x","article-title":"The Simplicity and Power model for inductive inference","volume":"26","author":"Pothos","year":"2006","journal-title":"Artif. Intell. Rev."},{"key":"ref_59","unstructured":"Pearl, J. (1997). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann. [revised second printing ed.]."},{"key":"ref_60","unstructured":"Likewise, a travel booking clerk using a database of all flights between cities will assume that, if no flight is shown between, say, Edinburgh and Paris, then no such flight exists. In systems like Prolog, the closed-world assumption is the basis of \u201cnegation as failure\u201d: If a proposition cannot be proven with the clauses provided in a Prolog program, then, in terms of that store of knowledge, the proposition is assumed to be false."},{"key":"ref_61","unstructured":"Some of the frequencies shown in Figure 22 are intended to reflect the two probabilities suggested for this example in [59] (p. 49): \u201c... the [alarm] is sensitive to earthquakes and can be accidentally (p = 0:20) triggered by one. ... if an earthquake had occurred, it surely (p = 0:40) would be on the [radio] news.\u201d"},{"key":"ref_62","unstructured":"Minsky, M.L. (1968). Semantic Information Processing, MIT Press."},{"key":"ref_63","first-page":"209","article-title":"Experiments in solving analogy problems using Minimal Length Encoding","volume":"95","author":"Belloti","year":"1996","journal-title":"Appl. Decis. Technol."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1007\/BF01531061","article-title":"The representation and manipulation of the algorithmic probability measure for problem solving","volume":"4","author":"Gammerman","year":"1991","journal-title":"Ann. Math. Artif. Intell."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1017\/S0140525X12001525","article-title":"Can quantum probability provide a new direction for cognitive modeling?","volume":"36","author":"Pothos","year":"2013","journal-title":"Behav. Brain Sci."},{"key":"ref_66","unstructured":"Hebb, D.O. (1949). The Organization of Behaviour, John Wiley & Sons."},{"key":"ref_67","unstructured":"See, for example, \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t          \u201cArtificial neural network\u201d\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t  Wikipedia\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t          Available online: http:\/\/winmerge.org\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t          (accessed on 31 July 2013)"},{"key":"ref_68","unstructured":"Zipf, G.K. (1949). Human Behaviour and the Principle of Least Effort, Hafner."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1080\/net.12.3.241.253","article-title":"Redundancy reduction revisited","volume":"12","author":"Barlow","year":"2001","journal-title":"Netw. Comput. Neural Syst."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1037\/h0041906","article-title":"Understanding language without the ability to speak","volume":"65","author":"Lenneberg","year":"1962","journal-title":"J. Abnorm. Soc. Psychol."},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Brown, R. (1973). A First Language: The Early Stages, Penguin.","DOI":"10.4159\/harvard.9780674732469"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/4\/3\/283\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:48:28Z","timestamp":1760219308000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/4\/3\/283"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,8,6]]},"references-count":71,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2013,9]]}},"alternative-id":["info4030283"],"URL":"https:\/\/doi.org\/10.3390\/info4030283","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2013,8,6]]}}}