{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T14:36:33Z","timestamp":1787495793839,"version":"build-2736575974"},"reference-count":51,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,2,16]],"date-time":"2024-02-16T00:00:00Z","timestamp":1708041600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,2,16]],"date-time":"2024-02-16T00:00:00Z","timestamp":1708041600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100012330","name":"Cognitive Sciences and Technologies Council","doi-asserted-by":"publisher","award":["10466"],"award-info":[{"award-number":["10466"]}],"id":[{"id":"10.13039\/100012330","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Process Lett"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Hierarchical temporal memory (HTM) is a promising unsupervised machine-learning algorithm that models key principles of neocortical computation. One of the main components of HTM is the spatial pooler (SP), which encodes binary input streams into sparse distributed representations (SDRs). In this paper, we propose an information-theoretic framework for the performance comparison of HTM-spatial pooler (SP) algorithms, specifically, for quantifying the similarities and differences between sparse distributed representations in SP algorithms. We evaluate SP's standalone performance, as well as HTM's overall performance. Our comparison of various SP algorithms using Renyi mutual information, Renyi divergence, and Henze\u2013Penrose divergence measures reveals that the SP algorithm with learning and a logarithmic boosting function yields the most effective and useful data representation. Moreover, the most effective SP algorithm leads to superior HTM results. In addition, we utilize our proposed framework to compare HTM with other state-of-the-art sequential learning algorithms. We illustrate that HTM exhibits superior adaptability to pattern changes over time than long short term memory (LSTM), gated recurrent unit (GRU) and online sequential extreme learning machine (OS-ELM) algorithms. This superiority is evident from the lower Renyi divergence of HTM (0.23) compared to LSTM6000 (0.33), LSTM3000 (0.38), GRU (0.41), and OS-ELM (0.49). HTM also achieved the highest Renyi mutual information value of 0.79, outperforming LSTM6000 (0.73), LSTM3000 (0.71), GRU (0.68), and OS-ELM (0.62). These findings not only confirm the numerous advantages of HTM over other sequential learning algorithm, but also demonstrate the effectiveness of our proposed information-theoretic approach as a powerful framework for comparing and evaluating various learning algorithms.<\/jats:p>","DOI":"10.1007\/s11063-024-11546-8","type":"journal-article","created":{"date-parts":[[2024,2,16]],"date-time":"2024-02-16T06:02:39Z","timestamp":1708063359000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Performance Comparison of Different HTM-Spatial Pooler Algorithms Based on Information-Theoretic Measures"],"prefix":"10.1007","volume":"56","author":[{"given":"Shiva","family":"Sanati","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Modjtaba","family":"Rouhani","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ghosheh Abed","family":"Hodtani","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,2,16]]},"reference":[{"issue":"6","key":"11546_CR1","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1016\/j.tics.2006.04.008","volume":"10","author":"AA Ghazanfar","year":"2006","unstructured":"Ghazanfar AA, Schroeder CE (2006) Is neocortex essentially multisensory? Trends Cogn Sci 10(6):278\u2013285","journal-title":"Trends Cogn Sci"},{"issue":"14","key":"11546_CR2","doi-asserted-by":"publisher","first-page":"2991","DOI":"10.1002\/jnr.22444","volume":"88","author":"SD Antic","year":"2010","unstructured":"Antic SD, Zhou WL, Moore AR, Short SM, Ikonomu KD (2010) The decade of the dendritic NMDA spike. J Neurosci Res 88(14):2991\u20133001","journal-title":"J Neurosci Res"},{"key":"11546_CR3","first-page":"11","volume":"5","author":"H Jeff","year":"2004","unstructured":"Jeff H, Sandra B (2004) On intelligence. Times Books 5:11","journal-title":"Times Books"},{"key":"11546_CR4","doi-asserted-by":"crossref","unstructured":"Daylidyonok I, Frolenkova A, Panov AI (2018) Extended hierarchical temporal memory for motion anomaly detection. In: Biologically inspired cognitive architectures meeting pp 69\u201381: Springer","DOI":"10.1007\/978-3-319-99316-4_10"},{"key":"11546_CR5","doi-asserted-by":"crossref","unstructured":"Khangamwa G (2010) Detecting network intrusions using hierarchical temporal memory. In: International conference on e-infrastructure and e-services for developing countries, pp 41\u201348: Springer","DOI":"10.1007\/978-3-642-23828-4_5"},{"key":"11546_CR6","doi-asserted-by":"publisher","first-page":"624","DOI":"10.1016\/j.future.2018.05.014","volume":"88","author":"MA Rodriguez","year":"2018","unstructured":"Rodriguez MA, Kotagiri R, Buyya R (2018) Detecting performance anomalies in scientific workflows using hierarchical temporal memory. Futur Gener Comput Syst 88:624\u2013635","journal-title":"Futur Gener Comput Syst"},{"issue":"14","key":"11546_CR7","doi-asserted-by":"publisher","first-page":"1630","DOI":"10.3390\/electronics10141630","volume":"10","author":"R Sousa","year":"2021","unstructured":"Sousa R, Lima T, Abelha A, Machado J (2021) Hierarchical temporal memory theory approach to stock market time series forecasting. Electronics 10(14):1630","journal-title":"Electronics"},{"key":"11546_CR8","doi-asserted-by":"publisher","first-page":"67241","DOI":"10.1109\/ACCESS.2021.3077004","volume":"9","author":"X Wang","year":"2021","unstructured":"Wang X, Yang K, Liu T (2021) Stock price prediction based on morphological similarity clustering and hierarchical temporal memory. IEEE Access 9:67241\u201367248","journal-title":"IEEE Access"},{"issue":"2021","key":"11546_CR9","first-page":"1","volume":"2021","author":"HM Khan","year":"2021","unstructured":"Khan HM, Khan FM, Khan A, Asghar MZ, Alghazzawi DM (2021) Anomalous behavior detection framework using HTM-based semantic folding technique. Comput Math Methods Med 2021(2021):1\u201314","journal-title":"Comput Math Methods Med"},{"key":"11546_CR10","doi-asserted-by":"publisher","DOI":"10.36227\/techrxiv.12404393.v1","author":"S Heiserman","year":"2023","unstructured":"Heiserman S, Zaychik K, Miller T (2023) Motion-print: a biometric for real-time pilot identification using hierarchical temporal memory. TechRxiv. https:\/\/doi.org\/10.36227\/techrxiv.12404393.v1","journal-title":"TechRxiv"},{"key":"11546_CR11","volume-title":"Anomaly detection using hierarchical temporal memory in smart homes","author":"NO Alshammari","year":"2018","unstructured":"Alshammari NO (2018) Anomaly detection using hierarchical temporal memory in smart homes. Staffordshire University, Stoke-on-Trent"},{"key":"11546_CR12","volume-title":"Anomaly detection with hierarchical temporal memory: a system for detecting potential distress in the elderly and those with dementia","author":"J Reynolds","year":"2020","unstructured":"Reynolds J (2020) Anomaly detection with hierarchical temporal memory: a system for detecting potential distress in the elderly and those with dementia. University of Delaware, Newark"},{"key":"11546_CR13","doi-asserted-by":"publisher","first-page":"101599","DOI":"10.1016\/j.cose.2019.101599","volume":"87","author":"T Li","year":"2019","unstructured":"Li T, Wang B, Shang F, Tian J, Cao K (2019) Online sequential attack detection for ADS-B data based on hierarchical temporal memory. Comput Secur 87:101599","journal-title":"Comput Secur"},{"issue":"7","key":"11546_CR14","doi-asserted-by":"publisher","first-page":"2596","DOI":"10.3390\/app10072596","volume":"10","author":"K Zhang","year":"2020","unstructured":"Zhang K, Zhao F, Luo S, Xin Y, Zhu H, Chen Y (2020) Online intrusion scenario discovery and prediction based on Hierarchical Temporal Memory (HTM). Appl Sci 10(7):2596","journal-title":"Appl Sci"},{"key":"11546_CR15","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1016\/j.neucom.2018.09.098","volume":"396","author":"J Struye","year":"2020","unstructured":"Struye J, Latr\u00e9 S (2020) Hierarchical temporal memory and recurrent neural networks for time series prediction: an empirical validation and reduction to multilayer perceptrons. Neurocomputing 396:291\u2013301","journal-title":"Neurocomputing"},{"issue":"1","key":"11546_CR16","first-page":"7","volume":"12","author":"A Bamaqa","year":"2022","unstructured":"Bamaqa A, Sedky M, Bastaki B (2022) Reactive and proactive anomaly detection in crowd management using hierarchical temporal memory. Int J Mach Learn Comput 12(1):7\u201316","journal-title":"Int J Mach Learn Comput"},{"issue":"10","key":"11546_CR17","doi-asserted-by":"publisher","first-page":"e1000532","DOI":"10.1371\/journal.pcbi.1000532","volume":"5","author":"D George","year":"2009","unstructured":"George D, Hawkins J (2009) Towards a mathematical theory of cortical micro-circuits. PLoS Comput Biol 5(10):e1000532","journal-title":"PLoS Comput Biol"},{"key":"11546_CR18","doi-asserted-by":"publisher","first-page":"81","DOI":"10.3389\/fncir.2017.00081","volume":"11","author":"J Hawkins","year":"2017","unstructured":"Hawkins J, Ahmad S, Cui Y (2017) A theory of how columns in the neocortex enable learning the structure of the world. Front Neural Circuits 11:81","journal-title":"Front Neural Circuits"},{"issue":"1","key":"11546_CR19","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1109\/TETCI.2018.2850314","volume":"3","author":"AM Zyarah","year":"2019","unstructured":"Zyarah AM, Kudithipudi D (2019) Neuromorphic architecture for the hierarchical temporal memory. IEEE Trans Emerg Topics Comput Intell 3(1):4\u201314","journal-title":"IEEE Trans Emerg Topics Comput Intell"},{"key":"11546_CR20","unstructured":"Streat L, Kudithipudi D, Gomez K (2016) Non-volatile hierarchical temporal memory: Hardware for spatial pooling. arXiv preprint arXiv:1611.0279"},{"key":"11546_CR21","doi-asserted-by":"crossref","unstructured":"Padilla DE, McDonnell MD (2014) A neurobiologically plausible vector symbolic architecture. In: 2014 IEEE International conference on semantic computing pp 242\u2013245: IEEE.","DOI":"10.1109\/ICSC.2014.40"},{"key":"11546_CR22","doi-asserted-by":"crossref","unstructured":"Kapu\u015bci\u0144ski T (2010) Hand shape recognition in real images using hierarchical temporal memory trained on synthetic data. In: Image processing and communications challenges 2: Springer, pp 193\u2013200","DOI":"10.1007\/978-3-642-16295-4_22"},{"key":"11546_CR23","unstructured":"Ahmad S, Hawkins J (2015) Properties of sparse distributed representations and their application to hierarchical temporal memory. arXiv preprint arXiv:1503.07469"},{"key":"11546_CR24","unstructured":"Lattner S (2014) Hierarchical temporal memory-investigations, ideas, and experiments. Johannes Kepler Universit\u00e4t, Linz, Austria"},{"key":"11546_CR25","unstructured":"Byrne F (201) Encoding reality: prediction-assisted cortical learning algorithm in hierarchical temporal memory. arXiv preprint arXiv:1509.08255"},{"issue":"5","key":"11546_CR26","first-page":"1111","volume":"9","author":"M Leake","year":"2015","unstructured":"Leake M, Xia L, Rocki K, Imaino W (2015) A probabilistic view of the spatial pooler in hierarchical temporal memory. World Acad Sci Eng Technol Int J Comput Electr Autom Control Inf Eng 9(5):1111\u20131118","journal-title":"World Acad Sci Eng Technol Int J Comput Electr Autom Control Inf Eng"},{"key":"11546_CR27","doi-asserted-by":"publisher","first-page":"81","DOI":"10.3389\/frobt.2016.00081","volume":"3","author":"J Mnatzaganian","year":"2017","unstructured":"Mnatzaganian J, Fokou\u00e9 E, Kudithipudi D (2017) A mathematical formalization of hierarchical temporal memory\u2019s spatial pooler. Front Robot AI 3:81","journal-title":"Front Robot AI"},{"key":"11546_CR28","doi-asserted-by":"publisher","first-page":"1140782","DOI":"10.3389\/fncom.2023.1140782","volume":"17","author":"S Sanati","year":"2023","unstructured":"Sanati S, Rouhani M, Hodtani GA (2023) Information-theoretic analysis of hierarchical temporal memory-spatial pooler algorithm with a new upper bound for the standard information bottleneck method. Front Comput Neurosci 17:1140782","journal-title":"Front Comput Neurosci"},{"key":"11546_CR29","doi-asserted-by":"publisher","first-page":"111","DOI":"10.3389\/fncom.2017.00111","volume":"11","author":"Y Cui","year":"2017","unstructured":"Cui Y, Ahmad S, Hawkins J (2017) The HTM spatial pooler\u2014A neocortical algorithm for online sparse distributed coding. Front Comput Neurosci 11:111","journal-title":"Front Comput Neurosci"},{"key":"11546_CR30","doi-asserted-by":"crossref","unstructured":"Boudreau L, Kudithipudi D (2019) Binding of sparse distributed representations in hierarchical temporal memory. In: Proceedings of the 7th annual neuro-inspired computational elements workshop, 2019, pp 1\u201312","DOI":"10.1145\/3320288.3320299"},{"key":"11546_CR31","doi-asserted-by":"publisher","first-page":"23","DOI":"10.3389\/fncir.2016.00023","volume":"10","author":"J Hawkins","year":"2016","unstructured":"Hawkins J, Ahmad S (2016) Why neurons have thousands of synapses, a theory of sequence memory in neocortex. Front Neural Circuits 10:23","journal-title":"Front Neural Circuits"},{"key":"11546_CR32","doi-asserted-by":"publisher","first-page":"121","DOI":"10.3389\/fncir.2018.00121","volume":"12","author":"J Hawkins","year":"2019","unstructured":"Hawkins J, Lewis M, Klukas M, Purdy S, Ahmad S (2019) A framework for intelligence and cortical function based on grid cells in the neocortex. Front Neural Circuits 12:121","journal-title":"Front Neural Circuits"},{"key":"11546_CR33","volume-title":"Hierarchical temporal memory cortical learning algorithm for pattern recognition on multi-core architectures","author":"RW Price","year":"2011","unstructured":"Price RW (2011) Hierarchical temporal memory cortical learning algorithm for pattern recognition on multi-core architectures. Portland State University, Portland"},{"key":"11546_CR34","doi-asserted-by":"crossref","unstructured":"Dauletkhanuly Y, Krestinskaya O, James AP (2020) HTM theory. In: Deep learning classifiers with memristive networks: Springer, 2020, pp 169\u2013180","DOI":"10.1007\/978-3-030-14524-8_13"},{"key":"11546_CR35","unstructured":"R\u00e9nyi A (1961) On measures of entropy and information. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, vol 4, pp 547\u2013562: University of California Press"},{"key":"11546_CR36","doi-asserted-by":"crossref","unstructured":"Henze N, Penrose MD (1999) On the multivariate runs test. Annals Stat pp 290\u2013298","DOI":"10.1214\/aos\/1018031112"},{"key":"11546_CR37","volume-title":"Probability theory","author":"A Renyi","year":"1970","unstructured":"Renyi A (1970) Probability theory. North-Holland Publishing Company, Dutch"},{"key":"11546_CR38","unstructured":"Ahmad S, Hawkins J (2016) How do neurons operate on sparse distributed representations? A mathematical theory of sparsity, neurons and active dendrites. arXiv preprint arXiv:1601.00720"},{"key":"11546_CR39","unstructured":"LeCun Y (1998) The MNIST database of handwritten digits, http:\/\/yann.lecun.com\/exdb\/mnist\/"},{"key":"11546_CR40","unstructured":"Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747"},{"issue":"11","key":"11546_CR41","doi-asserted-by":"publisher","first-page":"2474","DOI":"10.1162\/NECO_a_00893","volume":"28","author":"Y Cui","year":"2016","unstructured":"Cui Y, Ahmad S, Hawkins J (2016) Continuous online sequence learning with an unsupervised neural network model. Neural Comput 28(11):2474\u20132504","journal-title":"Neural Comput"},{"key":"11546_CR42","doi-asserted-by":"crossref","unstructured":"Cui Y, Surpur C, Ahmad S, Hawkins J (2016) A comparative study of HTM and other neural network models for online sequence learning with streaming data. In: 2016 International joint conference on neural networks (IJCNN), 2016, pp 1530\u20131538: IEEE","DOI":"10.1109\/IJCNN.2016.7727380"},{"key":"11546_CR43","doi-asserted-by":"publisher","first-page":"16801","DOI":"10.1109\/ACCESS.2023.3246388","volume":"11","author":"M Fazil","year":"2023","unstructured":"Fazil M, Khan S, Albahlal BM, Alotaibi RM, Siddiqui T, Shah MA (2023) Attentional multi-channel convolution with bidirectional LSTM cell toward hate speech prediction. IEEE Access 11:16801\u201316811","journal-title":"IEEE Access"},{"key":"11546_CR44","doi-asserted-by":"publisher","first-page":"6893","DOI":"10.1007\/s00521-018-3523-0","volume":"31","author":"CO Sakar","year":"2019","unstructured":"Sakar CO, Polat SO, Katircioglu M, Kastro Y (2019) Real-time prediction of online shoppers\u2019 purchasing intention using multilayer perceptron and LSTM recurrent neural networks. Neural Comput Appl 31:6893\u20136908","journal-title":"Neural Comput Appl"},{"key":"11546_CR45","doi-asserted-by":"publisher","first-page":"107391","DOI":"10.1016\/j.knosys.2021.107391","volume":"230","author":"I Abuqaddom","year":"2021","unstructured":"Abuqaddom I, Mahafzah BA, Faris H (2021) Oriented stochastic loss descent algorithm to train very deep multi-layer neural networks without vanishing gradients. Knowl-Based Syst 230:107391","journal-title":"Knowl-Based Syst"},{"issue":"2","key":"11546_CR46","doi-asserted-by":"publisher","first-page":"1527","DOI":"10.1007\/s11063-022-10950-2","volume":"55","author":"PB Weerakody","year":"2023","unstructured":"Weerakody PB, Wong KW, Wang G (2023) Cyclic gate recurrent neural networks for time series data with missing values. Neural Process Lett 55(2):1527\u20131554","journal-title":"Neural Process Lett"},{"key":"11546_CR47","doi-asserted-by":"crossref","unstructured":"Huang S, Zou F, Guo F, Ren Q (2023) Highway traffic volume prediction based on GRU and attention by ETC data. In: Advances in smart vehicular technology, transportation, communication and applications: proceedings of VTCA 2022: Springer, 2023, pp. 263\u2013275","DOI":"10.1007\/978-981-99-0848-6_20"},{"key":"11546_CR48","unstructured":"Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modelling. arXiv preprint arXiv:1412.3555"},{"issue":"6","key":"11546_CR49","doi-asserted-by":"publisher","first-page":"1411","DOI":"10.1109\/TNN.2006.880583","volume":"17","author":"N-Y Liang","year":"2006","unstructured":"Liang N-Y, Huang G-B, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411\u20131423","journal-title":"IEEE Trans Neural Netw"},{"key":"11546_CR50","doi-asserted-by":"crossref","unstructured":"Wong PK, Vong CM, Gao XH, Wong KI (2014) Adaptive control using fully online sequential-extreme learning machine and a case study on engine air-fuel ratio regulation. Math Problems Eng 2014","DOI":"10.1155\/2014\/246964"},{"key":"11546_CR51","doi-asserted-by":"crossref","unstructured":"Sanati S, Rouhani M, Hodtani GA (2022) Performance comparison of different HTM-spatial pooler algorithms based on information-theoretic measures","DOI":"10.21203\/rs.3.rs-1723895\/v1"}],"container-title":["Neural Processing Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11546-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11063-024-11546-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11546-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T16:19:22Z","timestamp":1715876362000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11063-024-11546-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,16]]},"references-count":51,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["11546"],"URL":"https:\/\/doi.org\/10.1007\/s11063-024-11546-8","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-1723895\/v1","asserted-by":"object"}]},"ISSN":["1573-773X"],"issn-type":[{"value":"1573-773X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,16]]},"assertion":[{"value":"19 January 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 February 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"An earlier version of this paper has been presented as a preprint in Research square [\n                      \n                      ].","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"Not applicable.","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Participate"}},{"value":"Not applicable.","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}}],"article-number":"44"}}