{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T01:35:12Z","timestamp":1777685712481,"version":"3.51.4"},"reference-count":24,"publisher":"SAGE Publications","issue":"3,4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["HIS"],"published-print":{"date-parts":[[2023,11,3]]},"abstract":"<jats:p>The cortical learning algorithm (CLA) is a time series prediction algorithm. Memory elements called columns and cells discretely represent data with their state combinations, whereas linking elements called synapses change their state combinations. For tasks requiring to take actions, the action-prediction CLA (ACLA) has an advantage to complement missing state values with their predictions. However, an increase in the number of missing state values (i) generates excess synapses negatively affect the action predictions and (ii) decreases the stability of data representation and makes the output of action values difficult. This paper proposes an adaptive ACLA using (i) adaptive synapse adjustment and (ii) adaptive action-separated decoding in an uncertain environment, missing multiple input state values probabilistically. (i) The proposed adaptive synapse adjustment suppresses unnecessary synapses. (ii) The proposed adaptive action-separated decoding adaptively outputs an action prediction separately for each action value. Experimental results using uncertain two- and three-dimensional mountain car tasks show that the proposed adaptive ACLA achieves a more robust action prediction performance than the conventional ACLA, DDPG, and the three LSTM-assisted reinforcement learning algorithms of DDPG, TD3, and SAC, even though the number of missing state values and their frequencies increase. These results implicate that the proposed adaptive ACLA is a way to making decisions for the future, even in cases where information surrounding the situation partially lacked.<\/jats:p>","DOI":"10.3233\/his-230013","type":"journal-article","created":{"date-parts":[[2023,7,18]],"date-time":"2023-07-18T11:52:59Z","timestamp":1689681179000},"page":"225-245","source":"Crossref","is-referenced-by-count":1,"title":["Adaptive action-prediction cortical learning algorithm under uncertain environments"],"prefix":"10.1177","volume":"19","author":[{"given":"Kazushi","family":"Fujino","sequence":"first","affiliation":[{"name":"The University of Electro-Communications, Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Takeru","family":"Aoki","sequence":"additional","affiliation":[{"name":"Tokyo University of Science, Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Keiki","family":"Takadama","sequence":"additional","affiliation":[{"name":"The University of Electro-Communications, Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hiroyuki","family":"Sato","sequence":"additional","affiliation":[{"name":"The University of Electro-Communications, Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"key":"10.3233\/HIS-230013_ref1","unstructured":"J. Hawkins, A. Subutai and D. Dubinsky, Hierarchical Temporal Memory Including HTM Cortical Learning Algorithms, Technical report, Numenta Inc., 2010."},{"key":"10.3233\/HIS-230013_ref2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/fncir.2016.00023","article-title":"Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex","volume":"10","author":"Hawkins","year":"2016","journal-title":"Frontiers in Neural Circuits"},{"key":"10.3233\/HIS-230013_ref3","unstructured":"HTM.core, 2020 April 21. Available from: https:\/\/github.com\/htm-community\/htm.core."},{"key":"10.3233\/HIS-230013_ref4","unstructured":"J. Hawkins and S. Blakeslee, On Intelligence: How a New Understanding of the Brain Will Lead to the Creation of Truly Intelligent Machines, Times Books, 2005."},{"key":"10.3233\/HIS-230013_ref5","unstructured":"S. Ahmad and J. Hawkins, Properties of Sparse Distributed Representations and their Application to Hierarchical Temporal Memory, Technical report, Numenta Inc., 2015, pp.\u00a01\u201318."},{"issue":"8","key":"10.3233\/HIS-230013_ref6","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Computation"},{"issue":"11","key":"10.3233\/HIS-230013_ref7","doi-asserted-by":"crossref","first-page":"2474","DOI":"10.1162\/NECO_a_00893","article-title":"Continuous Online Sequence Learning with an Unsupervised Neural Network Model","volume":"28","author":"Cui","year":"2016","journal-title":"Neural Computation"},{"issue":"4","key":"10.3233\/HIS-230013_ref8","doi-asserted-by":"crossref","first-page":"450","DOI":"10.20965\/jaciii.2021.p0450","article-title":"Adaptive Synapse Arrangement in Cortical Learning Algorithm","volume":"25","author":"Aoki","year":"2021","journal-title":"Journal of Advanced Computational Intelligence and Intelligent Informatics (JACIII)"},{"key":"10.3233\/HIS-230013_ref9","first-page":"63","article-title":"A Preliminary Study on Cortical Learning Algorithm for Action Decision Using Forecast","author":"Fujino","year":"2021","journal-title":"The 17th Computational Intelligence Workshop"},{"key":"10.3233\/HIS-230013_ref10","unstructured":"T. Lillicrap, J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver and D. Wierstra, Continuous Control with Deep Reinforcement Learning, in: Proc. in the International Conference on Learning Representation (ICLR 2016), 2016."},{"key":"10.3233\/HIS-230013_ref11","unstructured":"Z. Yang and H. Nguyen, Recurrent Off-policy Baselines for Memory-based Continuous Control, arXiv preprint arXiv:2110.12628, 2021."},{"key":"10.3233\/HIS-230013_ref12","unstructured":"N. Heess, J. Hunt, T. Lillicrap and D. Silver, Memory-based Control with Recurrent Neural Networks, arXiv preprint arXiv:1512.04455, 2015."},{"key":"10.3233\/HIS-230013_ref13","first-page":"5619","article-title":"Memory-based Deep Reinforcement Learning for POMDPs","author":"Meng","year":"2021","journal-title":"Proc. in 2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS 2021)"},{"issue":"19","key":"10.3233\/HIS-230013_ref14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/s21196571","article-title":"LSTM-DDPG for Trading with Variable Positions","volume":"21","author":"Jia","year":"2021","journal-title":"Sensors"},{"key":"10.3233\/HIS-230013_ref15","first-page":"16691","article-title":"Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs","volume":"162","author":"Ni","year":"2022","journal-title":"Proc. in the 39th International Conference on Machine Learning (ICML 2022)"},{"key":"10.3233\/HIS-230013_ref16","unstructured":"POMDP Baselines, 2022 July 26. Available from: https:\/\/github.com\/twni2016\/pomdp-baselines."},{"key":"10.3233\/HIS-230013_ref17","doi-asserted-by":"crossref","unstructured":"K. Fujino, T. Aoki, K. Takadama and H. Sato, Adaptive Synapse Adjustment for Multivariate Cortical Learning Algorithm, in: Proc. in 2022 Joint 12th International Conference on Soft Computing and Intelligent Systems and 23rd International Symposium on Advanced Intelligent Systems (SCIS & ISIS 2022), 2022.","DOI":"10.1109\/SCISISIS55246.2022.10002035"},{"key":"10.3233\/HIS-230013_ref18","doi-asserted-by":"crossref","unstructured":"K. Fujino, T. Aoki, K. Takadama and H. Sato, Adaptive Synapse Adjustment and Decoding in Action-prediction Cortical Learning Algorithm, in: Proc. in the 12th World Congress on Nature and Biologically Inspired Computing (NaBIC 2022), 2022.","DOI":"10.1007\/978-3-031-27524-1_79"},{"key":"10.3233\/HIS-230013_ref19","first-page":"1","article-title":"Column-based Decoder of Internal Prediction Representation in Cortical Learning Algorithms","author":"Aoki","year":"2020","journal-title":"Proc. in 2020 Joint 11th International Conference on Soft Computing and Intelligent Systems and 21st International Symposium on Advanced Intelligent Systems (SCIS & ISIS 2020)"},{"key":"10.3233\/HIS-230013_ref20","unstructured":"OpenAI Gym, 2020 August 18. Available from: https:\/\/github.com\/openai\/gym."},{"key":"10.3233\/HIS-230013_ref21","doi-asserted-by":"crossref","first-page":"488","DOI":"10.1007\/978-3-540-87481-2_32","article-title":"Transferring Instances for Model-Based Reinforcement Learning","author":"Taylor","year":"2008","journal-title":"Machine Learning and Knowledge Discovery in Databases"},{"key":"10.3233\/HIS-230013_ref22","first-page":"1587","article-title":"Addressing Function Approximation Error in Actor-critic Methods","author":"Fujimoto","year":"2018","journal-title":"Proc. in the International Conference on Machine Learning (ICML 2018)"},{"key":"10.3233\/HIS-230013_ref23","unstructured":"T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel and S. Levine, Soft Actor-Critic Algorithms and Applications, ArXiv abs\/1812.05905, 2018."},{"key":"10.3233\/HIS-230013_ref24","unstructured":"Stable Baselines, 2022 July 26. Available from: https:\/\/github.com\/Stable-Baselines-Team\/stable-baselines."}],"container-title":["International Journal of Hybrid Intelligent Systems"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/HIS-230013","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T08:52:57Z","timestamp":1777452777000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/HIS-230013"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,3]]},"references-count":24,"journal-issue":{"issue":"3,4"},"URL":"https:\/\/doi.org\/10.3233\/his-230013","relation":{},"ISSN":["1448-5869","1875-8819"],"issn-type":[{"value":"1448-5869","type":"print"},{"value":"1875-8819","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,3]]}}}