{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:10:15Z","timestamp":1760029815297,"version":"build-2065373602"},"reference-count":47,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,3,21]],"date-time":"2025-03-21T00:00:00Z","timestamp":1742515200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ningbo Science and Technology Major Project","award":["2024Z259","2022Z149"],"award-info":[{"award-number":["2024Z259","2022Z149"]}]},{"name":"Key Technology RD Program of Ningbo","award":["2024Z259","2022Z149"],"award-info":[{"award-number":["2024Z259","2022Z149"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>In recent years, the rapid development of large language models (LLMs) has led to a growing consensus in the industry regarding the integration of long-term and short-term memory. However, the widespread application of long-term and short-term memory systems faces two significant challenges: increased execution time and decreased answer accuracy from LLMs. To tackle these challenges, we propose the ILSTMA. This architecture uniquely combines fundamental theories of human forgetting with classical operating system principles, providing an unprecedented acceleration method that does not rely on traditional memory retrieval algorithms, which is all based on the systematic planning of available memory space. Furthermore, our proposed most relevant dialogue retrieval process substantially enhances the answer accuracy of LLMs while examining the potential of the two most commonly used memory retrieval algorithms. Experimental results demonstrate that our acceleration method improves the execution efficiency of the original architecture by 21.45%, and our most relevant dialogue retrieval process raises the answer accuracy to 88.4%, surpassing several benchmarks. These findings validate the high performance of the ILSTMA.<\/jats:p>","DOI":"10.3390\/info16040251","type":"journal-article","created":{"date-parts":[[2025,3,21]],"date-time":"2025-03-21T04:58:38Z","timestamp":1742533118000},"page":"251","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["ILSTMA: Enhancing Accuracy and Speed of Long-Term and Short-Term Memory Architecture"],"prefix":"10.3390","volume":"16","author":[{"given":"Zongyu","family":"Ming","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology (School of Artificial Intelligence), Zhejiang Sci-Tech University, Hangzhou 310018, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zimu","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Cyber Science and Engineering, Ningbo University of Technology, Ningbo 315211, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Genlang","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer and Data Engineering, Ningbo Tech University, Ningbo 315199, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,21]]},"reference":[{"key":"ref_1","unstructured":"OpenAI (2024, August 11). ChatGPT. Available online: https:\/\/chat.openai.com\/chat."},{"key":"ref_2","unstructured":"Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozi\u00e8re, B., Goyal, N., Hambro, E., and Azhar, F. (2023). Llama: Open and efficient foundation language models. arXiv."},{"key":"ref_3","unstructured":"Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., and Xia, X. (2022). Glm-130b: An open bilingual pre-trained model. arXiv."},{"key":"ref_4","unstructured":"Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., and Anadkat, S. (2023). Gpt-4 technical report. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ratner, N., Levine, Y., Belinkov, Y., Ram, O., Magar, I., Abend, O., Karpas, E., Shashua, A., Leyton-Brown, K., and Shoham, Y. (2022). Parallel context windows for large language models. arXiv.","DOI":"10.18653\/v1\/2023.acl-long.352"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, X., Salmani, M., Omidi, P., Ren, X., Rezagholizadeh, M., and Eshaghi, A. (2024). Beyond the limits: A survey of techniques to extend the context length in large language models. arXiv.","DOI":"10.24963\/ijcai.2024\/917"},{"key":"ref_7","unstructured":"Ng, Y., Miyashita, D., Hoshi, Y., Morioka, Y., Torii, O., Kodama, T., and Deguchi, J. (2023). Simplyretrieve: A private and lightweight retrieval-centric generative ai tool. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.J., and Huang, G. (2024, January 26\u201327). Expel: Llm agents are experiential learners. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i17.29936"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhong, W., Guo, L., Gao, Q., Ye, H., and Wang, Y. (2024, January 26\u201327). Memorybank: Enhancing large language models with long-term memory. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i17.29946"},{"key":"ref_10","unstructured":"Park, J.S., O\u2019Brien, J., Cai, C.J., Morris, M.R., Liang, P., and Bernstein, M.S. (November, January 29). Generative agents: Interactive simulacra of human behavior. Proceedings of the 36th Annual Acm Symposium on User Interface Software and Technology, San Francisco, CA, USA."},{"key":"ref_11","unstructured":"Liu, L., Yang, X., Shen, Y., Hu, B., Zhang, Z., Gu, J., and Zhang, G. (2023). Think-in-memory: Recalling and post-thinking enable llms with long-term memory. arXiv."},{"key":"ref_12","unstructured":"Ebbinghaus, H. (2024, July 17). Memory: A Contribution to Experimental, Available online: https:\/\/pmc.ncbi.nlm.nih.gov\/articles\/PMC4117135\/."},{"key":"ref_13","unstructured":"Brown, G.D., and Lewandowsky, S. (2010). Forgetting in memory models: Arguments against trace decay and consolidation failure. Forgetting, Psychology Press."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1037\/0033-2909.132.3.354","article-title":"Distributed practice in verbal recall tasks: A review and quantitative synthesis","volume":"132","author":"Cepeda","year":"2006","journal-title":"Psychol. Bull."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Lindsey, J., and Litwin-Kumar, A. (2023). Theory of systems memory consolidation via recall-gated plasticity. eLife, 12.","DOI":"10.7554\/eLife.90793"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1038\/npp.2010.169","article-title":"Update on memory systems and processes","volume":"36","author":"Nadel","year":"2011","journal-title":"Neuropsychopharmacology"},{"key":"ref_17","unstructured":"Smith, A.M. (2018). Examining the Role of Retrieval Practice in Improving Memory Accessibility Under Stress. [Ph.D. Thesis, Tufts University]."},{"key":"ref_18","first-page":"1","article-title":"Scaling instruction-finetuned language models","volume":"25","author":"Chung","year":"2024","journal-title":"J. Mach. Learn. Res."},{"key":"ref_19","unstructured":"Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., and Lin, X.V. (2022). Opt: Open pre-trained transformer language models. arXiv."},{"key":"ref_20","unstructured":"Le Scao, T., Fan, A., Akiki, C., Pavlick, E., Ili\u0107, S., Hesslow, D., Castagn\u00e9, R., Luccioni, A.S., Yvon, F., and Gall\u00e9, M. (2023, November 20). Bloom: A 176b-Parameter Open-Access Multilingual Language Model. Available online: https:\/\/inria.hal.science\/hal-03850124\/."},{"key":"ref_21","unstructured":"Rae, J.W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., and Young, S. (2021). Scaling language models: Methods, analysis & insights from training gopher. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Tay, Y., Wei, J., Chung, H.W., Tran, V.Q., So, D.R., Shakeri, S., Garcia, X., Zheng, H.S., Rao, J., and Chowdhery, A. (arXiv, 2022). Transcending scaling laws with 0.1% extra compute, arXiv.","DOI":"10.18653\/v1\/2023.emnlp-main.91"},{"key":"ref_23","unstructured":"Zeng, W., Ren, X., Su, T., Wang, H., Liao, Y., Wang, Z., Jiang, X., Yang, Z., Wang, K., and Zhang, X. (2021). Pangu-alpha: Large-scale autoregressive pretrained Chinese language models with auto-parallel computation. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Huawei Technologies Co., L. (2022). Huawei Technologies Co., L. Huawei mindspore ai development framework. Artificial Intelligence Technology, Springer.","DOI":"10.1007\/978-981-19-2879-6"},{"key":"ref_25","unstructured":"Nijkamp, E., Pang, B., Hayashi, H., Tu, L., Wang, H., Zhou, Y., Savarese, S., and Xiong, C. (2022). Codegen: An open large language model for code with multi-turn program synthesis. arXiv."},{"key":"ref_26","unstructured":"Iyer, S., Lin, X.V., Pasunuru, R., Mihaylov, T., Simig, D., Yu, P., Shuster, K., Wang, T., Liu, Q., and Koura, P.S. (2022). Opt-iml: Scaling language model instruction meta learning through the lens of generalization. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Scao, T.L., Bari, M.S., Shen, S., Yong, Z.X., and Schoelkopf, H. (2022). Crosslingual generalization through multitask finetuning. arXiv.","DOI":"10.18653\/v1\/2023.acl-long.891"},{"key":"ref_28","unstructured":"Li, R., Allal, L.B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., and Chim, J. (2023). Starcoder: May the source be with you!. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., and Lin, Y. (2024). A survey on large language model based autonomous agents. Front. Comput. Sci., 18.","DOI":"10.1007\/s11704-024-40231-1"},{"key":"ref_30","unstructured":"Rana, K., Haviland, J., Garg, S., Abou-Chakra, J., Reid, I., and Suenderhauf, N. (2023). Sayplan: Grounding large language models using 3d scene graphs for scalable task planning. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhu, A., Martin, L., Head, A., and Callison-Burch, C. (2023, January 8\u201312). CALYPSO: LLMs as Dungeon Master\u2019s Assistants. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Salt Lake City, UT, USA.","DOI":"10.1609\/aiide.v19i1.27534"},{"key":"ref_32","unstructured":"Fischer, K.A. (2023). Reflective linguistic programming (rlp): A stepping stone in socially-aware agi (socialagi). arXiv."},{"key":"ref_33","unstructured":"Wang, Z., Cai, S., Chen, G., Liu, A., Ma, X., and Liang, Y. (2023). Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv."},{"key":"ref_34","first-page":"8634","article-title":"Reflexion: Language agents with verbal reinforcement learning","volume":"36","author":"Shinn","year":"2024","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_35","unstructured":"Zhu, X., Chen, Y., Tian, H., Tao, C., Su, W., Yang, C., Huang, G., Li, B., Lu, L., and Wang, X. (2023). Ghost in the minecraft: Generally capable agents for open-world environments via large language models with text-based knowledge and memory. arXiv."},{"key":"ref_36","unstructured":"Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., and Anandkumar, A. (2023). Voyager: An open-ended embodied agent with large language models. arXiv."},{"key":"ref_37","unstructured":"Qian, C., Cong, X., Yang, C., Chen, W., Su, Y., Xu, J., Liu, Z., and Sun, M. (2023). Communicative agents for software development. arXiv."},{"key":"ref_38","unstructured":"Lin, J., Zhao, H., Zhang, A., Wu, Y., Ping, H., and Chen, Q. (2023). Agentsims: An open-source sandbox for large language model evaluation. arXiv."},{"key":"ref_39","unstructured":"Wang, B., Liang, X., Yang, J., Huang, H., Wu, S., Wu, P., Lu, L., Ma, Z., and Li, Z. (2023). Enhancing large language model with self-controlled memory framework. arXiv."},{"key":"ref_40","unstructured":"Huang, Z., Gutierrez, S., Kamana, H., and MacNeil, S. Memory sandbox: Transparent and interactive memory management for conversational agents. Proceedings of the Adjunct Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology."},{"key":"ref_41","unstructured":"Hu, C., Fu, J., Du, C., Luo, S., Zhao, J., and Zhao, H. (arXiv, 2023). Chatdb: Augmenting llms with databases as their symbolic memory, arXiv."},{"key":"ref_42","unstructured":"Zhou, X., Li, G., and Liu, Z. (2023). Llm as dba. arXiv."},{"key":"ref_43","unstructured":"Guti\u00e9rrez, B.J., Shu, Y., Gu, Y., Yasunaga, M., and Su, Y. (2024). HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. arXiv."},{"key":"ref_44","unstructured":"Zhou, W., Jiang, Y.E., Cui, P., Wang, T., Xiao, Z., Hou, Y., Cotterell, R., and Sachan, M. (2023). Recurrentgpt: Interactive generation of (arbitrarily) long text. arXiv."},{"key":"ref_45","unstructured":"Wang, Q., Ding, L., Cao, Y., Tian, Z., Wang, S., Tao, D., and Guo, L. (2023). Recursively summarizing enables long-term dialogue memory in large language models. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1109\/TBDATA.2019.2921572","article-title":"Billion-scale similarity search with GPUs","volume":"7","author":"Johnson","year":"2019","journal-title":"IEEE Trans. Big Data"},{"key":"ref_47","unstructured":"LangChain Inc (2024, August 13). LangChain. Available online: https:\/\/docs.langchain.com\/docs\/."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/4\/251\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:57:40Z","timestamp":1760029060000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/4\/251"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,21]]},"references-count":47,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["info16040251"],"URL":"https:\/\/doi.org\/10.3390\/info16040251","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2025,3,21]]}}}