{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:32:37Z","timestamp":1760149957501,"version":"build-2065373602"},"reference-count":40,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T00:00:00Z","timestamp":1695772800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Health &amp; Welfare, Republic of Korea","award":["HI21C1074"],"award-info":[{"award-number":["HI21C1074"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Catastrophic forgetting, which means a rapid forgetting of learned representations while learning new data\/samples, is one of the main problems of deep neural networks. In this paper, we propose a novel incremental learning framework that can address the forgetting problem by learning new incoming data in an online manner. We develop a new incremental learning framework that can learn extra data or new classes with less catastrophic forgetting. We adopt the hippocampal memory process to the deep neural networks by defining the effective maximum of neural activation and its boundary to represent a feature distribution. In addition, we incorporate incremental QR factorization into the deep neural networks to learn new data with both existing labels and new labels with less forgetting. The QR factorization can provide the accurate subspace prior, and incremental QR factorization can reasonably express the collaboration between new data with both existing classes and new class with less forgetting. In our framework, a set of appropriate features (i.e., nodes) provides improved representation for each class. We apply our method to the convolutional neural network (CNN) for learning Cifar-100 and Cifar-10 datasets. The experimental results show that the proposed method efficiently alleviates the stability and plasticity dilemma in the deep neural networks by providing the performance stability of a trained network while effectively learning unseen data and additional new classes.<\/jats:p>","DOI":"10.3390\/s23198117","type":"journal-article","created":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T07:50:26Z","timestamp":1695887426000},"page":"8117","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Incremental Learning for Online Data Using QR Factorization on Convolutional Neural Networks"],"prefix":"10.3390","volume":"23","author":[{"given":"Jonghong","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Neurology, Keimyung University Dongsan Hospital, Keimyung University School of Medicine, Daegu 42601, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5828-3048","authenticated-orcid":false,"given":"WonHee","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Neurology, Keimyung University Dongsan Hospital, Keimyung University School of Medicine, Daegu 42601, Republic of Korea"},{"name":"Department of Medical Informatics, Keimyung University School of Medicine, Daegu 42601, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sungdae","family":"Baek","sequence":"additional","affiliation":[{"name":"Graduate School of Artificial Intelligence, Kyungpook National University, Daegu 41566, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8235-9855","authenticated-orcid":false,"given":"Jeong-Ho","family":"Hong","sequence":"additional","affiliation":[{"name":"Department of Neurology, Keimyung University Dongsan Hospital, Keimyung University School of Medicine, Daegu 42601, Republic of Korea"},{"name":"Department of Medical Informatics, Keimyung University School of Medicine, Daegu 42601, Republic of Korea"},{"name":"Biolink Inc., Daegu 42601, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Minho","family":"Lee","sequence":"additional","affiliation":[{"name":"Graduate School of Artificial Intelligence, Kyungpook National University, Daegu 41566, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,27]]},"reference":[{"key":"ref_1","unstructured":"Hung, C.Y., Tu, C.H., Wu, C.E., Chen, C.H., Chan, Y.M., and Chen, C.S. (2019, January 8\u201314). Compacting, Picking and Growing for Unforgetting Continual Learning. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2935","DOI":"10.1109\/TPAMI.2017.2773081","article-title":"Learning without forgetting","volume":"40","author":"Li","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_3","unstructured":"Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., Zhang, Z., and Fu, Y. (2018). Incremental classifier learning with generative adversarial networks. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., and Fu, Y. (2019, January 16\u201320). Large scale incremental learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00046"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Rebuffi, S.A., Kolesnikov, A., Sperl, G., and Lampert, C.H. (2017, January 21\u201326). icarl: Incremental classifier and representation learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.587"},{"key":"ref_6","unstructured":"Liu, B. (2020, January 7\u201312). Learning on the job: Online lifelong and continual learning. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/S0079-7421(08)60536-8","article-title":"Catastrophic interference in connectionist networks: The sequential learning problem","volume":"Volume 24","author":"McCloskey","year":"1989","journal-title":"Psychology of Learning and Motivation"},{"key":"ref_8","unstructured":"Gepperth, A., and Hammer, B. (2016, January 27\u201329). Incremental learning algorithms and applications. Proceedings of the European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"6916","DOI":"10.1109\/TNNLS.2021.3083931","article-title":"Multilayer sparsity-based tensor decomposition for low-rank tensor completion","volume":"33","author":"Xue","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zeng, H., Xue, J., Luong, H.Q., and Philips, W. (2022). Multimodal core tensor factorization and its applications to low-rank tensor completion. IEEE Trans. Multimed.","DOI":"10.1109\/TMM.2022.3216746"},{"key":"ref_11","first-page":"2300","article-title":"Reconstruction of compressive sensing and semi-QR factorization","volume":"28","author":"Fu","year":"2008","journal-title":"J. Comput. Appl."},{"key":"ref_12","unstructured":"Glorot, X., Bordes, A., and Bengio, Y. (2011, January 11\u201313). Deep sparse rectifier neural networks. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1080\/09540099208946624","article-title":"Semi-distributed representations and catastrophic forgetting in connectionist networks","volume":"4","author":"French","year":"1992","journal-title":"Connect. Sci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1037\/0033-295X.97.2.285","article-title":"Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions","volume":"97","author":"Ratcliff","year":"1990","journal-title":"Psychol. Rev."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1111\/j.1551-6708.1987.tb00862.x","article-title":"Competitive learning: From interactive activation to adaptive resonance","volume":"11","author":"Grossberg","year":"1987","journal-title":"Cogn. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"504","DOI":"10.3389\/fpsyg.2013.00504","article-title":"The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects","volume":"4","author":"Mermillod","year":"2013","journal-title":"Front. Psychol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"759","DOI":"10.1016\/0893-6080(91)90056-B","article-title":"Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system","volume":"4","author":"Carpenter","year":"1991","journal-title":"Neural Netw."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"646","DOI":"10.1016\/j.neucom.2010.09.003","article-title":"Growing fuzzy topology adaptive resonance theory models with a push\u2013pull learning algorithm","volume":"74","author":"Kim","year":"2011","journal-title":"Neurocomputing"},{"key":"ref_19","unstructured":"Jung, H., Ju, J., Jung, M., and Kim, J. (2016). Less-forgetting learning in deep neural networks. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Shmelkov, K., Schmid, C., and Alahari, K. (2017, January 22\u201329). Incremental learning of object detectors without catastrophic forgetting. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.368"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3521","DOI":"10.1073\/pnas.1611835114","article-title":"Overcoming catastrophic forgetting in neural networks","volume":"114","author":"Kirkpatrick","year":"2017","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_22","unstructured":"Lee, S.W., Kim, J.H., Jun, J., Ha, J.W., and Zhang, B.T. (2017, January 4\u20139). Overcoming catastrophic forgetting by incremental moment matching. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Aljundi, R., Belilovsky, E., Tuytelaars, T., Charlin, L., Caccia, M., Lin, M., and Page-Caccia, L. (2019, January 8\u201314). Online continual learning with maximal interfered retrieval. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR.2019.01151"},{"key":"ref_24","first-page":"29193","article-title":"Gradient-based editing of memory examples for online task-free continual learning","volume":"34","author":"Jin","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Gao, Q., Zhao, C., Ghanem, B., and Zhang, J. (2022, January 23\u201327). R-dfcil: Relation-guided representation learning for data-free class incremental learning. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-20050-2_25"},{"key":"ref_26","first-page":"27075","article-title":"Exploring example influence in continual learning","volume":"35","author":"Sun","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Roy, A., Verma, V.K., Voonna, S., Ghosh, K., Ghosh, S., and Das, A. (2023). Exemplar-Free Continual Transformer with Convolutions. arXiv.","DOI":"10.1109\/ICCV51070.2023.00542"},{"key":"ref_28","unstructured":"Zhang, J., Chen, C., Zhuang, W., and Lv, L. (2023). Addressing Catastrophic Forgetting in Federated Class-Continual Learning. arXiv."},{"key":"ref_29","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Shi, Q., Eriksson, A., Van Den Hengel, A., and Shen, C. (2011, January 20\u201325). Is face recognition really a compressive sensing problem?. Proceedings of the CVPR 2011, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995556"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1109\/TPAMI.2008.79","article-title":"Robust face recognition via sparse representation","volume":"31","author":"Wright","year":"2008","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","unstructured":"Yang, A.Y., Wright, J., Ma, Y., and Sastry, S.S. (2007). Feature selection in face recognition: A sparse representation perspective. IEEE Trans. Pattern Anal. Mach. Intell., 2, Available online: https:\/\/people.eecs.berkeley.edu\/~yima\/recognition\/Files\/PAMI_Feature.pdf."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wang, L., Zhang, H., Wang, Z., and Li, J. (2014, January 13\u201314). A New Compressive Sensing Method for Face Recognition. Proceedings of the 2014 Seventh International Symposium on Computational Intelligence and Design, Hangzhou, China.","DOI":"10.1109\/ISCID.2014.73"},{"key":"ref_35","unstructured":"Axelsson, O. (1996). Iterative Solution Methods, Cambridge University Press."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1007\/BF01933262","article-title":"Iterative refinement enhances the stability ofQR factorization methods for solving linear equations","volume":"31","author":"Higham","year":"1991","journal-title":"BIT Numer. Math."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/j.procs.2016.07.418","article-title":"Convolutional neural network with biologically inspired retinal structure","volume":"88","author":"Kim","year":"2016","journal-title":"Procedia Comput. Sci."},{"key":"ref_38","unstructured":"Krizhevsky, A., and Hinton, G. (2023, August 08). Learning Multiple Layers of Features from Tiny Images. Available online: https:\/\/www.cs.toronto.edu\/~kriz\/learning-features-2009-TR.pdf."},{"key":"ref_39","unstructured":"Chaudhry, A., Rohrbach, M., Elhoseiny, M., Ajanthan, T., Dokania, P.K., Torr, P.H., and Ranzato, M. (2019). Continual learning with tiny episodic memories. arXiv."},{"key":"ref_40","unstructured":"Lopez-Paz, D., and Ranzato, M. (2017, January 4\u20139). Gradient episodic memory for continual learning. Proceedings of the Advances in Neural Information Processing Systems 30: 31st Annual Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/19\/8117\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:59:42Z","timestamp":1760129982000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/19\/8117"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,27]]},"references-count":40,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["s23198117"],"URL":"https:\/\/doi.org\/10.3390\/s23198117","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,9,27]]}}}