{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T16:27:07Z","timestamp":1781368027846,"version":"3.54.1"},"reference-count":140,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2020,10,22]],"date-time":"2020-10-22T00:00:00Z","timestamp":1603324800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>As a special case of machine learning, incremental learning can acquire useful knowledge from incoming data continuously while it does not need to access the original data. It is expected to have the ability of memorization and it is regarded as one of the ultimate goals of artificial intelligence technology. However, incremental learning remains a long term challenge. Modern deep neural network models achieve outstanding performance on stationary data distributions with batch training. This restriction leads to catastrophic forgetting for incremental learning scenarios since the distribution of incoming data is unknown and has a highly different probability from the old data. Therefore, a model must be both plastic to acquire new knowledge and stable to consolidate existing knowledge. This review aims to draw a systematic review of the state of the art of incremental learning methods. Published reports are selected from Web of Science, IEEEXplore, and DBLP databases up to May 2020. Each paper is reviewed according to the types: architectural strategy, regularization strategy and rehearsal and pseudo-rehearsal strategy. We compare and discuss different methods. Moreover, the development trend and research focus are given. It is concluded that incremental learning is still a hot research area and will be for a long period. More attention should be paid to the exploration of both biological systems and computational models.<\/jats:p>","DOI":"10.3390\/e22111190","type":"journal-article","created":{"date-parts":[[2020,10,22]],"date-time":"2020-10-22T10:27:58Z","timestamp":1603362478000},"page":"1190","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":110,"title":["An Appraisal of Incremental Learning Methods"],"prefix":"10.3390","volume":"22","author":[{"given":"Yong","family":"Luo","sequence":"first","affiliation":[{"name":"College of Software, Northeastern University, Shenyang 110004, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liancheng","family":"Yin","sequence":"additional","affiliation":[{"name":"College of Software, Northeastern University, Shenyang 110004, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenchao","family":"Bai","sequence":"additional","affiliation":[{"name":"College of Software, Northeastern University, Shenyang 110004, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1243-0123","authenticated-orcid":false,"given":"Keming","family":"Mao","sequence":"additional","affiliation":[{"name":"College of Software, Northeastern University, Shenyang 110004, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,10,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"He, K.M., Zhang, X.Y., Ren, S.Q., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3434","DOI":"10.1109\/ACCESS.2019.2955480","article-title":"Challenges in Task Incremental Learning for Assistive Robotics","volume":"8","author":"Fan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1016\/j.neucom.2014.10.011","article-title":"A robust safety-oriented autonomous cruise control scheme for electric vehicles based on model predictive control and online sequential extreme learning machine with a hyper-level fault tolerance-based supervisor","volume":"151","author":"Mozaffari","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Khannoussi, A., Olteanu, A.L., Labreuche, C., Narayan, P., Dezan, C., Diguet, J., Petit-Fr\u00e8re, J., and Meyer, P. (2019, January 27\u201330). Integrating Operators\u2019 Preferences into Decisions of Unmanned Aerial Vehicles: Multi-layer Decision Engine and Incremental Preference Elicitation. Proceedings of the International Conference on Algorithmic DecisionTheory, Durham, NC, USA.","DOI":"10.1007\/978-3-030-31489-7_4"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.tins.2004.12.003","article-title":"Memory retentions-the synaptic stability versus plasticity dilemma","volume":"28","author":"Abraham","year":"2005","journal-title":"Trends Neurosci."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kemker, R., McClure, M., Abitino, A., Hayes, T., and Kanan, C. (2018, January 2\u20137). Measuring catastrophic forgetting in neural networks. Proceedings of the AAAI, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11651"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"8925","DOI":"10.1016\/j.jfranklin.2020.04.033","article-title":"Non-iterative and Fast Deep Learning: Multilayer Extreme Learning Machines","volume":"357","author":"Zhang","year":"2020","journal-title":"J. Frankl. Inst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"e216","DOI":"10.1002\/wdev.216","article-title":"Neural plasticity across the lifespan","volume":"6","author":"Power","year":"2016","journal-title":"Wiley Interdiscip. Rev. Dev. Biol."},{"key":"ref_9","first-page":"166","article-title":"The temporal paradox of hebbian learning and homeostatic plasticity","volume":"43","author":"Zenke","year":"2017","journal-title":"Neurobiology"},{"key":"ref_10","unstructured":"Morris, R.G.M., and Hebb, D.O. (1949). The Organization of Behavior, Wiley."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1178","DOI":"10.1038\/81453","article-title":"Synaptic plasticity: Taming the beast","volume":"3","author":"Abbott","year":"2000","journal-title":"Nat. Neurosci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"919","DOI":"10.1038\/78829","article-title":"Competitive hebbian learning through spike-timing-dependent synaptic plasticity","volume":"3","author":"Song","year":"2000","journal-title":"Nat. Neurosci."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1037\/0033-295X.102.3.419","article-title":"Why there are complementary learning systems in the hippocampus and neocortex: Insights from he successes and failures of connectionist models of learning and memory","volume":"102","author":"McClelland","year":"1995","journal-title":"Psychol. Rev."},{"key":"ref_14","first-page":"1387","article-title":"An empirical investigation of catastrophic forgetting in gradient-based neural networks","volume":"84","author":"Goodfellow","year":"2013","journal-title":"Comput. Ence"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3521","DOI":"10.1073\/pnas.1611835114","article-title":"Overcoming catastrophic forgetting in neural networks","volume":"114","author":"Kirkpatrick","year":"2016","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_16","first-page":"614","article-title":"Learning Without Forgetting","volume":"4","author":"Li","year":"2016","journal-title":"ECCV"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Rebuffi, S.A., Kolesnikov, A., Sperl, G., and Lampert, C.H. (2017, January 21\u201326). iCaRL: Incremental classifier and representation learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.587"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1261","DOI":"10.1016\/j.neucom.2017.06.084","article-title":"Incremental on-line learning: A review and comparison of state of the art algorithms","volume":"275","author":"Losing","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1007\/s12065-019-00203-y","article-title":"Incremental supervised learning: Algorithms and applications in pattern recognition","volume":"12","author":"Chefrour","year":"2019","journal-title":"Evol. Intell."},{"key":"ref_20","unstructured":"Van de Ven, G.M., and Tolias, A.S. (2019). Three scenarios for continual learning. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1623","DOI":"10.1109\/TNNLS.2013.2264952","article-title":"Ensemble Learning in Fixed Expansion Layer Networks for Mitigating Catastrophic Forgetting","volume":"24","author":"Coop","year":"2013","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_22","first-page":"1","article-title":"Incremental Concept Learning via Online Generative Memory Recall","volume":"99","author":"Li","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_23","unstructured":"Lomonaco, V., and Maltoni, D. (2017). CORe50: A New Dataset and Benchmark for Continuous Object Recognition. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1109\/5326.983933","article-title":"Learn++: An incremental learning algorithm for supervised neural networks","volume":"31","author":"Polikar","year":"2001","journal-title":"IEEE Trans. Syst. Man Cybern Part C"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.eswa.2018.08.055","article-title":"An incremental learning method for spoof fingerprint detection","volume":"116","author":"Kho","year":"2019","journal-title":"Expert Syst. Appl."},{"key":"ref_26","first-page":"181","article-title":"A Novel Incremental Learning Algorithm Based on Incremental Support Vector Machina and Incremental Neural Network Learn++","volume":"33","author":"Chefrour","year":"2019","journal-title":"Rev. D\u2019Intell. Artif."},{"key":"ref_27","unstructured":"Rusu, A.A., Rabinowitz, N.C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., and Hadsell, R. (2016). Progressive neural networks. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.neunet.2019.09.010","article-title":"Tree-CNN: A hierarchical Deep Convolutional Neural Network for incremental learning","volume":"121","author":"Roy","year":"2020","journal-title":"Neural Netw."},{"key":"ref_29","unstructured":"Yoon, J., Yang, E., Lee, J., and Hwang, S.J. (2017). Lifelong Learning with Dynamically Expandable Networks. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Aljundi, R., Chakravarty, P., and Tuytelaars, T. (2017, January 21\u201326). Expert Gate: Lifelong Learning with a Network of Experts. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.753"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"924","DOI":"10.1007\/s12559-016-9389-5","article-title":"A bio-inspired incremental learning architecture for applied perceptual problems","volume":"8","author":"Gepperth","year":"2016","journal-title":"Cogn. Comput."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"19","DOI":"10.3389\/frai.2020.00019","article-title":"Self-Net: Lifelong Learning via Continual Self-Modeling","volume":"3","author":"Mandivarapu","year":"2020","journal-title":"Front. Artif. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Yang, Y., Zhou, D.W., Zhan, D.C., Xiong, H., and Jiang, Y. (2019, January 4\u20138). Adaptive Deep Models for Incremental Learning: Considering Capacity Scalability and Sustainability. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330865"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1016\/j.patcog.2018.01.025","article-title":"A novel random forests based class incremental learning method for activity recognition","volume":"78","author":"Hu","year":"2018","journal-title":"Pattern Recogn."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"4615","DOI":"10.1109\/ACCESS.2019.2963056","article-title":"Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing","volume":"8","author":"Sarwar","year":"2019","journal-title":"IEEE Access"},{"key":"ref_36","unstructured":"Peng, J., Tang, B., Hao, J., Li, Z., Lei, Y.J., Lin, T., and Li, H.F. (2019). Overcoming Long-term Catastrophic Forgetting through Adversarial Neural Pruning and Synaptic Consolidation. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1109\/TCYB.2018.2866869","article-title":"Incremental Class Learning for Hierarchical Classification","volume":"50","author":"Park","year":"2018","journal-title":"IEEE Trans. Cybern."},{"key":"ref_38","unstructured":"Zhu, Q.Y., He, Z.K., and Ye, X. (2019). Incremental Classifier Learning Based on PEDCC-Loss and Cosine Distance. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1016\/j.asoc.2017.03.005","article-title":"Life-long learning based on dynamic combination model","volume":"56","author":"Ren","year":"2017","journal-title":"Appl. Soft Comput."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Part, J.L., and Lemon, O. (2017, January 18\u201321). Incremental online learning of objects for robots operating in real environments. Proceedings of the 2017 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), Lisbon, Portugal.","DOI":"10.1109\/DEVLRN.2017.8329822"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Mallya, A., and Lazebnik, S. (2018, January 18\u201323). PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00810"},{"key":"ref_42","unstructured":"Hu, W.P., Lin, Z., Liu, B., Tao, C.Y., Tao, Z.W., Ma, J.W., Zhao, D.Y., and Yan, R. (2019, January 6\u20139). Overcoming Catastrophic Forgetting for Continual Learning via Model Adaptation. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zacarias, A.S., and Alexandre, L.A. (2018, January 19\u201321). SeNA-CNN: Overcoming Catastrophic Forgetting in Convolutional Neural Networks by Selective Network Augmentation. Proceedings of the IAPR Workshop on Artificial Neural Networks in Pattern Recognition, Siena, Italy.","DOI":"10.1007\/978-3-319-99978-4_8"},{"key":"ref_44","unstructured":"Li, X.L., Zhou, Y.B., Wu, T.F., Socher, R., and Xiong, C.M. (2019). Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting. arXiv."},{"key":"ref_45","unstructured":"Amer, M., and Maul, T. (2019). Reducing Catastrophic Forgetting in Modular Neural Networks by Dynamic Information Balancing. arXiv."},{"key":"ref_46","first-page":"3987","article-title":"Continual Learning Through Synaptic Intelligence","volume":"70","author":"Zenke","year":"2017","journal-title":"Proc. Mach. Learn. Res."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Chaudhry, A., Dokania, P.K., Ajanthan, T., and Torr, P.H.S. (2018, January 8\u201314). Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01252-6_33"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., and Tuytelaars, T. (2018, January 8\u201314). Memory Aware Synapses: Learning What (not) to Forget. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01219-9_9"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Aljundi, R., Kelchtermans, K., and Tuytelaars, T. (2019, January 15\u201320). Task-Free Continual Learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01151"},{"key":"ref_51","unstructured":"Zeng, G.X., Chen, Y., Cui, B., and Yu, S. (2018). Continuous Learning of Context-dependent Processing in Neural Networks. arXiv."},{"key":"ref_52","unstructured":"Farajtabar, M., Azizan, N., Mott, A., and Li, A. (2020, January 3). Orthogonal Gradient Descent for Continual Learning. Proceedings of the AISTATS, Palermo, Italy."},{"key":"ref_53","unstructured":"Choi, E., Lee, K., and Choi, K. (2019). Autoencoder-Based Incremental Class Learning without Retraining on Old Data. arXiv."},{"key":"ref_54","unstructured":"Lee, S.W., Kim, J.H., Jun, J., Ha, J.W., and Zhang, B.T. (2017, January 4\u20139). Overcoming Catastrophic Forgetting by Incremental Moment Matching. Proceedings of the NIPS, Long Beach, CA, USA."},{"key":"ref_55","unstructured":"Nguyen, G., Chen, S., Do, T., Jun, T.j., Choi, H.J., and Kim, D. (2020). Dissecting Catastrophic Forgetting in Continual Learning by Deep Visualization. arXiv."},{"key":"ref_56","unstructured":"Singh, A. (2020). Adversarial Incremental Learning. arXiv."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.neunet.2019.03.010","article-title":"Continuous learning in single-incremental-task scenarios","volume":"116","author":"Maltoni","year":"2019","journal-title":"Neural Netw."},{"key":"ref_58","unstructured":"Serr\u00e0, J., Suris, D., Miron, M., and Karatzoglou, A. (2018, January 10\u201315). Overcoming Catastrophic Forgetting with Hard Attention to the Task. Proceedings of the ICML, Stockholm, Sweden."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Liu, X.L., Masana, M., Herranz, L., Weijer, J., L\u00f3pez, A.M., and Bagdanov, A.D. (2018, January 20\u201324). Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgetting. Proceedings of the ICPR, Beijing, China.","DOI":"10.1109\/ICPR.2018.8545895"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1109\/TII.2018.2871084","article-title":"An Incremental Deep Convolutional Computation Model for Feature Learning on Industrial Big Data","volume":"15","author":"Li","year":"2019","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_61","unstructured":"Allred, J.M., and Roy, K. (2019). Stimulating STDP to Exploit Locality for Lifelong Learning without Catastrophic Forgetting. arXiv."},{"key":"ref_62","unstructured":"Kutalev, A. (2020). Natural Way to Overcome the Catastrophic Forgetting in Neural Networks. arXiv."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Paik, I., Oh, S., Kwak, T.Y., and Kim, I. (2019). Overcoming Catastrophic Forgetting by Neuron-level Plasticity Control. arXiv.","DOI":"10.1609\/aaai.v34i04.5981"},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"E10467","DOI":"10.1073\/pnas.1803839115","article-title":"Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization","volume":"115","author":"Masse","year":"2018","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_65","unstructured":"Gaurav, A., Vernekar, S., Lee, J., Abdelzad, V., Czarnecki, K., and Sedwards, S. (2020, January 7\u201312). Simple Continual Learning Strategies for Safer Classifers. Proceedings of the AAAI, New York, NY, USA."},{"key":"ref_66","first-page":"38","article-title":"Distilling the Knowledge in a Neural Network","volume":"14","author":"Hinton","year":"2015","journal-title":"Comput. Sci."},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Hao, T., Fu, Y.W., Jiang, Y.G., and Tian, Q. (2019, January 8\u201312). An end-to-end Architecture for Class-incremental Object Detection with Knowledge Distillation. Proceedings of the ICME, Shanghai, China.","DOI":"10.1109\/ICME.2019.00009"},{"key":"ref_68","doi-asserted-by":"crossref","unstructured":"Rannen, A., Aljundi, R., Blaschko, M.B., and Tuytelaars, T. (2017, January 21\u201326). Encoder based lifelong learning. Proceedings of the CVPR, Honolulu, HI, USA.","DOI":"10.1109\/ICCV.2017.148"},{"key":"ref_69","unstructured":"Schwarz, J., Luketina, J., Czarnecki, W.M., Agnieszka, G.B., Yee, W.T., Pascanu, R., and Hadsell, R. (2018, January 10\u201315). Progress & Compress: A scalable framework for continual learning. Proceedings of the ICML, Stockholm, Sweden."},{"key":"ref_70","doi-asserted-by":"crossref","unstructured":"Hou, S.H., Pan, X.Y., Chen, C.L., Wang, Z.L., and Lin, D.H. (2018, January 8\u201314). Lifelong Learning via Progressive Distillation and Retrospection. Proceedings of the ECCV, Munich, Germany.","DOI":"10.1007\/978-3-030-01219-9_27"},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/neco_a_01232","article-title":"Adversarial Feature Alignment: Avoid Catastrophic Forgetting in Incremental Task Lifelong Learning","volume":"31","author":"Yao","year":"2019","journal-title":"Neural Comput."},{"key":"ref_72","first-page":"1205","article-title":"Optimal kernel choice for large-scale twosample tests","volume":"1","author":"Gretton","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_73","unstructured":"Kim, D., Bae, J., Jo, Y., and Choi, J.Y. (2019). Incremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and Intransigence. arXiv."},{"key":"ref_74","unstructured":"Pereyra, G., Tucker, G., Chorowski, J., Kaiser, L., and Hinton, G.E. (2017, January 24\u201326). Regularizing Neural Networks by Penalizing Confident Output Distributions. Proceedings of the ICLR, Toulon, France."},{"key":"ref_75","doi-asserted-by":"crossref","unstructured":"Jung, H.C., Ju, J.W., Jung, M.J., and Kim, J.M. (2018, January 2\u20137). Less-Forgetful Learning for Domain Expansion in Deep Neural Networks. Proceedings of the AAMI, New Orleans, Louisiana, LA, USA.","DOI":"10.1609\/aaai.v32i1.11769"},{"key":"ref_76","unstructured":"Lee, K.B., Lee, K.M., Shin, J.W., and Lee, H.L. (November, January 27). Overcoming Catastrophic Forgetting With Unlabeled Data in the Wild. Proceedings of the ICCV, Seoul, Korea."},{"key":"ref_77","doi-asserted-by":"crossref","unstructured":"Zhang, J.T., Zhang, J., Ghosh, S., Li, D.W., Tasci, S., Heck, L.P., Zhang, H.M., and Kou, C.C. (2020, January 1\u20135). Class-incremental Learning via Deep Model Consolidation. Proceedings of the WACV, Snowmass Village, CO, USA.","DOI":"10.1109\/WACV45572.2020.9093365"},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"23090","DOI":"10.1109\/ACCESS.2019.2963461","article-title":"Efficient Incremental Learning Using Dynamic Correction Vector","volume":"8","author":"Xiang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_79","doi-asserted-by":"crossref","unstructured":"Zhao, B.W., Xiao, X., Gan, G.J., Zhang, B., and Xia, S.T. (2019). Maintaining Discrimination and Fairness in Class Incremental Learning. arXiv.","DOI":"10.1109\/CVPR42600.2020.01322"},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Javed, K., and Shafait, F. (2018, January 2\u20136). Revisiting Distillation and Incremental Classifier Learning. Proceedings of the ACCV, Perth, Australia.","DOI":"10.1007\/978-3-030-20876-9_1"},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Hou, S.H., Pan, X.Y., Loy, C.C., Wang, Z.L., and Lin, D.H. (2019, January 16\u201320). Learning a Unified Classifier Incrementally via Rebalancing. Proceedings of the CVPR, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00092"},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Michieli, U., and Zanuttigh, P. (2019). Knowledge Distillation for Incremental Learning in Semantic Segmentation. arXiv.","DOI":"10.1109\/ICCVW.2019.00400"},{"key":"ref_83","doi-asserted-by":"crossref","unstructured":"Li, D.W., Tasci, S., Ghosh, S., Zhu, J.W., Zhang, J.T., and Heck, L.P. (2019, January 7\u20139). RILOD: Near real-time incremental learning for object detection at the edge. Proceedings of the SEC, Washington, DC, USA.","DOI":"10.1145\/3318216.3363317"},{"key":"ref_84","doi-asserted-by":"crossref","unstructured":"Shmelkov, K., Schmid, C., and Alahari, K. (2017, January 22\u201327). Incremental Learning of Object Detectors without Catastrophic Forgetting. Proceedings of the ICCV, Venice, Italy.","DOI":"10.1109\/ICCV.2017.368"},{"key":"ref_85","unstructured":"Zhai, M.Y., Chen, L., Tung, F., He, J.W., Nawhal, M., and Mori, G. (November, January 27). Lifelong GAN: Continual Learning for Conditional Image Generation. Proceedings of the ICCV, Seoul, Korea."},{"key":"ref_86","unstructured":"Anthony, R. (1993, January 24\u201326). Catastrophic forgetting in neural networks: The role of rehearsal mechanisms. Proceedings of the New Zealand International Two-stream Conference on Artificial Neural Networks & Expert Systems IEEE, Dunedin, New Zealand."},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1080\/09540099550039318","article-title":"Catastrophic forgetting, rehearsal and pseudorehearsal","volume":"7","author":"Robins","year":"1995","journal-title":"Connect. Sci."},{"key":"ref_88","unstructured":"Knoblauch, J., Husain, H., and Diethe, T. (2020). Optimal Continual Learning has Perfect Memory and is NP-hard. arXiv."},{"key":"ref_89","doi-asserted-by":"crossref","unstructured":"Castro, F.M., Marin-Jimenez, M.J., Guil, N., Schmid, C., and Alahari, K. (2018, January 8\u201310). End-to-end incremental learning. Proceedings of the ECCV (12), Munich, Germany.","DOI":"10.1007\/978-3-030-01258-8_15"},{"key":"ref_90","unstructured":"Wu, Y., Chen, Y.P., Wang, L.J., Ye, Y.C., Liu, Z.C., Guo, Y.D., Zhang, Z.Y., and Fu, Y. (2018). Incremental Classifier Learning with Generative Adversarial Networks. arXiv."},{"key":"ref_91","unstructured":"Odena, A., Olah, C., and Shlens, J. (2017, January 6\u201311). Conditional image synthesis with auxiliary classifier GANs. Proceedings of the ICML, Sydney, Australia."},{"key":"ref_92","unstructured":"Kemker, R., and Kanan, C. (May, January 30). FearNet: Brain-Inspired Model for Incremental Learning. Proceedings of the ICLR, Vancouver, BC, Canada."},{"key":"ref_93","unstructured":"Shin, H., Lee, J.K., Kim, J., and Kim, J. (2017, January 4\u20139). Continual learning with deep generative replay. Proceedings of the NIPS, Long Beach, CA, USA."},{"key":"ref_94","unstructured":"Kingma, D.P., and Welling, M. (2014, January 14\u201316). Auto-encoding variational bayes. Proceedings of the ICLR, Banff, AB, Canada."},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Wu, Y., Chen, Y.P., Wang, L.J., Ye, Y.C., Liu, Z.C., Guo, Y.D., and Fu, Y. (2019, January 16\u201320). Large Scale Incremental Learning. Proceedings of the CVPR, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00046"},{"key":"ref_96","doi-asserted-by":"crossref","unstructured":"Ostapenko, O., Puscas, M.M., Klein, T., J\u00e4hnichen, P., and Nabi, M. (2019, January 16\u201320). Learning to remember: A synaptic driven framework for continual learning. Proceedings of the CVPR, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01158"},{"key":"ref_97","doi-asserted-by":"crossref","unstructured":"Mellado, D., Saavedra, C., Chabert, S., Torres, R., and Salas, R. (2019). Self-Improving Generative Artificial Neural Network for Pseudorehearsal Incremental Class Learning. Algorithms, 12.","DOI":"10.20944\/preprints201907.0121.v1"},{"key":"ref_98","doi-asserted-by":"crossref","first-page":"51276","DOI":"10.1109\/ACCESS.2020.2980386","article-title":"Exemplar-Supported Representation for Effective Class-incremental Learning","volume":"8","author":"Guo","year":"2020","journal-title":"IEEE Access"},{"key":"ref_99","unstructured":"Riemer, M., Klinger, R., Bouneffouf, D., and Franceschini, M. (February, January 27). Scalable Recollections for Continual Lifelong Learning. Proceedings of the AAAi, Honolulu, HI, USA."},{"key":"ref_100","doi-asserted-by":"crossref","unstructured":"Kim, B., Lee, J., Kim, K., Kim, S., and Kim, J. (2019, January 22\u201325). Collaborative Method for Incremental Learning on Classification and Generation. Proceedings of the ICIP, Taipei, Taiwan.","DOI":"10.1109\/ICIP.2019.8802976"},{"key":"ref_101","unstructured":"Shah, H., Javed, K., and Shafait, F. (2018). Distillation Techniques for Pseudo-rehearsal Based Incremental Learning. arXiv."},{"key":"ref_102","unstructured":"Lopez-Paz, D., and Marc\u2019Aurelio, R. (2017, January 4\u20139). Gradient episodic memory for continual learning. Proceedings of the NIPS, Long Beach, CA, USA."},{"key":"ref_103","doi-asserted-by":"crossref","unstructured":"Stojanov, S., Mishra, S., Thai, N.H., Dhanda, N., Humayun, A., Yu, C., Smith, L.B., and Rehg, J.M. (2019, January 16\u201320). Incremental Object Learning From Contiguous Views. Proceedings of the CVPR, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00898"},{"key":"ref_104","unstructured":"Li, Y., Li, Z.X., Ding, L.Z., Yang, P., Hu, Y.H., Chen, W., and Gao, X. (2018). SupportNet: Solving catastrophic forgetting in class incremental learning with support data. arXiv."},{"key":"ref_105","doi-asserted-by":"crossref","unstructured":"Xiong, W., Wang, Y.J., and Cheng, L. (2018, January 11\u201313). Fisher Discriminant Analysis Random Forest for Online Class Incremental Learning. Proceedings of the ISPA\/IUCC\/BDCloud\/SocialCom\/SustainCom, Melbourne, Australia.","DOI":"10.1109\/BDCloud.2018.00093"},{"key":"ref_106","doi-asserted-by":"crossref","first-page":"3524","DOI":"10.1109\/JSTARS.2019.2925416","article-title":"Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data","volume":"12","author":"Tasar","year":"2019","journal-title":"J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_107","first-page":"289","article-title":"Characteristics of contrastive Hebbian learning with pseudorehearsal for multilayer neural networks on reduction of catastrophic forgetting","volume":"7","author":"Hattori","year":"2018","journal-title":"Int. J. Comput."},{"key":"ref_108","unstructured":"van de Ven, G.M., and Tolias, A.S. (2018). Generative replay with feedback connections as a general strategy for continual learning. arXiv."},{"key":"ref_109","unstructured":"Venkatesan, R., Venkateswara, H., Panchanathan, S., and Li, B.X. (2017). A Strategy for an Uncompromising Incremental Learner. arXiv."},{"key":"ref_110","doi-asserted-by":"crossref","unstructured":"Lee, S., and Baek, J. (2019, January 1\u20133). Generative Pseudorehearsal Strategy for Fault Classification Under an Incremental Learning. Proceedings of the CSE\/EUC, New York, NY, USA.","DOI":"10.1109\/CSE\/EUC.2019.00035"},{"key":"ref_111","unstructured":"Xiang, Y., Fu, Y., Ji, P., and Huang, H. (November, January 27). Incremental Learning Using Conditional Adversarial Networks. Proceedings of the ICCV, Seoul, Korea."},{"key":"ref_112","doi-asserted-by":"crossref","unstructured":"Masarczyk, W., and Tautkute, I. (2020, January 16\u201318). Reducing catastrophic forgetting with learning on synthetic data. Proceedings of the CVPR, Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00134"},{"key":"ref_113","unstructured":"Atkinson, C., McCane, B., Szymanski, L., and Robins, A.V. (2018). Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networks. arXiv."},{"key":"ref_114","doi-asserted-by":"crossref","unstructured":"Rios, A., and Itti, L. (2019, January 10\u201316). Closed-Loop Memory GAN for Continual Learning. Proceedings of the IJCAI, Macao, China.","DOI":"10.24963\/ijcai.2019\/462"},{"key":"ref_115","doi-asserted-by":"crossref","first-page":"1870","DOI":"10.1109\/TNNLS.2019.2927320","article-title":"Incremental Reinforcement Learning in Continuous Spaces via Policy Relaxation and Importance Weighting","volume":"31","author":"Wang","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_116","doi-asserted-by":"crossref","unstructured":"Perez-Rua, J.M., Zhu, X.T., Hospedales, T.M., and Xiang, T. (2020, January 16\u201318). Incremental Few-Shot Object Detection. Proceedings of the CVPR, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01386"},{"key":"ref_117","unstructured":"Zhou, X.Y., Wang, D.Q., and Krahenbuhl, P. (2019). Objects as Points. arXiv."},{"key":"ref_118","doi-asserted-by":"crossref","unstructured":"Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., and Shah, M. (2020, January 16\u201318). iTAML: An Incremental Task-Agnostic Meta-learning Approach. Proceedings of the CVPR, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01360"},{"key":"ref_119","doi-asserted-by":"crossref","unstructured":"Gao, M., Zong, Y.B., Li, R., An, C.Z., Duan, Q., Yin, Q.S., and Wang, X.C. (2019, January 21\u201324). Online Anomaly Detection via Incremental Tensor Decomposition. Proceedings of the WISA, Jeju Island, Korea.","DOI":"10.1007\/978-3-030-30952-7_1"},{"key":"ref_120","unstructured":"Rodriguez, N.D., Lomonaco, V., Filliat, D., and Maltoni, D. (2018). Don\u2019t forget, there is more than forgetting: New metrics for Continual Learning. arXiv."},{"key":"ref_121","doi-asserted-by":"crossref","unstructured":"She, Q., Feng, F., Hao, X., Yang, Q., Lan, C., Lomonaco, V., Shi, X., Wang, Z., Guo, Y., and Zhang, Y. (August, January 31). OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9196887"},{"key":"ref_122","unstructured":"Lomonaco, V., Pellegrini, L., Rodriguez, P., Caccia, M., She, Q., Chen, Y., Jodelet, Q., Wang, R., Mai, Z., and Vazquez, D. (2020). CVPR 2020 Continual Learning in Computer Vision Competition: Approaches, Results, Current Challenges and Future Directions. arXiv."},{"key":"ref_123","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_124","unstructured":"Krizhevsky, A., and Hinton, G. (2009). Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases, Elsevier BV."},{"key":"ref_125","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., and Li, F.F. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_126","unstructured":"Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A.Y. (2020, October 21). Reading Digits in Natural Images with Unsupervised Feature Learning. Available online: https:\/\/research.google\/pubs\/pub37648\/."},{"key":"ref_127","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1007\/s11263-014-0733-5","article-title":"The pascal visual object classes challenge: A retrospective","volume":"111","author":"Everingham","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_128","doi-asserted-by":"crossref","unstructured":"Quattoni, A., and Torralba, A. (2009, January 20\u201325). Recognizing indoor scenes. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206537"},{"key":"ref_129","unstructured":"Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. (2011). The Caltech-UCSD Birds-200-2011 Dataset, California Institute of Technology. Technical Report CNS-TR-2011-001."},{"key":"ref_130","doi-asserted-by":"crossref","unstructured":"Nilsback, M.E., and Zisserman, A. (2008, January 16\u201319). Automated flower classification over a large number of classes. Proceedings of the 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, Bhubaneswar, India.","DOI":"10.1109\/ICVGIP.2008.47"},{"key":"ref_131","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1109\/MRA.2020.2987186","article-title":"IROS 2019 Lifelong Robotic Vision: Object Recognition Challenge [Competitions]","volume":"27","author":"Bae","year":"2020","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_132","unstructured":"He, C., Wang, R., Shan, S., and Chen, X. (2018, January 3\u20136). Exemplar-supported generative reproduction for class incremental learning. Proceedings of the BMVC, Newcastle, UK."},{"key":"ref_133","unstructured":"Seff, A., Beatson, A., Suo, D., and Liu, H. (2017). Continual learning in generative adversarial nets. arXiv."},{"key":"ref_134","unstructured":"Chaudhry, A., Ranzato, M., Rohrbach, M., and Elhoseiny, M. (2019, January 6\u20139). Efficient lifelong learning with A-GEM. Proceedings of the ICLR, New Orleans, LA, USA."},{"key":"ref_135","unstructured":"Wu, C., Herranz, L., Liu, X., Wang, Y., van de Weijer, J., and Raducanu, B. (2018, January 3\u20138). Memory Replay GANs: Learning to generate images from new categories without forgetting. Proceedings of the Advances in Neural Information Processing Systems, Montr\u00e9al, QC, Canada."},{"key":"ref_136","doi-asserted-by":"crossref","unstructured":"Cohen, G., Afshar, S., Tapson, J., and van Schaik, A. (2017, January 14\u201319). EMNIST: Extending MNIST to handwritten letters. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.","DOI":"10.1109\/IJCNN.2017.7966217"},{"key":"ref_137","doi-asserted-by":"crossref","unstructured":"Shi, X.S., Li, D.J., Zhao, P.P., Tian, Q.B., Tian, Y.X., Long, Q.W., Zhu, C.H., Song, J.W., Qiao, F., and Song, L. (August, January 31). Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAM. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9196638"},{"key":"ref_138","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.inffus.2019.12.004","article-title":"Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges","volume":"58","author":"Lesort","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_139","doi-asserted-by":"crossref","unstructured":"Hayes, T.L., Cahill, N.D., and Kanan, C. (2019, January 20\u201324). Memory efficient experience replay for streaming learning. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793982"},{"key":"ref_140","unstructured":"Raffaello, C., Silvio, T., Lorenzo, R., Giorgio, M., and Francesco, N. (2016, January 16\u201321). Incremental semiparametric inverse dynamics learning. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/11\/1190\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:25:52Z","timestamp":1760178352000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/11\/1190"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,22]]},"references-count":140,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["e22111190"],"URL":"https:\/\/doi.org\/10.3390\/e22111190","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,22]]}}}