{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T21:22:42Z","timestamp":1773868962250,"version":"3.50.1"},"reference-count":48,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2019,10,1]],"date-time":"2019-10-01T00:00:00Z","timestamp":1569888000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002848","name":"Comisi\u00f3n Nacional de Investigaci\u00f3n Cient\u00edfica y Tecnol\u00f3gica","doi-asserted-by":"publisher","award":["79140057"],"award-info":[{"award-number":["79140057"]}],"id":[{"id":"10.13039\/501100002848","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002848","name":"Comisi\u00f3n Nacional de Investigaci\u00f3n Cient\u00edfica y Tecnol\u00f3gica","doi-asserted-by":"publisher","award":["REDI170367"],"award-info":[{"award-number":["REDI170367"]}],"id":[{"id":"10.13039\/501100002848","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002848","name":"Comisi\u00f3n Nacional de Investigaci\u00f3n Cient\u00edfica y Tecnol\u00f3gica","doi-asserted-by":"publisher","award":["ACT1416"],"award-info":[{"award-number":["ACT1416"]}],"id":[{"id":"10.13039\/501100002848","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Deep learning models are part of the family of artificial neural networks and, as such, they suffer catastrophic interference when learning sequentially. In addition, the greater number of these models have a rigid architecture which prevents the incremental learning of new classes. To overcome these drawbacks, we propose the Self-Improving Generative Artificial Neural Network (SIGANN), an end-to-end deep neural network system which can ease the catastrophic forgetting problem when learning new classes. In this method, we introduce a novel detection model that automatically detects samples of new classes, and an adversarial autoencoder is used to produce samples of previous classes. This system consists of three main modules: a classifier module implemented using a Deep Convolutional Neural Network, a generator module based on an adversarial autoencoder, and a novelty-detection module implemented using an OpenMax activation function. Using the EMNIST data set, the model was trained incrementally, starting with a small set of classes. The results of the simulation show that SIGANN can retain previous knowledge while incorporating gradual forgetfulness of each learning sequence at a rate of about 7% per training step. Moreover, SIGANN can detect new classes that are hidden in the data with a median accuracy of     43 %     and, therefore, proceed with incremental class learning.<\/jats:p>","DOI":"10.3390\/a12100206","type":"journal-article","created":{"date-parts":[[2019,10,1]],"date-time":"2019-10-01T11:11:16Z","timestamp":1569928276000},"page":"206","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Self-Improving Generative Artificial Neural Network for Pseudorehearsal Incremental Class Learning"],"prefix":"10.3390","volume":"12","author":[{"given":"Diego","family":"Mellado","sequence":"first","affiliation":[{"name":"Escuela de Ingenier\u00eda C. Biom\u00e9dica, Universidad de Valara\u00edso, Valpara\u00edso 2362905, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4130-0010","authenticated-orcid":false,"given":"Carolina","family":"Saavedra","sequence":"additional","affiliation":[{"name":"Escuela de Ingenier\u00eda C. Biom\u00e9dica, Universidad de Valara\u00edso, Valpara\u00edso 2362905, Chile"},{"name":"Centro de Investigaci\u00f3n y Desarrollo en Ingenier\u00eda en Salud, CINGS-UV, Universidad de Valpara\u00edso, Valpara\u00edso 2362905, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2890-5077","authenticated-orcid":false,"given":"Steren","family":"Chabert","sequence":"additional","affiliation":[{"name":"Escuela de Ingenier\u00eda C. Biom\u00e9dica, Universidad de Valara\u00edso, Valpara\u00edso 2362905, Chile"},{"name":"Centro de Investigaci\u00f3n y Desarrollo en Ingenier\u00eda en Salud, CINGS-UV, Universidad de Valpara\u00edso, Valpara\u00edso 2362905, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2705-4298","authenticated-orcid":false,"given":"Romina","family":"Torres","sequence":"additional","affiliation":[{"name":"Engineering Faculty, Universidad Andres Bello, Vi\u00f1a del Mar 2531015, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0350-6811","authenticated-orcid":false,"given":"Rodrigo","family":"Salas","sequence":"additional","affiliation":[{"name":"Escuela de Ingenier\u00eda C. Biom\u00e9dica, Universidad de Valara\u00edso, Valpara\u00edso 2362905, Chile"},{"name":"Centro de Investigaci\u00f3n y Desarrollo en Ingenier\u00eda en Salud, CINGS-UV, Universidad de Valpara\u00edso, Valpara\u00edso 2362905, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1109\/5326.983933","article-title":"Learn++: An incremental learning algorithm for supervised neural networks","volume":"31","author":"Polikar","year":"2001","journal-title":"IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.)"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2744","DOI":"10.1016\/j.neucom.2006.04.011","article-title":"A robust and flexible model of hierarchical self-organizing maps for non-stationary environments","volume":"70","author":"Salas","year":"2007","journal-title":"Neurocomputing"},{"key":"ref_3","unstructured":"Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., and Lillicrap, T. (2016, January 19\u201324). Meta-learning with memory-augmented neural networks. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_4","unstructured":"Thrun, S. (December, January 27). Is learning the n-th thing any easier than learning the first?. Proceedings of the 8th International Conference on Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Mira, J., and \u00c1lvarez, J.R. (2003). Robust Expectation Maximization Learning Algorithm for Mixture of Experts. Computational Methods in Neural Modeling, Springer.","DOI":"10.1007\/3-540-44868-3"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/S0079-7421(08)60536-8","article-title":"Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem","volume":"Volume 24","author":"Bower","year":"1989","journal-title":"Psychology of Learning and Motivation"},{"key":"ref_7","unstructured":"Grossberg, S. (1982). Studies of Mind and Brain: Neural Principles of Learning, Perception, Development, Cognition, and Motor Control, Reidel Press."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Mermillod, M., Bugaiska, A., and Bonin, P. (2013). The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects. Front. Psychol., 4.","DOI":"10.3389\/fpsyg.2013.00504"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1080\/09540099208946624","article-title":"Semi-distributed Representations and Catastrophic Forgetting in Connectionist Networks","volume":"4","author":"French","year":"1992","journal-title":"Connect. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"962","DOI":"10.1016\/j.patrec.2011.01.020","article-title":"Machine fusion to enhance the topology preservation of vector quantization artificial neural networks","volume":"32","author":"Salas","year":"2011","journal-title":"Pattern Recognit. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kemker, R., McClure, M., Abitino, A., Hayes, T.L., and Kanan, C. (2018, January 2\u20137). Measuring catastrophic forgetting in neural networks. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11651"},{"key":"ref_12","unstructured":"Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., and Bengio, Y. (2013). An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ge, Z., Demyanov, S., Chen, Z., and Garnavi, R. (2017). Generative OpenMax for Multi-Class Open Set Classification. arXiv.","DOI":"10.5244\/C.31.42"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Rebuffi, S.A., Kolesnikov, A., and Lampert, C.H. (2017, January 21\u201326). iCaRL: Incremental Classifier and Representation Learning. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.587"},{"key":"ref_15","unstructured":"Li, Y., Li, Z., Ding, L., Yang, P., Hu, Y., Chen, W., and Gao, X. (2018). Supportnet: Solving catastrophic forgetting in class incremental learning with support data. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2935","DOI":"10.1109\/TPAMI.2017.2773081","article-title":"Learning without Forgetting","volume":"40","author":"Li","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","unstructured":"Shin, H., Lee, J.K., Kim, J., and Kim, J. (2016, January 5\u201310). Continual learning with deep generative replay. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Cherti, M., Kegl, B., and Kazakci, A. (2017, January 6\u20138). Out-of-Class Novelty Generation: An Experimental Foundation. Proceedings of the 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI), Boston, MA, USA.","DOI":"10.1109\/ICTAI.2017.00197"},{"key":"ref_19","unstructured":"Leveau, V., and Joly, A. (2017). Adversarial Autoencoders for Novelty Detection. [Ph.D. Thesis, Inria-Sophia Antipolis]."},{"key":"ref_20","unstructured":"Skv\u00e1ra, V., Pevn\u00fd, T., and Sm\u00eddl, V. (2018, January 20). Are generative deep models for novelty detection truly better?. Proceedings of the ACM SIGKDD Workshop on Outlier Detection De-constructed (KDD-ODD 2018), London, UK."},{"key":"ref_21","unstructured":"Mellado, D. (2018). A Biological Inspired Artificial Neural Network Model for Incremental Learning with Novelty Detection. [Master\u2019s Thesis, Universidad de Valparaiso]."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Mellado, D., Saavedra, C., Chabert, S., Torres, R., and Salas, R. (2019). Self-Improving Generative Artificial Neural Network for Pseudo-Rehearsal Incremental Class Learning. Preprints, 2019.","DOI":"10.20944\/preprints201907.0121.v1"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1037\/0033-295X.97.2.285","article-title":"Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions","volume":"97","author":"Ratcliff","year":"1990","journal-title":"Psychol. Rev."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1080\/09540099550039318","article-title":"Catastrophic forgetting, rehearsal and pseudorehearsal","volume":"7","author":"Robins","year":"1995","journal-title":"Connect. Sci."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Mellado, D., Saavedra, C., Chabert, S., and Salas, R. (2017, January 22\u201324). Pseudorehearsal Approach for Incremental Learning of Deep Convolutional Neural Networks. Proceedings of the Computational Neuroscience: First Latin American Workshop, LAWCN 2017, Porto Alegre, Brazil.","DOI":"10.1007\/978-3-319-71011-2_10"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1142\/S0218127404009405","article-title":"How and Why Brains Create Meaning From Sensory Information","volume":"14","author":"Freeman","year":"2004","journal-title":"Int. J. Bifurc. Chaos"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1016\/S0893-6080(99)00056-8","article-title":"The consolidation of learning during sleep: Comparing the pseudorehearsal and unlearning accounts","volume":"12","author":"Robins","year":"1999","journal-title":"Neural Networks"},{"key":"ref_28","first-page":"989","article-title":"Avoiding catastrophic forgetting by coupling two reverberating neural networks","volume":"320","author":"Ans","year":"1997","journal-title":"Comptes Rendus de l\u2019Acad\u00e9mie des Sciences\u2014Series III\u2014Sciences de la Vie"},{"key":"ref_29","unstructured":"Atkinson, C., McCane, B., Szymanski, L., and Robins, A. (2018). Pseudo-Recursal: Solving the Catastrophic Forgetting Problem in Deep Neural Networks. arXiv."},{"key":"ref_30","unstructured":"Besedin, A., Blanchart, P., Crucianu, M., and Ferecatu, M. (2017, January 18\u201322). Evolutive deep models for online learning on data streams with no storage. Proceedings of the Workshop on IoT Large Scale Learning from Data Streams Co-Located with the 2017 European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2017), Skopje, Macedonia."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.neunet.2019.01.012","article-title":"Continual lifelong learning with neural networks: A review","volume":"113","author":"Parisi","year":"2019","journal-title":"Neural Netw."},{"key":"ref_32","unstructured":"Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv."},{"key":"ref_33","unstructured":"Doersch, C. (2016). Tutorial on variational autoencoders. arXiv."},{"key":"ref_34","unstructured":"Gregor, K., Danihelka, I., Graves, A., Rezende, D.J., and Wierstra, D. (2015). DRAW: A Recurrent Neural Network for Image Generation. arXiv."},{"key":"ref_35","unstructured":"Kulkarni, T.D., Whitney, W.F., Kohli, P., and Tenenbaum, J. (2015, January 7\u201312). Deep convolutional inverse graphics network. Proceedings of the 28th International Conference on Neural Information Processing Systems\u2014Volume 2, Montreal, QC, Canada."},{"key":"ref_36","unstructured":"Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., and Frey, B. (2015). Adversarial Autoencoders. arXiv."},{"key":"ref_37","unstructured":"Creswell, A., Bharath, A.A., and Sengupta, B. (2017). Conditional Autoencoders with Adversarial Information Factorization. arXiv."},{"key":"ref_38","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the 27th International Conference on Neural Information Processing Systems\u2014Volume 2, Montreal, QC, Canada."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1016\/j.sigpro.2003.07.018","article-title":"Novelty detection: A review\u2014Part 1: Statistical approaches","volume":"83","author":"Markou","year":"2003","journal-title":"Signal Process."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.sigpro.2013.12.026","article-title":"A review of novelty detection","volume":"99","author":"Pimentel","year":"2014","journal-title":"Signal Process."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Richter, C., and Roy, N. (2017, January 12\u201316). Safe visual navigation via deep learning and novelty detection. Proceedings of the RSS 2017: Robotics: Science and Systems, Cambridge, MA, USA.","DOI":"10.15607\/RSS.2017.XIII.064"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Bendale, A., and Boult, T.E. (2016, January 27\u201330). Towards open set deep networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.173"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Kotz, S., and Nadarajah, S. (2000). Extreme Value Distributions: Theory and Applications, Imperial College Press.","DOI":"10.1142\/9781860944024"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1689","DOI":"10.1109\/TPAMI.2011.54","article-title":"Meta-Recognition: The Theory and Practice of Recognition Score Analysis","volume":"33","author":"Scheirer","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell. (PAMI)"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Lai, C.D., Murthy, D., and Xie, M. (2006). Weibull Distributions and Their Applications. Springer Handbook of Engineering Statistics, Springer.","DOI":"10.1007\/978-1-84628-288-1_3"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Cohen, G., Afshar, S., Tapson, J., and van Schaik, A. (2017). EMNIST: An extension of MNIST to handwritten letters. arXiv.","DOI":"10.1109\/IJCNN.2017.7966217"},{"key":"ref_47","unstructured":"Krizhevsky, A. (2019, September 25). Available online: https:\/\/www.cs.toronto.edu\/~kriz\/learning-features-2009-TR.pdf."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"346","DOI":"10.3758\/MC.37.3.346","article-title":"Interference between storage and processing in working memory: Feature overwriting, not similarity-based competition","volume":"37","author":"Oberauer","year":"2009","journal-title":"Mem. Cogn."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/10\/206\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:26:48Z","timestamp":1760189208000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/10\/206"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,1]]},"references-count":48,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2019,10]]}},"alternative-id":["a12100206"],"URL":"https:\/\/doi.org\/10.3390\/a12100206","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints201907.0121.v1","asserted-by":"object"}]},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,1]]}}}