{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T04:35:09Z","timestamp":1781584509946,"version":"3.54.5"},"reference-count":44,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2025,8,23]],"date-time":"2025-08-23T00:00:00Z","timestamp":1755907200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Symmetry and asymmetry between past and future knowledge are at the heart of continual learning. Deep neural networks typically lose the temporal symmetry that would preserve earlier knowledge when the network is trained sequentially, a phenomenon known as catastrophic forgetting. Dynamically expandable networks (DENs) attempt to restore symmetry by allocating a dedicated module\u2014such as a feature extractor or a task token\u2014for every new task while freezing all previously learned modules. Although this strategy yields high average accuracy, we observe a pronounced asymmetry: earlier tasks still degrade over time, indicating that frozen modules alone do not guarantee knowledge conservation. Moreover, feature bias, arising from the imbalance between old and new samples, further exacerbates the forgetting issue. This raises a fundamental challenge: how can multiple feature extractors be coordinated more effectively to mitigate catastrophic forgetting while enabling the robust acquisition of new tasks? To address this challenge, we propose two asymmetric, contrastive auxiliary losses that exploit rich information from previous tasks to guide new task learning. Specifically, our approach integrates features extracted by both frozen and current modules to reinforce task boundaries while facilitating the learning process. In addition, we introduce a feature adjustment mechanism to alleviate the bias caused by class imbalance. Extensive experiments on benchmarks, including DyTox and MCG, demonstrate that our approach reduces catastrophic forgetting and achieves state-of-the-art performance on ImageNet-100.<\/jats:p>","DOI":"10.3390\/sym17091379","type":"journal-article","created":{"date-parts":[[2025,8,25]],"date-time":"2025-08-25T00:09:32Z","timestamp":1756080572000},"page":"1379","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Rethinking the Stability\u2013Plasticity Dilemma of Dynamically Expandable Networks"],"prefix":"10.3390","volume":"17","author":[{"given":"Mingda","family":"Dong","sequence":"first","affiliation":[{"name":"Lab of Artificial Intelligence for Education, East China Normal University, Shanghai 200062, China"},{"name":"School of Computer Science and Technology, East China Normal University, Shanghai 200062, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rui","family":"Li","sequence":"additional","affiliation":[{"name":"China Mobile (HangZhou) Information Technology Co., Ltd., Hangzhou 311121, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,8,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/S0079-7421(08)60536-8","article-title":"Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem","volume":"Volume 24","author":"McCloskey","year":"1989","journal-title":"Psychology of Learning and Motivation"},{"key":"ref_2","unstructured":"Pham, Q., Liu, C., and Hoi, S. (2021, January 6\u201314). DualNet: Continual Learning, Fast and Slow. Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS), Virtual Event."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1037\/0033-295X.102.3.419","article-title":"Why There Are Complementary Learning Systems in the Hippocampus and Neocortex: Insights from the Successes and Failures of Connectionist Models of Learning and Memory","volume":"102","author":"McClelland","year":"1995","journal-title":"Psychol. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Yan, S., Xie, J., and He, X. (2021, January 19\u201325). DER: Dynamically Expandable Representation for Class Incremental Learning. Proceedings of the 34th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00303"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Douillard, A., Ram\u00e9, A., Couairon, G., and Cord, M. (2022, January 19\u201324). DyTox: Transformers for Continual Learning with Dynamic Token Expansion. Proceedings of the 35th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00907"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Cai, T., Zhang, Z., Tan, X., Qu, Y., Jiang, G., Wang, C., and Xie, Y. (2023, January 18\u201322). Multi-Centroid Task Descriptor for Dynamic Class Incremental Inference. Proceedings of the 36th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00705"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hu, Z., Li, Y., Lyu, J., Gao, D., and Vasconcelos, N. (2023, January 18\u201322). Dense Network Expansion for Class Incremental Learning. Proceedings of the 36th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01141"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3521","DOI":"10.1073\/pnas.1611835114","article-title":"Overcoming Catastrophic Forgetting in Neural Networks","volume":"114","author":"Kirkpatrick","year":"2017","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., and Tuytelaars, T. (2018, January 8\u201314). Memory Aware Synapses: Learning What (Not) to Forget. Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01219-9_9"},{"key":"ref_10","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Rebuffi, S.A., Kolesnikov, A., Sperl, G., and Lampert, C. (2017, January 21\u201326). iCarl: Incremental Classifier and Representation Learning. Proceedings of the 30th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.587"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Douillard, A., Cord, M., Ollion, C., Robert, T., and Valle, E. (2020, January 23\u201328). PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning. Proceedings of the 16th European Conference on Computer Vision (ECCV), Virtual Event.","DOI":"10.1007\/978-3-030-58565-5_6"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yu, L., Twardowski, B., Liu, X., Herranz, L., Wang, K., Cheng, Y., Jui, S., and van de Weijer, J. (2020, January 14\u201319). Semantic Drift Compensation for Class-Incremental Learning. Proceedings of the 33rd IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual Event.","DOI":"10.1109\/CVPR42600.2020.00701"},{"key":"ref_14","unstructured":"Zheng, B., Zhou, D.W., Ye, H.J., and Zhan, D.C. (2024, January 21\u201327). Multi-Layer Rehearsal Feature Augmentation for Class-Incremental Learning. Proceedings of the 41st International Conference on Machine Learning(ICML), Vienna, Austria."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Hou, S., Pan, X., Loy, C., Wang, Z., and Lin, D. (2019, January 16\u201320). Learning a Unified Classifier Incrementally via Rebalancing. Proceedings of the 32nd IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00092"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., and Fu, Y. (2019, January 16\u201320). Large Scale Incremental Learning. Proceedings of the 32nd IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00046"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhao, B., Xiao, X., Gan, G., Zhang, B., and Xia, S.T. (2020, January 14\u201319). Maintaining Discrimination and Fairness in Class Incremental Learning. Proceedings of the 33rd IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual Event.","DOI":"10.1109\/CVPR42600.2020.01322"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5497","DOI":"10.1109\/TPAMI.2022.3206549","article-title":"Class-Incremental Continual Learning into the Extended Der-Verse","volume":"45","author":"Boschini","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","unstructured":"He, J. (, January 17\u201321). Gradient Reweighting: Towards Imbalanced Class-Incremental Learning. Proceedings of the 41st IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kim, D., and Han, B. (2023, January 18\u201322). On the Stability-Plasticity Dilemma of Class-Incremental Learning. Proceedings of the 36th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01934"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, Y., Schiele, B., and Sun, Q. (2021, January 19\u201325). Adaptive Aggregation Networks for Class-Incremental Learning. Proceedings of the 34th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00257"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Wang, F.Y., Zhou, D.W., Ye, H.J., and Zhan, D.C. (2022, January 23\u201327). FOSTER: Feature Boosting and Compression for Class-Incremental Learning. Proceedings of the 17th European Conference on Computer Vision (ECCV), Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-19806-9_23"},{"key":"ref_23","unstructured":"Wang, F.Y., Zhou, D.W., Liu, L., Ye, H.J., Bian, Y., Zhan, D.C., and Zhao, P. (2023, January 1\u20135). BEEF: Bi-Compatible Class-Incremental Learning via Energy-Based Expansion and Fusion. Proceedings of the The 11th International Conference on Learning Representations (ICLR), Kigali, Rwanda."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, X., and Chang, X. (2023, January 2\u20136). Dynamic Residual Classifier for Class Incremental Learning. Proceedings of the 19th IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.01718"},{"key":"ref_25","unstructured":"Rype\u015b\u0107, G., Cygert, S., Khan, V., Trzcinski, T., Zieli\u0144ski, B., and Twardowski, B. (2024, January 7\u201311). Divide and Not Forget: Ensemble of Selectively Trained Experts in Continual Learning. Proceedings of the The 12th International Conference on Learning Representations (ICLR), Vienna, Austria."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"129807","DOI":"10.1016\/j.neucom.2025.129807","article-title":"Feature expansion and enhanced compression for class incremental learning","volume":"634","author":"Ferdinand","year":"2025","journal-title":"Neurocomputing"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020, January 14\u201319). Momentum Contrast for Unsupervised Visual Representation Learning. Proceedings of the 33rd IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual Event.","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"ref_28","unstructured":"Grill, J.B., Strub, F., Altch\u00e9, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., and Gheshlaghi Azar, M. (2020, January 6\u201312). Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS), Virtual Event."},{"key":"ref_29","unstructured":"Chen, T., Kornblith, S., Swersky, K., Norouzi, M., and Hinton, G. (2020). Big Self-Supervised Models are Strong Semi-Supervised Learners. arXiv."},{"key":"ref_30","unstructured":"Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., and Krishnan, D. (2020, January 6\u201312). Supervised Contrastive Learning. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS), Virtual Event."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cha, H., Lee, J., and Shin, J. (2021, January 11\u201317). Co2L: Contrastive Continual Learning. Proceedings of the 18th IEEE\/CVF International Conference on Computer Vision (ICCV), Virtual Event.","DOI":"10.1109\/ICCV48922.2021.00938"},{"key":"ref_32","unstructured":"Guo, Y., Liu, B., and Zhao, D. (2022, January 17\u201323). Online Continual Learning Through Mutual Information Maximization. Proceedings of the 39th International Conference on Machine Learning (ICML). PMLR, Baltimore, MD, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ahmed, N., Kukleva, A., and Schiele, B. (2024, January 17\u201321). OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning. Proceedings of the 41st IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.02717"},{"key":"ref_34","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 3\u20137). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Proceedings of the International Conference on Learning Representations (ICLR), Virtual Event."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wang, Z., Zhang, Z., Lee, C.Y., Zhang, H., Sun, R., Ren, X., Su, G., Perot, V., Dy, J., and Pfister, T. (2022, January 19\u201324). Learning to Prompt for Continual Learning. Proceedings of the 35th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00024"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Wang, Z., Zhang, Z., Ebrahimi, S., Sun, R., Zhang, H., Lee, C.Y., Ren, X., Su, G., Perot, V., and Dy, J. (2022, January 23\u201327). DualPrompt: Complementary Prompting for Rehearsal-Free Continual Learning. Proceedings of the 17th European Conference on Computer Vision (ECCV), Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-19809-0_36"},{"key":"ref_37","unstructured":"Wang, Y., Huang, Z., and Hong, X. (December, January 28). S-Prompts Learning with Pre-Trained Transformers: An Occam\u2019s Razor for Domain Incremental Learning. Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS), New Orleans, LA, USA."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Smith, J., Karlinsky, L., Gutta, V., Cascante-Bonilla, P., Kim, D., Arbelle, A., Panda, R., Feris, R., and Kira, Z. (2023, January 18\u201322). CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning. Proceedings of the 36th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01146"},{"key":"ref_39","unstructured":"Wang, L., Xie, J., Zhang, X., Huang, M., Su, H., and Zhu, J. (2023, January 10\u201316). Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-Optimality. Proceedings of the 37th Conference on Neural Information Processing Systems, New Orleans, LA, USA."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wang, R., Duan, X., Kang, G., Liu, J., Lin, S., Xu, S., L\u00fc, J., and Zhang, B. (2023, January 18\u201322). AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learning. Proceedings of the 36th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00356"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"113009","DOI":"10.1016\/j.knosys.2025.113009","article-title":"Knowledge-guided prompt-based continual learning: Aligning task-prompts through contrastive hard negatives","volume":"310","author":"Lu","year":"2025","journal-title":"Knowl.-Based Syst."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhou, D.W., Sun, H.L., Ye, H.J., and Zhan, D.C. (2024, January 17\u201321). Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning. Proceedings of the 36th IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.02223"},{"key":"ref_43","unstructured":"Lopez-Paz, D., and Ranzato, M. (2017). Gradient Episodic Memory for Continuum Learning. arXiv."},{"key":"ref_44","first-page":"2579","article-title":"Visualizing Data Using t-SNE","volume":"9","author":"Hinton","year":"2008","journal-title":"J. Mach. Learn. Res."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/9\/1379\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:34:35Z","timestamp":1760034875000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/9\/1379"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,23]]},"references-count":44,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["sym17091379"],"URL":"https:\/\/doi.org\/10.3390\/sym17091379","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,23]]}}}