{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T02:32:48Z","timestamp":1784169168453,"version":"3.55.0"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"3s","license":[{"start":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T00:00:00Z","timestamp":1677196800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2020AAA0106300"],"award-info":[{"award-number":["2020AAA0106300"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62250008, 62222209, and 62102222"],"award-info":[{"award-number":["62250008, 62222209, and 62102222"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2023,10,31]]},"abstract":"<jats:p>Class incremental continual learning aims to improve the ability of modern classification models to continually recognize new classes without forgetting the previous ones. Prior art in the field has largely considered using a replay buffer. In this article, we start from an observation that the existing replay-based method would fail when the stored exemplars are not hard enough to get a good decision boundary between a previously learned class and a new class. To prevent this situation, we propose a method from the perspective of remedy after forgetting for the first time. In the proposed method, a set of exemplars is preserved as a working memory, which helps to recognize new classes. When the working memory is insufficient to distinguish between new classes, more discriminating samples would be swapped from a long-term memory, which is built up during the early training process, in an adaptive way. Our continual recognition model with adaptive memory update is capable of overcoming the problem of catastrophic forgetting with various new classes coming in sequence, especially for similar but different classes. Extensive experiments on different real-world datasets demonstrate that the proposed model is superior to existing state-of-the-art algorithms. Moreover, our model can be used as a general plugin for any replay-based continual learning algorithm to further improve their performance.<\/jats:p>","DOI":"10.1145\/3573202","type":"journal-article","created":{"date-parts":[[2022,12,6]],"date-time":"2022-12-06T00:20:55Z","timestamp":1670286055000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Continual Recognition with Adaptive Memory Update"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9073-1592","authenticated-orcid":false,"given":"Xuanrong","family":"Yao","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0351-2939","authenticated-orcid":false,"given":"Xin","family":"Wang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2887-2946","authenticated-orcid":false,"given":"Yue","family":"Liu","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2236-9290","authenticated-orcid":false,"given":"Wenwu","family":"Zhu","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,2,24]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"3931","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Abati Davide","year":"2020","unstructured":"Davide Abati, Jakub Tomczak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, and Babak Ehteshami Bejnordi. 2020. Conditional channel gated networks for task-aware continual learning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 3931\u20133940."},{"key":"e_1_3_1_3_2","unstructured":"Rahaf Aljundi Eugene Belilovsky Tinne Tuytelaars Laurent Charlin Massimo Caccia Min Lin and Lucas Page-Caccia. 2019. Online continual learning with maximal interfered retrieval. In Proceedings of the 33rd International Conference on Neural Information Processing Systems (NIPS\u201919) . 11872\u201311883."},{"key":"e_1_3_1_4_2","doi-asserted-by":"crossref","first-page":"e10778","DOI":"10.7554\/eLife.10778","article-title":"Nanoconnectomic upper bound on the variability of synaptic plasticity","volume":"4","author":"Jr. Thomas M. Bartol","year":"2015","unstructured":"Thomas M. Bartol Jr., Cailey Bromer, Justin Kinney, Michael A. Chirillo, Jennifer N. Bourne, Kristen M. Harris, and Terrence J. Sejnowski. 2015. Nanoconnectomic upper bound on the variability of synaptic plasticity. eLife 4 (2015), e10778.","journal-title":"eLife"},{"key":"e_1_3_1_5_2","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV\u201918)","author":"Belouadah Eden","year":"2018","unstructured":"Eden Belouadah and Adrian Popescu. 2018. DeeSIL: Deep-shallow incremental learning. In Proceedings of the European Conference on Computer Vision (ECCV\u201918)."},{"key":"e_1_3_1_6_2","first-page":"583","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Belouadah Eden","year":"2019","unstructured":"Eden Belouadah and Adrian Popescu. 2019. IL2M: Class incremental learning with dual memory. In Proceedings of the IEEE International Conference on Computer Vision. 583\u2013592."},{"key":"e_1_3_1_7_2","first-page":"1240","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Caccia Lucas","year":"2020","unstructured":"Lucas Caccia, Eugene Belilovsky, Massimo Caccia, and Joelle Pineau. 2020. Online learned continual compression with adaptive quantization modules. In Proceedings of the International Conference on Machine Learning. 1240\u20131250."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01258-8_15"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-01581-6"},{"key":"e_1_3_1_10_2","doi-asserted-by":"crossref","unstructured":"J. Deng W. Dong R. Socher L.-J. Li K. Li and L. Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201909) .","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_1_11_2","first-page":"5138","volume-title":"Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201919)","author":"Dhar Prithviraj","year":"2019","unstructured":"Prithviraj Dhar, Rajat Vikram Singh, Kuan-Chuan Peng, Ziyan Wu, and Rama Chellappa. 2019. Learning without memorizing. In Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201919). 5138\u20135146."},{"key":"e_1_3_1_12_2","unstructured":"Bernd Fritzke et\u00a0al. 1994. A growing neural gas network learns topologies. In Proceedings of the 7th International Conference on Neural Information Processing Systems (NIPS\u201994) . 625\u2013632."},{"key":"e_1_3_1_13_2","first-page":"770","article-title":"Deep residual learning for image recognition","author":"He Kaiming","year":"2016","unstructured":"Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201916). 770\u2013778.","journal-title":"Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201916)"},{"key":"e_1_3_1_14_2","volume-title":"Proceedings of the NIPS Deep Learning and Representation Learning Workshop","author":"Hinton Geoffrey","year":"2015","unstructured":"Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. In Proceedings of the NIPS Deep Learning and Representation Learning Workshop."},{"key":"e_1_3_1_15_2","volume-title":"Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201919)","author":"Hou Saihui","year":"2019","unstructured":"Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. 2019. Learning a unified classifier incrementally via rebalancing. In Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201919)."},{"key":"e_1_3_1_16_2","first-page":"13669","volume-title":"Advances in Neural Information Processing Systems","author":"Hung Ching-Yi","year":"2019","unstructured":"Ching-Yi Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen. 2019. Compacting, picking and growing for unforgetting continual learning. In Advances in Neural Information Processing Systems. 13669\u201313679."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11595"},{"key":"e_1_3_1_18_2","article-title":"Adam: A method for stochastic optimization","volume":"1412","author":"Kingma Diederik P.","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. CoRR abs\/1412.6980 (2015).","journal-title":"CoRR"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1611835114"},{"key":"e_1_3_1_20_2","article-title":"Optimal continual learning has perfect memory and is NP-hard","author":"Knoblauch Jeremias","year":"2020","unstructured":"Jeremias Knoblauch, Hisham Husain, and Tom Diethe. 2020. Optimal continual learning has perfect memory and is NP-hard. In Proceedings of the International Conference on Machine Learning.","journal-title":"Proceedings of the International Conference on Machine Learning."},{"key":"e_1_3_1_21_2","volume-title":"Learning Multiple Layers of Features from Tiny Images","author":"Krizhevsky Alex","year":"2009","unstructured":"Alex Krizhevsky. 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report. University of Toronto."},{"key":"e_1_3_1_22_2","first-page":"312","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Lee Kibok","year":"2019","unstructured":"Kibok Lee, Kimin Lee, Jinwoo Shin, and Honglak Lee. 2019. Overcoming catastrophic forgetting with unlabeled data in the wild. In Proceedings of the IEEE International Conference on Computer Vision. 312\u2013321."},{"key":"e_1_3_1_23_2","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Li Xilai","year":"2019","unstructured":"Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. 2019. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In Proceedings of the International Conference on Machine Learning."},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2773081"},{"key":"e_1_3_1_25_2","doi-asserted-by":"crossref","first-page":"7765","DOI":"10.1109\/CVPR.2018.00810","article-title":"PackNet: Adding multiple tasks to a single network by iterative pruning","author":"Mallya Arun","year":"2018","unstructured":"Arun Mallya and S. Lazebnik. 2018. PackNet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition.7765\u20137773.","journal-title":"Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.102.3.419"},{"issue":"7","key":"e_1_3_1_27_2","doi-asserted-by":"crossref","first-page":"e0120644","DOI":"10.1371\/journal.pone.0120644","article-title":"Replication and analysis of Ebbinghaus\u2019 forgetting curve","volume":"10","author":"Murre Jaap M. J.","year":"2015","unstructured":"Jaap M. J. Murre and Joeri Dros. 2015. Replication and analysis of Ebbinghaus\u2019 forgetting curve. PloS One 10, 7 (2015), e0120644.","journal-title":"PloS One"},{"key":"e_1_3_1_28_2","first-page":"11321","volume-title":"Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201919)","author":"Ostapenko Oleksiy","year":"2019","unstructured":"Oleksiy Ostapenko, Mihai Puscas, Tassilo Klein, Patrick Jahnichen, and Moin Nabi. 2019. Learning to remember: A synaptic plasticity driven framework for continual learning. In Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201919). 11321\u201311329."},{"key":"e_1_3_1_29_2","unstructured":"Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury Gregory Chanan Trevor Killeen et\u00a0al. 2019. PyTorch: An imperative style high-performance deep learning library. In Proceedings of the 33rd International Conference on Neural Information Processing Systems (NIPS\u201919) . 8026\u20138037."},{"key":"e_1_3_1_30_2","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV\u201920)","author":"Prabhu Ameya","year":"2020","unstructured":"Ameya Prabhu, Philip Torr, and Puneet Dokania. 2020. GDumb: A simple approach that questions our progress in continual learning. In Proceedings of the European Conference on Computer Vision (ECCV\u201920)."},{"key":"e_1_3_1_31_2","first-page":"13588","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Rajasegaran Jathushan","year":"2020","unstructured":"Jathushan Rajasegaran, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Mubarak Shah. 2020. iTAML: An incremental task-agnostic meta-learning approach. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 13588\u201313597."},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.587"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33011352"},{"key":"e_1_3_1_34_2","first-page":"9630","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"35","author":"Shim Dongsub","year":"2021","unstructured":"Dongsub Shim, Zheda Mai, Jihwan Jeong, Scott Sanner, Hyunwoo Kim, and Jongseong Jang. 2021. Online class-incremental continual learning with adversarial Shapley value. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 9630\u20139638."},{"key":"e_1_3_1_35_2","first-page":"2990","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS\u201917)","author":"Shin Hanul","year":"2017","unstructured":"Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS\u201917). 2990\u20132999."},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1162\/1064546053278973"},{"key":"e_1_3_1_37_2","doi-asserted-by":"crossref","unstructured":"Xiaoyu Tao Xinyuan Chang Xiaopeng Hong Xing Wei and Yihong Gong. 2020. Topology-preserving class-incremental learning. In Computer Vision\u2014ECCV 2020 . Lecture Notes in Computer Science Vol. 12364. Springer 254\u2013270.","DOI":"10.1007\/978-3-030-58529-7_16"},{"key":"e_1_3_1_38_2","article-title":"Three scenarios for continual learning","author":"Ven Gido M. van de","year":"2019","unstructured":"Gido M. van de Ven and Andreas S. Tolias. 2019. Three scenarios for continual learning. arXiv preprint arXiv:1904.07734 (2019).","journal-title":"arXiv preprint arXiv:1904.07734"},{"key":"e_1_3_1_39_2","unstructured":"Chenshen Wu Luis Herranz Xialei Liu Joost van de Weijer and Bogdan Raducanu. 2018. Memory replay GANs: Learning to generate new categories without forgetting. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS\u201918) . 5962\u20135972."},{"key":"e_1_3_1_40_2","doi-asserted-by":"crossref","first-page":"374","DOI":"10.1109\/CVPR.2019.00046","article-title":"Large scale incremental learning","author":"Wu Y.","year":"2019","unstructured":"Y. Wu, Yan-Jia Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. 2019. Large scale incremental learning. In Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201919).374\u2013382.","journal-title":"Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201919)."}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3573202","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3573202","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:25Z","timestamp":1750178245000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3573202"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,24]]},"references-count":39,"journal-issue":{"issue":"3s","published-print":{"date-parts":[[2023,10,31]]}},"alternative-id":["10.1145\/3573202"],"URL":"https:\/\/doi.org\/10.1145\/3573202","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,24]]},"assertion":[{"value":"2022-05-10","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-20","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-24","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}