{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T17:41:12Z","timestamp":1789666872737,"version":"4.0.1"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,7,25]],"date-time":"2024-07-25T00:00:00Z","timestamp":1721865600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science and Technology Council (NSTC) of Taiwan","award":["110-2221-E-006-136-MY3, 111-2221-E-002-146-MY3, 112-2628-E-006-012-MY3, 111-2221-E-006-001, and 112-2634-F-002-006"],"award-info":[{"award-number":["110-2221-E-006-136-MY3, 111-2221-E-002-146-MY3, 112-2628-E-006-012-MY3, 111-2221-E-006-001, and 112-2634-F-002-006"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2024,8,31]]},"abstract":"<jats:p>Incremental data drifting is a common problem when employing a machine-learning model in industrial applications. The underlying data distribution evolves gradually, e.g., users change their buying preferences on an E-commerce website over time. The problem needs to be addressed to obtain high performance. Right now, studies regarding incremental data drifting suffer from several issues. For one thing, there is a lack of clear-defined incremental drift datasets for examination. Existing efforts use either collected real datasets or synthetic datasets that show two obvious limitations. One is in particular when and of which type of drifts the distribution undergoes is unknown, and the other is that a simple synthesized dataset cannot reflect the complex representation we would normally face in the real world. For another, there lacks a well-defined protocol to evaluate a learner\u2019s knowledge transfer capability on an incremental drift dataset. To provide a holistic discussion on these issues, we create approaches to generate datasets with specific drift types, and define a novel protocol for evaluation. Besides, we investigate recent advances in the transfer learning field, including Domain Adaptation and Lifelong Learning, and examine how they perform in the presence of incremental data drifting. The results unfold the relationships among drift types, knowledge preservation, and learning approaches.<\/jats:p>","DOI":"10.1145\/3655630","type":"journal-article","created":{"date-parts":[[2024,5,24]],"date-time":"2024-05-24T07:17:44Z","timestamp":1716535064000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Incremental Data Drifting: Evaluation Metrics, Data Generation, and Approach Comparison"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-9281-9886","authenticated-orcid":false,"given":"Yu-Tung","family":"Pai","sequence":"first","affiliation":[{"name":"CSIE, National Taiwan University, Taipei, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0694-9878","authenticated-orcid":false,"given":"Nien-En","family":"Sun","sequence":"additional","affiliation":[{"name":"CSIE, National Taiwan University, Taipei, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7995-4787","authenticated-orcid":false,"given":"Cheng-Te","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9970-1250","authenticated-orcid":false,"given":"Shou-de","family":"Lin","sequence":"additional","affiliation":[{"name":"CSIE, National Taiwan University, Taipei, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,7,25]]},"reference":[{"key":"e_1_3_2_2_2","article-title":"Invariant risk minimization","author":"Arjovsky Martin","year":"2019","unstructured":"Martin Arjovsky, L\u00e9on Bottou, Ishaan Gulrajani, and David Lopez-Paz. 2019. Invariant risk minimization. arXiv preprint arXiv:1907.02893 (2019).","journal-title":"arXiv preprint arXiv:1907.02893"},{"key":"e_1_3_2_3_2","unstructured":"Manuel Baena-Garc\u00eda Jos\u00e9 Campo-\u00c1vila Ra\u00fal Fidalgo-Merino Albert Bifet Ricard Gavald and Rafael Morales-Bueno. 2006. Early drift detection method. (012006)."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0168-1699(99)00046-0"},{"key":"e_1_3_2_5_2","volume-title":"International Conference on Learning Representations (ICLR \u201919)","author":"Brock Andrew","year":"2019","unstructured":"Andrew Brock, Jeff Donahue, and Karen Simonyan. 2019. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations (ICLR \u201919)."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-019-05840-z"},{"key":"e_1_3_2_7_2","first-page":"2189","volume-title":"International Conference on Machine Learning","author":"Creager Elliot","year":"2021","unstructured":"Elliot Creager, J\u00f6rn-Henrik Jacobsen, and Richard Zemel. 2021. Environment inference for invariant learning. In International Conference on Machine Learning. PMLR, 2189\u20132200."},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.5555\/3158281"},{"issue":"7","key":"e_1_3_2_9_2","first-page":"3366","article-title":"A continual learning survey: Defying forgetting in classification tasks","volume":"44","author":"Lange Matthias De","year":"2022","unstructured":"Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. 2022. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 7 (2022), 3366\u20133385.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_2_10_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4171\u20134186."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2011.2160459"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/1014052.1014069"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dib.2015.01.003"},{"key":"e_1_3_2_14_2","first-page":"286","article-title":"Learning with drift detection","volume":"8","author":"Gama Jo\u00e3o","year":"2004","unstructured":"Jo\u00e3o Gama, Pedro Medas, Gladys Castillo, and Pedro Rodrigues. 2004. Learning with drift detection. Intelligent Data Analysis 8, 286\u2013295.","journal-title":"Intelligent Data Analysis"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/2523813"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2523813"},{"key":"e_1_3_2_17_2","first-page":"59:1\u201359:35","article-title":"Domain-adversarial training of neural networks","volume":"17","author":"Ganin Yaroslav","year":"2016","unstructured":"Yaroslav Ganin, E. Ustinova, Hana Ajakan, Pascal Germain, H. Larochelle, Fran\u00e7ois Laviolette, M. Marchand, and V. Lempitsky. 2016. Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17 (2016), 59:1\u201359:35.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3054925"},{"key":"e_1_3_2_19_2","volume-title":"Advances in Neural Information Processing Systems (NeurIPS \u201914)","author":"Goodfellow Ian","year":"2014","unstructured":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial Networks. In Advances in Neural Information Processing Systems (NeurIPS \u201914)."},{"key":"e_1_3_2_20_2","article-title":"Out-of-distribution prediction with invariant risk minimization: The limitation and an effective fix","author":"Guo Ruocheng","year":"2021","unstructured":"Ruocheng Guo, Pengchuan Zhang, Hao Liu, and Emre Kiciman. 2021. Out-of-distribution prediction with invariant risk minimization: The limitation and an effective fix. arXiv preprint arXiv:2101.07732 (2021).","journal-title":"arXiv preprint arXiv:2101.07732"},{"key":"e_1_3_2_21_2","article-title":"Highly scalable deep learning training system with mixed-precision: Training ImageNet in four minutes","author":"Jia Xianyan","year":"2018","unstructured":"Xianyan Jia, Shutao Song, Wei He, Yangzihao Wang, Haidong Rong, Feihu Zhou, Liqiang Xie, Zhenyu Guo, Yuanzhou Yang, Liwei Yu, Tiegang Chen, Guangxiao Hu, Shaohuai Shi, and Xiaowen Chu. 2018. Highly scalable deep learning training system with mixed-precision: Training ImageNet in four minutes. arXiv preprint arXiv:1807.11205 (2018).","journal-title":"arXiv preprint arXiv:1807.11205"},{"key":"e_1_3_2_22_2","volume-title":"International Conference on Learning Representations (ICLR \u201918)","author":"Karras Tero","year":"2018","unstructured":"Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive growing of GANs for improved quality, stability, and variation. In International Conference on Learning Representations (ICLR \u201918)."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.5555\/1746286.1746291"},{"key":"e_1_3_2_25_2","volume-title":"International Conference on Learning Representations (ICLR \u201915)","author":"Kingma Diederik P.","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR \u201915)."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1611835114"},{"key":"e_1_3_2_27_2","first-page":"5815","volume-title":"International Conference on Machine Learning","author":"Krueger David","year":"2021","unstructured":"David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. 2021. Out-of-distribution generalization via risk extrapolation (rex). In International Conference on Machine Learning. PMLR, 5815\u20135826."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.5555\/3524938.3525445"},{"key":"e_1_3_2_29_2","volume-title":"International Conference on Learning Representations","author":"Lin Sen","year":"2022","unstructured":"Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. 2022. TRGP: Trust region gradient projection for continual learning. In International Conference on Learning Representations."},{"key":"e_1_3_2_30_2","volume-title":"Advances in Neural Information Processing Systems","author":"Lopez-Paz David","year":"2017","unstructured":"David Lopez-Paz and Marc'Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems."},{"issue":"12","key":"e_1_3_2_31_2","first-page":"2346","article-title":"Learning under concept drift: A review","volume":"31","author":"Lu Jie","year":"2018","unstructured":"Jie Lu, Anjin Liu, Fan Dong, Feng Gu, Jo\u00e1o Gama, and Guangquan Zhang. 2018. Learning under concept drift: A review. IEEE Trans. Knowl. Data Eng. 31, 12 (Oct.2018), 2346\u20132363.","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"e_1_3_2_32_2","article-title":"Revisiting small batch training for deep neural networks","author":"Masters Dominic","year":"2018","unstructured":"Dominic Masters and Carlo Luschi. 2018. Revisiting small batch training for deep neural networks. arXiv preprint arXiv:1804.07612 (2018).","journal-title":"arXiv preprint arXiv:1804.07612"},{"key":"e_1_3_2_33_2","first-page":"188","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP \u201919)","author":"Ni Jianmo","year":"2019","unstructured":"Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP \u201919). 188\u2013197."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2014.2347059"},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","unstructured":"Amanda Rios and Laurent Itti. 2019. Closed-loop memory GAN for continual learning. 3332\u20133338.","DOI":"10.24963\/ijcai.2019\/462"},{"key":"e_1_3_2_36_2","first-page":"9623","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"37","author":"Rostami Mohammad","year":"2023","unstructured":"Mohammad Rostami and Aram Galstyan. 2023. Overcoming concept shift in domain-aware settings through consolidated internal distributions. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 9623\u20139631."},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","first-page":"3723","DOI":"10.1109\/CVPR.2018.00392","article-title":"Maximum classifier discrepancy for unsupervised domain adaptation","author":"Saito Kuniaki","year":"2018","unstructured":"Kuniaki Saito, Kohei Watanabe, Y. Ushiku, and T. Harada. 2018. Maximum classifier discrepancy for unsupervised domain adaptation. 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (2018), 3723\u20133732.","journal-title":"2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"e_1_3_2_38_2","article-title":"InterFaceGAN: Interpreting the disentangled face representation learned by GANs","author":"Shen Yujun","year":"2020","unstructured":"Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou. 2020. InterFaceGAN: Interpreting the disentangled face representation learned by GANs. IEEE Transactions on Pattern Analysis and Machine Intelligence PP (2020).","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_2_39_2","volume-title":"2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201921)","author":"Shen Yujun","year":"2021","unstructured":"Yujun Shen and Bolei Zhou. 2021. Closed-form factorization of latent semantics in GANs. In 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR \u201921)."},{"key":"e_1_3_2_40_2","unstructured":"Hanul Shin Jung Lee Jaehong Kim and Jiwon Kim. 2017. Continual learning with deep generative replay. (052017)."},{"key":"e_1_3_2_41_2","unstructured":"Rui Shu Hung Bui Hirokazu Narui and Stefano Ermon. 2018. A DIRT-T approach to unsupervised domain adaptation. (022018)."},{"key":"e_1_3_2_42_2","article-title":"A segment-based drift adaptation method for data streams","author":"Song Yiliao","year":"2021","unstructured":"Yiliao Song, Jie Lu, Anjin Liu, Haiyan Lu, and Guangquan Zhang. 2021. A segment-based drift adaptation method for data streams. IEEE Transactions on Neural Networks and Learning Systems (2021).","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_3_2_43_2","article-title":"Learning data streams with changing distributions and temporal dependency","author":"Song Yiliao","year":"2021","unstructured":"Yiliao Song, Jie Lu, Haiyan Lu, and Guangquan Zhang. 2021. Learning data streams with changing distributions and temporal dependency. IEEE Transactions on Neural Networks and Learning Systems (2021).","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/502512.502568"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00020"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.snb.2012.01.074"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/956750.956778"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2018.05.083"},{"issue":"11","key":"e_1_3_2_49_2","first-page":"1","article-title":"Data stream mining: Methods and challenges for handling concept drift","volume":"1","author":"Wares Scott","year":"2019","unstructured":"Scott Wares, John Isaacs, and Eyad Elyan. 2019. Data stream mining: Methods and challenges for handling concept drift. SN Appl. Sci. 1, 11 (2019), 1\u201319.","journal-title":"SN Appl. Sci."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00116900"},{"key":"e_1_3_2_51_2","volume-title":"Advances in Neural Information Processing Systems (NeurIPS \u201919)","author":"Xu Lei","year":"2019","unstructured":"Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. 2019. Modeling tabular data using conditional GAN. In Advances in Neural Information Processing Systems (NeurIPS \u201919)."},{"key":"e_1_3_2_52_2","volume-title":"Advances in Neural Information Processing Systems","author":"Zhang Marvin Mengxin","year":"2021","unstructured":"Marvin Mengxin Zhang, Henrik Marklund, Nikita Dhawan, Abhishek Gupta, Sergey Levine, and Chelsea Finn. 2021. Adaptive risk minimization: Learning to adapt to domain shift. In Advances in Neural Information Processing Systems."},{"key":"e_1_3_2_53_2","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR \u201917)","author":"Zhang Zhifei","year":"2017","unstructured":"Zhifei Zhang, Yang Song, and Hairong Qi. 2017. Age progression\/regression by conditional adversarial autoencoder. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR \u201917)."}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3655630","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3655630","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:46Z","timestamp":1750291426000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3655630"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,25]]},"references-count":52,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,8,31]]}},"alternative-id":["10.1145\/3655630"],"URL":"https:\/\/doi.org\/10.1145\/3655630","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,25]]},"assertion":[{"value":"2023-04-18","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-28","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}