{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T01:43:54Z","timestamp":1787017434457,"version":"build-2736575974"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"8","license":[{"start":{"date-parts":[[2024,7,31]],"date-time":"2024-07-31T00:00:00Z","timestamp":1722384000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100018537","name":"National Science and Technology Major Project","doi-asserted-by":"crossref","award":["2022ZD0114800"],"award-info":[{"award-number":["2022ZD0114800"]}],"id":[{"id":"10.13039\/501100018537","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Science Foundation of China","doi-asserted-by":"crossref","award":["61921006, 62206125"],"award-info":[{"award-number":["61921006, 62206125"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Postdoctoral Program for Innovative Talent, and China Postdoctoral Science Foundation","award":["2023M731597"],"award-info":[{"award-number":["2023M731597"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>\n                    Learning with data streams has attracted much attention in recent decades. Conventional approaches typically assume that the feature and label of a data item can be timely observed at each round. In many real-world tasks, however, it often occurs that either the feature or the label is observed firstly while the other arrives with delay. For instance, in distributed learning systems, a central processor collects training data from different sub-processors to train a learning model, whereas the feature and label of certain data items can arrive asynchronously due to network latency. The problem of learning with\n                    <jats:italic>asynchronous<\/jats:italic>\n                    feature or label in streams encompasses many applications but still lacks sound solutions. In this article, we formulate the problem and propose a new approach to alleviate the negative effect of asynchronicity and mining asynchronous data streams. Our approach carefully exploits the timely arrived information and builds an online ensemble structure to adaptively reuse historical models and instances. We provide the theoretical guarantees of our approach and conduct extensive experiments to validate its effectiveness.\n                  <\/jats:p>","DOI":"10.1145\/3662186","type":"journal-article","created":{"date-parts":[[2024,5,3]],"date-time":"2024-05-03T11:51:02Z","timestamp":1714737062000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Learning with Asynchronous Labels"],"prefix":"10.1145","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5812-2807","authenticated-orcid":false,"given":"Yu-Yang","family":"Qian","sequence":"first","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, and School of Artificial Intelligence, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2101-1836","authenticated-orcid":false,"given":"Zhen-Yu","family":"Zhang","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China and RIKEN Center for Advanced Intelligence Project, Tokyo, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7925-8255","authenticated-orcid":false,"given":"Peng","family":"Zhao","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, and School of Artificial Intelligence, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0746-1494","authenticated-orcid":false,"given":"Zhi-Hua","family":"Zhou","sequence":"additional","affiliation":[{"name":"National Key Laboratory for Novel Software Technology, and School of Artificial Intelligence, Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,7,31]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Proceedings of the 21st European Symposium on Artificial Neural Networks","volume":"3","author":"Anguita Davide","year":"2013","unstructured":"Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge Luis Reyes-Ortiz. 2013. A Public Domain Dataset for Human Activity Recognition Using Smartphones. In Proceedings of the 21st European Symposium on Artificial Neural Networks. 3."},{"key":"e_1_3_2_3_2","first-page":"29960","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 35 (NeurIPS)","author":"Bai Yong","year":"2022","unstructured":"Yong Bai, Yu-Jie Zhang, Peng Zhao, Masashi Sugiyama, and Zhi-Hua Zhou. 2022. Adapting to Online Label Shift with Provable Guarantees. In Proceedings of the Advances in Neural Information Processing Systems 35 (NeurIPS). 29960\u201329974."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSIPN.2019.2902493"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3139240"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511546921"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623634"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1137\/0803026"},{"key":"e_1_3_2_9_2","first-page":"6.1","volume-title":"Proceedings of the 25th Annual Conference on Computational Learning Theory (COLT)","author":"Chiang Chao-Kai","year":"2012","unstructured":"Chao-Kai Chiang, Tianbao Yang, Chia-Jung Lee, Mehrdad Mahdavi, Chi-Jen Lu, Rong Jin, and Shenghuo Zhu. 2012. Online Optimization with Gradual Variations. In Proceedings of the 25th Annual Conference on Computational Learning Theory (COLT). 6.1\u20136.20."},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/2829955"},{"key":"e_1_3_2_11_2","first-page":"3363","volume-title":"Proceedings of the 38th International Conference on Machine Learning (ICML)","author":"Flaspohler Genevieve","year":"2021","unstructured":"Genevieve Flaspohler, Francesco Orabona, Judah Cohen, Soukayna Mouatadid, Miruna Oprescu, Paulo Orenstein, and Lester Mackey. 2021. Online Learning with Optimism and Delay. In Proceedings of the 38th International Conference on Machine Learning (ICML). 3363\u20133373."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCI.2017.2699865"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10710-017-9314-z"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","unstructured":"Priya Goyal Piotr Doll\u00e1r Ross B. Girshick Pieter Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia and Kaiming He. 2017. Accurate Large Minibatch SGD: Training ImageNet in 1 Hour. arXiv:1706.02677. DOI: 10.48550\/arXiv.1706.02677","DOI":"10.48550\/arXiv.1706.02677"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-019-00654-y"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1002\/acs.4480070604"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1561\/2400000013"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.5555\/3524938.3525328"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3466616"},{"key":"e_1_3_2_20_2","first-page":"78","article-title":"Multi-Agent Online Optimization with Delays: Asynchronicity, Adaptivity, and Optimism","volume":"23","author":"Hsieh Yu-Guan","year":"2022","unstructured":"Yu-Guan Hsieh, Franck Iutzeler, J\u00e9rome Malick, and Panayotis Mertikopoulos. 2022. Multi-Agent Online Optimization with Delays: Asynchronicity, Adaptivity, and Optimism. Journal of Machine Learning Research 23, 78 (2022), 1\u201349.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_21_2","first-page":"1453","volume-title":"Proceedings of the 34th International Conference on Machine Learning (ICML)","author":"Joulani Pooria","year":"2013","unstructured":"Pooria Joulani, Andr\u00e1s Gy\u00f6rgy, and Csaba Szepesv\u00e1ri. 2013. Online Learning under Delayed Feedback. In Proceedings of the 34th International Conference on Machine Learning (ICML). 1453\u20131461."},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2017.02.004"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1002\/nav.3800020109"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.5555\/3524938.3525445"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/1964897.1964918"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-022-1270-y"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553456"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/MPRV.2018.03367731"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2017.2691702"},{"key":"e_1_3_2_30_2","unstructured":"Francesco Orabona. 2019. A Modern Introduction to Online Learning. arXiv:1912.13213."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2016.7840877"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM58522.2023.00162"},{"key":"e_1_3_2_33_2","first-page":"1270","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 28 (NeurIPS)","author":"Quanrud Kent","year":"2015","unstructured":"Kent Quanrud and Daniel Khashabi. 2015. Online Learning with Adversarial Delays. In Proceedings of the Advances in Neural Information Processing Systems 28 (NeurIPS). 1270\u20131278."},{"key":"e_1_3_2_34_2","first-page":"993","volume-title":"Proceedings of the 26th Annual Conference on Computational Learning Theory (COLT)","author":"Rakhlin Alexander","year":"2013","unstructured":"Alexander Rakhlin and Karthik Sridharan. 2013. Online Learning with Predictable Sequences. In Proceedings of the 26th Annual Conference on Computational Learning Theory (COLT). 993\u20131019."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401282"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1561\/2200000018"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICMLA.2015.174"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","unstructured":"Suvrit Sra Adams Wei Yu Mu Li and Alexander J. Smola. 2015. AdaDelay: Delay Adaptive Distributed Stochastic Convex Optimization. arXiv:1508.05003. DOI: 10.48550\/arXiv.1508.05003","DOI":"10.48550\/arXiv.1508.05003"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2809695.2809718"},{"key":"e_1_3_2_40_2","first-page":"3522","volume-title":"Proceedings of the 29th International Conference on International Joint Conferences on Artificial Intelligence (IJCAI)","author":"Su Yumin","year":"2021","unstructured":"Yumin Su, Liang Zhang, Quanyu Dai, Bo Zhang, Jinyao Yan, Dan Wang, Yongjun Bao, Sulong Xu, Yang He, and Weipeng Yan. 2021. An Attention-Based Model for Conversion Rate Prediction with Delayed Feedback via Post-Click Calibration. In Proceedings of the 29th International Conference on International Joint Conferences on Artificial Intelligence (IJCAI). 3522\u20133528."},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-021-06072-w"},{"key":"e_1_3_2_42_2","first-page":"19703","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 35 (NeurIPS)","author":"Wan Yuanyu","year":"2022","unstructured":"Yuanyu Wan, Wei-Wei Tu, and Lijun Zhang. 2022b. Online Frank-Wolfe with Arbitrary Delays. In Proceedings of the Advances in Neural Information Processing Systems 35 (NeurIPS). 19703\u201319715."},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","unstructured":"Yuanyu Wan Yibo Wang Chang Yao Wei-Wei Tu and Lijun Zhang. 2022c. Projection-Free Online Learning with Arbitrary Delays. arXiv:2204.04964. DOI: 10.48550\/arXiv.2204.04964","DOI":"10.48550\/arXiv.2204.04964"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","unstructured":"Yuanyu Wan Chang Yao Mingli Song and Lijun Zhang. 2024. Improved Regret for Bandit Convex Optimization with Delayed Feedback. arXiv:2402.09152. DOI: 10.48550\/arXiv.2402.09152","DOI":"10.48550\/arXiv.2402.09152"},{"key":"e_1_3_2_45_2","first-page":"7913","volume-title":"Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS)","author":"Wang Jing","year":"2023","unstructured":"Jing Wang, Peng Zhao, and Zhi-Hua Zhou. 2023. Revisiting Weighted Strategy for Non-Stationary Parametric Bandits. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS). 7913\u20137942."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","unstructured":"Yanshi Wang Jie Zhang Qing Da and Anxiang Zeng. 2020. Delayed Feedback Modeling for the Entire Space Conversion Rate Prediction. arXiv:2011.11826. DOI: 10.48550\/arXiv.2011.11826","DOI":"10.48550\/arXiv.2011.11826"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2002.1013136"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-023-2689-5"},{"issue":"5","key":"e_1_3_2_49_2","doi-asserted-by":"crossref","first-page":"1178","DOI":"10.1109\/TPAMI.2012.197","article-title":"Online Feature Selection with Streaming Features","volume":"35","author":"Wu Xindong","year":"2012","unstructured":"Xindong Wu, Kui Yu, Wei Ding, Hao Wang, and Xingquan Zhu. 2012. Online Feature Selection with Streaming Features. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 5 (2012), 1178\u20131192.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401314"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462892"},{"key":"e_1_3_2_52_2","first-page":"29074","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 36 (NeurIPS)","author":"Zhang Yu-Jie","year":"2023","unstructured":"Yu-Jie Zhang, Zhen-Yu Zhang, Peng Zhao, and Masashi Sugiyama. 2023b. Adapting to Continuous Covariate Shift via Online Density Ratio Estimation. In Proceedings of the Advances in Neural Information Processing Systems 36 (NeurIPS). 29074\u201329113."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11432-020-3419-4"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539351"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-019-05835-w"},{"issue":"3","key":"e_1_3_2_56_2","first-page":"951","article-title":"Distribution-Free One-Pass Learning","volume":"33","author":"Zhao Peng","year":"2021","unstructured":"Peng Zhao, Xinqiang Wang, Siyu Xie, Lei Guo, and Zhi-Hua Zhou. 2021. Distribution-Free One-Pass Learning. IEEE Transaction on Knowledge and Data Engineering 33, 3 (2021), 951\u2013963.","journal-title":"IEEE Transaction on Knowledge and Data Engineering"},{"key":"e_1_3_2_57_2","first-page":"12510","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 33 (NeurIPS)","author":"Zhao Peng","year":"2020","unstructured":"Peng Zhao, Yu-Jie Zhang, Lijun Zhang, and Zhi-Hua Zhou. 2020b. Dynamic Regret of Convex and Smooth Functions. In Proceedings of the Advances in Neural Information Processing Systems 33 (NeurIPS). 12510\u201312520."},{"key":"e_1_3_2_58_2","first-page":"98","article-title":"Adaptivity and Non-Stationarity: Problem-Dependent Dynamic Regret for Online Convex Optimization","volume":"25","author":"Zhao Peng","year":"2024","unstructured":"Peng Zhao, Yu-Jie Zhang, Lijun Zhang, and Zhi-Hua Zhou. 2024. Adaptivity and Non-Stationarity: Problem-Dependent Dynamic Regret for Online Convex Optimization. Journal of Machine Learning Research 25, 98 (2024), 1\u201352.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.5555\/3305890.3306107"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11432-023-3823-6"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1201\/b12207"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1093\/nsr\/nwac123"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-022-2900-0"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3662186","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3662186","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T19:57:10Z","timestamp":1750276630000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3662186"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,31]]},"references-count":62,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3662186"],"URL":"https:\/\/doi.org\/10.1145\/3662186","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,31]]},"assertion":[{"value":"2022-10-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-31","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}