{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T09:51:00Z","timestamp":1773481860902,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":80,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,5,8]],"date-time":"2023-05-08T00:00:00Z","timestamp":1683504000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Swiss National Science Foundation","award":["200021_204620"],"award-info":[{"award-number":["200021_204620"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,5,8]]},"DOI":"10.1145\/3578356.3592585","type":"proceedings-article","created":{"date-parts":[[2023,5,4]],"date-time":"2023-05-04T19:44:37Z","timestamp":1683229477000},"page":"8-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Towards A Platform and Benchmark Suite for Model Training on Dynamic Datasets"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4093-4361","authenticated-orcid":false,"given":"Maximilian","family":"B\u00f6ther","sequence":"first","affiliation":[{"name":"ETH Zurich, Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3364-2109","authenticated-orcid":false,"given":"Foteini","family":"Strati","sequence":"additional","affiliation":[{"name":"ETH Zurich, Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6750-5500","authenticated-orcid":false,"given":"Viktor","family":"Gsteiger","sequence":"additional","affiliation":[{"name":"ETH Zurich, Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8559-0529","authenticated-orcid":false,"given":"Ana","family":"Klimovic","sequence":"additional","affiliation":[{"name":"ETH Zurich, Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,5,8]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/15825aee15eb335cc13f9b559f166ee8-Abstract.html","author":"Aljundi Rahaf","year":"2019","unstructured":"Rahaf Aljundi , Eugene Belilovsky , Tinne Tuytelaars , Laurent Charlin , Massimo Caccia , Min Lin , and Lucas Page-Caccia . 2019 . Online Continual Learning with Maximal Interfered Retrieval . In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/15825aee15eb335cc13f9b559f166ee8-Abstract.html Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. 2019. Online Continual Learning with Maximal Interfered Retrieval. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/15825aee15eb335cc13f9b559f166ee8-Abstract.html"},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/e562cd9c0768d5464b64cf61da7fc6bb-Abstract.html","author":"Aljundi Rahaf","year":"2019","unstructured":"Rahaf Aljundi , Min Lin , Baptiste Goujaud , and Yoshua Bengio . 2019 . Gradient based sample selection for online continual learning . In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/e562cd9c0768d5464b64cf61da7fc6bb-Abstract.html Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. 2019. Gradient based sample selection for online continual learning. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/e562cd9c0768d5464b64cf61da7fc6bb-Abstract.html"},{"key":"e_1_3_2_1_3_1","unstructured":"Amazon. 2023. Amazon SageMaker. https:\/\/docs.aws.amazon.com\/sagemaker\/index.html.  Amazon. 2023. Amazon SageMaker. https:\/\/docs.aws.amazon.com\/sagemaker\/index.html."},{"key":"e_1_3_2_1_4_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10","author":"Bang Jihwan","year":"2021","unstructured":"Jihwan Bang , Heesu Kim , YoungJoon Yoo , Jung-Woo Ha , and Jonghyun Choi . 2021 . Rainbow Memory: Continual Learning with a Memory of Diverse Samples . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10 .1109\/cvpr46437.2021.00812 Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung-Woo Ha, and Jonghyun Choi. 2021. Rainbow Memory: Continual Learning with a Memory of Diverse Samples. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10.1109\/cvpr46437.2021.00812"},{"key":"e_1_3_2_1_5_1","volume-title":"Proceedings of the International AAAI Conference on Web and Social Media (ICWSM). 10","author":"Baumgartner Jason","year":"2020","unstructured":"Jason Baumgartner , Savvas Zannettou , Brian Keegan , Megan Squire , and Jeremy Blackburn . 2020 . The Pushshift Reddit Dataset . In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM). 10 .48550\/ARXIV.2001.08435 Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The Pushshift Reddit Dataset. In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM). 10.48550\/ARXIV.2001.08435"},{"key":"e_1_3_2_1_6_1","volume-title":"Proceedings of the USENIX Conference on Operational Machine Learning (OpML). https:\/\/www.usenix.org\/conference\/opml19\/presentation\/baylor","author":"Baylor Denis","year":"2019","unstructured":"Denis Baylor , Kevin Haas , Konstantinos Katsiapis , Sammy Leong , Rose Liu , Clemens Mewald , Hui Miao , Neoklis Polyzotis , Mitchell Trott , and Martin Zinkevich . 2019 . Continuous Training for Production ML in the TensorFlow Extended (TFX) Platform . In Proceedings of the USENIX Conference on Operational Machine Learning (OpML). https:\/\/www.usenix.org\/conference\/opml19\/presentation\/baylor Denis Baylor, Kevin Haas, Konstantinos Katsiapis, Sammy Leong, Rose Liu, Clemens Mewald, Hui Miao, Neoklis Polyzotis, Mitchell Trott, and Martin Zinkevich. 2019. Continuous Training for Production ML in the TensorFlow Extended (TFX) Platform. In Proceedings of the USENIX Conference on Operational Machine Learning (OpML). https:\/\/www.usenix.org\/conference\/opml19\/presentation\/baylor"},{"key":"e_1_3_2_1_7_1","unstructured":"BentoML. 2023. BentoML: Github Organization. https:\/\/github.com\/bentoml\/. Accessed: 2023-03-09.  BentoML. 2023. BentoML: Github Organization. https:\/\/github.com\/bentoml\/. Accessed: 2023-03-09."},{"key":"e_1_3_2_1_8_1","volume-title":"Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). https:\/\/www.usenix.org\/conference\/nsdi22\/presentation\/bhardwaj","author":"Bhardwaj Romil","year":"2022","unstructured":"Romil Bhardwaj , Zhengxu Xia , Ganesh Ananthanarayanan , Junchen Jiang , Yuanchao Shu , Nikolaos Karianakis , Kevin Hsieh , Paramvir Bahl , and Ion Stoica . 2022 . Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers . In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). https:\/\/www.usenix.org\/conference\/nsdi22\/presentation\/bhardwaj Romil Bhardwaj, Zhengxu Xia, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, Nikolaos Karianakis, Kevin Hsieh, Paramvir Bahl, and Ion Stoica. 2022. Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers. In Proceedings of the USENIX Symposium on Networked Systems Design and Implementation (NSDI). https:\/\/www.usenix.org\/conference\/nsdi22\/presentation\/bhardwaj"},{"key":"e_1_3_2_1_9_1","unstructured":"Lukas Biewald. 2020. Experiment Tracking with Weights and Biases. https:\/\/www.wandb.com\/  Lukas Biewald. 2020. Experiment Tracking with Weights and Biases. https:\/\/www.wandb.com\/"},{"key":"e_1_3_2_1_10_1","volume-title":"Machine Unlearning. In Proceedings of the IEEE Symposium on Security and Privacy (S&P). 10","author":"Bourtoule Lucas","year":"2021","unstructured":"Lucas Bourtoule , Varun Chandrasekaran , Christopher A. Choquette-Choo , Hengrui Jia , Adelin Travers , Baiwu Zhang , David Lie , and Nicolas Papernot . 2021 . Machine Unlearning. In Proceedings of the IEEE Symposium on Security and Privacy (S&P). 10 .1109\/sp40001.2021.00019 Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Machine Unlearning. In Proceedings of the IEEE Symposium on Security and Privacy (S&P). 10.1109\/sp40001.2021.00019"},{"key":"e_1_3_2_1_11_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10","author":"Caesar Holger","year":"2020","unstructured":"Holger Caesar , Varun Bankiti , Alex H. Lang , Sourabh Vora , Venice Erin Liong , Qiang Xu , Anush Krishnan , Yu Pan , Giancarlo Baldan , and Oscar Beijbom . 2020 . nuScenes: A Multimodal Dataset for Autonomous Driving . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10 .1109\/cvpr42600.2020.01164 Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. 2020. nuScenes: A Multimodal Dataset for Autonomous Driving. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10.1109\/cvpr42600.2020.01164"},{"key":"e_1_3_2_1_12_1","volume-title":"Apache Flink\u2122: Stream and Batch Processing in a Single Engine. Bulletin of the Technical Committee on Data Engineering 38, 4","author":"Carbone Paris","year":"2015","unstructured":"Paris Carbone , Asterios Katsifodimos , Stephan Ewen , Volker Markl , Seif Haridi , and Kostas Tzoumas . 2015 . Apache Flink\u2122: Stream and Batch Processing in a Single Engine. Bulletin of the Technical Committee on Data Engineering 38, 4 (2015). Paris Carbone, Asterios Katsifodimos, Stephan Ewen, Volker Markl, Seif Haridi, and Kostas Tzoumas. 2015. Apache Flink\u2122: Stream and Batch Processing in a Single Engine. Bulletin of the Technical Committee on Data Engineering 38, 4 (2015)."},{"key":"e_1_3_2_1_13_1","volume-title":"Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2000\/hash\/155fa09596c7e18e50b58eb7e0c6ccb4-Abstract.html","author":"Cauwenberghs Gert","unstructured":"Gert Cauwenberghs and Tomaso A. Poggio . 2000. Incremental and Decremental Support Vector Machine Learning . In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2000\/hash\/155fa09596c7e18e50b58eb7e0c6ccb4-Abstract.html Gert Cauwenberghs and Tomaso A. Poggio. 2000. Incremental and Decremental Support Vector Machine Learning. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2000\/hash\/155fa09596c7e18e50b58eb7e0c6ccb4-Abstract.html"},{"key":"e_1_3_2_1_14_1","volume-title":"Proceedings of the International Workshop on Data Management for End-to-End Machine Learning (DEEM). 10","author":"Chen Andrew","year":"2020","unstructured":"Andrew Chen , Andy Chow , Aaron Davidson , Arjun D Cunha , Ali Ghodsi , Sue Ann Hong , Andy Konwinski , Clemens Mewald , Siddharth Murching , Tomas Nykodym , Paul Ogilvie , Mani Parkhe , Avesh Singh , Fen Xie , Matei Zaharia , Richard Zang , Juntai Zheng , and Corey Zumar . 2020 . Developments in MLflow: A System to Accelerate the Machine Learning Lifecycle . In Proceedings of the International Workshop on Data Management for End-to-End Machine Learning (DEEM). 10 .1145\/3399579.3399867 Andrew Chen, Andy Chow, Aaron Davidson, Arjun DCunha, Ali Ghodsi, Sue Ann Hong, Andy Konwinski, Clemens Mewald, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, Mani Parkhe, Avesh Singh, Fen Xie, Matei Zaharia, Richard Zang, Juntai Zheng, and Corey Zumar. 2020. Developments in MLflow: A System to Accelerate the Machine Learning Lifecycle. In Proceedings of the International Workshop on Data Management for End-to-End Machine Learning (DEEM). 10.1145\/3399579.3399867"},{"key":"e_1_3_2_1_15_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. 10","author":"Cordts Marius","year":"2016","unstructured":"Marius Cordts , Mohamed Omran , Sebastian Ramos , Timo Rehfeld , Markus Enzweiler , Rodrigo Benenson , Uwe Franke , Stefan Roth , and Bernt Schiele . 2016 . The Cityscapes Dataset for Semantic Urban Scene Understanding . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. 10 .1109\/cvpr.2016.350 Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The Cityscapes Dataset for Semantic Urban Scene Understanding. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE. 10.1109\/cvpr.2016.350"},{"key":"e_1_3_2_1_16_1","unstructured":"Criteo. 2013. Download Terabyte Click Logs. https:\/\/labs.criteo.com\/2013\/12\/download-terabyte-click-logs\/.  Criteo. 2013. Download Terabyte Click Logs. https:\/\/labs.criteo.com\/2013\/12\/download-terabyte-click-logs\/."},{"key":"e_1_3_2_1_17_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Deng Jia","year":"2009","unstructured":"Jia Deng , Wei Dong , Richard Socher , Li-Jia Li , Kai Li , and Li Fei-Fei . 2009 . ImageNet: A large-scale hierarchical image database . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_3_2_1_18_1","first-page":"19","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). 10","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding . In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). 10 .18653\/v1\/n 19 - 1423 Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). 10.18653\/v1\/n19-1423"},{"key":"e_1_3_2_1_19_1","volume-title":"Proceedings of the Workshop on Continual Learning at NeurIPS. 10","author":"Diethe Tom","year":"2018","unstructured":"Tom Diethe , Tom Borchert , Eno Thereska , Borja Balle , and Neil Lawrence . 2018 . Continual Learning in Practice . In Proceedings of the Workshop on Continual Learning at NeurIPS. 10 .48550\/ARXIV.1903.05202 Tom Diethe, Tom Borchert, Eno Thereska, Borja Balle, and Neil Lawrence. 2018. Continual Learning in Practice. In Proceedings of the Workshop on Continual Learning at NeurIPS. 10.48550\/ARXIV.1903.05202"},{"key":"e_1_3_2_1_20_1","volume-title":"Proceedings of the Conference on Recommender Systems (RecSys). 10","author":"Egg Alex","year":"2021","unstructured":"Alex Egg . 2021 . Online Learning for Recommendations at Grubhub . In Proceedings of the Conference on Recommender Systems (RecSys). 10 .1145\/3460231.3474599 Alex Egg. 2021. Online Learning for Recommendations at Grubhub. In Proceedings of the Conference on Recommender Systems (RecSys). 10.1145\/3460231.3474599"},{"key":"e_1_3_2_1_21_1","unstructured":"European Union. 2016. Art. 17 GDPR: Right to erasure ('right to be forgotten'). https:\/\/gdpr.eu\/article-17-right-to-be-forgotten\/.  European Union. 2016. Art. 17 GDPR: Right to erasure ('right to be forgotten'). https:\/\/gdpr.eu\/article-17-right-to-be-forgotten\/."},{"key":"e_1_3_2_1_22_1","volume-title":"Presentations of the USENIX Conference on Operational Machine Learning (OpML). https:\/\/www.usenix.org\/conference\/opml20\/presentation\/farabet","author":"Farabet Clement","year":"2020","unstructured":"Clement Farabet and Nicolas Koumchatzky . 2020 . Presentation: Inside NVIDIA's AI Infrastructure for Self-driving Cars . In Presentations of the USENIX Conference on Operational Machine Learning (OpML). https:\/\/www.usenix.org\/conference\/opml20\/presentation\/farabet Clement Farabet and Nicolas Koumchatzky. 2020. Presentation: Inside NVIDIA's AI Infrastructure for Self-driving Cars. In Presentations of the USENIX Conference on Operational Machine Learning (OpML). https:\/\/www.usenix.org\/conference\/opml20\/presentation\/farabet"},{"key":"e_1_3_2_1_23_1","volume-title":"Feast: Feature Store For Machine Learning. https:\/\/feast.dev\/.","author":"Authors Feast","year":"2023","unstructured":"Feast Authors . 2023 . Feast: Feature Store For Machine Learning. https:\/\/feast.dev\/. Feast Authors. 2023. Feast: Feature Store For Machine Learning. https:\/\/feast.dev\/."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2523813"},{"key":"e_1_3_2_1_25_1","volume-title":"Proceedings of the IEEE International Symposium on High Performance Computer Architecture (HPCA). 10","author":"Hazelwood Kim","year":"2018","unstructured":"Kim Hazelwood , Sarah Bird , David Brooks , Soumith Chintala , Utku Diril , Dmytro Dzhulgakov , Mohamed Fawzy , Bill Jia , Yangqing Jia , Aditya Kalro , James Law , Kevin Lee , Jason Lu , Pieter Noordhuis , Misha Smelyanskiy , Liang Xiong , and Xiaodong Wang . 2018 . Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective . In Proceedings of the IEEE International Symposium on High Performance Computer Architecture (HPCA). 10 .1109\/HPCA.2018.00059 Kim Hazelwood, Sarah Bird, David Brooks, Soumith Chintala, Utku Diril, Dmytro Dzhulgakov, Mohamed Fawzy, Bill Jia, Yangqing Jia, Aditya Kalro, James Law, Kevin Lee, Jason Lu, Pieter Noordhuis, Misha Smelyanskiy, Liang Xiong, and Xiaodong Wang. 2018. Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective. In Proceedings of the IEEE International Symposium on High Performance Computer Architecture (HPCA). 10.1109\/HPCA.2018.00059"},{"key":"e_1_3_2_1_26_1","volume-title":"Proceedings of the International Workshop on Data Mining for Online Advertising (ADKDD). 10","author":"He Xinran","year":"2014","unstructured":"Xinran He , Junfeng Pan , Ou Jin , Tianbing Xu , Bo Liu , Tao Xu , Yanxin Shi , Antoine Atallah , Ralf Herbrich , Stuart Bowers , and Joaquin Qui\u00f1onero Candela . 2014 . Practical Lessons from Predicting Clicks on Ads at Facebook . In Proceedings of the International Workshop on Data Mining for Online Advertising (ADKDD). 10 .1145\/2648584.2648589 Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, and Joaquin Qui\u00f1onero Candela. 2014. Practical Lessons from Predicting Clicks on Ads at Facebook. In Proceedings of the International Workshop on Data Mining for Online Advertising (ADKDD). 10.1145\/2648584.2648589"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1002\/qj.3803"},{"key":"e_1_3_2_1_28_1","unstructured":"Chip Huyen. 2020. Machine learning is going real-time. https:\/\/huyenchip.com\/2020\/12\/27\/real-time-machine-learning.html.  Chip Huyen. 2020. Machine learning is going real-time. https:\/\/huyenchip.com\/2020\/12\/27\/real-time-machine-learning.html."},{"key":"e_1_3_2_1_29_1","volume-title":"Designing Machine Learning Systems","author":"Huyen Chip","unstructured":"Chip Huyen . 2022. Designing Machine Learning Systems . O'Reilly Media, Inc. Chip Huyen. 2022. Designing Machine Learning Systems. O'Reilly Media, Inc."},{"key":"e_1_3_2_1_30_1","unstructured":"Chip Huyen. 2022. Real-time machine learning: challenges and solutions. https:\/\/huyenchip.com\/2022\/01\/02\/real-time-machine-learning-challenges-and-solutions.html.  Chip Huyen. 2022. Real-time machine learning: challenges and solutions. https:\/\/huyenchip.com\/2022\/01\/02\/real-time-machine-learning-challenges-and-solutions.html."},{"key":"e_1_3_2_1_31_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10","author":"Jia Ruoxi","year":"2021","unstructured":"Ruoxi Jia , Fan Wu , Xuehui Sun , Jiacen Xu , David Dao , Bhavya Kailkhura , Ce Zhang , Bo Li , and Dawn Song . 2021 . Scalability vs. Utility: Do We Have to Sacrifice One for the Other in Data Importance Quantification? . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10 .1109\/cvpr46437.2021.00814 Ruoxi Jia, Fan Wu, Xuehui Sun, Jiacen Xu, David Dao, Bhavya Kailkhura, Ce Zhang, Bo Li, and Dawn Song. 2021. Scalability vs. Utility: Do We Have to Sacrifice One for the Other in Data Importance Quantification?. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10.1109\/cvpr46437.2021.00814"},{"key":"e_1_3_2_1_32_1","volume-title":"Proceedings of Machine Learning and Systems (MLSys). https:\/\/proceedings.mlsys.org\/book\/319","author":"Kang Daniel","year":"2020","unstructured":"Daniel Kang , Deepti Raghavan , Peter Bailis , and Matei Zaharia . 2020 . Model Assertions for Monitoring and Improving ML Models . In Proceedings of Machine Learning and Systems (MLSys). https:\/\/proceedings.mlsys.org\/book\/319 .pdf Daniel Kang, Deepti Raghavan, Peter Bailis, and Matei Zaharia. 2020. Model Assertions for Monitoring and Improving ML Models. In Proceedings of Machine Learning and Systems (MLSys). https:\/\/proceedings.mlsys.org\/book\/319.pdf"},{"key":"e_1_3_2_1_33_1","volume-title":"Proceedings of the International Conference on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v80\/katharopoulos18a.html","author":"Katharopoulos Angelos","year":"2018","unstructured":"Angelos Katharopoulos and Fran\u00e7ois Fleuret . 2018 . Not All Samples Are Created Equal: Deep Learning with Importance Sampling . In Proceedings of the International Conference on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v80\/katharopoulos18a.html Angelos Katharopoulos and Fran\u00e7ois Fleuret. 2018. Not All Samples Are Created Equal: Deep Learning with Importance Sampling. In Proceedings of the International Conference on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v80\/katharopoulos18a.html"},{"key":"e_1_3_2_1_34_1","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR). https:\/\/openreview.net\/forum?id=nrGGfMbY_qK","author":"Koh Hyunseo","year":"2022","unstructured":"Hyunseo Koh , Dahyun Kim , Jung-Woo Ha , and Jonghyun Choi . 2022 . Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference . In Proceedings of the International Conference on Learning Representations (ICLR). https:\/\/openreview.net\/forum?id=nrGGfMbY_qK Hyunseo Koh, Dahyun Kim, Jung-Woo Ha, and Jonghyun Choi. 2022. Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference. In Proceedings of the International Conference on Learning Representations (ICLR). https:\/\/openreview.net\/forum?id=nrGGfMbY_qK"},{"key":"e_1_3_2_1_35_1","volume-title":"International Conference on Machine Learning (ICML). https:\/\/proceedings.mlr.press\/v139\/koh21a.html","author":"Koh Pang Wei","year":"2021","unstructured":"Pang Wei Koh , Shiori Sagawa , Henrik Marklund , Sang Michael Xie , Marvin Zhang , Akshay Balsubramani , Weihua Hu , Michihiro Yasunaga , Richard Lanas Phillips , Irena Gao , Tony Lee , Etienne David , Ian Stavness , Wei Guo , Berton A. Earnshaw , Imran S. Haque , Sara Beery , Jure Leskovec , Anshul Kundaje , Emma Pierson , Sergey Levine , Chelsea Finn , and Percy Liang . 2021 . WILDS: A Benchmark of in-the-Wild Distribution Shifts . In International Conference on Machine Learning (ICML). https:\/\/proceedings.mlr.press\/v139\/koh21a.html Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton A. Earnshaw, Imran S. Haque, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. 2021. WILDS: A Benchmark of in-the-Wild Distribution Shifts. In International Conference on Machine Learning (ICML). https:\/\/proceedings.mlr.press\/v139\/koh21a.html"},{"key":"e_1_3_2_1_36_1","unstructured":"Akinwande Komolafe. 2023. Retraining Model During Deployment: Continuous Training and Continuous Testing. https:\/\/neptune.ai\/blog\/retraining-model-during-deployment-continuous-training-continuous-testing.  Akinwande Komolafe. 2023. Retraining Model During Deployment: Continuous Training and Continuous Testing. https:\/\/neptune.ai\/blog\/retraining-model-during-deployment-continuous-training-continuous-testing."},{"key":"e_1_3_2_1_38_1","volume-title":"Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2021\/hash\/f5bf0ba0a17ef18f9607774722f5698c-Abstract.html","author":"Lazaridou Angeliki","year":"2021","unstructured":"Angeliki Lazaridou , Adhiguna Kuncoro , Elena Gribovskaya , Devang Agrawal , Adam Liska , Tayfun Terzi , Mai Gimenez , Cyprien de Masson d' Autume , Tom\u00e1s Kocisk\u00fd , Sebastian Ruder , Dani Yogatama , Kris Cao , Susannah Young , and Phil Blunsom . 2021 . Mind the Gap: Assessing Temporal Generalization in Neural Language Models . In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2021\/hash\/f5bf0ba0a17ef18f9607774722f5698c-Abstract.html Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Tom\u00e1s Kocisk\u00fd, Sebastian Ruder, Dani Yogatama, Kris Cao, Susannah Young, and Phil Blunsom. 2021. Mind the Gap: Assessing Temporal Generalization in Neural Language Models. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2021\/hash\/f5bf0ba0a17ef18f9607774722f5698c-Abstract.html"},{"key":"e_1_3_2_1_39_1","volume-title":"Proc. IEEE 86","author":"LeCun Yann","year":"1998","unstructured":"Yann LeCun , L\u00e9on Bottou , Yoshua Bengio , and Patrick Haffner . 1998 . Gradient-based learning applied to document recognition . Proc. IEEE 86 , 11 (1998). 10.1109\/5.726791 Yann LeCun, L\u00e9on Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998). 10.1109\/5.726791"},{"key":"e_1_3_2_1_40_1","volume-title":"Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2021\/hash\/362387494f6be6613daea643a7706a42-Abstract.html","author":"Li Aodong","year":"2021","unstructured":"Aodong Li , Alex Boyd , Padhraic Smyth , and Stephan Mandt . 2021 . Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning . In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2021\/hash\/362387494f6be6613daea643a7706a42-Abstract.html Aodong Li, Alex Boyd, Padhraic Smyth, and Stephan Mandt. 2021. Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2021\/hash\/362387494f6be6613daea643a7706a42-Abstract.html"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"crossref","unstructured":"Vincenzo Lomonaco Lorenzo Pellegrini Andrea Cossu Antonio Carta Gabriele Graffieti Tyler L. Hayes Matthias De Lange Marc Masana Jary Pomponi Gido M. van de Ven Martin Mundt Qi She Keiland Cooper Jeremy Forest Eden Belouadah Simone Calderara German Ignacio Parisi Fabio Cuzzolin Andreas S. Tolias Simone Scardapane Luca Antiga Subutai Ahmad Adrian Popescu Christopher Kanan Joost van de Weijer Tinne Tuytelaars Davide Bacciu and Davide Maltoni. 2021. Avalanche: An End-to-End Library for Continual Learning. In Workshop Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops (CVPR). 10.1109\/CVPRW53098.2021.00399  Vincenzo Lomonaco Lorenzo Pellegrini Andrea Cossu Antonio Carta Gabriele Graffieti Tyler L. Hayes Matthias De Lange Marc Masana Jary Pomponi Gido M. van de Ven Martin Mundt Qi She Keiland Cooper Jeremy Forest Eden Belouadah Simone Calderara German Ignacio Parisi Fabio Cuzzolin Andreas S. Tolias Simone Scardapane Luca Antiga Subutai Ahmad Adrian Popescu Christopher Kanan Joost van de Weijer Tinne Tuytelaars Davide Bacciu and Davide Maltoni. 2021. Avalanche: An End-to-End Library for Continual Learning. In Workshop Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops (CVPR). 10.1109\/CVPRW53098.2021.00399","DOI":"10.1109\/CVPRW53098.2021.00399"},{"key":"e_1_3_2_1_42_1","volume-title":"Proceedings of Advances in Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/f87522788a2be2d171666752f97ddebb-Abstract.html","author":"Lopez-Paz David","year":"2017","unstructured":"David Lopez-Paz and Marc'Aurelio Ranzato . 2017 . Gradient Episodic Memory for Continual Learning . In Proceedings of Advances in Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/f87522788a2be2d171666752f97ddebb-Abstract.html David Lopez-Paz and Marc'Aurelio Ranzato. 2017. Gradient Episodic Memory for Continual Learning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/f87522788a2be2d171666752f97ddebb-Abstract.html"},{"key":"e_1_3_2_1_43_1","unstructured":"Andrey Malinin Neil Band Yarin Gal Mark J. F. Gales Alexander Ganshin German Chesnokov Alexey Noskov Andrey Ploskonosov Liudmila Prokhorenkova Ivan Provilkov Vatsal Raina Vyas Raina Denis Roginskiy Mariya Shmatova Panagiotis Tigas and Boris Yangel. 2021. Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS) (Benchmark Track) Joaquin Vanschoren and Sai-Kit Yeung (Eds.). https:\/\/datasets-benchmarks-proceedings.neurips.cc\/paper\/2021\/hash\/ad61ab143223efbc24c7d2583be69251-Abstract-round2.html  Andrey Malinin Neil Band Yarin Gal Mark J. F. Gales Alexander Ganshin German Chesnokov Alexey Noskov Andrey Ploskonosov Liudmila Prokhorenkova Ivan Provilkov Vatsal Raina Vyas Raina Denis Roginskiy Mariya Shmatova Panagiotis Tigas and Boris Yangel. 2021. Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS) (Benchmark Track) Joaquin Vanschoren and Sai-Kit Yeung (Eds.). https:\/\/datasets-benchmarks-proceedings.neurips.cc\/paper\/2021\/hash\/ad61ab143223efbc24c7d2583be69251-Abstract-round2.html"},{"key":"e_1_3_2_1_44_1","volume-title":"Proceedings of Machine Learning and Systems (MLSys). https:\/\/proceedings.mlsys.org\/paper\/2022\/hash\/1c383cd30b7c298ab50293adfecb7b18-Abstract.html","author":"Mallick Ankur","year":"2022","unstructured":"Ankur Mallick , Kevin Hsieh , Behnaz Arzani , and Gauri Joshi . 2022 . Matchmaker: Data Drift Mitigation in Machine Learning for Large-Scale Systems . In Proceedings of Machine Learning and Systems (MLSys). https:\/\/proceedings.mlsys.org\/paper\/2022\/hash\/1c383cd30b7c298ab50293adfecb7b18-Abstract.html Ankur Mallick, Kevin Hsieh, Behnaz Arzani, and Gauri Joshi. 2022. Matchmaker: Data Drift Mitigation in Machine Learning for Large-Scale Systems. In Proceedings of Machine Learning and Systems (MLSys). https:\/\/proceedings.mlsys.org\/paper\/2022\/hash\/1c383cd30b7c298ab50293adfecb7b18-Abstract.html"},{"key":"e_1_3_2_1_45_1","volume-title":"Proceedings of the International Conference on Machine Learning (ICML). https:\/\/proceedings.mlr.press\/v162\/mindermann22a.html","author":"Mindermann S\u00f6ren","year":"2022","unstructured":"S\u00f6ren Mindermann , Jan Markus Brauner , Muhammed Razzak , Mrinank Sharma , Andreas Kirsch , Winnie Xu , Benedikt H\u00f6ltgen , Aidan N. Gomez , Adrien Morisot , Sebastian Farquhar , and Yarin Gal . 2022 . Prioritized Training on Points that are Learnable, Worth Learning, and not yet Learnt . In Proceedings of the International Conference on Machine Learning (ICML). https:\/\/proceedings.mlr.press\/v162\/mindermann22a.html S\u00f6ren Mindermann, Jan Markus Brauner, Muhammed Razzak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt H\u00f6ltgen, Aidan N. Gomez, Adrien Morisot, Sebastian Farquhar, and Yarin Gal. 2022. Prioritized Training on Points that are Learnable, Worth Learning, and not yet Learnt. In Proceedings of the International Conference on Machine Learning (ICML). https:\/\/proceedings.mlr.press\/v162\/mindermann22a.html"},{"key":"e_1_3_2_1_46_1","volume-title":"Proceedings of the International Conference on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v119\/mirzasoleiman20a.html","author":"Mirzasoleiman Baharan","year":"2020","unstructured":"Baharan Mirzasoleiman , Jeff A. Bilmes , and Jure Leskovec . 2020 . Core-sets for Data-efficient Training of Machine Learning Models . In Proceedings of the International Conference on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v119\/mirzasoleiman20a.html Baharan Mirzasoleiman, Jeff A. Bilmes, and Jure Leskovec. 2020. Core-sets for Data-efficient Training of Machine Learning Models. In Proceedings of the International Conference on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v119\/mirzasoleiman20a.html"},{"key":"e_1_3_2_1_47_1","volume-title":"Proceedings of the International Conference on Knowledge Discovery and Data Mining (KDD). 10","author":"Modi Akshay Naresh","year":"2017","unstructured":"Akshay Naresh Modi , Chiu Yuen Koo , Chuan Yu Foo , Clemens Mewald , Denis M. Baylor , Eric Breck , Heng-Tze Cheng , Jarek Wilkiewicz , Levent Koc , Lukasz Lew , Martin A. Zinkevich , Martin Wicke , Mustafa Ispir , Neoklis Polyzotis , Noah Fiedel , Salem Elie Haykal , Steven Whang , Sudip Roy , Sukriti Ramesh , Vihan Jain , Xin Zhang , and Zakaria Haque . 2017 . TFX: A TensorFlow-Based Production-Scale Machine Learning Platform . In Proceedings of the International Conference on Knowledge Discovery and Data Mining (KDD). 10 .1145\/3097983.3098021 Akshay Naresh Modi, Chiu Yuen Koo, Chuan Yu Foo, Clemens Mewald, Denis M. Baylor, Eric Breck, Heng-Tze Cheng, Jarek Wilkiewicz, Levent Koc, Lukasz Lew, Martin A. Zinkevich, Martin Wicke, Mustafa Ispir, Neoklis Polyzotis, Noah Fiedel, Salem Elie Haykal, Steven Whang, Sudip Roy, Sukriti Ramesh, Vihan Jain, Xin Zhang, and Zakaria Haque. 2017. TFX: A TensorFlow-Based Production-Scale Machine Learning Platform. In Proceedings of the International Conference on Knowledge Discovery and Data Mining (KDD). 10.1145\/3097983.3098021"},{"key":"e_1_3_2_1_48_1","unstructured":"Maxim Naumov Dheevatsa Mudigere Hao-Jun Michael Shi Jianyu Huang Narayanan Sundaraman Jongsoo Park Xiaodong Wang Udit Gupta Carole-Jean Wu Alisson G. Azzolini Dmytro Dzhulgakov Andrey Mallevich Ilia Cherniavskii Yinghai Lu Raghuraman Krishnamoorthi Ansha Yu Volodymyr Kondratenko Stephanie Pereira Xianjie Chen Wenlin Chen Vijay Rao Bill Jia Liang Xiong and Misha Smelyanskiy. 2019. Deep Learning Recommendation Model for Personalization and Recommendation Systems. (2019). 10.48550\/ARXIV.1906.00091  Maxim Naumov Dheevatsa Mudigere Hao-Jun Michael Shi Jianyu Huang Narayanan Sundaraman Jongsoo Park Xiaodong Wang Udit Gupta Carole-Jean Wu Alisson G. Azzolini Dmytro Dzhulgakov Andrey Mallevich Ilia Cherniavskii Yinghai Lu Raghuraman Krishnamoorthi Ansha Yu Volodymyr Kondratenko Stephanie Pereira Xianjie Chen Wenlin Chen Vijay Rao Bill Jia Liang Xiong and Misha Smelyanskiy. 2019. Deep Learning Recommendation Model for Personalization and Recommendation Systems. (2019). 10.48550\/ARXIV.1906.00091"},{"key":"e_1_3_2_1_49_1","unstructured":"Neptune. 2023. Neptune.ai ML Metadata Store. https:\/\/neptune.ai\/.  Neptune. 2023. Neptune.ai ML Metadata Store. https:\/\/neptune.ai\/."},{"key":"e_1_3_2_1_50_1","unstructured":"NVIDIA. 2023. NVIDIA DLRM Example Implementation. https:\/\/github.com\/NVIDIA\/DeepLearningExamples\/tree\/master\/PyTorch\/Recommendation\/DLRM. Accessed: 2023-03-08.  NVIDIA. 2023. NVIDIA DLRM Example Implementation. https:\/\/github.com\/NVIDIA\/DeepLearningExamples\/tree\/master\/PyTorch\/Recommendation\/DLRM. Accessed: 2023-03-08."},{"key":"e_1_3_2_1_51_1","unstructured":"NVIDIA. 2023. NVIDIA Triton Inference Server. https:\/\/developer.nvidia.com\/nvidia-triton-inference-server. Accessed: 2023-03-09.  NVIDIA. 2023. NVIDIA Triton Inference Server. https:\/\/developer.nvidia.com\/nvidia-triton-inference-server. Accessed: 2023-03-09."},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"crossref","unstructured":"Andrei Paleyes Raoul-Gabriel Urma and Neil D. Lawrence. 2022. Challenges in Deploying Machine Learning: A Survey of Case Studies. Comput. Surveys 55 6 (2022). 10.1145\/3533378  Andrei Paleyes Raoul-Gabriel Urma and Neil D. Lawrence. 2022. Challenges in Deploying Machine Learning: A Survey of Case Studies. Comput. Surveys 55 6 (2022). 10.1145\/3533378","DOI":"10.1145\/3533378"},{"key":"e_1_3_2_1_53_1","unstructured":"Jaideep Pathak Shashank Subramanian Peter Harrington Sanjeev Raja Ashesh Chattopadhyay Morteza Mardani Thorsten Kurth David Hall Zongyi Li Kamyar Azizzadenesheli Pedram Hassanzadeh Karthik Kashinath and Animashree Anandkumar. 2022. FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators. 10.48550\/ARXIV.2202.11214  Jaideep Pathak Shashank Subramanian Peter Harrington Sanjeev Raja Ashesh Chattopadhyay Morteza Mardani Thorsten Kurth David Hall Zongyi Li Kamyar Azizzadenesheli Pedram Hassanzadeh Karthik Kashinath and Animashree Anandkumar. 2022. FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators. 10.48550\/ARXIV.2202.11214"},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/5326.983933"},{"key":"e_1_3_2_1_55_1","volume-title":"Proceedings of the International Conference on Machine Learning (ICML). https:\/\/proceedings.mlr.press\/v162\/pooladzandi22a.html","author":"Pooladzandi Omead","year":"2022","unstructured":"Omead Pooladzandi , David Davini , and Baharan Mirzasoleiman . 2022 . Adaptive Second Order Coresets for Data-efficient Machine Learning . In Proceedings of the International Conference on Machine Learning (ICML). https:\/\/proceedings.mlr.press\/v162\/pooladzandi22a.html Omead Pooladzandi, David Davini, and Baharan Mirzasoleiman. 2022. Adaptive Second Order Coresets for Data-efficient Machine Learning. In Proceedings of the International Conference on Machine Learning (ICML). https:\/\/proceedings.mlr.press\/v162\/pooladzandi22a.html"},{"key":"e_1_3_2_1_56_1","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV). 10","author":"Prabhu Ameya","unstructured":"Ameya Prabhu , Philip H. S. Torr , and Puneet K. Dokania . 2020. GDumb: A Simple Approach that Questions Our Progress in Continual Learning . In Proceedings of the European Conference on Computer Vision (ECCV). 10 .1007\/978-3-030-58536-5_31 Ameya Prabhu, Philip H. S. Torr, and Puneet K. Dokania. 2020. GDumb: A Simple Approach that Questions Our Progress in Continual Learning. In Proceedings of the European Conference on Computer Vision (ECCV). 10.1007\/978-3-030-58536-5_31"},{"key":"e_1_3_2_1_57_1","unstructured":"PyTorch Serve Contributors. 2020. TorchServe: Docs. https:\/\/pytorch.org\/serve\/. Accessed: 2023-03-09.  PyTorch Serve Contributors. 2020. TorchServe: Docs. https:\/\/pytorch.org\/serve\/. Accessed: 2023-03-09."},{"key":"e_1_3_2_1_58_1","volume-title":"Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/846c260d715e5b854ffad5f70a516c88-Abstract.html","author":"Rabanser Stephan","unstructured":"Stephan Rabanser , Stephan G\u00fcnnemann , and Zachary C. Lipton . 2019. Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift . In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/846c260d715e5b854ffad5f70a516c88-Abstract.html Stephan Rabanser, Stephan G\u00fcnnemann, and Zachary C. Lipton. 2019. Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/846c260d715e5b854ffad5f70a516c88-Abstract.html"},{"key":"e_1_3_2_1_59_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10","author":"Redmon Joseph","year":"2016","unstructured":"Joseph Redmon , Santosh Divvala , Ross Girshick , and Ali Farhadi . 2016 . You Only Look Once: Unified, Real-Time Object Detection . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10 .1109\/cvpr.2016.91 Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10.1109\/cvpr.2016.91"},{"key":"e_1_3_2_1_60_1","volume-title":"On Challenges in Machine Learning Model Management","author":"Schelter Sebastian","year":"2018","unstructured":"Sebastian Schelter , Felix Bie\u00dfmann , Tim Januschowski , David Salinas , Stephan Seufert , and Gyuri Szarvas . 2018. On Challenges in Machine Learning Model Management . IEEE Data Engineering Bulletin 41, 4 ( 2018 ). http:\/\/sites.computer.org\/debull\/A18dec\/p5.pdf Sebastian Schelter, Felix Bie\u00dfmann, Tim Januschowski, David Salinas, Stephan Seufert, and Gyuri Szarvas. 2018. On Challenges in Machine Learning Model Management. IEEE Data Engineering Bulletin 41, 4 (2018). http:\/\/sites.computer.org\/debull\/A18dec\/p5.pdf"},{"key":"e_1_3_2_1_61_1","volume-title":"Proceedings of the ML Evaluation Standards Workshop at ICLR. 10","author":"Shankar Shreya","unstructured":"Shreya Shankar , Bernease Herman , and Aditya G. Parameswaran . 2022. Rethinking Streaming Machine Learning Evaluation . In Proceedings of the ML Evaluation Standards Workshop at ICLR. 10 .48550\/arXiv.2205.11473 Shreya Shankar, Bernease Herman, and Aditya G. Parameswaran. 2022. Rethinking Streaming Machine Learning Evaluation. In Proceedings of the ML Evaluation Standards Workshop at ICLR. 10.48550\/arXiv.2205.11473"},{"key":"e_1_3_2_1_62_1","volume-title":"Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). https:\/\/www.usenix.org\/conference\/osdi22\/presentation\/sima","author":"Sima Chijun","year":"2022","unstructured":"Chijun Sima , Yao Fu , Man-Kit Sit , Liyi Guo , Xuri Gong , Feng Lin , Junyu Wu , Yongsheng Li , Haidong Rong , Pierre-Louis Aublin , and Luo Mai . 2022 . Ekko: A Large-Scale Deep Learning Recommender System with Low-Latency Model Update . In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). https:\/\/www.usenix.org\/conference\/osdi22\/presentation\/sima Chijun Sima, Yao Fu, Man-Kit Sit, Liyi Guo, Xuri Gong, Feng Lin, Junyu Wu, Yongsheng Li, Haidong Rong, Pierre-Louis Aublin, and Luo Mai. 2022. Ekko: A Large-Scale Deep Learning Recommender System with Low-Latency Model Update. In Proceedings of the USENIX Symposium on Operating Systems Design and Implementation (OSDI). https:\/\/www.usenix.org\/conference\/osdi22\/presentation\/sima"},{"key":"e_1_3_2_1_63_1","unstructured":"State of California USA. 2018. Section 1798.130 CCPA. https:\/\/ccpa-info.com\/california-consumer-privacy-act-full-text\/.  State of California USA. 2018. Section 1798.130 CCPA. https:\/\/ccpa-info.com\/california-consumer-privacy-act-full-text\/."},{"key":"e_1_3_2_1_64_1","volume-title":"Proceedings International Conference on Extending Database Technology, (EDBT). 10","author":"Strassenburg Nils","year":"2023","unstructured":"Nils Strassenburg , Dominic Kupfer , Julia Kowal , and Tilmann Rabl . 2023 . Efficient Multi-Model Management . In Proceedings International Conference on Extending Database Technology, (EDBT). 10 .48786\/edbt.2023.37 Nils Strassenburg, Dominic Kupfer, Julia Kowal, and Tilmann Rabl. 2023. Efficient Multi-Model Management. In Proceedings International Conference on Extending Database Technology, (EDBT). 10.48786\/edbt.2023.37"},{"key":"e_1_3_2_1_65_1","unstructured":"Nils Strassenburg Ilin Tolovski and Tilmann Rabl. 2022. Efficiently Managing Deep Learning Models in a Distributed Environment. 10.48786\/EDBT.2022.12  Nils Strassenburg Ilin Tolovski and Tilmann Rabl. 2022. Efficiently Managing Deep Learning Models in a Distributed Environment. 10.48786\/EDBT.2022.12"},{"key":"e_1_3_2_1_66_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10","author":"Sun Pei","year":"2020","unstructured":"Pei Sun , Henrik Kretzschmar , Xerxes Dotiwalla , Aurelien Chouard , Vijaysai Patnaik , Paul Tsui , James Guo , Yin Zhou , Yuning Chai , Benjamin Caine , Vijay Vasudevan , Wei Han , Jiquan Ngiam , Hang Zhao , Aleksei Timofeev , Scott Ettinger , Maxim Krivokon , Amy Gao , Aditya Joshi , Yu Zhang , Jonathon Shlens , Zhifeng Chen , and Dragomir Anguelov . 2020 . Scalability in Perception for Autonomous Driving: Waymo Open Dataset . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10 .1109\/CVPR42600.2020.00252 Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Ettinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhang, Jonathon Shlens, Zhifeng Chen, and Dragomir Anguelov. 2020. Scalability in Perception for Autonomous Driving: Waymo Open Dataset. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10.1109\/CVPR42600.2020.00252"},{"key":"e_1_3_2_1_67_1","volume-title":"Proceedings of the VLDB Endowment 13","author":"Suprem Abhijit","year":"2020","unstructured":"Abhijit Suprem , Joy Arulraj , Calton Pu , and Joao Ferreira . 2020 . ODIN: Automated drift detection and recovery in video analytics . Proceedings of the VLDB Endowment 13 , 12 (2020). 10.14778\/3407790.3407837 Abhijit Suprem, Joy Arulraj, Calton Pu, and Joao Ferreira. 2020. ODIN: Automated drift detection and recovery in video analytics. Proceedings of the VLDB Endowment 13, 12 (2020). 10.14778\/3407790.3407837"},{"key":"e_1_3_2_1_68_1","volume-title":"Proceedings of the International Conference on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v139\/tahmasbi21a.html","author":"Tahmasbi Ashraf","unstructured":"Ashraf Tahmasbi , Ellango Jothimurugesan , Srikanta Tirthapura , and Phillip B. Gibbons . 2021. DriftSurf: Stable-State \/ Reactive-State Learning under Concept Drift . In Proceedings of the International Conference on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v139\/tahmasbi21a.html Ashraf Tahmasbi, Ellango Jothimurugesan, Srikanta Tirthapura, and Phillip B. Gibbons. 2021. DriftSurf: Stable-State \/ Reactive-State Learning under Concept Drift. In Proceedings of the International Conference on Machine Learning (ICML). http:\/\/proceedings.mlr.press\/v139\/tahmasbi21a.html"},{"key":"e_1_3_2_1_69_1","unstructured":"Tesla. 2019. Tesla Autonomy Day. https:\/\/www.youtube.com\/watch?v=Ucp0TTmvqOE&t=6678s.  Tesla. 2019. Tesla Autonomy Day. https:\/\/www.youtube.com\/watch?v=Ucp0TTmvqOE&t=6678s."},{"key":"e_1_3_2_1_70_1","unstructured":"Josh Tobin. 2021. Toward continual learning systems. https:\/\/gantry.io\/blog\/toward-continual-learning-systems\/.  Josh Tobin. 2021. Toward continual learning systems. https:\/\/gantry.io\/blog\/toward-continual-learning-systems\/."},{"key":"e_1_3_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-15245-z"},{"key":"e_1_3_2_1_72_1","unstructured":"Larysa Visengeriyeva Anja Kammer Isabel B\u00e4r Alexander Kniesz and Michael Pl\u00f6d. 2023. MLOps Infrastructure Stack. https:\/\/ml-ops.org\/content\/state-of-mlops.  Larysa Visengeriyeva Anja Kammer Isabel B\u00e4r Alexander Kniesz and Michael Pl\u00f6d. 2023. MLOps Infrastructure Stack. https:\/\/ml-ops.org\/content\/state-of-mlops."},{"key":"e_1_3_2_1_73_1","unstructured":"Steve Wang and Will Cukierski. 2014. The Avazu Click-Through Rate Prediction Dataset. https:\/\/kaggle.com\/competitions\/avazu-ctr-prediction  Steve Wang and Will Cukierski. 2014. The Avazu Click-Through Rate Prediction Dataset. https:\/\/kaggle.com\/competitions\/avazu-ctr-prediction"},{"key":"e_1_3_2_1_74_1","unstructured":"Weights&Biases. 2023. W&B: Dataset Versioning. https:\/\/docs.wandb.ai\/guides\/data-and-model-versioning\/dataset-versioning#25c79f05-174e-4d35-abda-e5c238b8d6d6.  Weights&Biases. 2023. W&B: Dataset Versioning. https:\/\/docs.wandb.ai\/guides\/data-and-model-versioning\/dataset-versioning#25c79f05-174e-4d35-abda-e5c238b8d6d6."},{"key":"e_1_3_2_1_75_1","volume-title":"Proceedings of Advances in Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2022\/hash\/be0a8ecf8b2743a4117557c5eca0fb79-Abstract-Conference.html","author":"Kaixu Ren Pengjie Wang Yufeng Cai","year":"2022","unstructured":"Yufeng Cai Kaixu Ren Pengjie Wang Huimin Yi Yue Song Jing Chen Hongbo Deng Jian Xu Lin Qu Bo Zheng Wenbo Su , Yuanxing Zhang . 2022 . GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation Models . In Proceedings of Advances in Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2022\/hash\/be0a8ecf8b2743a4117557c5eca0fb79-Abstract-Conference.html Yufeng Cai Kaixu Ren Pengjie Wang Huimin Yi Yue Song Jing Chen Hongbo Deng Jian Xu Lin Qu Bo Zheng Wenbo Su, Yuanxing Zhang. 2022. GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation Models. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS). https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2022\/hash\/be0a8ecf8b2743a4117557c5eca0fb79-Abstract-Conference.html"},{"key":"e_1_3_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2022.05.014"},{"key":"e_1_3_2_1_77_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10","author":"Wu Yue","year":"2019","unstructured":"Yue Wu , Yinpeng Chen , Lijuan Wang , Yuancheng Ye , Zicheng Liu , Yandong Guo , and Yun Fu . 2019 . Large Scale Incremental Learning . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10 .1109\/cvpr.2019.00046 Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. 2019. Large Scale Incremental Learning. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10.1109\/cvpr.2019.00046"},{"key":"e_1_3_2_1_78_1","volume-title":"Proceedings of the Conference on Neural Information Processing Systems (NeurIPS) (Benchmark Track). https:\/\/openreview.net\/forum?id=F9ENmZABB0","author":"Yao Huaxiu","year":"2022","unstructured":"Huaxiu Yao , Caroline Choi , Bochuan Cao , Yoonho Lee , Pang Wei Koh , and Chelsea Finn . 2022 . Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time . In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS) (Benchmark Track). https:\/\/openreview.net\/forum?id=F9ENmZABB0 Huaxiu Yao, Caroline Choi, Bochuan Cao, Yoonho Lee, Pang Wei Koh, and Chelsea Finn. 2022. Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS) (Benchmark Track). https:\/\/openreview.net\/forum?id=F9ENmZABB0"},{"key":"e_1_3_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/2934664"},{"key":"e_1_3_2_1_80_1","unstructured":"Giovanni Zappella Martin Wistuba Lukas Balles and Cedric Archambeau. 2022. Automatically retrain neural networks with Renate. https:\/\/aws.amazon.com\/de\/blogs\/machine-learning\/automatically-retrain-neural-networks-with-renate\/.  Giovanni Zappella Martin Wistuba Lukas Balles and Cedric Archambeau. 2022. Automatically retrain neural networks with Renate. https:\/\/aws.amazon.com\/de\/blogs\/machine-learning\/automatically-retrain-neural-networks-with-renate\/."},{"key":"e_1_3_2_1_81_1","volume-title":"Proceedings of the Annual International Symposium on Computer Architecture (ISCA). 10","author":"Zhao Mark","year":"2022","unstructured":"Mark Zhao , Niket Agarwal , Aarti Basant , Bu\u011fra Gedik , Satadru Pan , Mustafa Ozdal , Rakesh Komuravelli , Jerry Pan , Tianshu Bao , Haowei Lu , Sundaram Narayanan , Jack Langman , Kevin Wilfong , Harsha Rastogi , Carole-Jean Wu , Christos Kozyrakis , and Parik Pol . 2022 . Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training . In Proceedings of the Annual International Symposium on Computer Architecture (ISCA). 10 .1145\/3470496.3533044 Mark Zhao, Niket Agarwal, Aarti Basant, Bu\u011fra Gedik, Satadru Pan, Mustafa Ozdal, Rakesh Komuravelli, Jerry Pan, Tianshu Bao, Haowei Lu, Sundaram Narayanan, Jack Langman, Kevin Wilfong, Harsha Rastogi, Carole-Jean Wu, Christos Kozyrakis, and Parik Pol. 2022. Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA). 10.1145\/3470496.3533044"}],"event":{"name":"EuroMLSys '23: 3rd Workshop on Machine Learning and Systems","location":"Rome Italy","acronym":"EuroMLSys '23","sponsor":["SIGOPS ACM Special Interest Group on Operating Systems"]},"container-title":["Proceedings of the 3rd Workshop on Machine Learning and Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3578356.3592585","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:46:51Z","timestamp":1750178811000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3578356.3592585"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,8]]},"references-count":80,"alternative-id":["10.1145\/3578356.3592585","10.1145\/3578356"],"URL":"https:\/\/doi.org\/10.1145\/3578356.3592585","relation":{},"subject":[],"published":{"date-parts":[[2023,5,8]]},"assertion":[{"value":"2023-05-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}