{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T21:18:50Z","timestamp":1786223930439,"version":"build-2736575974"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:p>The industrial machine learning pipeline requires iterating on model features, training and deploying models, and monitoring deployed models at scale. Feature stores were developed to manage and standardize the engineer's workflow in this end-to-end pipeline, focusing on traditional tabular feature data. In recent years, however, model development has shifted towards using self-supervised pretrained embeddings as model features. Managing these embeddings and the downstream systems that use them introduces new challenges with respect to managing embedding training data, measuring embedding quality, and monitoring downstream models that use embeddings. These challenges are largely unaddressed in standard feature stores. Our goal in this tutorial is to introduce the feature store system and discuss the challenges and current solutions to managing these new embedding-centric pipelines.<\/jats:p>","DOI":"10.14778\/3476311.3476402","type":"journal-article","created":{"date-parts":[[2021,10,28]],"date-time":"2021-10-28T22:48:56Z","timestamp":1635461336000},"page":"3178-3181","source":"Crossref","is-referenced-by-count":22,"title":["Managing ML pipelines"],"prefix":"10.14778","volume":"14","author":[{"given":"Laurel","family":"Orr","sequence":"first","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Atindriyo","family":"Sanyal","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiao","family":"Ling","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karan","family":"Goel","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Megan","family":"Leszczynski","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,10,28]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2021. Feast: An Open Source Feature Store for Machine Learning. https:\/\/feast.dev\/.  2021. Feast: An Open Source Feature Store for Machine Learning. https:\/\/feast.dev\/."},{"key":"e_1_2_1_2_1","unstructured":"2021. Feature Store for ML. https:\/\/www.featurestore.org\/.  2021. Feature Store for ML. https:\/\/www.featurestore.org\/."},{"key":"e_1_2_1_3_1","unstructured":"2021. Hopsworks: The Enterprise Feature Store. https:\/\/www.hopsworks.ai\/.  2021. Hopsworks: The Enterprise Feature Store. https:\/\/www.hopsworks.ai\/."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209889.3209894"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2207710"},{"key":"e_1_2_1_6_1","volume-title":"Slice-based learning: A programming model for residual learning in critical data slices. Advances in neural information processing systems 32","author":"Chen Vincent S","year":"2019"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.14778\/3397230.3397235"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/RAISE.2019.00013"},{"key":"e_1_2_1_9_1","volume-title":"Model Patching: Closing the Subgroup Performance Gap with Data Augmentation. arXiv preprint arXiv:2008.06775","author":"Goel Karan","year":"2020"},{"key":"e_1_2_1_10_1","volume-title":"Robustness Gym: Unifying the NLP Evaluation Landscape. arXiv preprint arXiv:2101.04840","author":"Goel Karan","year":"2021"},{"key":"e_1_2_1_11_1","unstructured":"Ben Gomes. 2017. Our latest quality improvements for Search. https:\/\/blog.google\/products\/search\/our-latest-quality-improvements-search\/.  Ben Gomes. 2017. Our latest quality improvements for Search. https:\/\/blog.google\/products\/search\/our-latest-quality-improvements-search\/."},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of coling 2016, the 26th international conference on computational linguistics: Technical papers. 2785--2796","author":"Hellrich Johannes","year":"2016"},{"key":"e_1_2_1_13_1","volume-title":"Meet Michelangelo: Uber's Machine Learning Platform. https:\/\/eng.uber.com\/michelangelo-machine-learning-platform\/.","author":"Hermann Jeremy","year":"2021"},{"key":"e_1_2_1_14_1","unstructured":"Andrej Karpathy. 2020. Software 2.0. https:\/\/medium.com\/@karpathy\/software-2-0-a64152b37c35.  Andrej Karpathy. 2020. Software 2.0. https:\/\/medium.com\/@karpathy\/software-2-0-a64152b37c35."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401075"},{"key":"e_1_2_1_16_1","volume-title":"Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Sara Beery, et al.","author":"Koh Pang Wei","year":"2020"},{"key":"e_1_2_1_17_1","volume-title":"Understanding the downstream instability of word embeddings. arXiv preprint arXiv:2003.04983","author":"Leszczynski Megan","year":"2020"},{"key":"e_1_2_1_18_1","volume-title":"On the downstream performance of compressed word embeddings. Advances in neural information processing systems 32","author":"May Avner","year":"2019"},{"key":"e_1_2_1_19_1","volume-title":"Ludwig: a type-based declarative deep learning toolbox. arXiv preprint arXiv:1909.07930","author":"Molino Piero","year":"2019"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3196926"},{"key":"e_1_2_1_21_1","volume-title":"Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G Azzolini, et al.","author":"Naumov Maxim","year":"2019"},{"key":"e_1_2_1_22_1","volume-title":"Bootleg: Chasing the Tail with Self-Supervised Named Entity Disambiguation. CIDR 2021, 11th Conference on Innovative Data Systems Research","author":"Orr Laurel","year":"2021"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.14778\/3157794.3157797"},{"key":"e_1_2_1_24_1","volume-title":"Overton: A Data System for Monitoring and Improving Machine-Learned Products. In CIDR","author":"Re Christopher","year":"2020"},{"key":"e_1_2_1_25_1","volume-title":"Beyond accuracy: Behavioral testing of NLP models with CheckList. arXiv preprint arXiv:2005.04118","author":"Ribeiro Marco Tulio","year":"2020"},{"key":"e_1_2_1_26_1","unstructured":"Sebastian Schelter Felix Biessmann Tim Januschowski David Salinas Stephan Seufert and Gyuri Szarvas. 2018. On challenges in machine learning model management. (2018).  Sebastian Schelter Felix Biessmann Tim Januschowski David Salinas Stephan Seufert and Gyuri Szarvas. 2018. On challenges in machine learning model management. (2018)."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6403"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939502.2939516"},{"key":"e_1_2_1_29_1","volume-title":"Factors influencing the surprising instability of word embeddings. arXiv preprint arXiv:1804.09692","author":"Wendlandt Laura","year":"2018"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3476311.3476402","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T11:42:17Z","timestamp":1672227737000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3476311.3476402"}},"subtitle":["feature stores and the coming wave of embedding ecosystems"],"short-title":[],"issued":{"date-parts":[[2021,7]]},"references-count":29,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["10.14778\/3476311.3476402"],"URL":"https:\/\/doi.org\/10.14778\/3476311.3476402","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2021,7]]}}}