{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T01:31:39Z","timestamp":1773192699251,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":40,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,6,9]],"date-time":"2021-06-09T00:00:00Z","timestamp":1623196800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT)","award":["NRF-2021R1C1C1005999"],"award-info":[{"award-number":["NRF-2021R1C1C1005999"]}]},{"name":"National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT)","award":["NRF-2018R1A5A1059921"],"award-info":[{"award-number":["NRF-2018R1A5A1059921"]}]},{"name":"Google AI Focused Research Award"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,6,9]]},"DOI":"10.1145\/3448016.3452792","type":"proceedings-article","created":{"date-parts":[[2021,6,18]],"date-time":"2021-06-18T17:22:30Z","timestamp":1624036950000},"page":"1771-1783","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":29,"title":["Slice Tuner"],"prefix":"10.1145","author":[{"given":"Ki Hyun","family":"Tae","sequence":"first","affiliation":[{"name":"KAIST, Daejeon, Republic of Korea"}]},{"given":"Steven Euijong","family":"Whang","sequence":"additional","affiliation":[{"name":"KAIST, Daejeon, Republic of Korea"}]}],"member":"320","published-online":{"date-parts":[[2021,6,18]]},"reference":[{"key":"e_1_3_2_2_1_1","unstructured":"[n.d.]. Amazon Mechanical Turk. https:\/\/www.mturk.com\/.  [n.d.]. Amazon Mechanical Turk. https:\/\/www.mturk.com\/."},{"key":"e_1_3_2_2_2_1","unstructured":"[n.d.]. Slice Tuner Github repository. https:\/\/github.com\/systemT2021\/ SliceTuner.  [n.d.]. Slice Tuner Github repository. https:\/\/github.com\/systemT2021\/ SliceTuner."},{"key":"e_1_3_2_2_3_1","unstructured":"[n.d.]. Software 2.0. https:\/\/medium.com\/@karpathy\/software-2-0- a64152b37c35.  [n.d.]. Software 2.0. https:\/\/medium.com\/@karpathy\/software-2-0- a64152b37c35."},{"key":"e_1_3_2_2_4_1","unstructured":"[n.d.]. TensorFlow Model Analysis. https:\/\/github.com\/tensorflow\/modelanalysis.  [n.d.]. TensorFlow Model Analysis. https:\/\/github.com\/tensorflow\/modelanalysis."},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(93)90013-M"},{"key":"e_1_3_2_2_6_1","volume-title":"Benoit Steiner, Paul A. Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng.","author":"Abadi Mart\u00edn","year":"2016","unstructured":"Mart\u00edn Abadi , Paul Barham , Jianmin Chen , Zhifeng Chen , Andy Davis , Jeffrey Dean , Matthieu Devin , Sanjay Ghemawat , Geoffrey Irving , Michael Isard , Manjunath Kudlur , Josh Levenberg , Rajat Monga , Sherry Moore , Derek Gordon Murray , Benoit Steiner, Paul A. Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016 . TensorFlow: A System for Large-Scale Machine Learning. In OSDI. 265--283. Mart\u00edn Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek Gordon Murray, Benoit Steiner, Paul A. Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A System for Large-Scale Machine Learning. In OSDI. 265--283."},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"crossref","unstructured":"Abolfazl Asudeh Zhongjun Jin and H. V. Jagadish. 2019. Assessing and Remedying Coverage for a Given Dataset. ICDE (2019) 554--565.  Abolfazl Asudeh Zhongjun Jin and H. V. Jagadish. 2019. Assessing and Remedying Coverage for a Given Dataset. ICDE (2019) 554--565.","DOI":"10.1109\/ICDE.2019.00056"},{"key":"e_1_3_2_2_8_1","volume-title":"Finite-Time Analysis of the Multiarmed Bandit Problem. 47, 2--3","author":"Auer Peter","year":"2002","unstructured":"Peter Auer , Nicol\u00f2 Cesa-Bianchi , and Paul Fischer . 2002. Finite-Time Analysis of the Multiarmed Bandit Problem. 47, 2--3 ( 2002 ), 235--256. Peter Auer, Nicol\u00f2 Cesa-Bianchi, and Paul Fischer. 2002. Finite-Time Analysis of the Multiarmed Bandit Problem. 47, 2--3 (2002), 235--256."},{"key":"e_1_3_2_2_9_1","volume-title":"Zakaria Haque, Salem Haykal, Mustafa Ispir, Vihan Jain, Levent Koc, et al.","author":"Baylor Denis","year":"2017","unstructured":"Denis Baylor , Eric Breck , Heng-Tze Cheng , Noah Fiedel , Chuan Yu Foo , Zakaria Haque, Salem Haykal, Mustafa Ispir, Vihan Jain, Levent Koc, et al. 2017 . TFX : A TensorFlow-Based Production-Scale Machine Learning Platform. In KDD. 1387-- 1395. Denis Baylor, Eric Breck, Heng-Tze Cheng, Noah Fiedel, Chuan Yu Foo, Zakaria Haque, Salem Haykal, Mustafa Ispir, Vihan Jain, Levent Koc, et al. 2017. TFX: A TensorFlow-Based Production-Scale Machine Learning Platform. In KDD. 1387-- 1395."},{"key":"e_1_3_2_2_10_1","volume-title":"Noy","author":"Benjelloun Omar","year":"2020","unstructured":"Omar Benjelloun , Shiyu Chen , and Natasha F . Noy . 2020 . Google Dataset Search by the Numbers. CoRR abs\/2006.06894 (2020). arXiv:2006.06894 Omar Benjelloun, Shiyu Chen, and Natasha F. Noy. 2020. Google Dataset Search by the Numbers. CoRR abs\/2006.06894 (2020). arXiv:2006.06894"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2018.07.011"},{"key":"e_1_3_2_2_12_1","volume-title":"Sontag","author":"Chen Irene Y.","year":"2018","unstructured":"Irene Y. Chen , Fredrik D. Johansson , and David A . Sontag . 2018 . Why Is My Classifier Discriminatory?. In NeurIPS. 3543--3554. Irene Y. Chen, Fredrik D. Johansson, and David A. Sontag. 2018. Why Is My Classifier Discriminatory?. In NeurIPS. 3543--3554."},{"key":"e_1_3_2_2_13_1","volume-title":"Medical Image Deep Learning with Hospital PACS Dataset. CoRR abs\/1511.06348","author":"Cho Junghwan","year":"2015","unstructured":"Junghwan Cho , Kyewook Lee , Ellie Shin , Garry Choy , and Synho Do. 2015. Medical Image Deep Learning with Hospital PACS Dataset. CoRR abs\/1511.06348 ( 2015 ). arXiv:1511.06348 Junghwan Cho, Kyewook Lee, Ellie Shin, Garry Choy, and Synho Do. 2015. Medical Image Deep Learning with Hospital PACS Dataset. CoRR abs\/1511.06348 (2015). arXiv:1511.06348"},{"key":"e_1_3_2_2_14_1","volume-title":"Ki Hyun Tae, and Steven Euijong Whang","author":"Chung Yeounoh","year":"2019","unstructured":"Yeounoh Chung , Tim Kraska , Neoklis Polyzotis , Ki Hyun Tae, and Steven Euijong Whang . 2019 . Slice Finder : Automated Data Slicing for Model Validation. IEEE TKDE ( 2019). Yeounoh Chung, Tim Kraska, Neoklis Polyzotis, Ki Hyun Tae, and Steven Euijong Whang. 2019. Slice Finder: Automated Data Slicing for Model Validation. IEEE TKDE (2019)."},{"key":"e_1_3_2_2_15_1","volume-title":"Jost Tobias Springenberg, and Frank Hutter","author":"Domhan Tobias","year":"2015","unstructured":"Tobias Domhan , Jost Tobias Springenberg, and Frank Hutter . 2015 . Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves. In IJCAI. 3460--3468. Tobias Domhan, Jost Tobias Springenberg, and Frank Hutter. 2015. Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves. In IJCAI. 3460--3468."},{"key":"e_1_3_2_2_16_1","volume-title":"Fairness Through Awareness","author":"Dwork Cynthia","unstructured":"Cynthia Dwork , Moritz Hardt , Toniann Pitassi , Omer Reingold , and Richard Zemel . 2012. Fairness Through Awareness . In ITCS (Cambridge, Massachusetts ). ACM, New York, NY, USA , 214--226. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness Through Awareness. In ITCS (Cambridge, Massachusetts). ACM, New York, NY, USA, 214--226."},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"crossref","unstructured":"Michael Feldman Sorelle A. Friedler John Moeller Carlos Scheidegger and Suresh Venkatasubramanian. 2015. Certifying and Removing Disparate Impact. In KDD. 259--268.  Michael Feldman Sorelle A. Friedler John Moeller Carlos Scheidegger and Suresh Venkatasubramanian. 2015. Certifying and Removing Disparate Impact. In KDD. 259--268.","DOI":"10.1145\/2783258.2783311"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6947-12-8"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2014.02.013"},{"key":"e_1_3_2_2_20_1","unstructured":"Moritz Hardt Eric Price and Nati Srebro. 2016. Equality of Opportunity in Supervised Learning. In NeurIPS. 3315--3323.  Moritz Hardt Eric Price and Nati Srebro. 2016. Equality of Opportunity in Supervised Learning. In NeurIPS. 3315--3323."},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"crossref","unstructured":"K. He X. Zhang S. Ren and J. Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR. 770--778.  K. He X. Zhang S. Ren and J. Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR. 770--778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_2_22_1","volume-title":"Yang Yang, and Yanqi Zhou.","author":"Hestness Joel","year":"2017","unstructured":"Joel Hestness , Sharan Narang , Newsha Ardalani , Gregory F. Diamos , Heewoo Jun , Hassan Kianinejad , Md. Mostofa Ali Patwary , Yang Yang, and Yanqi Zhou. 2017 . Deep Learning Scaling is Predictable, Empirically. CoRR abs\/1712.00409 (2017). Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory F. Diamos, Heewoo Jun, Hassan Kianinejad, Md. Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. 2017. Deep Learning Scaling is Predictable, Empirically. CoRR abs\/1712.00409 (2017)."},{"key":"e_1_3_2_2_23_1","unstructured":"Niki Kilbertus Mateo Rojas-Carulla Giambattista Parascandolo Moritz Hardt Dominik Janzing and Bernhard Sch\u00f6lkopf. 2017. Avoiding Discrimination through Causal Reasoning. In NeurIPS. 656--666.  Niki Kilbertus Mateo Rojas-Carulla Giambattista Parascandolo Moritz Hardt Dominik Janzing and Bernhard Sch\u00f6lkopf. 2017. Avoiding Discrimination through Causal Reasoning. In NeurIPS. 656--666."},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"crossref","unstructured":"Hoon Kim Kangwook Lee Gyeongjo Hwang and Changho Suh. 2019. Crash to Not Crash: Learn to Identify Dangerous Vehicles Using a Simulator. In AAAI. 978--985.  Hoon Kim Kangwook Lee Gyeongjo Hwang and Changho Suh. 2019. Crash to Not Crash: Learn to Identify Dangerous Vehicles Using a Simulator. In AAAI. 978--985.","DOI":"10.1609\/aaai.v33i01.3301978"},{"key":"e_1_3_2_2_25_1","unstructured":"Ron Kohavi. 1996. Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision- Tree Hybrid. In KDD. 202--207.  Ron Kohavi. 1996. Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision- Tree Hybrid. In KDD. 202--207."},{"key":"e_1_3_2_2_26_1","unstructured":"Y. LeCun L. Bottou Y. Bengio and P. Haffner. 2001. Gradient-Based Learning Applied to Document Recognition. In Intelligent Signal Processing. IEEE Press 306--351.  Y. LeCun L. Bottou Y. Bengio and P. Haffner. 2001. Gradient-Based Learning Applied to Document Recognition. In Intelligent Signal Processing. IEEE Press 306--351."},{"key":"e_1_3_2_2_27_1","volume-title":"Advances in Neural Information Processing Systems 30. Curran Associates","author":"Levine Nir","unstructured":"Nir Levine , Koby Crammer , and Shie Mannor . 2017. Rotting Bandits . In Advances in Neural Information Processing Systems 30. Curran Associates , Inc ., 3074--3083. Nir Levine, Koby Crammer, and Shie Mannor. 2017. Rotting Bandits. In Advances in Neural Information Processing Systems 30. Curran Associates, Inc., 3074--3083."},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352116"},{"key":"e_1_3_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299887.3299891"},{"key":"e_1_3_2_2_30_1","volume-title":"Digital Communications","unstructured":"Proakis. 2007. Digital Communications 5 th Edition. McGraw Hill . Proakis. 2007. Digital Communications 5th Edition. McGraw Hill.","edition":"5"},{"key":"e_1_3_2_2_31_1","unstructured":"Alexander J. Ratner Braden Hancock and Christopher R\u00e9. 2019. The Role of Massively Multi-Task and Weak Supervision in Software 2.0. In CIDR.  Alexander J. Ratner Braden Hancock and Christopher R\u00e9. 2019. The Role of Massively Multi-Task and Weak Supervision in Software 2.0. In CIDR."},{"key":"e_1_3_2_2_32_1","volume-title":"A Survey on Data Collection for Machine Learning: a Big Data - AI Integration Perspective","author":"Roh Yuji","year":"2019","unstructured":"Yuji Roh , Geon Heo , and Steven Euijong Whang . 2019. A Survey on Data Collection for Machine Learning: a Big Data - AI Integration Perspective . IEEE Trans. Knowl. Data Eng . ( 2019 ). Yuji Roh, Geon Heo, and Steven Euijong Whang. 2019. A Survey on Data Collection for Machine Learning: a Big Data - AI Integration Perspective. IEEE Trans. Knowl. Data Eng. (2019)."},{"key":"e_1_3_2_2_33_1","volume-title":"Active Learning","author":"Settles Burr","unstructured":"Burr Settles . 2012. Active Learning . Morgan & Claypool Publishers . Burr Settles. 2012. Active Learning. Morgan & Claypool Publishers."},{"key":"e_1_3_2_2_34_1","volume-title":"Ipeirotis","author":"Sheng Victor S.","year":"2008","unstructured":"Victor S. Sheng , Foster J. Provost , and Panagiotis G . Ipeirotis . 2008 . Get another label? improving data quality and data mining using multiple, noisy labelers. In KDD. 614--622. Victor S. Sheng, Foster J. Provost, and Panagiotis G. Ipeirotis. 2008. Get another label? improving data quality and data mining using multiple, noisy labelers. In KDD. 614--622."},{"key":"e_1_3_2_2_35_1","volume-title":"Slice Tuner: A Selective Data Acquisition Framework for Accurate and Fair Machine Learning Models. arXiv:2003.04549 [cs.LG]","author":"Tae Ki Hyun","year":"2021","unstructured":"Ki Hyun Tae and Steven Euijong Whang . 2021 . Slice Tuner: A Selective Data Acquisition Framework for Accurate and Fair Machine Learning Models. arXiv:2003.04549 [cs.LG] Ki Hyun Tae and Steven Euijong Whang. 2021. Slice Tuner: A Selective Data Acquisition Framework for Accurate and Fair Machine Learning Models. arXiv:2003.04549 [cs.LG]"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3294052.3322192"},{"key":"e_1_3_2_2_37_1","unstructured":"Han Xiao Kashif Rasul and Roland Vollgraf. 2017. Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. arXiv:cs.LG\/1708.07747 [cs.LG]  Han Xiao Kashif Rasul and Roland Vollgraf. 2017. Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. arXiv:cs.LG\/1708.07747 [cs.LG]"},{"key":"e_1_3_2_2_38_1","volume-title":"Gummadi","author":"Zafar Muhammad Bilal","year":"2017","unstructured":"Muhammad Bilal Zafar , Isabel Valera , Manuel Gomez-Rodriguez , and Krishna P . Gummadi . 2017 . Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment. In WWW. 1171--1180. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P. Gummadi. 2017. Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment. In WWW. 1171--1180."},{"key":"e_1_3_2_2_39_1","first-page":"39","article-title":"Accelerating the Machine Learning Lifecycle with MLflow","volume":"41","author":"Zaharia Matei","year":"2018","unstructured":"Matei Zaharia , Andrew Chen , Aaron Davidson , Ali Ghodsi , Sue Ann Hong , Andy Konwinski , Siddharth Murching , Tomas Nykodym , Paul Ogilvie , Mani Parkhe , Fen Xie , and Corey Zumar . 2018 . Accelerating the Machine Learning Lifecycle with MLflow . IEEE Data Eng. Bull. 41 , 4 (2018), 39 -- 45 . Matei Zaharia, Andrew Chen, Aaron Davidson, Ali Ghodsi, Sue Ann Hong, Andy Konwinski, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, Mani Parkhe, Fen Xie, and Corey Zumar. 2018. Accelerating the Machine Learning Lifecycle with MLflow. IEEE Data Eng. Bull. 41, 4 (2018), 39--45.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_3_2_2_40_1","doi-asserted-by":"crossref","unstructured":"Zhifei Zhang Yang Song and Hairong Qi. 2017. Age Progression\/Regression by Conditional Adversarial Autoencoder. In CVPR.  Zhifei Zhang Yang Song and Hairong Qi. 2017. Age Progression\/Regression by Conditional Adversarial Autoencoder. In CVPR.","DOI":"10.1109\/CVPR.2017.463"}],"event":{"name":"SIGMOD\/PODS '21: International Conference on Management of Data","location":"Virtual Event China","acronym":"SIGMOD\/PODS '21","sponsor":["SIGMOD ACM Special Interest Group on Management of Data"]},"container-title":["Proceedings of the 2021 International Conference on Management of Data"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3448016.3452792","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3448016.3452792","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:28:05Z","timestamp":1750195685000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3448016.3452792"}},"subtitle":["A Selective Data Acquisition Framework for Accurate and Fair Machine Learning Models"],"short-title":[],"issued":{"date-parts":[[2021,6,9]]},"references-count":40,"alternative-id":["10.1145\/3448016.3452792","10.1145\/3448016"],"URL":"https:\/\/doi.org\/10.1145\/3448016.3452792","relation":{},"subject":[],"published":{"date-parts":[[2021,6,9]]},"assertion":[{"value":"2021-06-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}