{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T09:05:11Z","timestamp":1775639111165,"version":"3.50.1"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"8","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2021,4]]},"abstract":"<jats:p>\n            Partitioning is effective in avoiding expensive shuffling operations. However, it remains a significant challenge to automate this process for Big Data analytics workloads that extensively use user defined functions (UDFs), where sub-computations are hard to be reused for partitionings compared to relational applications. In addition, functional dependency that is widely utilized for partitioning selection is often unavailable in the unstructured data that is ubiquitous in UDF-centric analytics. We propose the\n            <jats:italic>Lachesis<\/jats:italic>\n            system, which represents UDF-centric workloads as workflows of analyzable and reusable sub-computations.\n            <jats:italic>Lachesis<\/jats:italic>\n            further adopts a deep reinforcement learning model to infer which sub-computations should be used to partition the underlying data. This analysis is then applied to automatically optimize the storage of the data across applications to improve the performance and users' productivity.\n          <\/jats:p>","DOI":"10.14778\/3457390.3457392","type":"journal-article","created":{"date-parts":[[2021,10,21]],"date-time":"2021-10-21T22:48:38Z","timestamp":1634856518000},"page":"1262-1275","source":"Crossref","is-referenced-by-count":9,"title":["Lachesis"],"prefix":"10.14778","volume":"14","author":[{"given":"Jia","family":"Zou","sequence":"first","affiliation":[{"name":"Arizona State University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Amitabh","family":"Das","sequence":"additional","affiliation":[{"name":"Arizona State University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pratik","family":"Barhate","sequence":"additional","affiliation":[{"name":"Arizona State University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arun","family":"Iyengar","sequence":"additional","affiliation":[{"name":"IBM T.J.Watson Research Center"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Binhang","family":"Yuan","sequence":"additional","affiliation":[{"name":"Rice University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dimitrije","family":"Jankov","sequence":"additional","affiliation":[{"name":"Rice University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chris","family":"Jermaine","sequence":"additional","affiliation":[{"name":"Rice University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,10,21]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007568.1007609"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-014-0357-y"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2750543"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824080"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1137\/0221065"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.14778\/3007263.3007279"},{"key":"e_1_2_1_7_1","volume-title":"HDFS architecture guide. HADOOP APACHE PROJECT http:\/\/hadoop.apache.org\/common\/docs\/current\/hdfsdesign.pdf","author":"Borthakur Dhruba","year":"2008"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.14778\/2367502.2367519"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352128"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/800157.805047"},{"key":"e_1_2_1_11_1","volume-title":"TPC-H benchmark specification. Published at http:\/\/www.tcp.org\/hspec.html 21","author":"Transaction Processing Performance Council","year":"2008"},{"key":"e_1_2_1_12_1","unstructured":"Andrew Crotty Alex Galakatos Kayhan Dursun Tim Kraska Ugur Cetintemel and Stanley B Zdonik. 2015. Tupleware: \"Big\" Data Big Analytics Small Clusters.. In CIDR.  Andrew Crotty Alex Galakatos Kayhan Dursun Tim Kraska Ugur Cetintemel and Stanley B Zdonik. 2015. Tupleware: \"Big\" Data Big Analytics Small Clusters.. In CIDR."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920908"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/0022-247X(65)90125-3"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2723709"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2588569"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376727"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.14778\/2002938.2002943"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.5555\/3045390.3045688"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2011.2165682"},{"key":"e_1_2_1_21_1","unstructured":"HBase [n.d.]. HBase. https:\/\/hbase.apache.org\/.  HBase [n.d.]. HBase. https:\/\/hbase.apache.org\/."},{"key":"e_1_2_1_22_1","volume-title":"Learning a Partitioning Advisor with Deep Reinforcement Learning. arXiv preprint arXiv:1904.01279","author":"Hilprecht Benjamin","year":"2019"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389704"},{"key":"e_1_2_1_24_1","first-page":"68","article-title":"Database Cracking","volume":"7","author":"Idreos Stratos","year":"2007","journal-title":"CIDR"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.14778\/3317315.3317323"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3190656"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/3026877.3026887"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732951.2732959"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.14778\/3342263.3342633"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.2307\/2685263"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389750"},{"key":"e_1_2_1_32_1","volume-title":"Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971","author":"Lillicrap Timothy P","year":"2015"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1809049.1809051"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.14778\/3055540.3055551"},{"key":"e_1_2_1_35_1","volume-title":"Beginning Apache Spark 2","author":"Luu Hien"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098822.3098843"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.5555\/3045390.3045594"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989444"},{"key":"e_1_2_1_40_1","volume-title":"Conference on Innovative Data Systems Research (CIDR).","author":"Palkar Shoumik","year":"2017"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3186728.3164140"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.5555\/556803"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/564691.564757"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1002\/sapm194221183"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISM.2006.143"},{"key":"e_1_2_1_46_1","volume-title":"Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347","author":"Schulman John","year":"2017"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183653"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.14778\/2831360.2831365"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.14778\/2733004.2733005"},{"key":"e_1_2_1_50_1","volume-title":"Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al.","author":"Silver David","year":"2016"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.5555\/551283"},{"key":"e_1_2_1_52_1","unstructured":"tensorflow [n.d.]. FFN example with tensorflow. https:\/\/www.kaggle.com\/hbaderts\/simple-feed-forward-neural-network-with-tensorflow.  tensorflow [n.d.]. FFN example with tensorflow. https:\/\/www.kaggle.com\/hbaderts\/simple-feed-forward-neural-network-with-tensorflow."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687553.1687609"},{"key":"e_1_2_1_54_1","volume-title":"The digital universe of opportunities: Rich data and the increasing value of the internet of things. IDC Analyze the Future","author":"Turner Vernon","year":"2014"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1137\/0208032"},{"key":"e_1_2_1_56_1","volume-title":"The Workflow Trace Archive: Open-Access Data from Public and Private Computing Infrastructures","author":"Versluis Laurens","year":"2020"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.5555\/3388242.3388275"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.14778\/2777598.2777601"},{"key":"e_1_2_1_59_1","volume-title":"Machine learning 8, 3--4","author":"Watkins Christopher JCH","year":"1992"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.5555\/2285539"},{"key":"e_1_2_1_61_1","unstructured":"Yahoo [n.d.]. Yahoo! Cloud Trace. https:\/\/webscope.sandbox.yahoo.com\/catalog.php?datatype=s.  Yahoo [n.d.]. Yahoo! Cloud Trace. https:\/\/webscope.sandbox.yahoo.com\/catalog.php?datatype=s."},{"key":"e_1_2_1_62_1","volume-title":"Tensor Relational Algebra for Machine Learning System Design. arXiv preprint arXiv:2009.00524","author":"Yuan Binhang","year":"2020"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.5555\/1863103.1863113"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2723718"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2213836.2213839"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2010.5447802"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3196933"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.14778\/3311880.3311885"},{"key":"e_1_2_1_69_1","volume-title":"Architecture of a distributed storage that combines file system, memory and computation in a single layer. The VLDB Journal","author":"Zou Jia","year":"2020"},{"key":"e_1_2_1_70_1","volume-title":"WATSON: A Workflow-based Data Storage Optimizer for Analytics. In Proceedings of the 36th International Conference on Massive Storage Systems and Technology (MSST","author":"Zou Jia","year":"2020"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3457390.3457392","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:44:33Z","timestamp":1672224273000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3457390.3457392"}},"subtitle":["automatic partitioning for UDF-centric analytics"],"short-title":[],"issued":{"date-parts":[[2021,4]]},"references-count":69,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2021,4]]}},"alternative-id":["10.14778\/3457390.3457392"],"URL":"https:\/\/doi.org\/10.14778\/3457390.3457392","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2021,4]]}}}