{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,20]],"date-time":"2026-09-20T06:29:14Z","timestamp":1789885754047,"version":"4.0.1"},"reference-count":26,"publisher":"Association for Computing Machinery (ACM)","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2017,1]]},"abstract":"<jats:p>Big data analytics often involves complex join queries over two or more tables. Such join processing is expensive in a distributed setting both because large amounts of data must be read from disk, and because of data shuffling across the network. Many techniques based on data partitioning have been proposed to reduce the amount of data that must be accessed, often focusing on finding the best partitioning scheme for a particular workload, rather than adapting to changes in the workload over time.<\/jats:p>\n          <jats:p>\n            In this paper, we present AdaptDB, an adaptive storage manager for analytical database workloads in a distributed setting. It works by partitioning datasets across a cluster and incrementally refining data partitioning as queries are run. AdaptDB introduces a novel\n            <jats:italic>hyper-join<\/jats:italic>\n            that avoids expensive data shuffling by identifying storage blocks of the joining tables that overlap on the join attribute, and only joining those blocks. Hyper-join performs well when each block in one table overlaps with few blocks in the other table, since that will minimize the number of blocks that have to be accessed. To minimize the number of overlapping blocks for common join queries, AdaptDB users\n            <jats:italic>smooth repartitioning<\/jats:italic>\n            to repartition small portions of the tables on join attributes as queries run. A prototype of AdaptDB running on top of Spark improves query performance by 2--3x on TPC-H as well as real-world dataset, versus a system that employs scans and shuffle-joins.\n          <\/jats:p>","DOI":"10.14778\/3055540.3055551","type":"journal-article","created":{"date-parts":[[2017,3,15]],"date-time":"2017-03-15T14:27:29Z","timestamp":1489588049000},"page":"589-600","source":"Crossref","is-referenced-by-count":44,"title":["AdaptDB"],"prefix":"10.14778","volume":"10","author":[{"given":"Yi","family":"Lu","sequence":"first","affiliation":[{"name":"MIT CSAIL"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anil","family":"Shanbhag","sequence":"additional","affiliation":[{"name":"MIT CSAIL"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alekh","family":"Jindal","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samuel","family":"Madden","sequence":"additional","affiliation":[{"name":"MIT CSAIL"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2017,1]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Apache hadoop. http:\/\/hadoop.apache.org.  Apache hadoop. http:\/\/hadoop.apache.org."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007568.1007609"},{"key":"e_1_2_1_3_1","volume-title":"HotOS","author":"Ananthanarayanan G.","year":"2011","unstructured":"G. Ananthanarayanan , A. Ghodsi , S. Shenker , and I. Stoica . Disk-locality in datacenter computing considered irrelevant . In HotOS , 2011 . G. Ananthanarayanan, A. Ghodsi, S. Shenker, and I. Stoica. Disk-locality in datacenter computing considered irrelevant. In HotOS, 2011."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.14778\/2904483.2904485"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipl.2010.12.003"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920853"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920908"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376727"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732279.2732281"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.14778\/2002938.2002943"},{"key":"e_1_2_1_11_1","volume-title":"Database cracking: Towards auto-tuning database kernels","author":"Idreos S.","year":"2010","unstructured":"S. Idreos . Database cracking: Towards auto-tuning database kernels . CWI and University of Amsterdam , 2010 . S. Idreos. Database cracking: Towards auto-tuning database kernels. CWI and University of Amsterdam, 2010."},{"key":"e_1_2_1_12_1","first-page":"68","volume-title":"CIDR","author":"Idreos S.","year":"2007","unstructured":"S. Idreos , M. L. Kersten , and S. Manegold . Database cracking . In CIDR , pages 68 -- 78 , 2007 . S. Idreos, M. L. Kersten, and S. Manegold. Database cracking. In CIDR, pages 68--78, 2007."},{"key":"e_1_2_1_13_1","first-page":"213","volume-title":"CIDR","author":"Kersten M. L.","year":"2005","unstructured":"M. L. Kersten and S. Manegold . Cracking the database store . In CIDR , pages 213 -- 224 , 2005 . M. L. Kersten and S. Manegold. Cracking the database store. In CIDR, pages 213--224, 2005."},{"key":"e_1_2_1_14_1","first-page":"488","volume-title":"VLDB","author":"Merrett T. H.","year":"1981","unstructured":"T. H. Merrett , Y. Kambayashi , and H. Yasuura . Scheduling of page-fetches in join operations . In VLDB , pages 488 -- 498 , 1981 . T. H. Merrett, Y. Kambayashi, and H. Yasuura. Scheduling of page-fetches in join operations. In VLDB, pages 488--498, 1981."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989444"},{"key":"e_1_2_1_16_1","first-page":"1","volume-title":"OSDI","author":"Nightingale E. B.","year":"2012","unstructured":"E. B. Nightingale , J. Elson , J. Fan , O. S. Hofmann , J. Howell , and Y. Suzue . Flat datacenter storage . In OSDI , pages 1 -- 15 , 2012 . E. B. Nightingale, J. Elson, J. Fan, O. S. Hofmann, J. Howell, and Y. Suzue. Flat datacenter storage. In OSDI, pages 1--15, 2012."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2213836.2213844"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3148.3325"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2452376.2452427"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2016.7498324"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.14778\/3007263.3007311"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2610515"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.14778\/2876473.2876477"},{"key":"e_1_2_1_24_1","first-page":"15","volume-title":"NSDI","author":"Zaharia M.","year":"2012","unstructured":"M. Zaharia , M. Chowdhury , T. Das , A. Dave , J. Ma , M. McCauly , M. J. Franklin , S. Shenker , and I. Stoica . Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing . In NSDI , pages 15 -- 28 , 2012 . M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauly, M. J. Franklin, S. Shenker, and I. Stoica. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In NSDI, pages 15--28,2012."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2723718"},{"key":"e_1_2_1_26_1","first-page":"1087","volume-title":"VLDB","author":"Zilio D. C.","year":"2004","unstructured":"D. C. Zilio , J. Rao , S. Lightstone , G. M. Lohman , A. J. Storm , C. Garcia-Arellano , and S. Fadden . DB2 design advisor: Integrated automatic physical database design . In VLDB , pages 1087 -- 1097 , 2004 . D. C. Zilio, J. Rao, S. Lightstone, G. M. Lohman, A. J. Storm, C. Garcia-Arellano, and S. Fadden. DB2 design advisor: Integrated automatic physical database design. In VLDB, pages 1087--1097, 2004."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3055540.3055551","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T11:13:20Z","timestamp":1672226000000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3055540.3055551"}},"subtitle":["adaptive partitioning for distributed joins"],"short-title":[],"issued":{"date-parts":[[2017,1]]},"references-count":26,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2017,1]]}},"alternative-id":["10.14778\/3055540.3055551"],"URL":"https:\/\/doi.org\/10.14778\/3055540.3055551","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2017,1]]}}}