{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T04:03:38Z","timestamp":1783656218134,"version":"3.55.0"},"reference-count":26,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2016,11]]},"abstract":"<jats:p>\n            We present a novel storage manager for\n            <jats:italic>multi-dimensional arrays<\/jats:italic>\n            that arise in scientific applications, which is part of a larger scientific data management system called TileDB. In contrast to existing solutions, TileDB is optimized for\n            <jats:italic>both<\/jats:italic>\n            dense and sparse arrays. Its key idea is to organize array elements into ordered collections called\n            <jats:italic>fragments.<\/jats:italic>\n            Each fragment is dense or sparse, and groups contiguous array elements into\n            <jats:italic>data tiles<\/jats:italic>\n            of fixed capacity. The organization into fragments turns random writes into sequential writes, and, coupled with a novel read algorithm, leads to very efficient reads. TileDB enables parallelization via multi-threading and multi-processing, offering thread-\/process-safety and atomicity via lightweight locking. We show that TileDB delivers comparable performance to the HDF5 dense array storage manager, while providing much faster random writes. We also show that TileDB offers substantially faster reads and writes than the SciDB array database system with both dense and sparse arrays. Finally, we demonstrate that TileDB is considerably faster than adaptations of the Vertica relational column-store for dense array storage management, and at least as fast for the case of sparse arrays.\n          <\/jats:p>","DOI":"10.14778\/3025111.3025117","type":"journal-article","created":{"date-parts":[[2017,1,24]],"date-time":"2017-01-24T15:29:41Z","timestamp":1485271781000},"page":"349-360","source":"Crossref","is-referenced-by-count":108,"title":["The TileDB array data storage manager"],"prefix":"10.14778","volume":"10","author":[{"given":"Stavros","family":"Papadopoulos","sequence":"first","affiliation":[{"name":"Intel Labs &amp; MIT"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kushal","family":"Datta","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samuel","family":"Madden","sequence":"additional","affiliation":[{"name":"MIT"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Timothy","family":"Mattson","sequence":"additional","affiliation":[{"name":"Intel Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2016,11]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Apache Kylin. http:\/\/kylin.apache.org\/.  Apache Kylin. http:\/\/kylin.apache.org\/."},{"key":"e_1_2_1_2_1","unstructured":"Broad Institute Intel work together to develop tools to accelerate biomedical research. http:\/\/genomicinfo.broadinstitute.org\/acton\/media\/13431\/broad-intel-collaboration.  Broad Institute Intel work together to develop tools to accelerate biomedical research. http:\/\/genomicinfo.broadinstitute.org\/acton\/media\/13431\/broad-intel-collaboration."},{"key":"e_1_2_1_3_1","unstructured":"Charm++. http:\/\/charm.cs.illinois.edu\/research\/charm.  Charm++. http:\/\/charm.cs.illinois.edu\/research\/charm."},{"key":"e_1_2_1_4_1","unstructured":"Enabling a Strict Consistency Semantics Model in Parallel HDF5. https:\/\/www.hdfgroup.org\/HDF5\/doc\/Advanced\/PHDF5FileConsistencySemantics\/PHDF5FileConsistencySemantics.pdf.  Enabling a Strict Consistency Semantics Model in Parallel HDF5. https:\/\/www.hdfgroup.org\/HDF5\/doc\/Advanced\/PHDF5FileConsistencySemantics\/PHDF5FileConsistencySemantics.pdf."},{"key":"e_1_2_1_5_1","unstructured":"GenericIO. http:\/\/trac.alcf.anl.gov\/projects\/genericio.  GenericIO. http:\/\/trac.alcf.anl.gov\/projects\/genericio."},{"key":"e_1_2_1_6_1","unstructured":"HDF5 for Python. http:\/\/www.h5py.org\/.  HDF5 for Python. http:\/\/www.h5py.org\/."},{"key":"e_1_2_1_7_1","unstructured":"Legion Parallel System. http:\/\/legion.stanford.edu\/.  Legion Parallel System. http:\/\/legion.stanford.edu\/."},{"key":"e_1_2_1_8_1","unstructured":"National Oceanic and Atmospheric Administration. Marine Cadastre. http:\/\/marinecadastre.gov\/ais\/.  National Oceanic and Atmospheric Administration. Marine Cadastre. http:\/\/marinecadastre.gov\/ais\/."},{"key":"e_1_2_1_9_1","unstructured":"NetCDF. http:\/\/www.unidata.ucar.edu\/software\/netcdf.  NetCDF. http:\/\/www.unidata.ucar.edu\/software\/netcdf."},{"key":"e_1_2_1_10_1","unstructured":"Parallel HDF5. https:\/\/www.hdfgroup.org\/HDF5\/PHDF5\/.  Parallel HDF5. https:\/\/www.hdfgroup.org\/HDF5\/PHDF5\/."},{"key":"e_1_2_1_11_1","unstructured":"PLASMA. http:\/\/www.netlib.org\/plasma\/.  PLASMA. http:\/\/www.netlib.org\/plasma\/."},{"key":"e_1_2_1_12_1","unstructured":"PostgreSQL. http:\/\/www.postgresql.org\/.  PostgreSQL. http:\/\/www.postgresql.org\/."},{"key":"e_1_2_1_13_1","unstructured":"PyTables. http:\/\/www.pytables.org\/.  PyTables. http:\/\/www.pytables.org\/."},{"key":"e_1_2_1_14_1","unstructured":"ROMIO\n\n  \n  : A High-Performance Portable MPI-IO Implementation. http:\/\/www.mcs.anl.gov\/projects\/romio\/.  ROMIO: A High-Performance Portable MPI-IO Implementation. http:\/\/www.mcs.anl.gov\/projects\/romio\/."},{"key":"e_1_2_1_15_1","unstructured":"ScaLAPACK. http:\/\/www.netlib.org\/scalapack\/.  ScaLAPACK. http:\/\/www.netlib.org\/scalapack\/."},{"key":"e_1_2_1_16_1","unstructured":"The HDF5 Format. http:\/\/www.hdfgroup.org\/HDF5\/.  The HDF5 Format. http:\/\/www.hdfgroup.org\/HDF5\/."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/276305.276386"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807167.1807271"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-007-0071-0"},{"issue":"1","key":"e_1_2_1_20_1","first-page":"40","volume":"35","author":"Idreos S.","year":"2012","journal-title":"IEEE Data Engin. Bulletin"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.14778\/2367502.2367518"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s002360050048"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/146941.146943"},{"key":"e_1_2_1_24_1","unstructured":"F. Rusu and Y. Cheng. A Survey on Array Storage Query Languages and Systems. ArXiv e-prints 2013.  F. Rusu and Y. Cheng. A Survey on Array Storage Query Languages and Systems. ArXiv e-prints 2013."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989351"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30192-9_15"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3025111.3025117","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:19:33Z","timestamp":1672222773000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3025111.3025117"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,11]]},"references-count":26,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2016,11]]}},"alternative-id":["10.14778\/3025111.3025117"],"URL":"https:\/\/doi.org\/10.14778\/3025111.3025117","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2016,11]]}}}