{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T07:13:18Z","timestamp":1784099598609,"version":"3.55.0"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2020,8]]},"abstract":"<jats:p>Hybrid Transactional and Analytical Processing (HTAP) databases require processing transactional and analytical queries in isolation to remove the interference between them. To achieve this, it is necessary to maintain different replicas of data specified for the two types of queries. However, it is challenging to provide a consistent view for distributed replicas within a storage system, where analytical requests can efficiently read consistent and fresh data from transactional workloads at scale and with high availability.<\/jats:p>\n          <jats:p>To meet this challenge, we propose extending replicated state machine-based consensus algorithms to provide consistent replicas for HTAP workloads. Based on this novel idea, we present a Raft-based HTAP database: TiDB. In the database, we design a multi-Raft storage system which consists of a row store and a column store. The row store is built based on the Raft algorithm. It is scalable to materialize updates from transactional requests with high availability. In particular, it asynchronously replicates Raft logs to learners which transform row format to column format for tuples, forming a real-time updatable column store. This column store allows analytical queries to efficiently read fresh and consistent data with strong isolation from transactions on the row store. Based on this storage system, we build an SQL engine to process large-scale distributed transactions and expensive analytical queries. The SQL engine optimally accesses row-format and column-format replicas of data. We also include a powerful analysis engine, TiSpark, to help TiDB connect to the Hadoop ecosystem. Comprehensive experiments show that TiDB achieves isolated high performance under CH-benCHmark, a benchmark focusing on HTAP workloads.<\/jats:p>","DOI":"10.14778\/3415478.3415535","type":"journal-article","created":{"date-parts":[[2020,9,14]],"date-time":"2020-09-14T18:46:35Z","timestamp":1600109195000},"page":"3072-3084","source":"Crossref","is-referenced-by-count":294,"title":["TiDB"],"prefix":"10.14778","volume":"13","author":[{"given":"Dongxu","family":"Huang","sequence":"first","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qi","family":"Liu","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qiu","family":"Cui","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhuhe","family":"Fang","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoyu","family":"Ma","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fei","family":"Xu","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Li","family":"Shen","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liu","family":"Tang","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuxing","family":"Zhou","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Menglong","family":"Huang","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wan","family":"Wei","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cong","family":"Liu","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian","family":"Zhang","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianjun","family":"Li","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuelian","family":"Wu","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lingyu","family":"Song","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ruoxi","family":"Sun","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuaipeng","family":"Yu","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lei","family":"Zhao","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicholas","family":"Cameron","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liquan","family":"Pei","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xin","family":"Tang","sequence":"additional","affiliation":[{"name":"PingCAP"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,8]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Clickhouse. https:\/\/clickhouse.tech."},{"key":"e_1_2_1_2_1","unstructured":"LZ4. https:\/\/github.com\/lz4\/lz4."},{"key":"e_1_2_1_3_1","unstructured":"MemSQL. https:\/\/www.memsql.com."},{"key":"e_1_2_1_4_1","unstructured":"Parquet. https:\/\/parquet.apache.org."},{"key":"e_1_2_1_5_1","unstructured":"RocksDB. https:\/\/rocksdb.org."},{"key":"e_1_2_1_6_1","unstructured":"Sysbench. https:\/\/github.com\/akopytov\/sysbench."},{"key":"e_1_2_1_7_1","unstructured":"TiDB. https:\/\/github.com\/pingcap\/tidb."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2915231"},{"key":"e_1_2_1_9_1","volume-title":"CIDR. www.cidrdb.org","author":"Barber R.","year":"2017","unstructured":"R. Barber, C. Garcia-Arellano, R. Grosman, R. M\u00fcller, et al. Evolving Databases for New-Gen Big Data Applications. In CIDR. www.cidrdb.org, 2017."},{"key":"e_1_2_1_10_1","first-page":"2077","volume-title":"SIGMOD","author":"Barber R.","year":"2016","unstructured":"R. Barber, M. Huras, G. M. Lohman, C. Mohan, et al. Wildfire: Concurrent Blazing Data Ingest and Analytics. In SIGMOD, pages 2077--2080. ACM, 2016."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978915.1978919"},{"key":"e_1_2_1_12_1","first-page":"205","volume-title":"OSDI","author":"Chang F.","year":"2006","unstructured":"F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. Gruber. Bigtable: A Distributed Storage System for Structured Data. In OSDI, pages 205--218. USENIX Association, 2006."},{"key":"e_1_2_1_13_1","volume-title":"ACM","author":"Cole R. L.","year":"2011","unstructured":"R. L. Cole, F. Funke, L. Giakoumakis, W. Guy, et al. The mixed workload CH-benCHmark. In DBTest 2011, page 8. ACM, 2011."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491245"},{"issue":"3","key":"e_1_2_1_15_1","first-page":"226","volume":"13","author":"Fang Z.","year":"2019","unstructured":"Z. Fang, B. Zheng, and C. Weng. Interleaved Multi-Vectorizing. PVLDB, 13(3):226--238, 2019.","journal-title":"Interleaved Multi-Vectorizing. PVLDB"},{"issue":"12","key":"e_1_2_1_16_1","first-page":"1295","article-title":"Full Circle Back to Shared-Nothing Database Architectures","volume":"7","author":"Floratou A.","year":"2014","unstructured":"A. Floratou, U. F. Minhas, and F. \u00d6zcan. SQL-on-Hadoop: Full Circle Back to Shared-Nothing Database Architectures. PVLDB, 7(12):1295--1306, 2014.","journal-title":"PVLDB"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/69.273032"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2011.5767867"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2015.7113373"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/279227.279229"},{"issue":"12","key":"e_1_2_1_21_1","first-page":"1740","volume":"8","author":"Larson P.","year":"2015","unstructured":"P. Larson, A. Birka, E. N. Hanson, W. Huang, M. Nowakiewicz, and V. Papadimos. Real-Time Analytical Processing with SQL Server. PVLDB, 8(12):1740--1751, 2015.","journal-title":"Real-Time Analytical Processing with SQL Server. PVLDB"},{"issue":"12","key":"e_1_2_1_22_1","first-page":"1598","article-title":"Parallel Replication across Formats","volume":"10","author":"Lee J.","year":"2017","unstructured":"J. Lee, S. Moon, K. H. Kim, D. H. Kim, S. K. Cha, W. Han, C. G. Park, H. J. Na, and J. Lee. Parallel Replication across Formats in SAP HANA for Scaling Out Mixed OLTP\/OLAP Workloads. PVLDB, 10(12):1598--1609, 2017.","journal-title":"SAP HANA for Scaling Out Mixed OLTP\/OLAP Workloads. PVLDB"},{"key":"e_1_2_1_23_1","first-page":"1","volume-title":"EDBT","author":"Luo C.","year":"2019","unstructured":"C. Luo, P. T\u00f6z\u00fcn, Y. Tian, R. Barber, etal. Umzi: Unified Multi-Zone Indexing for Large-Scale HTAP. In EDBT, pages 1--12. OpenProceedings.org, 2019."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3035959"},{"key":"e_1_2_1_25_1","volume-title":"CIDR. www.cidrdb.org","author":"Mozafari B.","year":"2017","unstructured":"B. Mozafari, J. Ramnarayan, S. Menon, Y. Mahajan, S. Chakraborty, H. Bhanawat, and K. Bachhav. SnappyData: A Unified Cluster for Streaming, Transactions and Interactive Analytics. In CIDR. www.cidrdb.org, 2017."},{"key":"e_1_2_1_26_1","series-title":"LNI","first-page":"499","volume-title":"DBIS","author":"M\u00fchlbauer T.","year":"2013","unstructured":"T. M\u00fchlbauer, W. R\u00f6diger, A. Reiser, A. Kemper, and T. Neumann. ScyPer: A Hybrid OLTP&OLAP Distributed Main Memory Database System for Scalable Real-Time Analytics. In DBIS, volume P-214 of LNI, pages 499--502. GI, 2013."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2016.7498333"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/s002360050048"},{"key":"e_1_2_1_29_1","first-page":"305","volume-title":"USENIX ATC","author":"Ongaro D.","year":"2014","unstructured":"D. Ongaro and J. K. Ousterhout. In Search of an Understandable Consensus Algorithm. In USENIX ATC, pages 305--319. USENIX Association, 2014."},{"key":"e_1_2_1_30_1","first-page":"1771","volume-title":"SIGMOD","author":"\u00d6zcan F.","year":"2017","unstructured":"F. \u00d6zcan, Y. Tian, and P. T\u00f6z\u00fcn. Hybrid Transactional\/Analytical Processing: A Survey. In SIGMOD, pages 1771--1775. ACM, 2017."},{"key":"e_1_2_1_31_1","volume-title":"CIDR. www.cidrdb.org","author":"Pavlo A.","year":"2017","unstructured":"A. Pavlo, G. Angulo, J. Arulraj, H. Lin, J. Lin, et al. Self-Driving Database Management Systems. In CIDR. www.cidrdb.org, 2017."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3003665.3003674"},{"key":"e_1_2_1_33_1","first-page":"251","volume-title":"OSDI","author":"Peng D.","year":"2010","unstructured":"D. Peng and F. Dabek. Large-scale Incremental Processing Using Distributed Transactions and Notifications. In OSDI, pages 251--264. USENIX Association, 2010."},{"key":"e_1_2_1_34_1","first-page":"97","volume-title":"TPCTC","volume":"8904","author":"Psaroudakis I.","year":"2014","unstructured":"I. Psaroudakis, F. Wolf, N. May, T. Neumann, A. B\u00f6hm, A. Ailamaki, and K. Sattler. Scaling Up Mixed Workloads: A Battle of Data Freshness, Flexibility, and Scheduling. In TPCTC, volume 8904, pages 97--112. Springer, 2014."},{"key":"e_1_2_1_35_1","first-page":"540","volume-title":"EDBT","author":"Sadoghi M.","year":"2018","unstructured":"M. Sadoghi, S. Bhattacherjee, B. Bhattacharjee, and M. Canim. L-Store: A Real-time OLTP and OLAP System. In EDBT, pages 540--551. OpenProceedings.org, 2018."},{"key":"e_1_2_1_36_1","first-page":"729","volume-title":"SIGMOD","author":"Sivasubramanian S.","year":"2012","unstructured":"S. Sivasubramanian. Amazon dynamoDB: a seamlessly scalable non-relational database service. In SIGMOD, pages 729--730. ACM, 2012."},{"key":"e_1_2_1_37_1","first-page":"2","volume-title":"ICDE","author":"Stonebraker M.","year":"2005","unstructured":"M. Stonebraker and U. \u00c7etintemel. \"One Size Fits All\": An Idea Whose Time Has Come and Gone (Abstract). In ICDE, pages 2--11. IEEE Computer Society, 2005."},{"key":"e_1_2_1_38_1","first-page":"1493","volume-title":"SIGMOD","author":"Taft R.","year":"2020","unstructured":"R. Taft, I. Sharif, A. Matei, N. VanBenschoten, J. Lewis, et al. CockroachDB: The Resilient Geo-Distributed SQL Database. In SIGMOD, pages 1493--1509. ACM, 2020."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3415478.3415535","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,17]],"date-time":"2025-09-17T02:32:05Z","timestamp":1758076325000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3415478.3415535"}},"subtitle":["a Raft-based HTAP database"],"short-title":[],"issued":{"date-parts":[[2020,8]]},"references-count":38,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2020,8]]}},"alternative-id":["10.14778\/3415478.3415535"],"URL":"https:\/\/doi.org\/10.14778\/3415478.3415535","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2020,8]]}}}