{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T10:05:08Z","timestamp":1775815508002,"version":"3.50.1"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2020,8]]},"abstract":"<jats:p>Monarch is a globally-distributed in-memory time series database system in Google. Monarch runs as a multi-tenant service and is used mostly to monitor the availability, correctness, performance, load, and other aspects of billion-user-scale applications and systems at Google. Every second, the system ingests terabytes of time series data into memory and serves millions of queries. Monarch has a regionalized architecture for reliability and scalability, and global query and configuration planes that integrate the regions into a unified system. On top of its distributed architecture, Monarch has flexible configuration, an expressive relational data model, and powerful queries. This paper describes the structure of the system and the novel mechanisms that achieve a reliable and flexible unified system on a regionalized distributed architecture. We also share important lessons learned from a decade's experience of developing and running Monarch as a service in Google.<\/jats:p>","DOI":"10.14778\/3181-3194","type":"journal-article","created":{"date-parts":[[2020,9,14]],"date-time":"2020-09-14T18:46:35Z","timestamp":1600109195000},"page":"3181-3194","source":"Crossref","is-referenced-by-count":27,"title":["Monarch"],"prefix":"10.14778","volume":"13","author":[{"given":"Colin","family":"Adams","sequence":"first","affiliation":[{"name":"Google LLC"}]},{"given":"Luis","family":"Alonso","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Benjamin","family":"Atkin","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"John","family":"Banning","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Sumeer","family":"Bhola","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Rick","family":"Buskens","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Ming","family":"Chen","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Xi","family":"Chen","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Yoo","family":"Chung","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Qin","family":"Jia","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Nick","family":"Sakharov","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"George","family":"Talbot","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Adam","family":"Tart","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Nick","family":"Taylor","sequence":"additional","affiliation":[{"name":"Google LLC"}]}],"member":"320","published-online":{"date-parts":[[2020,8]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"739","volume-title":"Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, OSDI'16","author":"Adya A.","year":"2016","unstructured":"A. Adya, D. Myers, J. Howell, J. Elson, C. Meek, V. Khemani, S. Fulger, P. Gu, L. Bhuvanagiri, J. Hunter, and et al. Slicer: Auto-sharding for datacenter applications. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, OSDI'16, page 739--753, USA, 2016. USENIX Association."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2015.04.007"},{"key":"e_1_2_1_3_1","first-page":"39","volume-title":"Proceedings of the 14th Usenix Conference on File and Storage Technologies","author":"Andersen M. P.","year":"2016","unstructured":"M. P. Andersen and D. E. Culler. BTrDB: Optimizing storage system design for timeseries processing. In Proceedings of the 14th Usenix Conference on File and Storage Technologies, pages 39--52. USENIX Association, Feb. 2016."},{"key":"e_1_2_1_4_1","unstructured":"A. Arasu S. Babu and J. Widom. The CQL continuous query language: Semantic foundations and query execution. Technical Report 2003-67 Stanford InfoLab 2003."},{"key":"e_1_2_1_5_1","first-page":"249","volume-title":"Datenbanksysteme f\u00fcr Business, Technologie und Web (BTW 2017) - Workshopband","author":"Bader A.","year":"2017","unstructured":"A. Bader, O. Kopp, and M. Falkenthal. Survey and comparison of open source time series databases. In B. Mitschang, D. Nicklas, F. Leymann, H. Sch\u00f6ning, M. Herschel, J. Teubner, T. H\u00e4rder, O. Kopp, and M. Wieland, editors, Datenbanksysteme f\u00fcr Business, Technologie und Web (BTW 2017) - Workshopband, pages 249--268. Gesellschaft f\u00fcr Informatik e.V., 2017."},{"issue":"3","key":"e_1_2_1_6_1","first-page":"1","article-title":"Sprintz: Time series compression for the internet of things. Proceedings of the ACM on Interactive, Mobile","volume":"2","author":"Blalock D.","year":"2018","unstructured":"D. Blalock, S. Madden, and J. Guttag. Sprintz: Time series compression for the internet of things. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(3):93:1--93:23, Sept. 2018.","journal-title":"Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/362686.362692"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2012.37"},{"key":"e_1_2_1_9_1","volume-title":"truetime and the cap theorem. Technical report","author":"Brewer E.","year":"2017","unstructured":"E. Brewer. Spanner, truetime and the cap theorem. Technical report, 2017."},{"key":"e_1_2_1_10_1","volume-title":"Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems (TOCS), 26(2):1--26","author":"Chang F.","year":"2008","unstructured":"F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber. Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems (TOCS), 26(2):1--26, 2008."},{"key":"e_1_2_1_11_1","first-page":"124","volume-title":"2004 IEEE International Conference on Mobile Ad-hoc and Sensor Systems","author":"Chen H.","year":"2004","unstructured":"H. Chen, J. Li, and P. Mohapatra. Race: time series compression with rate adaptivity and error bound for sensor networks. In 2004 IEEE International Conference on Mobile Ad-hoc and Sensor Systems, pages 124--133, Oct 2004."},{"key":"e_1_2_1_12_1","unstructured":"B. S. Chris Larsen. OpenTSDB - a distributed scalable monitoring system. http:\/\/opentsdb.net."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491245"},{"key":"e_1_2_1_14_1","volume-title":"Regular expression matching with a trigram index","author":"Cox R.","year":"2012","unstructured":"R. Cox. Regular expression matching with a trigram index, 2012. https:\/\/swtch.com\/~rsc\/regexp\/regexp4.html."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-28534-9_16"},{"key":"e_1_2_1_16_1","volume-title":"Monitoring with Graphite: Tracking Dynamic Host and Application Metrics at Scale","author":"Dixon J.","year":"2017","unstructured":"J. Dixon. Monitoring with Graphite: Tracking Dynamic Host and Application Metrics at Scale. O'Reilly Media, 1 edition, March 2017."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2379776.2379788"},{"key":"e_1_2_1_18_1","unstructured":"A. Fikes. Storage architecture and challenges. https:\/\/cloud.google.com\/files\/storage_architecture_and_challenges.pdf."},{"key":"e_1_2_1_19_1","volume-title":"HBase: The Definitive Guide","author":"George L.","year":"2011","unstructured":"L. George. HBase: The Definitive Guide. O'Reilly Media, 1 edition, 2011."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/945445.945450"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2011.389"},{"key":"e_1_2_1_22_1","unstructured":"Google. Snappy --- a fast compressor\/decompressor. https:\/\/github.com\/google\/snappy."},{"key":"e_1_2_1_23_1","series-title":"Developments in Water Science","volume-title":"Time series modelling of water resources and environmental systems","author":"Hipel K. W.","year":"1994","unstructured":"K. W. Hipel and A. I. McLeod. Time series modelling of water resources and environmental systems, volume 45 of Developments in Water Science. Elsevier, 1994."},{"key":"e_1_2_1_24_1","first-page":"613","volume-title":"Proceedings of the 2015 USENIX Annual Technical Conference","author":"Huang J.","year":"2015","unstructured":"J. Huang, A. Badam, R. Chandra, and E. B. Nightingale. Weardrive: Fast and energy-efficient storage for wearables. In Proceedings of the 2015 USENIX Annual Technical Conference, pages 613--625. USENIX Association, 2015."},{"key":"e_1_2_1_25_1","volume-title":"a high performance, in memory time series storage engine","author":"F. Inc. Beringei","year":"2016","unstructured":"F. Inc. Beringei: a high performance, in memory time series storage engine, 2016. https:\/\/github.com\/facebookarchive\/beringei."},{"key":"e_1_2_1_26_1","volume-title":"gRPC: Bidirectional streaming RPC","author":"G. Inc.","year":"2017","unstructured":"G. Inc. gRPC: Bidirectional streaming RPC, 2017. https:\/\/grpc.io\/docs\/guides\/concepts\/."},{"key":"e_1_2_1_27_1","unstructured":"InfluxData. InfluxDB --- open source time series metrics and analytics database. http:\/\/influxdata.com."},{"key":"e_1_2_1_28_1","first-page":"256","volume-title":"PVLDB","author":"Ioannidis Y. E.","year":"1993","unstructured":"Y. E. Ioannidis. Universality of serial histograms. PVLDB, pages 256--267, 1993."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2017.2740932"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.14778\/3236187.3236215"},{"issue":"10","key":"e_1_2_1_31_1","first-page":"797","article-title":"M4: A visualization-oriented time series data aggregation","volume":"7","author":"Jugel U.","year":"2014","unstructured":"U. Jugel, Z. Jerzak, G. Hackenbroich, and V. Markl. M4: A visualization-oriented time series data aggregation. PVLDB, 7(10):797--808, 2014.","journal-title":"PVLDB"},{"issue":"6","key":"e_1_2_1_32_1","first-page":"677","article-title":"Coconut: A scalable bottom-up approach for building data series indexes","volume":"11","author":"Kondylakis H.","year":"2018","unstructured":"H. Kondylakis, N. Dayan, K. Zoumpatianos, and T. Palpanas. Coconut: A scalable bottom-up approach for building data series indexes. PVLDB, 11(6):677--690, 2018.","journal-title":"PVLDB"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2003.1260811"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2005.01.025"},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the 8th Biennial Conference on Innovative Data Systems Research","author":"Meehan J.","year":"2017","unstructured":"J. Meehan, C. Aslantas, S. Zdonik, N. Tatbul, and J. Du. Data ingestion for the connected world. In Proceedings of the 8th Biennial Conference on Innovative Data Systems Research, Jan. 2017."},{"key":"e_1_2_1_36_1","volume-title":"CO","author":"Merchant A.","year":"2014","unstructured":"A. Merchant. Keynote address II: Optimal flash partitioning for storage workloads in google's colossus file system. Broomfield, CO, Oct. 2014. USENIX Association. The 2nd workshop on interactions of NVM\/Flash with operating systems and workloads."},{"issue":"12","key":"e_1_2_1_37_1","first-page":"1816","article-title":"Gorilla: A fast, scalable, in-memory time series database","volume":"8","author":"Pelkonen T.","year":"2015","unstructured":"T. Pelkonen, S. Franklin, J. Teller, P. Cavallaro, Q. Huang, J. Meza, and K. Veeraraghavan. Gorilla: A fast, scalable, in-memory time series database. PVLDB, 8(12):1816--1827, 2015.","journal-title":"PVLDB"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339576"},{"key":"e_1_2_1_39_1","unstructured":"F. Reinartz J. Volz and B. Rabenstein. Prometheus --- monitoring system & time series database. http:\/\/prometheus.io\/."},{"issue":"12","key":"e_1_2_1_40_1","first-page":"1835","article-title":"F1 query: Declarative querying at scale","volume":"11","author":"Samwel B.","year":"2018","unstructured":"B. Samwel, J. Cieslewicz, B. Handy, J. Govig, P. Venetis, C. Yang, K. Peters, J. Shute, D. Tenedorio, H. Apte, and et al. F1 query: Declarative querying at scale. PVLDB, 11(12):1835--1848, 2018.","journal-title":"PVLDB"},{"key":"e_1_2_1_41_1","volume-title":"Google","author":"Sigelman B. H.","year":"2010","unstructured":"B. H. Sigelman, L. A. Barroso, M. Burrows, P. Stephenson, M. Plakal, D. Beaver, S. Jaspan, and C. Shanbhag. Dapper, a large-scale distributed systems tracing infrastructure. Technical report, Google, Inc., 2010."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJSNET.2006.012035"},{"key":"e_1_2_1_43_1","first-page":"799","volume-title":"PVLDB","author":"Tatbul N.","year":"2006","unstructured":"N. Tatbul and S. Zdonik. Window-aware load shedding for aggregation queries over data streams. PVLDB, pages 799--810, 2006."},{"key":"e_1_2_1_44_1","volume-title":"World Scientific","author":"Taylor S. J.","year":"2007","unstructured":"S. J. Taylor. Modelling Financial Time Series. World Scientific, second edition, 2007."},{"key":"e_1_2_1_45_1","unstructured":"The Linux man-pages project. cgroups --- Linux control groups. http:\/\/man7.org\/linux\/man-pages\/man7\/cgroups.7.html."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2741948.2741964"},{"key":"e_1_2_1_47_1","first-page":"107","volume-title":"Site Reliability Engineering","author":"Wilkinson J.","year":"2016","unstructured":"J. Wilkinson. Practical alerting from time-series data. In B. Beyer, C. Jones, J. Petoff, and N. R. Murphy, editors, Site Reliability Engineering, pages 107--123. O'Reilly Media, 2016."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CloudCom.2012.6427510"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3181-3194","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,17]],"date-time":"2025-09-17T02:13:00Z","timestamp":1758075180000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3181-3194"}},"subtitle":["Google's planet-scale in-memory time series database"],"short-title":[],"issued":{"date-parts":[[2020,8]]},"references-count":48,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2020,8]]}},"alternative-id":["10.14778\/3181-3194"],"URL":"https:\/\/doi.org\/10.14778\/3181-3194","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2020,8]]}}}