{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T15:32:25Z","timestamp":1775230345039,"version":"3.50.1"},"reference-count":28,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,3,8]],"date-time":"2021-03-08T00:00:00Z","timestamp":1615161600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGMOD Rec."],"published-print":{"date-parts":[[2021,3,8]]},"abstract":"<jats:p>Observability has been gaining importance as a key capability in today's large-scale software systems and services. Motivated by current experience in industry exemplified by Slack and as a call to arms for database research, this paper outlines the challenges and opportunities involved in designing and building Observability Data Management Systems (ODMSs) to handle this emerging workload at scale.<\/jats:p>","DOI":"10.1145\/3456859.3456863","type":"journal-article","created":{"date-parts":[[2021,3,11]],"date-time":"2021-03-11T07:04:51Z","timestamp":1615446291000},"page":"18-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":39,"title":["Towards Observability Data Management at Scale"],"prefix":"10.1145","volume":"49","author":[{"given":"Suman","family":"Karumuri","sequence":"first","affiliation":[{"name":"Slack Technologies"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Franco","family":"Solleza","sequence":"additional","affiliation":[{"name":"Brown University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stan","family":"Zdonik","sequence":"additional","affiliation":[{"name":"Brown University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nesime","family":"Tatbul","sequence":"additional","affiliation":[{"name":"Intel Labs and MIT"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,3,10]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Cloud-Native File Systems. In USENIX Conference on Hot Topics in Cloud Computing (HotCloud).","author":"R.","unstructured":"R. H. Arpaci-Dusseau et al. 2018 . Cloud-Native File Systems. In USENIX Conference on Hot Topics in Cloud Computing (HotCloud). R. H. Arpaci-Dusseau et al. 2018. Cloud-Native File Systems. In USENIX Conference on Hot Topics in Cloud Computing (HotCloud)."},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","unstructured":"C. Chan etal 2020. Debugging Incidents in Google's Distributed Systems. ACM Queue 18 2 (2020).  C. Chan et al. 2020. Debugging Incidents in Google's Distributed Systems. ACM Queue 18 2 (2020).","DOI":"10.1145\/3400899.3404974"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2814710.2814713"},{"key":"e_1_2_1_4_1","volume-title":"Elasticsearch: The Definitive Guide","author":"Gormley C.","year":"2015","unstructured":"C. Gormley 2015 . Elasticsearch: The Definitive Guide . O'Reilly Media . C. Gormley et al. 2015. Elasticsearch: The Definitive Guide. O'Reilly Media."},{"key":"e_1_2_1_5_1","unstructured":"M. Hausenblas etal 2017. Lambda Architecture. http:\/\/lambdaarchitecture. net.  M. Hausenblas et al. 2017. Lambda Architecture. http:\/\/lambdaarchitecture. net."},{"key":"e_1_2_1_6_1","volume-title":"Kafka: The Definitive Guide Real-Time Data and Stream Processing at Scale","author":"Narkhede N.","year":"2017","unstructured":"N. Narkhede 2017 . Kafka: The Definitive Guide Real-Time Data and Stream Processing at Scale . O'Reilly Media . N. Narkhede et al. 2017. Kafka: The Definitive Guide Real-Time Data and Stream Processing at Scale. O'Reilly Media."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/13677.22723"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132749"},{"key":"e_1_2_1_9_1","unstructured":"R. Katkov. 2020. All Hands on Deck. https:\/\/slack.engineering\/allhands- on-deck-91d6986c3ee.  R. Katkov. 2020. All Hands on Deck. https:\/\/slack.engineering\/allhands- on-deck-91d6986c3ee."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/359545.359563"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2815400.2815415"},{"key":"e_1_2_1_12_1","unstructured":"S. More. 2018. A Practical Observability Primer. mStakx.  S. More. 2018. A Practical Observability Primer. mStakx."},{"key":"e_1_2_1_13_1","doi-asserted-by":"crossref","unstructured":"S. Niedermaier etal 2019. On Observability and Monitoring of Distributed Systems -- An Industry Interview Study. In ICSOC.  S. Niedermaier et al. 2019. On Observability and Monitoring of Distributed Systems -- An Industry Interview Study. In ICSOC.","DOI":"10.1007\/978-3-030-33702-5_3"},{"key":"e_1_2_1_14_1","unstructured":"OpenTelemetry. 2019. The OpenTelemetry Open-Source Observability Framework. https:\/\/opentelemetry.io\/.  OpenTelemetry. 2019. The OpenTelemetry Open-Source Observability Framework. https:\/\/opentelemetry.io\/."},{"key":"e_1_2_1_15_1","unstructured":"J. O'Shea. 2020. Building Dashboards for Operational Visibility. https:\/\/aws.amazon.com\/builders-library\/building-dashboards-foroperational- visibility\/.  J. O'Shea. 2020. Building Dashboards for Operational Visibility. https:\/\/aws.amazon.com\/builders-library\/building-dashboards-foroperational- visibility\/."},{"key":"e_1_2_1_16_1","volume-title":"Hybrid Transactional\/Analytical Processing: A Survey. In ACM SIGMOD Conference. 1771--1775","author":"F.","unstructured":"F. \u00d6zcan et al. 2017 . Hybrid Transactional\/Analytical Processing: A Survey. In ACM SIGMOD Conference. 1771--1775 . F. \u00d6zcan et al. 2017. Hybrid Transactional\/Analytical Processing: A Survey. In ACM SIGMOD Conference. 1771--1775."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2814710.2814719"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377391.3377400"},{"key":"e_1_2_1_19_1","volume-title":"Pinterest Secor: A Service for Implementing Kafka Log Persistence. https:\/\/github.com\/pinterest\/secor.","year":"2017","unstructured":"Pinterest. 2017 . Pinterest Secor: A Service for Implementing Kafka Log Persistence. https:\/\/github.com\/pinterest\/secor. Pinterest. 2017. Pinterest Secor: A Service for Implementing Kafka Log Persistence. https:\/\/github.com\/pinterest\/secor."},{"key":"e_1_2_1_20_1","unstructured":"Prometheus. 2012. Prometheus Documentation. https:\/\/prometheus.io\/ docs\/concepts\/metric_types\/.  Prometheus. 2012. Prometheus Documentation. https:\/\/prometheus.io\/ docs\/concepts\/metric_types\/."},{"key":"e_1_2_1_21_1","volume-title":"ACM Middleware Conference. 14-- 27","author":"Rodrigues J.","year":"2017","unstructured":"J. Rodrigues 2017 . Sieve: Actionable Insights from Monitored Metrics in Distributed Systems . In ACM Middleware Conference. 14-- 27 . J. Rodrigues et al. 2017. Sieve: Actionable Insights from Monitored Metrics in Distributed Systems. In ACM Middleware Conference. 14-- 27."},{"key":"e_1_2_1_22_1","volume-title":"Presto: SQL on Everything","author":"Sethi R.","year":"2019","unstructured":"R. Sethi 2019 . Presto: SQL on Everything . In IEEE ICDE. R. Sethi et al. 2019. Presto: SQL on Everything. In IEEE ICDE."},{"key":"e_1_2_1_23_1","volume-title":"Mastering Distributed Tracing: Analyzing Performance in Microservices and Complex Systems","author":"Shkuro Y.","unstructured":"Y. Shkuro . 2019. Mastering Distributed Tracing: Analyzing Performance in Microservices and Complex Systems . Packt Publishing . Y. Shkuro. 2019. Mastering Distributed Tracing: Analyzing Performance in Microservices and Complex Systems. Packt Publishing."},{"key":"e_1_2_1_24_1","volume-title":"Dapper: A Large-Scale Distributed Systems Tracing Infrastructure. Technical Report. Google","author":"Sigelman B. H.","year":"2010","unstructured":"B. H. Sigelman 2010 . Dapper: A Large-Scale Distributed Systems Tracing Infrastructure. Technical Report. Google , Inc . B. H. Sigelman et al. 2010. Dapper: A Large-Scale Distributed Systems Tracing Infrastructure. Technical Report. Google, Inc."},{"key":"e_1_2_1_25_1","volume-title":"Distributed Systems Observability: A Guide to Building Robust Systems","author":"Sridharan C.","unstructured":"C. Sridharan . 2018. Distributed Systems Observability: A Guide to Building Robust Systems . O'Reilly Media . C. Sridharan. 2018. Distributed Systems Observability: A Guide to Building Robust Systems. O'Reilly Media."},{"key":"e_1_2_1_26_1","doi-asserted-by":"crossref","unstructured":"D. Vohra. 2016. Apache Parquet. In Practical Hadoop Ecosystem: A Definitive Guide to Hadoop-Related Frameworks and Tools. 325--335.  D. Vohra. 2016. Apache Parquet. In Practical Hadoop Ecosystem: A Definitive Guide to Hadoop-Related Frameworks and Tools. 325--335.","DOI":"10.1007\/978-1-4842-2199-0_8"},{"key":"e_1_2_1_27_1","doi-asserted-by":"crossref","unstructured":"A. Wiedemann etal 2019. The DevOps Phenomenon. ACM Queue 17 2 (2019).  A. Wiedemann et al. 2019. The DevOps Phenomenon. ACM Queue 17 2 (2019).","DOI":"10.1145\/3329781.3338532"},{"key":"e_1_2_1_28_1","volume-title":"Spark: Cluster Computing withWorking Sets. In USENIX Conference on Hot Topics in Cloud Computing (HotCloud).","author":"Zaharia M.","year":"2010","unstructured":"M. Zaharia 2010 . Spark: Cluster Computing withWorking Sets. In USENIX Conference on Hot Topics in Cloud Computing (HotCloud). M. Zaharia et al. 2010. Spark: Cluster Computing withWorking Sets. In USENIX Conference on Hot Topics in Cloud Computing (HotCloud)."}],"container-title":["ACM SIGMOD Record"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3456859.3456863","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3456859.3456863","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:26Z","timestamp":1750195886000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3456859.3456863"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,8]]},"references-count":28,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,3,8]]}},"alternative-id":["10.1145\/3456859.3456863"],"URL":"https:\/\/doi.org\/10.1145\/3456859.3456863","relation":{},"ISSN":["0163-5808"],"issn-type":[{"value":"0163-5808","type":"print"}],"subject":[],"published":{"date-parts":[[2021,3,8]]},"assertion":[{"value":"2021-03-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}