{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T21:29:34Z","timestamp":1773523774447,"version":"3.50.1"},"reference-count":17,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2016,3,1]],"date-time":"2016-03-01T00:00:00Z","timestamp":1456790400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Queue"],"published-print":{"date-parts":[[2016,3]]},"abstract":"<jats:p>Distributed systems pose unique challenges for software developers. Reasoning about concurrent activities of system nodes and even understanding the system\u2019s communication topology can be difficult. A standard approach to gaining insight into system activity is to analyze system logs. Unfortunately, this can be a tedious and complex process. This article looks at several key features and debugging challenges that differentiate distributed systems from other kinds of software. The article presents several promising tools and ongoing research to help resolve these challenges.<\/jats:p>","DOI":"10.1145\/2927299.2940294","type":"journal-article","created":{"date-parts":[[2020,9,3]],"date-time":"2020-09-03T04:20:10Z","timestamp":1599106810000},"page":"91-110","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":20,"title":["Debugging Distributed Systems"],"prefix":"10.1145","volume":"14","author":[{"given":"Ivan","family":"Beschastnikh","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Patty","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuriy","family":"Brun","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael D,","family":"Ernst","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2016,3,28]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Concurrency Control and Recovery in Database Systems","author":"Bernstein P."},{"key":"e_1_2_1_2_1","volume-title":"10th Usenix Symposium on Operating Systems Design and Implementation; https:\/\/www.usenix.org\/conference\/osdi12\/technical-sessions\/presentation\/corbett.","author":"Corbett J. C.","year":"2012"},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the 26th International Conference on Large Installation System Administration: 33-42; https:\/\/users.ece.cmu.edu\/~spertet\/papers\/hadoopvis-lisa12-cameraready-v3.pdf.","author":"Garduno E.","year":"2012"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the Fourth Usenix Conference on Networked Systems Design and Implementation; https:\/\/www.usenix.org\/legacy\/event\/nsdi07\/tech\/full_papers\/geels\/geels.pdf.","author":"Geels D.","year":"2007"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2815400.2815428"},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the Fourth Usenix Conference on Networked Systems Design and Implementation; https:\/\/www.usenix.org\/legacy\/event\/nsdi07\/tech\/killian\/killian.pdf.","author":"Killian C.","year":"2007"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the Fifth Usenix Symposium on Networked Systems Design and Implementation: 423-437; http:\/\/static.usenix.org\/event\/nsdi08\/tech\/full_papers\/liu_xuezheng\/liu_xuezheng.pdf.","author":"Liu X.","year":"2008"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2815400.2815415"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the International Workshop on Parallel and Distributed Algorithms; http:\/\/homes.cs.washington.edu\/~arvind\/cs425\/doc\/mattern89virtual.pdf","author":"Mattern F.","year":"1989"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2699417"},{"key":"e_1_2_1_11_1","unstructured":"Project Voldemort; http:\/\/www.project-voldemort.com\/voldemort\/.  Project Voldemort; http:\/\/www.project-voldemort.com\/voldemort\/."},{"key":"e_1_2_1_12_1","unstructured":"Sambasivan R. R. Fonseca R. Shafer I. Ganger G. 2014. So you want to trace your distributed system? Key design insights from years of practical experience. Parallel Data Laboratory Carnegie Mellon University; http:\/\/www.pdl.cmu.edu\/PDL-FTP\/SelfStar\/CMU-PDL-14-102.pdf.  Sambasivan R. R. Fonseca R. Shafer I. Ganger G. 2014. So you want to trace your distributed system? Key design insights from years of practical experience. Parallel Data Laboratory Carnegie Mellon University; http:\/\/www.pdl.cmu.edu\/PDL-FTP\/SelfStar\/CMU-PDL-14-102.pdf."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2619239.2626304"},{"key":"e_1_2_1_14_1","unstructured":"Sigelman B. H. Barroso L. A. Burrows M. Stephenson P. Plakal M. Beaver D. Jaspan S. Shanbhag C. 2010. Dapper a large-scale distributed systems tracing infrastructure. Research at Google; http:\/\/research.google.com\/pubs\/pub36356.html.  Sigelman B. H. Barroso L. A. Burrows M. Stephenson P. Plakal M. Beaver D. Jaspan S. Shanbhag C. 2010. Dapper a large-scale distributed systems tracing infrastructure. Research at Google; http:\/\/research.google.com\/pubs\/pub36356.html."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2737924.2737958"},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the Workshop on Managing Systems via Log Analysis and Machine Learning Techniques; http:\/\/iiis.tsinghua.edu.cn\/~weixu\/files\/slaml10","author":"Xu W.","year":"2010"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the Sixth Usenix Symposium on Networked Systems Design and Implementation: 213-228; https:\/\/www.usenix.org\/legacy\/event\/nsdi09\/tech\/full_papers\/yang\/yang_html\/.","author":"Yang J.","year":"2009"}],"container-title":["Queue"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2927299.2940294","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2927299.2940294","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:56:21Z","timestamp":1750222581000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2927299.2940294"}},"subtitle":["Challenges and options for validation and debugging"],"short-title":[],"issued":{"date-parts":[[2016,3]]},"references-count":17,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2016,3]]}},"alternative-id":["10.1145\/2927299.2940294"],"URL":"https:\/\/doi.org\/10.1145\/2927299.2940294","relation":{},"ISSN":["1542-7730","1542-7749"],"issn-type":[{"value":"1542-7730","type":"print"},{"value":"1542-7749","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,3]]},"assertion":[{"value":"2016-03-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}