{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T22:44:50Z","timestamp":1777675490668,"version":"3.51.4"},"reference-count":27,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2018,11,15]],"date-time":"2018-11-15T00:00:00Z","timestamp":1542240000000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/www.sagepub.com\/licence-information-for-chorus"}],"funder":[{"DOI":"10.13039\/100006192","name":"Advanced Scientific Computing Research","doi-asserted-by":"publisher","award":["82707"],"award-info":[{"award-number":["82707"]}],"id":[{"id":"10.13039\/100006192","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2019,3]]},"abstract":"<jats:p>Because of the vast volume of data being produced by today\u2019s scientific simulations and experiments, lossy data compressor allowing user-controlled loss of accuracy during the compression is a relevant solution for significantly reducing the data size. However, lossy compressor developers and users are missing a tool to explore the features of scientific data sets and understand the data alteration after compression in a systematic and reliable way. To address this gap, we have designed and implemented a generic framework called Z-checker. On the one hand, Z-checker combines a battery of data analysis components for data compression. On the other hand, Z-checker is implemented as an open-source community tool to which users and developers can contribute and add new analysis components based on their additional analysis demands. In this article, we present a survey of existing lossy compressors. Then, we describe the design framework of Z-checker, in which we integrated evaluation metrics proposed in prior work as well as other analysis tools. Specifically, for lossy compressor developers, Z-checker can be used to characterize critical properties (such as entropy, distribution, power spectrum, principal component analysis, and autocorrelation) of any data set to improve compression strategies. For lossy compression users, Z-checker can detect the compression quality (compression ratio and bit rate) and provide various global distortion analysis comparing the original data with the decompressed data (peak signal-to-noise ratio, normalized mean squared error, rate\u2013distortion, rate-compression error, spectral, distribution, and derivatives) and statistical analysis of the compression error (maximum, minimum, and average error; autocorrelation; and distribution of errors). Z-checker can perform the analysis with either coarse granularity (throughout the whole data set) or fine granularity (by user-defined blocks), such that the users and developers can select the best fit, adaptive compressors for different parts of the data set. Z-checker features a visualization interface displaying all analysis results in addition to some basic views of the data sets such as time series. To the best of our knowledge, Z-checker is the first tool designed to assess lossy compression comprehensively for scientific data sets.<\/jats:p>","DOI":"10.1177\/1094342017737147","type":"journal-article","created":{"date-parts":[[2017,11,15]],"date-time":"2017-11-15T06:15:46Z","timestamp":1510726546000},"page":"285-303","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":46,"title":["Z-checker: A framework for assessing lossy compression of scientific data"],"prefix":"10.1177","volume":"33","author":[{"given":"Dingwen","family":"Tao","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, University of California, Riverside, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sheng","family":"Di","sequence":"additional","affiliation":[{"name":"Division of Computer Science and Mathematics, Argonne National Laboratory, Lemont, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hanqi","family":"Guo","sequence":"additional","affiliation":[{"name":"Division of Computer Science and Mathematics, Argonne National Laboratory, Lemont, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zizhong","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of California, Riverside, CA, USA"},{"name":"Beijing University of Technology, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Franck","family":"Cappello","sequence":"additional","affiliation":[{"name":"Division of Computer Science and Mathematics, Argonne National Laboratory, Lemont, IL, USA"},{"name":"Parallel Computing Institute, University of Illinois Urbana\u2013Champaign, Champaign, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2017,11,15]]},"reference":[{"key":"bibr1-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1080\/08940886.2016.1149431"},{"key":"bibr2-1094342017737147","first-page":"912","volume-title":"Parallel and Distributed Processing Symposium, 2016 IEEE International","author":"Austin W","year":"2015"},{"key":"bibr3-1094342017737147","doi-asserted-by":"crossref","unstructured":"Baker AH, Hammerling DM, Mickelson SA, (2016) Evaluating lossy data compression on climate simulation data within a large ensemble. Geoscientific Model Development 9(12): 4381.","DOI":"10.5194\/gmd-9-4381-2016"},{"key":"bibr4-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1145\/2600212.2600217"},{"key":"bibr5-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2004.842745"},{"key":"bibr6-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2014.65"},{"key":"bibr8-1094342017737147","unstructured":"Community Earth Simulation Model (CESM) (2017) Available at: http:\/\/www.cesm.ucar.edu\/ (accessed 28 October 2017)."},{"key":"bibr9-1094342017737147","doi-asserted-by":"crossref","unstructured":"Deutsch LP (1996) Gzip file format specification version 4.3.","DOI":"10.17487\/rfc1952"},{"key":"bibr10-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2016.11"},{"key":"bibr11-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1038\/nclimate2915"},{"key":"bibr12-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1145\/3015569"},{"key":"bibr13-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1109\/JRPROC.1952.273898"},{"key":"bibr14-1094342017737147","volume-title":"Gnuplot in Action","author":"Janert PK","year":"2009"},{"key":"bibr15-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.2887"},{"key":"bibr16-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1155\/2014\/835419"},{"key":"bibr17-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346458"},{"issue":"5","key":"bibr18-1094342017737147","first-page":"1245","volume":"12","author":"Lindstrom P","year":"2006","journal-title":"TVCG"},{"key":"bibr19-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1109\/DCC.2006.35"},{"key":"bibr20-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2015.67"},{"key":"bibr21-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2017.115"},{"key":"bibr22-1094342017737147","volume-title":"JPEG2000 Image Compression Fundamentals, Standards and Practice: Image Compression Fundamentals, Standards and Practice","volume":"642","author":"Taubman D","year":"2012"},{"key":"bibr23-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1109\/30.125072"},{"key":"bibr24-1094342017737147","volume-title":"Adaptive Compression and Decompression of Bandlimited Signals","author":"Wegener AW"},{"key":"bibr25-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1098\/rspa.2003.1221"},{"key":"bibr26-1094342017737147","unstructured":"ZFP and Derivatives (2016) Available at: http:\/\/computation.llnl.gov\/projects\/floating-point-compression\/zfp-and-derivatives (accessed 28 October 2017)."},{"key":"bibr27-1094342017737147","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1977.1055714"},{"key":"bibr27a-1094342017737147","first-page":"1","author":"Zuras D","year":"2008","journal-title":"IEEE Std 754-2008"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342017737147","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/1094342017737147","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342017737147","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342017737147","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T08:15:39Z","timestamp":1777450539000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1094342017737147"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,11,15]]},"references-count":27,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2019,3]]}},"alternative-id":["10.1177\/1094342017737147"],"URL":"https:\/\/doi.org\/10.1177\/1094342017737147","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"value":"1094-3420","type":"print"},{"value":"1741-2846","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,11,15]]}}}