{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,11]],"date-time":"2026-07-11T15:43:42Z","timestamp":1783784622503,"version":"3.55.0"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,3,12]],"date-time":"2024-03-12T00:00:00Z","timestamp":1710201600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"DOE U.S. Department of Energy","award":["ASCR and 17-SC-20-SC"],"award-info":[{"award-number":["ASCR and 17-SC-20-SC"]}]},{"name":"NSF","award":["OAC-2003709, OAC-2104023, OAC-2311875, OAC-2311877, and OAC-2153451"],"award-info":[{"award-number":["OAC-2003709, OAC-2104023, OAC-2311875, OAC-2311877, and OAC-2153451"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2024,3,12]]},"abstract":"<jats:p>Error-bounded lossy compression has been identified as a promising solution for significantly reducing scientific data volumes upon users' requirements on data distortion. For the existing scientific error-bounded lossy compressors, some of them (such as SPERR and FAZ) can reach fairly high compression ratios and some others (such as SZx, SZ, and ZFP) feature high compression speeds, but they rarely exhibit both high ratio and high speed meanwhile. In this paper, we propose HPEZ with newly-designed interpolations and quality-metric-driven auto-tuning, which features significantly improved compression quality upon the existing high-performance compressors, meanwhile being exceedingly faster than high-ratio compressors. The key contributions lie as follows: (1) We develop a series of advanced techniques such as interpolation re-ordering, multi-dimensional interpolation, and natural cubic splines to significantly improve compression qualities with interpolation-based data prediction. (2) The auto-tuning module in HPEZ has been carefully designed with novel strategies, including but not limited to block-wise interpolation tuning, dynamic dimension freezing, and Lorenzo tuning. (3) We thoroughly evaluate HPEZ compared with many other compressors on six real-world scientific datasets. Experiments show that HPEZ outperforms other high-performance error-bounded lossy compressors in compression ratio by up to 140% under the same error bound, and by up to 360% under the same PSNR. In parallel data transfer experiments on the distributed database, HPEZ achieves a significant performance gain with up to 40% time cost reduction over the second-best compressor.<\/jats:p>","DOI":"10.1145\/3639259","type":"journal-article","created":{"date-parts":[[2024,3,26]],"date-time":"2024-03-26T18:51:32Z","timestamp":1711479092000},"page":"1-27","source":"Crossref","is-referenced-by-count":27,"title":["High-performance Effective Scientific Error-bounded Lossy Compression with Auto-tuned Multi-component Interpolation"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0177-502X","authenticated-orcid":false,"given":"Jinyang","family":"Liu","sequence":"first","affiliation":[{"name":"University of California, Riverside, Riverside, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7339-5256","authenticated-orcid":false,"given":"Sheng","family":"Di","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5328-3962","authenticated-orcid":false,"given":"Kai","family":"Zhao","sequence":"additional","affiliation":[{"name":"Florida State University, Tallahassee, FL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0630-1600","authenticated-orcid":false,"given":"Xin","family":"Liang","sequence":"additional","affiliation":[{"name":"University of Kentucky, Lexington, KY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-9250-0611","authenticated-orcid":false,"given":"Sian","family":"Jin","sequence":"additional","affiliation":[{"name":"Indiana University, Bloomington, IN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-2079-8130","authenticated-orcid":false,"given":"Zizhe","family":"Jian","sequence":"additional","affiliation":[{"name":"University of California, Riverside, Riverside, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5092-3987","authenticated-orcid":false,"given":"Jiajun","family":"Huang","sequence":"additional","affiliation":[{"name":"University of California, Riverside, Riverside, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8637-3307","authenticated-orcid":false,"given":"Shixun","family":"Wu","sequence":"additional","affiliation":[{"name":"University of California, Riverside, Riverside, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2578-4940","authenticated-orcid":false,"given":"Zizhong","family":"Chen","sequence":"additional","affiliation":[{"name":"University of California, Riverside, Riverside, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7890-3934","authenticated-orcid":false,"given":"Franck","family":"Cappello","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,3,26]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"[n. d.]. Miranda application. https:\/\/wci.llnl.gov\/simulation\/computer-codes\/miranda"},{"key":"e_1_2_2_2_1","unstructured":"[n. d.]. NSTX-GPI. https:\/\/w3.pppl.gov\/~szweben\/NSTX%20Blob%20Library\/NSTXblobs.html"},{"key":"e_1_2_2_3_1","unstructured":"[n. d.]. Scalable Computing for Advanced Library and Environment (SCALE) -- LETKF. https:\/\/github.com\/gylien\/scale-letkf."},{"key":"e_1_2_2_4_1","unstructured":"[n. d.]. SEGSalt. https:\/\/wiki.seg.org\/wiki\/SEG\/EAGE_Salt_and_Overthrust_Models."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00791-018-00303-9"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.3262"},{"key":"e_1_2_2_7_1","volume-title":"TTHRESH: Tensor compression for multidimensional visual data","author":"Ballester-Ripoll Rafael","year":"2019","unstructured":"Rafael Ballester-Ripoll, Peter Lindstrom, and Renato Pajarola. 2019. TTHRESH: Tensor compression for multidimensional visual data. IEEE transactions on visualization and computer graphics 26, 9 (2019), 2891--2903."},{"key":"e_1_2_2_8_1","volume-title":"arXiv preprint arXiv:2003.05991","author":"Bank Dor","year":"2020","unstructured":"Dor Bank, Noam Koenigstein, and Raja Giryes. 2020. Autoencoders. arXiv preprint arXiv:2003.05991 (2020)."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1177\/1094342019853336"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/eScience.2015.68"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2949550.2949554"},{"key":"e_1_2_2_12_1","unstructured":"Yann Collet. 2015. Zstandard -- Real-time data compression algorithm. http:\/\/facebook.github.io\/zstd\/ (2015)."},{"key":"e_1_2_2_13_1","volume-title":"2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, 696--707","author":"Fang Ziquan","year":"2021","unstructured":"Ziquan Fang, Yuntao Du, Lu Chen, Yujia Hu, Yunjun Gao, and Gang Chen. 2021. E 2 dtc: An end to end deep trajectory clustering framework via self-training. In 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, 696--707."},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevFluids.5.114602"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.newast.2015.06.003"},{"key":"e_1_2_2_16_1","volume-title":"Coordnet: Data generation and visualization generation for time-varying volumes via a coordinate-based neural network","author":"Han Jun","year":"2022","unstructured":"Jun Han and Chaoli Wang. 2022. Coordnet: Data generation and visualization generation for time-varying volumes via a coordinate-based neural network. IEEE Transactions on Visualization and Computer Graphics (2022)."},{"key":"e_1_2_2_17_1","volume-title":"Using Neural Networks for Two Dimensional Scientific Data Compression. In 2021 IEEE International Conference on Big Data (Big Data). IEEE, 2956--2965","author":"Hayne Lucas","year":"2021","unstructured":"Lucas Hayne, John Clyne, and Shaomeng Li. 2021. Using Neural Networks for Two Dimensional Scientific Data Compression. In 2021 IEEE International Conference on Big Data (Big Data). IEEE, 2956--2965."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.14778\/3236187.3236215"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.14778\/3574245.3574255"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1175\/BAMS-D-13-00255.1"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.3997\/1365-2397.fb2020015"},{"key":"e_1_2_2_22_1","volume-title":"Torben Bach Pedersen, and Christian Thomsen","author":"Jensen S\u00f8ren Kejser","year":"2019","unstructured":"S\u00f8ren Kejser Jensen, Torben Bach Pedersen, and Christian Thomsen. 2019. Scalable Model-Based Management of Correlated Dimensional Time Series in ModelarDB. arXiv e-prints (2019), arXiv--1903."},{"key":"e_1_2_2_23_1","volume-title":"Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114","author":"Kingma Diederik P","year":"2013","unstructured":"Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)."},{"key":"e_1_2_2_24_1","volume-title":"International Conference on Learning Representations.","author":"Kolouri Soheil","year":"2018","unstructured":"Soheil Kolouri, Phillip E Pope, Charles E Martin, and Gustavo K Rohde. 2018. Sliced Wasserstein auto-encoders. In International Conference on Learning Representations."},{"key":"e_1_2_2_25_1","volume-title":"Samatova","author":"Lakshminarasimhan Sriram","year":"2011","unstructured":"Sriram Lakshminarasimhan, Neil Shah, Stephane Ethier, Scott Klasky, Rob Latham, Rob Ross, and Nagiza F. Samatova. 2011. Compressing the Incompressible with ISABELA: In-situ Reduction of Spatio-temporal Data. In Euro-Par 2011 Parallel Processing, Emmanuel Jeannot, Raymond Namyst, and Jean Roman (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 366--379."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3458817.3476195"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS54959.2023.00104"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2018.00062"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1080\/14685240802376389"},{"key":"e_1_2_2_30_1","volume-title":"Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets. In 2018 IEEE International Conference on Big Data. IEEE.","author":"Liang Xin","year":"2018","unstructured":"Xin Liang, Sheng Di, Dingwen Tao, Sihuan Li, Shaomeng Li, Hanqi Guo, Zizhong Chen, and Franck Cappello. 2018. Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets. In 2018 IEEE International Conference on Big Data. IEEE."},{"key":"e_1_2_2_31_1","volume-title":"et al","author":"Liang Xin","year":"2021","unstructured":"Xin Liang, Ben Whitney, Jieyang Chen, Lipeng Wan, Qing Liu, Dingwen Tao, James Kress, David R Pugmire, Matthew Wolf, Norbert Podhorszki, et al . 2021. MGARD: Optimizing multilevel methods for error-bounded scientific data reduction. IEEE Trans. Comput. (2021)."},{"key":"e_1_2_2_32_1","doi-asserted-by":"crossref","unstructured":"Xin Liang Kai Zhao Sheng Di Sihuan Li Robert Underwood Ali M Gok Jiannan Tian Junjing Deng Jon C Calhoun Dingwen Tao et al. 2022. SZ3: A modular framework for composing prediction-based error-bounded lossy compressors. IEEE Transactions on Big Data (2022).","DOI":"10.1109\/TBDATA.2022.3201176"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346458"},{"key":"e_1_2_2_34_1","volume-title":"Exploring Autoencoder-based Error-bounded Compression for Scientific Data. In 2021 IEEE International Conference on Cluster Computing (CLUSTER). IEEE, 294--306","author":"Liu Jinyang","year":"2021","unstructured":"Jinyang Liu, Sheng Di, Kai Zhao, Sian Jin, Dingwen Tao, Xin Liang, Zizhong Chen, and Franck Cappello. 2021. Exploring Autoencoder-based Error-bounded Compression for Scientific Data. In 2021 IEEE International Conference on Cluster Computing (CLUSTER). IEEE, 294--306."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC41404.2022.00067"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3577193.3593721"},{"key":"e_1_2_2_37_1","volume-title":"High-Ratio Lossy Compression: Exploring the Autoencoder to Compress Scientific Data","author":"Liu Tong","year":"2021","unstructured":"Tong Liu, Jinzhen Wang, Qing Liu, Shakeel Alibhai, Tao Lu, and Xubin He. 2021. High-Ratio Lossy Compression: Exploring the Autoencoder to Compress Scientific Data. IEEE Transactions on Big Data (2021)."},{"key":"e_1_2_2_38_1","unstructured":"Yuanjian Liu Sheng Di Kyle Chard Ian Foster and Franck Cappello. 2023. Optimizing Scientific Data Transfer on Globus with Error-bounded Lossy Compression. arXiv:2307.05416 [cs.DC]"},{"key":"e_1_2_2_39_1","volume-title":"Computer Graphics Forum","author":"Lu Yuzhe","unstructured":"Yuzhe Lu, Kairong Jiang, Joshua A Levine, and Matthew Berger. 2021. Compressive neural representations of volumetric scalar fields. In Computer Graphics Forum, Vol. 40. Wiley Online Library, 135--146."},{"key":"e_1_2_2_40_1","first-page":"12","article-title":"Gorilla: A Fast, Scalable, in-Memory Time Series Database","volume":"8","author":"Tuomas Pelkonen","year":"2015","unstructured":"Tuomas Pelkonen et al. 2015. Gorilla: A Fast, Scalable, in-Memory Time Series Database. Proc. VLDB Endow. 8, 12 (Aug. 2015), 1816--1827.","journal-title":"Proc. VLDB Endow."},{"key":"e_1_2_2_41_1","volume-title":"Exascale scientific applications: Scalability and performance portability","author":"Straatsma Tjerk P","unstructured":"Tjerk P Straatsma, Katerina B Antypas, and Timothy J Williams. 2017. Exascale scientific applications: Scalability and performance portability. CRC Press."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2012.2221191"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1177\/1094342017737147"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1117\/1.1469618"},{"key":"e_1_2_2_45_1","volume-title":"Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques (PACT '20)","author":"Jiannan","unstructured":"Jiannan Tian et al. 2020. CuSZ: An Efficient GPU-Based Error-Bounded Lossy Compression Framework for Scientific Data. In Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques (PACT '20). 3--15."},{"key":"e_1_2_2_46_1","volume-title":"cuSZ (x): Optimizing Error-Bounded Lossy Compression for Scientific Data on GPUs. CoRR","author":"Tian Jiannan","year":"2021","unstructured":"Jiannan Tian, Sheng Di, Xiaodong Yu, Cody Rivera, Kai Zhao, Sian Jin, Yunhe Feng, Xin Liang, Dingwen Tao, and Franck Cappello. 2021. cuSZ (x): Optimizing Error-Bounded Lossy Compression for Scientific Data on GPUs. CoRR (2021)."},{"key":"e_1_2_2_47_1","volume-title":"Image quality assessment: from error visibility to structural similarity","author":"Wang Zhou","year":"2004","unstructured":"Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600--612."},{"key":"e_1_2_2_48_1","doi-asserted-by":"crossref","unstructured":"Xinyang Yu et al . 2020. Two-Level Data Compression using Machine Learning in Time Series Database. In 36th IEEE International Conference on Data Engineering. 1333--1344.","DOI":"10.1109\/ICDE48307.2020.00119"},{"key":"e_1_2_2_49_1","volume-title":"Dingwen Tao, Xin Liang, and Franck Cappello.","author":"Yu Xiaodong","year":"2022","unstructured":"Xiaodong Yu, Sheng Di, Kai Zhao, jiannan Tian, Dingwen Tao, Xin Liang, and Franck Cappello. 2022. SZx: an Ultra-fast Error-bounded Lossy Compressor for Scientific Datasets. arXiv preprint arXiv:2201.13020 (2022)."},{"key":"e_1_2_2_50_1","volume-title":"FZ-GPU: A Fast and High-Ratio Lossy Compressor for Scientific Computing Applications on GPUs. arXiv preprint arXiv:2304.12557","author":"Zhang Boyuan","year":"2023","unstructured":"Boyuan Zhang, Jiannan Tian, Sheng Di, Xiaodong Yu, Yunhe Feng, Xin Liang, Dingwen Tao, and Franck Cappello. 2023. FZ-GPU: A Fast and High-Ratio Lossy Compressor for Scientific Computing Applications on GPUs. arXiv preprint arXiv:2304.12557 (2023)."},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.14778\/3213880.3213885"},{"key":"e_1_2_2_52_1","volume-title":"MDZ: An Efficient Error-bounded Lossy Compressor for Molecular Dynamics Simulations. In 2022 IEEE 38th International Conference on Data Engineering (ICDE).","author":"Zhao Kai","year":"2022","unstructured":"Kai Zhao, Sheng Di, Perez Danny, Zizhong Chen, and Franck Cappello. 2022. MDZ: An Efficient Error-bounded Lossy Compressor for Molecular Dynamics Simulations. In 2022 IEEE 38th International Conference on Data Engineering (ICDE)."},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE51399.2021.00145"},{"key":"e_1_2_2_54_1","volume-title":"SDRBench: Scientific Data Reduction Benchmark for Lossy Compressors. In 2020 IEEE International Conference on Big Data (Big Data). 2716--2724","author":"Zhao Kai","year":"2020","unstructured":"Kai Zhao, Sheng Di, Xin Lian, Sihuan Li, Dingwen Tao, Julie Bessac, Zizhong Chen, and Franck Cappello. 2020. SDRBench: Scientific Data Reduction Benchmark for Lossy Compressors. In 2020 IEEE International Conference on Big Data (Big Data). 2716--2724."},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3369583.3392688"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639259","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3639259","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T15:18:25Z","timestamp":1755789505000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639259"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,12]]},"references-count":55,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3,12]]}},"alternative-id":["10.1145\/3639259"],"URL":"https:\/\/doi.org\/10.1145\/3639259","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,12]]}}}