{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:58:54Z","timestamp":1760234334740,"version":"build-2065373602"},"reference-count":49,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2021,5,12]],"date-time":"2021-05-12T00:00:00Z","timestamp":1620777600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["20K21794"],"award-info":[{"award-number":["20K21794"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Increasing processing capabilities and input\/output constraints of supercomputers have increased the use of co-processing approaches, i.e., visualizing and analyzing data sets of simulations on the fly. We present a method that evaluates the importance of different regions of simulation data and a data-driven approach that uses the proposed method to accelerate in-transit co-processing of large-scale simulations. We use the importance metrics to simultaneously employ multiple compression methods on different data regions to accelerate the in-transit co-processing. Our approach strives to adaptively compress data on the fly and uses load balancing to counteract memory imbalances. We demonstrate the method\u2019s efficiency through a fluid mechanics application, a Richtmyer\u2013Meshkov instability simulation, showing how to accelerate the in-transit co-processing of simulations. The results show that the proposed method expeditiously can identify regions of interest, even when using multiple metrics. Our approach achieved a speedup of 1.29\u00d7 in a lossless scenario. The data decompression time was sped up by 2\u00d7 compared to using a single compression method uniformly.<\/jats:p>","DOI":"10.3390\/a14050154","type":"journal-article","created":{"date-parts":[[2021,5,12]],"date-time":"2021-05-12T10:59:12Z","timestamp":1620817152000},"page":"154","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Accelerating In-Transit Co-Processing for Scientific Simulations Using Region-Based Data-Driven Analysis"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3000-9439","authenticated-orcid":false,"given":"Marcus","family":"Walld\u00e9n","sequence":"first","affiliation":[{"name":"Graduate School of Information Science and Technology, Osaka University, Suita-shi 565-0871, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Masao","family":"Okita","sequence":"additional","affiliation":[{"name":"Graduate School of Information Science and Technology, Osaka University, Suita-shi 565-0871, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fumihiko","family":"Ino","sequence":"additional","affiliation":[{"name":"Graduate School of Information Science and Technology, Osaka University, Suita-shi 565-0871, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dimitris","family":"Drikakis","sequence":"additional","affiliation":[{"name":"University of Nicosia, Nicosia CY-2417, Cyprus"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ioannis","family":"Kokkinakis","sequence":"additional","affiliation":[{"name":"Department of Mechanical and Aerospace Engineering, University of Strathclyde, Glasgow G1 1XJ, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,5,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1109\/MCG.2016.35","article-title":"The Tensions of In Situ Visualization","volume":"36","author":"Moreland","year":"2016","journal-title":"IEEE Comput. Graph. Appl."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1109\/MCG.2009.120","article-title":"In Situ Visualization at Extreme Scale: Challenges and Opportunities","volume":"29","author":"Ma","year":"2009","journal-title":"IEEE Comput. Graph. Appl."},{"key":"ref_3","unstructured":"Rivi, M., Calori, L., Muscianisi, G., and Slavni\u0107, V. (2021, April 10). In-situ Visualization: State-of-the-Art and Some Use Cases. Available online: https:\/\/www.hpc.cineca.it\/sites\/default\/files\/In-situ_Visualization_State-of-the-art_and_Some_Use_Cases.pdf."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/MCG.2010.55","article-title":"In Situ Visualization for Large-Scale Combustion Simulations","volume":"30","author":"Yu","year":"2010","journal-title":"IEEE Comput. Graph. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Bennett, J.C., Abbasi, H., Bremer, P., Grout, R., Gyulassy, A., Jin, T., Klasky, S., Kolla, H., Parashar, M., and Pascucci, V. (2012, January 24\u201329). Combining in-situ and in-transit processing to enable extreme-scale scientific analysis. Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (SC\u201912), Salt Lake City, UT, USA.","DOI":"10.1109\/SC.2012.31"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40668-016-0017-2","article-title":"In situ and in-transit analysis of cosmological simulations","volume":"3","author":"Friesen","year":"2016","journal-title":"Comput. Astrophys. Cosmol."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Flatken, M., Wagner, C., and Gerndt, A. (2014). Distributed Post-processing and Rendering for Large-Scale Scientific Simulations. Scientific Visualization: Uncertainty, Multifield, Biomedical, and Scalable Visualization, Springer.","DOI":"10.1007\/978-1-4471-6497-5_31"},{"key":"ref_8","unstructured":"Moreland, K., Oldfield, R., Marion, P., Jourdain, S., Podhorszki, N., Vishwanath, V., Fabian, N., Docan, C., Parashar, M., and Hereld, M. (2011, January 14). Examples of in Transit Visualization. Proceedings of the 2nd International Workshop on Petascal Data Analytics: Challenges and Opportunities (PDAC\u201911), Seattle, WA, USA."},{"key":"ref_9","unstructured":"Kress, J. (2020). In-Line vs. In-Transit In Situ: Which Technique to Use at Scale?. [Ph.D. Thesis, University of Oregon]."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Dorier, M., Sisneros, R., Gomez, L.B., Peterka, T., Orf, L., Rahmani, L., Antoniu, G., and Boug\u00e9, L. (2016, January 12\u201316). Adaptive Performance-Constrained In Situ Visualization of Atmospheric Simulations. Proceedings of the 2016 IEEE International Conference on Cluster Computing (CLUSTER), Taipei, Taiwan.","DOI":"10.1109\/CLUSTER.2016.25"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1547","DOI":"10.1109\/TVCG.2008.140","article-title":"Importance-Driven Time-Varying Data Visualization","volume":"14","author":"Wang","year":"2008","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Nouanesengsy, B., Woodring, J., Patchett, J., Myers, K., and Ahrens, J. (2014, January 9\u201310). ADR Visualization: A Generalized Framework for Ranking Large-Scale Scientific Data Using Analysis-Driven Refinement. Proceedings of the 2014 IEEE 4th Symposium on Large Data Analysis and Visualization (LDAV), Paris, France.","DOI":"10.1109\/LDAV.2014.7013203"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1016\/0021-9991(84)90073-1","article-title":"Adaptive mesh refinement for hyperbolic partial differential equations","volume":"53","author":"Berger","year":"1984","journal-title":"J. Comput. Phys."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4815","DOI":"10.1093\/mnras\/sty2586","article-title":"gamer-2: A GPU-accelerated adaptive mesh refinement code\u2014Accuracy, performance, and scalability","volume":"481","author":"Schive","year":"2018","journal-title":"Mon. Not. R. Astron. Soc."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Shimokawabe, T., and Onodera, N. (2019, January 12\u201314). A High-Productivity Framework for Adaptive Mesh Refinement on Multiple GPUs. Proceedings of the International Conference on Computational Science (ICCS 2019), Faro, Portugal.","DOI":"10.1007\/978-3-030-22734-0_21"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Biswas, A., Dutta, S., Pulido, J., and Ahrens, J. (2018, January 12). In Situ Data-Driven Adaptive Sampling for Large-Scale Simulation Data Summarization. Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV\u201918), Dallas, TX, USA.","DOI":"10.1145\/3281464.3281467"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Dutta, S., Biswas, A., and Ahrens, J. (2019). Multivariate Pointwise Information-Driven Data Sampling and Visualization. Entropy, 21.","DOI":"10.3390\/e21070699"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Fabian, N., Moreland, K., Thompson, D., Bauer, A.C., Marion, P., Gevecik, B., Rasquin, M., and Jansen, K.E. (2011, January 23\u201324). The ParaView Coprocessing Library: A scalable, general purpose in situ visualization library. Proceedings of the 2011 IEEE Symposium on Large Data Analysis and Visualization, Providence, RI, USA.","DOI":"10.1109\/LDAV.2011.6092322"},{"key":"ref_19","unstructured":"Childs, H., Brugger, E., Whitlock, B., Meredith, J., Ahern, S., Bonnell, K., Miller, M., Weber, G.H., Harrison, C., and Fogal, T. (2011, January 10\u201314). VisIt: An End-User Tool for Visualizing and Analyzing Very Large Data. Proceedings of the SciDAC, Denver, CO, USA."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"931","DOI":"10.1109\/TVCG.2016.2599041","article-title":"OSPRay\u2014A CPU Ray Tracing Framework for Scientific Visualization","volume":"23","author":"Wald","year":"2017","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/0146-664X(82)90104-6","article-title":"Geometric modeling using octree encoding","volume":"19","author":"Meagher","year":"1982","journal-title":"Comput. Graph. Image Process."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.cpc.2013.08.017","article-title":"An approach to exascale visualization: Interactive viewing of in-situ visualization","volume":"185","author":"Kageyama","year":"2014","journal-title":"Comput. Phys. Commun."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ahrens, J., Jourdain, S., O\u2019Leary, P., Patchett, J., Rogers, D.H., and Petersen, M. (2014, January 16\u201321). An Image-based Approach to Extreme Scale in Situ Visualization and Analysis. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC\u201914), New Orleans, LA, USA.","DOI":"10.1109\/SC.2014.40"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Woodring, J., Ahrens, J., Figg, J., Wendelberger, J., Habib, S., and Heitmann, K. (June, January 31). In-situ Sampling of a Large-Scale Particle Simulation for Interactive Visualization and Analysis. Proceedings of the Eurographics\/IEEE-VGTC Symposium on Visualization (EuroVis 2011), Bergen, Norway.","DOI":"10.1111\/j.1467-8659.2011.01964.x"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Landge, A.G., Pascucci, V., Gyulassy, A., Bennett, J.C., Kolla, H., Chen, J., and Bremer, P. (2014, January 16\u201321). In-Situ Feature Extraction of Large Scale Combustion Simulations Using Segmented Merge Trees. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC\u201914), New Orleans, LA, USA.","DOI":"10.1109\/SC.2014.88"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1307","DOI":"10.1109\/TVCG.2010.253","article-title":"Interactive Exploration and Analysis of Large-Scale Simulations Using Topology-Based Data Segmentation","volume":"17","author":"Bremer","year":"2011","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zou, H., Zheng, F., Wolf, M., Eisenhauer, G., Schwan, K., Abbasi, H., Liu, Q., Podhorszki, N., and Klasky, S. (2012, January 10\u201316). Quality-Aware Data Management for Large Scale Scientific Applications. Proceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, Salt Lake City, UT, USA.","DOI":"10.1109\/SC.Companion.2012.114"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jin, T., Zhang, F., Sun, Q., Bui, H., Parashar, M., Yu, H., Klasky, S., Podhorszki, N., and Abbasi, H. (2013, January 17\u201322). Using Cross-layer Adaptations for Dynamic Data Management in Large Scale Coupled Scientific Workflows. Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (SC\u201913), Denver, CO, USA.","DOI":"10.1145\/2503210.2503301"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Gu, J., Loring, B., Wu, K., and Bethel, E.W. (2019, January 18). HDF5 as a Vehicle for in Transit Data Movement. Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV\u201919), Denver, CO, USA.","DOI":"10.1145\/3364228.3364237"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2674","DOI":"10.1109\/TVCG.2014.2346458","article-title":"Fixed-Rate Compressed Floating-Point Arrays","volume":"20","author":"Lindstrom","year":"2014","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Liang, X., Di, S., Tao, D., Li, S., Li, S., Guo, H., Chen, Z., and Cappello, F. (2018, January 10\u201313). Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.","DOI":"10.1109\/BigData.2018.8622520"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1109\/TIT.1977.1055714","article-title":"A universal algorithm for sequential data compression","volume":"23","author":"Ziv","year":"1977","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Di, S., and Cappello, F. (2016, January 23\u201327). Fast Error-Bounded Lossy HPC Data Compression with SZ. Proceedings of the 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS), Chicago, IL, USA.","DOI":"10.1109\/IPDPS.2016.11"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wang, G., Xu, J., and He, B. (2016, January 12\u201314). A Novel Method for Tuning Configuration Parameters for Spark Based on Machine Learning. Proceedings of the 18th International Conference on High Performance Computing and Communications (HPCC\u201916), Sydney, NSW, Australia.","DOI":"10.1109\/HPCC-SmartCity-DSS.2016.0088"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2306","DOI":"10.1587\/transinf.2019PAP0003","article-title":"Memory Efficient Load Balancing for Distributed Large-Scale Volume Rendering Using a Two-layered Group Structure","volume":"102","author":"Wallden","year":"2019","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_36","unstructured":"Marchesin, S., Mongenet, C., and Dischler, J. (2006, January 6\u20137). Dynamic Load Balancing for Parallel Volume Rendering. Proceedings of the 6th Eurographics Conference on Parallel Graphics and Visualization, Groningen, The Netherlands."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1109\/TVCG.2017.2744059","article-title":"Dynamic Load Balancing Based on Constrained K-D Tree Decomposition for Parallel Particle Tracing","volume":"24","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_38","first-page":"106","article-title":"Prediction-based load balancing and resolution tuning for interactive volume raycasting","volume":"1","author":"Bruder","year":"2017","journal-title":"Vis. Inform."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1016\/j.parco.2006.12.002","article-title":"Adaptive Load Balancing for Raycasting of Non-uniformly Bricked Volumes","volume":"33","author":"Strengert","year":"2007","journal-title":"Parallel Comput."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"846","DOI":"10.1093\/ietisy\/e91-d.3.846","article-title":"An Effective Load Balancing Scheme for 3D Texture-Based Sort-Last Parallel Volume Rendering on GPU Clusters","volume":"91","author":"Lee","year":"2008","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1145\/361002.361007","article-title":"Multidimensional binary search trees used for associative searching","volume":"18","author":"Bentley","year":"1975","journal-title":"Commun. ACM"},{"key":"ref_42","unstructured":"Li, W., Mueller, K., and Kaufman, A. (2003, January 19\u201324). Empty Space Skipping and Occlusion Clipping for Texture-based Volume Rendering. Proceedings of the 14th IEEE Visualization 2003 (VIS\u201903), Seattle, WA, USA."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1145\/78964.78965","article-title":"Efficient Ray Tracing of Volume Data","volume":"9","author":"Levoy","year":"1990","journal-title":"ACM Trans. Graph."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Matsui, M., Ino, F., and Hagihara, K. (2005, January 2\u20135). Parallel Volume Rendering with Early Ray Termination for Visualizing Large-Scale Datasets. Proceedings of the International Symposium on Parallel and Distributed Processing and Applications, Nanjing, China.","DOI":"10.1007\/978-3-540-30566-8_30"},{"key":"ref_45","unstructured":"Cybermedia Center (2019, December 11). Osaka University \u00bb Blog Archive \u00bb OCTOPUS. Available online: http:\/\/www.hpc.cmc.osaka-u.ac.jp\/en\/octopus\/."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"046101","DOI":"10.1063\/1.3576187","article-title":"Richtmyer\u2013Meshkov turbulent mixing arising from an inclined material interface with realistic surface perturbations and reshocked flow","volume":"23","author":"Hahn","year":"2011","journal-title":"Phys. Fluids"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"132459","DOI":"10.1016\/j.physd.2020.132459","article-title":"Vortex morphology in Richtmyer\u2013Meshkov-induced turbulent mixing","volume":"407","author":"Kokkinakis","year":"2020","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"3692","DOI":"10.1063\/1.1504452","article-title":"Three-dimensional simulation of a Richtmyer\u2013Meshkov instability with a two-scale initial perturbation","volume":"14","author":"Cohen","year":"2002","journal-title":"Phys. Fluids"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"034106","DOI":"10.1063\/1.3555635","article-title":"Simulations of Richtmyer\u2013Meshkov instabilities in planar shock-tube experiments","volume":"23","author":"Grinstein","year":"2011","journal-title":"Phys. Fluids"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/5\/154\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:59:37Z","timestamp":1760162377000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/5\/154"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,12]]},"references-count":49,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2021,5]]}},"alternative-id":["a14050154"],"URL":"https:\/\/doi.org\/10.3390\/a14050154","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2021,5,12]]}}}