{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T22:53:55Z","timestamp":1777676035625,"version":"3.51.4"},"reference-count":35,"publisher":"SAGE Publications","issue":"3-4","license":[{"start":{"date-parts":[[2023,4,14]],"date-time":"2023-04-14T00:00:00Z","timestamp":1681430400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"National Science Foundation","award":["#1841758"],"award-info":[{"award-number":["#1841758"]}]},{"name":"National Science Foundation","award":["#2028923"],"award-info":[{"award-number":["#2028923"]}]},{"name":"National Science Foundation","award":["#2103845"],"award-info":[{"award-number":["#2103845"]}]},{"name":"National Science Foundation","award":["#2138811"],"award-info":[{"award-number":["#2138811"]}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2023,7]]},"abstract":"<jats:p>In the era of big data, materials science workflows need to handle large-scale data distribution, storage, and computation. Any of these areas can become a performance bottleneck. We present a framework for analyzing internal material structures (e.g., cracks) to mitigate these bottlenecks. We demonstrate the effectiveness of our framework for a workflow performing synchrotron X-ray computed tomography reconstruction and segmentation of a silica-based structure. Our framework provides a cloud-based, cutting-edge solution to challenges such as growing intermediate and output data and heavy resource demands during image reconstruction and segmentation. Specifically, our framework efficiently manages data storage, scaling up compute resources on the cloud. The multi-layer software structure of our framework includes three layers. A top layer uses Jupyter notebooks and serves as the user interface. A middle layer uses Ansible for resource deployment and managing the execution environment. A low layer is dedicated to resource management and provides resource management and job scheduling on heterogeneous nodes (i.e., GPU and CPU). At the core of this layer, Kubernetes supports resource management, and Dask enables large-scale job scheduling for heterogeneous resources. The broader impact of our work is four-fold: through our framework, we hide the complexity of the cloud\u2019s software stack to the user who otherwise is required to have expertise in cloud technologies; we manage job scheduling efficiently and in a scalable manner; we enable resource elasticity and workflow orchestration at a large scale; and we facilitate moving the study of nonporous structures, which has wide applications in engineering and scientific fields, to the cloud. While we demonstrate the capability of our framework for a specific materials science application, it can be adapted for other applications and domains because of its modular, multi-layer architecture.<\/jats:p>","DOI":"10.1177\/10943420231167800","type":"journal-article","created":{"date-parts":[[2023,4,14]],"date-time":"2023-04-14T05:32:46Z","timestamp":1681450366000},"page":"260-271","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":5,"title":["Orchestration of materials science workflows for heterogeneous resources at large scale"],"prefix":"10.1177","volume":"37","author":[{"given":"Naweiluo","family":"Zhou","sequence":"first","affiliation":[{"name":"University of Tennessee Knoxville College of Engineering, Knoxville, TN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Giorgio","family":"Scorzelli","sequence":"additional","affiliation":[{"name":"University of Utah, Salt Lake, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jakob","family":"Luettgau","sequence":"additional","affiliation":[{"name":"University of Tennessee Knoxville College of Engineering, Knoxville, TN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rahul R","family":"Kancharla","sequence":"additional","affiliation":[{"name":"Idaho National Laboratory, Idaho Falls, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joshua J","family":"Kane","sequence":"additional","affiliation":[{"name":"Idaho National Laboratory, Idaho Falls, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert","family":"Wheeler","sequence":"additional","affiliation":[{"name":"MicroTesting Solutions LLC, Hilliard, OH, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Brendan P","family":"Croom","sequence":"additional","affiliation":[{"name":"Johns Hopkins University Applied Physics Laboratory, Laurel, MD, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pania","family":"Newell","sequence":"additional","affiliation":[{"name":"University of Utah, Salt Lake, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Valerio","family":"Pascucci","sequence":"additional","affiliation":[{"name":"University of Utah, Salt Lake, Utah, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0031-6377","authenticated-orcid":false,"given":"Michela","family":"Taufer","sequence":"additional","affiliation":[{"name":"University of Tennessee Knoxville College of Engineering, Knoxville, TN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2023,4,14]]},"reference":[{"key":"bibr1-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2021.3059437"},{"key":"bibr2-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1145\/3458817.3476210"},{"key":"bibr3-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/RoEduNet51892.2020.9324857"},{"key":"bibr4-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2014.10.008"},{"key":"bibr5-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2015.96"},{"key":"bibr35-10943420231167800","unstructured":"en-Kiki O, Evans C, Ingerson B (2009) Yaml ain\u2019t markup language (yaml) version 1.1. Working Draft 2008."},{"key":"bibr6-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/LGDI.2005.1612479"},{"key":"bibr7-10943420231167800","unstructured":"Group H (2000-2010). Hierarchical data format version 5 http:\/\/www.hdfgroup.org\/HDF5."},{"key":"bibr8-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-41321-1_14"},{"key":"bibr9-10943420231167800","volume-title":"Kubernetes: Up and Running Dive into the Future of Infrastructure","author":"Hightower K","year":"2017"},{"key":"bibr10-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.3505"},{"key":"bibr11-10943420231167800","volume-title":"Proceedings of the 2020 USENIX Conference on Usenix Annual Technical Conference","author":"Keahey K","year":"2020"},{"key":"bibr12-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1002\/adfm.201909539"},{"key":"bibr13-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2016.05.410"},{"key":"bibr14-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1002\/1527-2648(200108)3:8<539::AID-ADEM539>3.0.CO;2-6"},{"key":"bibr15-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1016\/j.jnca.2013.10.004"},{"key":"bibr16-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2015.77"},{"issue":"239","key":"bibr17-10943420231167800","volume":"2014","author":"Merkel D","year":"2014","journal-title":"Linux J"},{"key":"bibr18-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2015.99"},{"key":"bibr19-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2022.3220539"},{"key":"bibr20-10943420231167800","volume-title":"High Performance Visualization\u2014Enabling Extreme-Scale Scientific Insight, Chapman and Hall\/CRC computational science series","author":"Pascucci V","year":"2012"},{"key":"bibr21-10943420231167800","unstructured":"Penny D (2020) NASA: We Forgot the $30m Bandwidth Charges. https:\/\/www.linkedin.com\/pulse\/oops-we-forgot-30m-bandwidth-charges-david-penny\/."},{"key":"bibr22-10943420231167800","unstructured":"Petruzza S, Venkat A, Morrical N, et al. (2020) The OpenVisus Framework for Extreme Data Management, Analysis and Visualization. https:\/\/github.com\/sci-visus\/OpenVisus."},{"key":"bibr23-10943420231167800","first-page":"1","volume":"46","author":"Peukert W","year":"2015","journal-title":"Unified Design Strategies for Particulate Products"},{"key":"bibr24-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/JCDL.2017.7991618"},{"key":"bibr25-10943420231167800","doi-asserted-by":"crossref","unstructured":"Rocklin M (2015) Dask: parallel computation with blocked algorithms and task scheduling In: Proceedings of the 14th Python in Science Conference (SCIPY), Los Angeles, C. 126\u2013132.","DOI":"10.25080\/Majora-7b98e3ed-013"},{"key":"bibr26-10943420231167800","volume-title":"Exploring Ansible 2: Fast and Easy Guide","author":"Sammons G","year":"2016"},{"key":"bibr27-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/BigData47090.2019.9006223"},{"key":"bibr28-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1126\/science.aah6168"},{"key":"bibr29-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1016\/j.ascom.2020.100412"},{"key":"bibr30-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/eScience51609.2021.00044"},{"key":"bibr31-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/Cluster48925.2021.00068"},{"key":"bibr32-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2015.7364047"},{"key":"bibr33-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1016\/j.parco.2011.05.005"},{"key":"bibr34-10943420231167800","doi-asserted-by":"publisher","DOI":"10.1007\/s11227-015-1438-4"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420231167800","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/10943420231167800","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420231167800","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T08:17:28Z","timestamp":1777450648000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/10943420231167800"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,14]]},"references-count":35,"journal-issue":{"issue":"3-4","published-print":{"date-parts":[[2023,7]]}},"alternative-id":["10.1177\/10943420231167800"],"URL":"https:\/\/doi.org\/10.1177\/10943420231167800","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"value":"1094-3420","type":"print"},{"value":"1741-2846","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,14]]}}}