{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T22:55:20Z","timestamp":1777676120921,"version":"3.51.4"},"reference-count":34,"publisher":"SAGE Publications","issue":"4","license":[{"start":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T00:00:00Z","timestamp":1777075200000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/www.sagepub.com\/licence-information-for-chorus"},{"start":{"date-parts":[[2025,4,25]],"date-time":"2025-04-25T00:00:00Z","timestamp":1745539200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"Exascale Computing Project","award":["17-SC-20-SC"],"award-info":[{"award-number":["17-SC-20-SC"]}]},{"name":"U.S. Department of Energy by Lawrence Livermore National Laboratory","award":["DE-AC52-07NA27344 (LLNL-CONF-826133)"],"award-info":[{"award-number":["DE-AC52-07NA27344 (LLNL-CONF-826133)"]}]},{"DOI":"10.13039\/100006224","name":"Argonne National Laboratory","doi-asserted-by":"crossref","award":["DE-AC02-06CH11357"],"award-info":[{"award-number":["DE-AC02-06CH11357"]}],"id":[{"id":"10.13039\/100006224","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100006231","name":"Brookhaven National Laboratory","doi-asserted-by":"crossref","award":["DESC0012704"],"award-info":[{"award-number":["DESC0012704"]}],"id":[{"id":"10.13039\/100006231","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100006132","name":"Office of Science of the U.S. DOE","doi-asserted-by":"crossref","award":["DE-AC05-00OR22725"],"award-info":[{"award-number":["DE-AC05-00OR22725"]}],"id":[{"id":"10.13039\/100006132","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2025,7]]},"abstract":"<jats:p>\n                    Exascale computers offer transformative capabilities to combine data-driven and learning-based approaches wi\n                    <jats:sup>th<\/jats:sup>\n                    traditional simulation applications to accelerate scientific discovery and insight. However, these software combinations and integrations are difficult to achieve due to the challenges of coordinating and deploying heterogeneous software components on diverse and massive platforms. We present the ExaWorks project, which addresses many of these challenges. We developed a workflow Software Development Toolkit (SDK), a curated collection of workflow technologies that can be composed and interoperated through a common interface, engineered following current best practices, and specifically designed to work on HPC platforms. ExaWorks also developed PSI\/J, a job management abstraction API, to simplify the construction of portable software components and applications that can be used over various HPC schedulers. The PSI\/J API is a minimal interface for submitting and monitoring jobs and their execution state across multiple and commonly used HPC schedulers. We also describe several leading and innovative workflow examples of ExaWorks tools used on DOE leadership platforms. Furthermore, we discuss how our project is working with the workflow community, large computing facilities, and HPC platform vendors to address the requirements of workflows sustainably at the exascale.\n                  <\/jats:p>","DOI":"10.1177\/10943420251331674","type":"journal-article","created":{"date-parts":[[2025,4,25]],"date-time":"2025-04-25T05:14:56Z","timestamp":1745558096000},"page":"579-593","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["Exascale workflow applications and middleware: An ExaWorks retrospective"],"prefix":"10.1177","volume":"39","author":[{"given":"Aymen","family":"Alsaadi","sequence":"first","affiliation":[{"name":"Rutgers University, New Brunswick, NJ, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mihael","family":"Hategan-Marandiuc","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"},{"name":"University of Chicago, Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ketan","family":"Maheshwari","sequence":"additional","affiliation":[{"name":"Oak Ridge National Laboratory, Oak Ridge, TN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andre","family":"Merzky","sequence":"additional","affiliation":[{"name":"Incomputable LLC, Highland Park, NJ, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mikhail","family":"Titov","sequence":"additional","affiliation":[{"name":"Brookhaven National Laboratory, Upton, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matteo","family":"Turilli","sequence":"additional","affiliation":[{"name":"Rutgers University, New Brunswick, NJ, USA"},{"name":"Brookhaven National Laboratory, Upton, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andreas","family":"Wilke","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Justin M.","family":"Wozniak","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"},{"name":"University of Chicago, Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kyle","family":"Chard","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"},{"name":"University of Chicago, Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rafael","family":"Ferreira da Silva","sequence":"additional","affiliation":[{"name":"Oak Ridge National Laboratory, Oak Ridge, TN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5040-026X","authenticated-orcid":false,"given":"Shantenu","family":"Jha","sequence":"additional","affiliation":[{"name":"Rutgers University, New Brunswick, NJ, USA"},{"name":"Princeton Plasma Physics Lab., Princeton, NJ, USA"},{"name":"Princeton University, Princeton, NJ, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Laney","sequence":"additional","affiliation":[{"name":"Lawrence Livermore National Laboratory, Livermore, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2025,4,25]]},"reference":[{"key":"e_1_3_5_2_1","doi-asserted-by":"crossref","unstructured":"Ahn DH Garlick J Grondona M et al. (2014) Flux: a next-generation resource management framework for large hpc centers. In: 2014 43rd International conference on parallel processing workshops Minneapolis MN USA 09\u201312 September 2014 9\u201317.","DOI":"10.1109\/ICPPW.2014.15"},{"key":"e_1_3_5_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2020.04.006"},{"key":"e_1_3_5_4_1","doi-asserted-by":"crossref","unstructured":"Al-Saadi A Ahn DH Babuji Y et al. (2021) Exaworks: workflows for exascale. In: 2021 IEEE workshop on Workflows in Support of Large-Scale Science (WORKS) 50\u201357.","DOI":"10.1109\/WORKS54523.2021.00012"},{"key":"e_1_3_5_5_1","doi-asserted-by":"publisher","unstructured":"Alsaadi A Ward L Merzky A et al. (2022) RADICAL-Pilot and Parsl: executing heterogeneous workflows on HPC platforms. In: 2022 IEEE\/ACM workshop on Workflows in Support of Large-Scale Science (WORKS) Dallas TX USA 13\u201318 November 2022 27\u201334. DOI:10.1109\/WORKS56498.2022.00009.","DOI":"10.1109\/WORKS56498.2022.00009"},{"key":"e_1_3_5_6_1","doi-asserted-by":"crossref","unstructured":"Babuji Y Woodard A Li Z et al. (2019) Parsl: pervasive parallel programming in Python. In: 28th ACM international symposium on High-Performance Parallel and Distributed Computing (HPDC) Phoenix AZ USA 17 June 2019 25\u201336.","DOI":"10.1145\/3307681.3325400"},{"issue":"1","key":"e_1_3_5_7_1","first-page":"27","article-title":"Workflows for science: a challenge when facing the convergence of hpc and big data","volume":"4","author":"Badia Sala RM","year":"2017","unstructured":"Badia Sala RM, Ayguade Parra E, Labarta Mancho JJ (2017) Workflows for science: a challenge when facing the convergence of hpc and big data. Supercomputing Frontiers and Innovations 4(1): 27\u201347.","journal-title":"Supercomputing Frontiers and Innovations"},{"key":"e_1_3_5_8_1","doi-asserted-by":"crossref","unstructured":"Balasubramanian V Treikalis A Weidner O et al. (2016) Ensemble toolkit: scalable and flexible execution of ensembles of tasks. In: 2016 45th International Conference on Parallel Processing (ICPP) 458\u2013463.","DOI":"10.1109\/ICPP.2016.59"},{"key":"e_1_3_5_9_1","first-page":"57","volume":"3","author":"Ben-Nun T","year":"2020","unstructured":"Ben-Nun T, Gamblin T, Hollman DS, et al. (2020) Workflows are the new applications: challenges in performance, portability, and productivity. In: 2020 IEEE\/ACM international workshop on Performance, Portability and Productivity in HPC (P3HPC), 57\u201369.","journal-title":"Workflows are the new applications: challenges in performance, portability, and productivity"},{"key":"e_1_3_5_10_1","doi-asserted-by":"publisher","DOI":"10.1177\/10943420231201154"},{"key":"e_1_3_5_11_1","doi-asserted-by":"crossref","unstructured":"den Burger M Jacobs C Kielmann T et al. (2010) What is the price of simplicity? a cross-platform evaluation of the SAGA API. In: Proceedings of the 16th international euro-par conference on parallel processing: part I Ischia Italy 31 August\u20133 September 2010 392\u2013404.","DOI":"10.1007\/978-3-642-15277-1_38"},{"key":"e_1_3_5_12_1","doi-asserted-by":"crossref","unstructured":"Di Natale F Bhatia H Carpenter TS et al. (2019) A massively parallel infrastructure\u00b4 for adaptive multiscale simulations: modeling RAS initiation pathway for cancer. In: Proceedings of the international conference for high performance computing networking storage and analysis Denver CO USA 12\u201317 November 2023 1\u201316.","DOI":"10.1145\/3295500.3356197"},{"key":"e_1_3_5_13_1","unstructured":"exa (2021) ExaWorks workflow survey. Technical report Exascale computing project. https:\/\/exaworks.org\/documents\/ExaWorksSurveyReport.pdf"},{"key":"e_1_3_5_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-23606-8_11"},{"key":"e_1_3_5_15_1","doi-asserted-by":"crossref","unstructured":"Ferreira da Silva R Casanova H Chard K et al. (2021) A community roadmap for scientific workflows research and development. In: 2021 IEEE workshop on Workflows in Support of Large-Scale Science (WORKS) St. Louis MO USA 15\u201315 November 2021 81\u201390.","DOI":"10.1109\/WORKS54523.2021.00016"},{"key":"e_1_3_5_16_1","doi-asserted-by":"publisher","DOI":"10.2172\/2006942"},{"key":"e_1_3_5_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2024.3401542"},{"key":"e_1_3_5_18_1","doi-asserted-by":"publisher","DOI":"10.2172\/2474744"},{"key":"e_1_3_5_19_1","unstructured":"git (2024) Awesome workflow engines. https:\/\/github.com\/meirwah\/awesome-workflow-engines"},{"key":"e_1_3_5_20_1","doi-asserted-by":"crossref","unstructured":"Hategan-Marandiuc M Merzky A Collier N et al. (2023) PSI\/J: a portable interface for submitting monitoring and managing jobs. In: 2023 IEEE 19th international conference on E-Science (E-Science) 1\u201310.","DOI":"10.1109\/e-Science58273.2023.10254912"},{"key":"e_1_3_5_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2021.3105994"},{"key":"e_1_3_5_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jocs.2022.101707"},{"key":"e_1_3_5_23_1","doi-asserted-by":"crossref","unstructured":"Saadi AA Alfe D Babuji Y et al. (2021) Impeccable: integrated modeling pipeline for covid cure by assessing better leads. In: 50th International Conference on Parallel Processing (ICPP \u201921) Lemont IL USA 9\u201312 August 2021 12.","DOI":"10.1145\/3472456.3473524"},{"key":"e_1_3_5_24_1","unstructured":"The Binder Project (2024) Reproducible sharable open interactive computing environments. https:\/\/mybinder.org\/"},{"key":"e_1_3_5_25_1","unstructured":"The ExaWorks Project (2024a) ExaWorks: software development kit. https:\/\/exaworkssdk.readthedocs.io\/en\/latest\/"},{"key":"e_1_3_5_26_1","unstructured":"The ExaWorks Project (2024b) ExaWorks software development kit docker container. https:\/\/github.com\/ExaWorks\/SDK\/tree\/master\/docker"},{"key":"e_1_3_5_27_1","unstructured":"The ExaWorks Team (2024) A portable submission interface for jobs (PSI\/J). https:\/\/purl.org\/psij.io\/spec"},{"key":"e_1_3_5_28_1","unstructured":"The SWAS Project (2024) SWAS: sustaining workflows & application services. https:\/\/swas.center\/"},{"key":"e_1_3_5_29_1","unstructured":"The Vuejs Team (2024) Vue.js. https:\/\/vuejs.org\/"},{"key":"e_1_3_5_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2019.2920048"},{"key":"e_1_3_5_31_1","doi-asserted-by":"publisher","DOI":"10.1177\/10943420211042558"},{"key":"e_1_3_5_32_1","doi-asserted-by":"publisher","DOI":"10.1177\/10943420241288242"},{"key":"e_1_3_5_33_1","doi-asserted-by":"publisher","unstructured":"Ward L Sivaraman G Pauloski J et al. (2021) Colmena: scalable machine-learning-based steering of ensemble simulations for high performance computing. In: 2021 IEEE\/ACM workshop on Machine Learning in High Performance Computing Environments (MLHPC) Los Alamitos CA USA 15 November 9\u201320. DOI: 10.1109\/MLHPC54614.2021.00007.","DOI":"10.1109\/MLHPC54614.2021.00007"},{"key":"e_1_3_5_34_1","volume-title":"ECP Workflow Survey Report","author":"Wozniak JM","year":"2017","unstructured":"Wozniak JM (2017) ECP Workflow Survey Report. Lemont, IL: Argonne National Laboratory. Technical Report ANL-24\/20."},{"key":"e_1_3_5_35_1","doi-asserted-by":"crossref","unstructured":"Wozniak JM Armstrong TG Wilde M et al. (2013) Swift\/T: large-scale application composition via distributed-memory dataflow processing. In: 2013 13th IEEE\/ACM international symposium on cluster cloud and grid computing Delft Netherlands 13\u201316 May 2013 95\u2013102.","DOI":"10.1109\/CCGrid.2013.99"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420251331674","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/10943420251331674","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420251331674","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420251331674","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T08:17:44Z","timestamp":1777450664000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/10943420251331674"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,25]]},"references-count":34,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,7]]}},"alternative-id":["10.1177\/10943420251331674"],"URL":"https:\/\/doi.org\/10.1177\/10943420251331674","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"value":"1094-3420","type":"print"},{"value":"1741-2846","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,25]]}}}