{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T20:21:47Z","timestamp":1784060507524,"version":"3.55.0"},"reference-count":40,"publisher":"SAGE Publications","issue":"3","license":[{"start":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T00:00:00Z","timestamp":1732752000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T00:00:00Z","timestamp":1732752000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100004359","name":"Vetenskapsr\u00e5det","doi-asserted-by":"publisher","award":["2019-04723"],"award-info":[{"award-number":["2019-04723"]}],"id":[{"id":"10.13039\/501100004359","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100018703","name":"HORIZON EUROPE European Innovation Council","doi-asserted-by":"publisher","award":["955606 (DEEP-SEA)"],"award-info":[{"award-number":["955606 (DEEP-SEA)"]}],"id":[{"id":"10.13039\/100018703","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100017156","name":"Swedish e-Science Research Centre","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100017156","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2025,5]]},"abstract":"<jats:p>The never-ending computational demand from simulations of turbulence makes computational fluid dynamics (CFD) a prime application use case for current and future exascale systems. High-order finite element methods, such as the spectral element method, have been gaining traction as they offer high performance on both multicore CPUs and modern GPU-based accelerators. In this work, we assess how high-fidelity CFD using the spectral element method can exploit the modular supercomputing architecture at scale through domain partitioning, where the computational domain is split between a Booster module powered by GPUs and a Cluster module with conventional CPU nodes. We investigate several different flow cases and computer systems based on the Modular Supercomputing Architecture (MSA). We observe that for our simulations, the communication overhead and load balancing issues incurred by incorporating different computing architectures are seldom worthwhile, especially when I\/O is also considered, but when the simulation at hand requires more than the combined global memory on the GPUs, utilizing additional CPUs to increase the available memory can be fruitful. We support our results with a simple performance model to assess when running across modules might be beneficial. As MSA is becoming more widespread and efforts to increase system utilization are growing more important our results give insight into when and how a monolithic application can utilize and spread out to more than one module and obtain a faster time to solution.<\/jats:p>","DOI":"10.1177\/10943420241303163","type":"journal-article","created":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T18:30:59Z","timestamp":1732818659000},"page":"329-344","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":6,"title":["Experience and analysis of scalable high-fidelity computational fluid dynamics on modular supercomputing architectures"],"prefix":"10.1177","volume":"39","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3374-8093","authenticated-orcid":false,"given":"Martin","family":"Karp","sequence":"first","affiliation":[{"name":"KTH Royal Institute of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Estela","family":"Suarez","sequence":"additional","affiliation":[{"name":"Forschungszentrum J\u00fclich GmbH"},{"name":"University of Bonn"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jan H","family":"Meinke","sequence":"additional","affiliation":[{"name":"Forschungszentrum J\u00fclich GmbH"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M\u00e5ns I","family":"Andersson","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9627-5903","authenticated-orcid":false,"given":"Philipp","family":"Schlatter","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology"},{"name":"Friedrich-Alexander Universit\u00e4t (FAU) Erlangen-N\u00fcrnberg"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefano","family":"Markidis","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5020-1631","authenticated-orcid":false,"given":"Niclas","family":"Jansson","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2024,11,28]]},"reference":[{"key":"e_1_3_5_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.parco.2021.102841"},{"key":"e_1_3_5_3_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1505.07630"},{"key":"e_1_3_5_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-23220-6_14"},{"key":"e_1_3_5_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3581784.3607089"},{"key":"e_1_3_5_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2020.01.045"},{"key":"e_1_3_5_7_1","doi-asserted-by":"publisher","DOI":"10.1177\/1094342017703771"},{"key":"e_1_3_5_8_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511546792"},{"key":"e_1_3_5_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10494-013-9482-8"},{"key":"e_1_3_5_10_1","doi-asserted-by":"publisher","DOI":"10.2514\/6.2015-3049"},{"key":"e_1_3_5_11_1","unstructured":"Fischer PF Lottes JW Kerkemeier SG (2008) nek5000 Web page. Available at: https:\/\/nek5000.mcs.anl.gov."},{"key":"e_1_3_5_12_1","doi-asserted-by":"publisher","DOI":"10.1177\/1094342020915762"},{"key":"e_1_3_5_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2015.131"},{"key":"e_1_3_5_14_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1922794117"},{"key":"e_1_3_5_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3581784.3627039"},{"key":"e_1_3_5_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compfluid.2024.106243"},{"key":"e_1_3_5_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/e-Science58273.2023.10254865"},{"key":"e_1_3_5_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/0021-9991(91)90007-8"},{"key":"e_1_3_5_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3539781.3539785"},{"key":"e_1_3_5_20_1","doi-asserted-by":"publisher","DOI":"10.1177\/10943420231158616"},{"key":"e_1_3_5_21_1","volume-title":"Parallel Graph Partitioning and Sparse Matrix Ordering Library. Version 2","author":"Karypis G","year":"2003","unstructured":"Karypis G, Schloegel K, Kumar V (2003) Parmetis In: Parallel Graph Partitioning and Sparse Matrix Ordering Library. Version 2."},{"key":"e_1_3_5_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1394608.1382129"},{"key":"e_1_3_5_23_1","doi-asserted-by":"publisher","DOI":"10.1177\/10943420211020803"},{"key":"e_1_3_5_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compfluid.2018.01.010"},{"key":"e_1_3_5_25_1","doi-asserted-by":"publisher","DOI":"10.17815\/jlsrf-5-171"},{"key":"e_1_3_5_26_1","doi-asserted-by":"publisher","DOI":"10.17815\/jlsrf-4-121-1"},{"key":"e_1_3_5_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPSW.2018.00019"},{"key":"e_1_3_5_28_1","volume-title":"Porting Applications to a Modular Supercomputer-Experiences from the Deepest Project","author":"Kreuzer A","year":"2021","unstructured":"Kreuzer A, Lippert T, Suarez E, et al. (2021) Porting Applications to a Modular Supercomputer-Experiences from the Deepest Project. Technical report, J\u00fclich Supercomputing Center."},{"key":"e_1_3_5_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2015.08.002"},{"key":"e_1_3_5_30_1","first-page":"359","volume-title":"International Conference on Variability of the Sun and Sun-like Stars: From Asteroseismology to Space Weather","author":"Markov S","year":"2019","unstructured":"Markov S, Petkov P, Pavlov V (2019) Large-scale molecular dynamics simulations on modular supercomputer architecture with gromacs. In: International Conference on Variability of the Sun and Sun-like Stars: From Asteroseismology to Space Weather. Springer, 359\u2013367."},{"key":"e_1_3_5_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3581784.3627038"},{"key":"e_1_3_5_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11227-013-1015-7"},{"key":"e_1_3_5_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2938615.2938617"},{"key":"e_1_3_5_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPSW52791.2021.00019"},{"key":"e_1_3_5_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/HiPINEB.2017.11"},{"key":"e_1_3_5_36_1","unstructured":"Slotnick JP Khodadoust A Alonso J et al. (2014) CFD vision 2030 study: a path to revolutionary computational aerosciences. Technical report."},{"key":"e_1_3_5_37_1","doi-asserted-by":"publisher","DOI":"10.1201\/9781351036863-9"},{"key":"e_1_3_5_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/331532.331599"},{"key":"e_1_3_5_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2010.11.031"},{"key":"e_1_3_5_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cpc.2014.07.011"},{"key":"e_1_3_5_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2014.2375202"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420241303163","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/10943420241303163","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420241303163","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T08:17:41Z","timestamp":1777450661000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/10943420241303163"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,28]]},"references-count":40,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,5]]}},"alternative-id":["10.1177\/10943420241303163"],"URL":"https:\/\/doi.org\/10.1177\/10943420241303163","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"value":"1094-3420","type":"print"},{"value":"1741-2846","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,28]]}}}