{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,28]],"date-time":"2026-06-28T05:25:09Z","timestamp":1782624309505,"version":"3.54.5"},"reference-count":39,"publisher":"SAGE Publications","issue":"5","license":[{"start":{"date-parts":[[2023,5,31]],"date-time":"2023-05-31T00:00:00Z","timestamp":1685491200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2023,9]]},"abstract":"<jats:p>We present hipBone, an open-source performance-portable proxy application for the Nek5000 (and NekRS) computational fluid dynamics applications. HipBone is a fully GPU-accelerated C++ implementation of the original NekBone CPU proxy application with several novel algorithmic and implementation improvements which optimize its performance on modern fine-grain parallel GPU accelerators. Our optimizations include a conversion to store the degrees of freedom of the problem in assembled form in order to reduce the amount of data moved during the main iteration and a portable implementation of the main Poisson operator kernel. We demonstrate near-roofline performance of the operator kernel on three different modern GPU accelerators from two different vendors. We present a novel algorithm for splitting the application of the Poisson operator on GPUs which aggressively hides MPI communication required for both halo exchange and assembly. Our implementation of nearest-neighbor MPI communication then leverages several different routing algorithms and GPU-Direct RDMA capabilities, when available, which improves scalability of the benchmark. We demonstrate the performance of hipBone on three different clusters housed at Oak Ridge National Laboratory, namely, the Summit supercomputer and the Frontier early-access clusters, Spock and Crusher. Our tests demonstrate both portability across different clusters and very good scaling efficiency, especially on large problems.<\/jats:p>","DOI":"10.1177\/10943420231178552","type":"journal-article","created":{"date-parts":[[2023,5,31]],"date-time":"2023-05-31T18:59:59Z","timestamp":1685559599000},"page":"560-577","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":9,"title":["HipBone: A performance-portable graphics processing unit-accelerated C++ version of the NekBone benchmark"],"prefix":"10.1177","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1293-7525","authenticated-orcid":false,"given":"Noel","family":"Chalmers","sequence":"first","affiliation":[{"name":"Data Center GPU and Accelerated Processing, Advanced Micro Devices Inc, Austin, TX, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7445-066X","authenticated-orcid":false,"given":"Abhishek","family":"Mishra","sequence":"additional","affiliation":[{"name":"Institute for Computational and Data Sciences, University at Buffalo, Buffalo, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5865-9322","authenticated-orcid":false,"given":"Damon","family":"McDougall","sequence":"additional","affiliation":[{"name":"Data Center GPU and Accelerated Processing, Advanced Micro Devices Inc, Austin, TX, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tim","family":"Warburton","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Virginia Tech, McBryde Hall, VA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2023,5,31]]},"reference":[{"key":"bibr1-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1016\/j.camwa.2020.06.009"},{"key":"bibr2-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1109\/ScalA51936.2020.00012"},{"key":"bibr3-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1109\/H2RC51942.2020.00008"},{"key":"bibr4-10943420231178552","volume-title":"Spectral Methods in Fluid Dynamics","author":"Canuto C","year":"2012"},{"key":"bibr5-10943420231178552","first-page":"69","volume-title":"International conference on exascale applications and software","author":"Cebamanos L","year":"2014"},{"key":"bibr6-10943420231178552","unstructured":"Chalmers N, Karakus A, Austin AP, et al. (2020) Libparanumal: a performance portable high-order finite element library. Available at: https:\/\/github.com\/paranumal\/libparanumal."},{"key":"bibr7-10943420231178552","volume-title":"Portable High-Order Finite Element Kernels I: Streaming Operations","author":"Chalmers N","year":"2020"},{"key":"bibr8-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511546792"},{"key":"bibr9-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2014.103"},{"key":"bibr10-10943420231178552","volume-title":"NekRS, a GPU-Accelerated Spectral Element Navier-Stokes Solver","author":"Fischer P","year":"2021"},{"key":"bibr11-10943420231178552","unstructured":"Fischer P, Lottes J, Kerkemeier S (2008a) Nek5000 web page. Available at: http:\/\/nek5000.mcs.anl.gov"},{"key":"bibr12-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/125\/1\/012076"},{"key":"bibr13-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1177\/1094342020915762"},{"key":"bibr14-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1016\/j.parco.2007.09.002"},{"key":"bibr15-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1080\/17445760601122076"},{"key":"bibr16-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1007\/s11227-016-1744-5"},{"key":"bibr17-10943420231178552","first-page":"57","volume-title":"International Conference on Exascale Applications and Software","author":"Gong J","year":"2014"},{"key":"bibr18-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1006\/jcph.1995.1208"},{"key":"bibr19-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1016\/0010-4655(82)90118-7"},{"key":"bibr20-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1016\/0167-8191(85)90014-6"},{"key":"bibr21-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2015.131"},{"key":"bibr22-10943420231178552","volume-title":"Optimization of Tensor-Product Operations in NekBone on GPUs","author":"Karp M","year":"2020"},{"key":"bibr23-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2009.06.041"},{"key":"bibr24-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2009.01.006"},{"key":"bibr25-10943420231178552","author":"Kronbichler M","year":"2018","journal-title":"Supercomputing"},{"key":"bibr26-10943420231178552","author":"Kronbichler M","year":"2017","journal-title":"II. Supercomputing"},{"key":"bibr27-10943420231178552","volume-title":"Solving Problems on Concurrent Processors: General techniques and regular problems, volume 1","author":"Lamb DA","year":"1988"},{"key":"bibr28-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-004-4787-3"},{"key":"bibr29-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1177\/1094342015576846"},{"key":"bibr30-10943420231178552","volume-title":"OCCA: A Unified Approach to Multi-Threading Languages","author":"Medina DS","year":"2014"},{"key":"bibr32-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2019.05.010"},{"key":"bibr33-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1016\/0021-9991(84)90128-1"},{"key":"bibr34-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2016.08.005"},{"key":"bibr35-10943420231178552","first-page":"130","volume-title":"International Conference on Exascale Applications and Software","author":"Schliephake M","year":"2014"},{"key":"bibr36-10943420231178552","volume-title":"gNek: A GPU Accelerated Incompressible Navier Stokes Solver","author":"Stilwell N","year":"2013"},{"key":"bibr37-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1177\/1094342018816368"},{"key":"bibr38-10943420231178552","volume-title":"Algorithms for Large-Scale Parallel Simulation of Unsteady Incompressible Flows in Three-Dimensional Complex Geometries","author":"Tufo HM","year":"1998"},{"key":"bibr39-10943420231178552","doi-asserted-by":"publisher","DOI":"10.1145\/331532.331599"},{"key":"bibr40-10943420231178552","volume-title":"Strong Scaling of OpenACC Enabled Nek5000 on Several GPU Based HPC Systems","author":"Vincent J","year":"2021"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420231178552","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/10943420231178552","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420231178552","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T08:17:30Z","timestamp":1777450650000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/10943420231178552"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,31]]},"references-count":39,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,9]]}},"alternative-id":["10.1177\/10943420231178552"],"URL":"https:\/\/doi.org\/10.1177\/10943420231178552","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"value":"1094-3420","type":"print"},{"value":"1741-2846","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,31]]}}}