{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,10]],"date-time":"2026-01-10T07:39:19Z","timestamp":1768030759803,"version":"3.49.0"},"reference-count":36,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,12,14]],"date-time":"2023-12-14T00:00:00Z","timestamp":1702512000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science, Research and Arts (MWK), Baden-W\u00fcrttemberg, Germany"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2023,12,31]]},"abstract":"<jats:p>Dense linear algebra operations appear very frequently in high-performance computing (HPC) applications, rendering their performance crucial to achieve optimal scalability. As many modern HPC clusters contain multi-GPU nodes, BLAS operations are frequently offloaded on GPUs, necessitating the use of optimized libraries to ensure good performance. Unfortunately, multi-GPU systems are accompanied by two significant optimization challenges: data transfer bottlenecks as well as problem splitting and scheduling in multiple workers (GPUs) with distinct memories. We demonstrate that the current multi-GPU BLAS methods for tackling these challenges target very specific problem and data characteristics, resulting in serious performance degradation for any slightly deviating workload. Additionally, an even more critical decision is omitted because it cannot be addressed using current scheduler-based approaches: the determination of which devices should be used for a certain routine invocation. To address these issues we propose a model-based approach: using performance estimation to provide problem-specific autotuning during runtime. We integrate this autotuning into an end-to-end BLAS framework named PARALiA. This framework couples autotuning with an optimized task scheduler, leading to near-optimal data distribution and performance-aware resource utilization. We evaluate PARALiA in an HPC testbed with 8 NVIDIA-V100 GPUs, improving the average performance of GEMM by 1.7\u00d7 and energy efficiency by 2.5\u00d7 over the state-of-the-art in a large and diverse dataset and demonstrating the adaptability of our performance-aware approach to future heterogeneous systems.<\/jats:p>","DOI":"10.1145\/3624569","type":"journal-article","created":{"date-parts":[[2023,9,15]],"date-time":"2023-09-15T11:59:45Z","timestamp":1694779185000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["PARALiA: A Performance Aware Runtime for Auto-tuning Linear Algebra on Heterogeneous Systems"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7821-3610","authenticated-orcid":false,"given":"Petros","family":"Anastasiadis","sequence":"first","affiliation":[{"name":"Cslab, National Technical University of Athens, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2141-5654","authenticated-orcid":false,"given":"Nikela","family":"Papadopoulou","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, Chalmers University of Technology, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7811-4831","authenticated-orcid":false,"given":"Georgios","family":"Goumas","sequence":"additional","affiliation":[{"name":"Cslab, National Technical University of Athens, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4890-8427","authenticated-orcid":false,"given":"Nectarios","family":"Koziris","sequence":"additional","affiliation":[{"name":"Cslab, National Technical University of Athens, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8976-8603","authenticated-orcid":false,"given":"Dennis","family":"Hoppe","sequence":"additional","affiliation":[{"name":"HLRS, University of Stuttgart, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6473-1633","authenticated-orcid":false,"given":"Li","family":"Zhong","sequence":"additional","affiliation":[{"name":"HLRS, University of Stuttgart, Germany"}]}],"member":"320","published-online":{"date-parts":[[2023,12,14]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1177\/10943420020160020101"},{"key":"e_1_3_1_3_2","unstructured":"developer.nvidia.com\/cublas. ([n. d.])."},{"key":"e_1_3_1_4_2","volume-title":"SuperMatrix: A Multithreaded Runtime Scheduling System for Algorithms-by-blocks","author":"Chan E. W.","year":"2007","unstructured":"E. W. Chan, F. G. Van Zee, P. Bientinesi, E. S. Quintana-Orti, G. Quintana-Orti, and R. Van de Geijn. 2007. SuperMatrix: A Multithreaded Runtime Scheduling System for Algorithms-by-blocks. Computer Science Department, University of Texas at Austin. https:\/\/books.google.gr\/books?id=ggn-jwEACAAJ"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.parco.2009.12.005"},{"key":"e_1_3_1_6_2","volume-title":"GPU Computing Gems","author":"Agullo Emmanuel","year":"2010","unstructured":"Emmanuel Agullo, C\u00e9dric Augonnet, Jack Dongarra, Hatem Ltaief, Raymond Namyst, Samuel Thibault, and Stanimire Tomov. 2010. Faster, cheaper, better\u2013a hybridization methodology to develop linear algebra software for GPUs. In GPU Computing Gems, Wen mei W. Hwu (Ed.). Vol. 2. Morgan Kaufmann. https:\/\/hal.inria.fr\/inria-00547847"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2015.56"},{"key":"e_1_3_1_8_2","unstructured":"NVIDIA cublasxt. 2015. [online] Available: https:\/\/docs.nvidia.com\/cuda\/cublas\/"},{"key":"e_1_3_1_9_2","unstructured":"NVIDIA nvblas. 2015. [online] Available: https:\/\/docs.nvidia.com\/cuda\/nvblas\/"},{"key":"e_1_3_1_10_2","article-title":"BLASX: A high performance level-3 BLAS library for heterogeneous multi-GPU computing","volume":"1510","author":"Wang Linnan","year":"2015","unstructured":"Linnan Wang, Wei Wu, Jianxiong Xiao, and Yi Yang. 2015. BLASX: A high performance level-3 BLAS library for heterogeneous multi-GPU computing. CoRR abs\/1510.05041 (2015). arxiv:1510.05041http:\/\/arxiv.org\/abs\/1510.05041","journal-title":"CoRR"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/PDP50117.2020.00008"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-8191(00)00087-9"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS51385.2021.00015"},{"key":"e_1_3_1_14_2","first-page":"1","volume-title":"SC\u201911: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis","author":"Hoefler T.","year":"2011","unstructured":"T. Hoefler, W. Gropp, W. Kramer, and M. Snir. 2011. Performance modeling for systematic performance tuning. In SC\u201911: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis. 1\u201312."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2011.07.011"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1109\/CCGrid.2014.16","volume-title":"2014 14th IEEE\/ACM International Symposium on Cluster, Cloud and Grid Computing","author":"Werkhoven B. v.","year":"2014","unstructured":"B. v. Werkhoven, J. Maassen, F. J. Seinstra, and H. E. Bal. 2014. Performance models for CPU-GPU data transfers. In 2014 14th IEEE\/ACM International Symposium on Cluster, Cloud and Grid Computing. 11\u201320."},{"key":"e_1_3_1_17_2","unstructured":"NVIDIA P\u00e9ter Vingelmann and Frank H. P. Fitzek. 2020. CUDA release: 10.2.89. (2020). https:\/\/developer.nvidia.com\/cuda-toolkit"},{"key":"e_1_3_1_18_2","doi-asserted-by":"crossref","first-page":"1097","DOI":"10.1109\/IPDPSW.2013.236","volume-title":"2013 IEEE International Symposium on Parallel Distributed Processing, Workshops and PhD Forum","author":"Boyer M.","year":"2013","unstructured":"M. Boyer, J. Meng, and K. Kumaran. 2013. Improving GPU performance prediction with data transfer modeling. In 2013 IEEE International Symposium on Parallel Distributed Processing, Workshops and PhD Forum. 1097\u20131106."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2014.05.003"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1177\/1094342015585845"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.simpat.2009.06.007"},{"key":"e_1_3_1_22_2","unstructured":"VULCAN HPC cluster at HLRS [online] Available: https:\/\/kb.hlrs.de\/platforms\/index.php\/NEC_Cluster_Hardware_and_Architecture_(vulcan)"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/2807591.2807644"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/2764454"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/2925987"},{"key":"e_1_3_1_26_2","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1109\/ISPASS.2011.5762730","volume-title":"(IEEE ISPASS) IEEE International Symposium on Performance Analysis of Systems and Software","author":"Gregg C.","year":"2011","unstructured":"C. Gregg and K. Hazelwood. 2011. Where is the data? Why you cannot debate CPU vs. GPU performance without the answer. In (IEEE ISPASS) IEEE International Symposium on Performance Analysis of Systems and Software. 134\u2013144."},{"key":"e_1_3_1_27_2","first-page":"1","volume-title":"2009 IEEE International Symposium on Parallel Distributed Processing","author":"Schaa D.","year":"2009","unstructured":"D. Schaa and D. Kaeli. 2009. Exploring the multiple-GPU design space. In 2009 IEEE International Symposium on Parallel Distributed Processing. 1\u201312."},{"key":"e_1_3_1_28_2","article-title":"Comparison of analytical and ML-based models for predicting CPU-GPU data transfer time","author":"Riahi Ali","year":"2020","unstructured":"Ali Riahi, Abdorreza Savadi, and Mahmoud Naghibzadeh. 2020. Comparison of analytical and ML-based models for predicting CPU-GPU data transfer time. Computing (January 2020).","journal-title":"Computing"},{"key":"e_1_3_1_29_2","first-page":"1828","volume-title":"2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops PhD Forum","author":"Meswani M. R.","year":"2012","unstructured":"M. R. Meswani, L. Carrington, D. Unat, A. Snavely, S. Baden, and S. Poole. 2012. Modeling and predicting performance of high performance computing applications on hardware accelerators. In 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops PhD Forum. 1828\u20131837."},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/CCGrid.2015.105"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3148173.3148184"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3297663.3310299"},{"key":"e_1_3_1_33_2","first-page":"45","volume-title":"2009 42nd Annual IEEE\/ACM International Symposium on Microarchitecture (MICRO)","author":"Luk Chi-Keung","year":"2009","unstructured":"Chi-Keung Luk, Sunpyo Hong, and Hyesoon Kim. 2009. Qilin: Exploiting parallelism on heterogeneous multiprocessors with adaptive mapping. In 2009 42nd Annual IEEE\/ACM International Symposium on Microarchitecture (MICRO). 45\u201355."},{"key":"e_1_3_1_34_2","volume-title":"Defense + Commercial Sensing","author":"Humphrey John R.","year":"2010","unstructured":"John R. Humphrey, Daniel K. Price, Kyle E. Spagnoli, Aaron L. Paolini, and Eric J. Kelmelis. 2010. CULA: Hybrid GPU accelerated linear algebra routines. In Defense + Commercial Sensing."},{"key":"e_1_3_1_35_2","volume-title":"Defense + Commercial Sensing","author":"Spagnoli Kyle E.","year":"2011","unstructured":"Kyle E. Spagnoli, John R. Humphrey, Daniel K. Price, and Eric J. Kelmelis. 2011. Accelerating sparse linear algebra using graphics processing units. In Defense + Commercial Sensing."},{"key":"e_1_3_1_36_2","volume-title":".","author":"Humphrey John R.","year":"2012","unstructured":"John R. Humphrey, Daniel K. Price, Kyle E. Spagnoli, and Eric J. Kelmelis. 2012. Accelerating CULA linear algebra routines with hybrid GPU and multicore computing. In ."},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPP.2012.31"}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3624569","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3624569","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T17:49:46Z","timestamp":1750268986000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3624569"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,14]]},"references-count":36,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12,31]]}},"alternative-id":["10.1145\/3624569"],"URL":"https:\/\/doi.org\/10.1145\/3624569","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,14]]},"assertion":[{"value":"2023-02-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-08-23","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}