{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T12:55:46Z","timestamp":1792241746941,"version":"4.4.0"},"reference-count":35,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"1","funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/P020720\/1"],"award-info":[{"award-number":["EP\/P020720\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014437","name":"Wenner-Gren Stiftelserna","doi-asserted-by":"publisher","award":["UPD2019-0067"],"award-info":[{"award-number":["UPD2019-0067"]}],"id":[{"id":"10.13039\/100014437","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000288","name":"Royal Society","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000288","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000015","name":"U.S. Department of Energy","doi-asserted-by":"publisher","award":["17-SC-20-SC"],"award-info":[{"award-number":["17-SC-20-SC"]}],"id":[{"id":"10.13039\/100000015","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Sci. Comput."],"published-print":{"date-parts":[[2023,2,28]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>In multiword arithmetic, a matrix is represented as the unevaluated sum of two or more lower precision matrices, and a matrix product is formed by multiplying the constituents in low precision. We investigate the use of multiword arithmetic for improving the performance-accuracy tradeoff of matrix multiplication with mixed precision block fused multiply\u2013add (FMA) hardware, focusing especially on the tensor cores available on NVIDIA GPUs. Building on a general block FMA framework, we develop a comprehensive error analysis of multiword matrix multiplication. After confirming the theoretical error bounds experimentally by simulating low precision in software, we use the cuBLAS and CUTLASS libraries to implement a number of matrix multiplication algorithms using double-fp16 (double-binary16) arithmetic. When running the algorithms on NVIDIA V100 and A100 GPUs, we find that double-fp16 is not as accurate as fp32 (binary32) arithmetic despite satisfying the same worst-case error bound. Using probabilistic error analysis, we explain why this issue is likely to be caused by the rounding mode used by the NVIDIA tensor cores, and we propose a parameterized blocked summation algorithm that alleviates the problem and significantly improves the performance-accuracy tradeoff.<\/jats:p>","DOI":"10.1137\/21m1465032","type":"journal-article","created":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T12:46:52Z","timestamp":1675169212000},"page":"C1-C19","source":"Crossref","is-referenced-by-count":19,"title":["Matrix Multiplication in Multiword Arithmetic: Error Analysis and Application to GPU Tensor Cores"],"prefix":"10.1137","volume":"45","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6015-391X","authenticated-orcid":true,"given":"Massimiliano","family":"Fasi","sequence":"first","affiliation":[{"name":"Department of Computer Science, Upper Mountjoy Campus, Durham University, Durham DH1 3LE, UK."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5956-4976","authenticated-orcid":true,"given":"Nicholas J.","family":"Higham","sequence":"additional","affiliation":[{"name":"Department of Mathematics, University of Manchester, Manchester M13 9PL, UK."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Florent","family":"Lopez","sequence":"additional","affiliation":[{"name":"Livermore Software Technology, ANSYS Company, Livermore, CA 94551 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Theo","family":"Mary","sequence":"additional","affiliation":[{"name":"Sorbonne Universit\u00e9, CNRS, LIP6, Paris 75005, France."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8706-1436","authenticated-orcid":true,"given":"Mantas","family":"Mikaitis","sequence":"additional","affiliation":[{"name":"Department of Mathematics, University of Manchester, Manchester M13 9PL, UK."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2023,11,21]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1177\/10943420211003313"},{"key":"ref2","unstructured":"Advanced Micro Devices, \u201cAMD instinct MI200\u201d Instruction Set Architecture, Reference Guide, Advanced Micro Devices, Inc., Santa Clara, CA, 2021, https:\/\/developer.amd.com\/wp-content\/resources\/CDNA2_Shader_ISA_18November2021.pdf."},{"key":"ref3","unstructured":"Advanced Micro Devices, AMD Instinct\u2122 MI200 Series Accelerator, Advanced Micro Devices, Inc., 2021; also available online from https:\/\/www.amd.com\/system\/files\/documents\/amd-instinct-mi200-datasheet.pdf."},{"key":"ref4","unstructured":"Advanced Micro Devices, Introducing AMD CDNA\u2122 2 Architecture, Advanced Micro Devices, Inc., 2021; also available online from https:\/\/www.amd.com\/system\/files\/documents\/amd-cdna2-white-paper.pdf."},{"key":"ref5","doi-asserted-by":"crossref","unstructured":"H. Anzt , \nY. M. Tsai , \nA. Abdelfattah , \nT. Cojean , and \nJ. Dongarra , Evaluating the performance of NVIDIA\u2019s A100 Ampere GPU for sparse and batched computations, in Proceedings of the 2020 IEEE\/ACM Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems, IEEE, Washington, DC; ACM, New York, 2021, pp. 26\u201338, https:\/\/doi.org\/10.1109\/PMBS51919.2020.00009.","DOI":"10.1109\/PMBS51919.2020.00009"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1137\/19M1289546"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1137\/19M1257780"},{"key":"ref8","doi-asserted-by":"crossref","unstructured":"J. Choquette , \nE. Lee , \nR. Krashinsky , \nV. Balan , and \nB. Khailany , 3.2 the A100 datacenter GPU and Ampere architecture, in Proceedings of the 2021 IEEE International Solid-State Circuits Conference San Francisco, CA; IEEE, Washington, DC, 2021, pp. 48\u201350, https:\/\/doi.org\/10.1109\/ISSCC42613.2021.9365803.","DOI":"10.1109\/ISSCC42613.2021.9365803"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1137\/20M1334796"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.211631"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.330"},{"key":"ref12","unstructured":"M. Fasi  and \nM. Mikaitis , CPFloat: A C Library for Emulating Low-Precision Arithmetic, MIMS EPrint 2020.22, Manchester Institute for Mathematical Sciences, The University of Manchester, Manchester, UK, 2020; also available online from http:\/\/eprints.maths.manchester.ac.uk\/2873\/."},{"key":"ref13","first-page":"20200110","volume":"476","author":"Haidar A.","year":"2020","journal-title":"Proc Roy. Soc. London A"},{"key":"ref14","doi-asserted-by":"crossref","unstructured":"A. Haidar , \nS. Tomov , \nJ. Dongarra , and \nN. J. Higham , Harnessing GPU tensor cores for fast FP16 arithmetic to speed up mixed-precision iterative refinement solvers, in Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis, SC18 (Dallas, TX), IEEE, Piscataway, NJ, 2018, pp. 606\u2013613, https:\/\/doi.org\/10.1109\/SC.2018.00050.","DOI":"10.1109\/SC.2018.00050"},{"key":"ref15","doi-asserted-by":"crossref","unstructured":"G. Henry , \nP. T. P. Tang , and \nA. Heinecke , Leveraging the bfloat16 artificial intelligence datatype for higher-precision computations, in Proceedings of the 26th IEEE Symposium on Computer Arithmetic, Kyoto, Japan, 2019, pp. 69\u201376, https:\/\/doi.org\/10.1109\/ARITH.2019.00019.","DOI":"10.1109\/ARITH.2019.00019"},{"key":"ref16","doi-asserted-by":"crossref","unstructured":"B. Hickmann  and \nD. Bradford , Experimental analysis of matrix multiplication functional units, in Proceedings of the 26th IEEE Symposium on Computer Arithmetic, Kyoto, Japan, 2019, pp. 116\u2013119, https:\/\/doi.org\/10.1109\/ARITH.2019.00031.","DOI":"10.1109\/ARITH.2019.00031"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898718027"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1137\/18M1226312"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1137\/20M1314355"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1017\/S0962492922000022"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.1137\/18M1229511"},{"key":"ref22","volume-title":"IEEE Standard for Floating-Point Arithmetic (revision of IEEE Std 754-1985)","author":"IEEE Std 754-2008","year":"2008"},{"key":"ref23","volume-title":"IEEE Standard for Floating-Point Arithmetic (revision of IEEE Std 754-2008)","author":"IEEE Std 754-2019","year":"2019"},{"key":"ref24","unstructured":"Intel Corporation, BFLOAT16\u2014Hardware Numerics Definition, White paper 338302-001US, Intel Corporation, 2018; also available online from https:\/\/software.intel.com\/en-us\/download\/bfloat16-hardware-numerics-definition."},{"key":"ref25","author":"Lopez F.","journal-title":"Int. J. High Perform. Comput. Appl."},{"key":"ref26","doi-asserted-by":"crossref","unstructured":"S. Markidis , \nChien S. W. D. \nE. Laure , \nI. B. Peng , and \nJ. S. Vetter , NVIDIA tensor core programmability, performance & precision, in Proceedings of the 32nd IEEE International Parallel and Distributed Processing Symposium Workshops, Vancouver, BC, Canada, 2018, pp. 522\u2013531, https:\/\/doi.org\/10.1109\/IPDPSW.2018.00091.","DOI":"10.1109\/IPDPSW.2018.00091"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1016\/j.cam.2019.112701"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-50743-5_12"},{"key":"ref29","doi-asserted-by":"crossref","unstructured":"J.M. Muller , \nN. Brunie , \nF. de Dinechin , \nC.P. Jeannerod , \nM. Joldes , \nV. Lef\u00e8vre , \nG. Melquiond , \nN. Revol , and \nS. Torres , Handbook of Floating-Point Arithmetic, Birkh\u00e4user, Cham, Switzerland, 2018, https:\/\/doi.org\/10.1007\/978-3-319-76526-6.","DOI":"10.1007\/978-3-319-76526-6"},{"key":"ref30","unstructured":"NVIDIA, NVIDIA tesla V100 GPU architecture, NVIDIA whitepaper WP-08608-001_v1.1, 2017, https:\/\/images.nvidia.com\/content\/volta-architecture\/pdf\/volta-architecture-whitepaper.pdf."},{"key":"ref31","unstructured":"NVIDIA, NVIDIA A100 tensor core GPU architecture, NVIDIA whitepaper v1.0, 2020, https:\/\/www.nvidia.com\/content\/dam\/en-zz\/Solutions\/Data-Center\/nvidia-ampere-architecture-whitepaper.pdf."},{"key":"ref32","unstructured":"NVIDIA, NVIDIA H100 tensor core GPU architecture, NVIDIA whitepaper v1.2, 2022, https:\/\/nvdam.widen.net\/content\/tdwwiwotwr\/original\/gtc22-whitepaper-hopper.pdf."},{"key":"ref33","doi-asserted-by":"crossref","unstructured":"L. Pisha  and \nLigowski \u0141. , Accelerating non-power-of-2 size Fourier transforms with GPU tensor cores, in Proceedings of the 2021 IEEE International Parallel and Distributed Processing Symposium, Portland, OR, 2021, pp. 507\u2013516, https:\/\/doi.org\/10.1109\/IPDPS49936.2021.00059.","DOI":"10.1109\/IPDPS49936.2021.00059"},{"key":"ref34","doi-asserted-by":"crossref","unstructured":"A. Sorna , \nX. Cheng , \nE. D\u2019Azevedo , \nK. Won , and \nS. Tomov , Optimizing the fast Fourier transform using mixed precision on tensor core hardware, in Proceedings of the 25th IEEE International Conference on High Performance Computing Workshops, Bengaluru, India, 2018, pp. 3\u20137, https:\/\/doi.org\/10.1109\/hipcw.2018.8634417.","DOI":"10.1109\/HiPCW.2018.8634417"},{"key":"ref35","doi-asserted-by":"crossref","unstructured":"D. Yan , \nW. Wang , and \nX. Chu , Demystifying tensor cores to optimize half-precision matrix multiply, in Proceedings of the 2020 IEEE International Parallel and Distributed Processing Symposium, New Orleans, LA, 2020, pp. 634\u2013643, https:\/\/doi.org\/10.1109\/IPDPS47924.2020.00071.","DOI":"10.1109\/IPDPS47924.2020.00071"}],"container-title":["SIAM Journal on Scientific Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/21M1465032","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T15:39:25Z","timestamp":1787326765000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/21M1465032"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,28]]},"references-count":35,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,2,28]]}},"alternative-id":["10.1137\/21M1465032"],"URL":"https:\/\/doi.org\/10.1137\/21m1465032","relation":{},"ISSN":["1064-8275","1095-7197"],"issn-type":[{"value":"1064-8275","type":"print"},{"value":"1095-7197","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,28]]}}}