{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,2]],"date-time":"2025-08-02T14:33:22Z","timestamp":1754145202045,"version":"3.41.2"},"reference-count":84,"publisher":"Association for Computing Machinery (ACM)","issue":"ISSTA","funder":[{"name":"NSF grants","award":["NSF CCF2146443"],"award-info":[{"award-number":["NSF CCF2146443"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,22]]},"abstract":"<jats:p>General-purpose graphics processing unit (GPU) computing has emerged as a leading parallel computing paradigm, offering significant performance gains in various domains such as scientific computing and deep learning. However, GPU programs are susceptible to numerical bugs, which can lead to incorrect results or crashes. These bugs are difficult to detect, debug, and fix due to their dependence on specific input values or types and the absence of reliable error-checking mechanisms and oracles. Additionally, the unique programming conventions of GPUs complicate identifying the root causes of bugs, while fixing them requires domain-specific knowledge of GPU computing and numerical libraries. Therefore, understanding the characteristics of GPU numerical bugs (GPU-NBs) is crucial for developing effective solutions.<\/jats:p>\n          <jats:p>In this paper, we conduct a comprehensive study of GPU-NBs by analyzing 397 real-world bug samples from GitHub. We identify common root causes, symptoms, input patterns, test oracles that trigger these bugs and the strategies used to fix them. We also present GPU-NBDetect, a preliminary tool designed to detect numerical bugs across six distinct bug categories. GPU-NBDetect detected a total of 226 bugs across 186 mathematical functions in four libraries, with 60 of the bugs confirmed by developers. Our findings lay the groundwork for developing detection and prevention techniques for GPU-NBs and offer insights for building more effective debugging and auto-repair tool.<\/jats:p>","DOI":"10.1145\/3728950","type":"journal-article","created":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T10:52:56Z","timestamp":1750589576000},"page":"1654-1677","source":"Crossref","is-referenced-by-count":0,"title":["An Investigation on Numerical Bugs in GPU Programs Towards Automated Bug Detection"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-6129-2865","authenticated-orcid":false,"given":"Ravishka","family":"Rathnasuriya","sequence":"first","affiliation":[{"name":"University of Texas at Dallas, Richardson, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-2241-0126","authenticated-orcid":false,"given":"Nidhi","family":"Majoju","sequence":"additional","affiliation":[{"name":"University of Texas at Dallas, Richardson, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-6651-1560","authenticated-orcid":false,"given":"Zihe","family":"Song","sequence":"additional","affiliation":[{"name":"University of Texas at Dallas, Richardson, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5338-7347","authenticated-orcid":false,"given":"Wei","family":"Yang","sequence":"additional","affiliation":[{"name":"University of Texas at Dallas, Richardson, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,6,22]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"ARM. 2024. ARM architecture reference manual ARMv7-A and ARMv7-R edition. https:\/\/developer.arm.com\/documentation\/ddi0406\/c\/Application-Level-Architecture\/Application-Level-Programmers\u2013Model\/Floating-point-data-types-and-arithmetic\/NaN-handling-and-the-Default-NaN"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3182657"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2429069.2429133"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2384616.2384625"},{"volume-title":"Textbook of computer science for class XI. PHI Learning Pvt","author":"Bhatnagar Seema","key":"e_1_2_1_5_1","unstructured":"Seema Bhatnagar. 2008. Textbook of computer science for class XI. PHI Learning Pvt. Ltd.. isbn:978-81-203-2993-5 https:\/\/books.google.com\/books?id=bjE5EHw35DkC"},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the Third Workshop on Software Tools for MultiCore Systems. 33","author":"Boyer Michael","year":"2008","unstructured":"Michael Boyer, Kevin Skadron, and Westley Weimer. 2008. Automated dynamic analysis of CUDA programs. In Proceedings of the Third Workshop on Software Tools for MultiCore Systems. 33."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/IEEESTD.2019.8766229"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2017.8115662"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3062341.3062342"},{"key":"e_1_2_1_10_1","unstructured":"NVIDIA Developer Forums. 2012. Floating point operations difference between CPU and GPU. https:\/\/forums.developer.nvidia.com\/t\/floating-point-operations-difference-between-cpu-and-gpu\/27334"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3314221.3314632"},{"volume-title":"Ginkgo: Numerical linear algebra software package. https:\/\/github.com\/ginkgo-project\/ginkgo","year":"2021","key":"e_1_2_1_12_1","unstructured":"Ginkgo-Project. 2021. Ginkgo: Numerical linear algebra software package. https:\/\/github.com\/ginkgo-project\/ginkgo"},{"key":"e_1_2_1_13_1","unstructured":"GitHub. 2021. GitHub Archive. https:\/\/archiveprogram.github.com\/"},{"key":"e_1_2_1_14_1","unstructured":"Google. 2021. TensorFlow: TensorFlow 2.2 using tf.float16 executes only on CPU (issue #41783). https:\/\/github.com\/tensorflow\/tensorflow\/issues\/41783"},{"key":"e_1_2_1_15_1","unstructured":"Google. 2021. TensorFlow: TFLite conv_2D unsupported data type for float32 tensor (issue #40357). https:\/\/github.com\/tensorflow\/tensorflow\/issues\/40357"},{"key":"e_1_2_1_16_1","unstructured":"Google. 2023. TensorFlow: Mixed precision. https:\/\/www.tensorflow.org\/guide\/mixed_precision##overview"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/Correctness54621.2021.00007"},{"key":"e_1_2_1_18_1","unstructured":"GPUNB. 2024. GPU program numerical bug study. https:\/\/gpu-program-bug-study.github.io\/Comprehensive-Study-on-GPU-Program-Numerical-Issues.github.io\/"},{"key":"e_1_2_1_19_1","volume-title":"Deep learning framework power scores","author":"Hales Jeff","year":"2018","unstructured":"Jeff Hales. 2018. Deep learning framework power scores 2018. https:\/\/www.kaggle.com\/code\/discdiver\/deep-learning-framework-power-scores-2018"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3441830"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPEC.2017.8091072"},{"key":"e_1_2_1_22_1","unstructured":"IBM. 2021. Numbers. https:\/\/www.ibm.com\/docs\/en\/idr\/11.4.0?topic=types-numbers"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2018.00028"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC48688.2020.0-156"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510095"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00118"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC41404.2022.00038"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3520313.3534655"},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of the BSD Conference. 5, 1\u201320","author":"Lattner Chris","year":"2008","unstructured":"Chris Lattner. 2008. LLVM and Clang: Next generation compiler technology. In Proceedings of the BSD Conference. 5, 1\u201320."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2370036.2145844"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2954143"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588195.3592991"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC57359.2022.00046"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2024.112226"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-76526-6"},{"key":"e_1_2_1_36_1","unstructured":"NVIDIA. 2009. NVIDIA\u2019s next generation CUDA compute architecture: Fermi v1.1. https:\/\/www.nvidia.com\/content\/PDF\/fermi_white_papers\/NVIDIA_Fermi_Compute_Architecture_Whitepaper.pdf"},{"key":"e_1_2_1_37_1","unstructured":"NVIDIA. 2012. NVIDIA\u2019s next generation CUDA compute architecture: Kepler GK110\/210 v1.1. https:\/\/www.nvidia.com\/content\/dam\/en-zz\/Solutions\/Data-Center\/tesla-product-literature\/NVIDIA-Kepler-GK110-GK210-Architecture-Whitepaper.pdf"},{"key":"e_1_2_1_38_1","volume-title":"AMGX: Algebraic multigrid solver (AmgX) library. https:\/\/developer.nvidia.com\/amgx","author":"NVIDIA.","year":"2019","unstructured":"NVIDIA. 2019. AMGX: Algebraic multigrid solver (AmgX) library. https:\/\/developer.nvidia.com\/amgx"},{"key":"e_1_2_1_39_1","unstructured":"NVIDIA. 2019. CUDA math library. https:\/\/developer.nvidia.com\/cuda-math-library"},{"key":"e_1_2_1_40_1","unstructured":"NVIDIA. 2021. cuBLAS: Basic linear algebra on NVIDIA GPUs. https:\/\/developer.nvidia.com\/cublas"},{"key":"e_1_2_1_41_1","unstructured":"NVIDIA. 2021. cuFFT: CUDA fast Fourier transform library. https:\/\/developer.nvidia.com\/cufft"},{"key":"e_1_2_1_42_1","unstructured":"NVIDIA. 2021. cuRAND: Random number generation on NVIDIA GPUs. https:\/\/developer.nvidia.com\/curand"},{"key":"e_1_2_1_43_1","unstructured":"NVIDIA. 2021. cuSOLVER: Direct linear solvers on NVIDIA GPUs. https:\/\/developer.nvidia.com\/cusolver"},{"key":"e_1_2_1_44_1","unstructured":"NVIDIA. 2021. cuSPARSE: GPU library APIs for sparse computation. https:\/\/developer.nvidia.com\/cusparse"},{"key":"e_1_2_1_45_1","unstructured":"NVIDIA. 2021. cuTENSOR: A high-performance CUDA library for tensor primitives. https:\/\/docs.nvidia.com\/cuda\/cutensor\/latest\/index.html"},{"key":"e_1_2_1_46_1","volume-title":"MATX: GPU-accelerated numerical computing in modern C++. https:\/\/github.com\/NVIDIA\/MatX","author":"NVIDIA.","year":"2021","unstructured":"NVIDIA. 2021. MATX: GPU-accelerated numerical computing in modern C++. https:\/\/github.com\/NVIDIA\/MatX"},{"key":"e_1_2_1_47_1","unstructured":"NVIDIA. 2021. RAPIDS AI cuDF: [Bug] Cast from decimal64 to decimal128 is really slow (issue #9597). https:\/\/github.com\/rapidsai\/cudf\/issues\/9597"},{"key":"e_1_2_1_48_1","unstructured":"NVIDIA. 2021. RAPIDS AI cuDF: [Bug] CUDA error invalid value when creating cudf.series from float16 CuPy series (issue #9065). https:\/\/github.com\/rapidsai\/cudf\/issues\/9065"},{"key":"e_1_2_1_49_1","unstructured":"NVIDIA. 2021. RAPIDS AI cuDF: [Bug] libcudf unary cast returns unexpected results when casting between decimal and int types (issue #7689). https:\/\/github.com\/rapidsai\/cudf\/issues\/7689"},{"key":"e_1_2_1_50_1","unstructured":"NVIDIA. 2023. CUDA C++ programming guide. https:\/\/docs.nvidia.com\/cuda\/cuda-c-programming-guide\/index.html##floating-point-standard"},{"key":"e_1_2_1_51_1","volume-title":"Thrust: The C++ parallel algorithms library. https:\/\/docs.nvidia.com\/cuda\/thrust\/index.html","author":"NVIDIA.","year":"2023","unstructured":"NVIDIA. 2023. Thrust: The C++ parallel algorithms library. https:\/\/docs.nvidia.com\/cuda\/thrust\/index.html"},{"key":"e_1_2_1_52_1","unstructured":"NVIDIA. 2025. NVIDIA graphics cards. https:\/\/www.nvidia.com\/en-us\/geforce\/graphics-cards\/"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00107"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.22360\/SpringSim.2016.HPC.032"},{"key":"e_1_2_1_55_1","unstructured":"Meta Research. 2022. PyTorch: torch.norm gives NaN gradient when I input small-value float16 tensor (issue #43211). https:\/\/github.com\/pytorch\/pytorch\/issues\/43211"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468591"},{"key":"e_1_2_1_57_1","first-page":"44","volume-title":"Proceedings of the 33rd USENIX Security Symposium (USENIX Security). 5341\u20135358","author":"Solt Flavien","year":"2024","unstructured":"Flavien Solt, Katharina Ceesay-Seitz, and Kaveh Razavi. 2024. Cascade: CPU fuzzing via intricate program generation. In Proceedings of the 33rd USENIX Security Symposium (USENIX Security). 5341\u20135358. isbn:978-1-939133-44-1"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908080.2908114"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2017.8167778"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/1831708.1831724"},{"key":"e_1_2_1_61_1","unstructured":"CuPy Development Team. 2021. CuPy: NumPy and SciPy for GPU. https:\/\/github.com\/cupy\/cupy"},{"key":"e_1_2_1_62_1","unstructured":"CuPy Development Team. 2021. CuPy: Passing CuPy functions into cupyx.scipy.ndimage.filters.generic_filter causes a TypeError (issue #3909). https:\/\/github.com\/cupy\/cupy\/issues\/3909"},{"key":"e_1_2_1_63_1","unstructured":"CuPy Development Team. 2021. CuPy: cp.matmul slower than np.matmul (issue #4891). https:\/\/github.com\/cupy\/cupy\/issues\/4891"},{"key":"e_1_2_1_64_1","unstructured":"CuPy Development Team. 2021. CuPy: ndimage.map_coordinates performs incorrectly with several order values (issue #4550). https:\/\/github.com\/cupy\/cupy\/issues\/4550"},{"key":"e_1_2_1_65_1","unstructured":"CuPy Development Team. 2021. cupyx.scipy.signal.convolve2d: Integer overflow when using mixed dtypes (issue #6047). https:\/\/github.com\/cupy\/cupy\/issues\/6047"},{"key":"e_1_2_1_66_1","unstructured":"CuPy Development Team. 2022. CuPy: Unexpected NaN when using big-endian arrays (issue #3652). https:\/\/github.com\/cupy\/cupy\/issues\/3652"},{"key":"e_1_2_1_67_1","unstructured":"CuPy Development Team. 2022. CuPy: User-defined kernels. https:\/\/docs.cupy.dev\/en\/stable\/user_guide\/kernel.html"},{"key":"e_1_2_1_68_1","volume-title":"Numba: NumPy aware dynamic Python compiler using LLVM. https:\/\/github.com\/numba\/numba","author":"Team Numba Development","year":"2021","unstructured":"Numba Development Team. 2021. Numba: NumPy aware dynamic Python compiler using LLVM. https:\/\/github.com\/numba\/numba"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2015.7056044"},{"key":"e_1_2_1_70_1","unstructured":"Vedantu. 2022. Numerical expression. https:\/\/www.vedantu.com\/maths\/numerical-expression"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3559561"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950355"},{"key":"e_1_2_1_73_1","first-page":"18749","article-title":"Precision & performance: Floating point and IEEE 754 compliance for NVIDIA GPUs","volume":"21","author":"Whitehead Nathan","year":"2011","unstructured":"Nathan Whitehead and Alex Fit-Florea. 2011. Precision & performance: Floating point and IEEE 754 compliance for NVIDIA GPUs. NVIDIA Documentation, 21 (2011), 18749\u201319424.","journal-title":"NVIDIA Documentation"},{"key":"e_1_2_1_74_1","unstructured":"Nicholas Wilt. 2013. The CUDA Handbook: A Comprehensive Guide to GPU Programming. Pearson Education."},{"key":"e_1_2_1_75_1","unstructured":"Nicholas Wilt. 2013. Floating point: CPU and GPU differences. http:\/\/www.cudahandbook.com\/2013\/08\/floating-point-cpu-and-gpu-differences\/"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380358"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3508035"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468612"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPP.2012.30"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290369"},{"volume-title":"The Fuzzing Book","author":"Zeller Andreas","key":"e_1_2_1_81_1","unstructured":"Andreas Zeller, Rahul Gopinath, Marcel B\u00f6hme, Gordon Fraser, and Christian Holler. 2023. Greybox Fuzzing. In The Fuzzing Book. CISPA Helmholtz Center for Information Security. https:\/\/www.fuzzingbook.org\/html\/GreyboxFuzzer.html"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409720"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.70"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1145\/3371128"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3728950","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,16]],"date-time":"2025-07-16T16:48:18Z","timestamp":1752684498000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3728950"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,22]]},"references-count":84,"journal-issue":{"issue":"ISSTA","published-print":{"date-parts":[[2025,6,22]]}},"alternative-id":["10.1145\/3728950"],"URL":"https:\/\/doi.org\/10.1145\/3728950","relation":{},"ISSN":["2994-970X"],"issn-type":[{"type":"electronic","value":"2994-970X"}],"subject":[],"published":{"date-parts":[[2025,6,22]]}}}