{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T00:01:41Z","timestamp":1775692901731,"version":"3.50.1"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,2,15]],"date-time":"2024-02-15T00:00:00Z","timestamp":1707955200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2024,6,30]]},"abstract":"<jats:p>General Matrix Multiplication (GEMM) is a computationally expensive operation that is used in many applications such as machine learning. Hardware accelerators are increasingly popular for speeding up GEMM computation, with Tiled Matrix Multiplication (TMUL) in recent Intel processors being an example. Unfortunately, the TMUL hardware is susceptible to errors, necessitating online error detection. The Algorithm-based Error Detection (ABED) technique is a powerful technique to detect errors in matrix multiplications. In this article, we consider implementation of an ABED technique that integrates seamlessly with the TMUL hardware to minimize performance overhead. Unfortunately, rounding errors introduced by floating-point operations do not allow a straightforward implementation of ABED in TMUL. Previously an error bound was considered for addressing rounding errors in ABED. If the error detection threshold is set too low, it will a trigger false alarm, while a loose bound will allow errors to escape detection. In this article, we propose an adaptive error threshold that takes into account the TMUL input values to address the problem of false triggers and error escapes and provide a taxonomy of various error classes. This threshold is obtained from theoretical error analysis but is not easy to implement in hardware. Consequently, we relax the threshold such that it can be easily computed in hardware. While ABED ensures error-free computation, it does not guarantee full coverage of all hardware faults. To address this problem, we propose an algorithmic pattern generation technique to ensure full coverage for all hardware faults. To evaluate the benefits of our proposed solution, we conducted fault injection experiments and show that our approach does not produce any false alarms or detection escapes for observable errors. We conducted additional fault injection experiments on a Deep Neural Network (DNN) model and find that if a fault is not detected, it does not cause any misclassification.<\/jats:p>","DOI":"10.1145\/3633332","type":"journal-article","created":{"date-parts":[[2023,11,22]],"date-time":"2023-11-22T12:32:17Z","timestamp":1700656337000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Highly Efficient Self-checking Matrix Multiplication on Tiled AMX Accelerators"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3600-9432","authenticated-orcid":false,"given":"Chandra Sekhar","family":"Mummidi","sequence":"first","affiliation":[{"name":"University of Massachusetts, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6238-5856","authenticated-orcid":false,"given":"Victor C.","family":"Ferreira","sequence":"additional","affiliation":[{"name":"Intel Corporation, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-8662-5820","authenticated-orcid":false,"given":"Sudarshan","family":"Srinivasan","sequence":"additional","affiliation":[{"name":"Intel Corporation, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8221-3824","authenticated-orcid":false,"given":"Sandip","family":"Kundu","sequence":"additional","affiliation":[{"name":"University of Massachusetts, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,2,15]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"BFLOAT16: The secret to high performance on cloud TPUs. Google cloud blog. Retrieved July 26 2023 from https:\/\/cloud.google.com\/blog\/products\/ai-machine-learning\/bfloat16-the-secret-to-high-performance-on-cloud-tpus"},{"key":"e_1_3_1_3_2","unstructured":"ImageNet. Retrieved July 26 2023 from https:\/\/image-net.org\/"},{"key":"e_1_3_1_4_2","unstructured":"Intel\u00ae C++ Compiler 19.1 Developer Guide and Reference. Retrieved July 26 2023 from https:\/\/www.intel.com\/content\/www\/us\/en\/develop\/documentation\/cpp-compiler-developer-guide-and-reference\/top.html"},{"key":"e_1_3_1_5_2","first-page":"99","volume-title":"2020 IEEE 32nd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD \u201920)","author":"Anderson Andrew","year":"2020","unstructured":"Andrew Anderson, Aravind Vasudevan, Cormac Keane, and David Gregg. 2020. High-performance low-memory lowering: GEMM-based algorithms for DNN convolution. In 2020 IEEE 32nd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD \u201920). IEEE, 99\u2013106."},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.3390\/mti2030047"},{"key":"e_1_3_1_7_2","first-page":"1","volume-title":"2023 Design, Automation & Test in Europe Conference & Exhibition (DATE \u201923)","author":"Bal Sandeep","year":"2023","unstructured":"Sandeep Bal, Chandra Sekhar Mummidi, Victor Da Cruz Ferreira, Sudarshan Srinivasan, and Sandip Kundu. 2023. A novel fault-tolerant architecture for tiled matrix multiplication. In 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE \u201923). IEEE, 1\u20136."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.44381"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/12.57055"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2004.4"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDMR.2005.853449"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/567806.567807"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2008.12.002"},{"key":"e_1_3_1_14_2","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1109\/DSN.2014.48","volume-title":"2014 44th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks","author":"Braun Claus","year":"2014","unstructured":"Claus Braun, Sebastian Halder, and Hans Joachim Wunderlich. 2014. A-ABFT: Autonomous algorithm-based fault tolerance for matrix multiplications on graphics processing units. In 2014 44th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks. IEEE, 443\u2013454."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSPEC.2019.8701189"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1109\/DSN.2008.4630077","volume-title":"2008 IEEE International Conference on Dependable Systems and Networks with FTCS and DCC (DSN \u201908)","author":"Constantinescu Cristian","year":"2008","unstructured":"Cristian Constantinescu, Ishwar Parulkar, Rick Harper, and Sarah Michalak. 2008. Silent data corruption\u2013Myth or reality? In 2008 IEEE International Conference on Dependable Systems and Networks with FTCS and DCC (DSN \u201908). IEEE, 108\u2013109."},{"key":"e_1_3_1_17_2","article-title":"Silent data corruptions at scale","author":"Dixit Harish Dattatraya","year":"2021","unstructured":"Harish Dattatraya Dixit, Sneha Pendharkar, Matt Beadon, Chris Mason, Tejasvi Chakravarthy, Bharath Muthiah, and Sriram Sankar. 2021. Silent data corruptions at scale. arXiv preprint arXiv:2102.11245 (2021).","journal-title":"arXiv preprint arXiv:2102.11245"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2018.2878387"},{"key":"e_1_3_1_19_2","article-title":"HPE Superdome Flex server architecture and RAS","volume":"4","author":"Enterprise Hewlett Packard","year":"2017","unstructured":"Hewlett Packard Enterprise. 2017. HPE Superdome Flex server architecture and RAS. Technical White Paper, a00036491ENW, Rev 4.","journal-title":"Technical White Paper, a00036491ENW, Rev"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368086"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21918"},{"key":"e_1_3_1_22_2","article-title":"https:\/\/software. intel. com\/sites\/landingpage","author":"Guide Intel Intrinsics","year":"2015","unstructured":"Intel Intrinsics Guide. 2015. https:\/\/software. intel. com\/sites\/landingpage IntrinsicsGuide (: 31.08. 2016) (2015).","journal-title":"IntrinsicsGuide (: 31.08. 2016)"},{"key":"e_1_3_1_23_2","doi-asserted-by":"crossref","first-page":"488","DOI":"10.1109\/HPCA47549.2020.00047","volume-title":"2020 IEEE International Symposium on High Performance Computer Architecture (HPCA \u201920)","author":"Gupta Udit","year":"2020","unstructured":"Udit Gupta, Carole-Jean Wu, Xiaodong Wang, Maxim Naumov, Brandon Reagen, David Brooks, Bradford Cottel, Kim Hazelwood, Mark Hempstead, Bill Jia, et\u00a0al. 2020. The architectural implications of Facebook\u2019s DNN-based personalized recommendation. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA \u201920). IEEE, 488\u2013501."},{"key":"e_1_3_1_24_2","article-title":"Making convolutions resilient via algorithm-based error detection techniques","author":"Hari Siva Kumar Sastry","year":"2021","unstructured":"Siva Kumar Sastry Hari, Michael Sullivan, Timothy Tsai, and Stephen W. Keckler. 2021. Making convolutions resilient via algorithm-based error detection techniques. IEEE Transactions on Dependable and Secure Computing (2021).","journal-title":"IEEE Transactions on Dependable and Secure Computing"},{"key":"e_1_3_1_25_2","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1109\/HPCA.2018.00059","volume-title":"2018 IEEE International Symposium on High Performance Computer Architecture (HPCA \u201918)","author":"Hazelwood Kim","year":"2018","unstructured":"Kim Hazelwood, Sarah Bird, David Brooks, Soumith Chintala, Utku Diril, Dmytro Dzhulgakov, Mohamed Fawzy, Bill Jia, Yangqing Jia, Aditya Kalro, et\u00a0al. 2018. Applied machine learning at Facebook: A datacenter infrastructure perspective. In 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA \u201918). IEEE, 620\u2013629."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458336.3465297"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.1984.1676475"},{"key":"e_1_3_1_29_2","article-title":"Accelerate AI with AMX","year":"2022","unstructured":"Intel. 2022. Accelerate AI with AMX. Retrieved May 12, 2023, from https:\/\/www.intel.com\/content\/dam\/www\/central-libraries\/us\/en\/documents\/2022-12\/accelerate-ai-with-amx-sb.pdf","journal-title":"R"},{"key":"e_1_3_1_30_2","article-title":"Performance of SSE and AVX instruction sets","author":"Jeong Hwancheol","year":"2012","unstructured":"Hwancheol Jeong, Sunghoon Kim, Weonjong Lee, and Seok-Ho Myung. 2012. Performance of SSE and AVX instruction sets. arXiv preprint arXiv:1211.0820 (2012).","journal-title":"arXiv preprint arXiv:1211.0820"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCAD.2017.8203882"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3079856.3080246"},{"key":"e_1_3_1_33_2","article-title":"A study of BFLOAT16 for deep learning training","author":"Kalamkar Dhiraj","year":"2019","unstructured":"Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja Vooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, et\u00a0al. 2019. A study of BFLOAT16 for deep learning training. arXiv:1905.12322 (2019).","journal-title":"arXiv:1905.12322"},{"key":"e_1_3_1_34_2","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1109\/LLVMHPC54804.2021.00008","volume-title":"2021 IEEE\/ACM 7th Workshop on the LLVM Compiler Infrastructure in HPC (LLVM-HPC \u201921)","author":"Khaldi Dounia","year":"2021","unstructured":"Dounia Khaldi, Yuanke Luo, Bing Yu, Alexey Sotkin, Bruno Morais, and Milind Girkar. 2021. Extending LLVM IR for DPC++ matrix support: A case study with Intel\u00ae Advanced Matrix Extensions (Intel\u00ae AMX). In 2021 IEEE\/ACM 7th Workshop on the LLVM Compiler Infrastructure in HPC (LLVM-HPC \u201921). IEEE, 20\u201326."},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3065386"},{"key":"e_1_3_1_36_2","volume-title":"IBM Z14 Model ZR1 Technical Introduction","author":"Lascu Octavian","year":"2018","unstructured":"Octavian Lascu, Bill White, Hervey Kamga, Frank Packheiser, Martijn Raave, John Troy, et\u00a0al. 2018. IBM Z14 Model ZR1 Technical Introduction. IBM Redbooks."},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCAS.2010.5537907"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.3390\/aerospace7060073"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3126908.3126964"},{"key":"e_1_3_1_40_2","article-title":"Efficient soft-error detection for low-precision deep learning recommendation models","author":"Li Sihuan","year":"2021","unstructured":"Sihuan Li, Jianyu Huang, Ping Tak Peter Tang, Daya Khudia, Jongsoo Park, Harish Dattatraya Dixit, and Zizhong Chen. 2021. Efficient soft-error detection for low-precision deep learning recommendation models. arXiv preprint arXiv:2103.00130 (2021).","journal-title":"arXiv preprint arXiv:2103.00130"},{"key":"e_1_3_1_41_2","article-title":"Enabling sparse Winograd convolution by native pruning","author":"Li Sheng","year":"2017","unstructured":"Sheng Li, Jongsoo Park, and Ping Tak Peter Tang. 2017. Enabling sparse Winograd convolution by native pruning. arXiv preprint arXiv:1702.08597 (2017).","journal-title":"arXiv preprint arXiv:1702.08597"},{"key":"e_1_3_1_42_2","first-page":"529","volume-title":"2022 IEEE 40th International Conference on Computer Design (ICCD \u201922)","author":"Mummidi Chandra Sekhar","year":"2022","unstructured":"Chandra Sekhar Mummidi, Sandeep Bal, Brunno F. Goldstein, Sudarshan Srinivasan, and Sandip Kundu. 2022. A highly-efficient error detection technique for general matrix multiplication using tiled processing on SIMD architecture. In 2022 IEEE 40th International Conference on Computer Design (ICCD \u201922). IEEE, 529\u2013536."},{"key":"e_1_3_1_43_2","volume-title":"Proceedings of the 6th Conference on Computer Systems","author":"Nightingale Edmund B.","year":"2011","unstructured":"Edmund B. Nightingale, John R. Douceur, and Vince Orgovan. 2011. Cycles, cells and platters: An empirical analysis of hardware failures on a million consumer PCs. In Proceedings of the 6th Conference on Computer Systems."},{"key":"e_1_3_1_44_2","unstructured":"NVIDIA. cuBLAS 12.2 documentation. Retrieved July 26 2023 from https:\/\/docs.nvidia.com\/cuda\/cublas\/"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/QEST.2012.37"},{"key":"e_1_3_1_46_2","doi-asserted-by":"crossref","unstructured":"Semeen Rehman. 2015. Reliable software for unreliable hardware-a cross-layer approach. (2015).","DOI":"10.1007\/978-3-319-25772-3"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00474"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVLSI.2023.3238907"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2011.6033395"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TC.1980.1675557"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/MM.2020.2975764"},{"key":"e_1_3_1_52_2","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1007\/978-3-319-06486-4_7","volume-title":"High-Performance Computing on the Intel\u00ae Xeon Phi\u2122","author":"Wang Endong","year":"2014","unstructured":"Endong Wang, Qing Zhang, Bo Shen, Guangyong Zhang, Xiaowei Lu, Qing Wu, and Yajuan Wang. 2014. Intel math kernel library. In High-Performance Computing on the Intel\u00ae Xeon Phi\u2122. Springer, 167\u2013188."},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2905842"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503222.3507767"},{"key":"e_1_3_1_55_2","article-title":"Ai matrix: A deep learning benchmark for Alibaba data centers","author":"Zhang Wei","year":"2019","unstructured":"Wei Zhang, Wei Wei, Lingjie Xu, Lingling Jin, and Cheng Li. 2019. Ai matrix: A deep learning benchmark for Alibaba data centers. arXiv preprint arXiv:1909.10562 (2019).","journal-title":"arXiv preprint arXiv:1909.10562"}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3633332","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3633332","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:54:01Z","timestamp":1750287241000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3633332"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,15]]},"references-count":54,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,6,30]]}},"alternative-id":["10.1145\/3633332"],"URL":"https:\/\/doi.org\/10.1145\/3633332","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,15]]},"assertion":[{"value":"2023-07-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-03","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-15","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}