{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T01:44:04Z","timestamp":1787017444787,"version":"build-2736575974"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"5s","license":[{"start":{"date-parts":[[2021,9,17]],"date-time":"2021-09-17T00:00:00Z","timestamp":1631836800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"US National Science Foundation","doi-asserted-by":"crossref","award":["CNS-1955353, CNS-1955196"],"award-info":[{"award-number":["CNS-1955353, CNS-1955196"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000183","name":"USA Army Research Office","doi-asserted-by":"crossref","award":["W911NF-17-1-0485"],"award-info":[{"award-number":["W911NF-17-1-0485"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"crossref"}]},{"name":"NSF to the Computing Research Association for the CIFellows Project","award":["#2030859"],"award-info":[{"award-number":["#2030859"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2021,10,31]]},"abstract":"<jats:p>\n                    The growing popularity of convolutional neural networks (CNNs) has led to the search for efficient computational platforms to accelerate CNN training. Resistive random-access memory (ReRAM)-based manycore architectures offer a promising alternative to commonly used GPU-based platforms for training CNNs. However, due to the immature fabrication process and limited write endurance, ReRAMs suffer from different types of faults. This makes training of CNNs challenging as weights are misrepresented when they are mapped to faulty ReRAM cells. This results in unstable training, leading to unacceptably low accuracy for the trained model. Due to the distributed nature of the mapping of the individual bits of a weight to different ReRAM cells, faulty weights often lead to exploding gradients. This in turn introduces a positive feedback in the training loop, resulting in extremely large and unstable weights. In this paper, we propose a lightweight and reliable CNN training methodology using weight clipping to prevent this phenomenon and enable training even in the presence of many faults. Weight clipping prevents large weights from destabilizing CNN training and provides the backpropagation algorithm with the opportunity to compensate for the weights mapped to faulty cells. The proposed methodology achieves near-GPU accuracy without introducing significant area or performance overheads. Experimental evaluation indicates that weight clipping enables the successful training of CNNs in the presence of faults, while also reducing training time by 4\n                    <jats:italic>X<\/jats:italic>\n                    on average compared to a conventional GPU platform. Moreover, we also demonstrate that weight clipping outperforms a recently proposed error correction code (ECC)-based method when training is carried out using faulty ReRAMs.\n                  <\/jats:p>","DOI":"10.1145\/3476986","type":"journal-article","created":{"date-parts":[[2021,9,17]],"date-time":"2021-09-17T14:36:51Z","timestamp":1631889411000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":17,"title":["Learning to Train CNNs on Faulty ReRAM-based Manycore Accelerators"],"prefix":"10.1145","volume":"20","author":[{"given":"Biresh Kumar","family":"Joardar","sequence":"first","affiliation":[{"name":"Duke University, Durham, NC"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Janardhan Rao","family":"Doppa","sequence":"additional","affiliation":[{"name":"Washington State University, Pullman, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hai","family":"Li","sequence":"additional","affiliation":[{"name":"Washington State University, Durham, NC"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Krishnendu","family":"Chakrabarty","sequence":"additional","affiliation":[{"name":"Duke University, Durham, NC"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Partha Pratim","family":"Pande","sequence":"additional","affiliation":[{"name":"Washington State University, Pullman, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,9,17]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"2017 IEEE International Symposium on High Performance Computer Architecture (HPCA), Austin, TX, USA 2017, 541\u2014552","author":"Song L.","unstructured":"L. Song , X. Qian , H. Li , and Y. Chen . 2017. PipeLayer: A pipelined reram-based accelerator for deep learning . 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA), Austin, TX, USA 2017, 541\u2014552 . L. Song, X. Qian, H. Li, and Y. Chen. 2017. PipeLayer: A pipelined reram-based accelerator for deep learning. 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA), Austin, TX, USA 2017, 541\u2014552."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3007787.3001139"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2020.2998456"},{"key":"e_1_2_1_4_1","doi-asserted-by":"crossref","unstructured":"B. K. Joardar J. R. Doppa P. P. Pande H. Li and K. Chakrabarty. 2020. AccuReD: High accuracy training of CNNs on ReRAM\/GPU heterogeneous 3D architecture. In IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 2020.  B. K. Joardar J. R. Doppa P. P. Pande H. Li and K. Chakrabarty. 2020. AccuReD: High accuracy training of CNNs on ReRAM\/GPU heterogeneous 3D architecture. In IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 2020.","DOI":"10.1109\/TCAD.2020.3013194"},{"key":"e_1_2_1_5_1","first-page":"6","volume-title":"2019 56th ACM\/IEEE Design Automation Conference (DAC), Las Vegas, NV, USA 2019","author":"He Z.","unstructured":"Z. He , J. Lin , R. Ewetz , J. Yuan , and D. Fan . 2019. Noise injection adaption: End-to-End ReRAM crossbar non-ideal effect adaption for neural network mapping . 2019 56th ACM\/IEEE Design Automation Conference (DAC), Las Vegas, NV, USA 2019 , pp. 1\u2013 6 . Z. He, J. Lin, R. Ewetz, J. Yuan, and D. Fan. 2019. Noise injection adaption: End-to-End ReRAM crossbar non-ideal effect adaption for neural network mapping. 2019 56th ACM\/IEEE Design Automation Conference (DAC), Las Vegas, NV, USA 2019, pp. 1\u20136."},{"key":"e_1_2_1_6_1","first-page":"10","volume-title":"2018 IEEE International Test Conference (ITC), Phoenix, AZ, USA 2018","author":"Chaudhuri A.","unstructured":"A. Chaudhuri and K. Chakrabarty . 2018. Analysis of process variations, defects, and design-induced coupling in memristors . 2018 IEEE International Test Conference (ITC), Phoenix, AZ, USA 2018 , pp. 1\u2013 10 . A. Chaudhuri and K. Chakrabarty. 2018. Analysis of process variations, defects, and design-induced coupling in memristors. 2018 IEEE International Test Conference (ITC), Phoenix, AZ, USA 2018, pp. 1\u201310."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2018.2855145"},{"key":"e_1_2_1_8_1","first-page":"10","volume-title":"2020 IEEE International Test Conference (ITC), Washington, DC, USA 2020","author":"Liu M.","unstructured":"M. Liu and K. Chakrabarty . 2020. Online fault detection in ReRAM-Based computing systems by monitoring dynamic power consumption . 2020 IEEE International Test Conference (ITC), Washington, DC, USA 2020 , pp. 1\u2013 10 . M. Liu and K. Chakrabarty. 2020. Online fault detection in ReRAM-Based computing systems by monitoring dynamic power consumption. 2020 IEEE International Test Conference (ITC), Washington, DC, USA 2020, pp. 1\u201310."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2018.00015"},{"key":"e_1_2_1_10_1","first-page":"1","article-title":"Rescuing memristor-based neuromorphic design with high defects. In Proceedings of the 54th Annual Design Automation Conference 2017 (DAC'17). Association for Computing Machinery, New York, NY, USA","volume":"87","author":"Liu C.","year":"2017","unstructured":"C. Liu , M. Hu , J. P. Strachan , and H. Li . 2017 . Rescuing memristor-based neuromorphic design with high defects. In Proceedings of the 54th Annual Design Automation Conference 2017 (DAC'17). Association for Computing Machinery, New York, NY, USA , Article 87 , 1 \u2013 6 . C. Liu, M. Hu, J. P. Strachan, and H. Li. 2017. Rescuing memristor-based neuromorphic design with high defects. In Proceedings of the 54th Annual Design Automation Conference 2017 (DAC'17). Association for Computing Machinery, New York, NY, USA, Article 87, 1\u20136.","journal-title":"Article"},{"key":"e_1_2_1_11_1","first-page":"527","volume-title":"Automation & Test in Europe Conference & Exhibition (DATE)","author":"Joardar B. K.","year":"2019","unstructured":"B. K. Joardar , B. Li , J. R. Doppa , H. Li , P. P. Pande , and K. Chakrabarty . 2019. REGENT: A heterogeneous ReRAM\/GPU-based architecture enabled by NoC for training CNNs. 2019 Design , Automation & Test in Europe Conference & Exhibition (DATE) , Florence, Italy 2019 , pp. 522\u2013 527 . B. K. Joardar, B. Li, J. R. Doppa, H. Li, P. P. Pande, and K. Chakrabarty. 2019. REGENT: A heterogeneous ReRAM\/GPU-based architecture enabled by NoC for training CNNs. 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE), Florence, Italy 2019, pp. 522\u2013527."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.2014.58"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3375699"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2014.12"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TED.2019.2894387"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/IEDM13553.2020.9372019"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/IEDM.2013.6724685"},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'18)","author":"Fujiki D.","unstructured":"D. Fujiki , S. Mahlke , and R. Das . 2018. In-Memory data parallel processor . In Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'18) . Association for Computing Machinery, New York, NY, USA, 1\u201314. D. Fujiki, S. Mahlke, and R. Das. 2018. In-Memory data parallel processor. In Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'18). Association for Computing Machinery, New York, NY, USA, 1\u201314."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNANO.2013.2253329"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/NVMTS.2015.7457488"},{"key":"e_1_2_1_21_1","first-page":"10","volume-title":"2018 IEEE International Test Conference (ITC), Phoenix, AZ, USA 2018","author":"Liu M.","unstructured":"M. Liu , L. Xia , Y. Wang , and K. Chakrabarty . 2018. Fault tolerance for RRAM-Based matrix operations . 2018 IEEE International Test Conference (ITC), Phoenix, AZ, USA 2018 , pp. 1\u2013 10 . M. Liu, L. Xia, Y. Wang, and K. Chakrabarty. 2018. Fault tolerance for RRAM-Based matrix operations. 2018 IEEE International Test Conference (ITC), Phoenix, AZ, USA 2018, pp. 1\u201310."},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the ACM\/IEEE International Symposium on Low Power Electronics and Design (ISLPED'20)","author":"Lou Q.","unstructured":"Q. Lou , T. Gao , P. Faley , M. Niemier , X. S. Hu , and S. Joshi . 2020. Embedding error correction into crossbars for reliable matrix vector multiplication using emerging devices . In Proceedings of the ACM\/IEEE International Symposium on Low Power Electronics and Design (ISLPED'20) . Association for Computing Machinery, New York, NY, USA, 139\u2013144. Q. Lou, T. Gao, P. Faley, M. Niemier, X. S. Hu, and S. Joshi. 2020. Embedding error correction into crossbars for reliable matrix vector multiplication using emerging devices. In Proceedings of the ACM\/IEEE International Symposium on Low Power Electronics and Design (ISLPED'20). Association for Computing Machinery, New York, NY, USA, 139\u2013144."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASSCC.2017.8240203"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the 2020 on Great Lakes Symposium on VLSI (GLSVLSI'20)","author":"Zhang B.","unstructured":"B. Zhang , N. Uysal , D. Fan , and R. Ewetz . 2020. Redundant neurons and shared redundant synapses for robust memristor-based dnns with reduced overhead . In Proceedings of the 2020 on Great Lakes Symposium on VLSI (GLSVLSI'20) . Association for Computing Machinery, New York, NY, USA, 339\u2013344. B. Zhang, N. Uysal, D. Fan, and R. Ewetz. 2020. Redundant neurons and shared redundant synapses for robust memristor-based dnns with reduced overhead. In Proceedings of the 2020 on Great Lakes Symposium on VLSI (GLSVLSI'20). Association for Computing Machinery, New York, NY, USA, 339\u2013344."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_2_1_26_1","volume-title":"ICLR","author":"Simonyan K.","year":"2015","unstructured":"K. Simonyan 2015 . Very deep convolutional networks for large-scale image recognition . In ICLR 2015. K. Simonyan et al. 2015. Very deep convolutional networks for large-scale image recognition. In ICLR 2015."},{"key":"e_1_2_1_27_1","volume-title":"CVPR","author":"He K.","year":"2016","unstructured":"K. He , X. Zhang , S. Ren , and J. Sun . 2016. Deep residual learning for image recognition . In CVPR 2016 . K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep residual learning for image recognition. In CVPR 2016."},{"key":"e_1_2_1_28_1","volume-title":"Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms. In arXiv preprint arXiv:1708.07747","author":"Xiao H.","year":"2017","unstructured":"H. Xiao , K. Rasul , and R. Vollgraf . 2017 . Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms. In arXiv preprint arXiv:1708.07747 , 2017. H. Xiao, K. Rasul, and R. Vollgraf. 2017. Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms. In arXiv preprint arXiv:1708.07747, 2017."},{"key":"e_1_2_1_29_1","unstructured":"A. Krizhevsky and G. Hinton. 2009. Learning multiple layers of features from tiny images 2009.  A. Krizhevsky and G. Hinton. 2009. Learning multiple layers of features from tiny images 2009."},{"key":"e_1_2_1_30_1","volume-title":"Deep neural networks are robust to weight binarization and other non-linear distortions. In arXiv:1606.01981","author":"Esser K.","year":"2016","unstructured":"Paul Merolla, Rathinakumar Appuswamy, John Arthur, Steve K. Esser , and Dharmendra Modha. 2016. Deep neural networks are robust to weight binarization and other non-linear distortions. In arXiv:1606.01981 , 2016 . Paul Merolla, Rathinakumar Appuswamy, John Arthur, Steve K. Esser, and Dharmendra Modha. 2016. Deep neural networks are robust to weight binarization and other non-linear distortions. In arXiv:1606.01981, 2016."},{"key":"e_1_2_1_31_1","first-page":"189","article-title":"Financial ratios, discriminant analysis and the prediction of corporate bankruptcy","volume":"23","author":"Altman E. I.","year":"1968","unstructured":"E. I. Altman . 1968 . Financial ratios, discriminant analysis and the prediction of corporate bankruptcy . Journal of Finance 23 (1968), 189 \u2013 209 . E. I. Altman. 1968. Financial ratios, discriminant analysis and the prediction of corporate bankruptcy. Journal of Finance 23 (1968), 189\u2013209.","journal-title":"Journal of Finance"},{"key":"e_1_2_1_32_1","volume-title":"ICLR","author":"Micikevicius P.","year":"2018","unstructured":"P. Micikevicius 2018 . Mixed precision training . In ICLR 2018. P. Micikevicius et al. 2018. Mixed precision training. In ICLR 2018."},{"key":"e_1_2_1_33_1","volume-title":"ICML","author":"Ioffe S.","year":"2015","unstructured":"S. Ioffe and C. Szegedy . Batch normalization: accelerating deep network training by reducing internal covariate shift . In ICML 2015 . S. Ioffe and C. Szegedy. Batch normalization: accelerating deep network training by reducing internal covariate shift. In ICML 2015."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2009.4919648"},{"key":"e_1_2_1_35_1","first-page":"5","volume-title":"2020 IEEE International Test Conference (ITC), Washington, DC, USA 2020","author":"Chaudhuri A.","unstructured":"A. Chaudhuri , J. Talukdar , F. Su , and K. Chakrabarty . 2020. Functional criticality classification of structural faults in ai accelerators . 2020 IEEE International Test Conference (ITC), Washington, DC, USA 2020 , pp. 1\u2013 5 . DOI:10.1109\/ITC44778.2020.9325272. 10.1109\/ITC44778.2020.9325272 A. Chaudhuri, J. Talukdar, F. Su, and K. Chakrabarty. 2020. Functional criticality classification of structural faults in ai accelerators. 2020 IEEE International Test Conference (ITC), Washington, DC, USA 2020, pp. 1\u20135. DOI:10.1109\/ITC44778.2020.9325272."},{"key":"e_1_2_1_36_1","volume-title":"2011 IEEE\/IFIP 19th International Conference","author":"Wang X.","year":"2011","unstructured":"X. Wang , M. Palesi , M. Yang , Y. Jiang , M. C. Huang , and P. Liu . 2011. Low latency and energy efficient multicasting schemes for 3D NoC-based SoCs . 2011 IEEE\/IFIP 19th International Conference , 2011 . X. Wang, M. Palesi, M. Yang, Y. Jiang, M. C. Huang, and P. Liu. 2011. Low latency and energy efficient multicasting schemes for 3D NoC-based SoCs. 2011 IEEE\/IFIP 19th International Conference, 2011."},{"key":"e_1_2_1_37_1","first-page":"233","volume-title":"Automation & Test in Europe Conference & Exhibition (DATE)","author":"Joardar B. K.","year":"2020","unstructured":"B. K. Joardar 2020 . GRAMARCH: A GPU-ReRAM based heterogeneous architecture for neural image segmentation. 2020 Design , Automation & Test in Europe Conference & Exhibition (DATE) 2020, pp. 228\u2013 233 . B. K. Joardar et al. 2020. GRAMARCH: A GPU-ReRAM based heterogeneous architecture for neural image segmentation. 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE) 2020, pp. 228\u2013233."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1561\/2400000003"},{"key":"e_1_2_1_39_1","volume-title":"In: Advances in Neural Information Processing Systems","author":"Krizhevsky A.","year":"2012","unstructured":"A. Krizhevsky , I. Sutskever , and G. E. Hinton . 2012 . Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012), 1097\u20131105. A. Krizhevsky, I. Sutskever, and G. E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012), 1097\u20131105."}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3476986","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3476986","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T15:30:46Z","timestamp":1750174246000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3476986"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,17]]},"references-count":39,"journal-issue":{"issue":"5s","published-print":{"date-parts":[[2021,10,31]]}},"alternative-id":["10.1145\/3476986"],"URL":"https:\/\/doi.org\/10.1145\/3476986","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"value":"1539-9087","type":"print"},{"value":"1558-3465","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,17]]},"assertion":[{"value":"2021-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-09-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}