{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T21:36:55Z","timestamp":1784065015669,"version":"3.55.0"},"reference-count":27,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2018,3,22]],"date-time":"2018-03-22T00:00:00Z","timestamp":1521676800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["4137411, 91530323, 61702297 and 61672312"],"award-info":[{"award-number":["4137411, 91530323, 61702297 and 61672312"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Key R8D Program of China","award":["2016YFA0602200"],"award-info":[{"award-number":["2016YFA0602200"]}]},{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"crossref","award":["2016M601031"],"award-info":[{"award-number":["2016M601031"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2018,3,31]]},"abstract":"<jats:p>The Sunway TaihuLight supercomputer is powered by SW26010, a new 260-core processor designed with on-chip fusion of heterogeneous cores. In this article, we present our work on optimizing the training process of convolutional neural networks (CNNs) on the Sunway TaihuLight supercomputer. Specifically, a highly efficient library (swDNN) and a customized Caffe framework (swCaffe) are proposed. Architecture-oriented optimization methods targeting the many-core architecture of SW26010 are introduced and are able to achieve 48\u00d7 speedup for the convolution routine in swDNN and 4\u00d7 speedup for the complete training process of the VGG-16 network using swCaffe, compared to the unoptimized algorithm and framework. Compared to the cuDNN library and the Caffe framework based on the NVIDIA K40m GPU, the proposed swDNN library and swCaffe framework on SW26010 have nearly half the performance of K40m in single -precision and have 3.6\u00d7 and 1.8\u00d7 speedup over K40m in double precision, respectively.<\/jats:p>","DOI":"10.1145\/3177885","type":"journal-article","created":{"date-parts":[[2018,3,23]],"date-time":"2018-03-23T12:29:49Z","timestamp":1521808189000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Optimizing Convolutional Neural Networks on the Sunway TaihuLight Supercomputer"],"prefix":"10.1145","volume":"15","author":[{"given":"Wenlai","family":"Zhao","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haohuan","family":"Fu","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiarui","family":"Fang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weijie","family":"Zheng","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Gan","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guangwen","family":"Yang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,3,22]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Mart\u00edn Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado etal 2016. TensorFlow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.044670  Mart\u00edn Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado et al. 2016. TensorFlow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.044670"},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the 10th International Workshop on Frontiers in Handwriting Recognition.","author":"Chellapilla Kumar","year":"2006","unstructured":"Kumar Chellapilla , Sidd Puri , and Patrice Simard . 2006 . High performance convolutional neural networks for document processing . In Proceedings of the 10th International Workshop on Frontiers in Handwriting Recognition. Kumar Chellapilla, Sidd Puri, and Patrice Simard. 2006. High performance convolutional neural networks for document processing. In Proceedings of the 10th International Workshop on Frontiers in Handwriting Recognition."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2644865.2541967"},{"key":"e_1_2_1_4_1","unstructured":"Tianqi Chen Mu Li Yutian Li Min Lin Naiyan Wang Minjie Wang Tianjun Xiao Bing Xu Chiyuan Zhang and Zheng Zhang. 2015. MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv:1512.01274.  Tianqi Chen Mu Li Yutian Li Min Lin Naiyan Wang Minjie Wang Tianjun Xiao Bing Xu Chiyuan Zhang and Zheng Zhang. 2015. MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv:1512.01274."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.2014.58"},{"key":"e_1_2_1_6_1","unstructured":"Sharan Chetlur Cliff Woolley Philippe Vandermersch Jonathan Cohen John Tran Bryan Catanzaro and Evan Shelhamer. 2014. cuDNN: Efficient primitives for deep learning. arXiv:1410.0759.  Sharan Chetlur Cliff Woolley Philippe Vandermersch Jonathan Cohen John Tran Bryan Catanzaro and Evan Shelhamer. 2014. cuDNN: Efficient primitives for deep learning. arXiv:1410.0759."},{"key":"e_1_2_1_7_1","volume-title":"Torch: A Modular Machine Learning Software Library. Idiap.","author":"Collobert Ronan","year":"2002","unstructured":"Ronan Collobert , Samy Bengio , and Johnny Marithoz . 2002 . Torch: A Modular Machine Learning Software Library. Idiap. Ronan Collobert, Samy Bengio, and Johnny Marithoz. 2002. Torch: A Modular Machine Learning Software Library. Idiap."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASL.2011.2134090"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2749469.2750389"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2017.20"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11432-016-5588-7"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2012.2205597"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654889"},{"key":"e_1_2_1_14_1","volume-title":"Hinton","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E . Hinton . 2012 . ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems . 1097--1105. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097--1105."},{"key":"e_1_2_1_15_1","unstructured":"Andrew Lavin. 2015. maxDNN: An efficient convolution kernel for deep learning with maxwell GPUs. arXiv:1501.06633.  Andrew Lavin. 2015. maxDNN: An efficient convolution kernel for deep learning with maxwell GPUs. arXiv:1501.06633."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.435"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786763.2694358"},{"key":"e_1_2_1_18_1","unstructured":"Michael Mathieu Mikael Henaff and Yann LeCun. 2013. Fast training of convolutional networks through FFTs. arXiv:1312.5851.  Michael Mathieu Mikael Henaff and Yann LeCun. 2013. Fast training of convolutional networks through FFTs. arXiv:1312.5851."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2847263.2847265"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature16961"},{"key":"e_1_2_1_21_1","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556.  Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556."},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2892--2900","author":"Sun Yi","year":"2014","unstructured":"Yi Sun , Xiaogang Wang , and Xiaoou Tang . 2014 . Deeply learned face representations are sparse, selective, and robust . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2892--2900 . Yi Sun, Xiaogang Wang, and Xiaoou Tang. 2014. Deeply learned face representations are sparse, selective, and robust. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2892--2900."},{"key":"e_1_2_1_23_1","unstructured":"Nicolas Vasilache Jeff Johnson Michael Mathieu Soumith Chintala Serkan Piantino and Yann LeCun. 2014. Fast convolutional nets with fbfft: A GPU performance evaluation. arXiv:1412.7580.  Nicolas Vasilache Jeff Johnson Michael Mathieu Soumith Chintala Serkan Piantino and Yann LeCun. 2014. Fast convolutional nets with fbfft: A GPU performance evaluation. arXiv:1412.7580."},{"key":"e_1_2_1_24_1","volume-title":"Zeiler and Rob Fergus","author":"Matthew","year":"2014","unstructured":"Matthew D. Zeiler and Rob Fergus . 2014 . Visualizing and understanding convolutional networks. In Computer Vision\u2014ECCV 2014. Lecture Notes in Computer Science, Vol. 8689 . Springer , 818--833. Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In Computer Vision\u2014ECCV 2014. Lecture Notes in Computer Science, Vol. 8689. Springer, 818--833."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2684746.2689060"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2934583.2934644"},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the IEEE International Conference on Application-Specific Systems, Architectures, and Processors. 107--114","author":"Zhao Wenlai","year":"2016","unstructured":"Wenlai Zhao , Haohuan Fu , Wayne Luk , Teng Yu , Shaojun Wang , Bo Feng , Yuchun Ma , and Guangwen Yang . 2016 . F-CNN: An FPGA-based framework for training convolutional neural networks . In Proceedings of the IEEE International Conference on Application-Specific Systems, Architectures, and Processors. 107--114 . Wenlai Zhao, Haohuan Fu, Wayne Luk, Teng Yu, Shaojun Wang, Bo Feng, Yuchun Ma, and Guangwen Yang. 2016. F-CNN: An FPGA-based framework for training convolutional neural networks. In Proceedings of the IEEE International Conference on Application-Specific Systems, Architectures, and Processors. 107--114."}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3177885","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3177885","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T03:02:55Z","timestamp":1750215775000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3177885"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,3,22]]},"references-count":27,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2018,3,31]]}},"alternative-id":["10.1145\/3177885"],"URL":"https:\/\/doi.org\/10.1145\/3177885","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,3,22]]},"assertion":[{"value":"2017-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-03-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}