{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,13]],"date-time":"2024-09-13T11:10:10Z","timestamp":1726225810336},"reference-count":17,"publisher":"Institute of Electronics, Information and Communications Engineers (IEICE)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IEICE Trans. Inf. & Syst."],"published-print":{"date-parts":[[2021,12,1]]},"DOI":"10.1587\/transinf.2021pap0008","type":"journal-article","created":{"date-parts":[[2021,11,30]],"date-time":"2021-11-30T22:44:47Z","timestamp":1638312287000},"page":"2057-2067","source":"Crossref","is-referenced-by-count":0,"title":["An FPGA-Based Optimizer Design for Distributed Deep Learning with Multiple GPUs"],"prefix":"10.1587","volume":"E104.D","author":[{"given":"Tomoya","family":"ITSUBO","sequence":"first","affiliation":[{"name":"Graduate School of Science and Technology, Keio University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michihiro","family":"KOIBUCHI","sequence":"additional","affiliation":[{"name":"National Institute of Informatics"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hideharu","family":"AMANO","sequence":"additional","affiliation":[{"name":"Graduate School of Science and Technology, Keio University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hiroki","family":"MATSUTANI","sequence":"additional","affiliation":[{"name":"Graduate School of Science and Technology, Keio University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"532","reference":[{"key":"1","unstructured":"[1] T. Akiba, S. Suzuki, and K. Fukuda, \u201cExtremely large minibatch SGD: Training ResNet-50 on ImageNet in 15 minutes,\u201d CoRR, vol.abs\/1711.04325, Nov. 2017."},{"key":"2","unstructured":"[2] H. Mikami, H. Suganuma, P. U-chupala, Y. Tanaka, and Y. Kageyama, \u201cMassively distributed SGD: ImageNet\/ResNet-50 training in a flash,\u201d CoRR, vol.abs\/1811.05233, Nov. 2018."},{"key":"3","unstructured":"[3] M. Yamazaki, A. Kasagi, A. Tabuchi, T. Honda, M. Miwa, N. Fukumoto, T. Tabaru, A. Ike, and K. Nakashima, \u201cYet another accelerated SGD: ResNet-50 training on ImageNet in 74.7 seconds,\u201d CoRR, vol.abs\/1903.12650, March 2019."},{"key":"4","unstructured":"[4] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, and X. Zheng, \u201cTensorFlow: A system for large-scale machine learning,\u201d Proc. USENIX Symposium on Operating Systems Design and Implementation (OSDI'16), pp.265-283, Nov. 2016."},{"key":"5","doi-asserted-by":"crossref","unstructured":"[5] J. Suzuki, Y. Hidaka, J. Higuchi, T. Yoshikawa, and A. Iwata, \u201cExpressEther-Ethernet-based virtualization technology for reconfigurable hardware platform,\u201d Proc. IEEE Symposium on High-Performance Interconnects (HOTI'06), pp.45-51, Aug. 2006. 10.1109\/HOTI.2006.12","DOI":"10.1109\/HOTI.2006.12"},{"key":"6","unstructured":"[6] \u201cThe NetFPGA Project.\u201d https:\/\/netfpga.org."},{"key":"7","doi-asserted-by":"crossref","unstructured":"[7] T. Itsubo, M. Koibuchi, H. Amano, and H. Matsutani, \u201cAccelerating deep learning using multiple GPUs and FPGA-based 10GbE switch,\u201d Proc. Int. Conf. Parallel, Distributed and Network-Based Processing (PDP'2020), pp.102-109, March 2020. 10.1109\/PDP50117.2020.00022","DOI":"10.1109\/PDP50117.2020.00022"},{"key":"8","unstructured":"[8] J. Duchi, E. Hazan, and Y. Singer, \u201cAdaptive subgradient methods for online learning and stochastic optimization,\u201d J. Mach. Learn. Res., vol.12, pp.2121-2159, July 2011."},{"key":"9","unstructured":"[9] D.P. Kingma and J. Ba, \u201cAdam: A method for stochastic optimization,\u201d CoRR, vol.abs\/1412.6980, Dec. 2014."},{"key":"10","unstructured":"[10] \u201cRMSprop loses to SMORMS3.\u201d https:\/\/sifter.org\/simon\/journal\/20150420.html."},{"key":"11","doi-asserted-by":"crossref","unstructured":"[11] Y. Li, J. Park, M. Alian, Y. Yuan, Z. Qu, P. Pan, R. Wang, A. Schwing, H. Esmaeilzadeh, and N.S. Kim, \u201cA network-centric hardware\/algorithm co-design to accelerate distributed training of deep neural networks,\u201d Proc. Int. Symp. Microarchitecture (MICRO'18), pp.175-188, Oct. 2018. 10.1109\/MICRO.2018.00023","DOI":"10.1109\/MICRO.2018.00023"},{"key":"12","doi-asserted-by":"publisher","unstructured":"[12] S. Morishima and H. Matsutani, \u201cHigh-performance with an In-GPU graph database cache,\u201d IEEE IT Professional, vol.19, no.6, pp.58-64, Nov. 2017. 10.1109\/MITP.2017.4241461","DOI":"10.1109\/MITP.2017.4241461"},{"key":"13","unstructured":"[13] \u201cChainer: A powerful, flexible, and intuitive framework for neural networks.\u201d https:\/\/chainer.org."},{"key":"14","unstructured":"[14] \u201cNVIDIA CUDA.\u201d https:\/\/developer.nvidia.com\/cuda-zone."},{"key":"15","doi-asserted-by":"crossref","unstructured":"[15] J. Suzuki, Y. Hayashi, M. Kan, S. Miyakawa, and T. Yoshikawa, \u201cEnd-to-end adaptive packet aggregation for high-throughput I\/O bus network using ethernet,\u201d Proc. IEEE Symposium on High-Performance Interconnects (HOTI'14), pp.17-24, Aug. 2014. 10.1109\/HOTI.2014.16","DOI":"10.1109\/HOTI.2014.16"},{"key":"16","unstructured":"[16] \u201cOSNT 10G Home.\u201d https:\/\/github.com\/NetFPGA\/OSNT-Public\/wiki\/OSNT-10G-Home."},{"key":"17","doi-asserted-by":"crossref","unstructured":"[17] G. Huang, Z. Liu, L. van der Matten, and K.Q. Weinberger, \u201cDensely connected convolutional networks,\u201d Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR'17), pp.2261-2269, July 2017. 10.1109\/CVPR.2017.243","DOI":"10.1109\/CVPR.2017.243"}],"container-title":["IEICE Transactions on Information and Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jstage.jst.go.jp\/article\/transinf\/E104.D\/12\/E104.D_2021PAP0008\/_pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,13]],"date-time":"2024-09-13T10:27:42Z","timestamp":1726223262000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jstage.jst.go.jp\/article\/transinf\/E104.D\/12\/E104.D_2021PAP0008\/_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,1]]},"references-count":17,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2021]]}},"URL":"https:\/\/doi.org\/10.1587\/transinf.2021pap0008","relation":{},"ISSN":["0916-8532","1745-1361"],"issn-type":[{"type":"print","value":"0916-8532"},{"type":"electronic","value":"1745-1361"}],"subject":[],"published":{"date-parts":[[2021,12,1]]},"article-number":"2021PAP0008"}}