{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,14]],"date-time":"2025-05-14T02:28:07Z","timestamp":1747189687190,"version":"3.40.5"},"reference-count":30,"publisher":"World Scientific Pub Co Pte Ltd","issue":"02","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Parallel Process. Lett."],"published-print":{"date-parts":[[2021,6]]},"abstract":"<jats:p> Recent advances in artificial intelligence has shown a direct correlation between the performance of a network and the number of hidden layers within the network. The Compute Unified Device Architecture (CUDA) framework facilitates the movement of heavy computation from the CPU to the graphics processing unit (GPU) and is used to accelerate the training of neural networks. In this paper, we consider the problem of data-parallel neural network training. We compare the performance of training the same neural network on the GPU with and without data parallelism. When data parallelism is used, we compare with both the conventional averaging of coefficients and our proposed method. We set out to show that not all sub-networks are equal and thus, should not be treated as equals when normalising weight vectors. The proposed method achieved state of the art accuracy faster than conventional training along with better classification performance in some cases. <\/jats:p>","DOI":"10.1142\/s0129626421500092","type":"journal-article","created":{"date-parts":[[2021,5,6]],"date-time":"2021-05-06T03:53:24Z","timestamp":1620273204000},"page":"2150009","source":"Crossref","is-referenced-by-count":0,"title":["Accelerating Data-Parallel Neural Network Training with Weighted-Averaging Reparameterisation"],"prefix":"10.1142","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0521-2152","authenticated-orcid":false,"given":"Sterling","family":"Ramroach","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, The University of the West Indies, Saint Augustine, Trinidad and Tobago"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ajay","family":"Joshi","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, The University of the West Indies, Saint Augustine, Trinidad and Tobago"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"219","published-online":{"date-parts":[[2021,5,6]]},"reference":[{"key":"S0129626421500092BIB001","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2019.03.022"},{"key":"S0129626421500092BIB002","doi-asserted-by":"publisher","DOI":"10.1145\/3325917.3325934"},{"key":"S0129626421500092BIB003","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2018.12.029"},{"key":"S0129626421500092BIB004","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP40776.2020.9053569"},{"key":"S0129626421500092BIB005","doi-asserted-by":"publisher","DOI":"10.1109\/TCBB.2018.2841396"},{"key":"S0129626421500092BIB006","doi-asserted-by":"publisher","DOI":"10.1109\/SSPS.2017.8071623"},{"journal-title":"IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems","year":"2018","author":"Zhang C.","key":"S0129626421500092BIB007"},{"key":"S0129626421500092BIB008","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2018.09.038"},{"key":"S0129626421500092BIB009","doi-asserted-by":"publisher","DOI":"10.1007\/s11004-019-09835-3"},{"issue":"18","key":"S0129626421500092BIB010","first-page":"8","volume":"120","author":"Nvidia C.","year":"2011","journal-title":"NVIDIA Corporation"},{"volume-title":"Thirtieth AAAI Conference on Artificial Intelligence","author":"Ballester P.","key":"S0129626421500092BIB011"},{"key":"S0129626421500092BIB012","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"S0129626421500092BIB013","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-72344-0_13"},{"key":"S0129626421500092BIB014","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2018.00105"},{"key":"S0129626421500092BIB015","doi-asserted-by":"publisher","DOI":"10.1109\/GlobalSIP.2018.8646456"},{"key":"S0129626421500092BIB017","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2019.10.004"},{"key":"S0129626421500092BIB018","doi-asserted-by":"publisher","DOI":"10.1016\/j.bdr.2017.01.005"},{"key":"S0129626421500092BIB019","first-page":"1321","volume-title":"Proceedings of the 34th International Conference on Machine Learning","volume":"70","author":"Guo C."},{"first-page":"901","volume-title":"Advances in Neural Information Processing Systems","author":"Salimans T.","key":"S0129626421500092BIB020"},{"key":"S0129626421500092BIB021","doi-asserted-by":"publisher","DOI":"10.1109\/NEMO.2015.7415056"},{"volume-title":"Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems","year":"2019","author":"G\u00e9ron A.","key":"S0129626421500092BIB022"},{"first-page":"675","volume-title":"Proceedings of the 22nd ACM International Conference on Multimedia","author":"Jia Y.","key":"S0129626421500092BIB023"},{"first-page":"265","volume-title":"12th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u00a016)","author":"Abadi M.","key":"S0129626421500092BIB025"},{"key":"S0129626421500092BIB026","doi-asserted-by":"publisher","DOI":"10.1109\/MLHPC.2016.004"},{"key":"S0129626421500092BIB027","doi-asserted-by":"publisher","DOI":"10.1007\/11503415_37"},{"key":"S0129626421500092BIB028","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2015.7280578"},{"key":"S0129626421500092BIB029","first-page":"1438","volume-title":"Advances in Neural Information Processing Systems","author":"Huh D.","year":"2018"},{"key":"S0129626421500092BIB030","doi-asserted-by":"publisher","DOI":"10.1109\/ICNN.1993.298623"},{"key":"S0129626421500092BIB031","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2971969"},{"volume-title":"Deep Learning with Keras","year":"2017","author":"Gulli A.","key":"S0129626421500092BIB032"}],"container-title":["Parallel Processing Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0129626421500092","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,18]],"date-time":"2021-06-18T05:26:37Z","timestamp":1623993997000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/abs\/10.1142\/S0129626421500092"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,6]]},"references-count":30,"journal-issue":{"issue":"02","published-print":{"date-parts":[[2021,6]]}},"alternative-id":["10.1142\/S0129626421500092"],"URL":"https:\/\/doi.org\/10.1142\/s0129626421500092","relation":{},"ISSN":["0129-6264","1793-642X"],"issn-type":[{"type":"print","value":"0129-6264"},{"type":"electronic","value":"1793-642X"}],"subject":[],"published":{"date-parts":[[2021,5,6]]}}}