{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T16:41:34Z","timestamp":1771260094385,"version":"3.50.1"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"23","license":[{"start":{"date-parts":[[2020,9,9]],"date-time":"2020-09-09T00:00:00Z","timestamp":1599609600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,9,9]],"date-time":"2020-09-09T00:00:00Z","timestamp":1599609600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010025","name":"University of Derby","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100010025","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Soft Comput"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The development of deep learning has led to a dramatic increase in the number of applications of artificial intelligence. However, the training of deeper neural networks for stable and accurate models translates into artificial neural networks (ANNs) that become unmanageable as the number of features increases. This work extends our earlier study where we explored the acceleration effects obtained by enforcing, in turn, scale freeness, small worldness, and sparsity during the ANN training process. The efficiency of that approach was confirmed by recent studies (conducted independently) where a million-node ANN was trained on non-specialized laptops. Encouraged by those results, our study is now focused on some tunable parameters, to pursue a further acceleration effect. We show that, although optimal parameter tuning is unfeasible, due to the high non-linearity of ANN problems, we can actually come up with a set of useful guidelines that lead to speed-ups in practical cases. We find that significant reductions in execution time can generally be achieved by setting the revised fraction parameter (<jats:inline-formula><jats:alternatives><jats:tex-math>$$\\zeta $$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><mml:mi>\u03b6<\/mml:mi><\/mml:math><\/jats:alternatives><\/jats:inline-formula>) to relatively low values.<\/jats:p>","DOI":"10.1007\/s00500-020-05302-y","type":"journal-article","created":{"date-parts":[[2020,9,10]],"date-time":"2020-09-10T11:52:52Z","timestamp":1599738772000},"page":"17787-17795","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Artificial neural networks training acceleration through network science strategies"],"prefix":"10.1007","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2367-6084","authenticated-orcid":false,"given":"Lucia","family":"Cavallaro","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4193-9842","authenticated-orcid":false,"given":"Ovidiu","family":"Bagdasar","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7421-216X","authenticated-orcid":false,"given":"Pasquale","family":"De Meo","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1528-7203","authenticated-orcid":false,"given":"Giacomo","family":"Fiumara","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2773-4421","authenticated-orcid":false,"given":"Antonio","family":"Liotta","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,9,9]]},"reference":[{"key":"5302_CR1","volume-title":"Network science","author":"A-L Barab\u00e1si","year":"2016","unstructured":"Barab\u00e1si A-L, P\u00f3sfai M (2016) Network science. Cambridge University Press, Cambridge UK"},{"key":"5302_CR2","unstructured":"Bellec\u00a0G, Kappel\u00a0D, Maass\u00a0W, Legenstein\u00a0R (2018) Deep rewiring: training very sparse deep networks. arXiv preprint arXiv:1711.05136"},{"key":"5302_CR3","doi-asserted-by":"publisher","first-page":"122","DOI":"10.3390\/info10040122","volume":"4","author":"DS Berman","year":"2019","unstructured":"Berman DS, Buczak AL, Chavis JS, Corbett CL (2019) A survey of deep learning methods for cyber security. Information 4:122. https:\/\/doi.org\/10.3390\/info10040122","journal-title":"Information"},{"key":"5302_CR4","unstructured":"Bourely\u00a0A, Boueri\u00a0JP, Choromonski\u00a0K (2017) Sparse neural networks topologies. arXiv preprint arXiv:1706.05683"},{"issue":"8","key":"5302_CR5","doi-asserted-by":"publisher","first-page":"1548","DOI":"10.1109\/TPAMI.2010.231","volume":"33","author":"D Cai","year":"2011","unstructured":"Cai D, He X, Han J, Huang TS (2011) Graph regularized non-negative matrix factorization for data representation. PAMI 33(8):1548\u20131560","journal-title":"PAMI"},{"key":"5302_CR6","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1007\/s00778-010-0189-3","volume":"20","author":"D Cai","year":"2011","unstructured":"Cai D, He X, Han J (2011) Speed up kernel discriminant analysis. VLDB J 20:21\u201333. https:\/\/doi.org\/10.1007\/s00778-010-0189-3","journal-title":"VLDB J"},{"issue":"1","key":"5302_CR7","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1016\/j.gpb.2017.07.003","volume":"16","author":"C Cao","year":"2018","unstructured":"Cao C, Liu F, Tan H, Song D, Shu W, Li W, Zhou Y, Bo X, Xie Z (2018) Deep learning and its applications in biomedicine. Genom Proteomics Bioinform 16(1):17\u201332. https:\/\/doi.org\/10.1016\/j.gpb.2017.07.003","journal-title":"Genom Proteomics Bioinform"},{"key":"5302_CR8","doi-asserted-by":"publisher","unstructured":"Cavallaro\u00a0L, Bagdasar\u00a0O, De\u00a0Meo\u00a0P, Fiumara\u00a0G, Liotta\u00a0A (2020) Artificial neural networks training acceleration through network science strategies. In: Sergeyev YD, Kvasov DE (eds) Numerical computations: theory and algorithms, NUMTA 2019. Lecture Notes in Computer Science, Springer, Cham 11974:330\u2013336. https:\/\/doi.org\/10.1007\/978-3-030-40616-5_27","DOI":"10.1007\/978-3-030-40616-5_27"},{"issue":"6","key":"5302_CR9","doi-asserted-by":"publisher","first-page":"1241","DOI":"10.1016\/j.drudis.2018.01.039","volume":"23","author":"H Chen","year":"2018","unstructured":"Chen H, Engkvist O, Wang Y, Olivecrona M, Blaschke T (2018) The rise of deep learning in drug discovery. Drug Discov Today 23(6):1241\u20131250. https:\/\/doi.org\/10.1016\/j.drudis.2018.01.039","journal-title":"Drug Discov Today"},{"key":"5302_CR10","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1109\/MSP.2010.939038","volume":"1","author":"Y Dong","year":"2011","unstructured":"Dong Y, Li D (2011) Deep learning and its applications to signal and information processing [exploratory DSP]. IEEE Signal Process Mag 1:145. https:\/\/doi.org\/10.1109\/MSP.2010.939038","journal-title":"IEEE Signal Process Mag"},{"key":"5302_CR11","doi-asserted-by":"crossref","unstructured":"Erd\u0151s P, R\u00e9nyi A (1959) On random graphs i. Publ Math-Debr 6:290\u2013297","DOI":"10.5486\/PMD.1959.6.3-4.12"},{"key":"5302_CR12","unstructured":"Frankle\u00a0J, Carbin\u00a0M (2018) The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. arXiv preprint arXiv:1803.03635"},{"key":"5302_CR13","unstructured":"Gale\u00a0T, Elsen\u00a0E, Hooker\u00a0S (2019) the state of sparsity in deep neural networks. arXiv preprint arXiv:1902.09574"},{"key":"5302_CR14","volume-title":"Deep learning","author":"I Goodfellow","year":"2016","unstructured":"Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge US"},{"issue":"19","key":"5302_CR15","doi-asserted-by":"publisher","first-page":"3937","DOI":"10.1200\/JCO.2004.12.133","volume":"22","author":"C Haslinger","year":"2004","unstructured":"Haslinger C, Schweifer N, Stilgenbauer S, D\u00f6hner H, Lichter P, Kraut N, Stratowa C, Abseher R (2004) Microarray gene expression profiling of B-cell chronic lymphocytic leukemia subgroups defined by genomic aberrations and VH mutation status. J Clin Oncol 22(19):3937\u201349. https:\/\/doi.org\/10.1200\/JCO.2004.12.133","journal-title":"J Clin Oncol"},{"key":"5302_CR16","unstructured":"Hestness\u00a0J, Narang\u00a0S, Ardalani\u00a0N, Diamos\u00a0GF, Jun\u00a0H, Kianinejad\u00a0H, Patwary\u00a0MMA, Yang\u00a0Y, Zhou\u00a0Y (2017) Deep learning scaling is predictable, empirically. arXiv preprint arXiv:1712.00409"},{"key":"5302_CR17","doi-asserted-by":"crossref","unstructured":"He\u00a0K, Zhang\u00a0X, Ren\u00a0S, Sun\u00a0J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, CVPR 2016, Las Vegas USA, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"issue":"4","key":"5302_CR18","doi-asserted-by":"publisher","first-page":"2361","DOI":"10.1007\/s00429-015-1035-6","volume":"221","author":"CC Hilgetag","year":"2016","unstructured":"Hilgetag CC, Goulas A (2016) Is the brain really a small-world network? Brain Struct Funct 221(4):2361\u20132366","journal-title":"Brain Struct Funct"},{"key":"5302_CR19","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","volume":"29","author":"G Hinton","year":"2012","unstructured":"Hinton G, Deng L, Yu D, Dahl GE, Mohamed A, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN, Kingsbury B (2012) Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process Mag 29:82\u201397","journal-title":"IEEE Signal Process Mag"},{"key":"5302_CR20","unstructured":"Kalchbrenner\u00a0N, Elsen\u00a0E, Simonyan\u00a0K, Noury\u00a0S, Casagrande\u00a0N, Lockhart\u00a0E, Stimberg\u00a0F, van den Oord\u00a0A, Dieleman\u00a0S, Kavukcuoglu\u00a0K (2018) Efficient neural audio synthesis. In: Proceedings of the international conference on machine learning, ICML 2018, Stockholm, pp 2415\u20132424"},{"issue":"6","key":"5302_CR21","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1145\/3065386","volume":"60","author":"A Krizhevsky","year":"2017","unstructured":"Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60(6):84\u201390. https:\/\/doi.org\/10.1145\/3065386","journal-title":"Commun ACM"},{"key":"5302_CR22","doi-asserted-by":"publisher","DOI":"10.1017\/9781316216002","volume-title":"Complex networks: principles, methods and applications","author":"V Latora","year":"2017","unstructured":"Latora V, Nicosia V, Russo G (2017) Complex networks: principles, methods and applications. Cambridge University Press, Cambridge UK"},{"key":"5302_CR23","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"Y LeCun","year":"2015","unstructured":"LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436\u2013444. https:\/\/doi.org\/10.1038\/nature14539","journal-title":"Nature"},{"key":"5302_CR24","doi-asserted-by":"crossref","unstructured":"Liu\u00a0S, Mocanu\u00a0DC, Matavalam\u00a0ARR, Pei\u00a0Y, Pechenizkiy\u00a0M (2019) Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware. ArXiv, arXiv:1901.09181","DOI":"10.1007\/s00521-020-05136-7"},{"key":"5302_CR25","unstructured":"Louizos\u00a0C, Welling\u00a0M, Kingma\u00a0DP (2017) Learning sparse neural networks through $$L_0$$ Regularization. arXiv preprint arXiv:1712.01312"},{"key":"5302_CR26","doi-asserted-by":"publisher","first-page":"2383","DOI":"10.1038\/s41467-018-04316-3","volume":"9","author":"DC Mocanu","year":"2018","unstructured":"Mocanu DC, Mocanu E, Stone P, Nguyen PH, Gibescu M, Liotta A (2018) Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science. Nat Commun 9:2383. https:\/\/doi.org\/10.1038\/s41467-018-04316-3","journal-title":"Nat Commun"},{"key":"5302_CR27","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1016\/j.eswa.2018.12.032","volume":"121","author":"D Ruano-Ord\u00e1s","year":"2019","unstructured":"Ruano-Ord\u00e1s D, Yevseyeva I, Fernandes VB, M\u00e9ndez JR, Emmerich MTM (2019) Improving the drug discovery process by using multiple classifier systems. Expert Syst Appl 121:292\u2013303. https:\/\/doi.org\/10.1016\/j.eswa.2018.12.032","journal-title":"Expert Syst Appl"},{"key":"5302_CR28","doi-asserted-by":"crossref","unstructured":"Srinivas\u00a0S, Subramanya\u00a0A, Babu\u00a0RV (2017) Training sparse neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, Honolulu, pp 455\u2013462. https:\/\/doi.org\/10.1109\/CVPRW.2017.61","DOI":"10.1109\/CVPRW.2017.61"},{"key":"5302_CR29","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/j.procs.2019.09.165","volume":"159","author":"J Stier","year":"2019","unstructured":"Stier J, Granitzer M (2019) Structural analysis of sparse neural networks. Procedia Comput Sci 159:107\u2013116","journal-title":"Procedia Comput Sci"},{"key":"5302_CR30","unstructured":"Ullrich\u00a0K, Meeds\u00a0E, Welling\u00a0M (2017) Soft weight-sharing for neural network compression. arXiv preprint arXiv:1702.04008"},{"key":"5302_CR31","unstructured":"Vaswani\u00a0A, Shazeer\u00a0N, Parmar\u00a0N, Uszkoreit\u00a0J, Jones\u00a0L, Gomez\u00a0AN, Kaiser\u00a0L, Polosukhin\u00a0I (2017) Attention is all you need. In: Proceedings of the annual conference on neural information processing systems, Long Beach, USA, pp 6000\u20136010"},{"key":"5302_CR32","doi-asserted-by":"publisher","first-page":"440","DOI":"10.1038\/30918","volume":"393","author":"DJ Watts","year":"1998","unstructured":"Watts DJ, Strogatz SH (1998) Collective dynamics of \u2018small-world\u2019 networks. Nature 393:440\u2013442","journal-title":"Nature"}],"container-title":["Soft Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00500-020-05302-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00500-020-05302-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00500-020-05302-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,7]],"date-time":"2023-10-07T04:19:58Z","timestamp":1696652398000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00500-020-05302-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,9]]},"references-count":32,"journal-issue":{"issue":"23","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["5302"],"URL":"https:\/\/doi.org\/10.1007\/s00500-020-05302-y","relation":{},"ISSN":["1432-7643","1433-7479"],"issn-type":[{"value":"1432-7643","type":"print"},{"value":"1433-7479","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,9]]},"assertion":[{"value":"9 September 2020","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"All authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This article does not contain any studies with human participants or animals performed by any of the authors.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Human and animal rights"}}]}}