{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T10:59:42Z","timestamp":1777546782292,"version":"3.51.4"},"reference-count":41,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2018,4,30]],"date-time":"2018-04-30T00:00:00Z","timestamp":1525046400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Spanish Agency of Research","award":["FPA2016-78595-C3-3-R"],"award-info":[{"award-number":["FPA2016-78595-C3-3-R"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In the optimization of deep neural networks (DNNs) via evolutionary algorithms (EAs) and the implementation of the training necessary for the creation of the objective function, there is often a trade-off between efficiency and flexibility. Pure software solutions implemented on general-purpose processors tend to be slow because they do not take advantage of the inherent parallelism of these devices, whereas hardware realizations based on heterogeneous platforms (combining central processing units (CPUs), graphics processing units (GPUs) and\/or field-programmable gate arrays (FPGAs)) are designed based on different solutions using methodologies supported by different languages and using very different implementation criteria. This paper first presents a study that demonstrates the need for a heterogeneous (CPU-GPU-FPGA) platform to accelerate the optimization of artificial neural networks (ANNs) using genetic algorithms. Second, the paper presents implementations of the calculations related to the individuals evaluated in such an algorithm on different (CPU- and FPGA-based) platforms, but with the same source files written in OpenCL. The implementation of individuals on remote, low-cost FPGA systems on a chip (SoCs) is found to enable the achievement of good efficiency in terms of performance per watt.<\/jats:p>","DOI":"10.3390\/s18051384","type":"journal-article","created":{"date-parts":[[2018,4,30]],"date-time":"2018-04-30T11:09:46Z","timestamp":1525086586000},"page":"1384","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Optimization of Deep Neural Networks Using SoCs with OpenCL"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2857-8667","authenticated-orcid":false,"given":"Rafael","family":"Gadea-Giron\u00e9s","sequence":"first","affiliation":[{"name":"Department of Electronic Engineering, Universitat Polit\u00e8cnica de Val\u00e8ncia, Camino de Vera, s\/n, 46022 Val\u00e8ncia, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ricardo","family":"Colom-Palero","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, Universitat Polit\u00e8cnica de Val\u00e8ncia, Camino de Vera, s\/n, 46022 Val\u00e8ncia, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vicente","family":"Herrero-Bosch","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, Universitat Polit\u00e8cnica de Val\u00e8ncia, Camino de Vera, s\/n, 46022 Val\u00e8ncia, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,4,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/j.engappai.2009.12.001","article-title":"Parallel scalable hardware implementation of asynchronous discrete particle swarm optimization","volume":"23","author":"Vakili","year":"2010","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1002\/cem.1401","article-title":"Neural networks applied in chemistry. I. Determination of the optimal topology of multilayer perceptron neural networks","volume":"25","author":"Curteanu","year":"2011","journal-title":"J. Chem."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1109\/TSMCB.2008.2008724","article-title":"A New Adaptive Merging and Growing Algorithm for Designing Artificial Neural Networks","volume":"39","author":"Islam","year":"2009","journal-title":"IEEE Trans. Syst. Man Cybern. Part B Cybern."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1109\/TEVC.2004.823467","article-title":"Quantum-inspired evolutionary algorithms with a new termination criterion, H-epsilon gate, and two-phase scheme","volume":"8","author":"Han","year":"2004","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1109\/TNN.2002.804317","article-title":"Tuning of the structure and parameters of a neural network using an improved genetic algorithm","volume":"14","author":"Leung","year":"2003","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1109\/TNN.2005.860885","article-title":"Tuning the structure and parameters of a neural network by using hybrid Taguchi-genetic algorithm","volume":"17","author":"Tsai","year":"2006","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1452","DOI":"10.1109\/TNN.2006.881047","article-title":"An optimization methodology for neural network weights and architectures","volume":"17","author":"Ludermir","year":"2006","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1109\/TNN.2005.844858","article-title":"Mutation-based Genetic Neural Network","volume":"16","author":"Palmes","year":"2005","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_9","unstructured":"Huang, D.-S., Wunsch, D.C., Levine, D.S., and Jo, K.-H. (2008). A Hybrid Particle Swarm Optimization for Feed-Forward Neural Network Training. Advanced Intelligent Computing Theories and Applications: With Aspects of Artificial Intelligence, Springer."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1266","DOI":"10.1109\/TNNLS.2013.2249089","article-title":"Quantum-Based Algorithm for Optimizing Artificial Neural Networks","volume":"24","author":"Lu","year":"2013","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"369298","DOI":"10.1155\/2015\/369298","article-title":"Designing Artificial Neural Networks Using Particle Swarm Optimization Algorithms","volume":"2015","author":"Garro","year":"2015","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Young, S.R., Rose, D.C., Karnowski, T.P., Lim, S.-H., and Patton, R.M. (2015, January 15). Optimizing Deep Learning Hyper-parameters Through an Evolutionary Algorithm. Proceedings of the MLHPC \u201915 Workshop on Machine Learning in High-Performance Computing Environments, Austin, TX, USA.","DOI":"10.1145\/2834892.2834896"},{"key":"ref_13","unstructured":"Miikkulainen, R., Liang, J.-Z., Meyerson, E., Rawal, A., Fink, D., Francon, O., Raju, B., Shahrzad, H., Navruzyan, A., and Duffy, N. (arXiv, 2017). Evolving Deep Neural Networks, arXiv."},{"key":"ref_14","unstructured":"Real, E., Moore, S., Selle, A., Saxena, S., Suematsu, Y.L., Le, Q.V., and Kurakin, A. (arXiv, 2017). Large-Scale Evolution of Image Classifiers, arXiv."},{"key":"ref_15","unstructured":"Baker, B., Gupta, O., Naik, N., and Raskar, R. (arXiv, 2016). Designing Neural Network Architectures Using Reinforcement Learning, arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1423","DOI":"10.1109\/5.784219","article-title":"Evolving artificial neural networks","volume":"87","author":"Yao","year":"1999","journal-title":"Proc. IEEE"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"694","DOI":"10.1109\/72.572107","article-title":"A new evolutionary system for evolving artificial neural networks","volume":"8","author":"Yao","year":"1997","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2017","DOI":"10.1016\/j.neucom.2009.11.032","article-title":"Approximate k-NN delta test minimization method using genetic algorithms: Application to time series","volume":"73","author":"Mateo","year":"2010","journal-title":"Neurocomputing"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Hawkins, S., He, H., Williams, G., and Baxter, R. (2002, January 2). Outlier Detection Using Replicator Neural Networks. Proceedings of the Fifth International Conference on Data Warehousing and Knowledge Discovery (DaWaK02), Toulouse, France.","DOI":"10.1007\/3-540-46145-0_17"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2944","DOI":"10.1007\/s11227-015-1419-7","article-title":"Gadea-Giron\u00e9s, R. Evolutionary Optimization of Neural Networks with Heterogeneous Computation: Study and Implementation","volume":"71","author":"Fe","year":"2015","journal-title":"J. Supercomput."},{"key":"ref_21","unstructured":"Prechelt, L. (1994). PROBEN1\u2014A Set of Neural Network Benchmark Problems and Benchmarking Rules, The University of Karlsruhe. Technical Report."},{"key":"ref_22","unstructured":"Kingma, D.P., and Jimmy, B. (arXiv, 2014). Adam: A Method for Stochastic Optimization, arXiv."},{"key":"ref_23","unstructured":"Zeiler, M.D. (arXiv, 2012). ADADELTA: An Adaptive Learning Rate Method, arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1162\/neco.1992.4.3.415","article-title":"Bayesian Interpolation","volume":"4","author":"MacKay","year":"1991","journal-title":"Neural Comput."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/S0933-3657(02)00028-3","article-title":"An Evolutionary Artificial Neural Networks Approach for Breast Cancer Diagnosis","volume":"25","author":"Abbass","year":"2002","journal-title":"Artif. Intell. Med."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1427","DOI":"10.1007\/s00521-012-1092-1","article-title":"A genetic algorithm-based multi-objective optimization of an artificial neural network classifier for breast cancer diagnosis","volume":"23","author":"Ahmad","year":"2013","journal-title":"Neural Comput. Appl."},{"key":"ref_27","first-page":"1","article-title":"Article: A Hybrid Differential Evolution and Back-Propagation Algorithm for Feedforward Neural Network Training","volume":"84","author":"Sarangi","year":"2013","journal-title":"Int. J. Comput. Appl."},{"key":"ref_28","first-page":"153","article-title":"Greedy layer-wise training of deep networks","volume":"19","author":"Bengio","year":"2007","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Sankaradas, M., Jakkula, V., Cadambi, S., Chakradhar, S., Durdanovic, I., Cosatto, E., and Graf, H.P. (2009, January 7\u20139). A Massively Parallel Coprocessor for Convolutional Neural Networks. Proceedings of the ASAP 2009 20th IEEE International Conference on Application-Specific Systems, Architectures and Processors, Boston, MA, USA.","DOI":"10.1109\/ASAP.2009.25"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Prado, R.N.A., Melo, J.D., Oliveira, J.A.N., and Neto, A.D.D. (2012, January 10\u201315). FPGA based implementation of a Fuzzy Neural Network modular architecture for embedded systems. Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, Australia.","DOI":"10.1109\/IJCNN.2012.6252447"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1007\/s00521-010-0423-3","article-title":"Neural network training based on FPGA with floating point number format and it\u2019s performance","volume":"20","author":"Karakuzu","year":"2011","journal-title":"Neural Comput. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wu, G.-D., Zhu, Z.-W., and Lin, B.-W. (2011, January 12\u201314). Reconfigurable back propagation based neural network architecture. Proceedings of the 2011 13th International Symposium on Integrated Circuits (ISIC), Singapore.","DOI":"10.1109\/ISICir.2011.6131881"},{"key":"ref_33","first-page":"1","article-title":"Implementation of Neural Network Back Propagation Training Algorithm on FPGA","volume":"52","author":"Pinjare","year":"2012","journal-title":"Int. J. Comput. Appl."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., and Cong, J. (2015, January 22\u201324). Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks. Proceedings of the FPGA \u201915 2015 ACM\/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA.","DOI":"10.1145\/2684746.2689060"},{"key":"ref_35","first-page":"513","article-title":"DLAU: A Scalable Deep Learning Accelerator Unit on FPGA","volume":"36","author":"Wang","year":"2017","journal-title":"IEEE Trans. Comput. Aided Des. Integr. Circuits Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhao, R., Song, W., Zhang, W., Xing, T., Lin, J.-H., Srivastava, M., Gupta, R., and Zhang, Z. (2017, January 22\u201324). Accelerating Binarized Convolutional Neural Networks with Software-Programmable FPGAs. Proceedings of the FPGA \u201917 2017 ACM\/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA.","DOI":"10.1145\/3020078.3021741"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Bettoni, M., Urgese, G., Kobayashi, Y., Macii, E., and Acquaviva, A. (2017). A Convolutional Neural Network Fully Implemented on FPGA for Embedded Platforms. 2017 New Generation of CAS (NGCAS), IEEE.","DOI":"10.1109\/NGCAS.2017.16"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Shi, X., An, H., Wang, C., Kandemir, M., and Jin, H. (2017). Optimizing OpenCL Implementation of Deep Convolutional Neural Network on FPGA. Network and Parallel Computing, Springer International Publishing.","DOI":"10.1007\/978-3-319-68210-5"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Suda, N., Chandra, V., Dasika, G., Mohanty, A., Ma, Y., Vrudhula, S., Seo, J.-S., and Cao, Y. (2016, January 21\u201323). Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Networks. Proceedings of the FPGA \u201916 2016 ACM\/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA.","DOI":"10.1145\/2847263.2847276"},{"key":"ref_40","unstructured":"Zhu, M., Liu, L., Wang, C., and Xie, Y. (arXiv, 2016). CNNLab: A Novel Parallel Framework for Neural Networks using GPU and FPGA-a Practical Study with Trade-off Analysis, arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Gadea-Giron, R., Herrero, V., Sebastia, A., and Salcedo, A.-M. (2000, January 27\u201330). The Role of the Embedded Memories in the Implementation of Artificial Neural Networks. Proceedings of the FPL \u201900 The Roadmap to Reconfigurable Computing, 10th International Workshop on Field-Programmable Logic and Applications, Villach, Austria.","DOI":"10.1007\/3-540-44614-1_85"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/5\/1384\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:02:43Z","timestamp":1760194963000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/5\/1384"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,4,30]]},"references-count":41,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2018,5]]}},"alternative-id":["s18051384"],"URL":"https:\/\/doi.org\/10.3390\/s18051384","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,4,30]]}}}