{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T18:42:55Z","timestamp":1775068975173,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T00:00:00Z","timestamp":1687996800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001602","name":"Science Foundation Ireland","doi-asserted-by":"publisher","award":["16\/IA\/4605"],"award-info":[{"award-number":["16\/IA\/4605"]}],"id":[{"id":"10.13039\/501100001602","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Neural networks have revolutionised the way we approach problem solving across multiple domains; however, their effective design and efficient use of computational resources is still a challenging task. One of the most important factors influencing this process is model hyperparameters which vary significantly with models and datasets. Recently, there has been an increased focus on automatically tuning these hyperparameters to reduce complexity and to optimise resource utilisation. From traditional human-intuitive tuning methods to random search, grid search, Bayesian optimisation, and evolutionary algorithms, significant advancements have been made in this direction that promise improved performance while using fewer resources. In this article, we propose HyperGE, a two-stage model for automatically tuning hyperparameters driven by grammatical evolution (GE), a bioinspired population-based machine learning algorithm. GE provides an advantage in that it allows users to define their own grammar for generating solutions, making it ideal for defining search spaces across datasets and models. We test HyperGE to fine-tune VGG-19 and ResNet-50 pre-trained networks using three benchmark datasets. We demonstrate that the search space is significantly reduced by a factor of ~90% in Stage 2 with fewer number of trials. HyperGE could become an invaluable tool within the deep learning community, allowing practitioners greater freedom when exploring complex problem domains for hyperparameter fine-tuning.<\/jats:p>","DOI":"10.3390\/a16070319","type":"journal-article","created":{"date-parts":[[2023,6,30]],"date-time":"2023-06-30T01:02:41Z","timestamp":1688086961000},"page":"319","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Grammatical Evolution-Driven Algorithm for Efficient and Automatic Hyperparameter Optimisation of Neural Networks"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9699-522X","authenticated-orcid":false,"given":"Gauri","family":"Vaidya","sequence":"first","affiliation":[{"name":"Biocomputing and Developmental Systems Research Group, University of Limerick, V94 T9PX Limerick, Ireland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8182-2465","authenticated-orcid":false,"given":"Meghana","family":"Kshirsagar","sequence":"additional","affiliation":[{"name":"Biocomputing and Developmental Systems Research Group, University of Limerick, V94 T9PX Limerick, Ireland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7002-5815","authenticated-orcid":false,"given":"Conor","family":"Ryan","sequence":"additional","affiliation":[{"name":"Biocomputing and Developmental Systems Research Group, University of Limerick, V94 T9PX Limerick, Ireland"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kshirsagar, M., More, T., Lahoti, R., Adgaonkar, S., Jain, S., and Ryan, C. (2022, January 3\u20135). Rethinking Traffic Management with Congestion Pricing and Vehicular Routing for Sustainable and Clean Transport. Proceedings of the 14th International Conference on Agents and Artificial Intelligence\u2014Volume 3: ICAART, Online.","DOI":"10.5220\/0010830300003116"},{"key":"ref_2","unstructured":"Bahja, M. (2020). E-Business-Higher Education and Intelligence Applications, BoD\u2013Books on Demand."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"388","DOI":"10.1016\/j.ijforecast.2020.06.008","article-title":"Recurrent Neural Networks for Time Series Forecasting: Current status and future directions","volume":"37","author":"Hewamalage","year":"2021","journal-title":"Int. J. Forecast."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Xiao, Y., Wu, L., Guo, J., Li, J., Zhang, M., Qin, T., and Liu, T.Y. (2023). A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond. IEEE Trans. Pattern Anal. Mach. Intell., 1\u201320.","DOI":"10.1109\/TPAMI.2023.3277122"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"9:1","DOI":"10.1147\/JRD.2017.2709578","article-title":"An effective algorithm for hyperparameter optimisation of neural networks","volume":"61","author":"Diaz","year":"2017","journal-title":"IBM J. Res. Dev."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 26\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Bochinski, E., Senst, T., and Sikora, T. (2017, January 17\u201320). Hyper-parameter optimisation for convolutional neural network committees based on evolutionary algorithms. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.","DOI":"10.1109\/ICIP.2017.8297018"},{"key":"ref_8","first-page":"281","article-title":"Random Search for Hyper-Parameter Optimisation","volume":"13","author":"Bergstra","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_9","unstructured":"Shawe-Taylor, J., Zemel, R., Bartlett, P., Pereira, F., and Weinberger, K. (2011). Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"6278908","DOI":"10.1155\/2019\/6278908","article-title":"Effect of the Sampling of a Dataset in the Hyperparameter Optimisation Phase over the Efficiency of a Machine Learning Algorithm","volume":"2019","author":"Carriegos","year":"2019","journal-title":"Complexity"},{"key":"ref_11","unstructured":"Hensman, J., Fusi, N., and Lawrence, N.D. (2013). Gaussian Processes for Big Data. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1186\/s13640-018-0383-6","article-title":"Application research of digital media image processing technology based on wavelet transform","volume":"2018","author":"Zhang","year":"2018","journal-title":"EURASIP J. Image Video Process"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1016\/j.aiopen.2021.08.002","article-title":"Pre-trained models: Past, present and future","volume":"2","author":"Han","year":"2021","journal-title":"AI Open"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei, L. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"014008","DOI":"10.1088\/1749-4699\/8\/1\/014008","article-title":"Hyperopt: A Python library for model selection and hyperparameter optimisation","volume":"8","author":"Bergstra","year":"2015","journal-title":"Comput. Sci. Discov."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019). Optuna: A Next-generation Hyperparameter Optimisation Framework. arXiv.","DOI":"10.1145\/3292500.3330701"},{"key":"ref_18","unstructured":"Liaw, R., Liang, E., Nishihara, R., Moritz, P., Gonzalez, J.E., and Stoica, I. (2018). Tune: A Research Platform for Distributed Model Selection and Training. arXiv."},{"key":"ref_19","unstructured":"Hayes, P., Anderson, D., Cheng, B., Spriggs, T.J., Johnson, A., and McCourt, M. (2019). SigOpt Documentation, SigOpt, Inc.. Technical Report SO-12\/14 \u2013 Revision 1.07."},{"key":"ref_20","first-page":"6765","article-title":"Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimisation","volume":"18","author":"Li","year":"2017","journal-title":"J. Mach. Learn. Res."},{"key":"ref_21","unstructured":"Jaderberg, M., Dalibard, V., Osindero, S., Czarnecki, W.M., Donahue, J., Razavi, A., Vinyals, O., Green, T., Dunning, I., and Simonyan, K. (2017). Population Based Training of Neural Networks. arXiv."},{"key":"ref_22","unstructured":"Yu, T., and Zhu, H. (2020). Hyper-Parameter Optimization: A Review of Algorithms and Applications. arXiv."},{"key":"ref_23","unstructured":"Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Garnett, R. (2016). Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_24","unstructured":"Levesque, J., Gagn\u00e9, C., and Sabourin, R. (2016). Bayesian Hyperparameter Optimization for Ensemble Learning. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Stang, M., Meier, C., Rau, V., and Sax, E. (2020). An Evolutionary Approach to Hyper-Parameter Optimisation of Neural Networks, Springer.","DOI":"10.1007\/978-3-030-25629-6_111"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"721","DOI":"10.1007\/s42835-020-00343-7","article-title":"Hyperparameter Optimisation Using a Genetic Algorithm Considering Verification Time in a Convolutional Neural Network","volume":"15","author":"Han","year":"2020","journal-title":"J. Electr. Eng. Technol."},{"key":"ref_27","unstructured":"Xiao, X., Yan, M., Basodi, S., Ji, C., and Pan, Y. (2020). Efficient Hyperparameter Optimisation in Deep Learning Using a Variable Length Genetic Algorithm. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"109076","DOI":"10.1016\/j.cie.2023.109076","article-title":"Simplified swarm optimisation for hyperparameters of convolutional neural networks","volume":"177","author":"Yeh","year":"2023","journal-title":"Comput. Ind. Eng."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Basha, S., Vinakota, S.K., Dubey, S.R., Pulabaigari, V., and Mukherjee, S. (2020). Autofcl: Automatically tuning fully connected layers for transfer learning. arXiv.","DOI":"10.1007\/s00521-020-05549-4"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1016\/j.neunet.2020.10.009","article-title":"AutoTune: Automatically Tuning Convolutional Neural Networks for Improved Transfer Learning","volume":"133","author":"Basha","year":"2021","journal-title":"Neural Netw."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Vaidya, G., Ilg, L., Kshirsagar, M., Naredo, E., and Ryan, C. (2022, January 14\u201316). HyperEstimator: Evolving Computationally Efficient CNN Models with Grammatical Evolution. Proceedings of the 19th International Conference on Smart Business Technologies, Lisbon, Portugal.","DOI":"10.5220\/0011324800003280"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Banzhaf, W., Poli, R., Schoenauer, M., and Fogarty, T.C. (1998). Genetic Programming, Springer.","DOI":"10.1007\/BFb0055923"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1109\/4235.942529","article-title":"Grammatical evolution","volume":"5","author":"Ryan","year":"2001","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Ryan, C., Kshirsagar, M., Chaudhari, P., and Jachak, R. (2020, January 22\u201324). GETS: Grammatical Evolution based Optimisation of Smoothing Parameters in Univariate Time Series Forecasting. Proceedings of the 12th International Conference, ICAART, Valletta, Malta.","DOI":"10.5220\/0008963305950602"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"8602","DOI":"10.1038\/s41598-022-11613-x","article-title":"Design of a cryptographically secure pseudo random number generator with grammatical evolution","volume":"12","author":"Ryan","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_36","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). Advances in Neural Information Processing Systems 32, Curran Associates, Inc."},{"key":"ref_37","unstructured":"Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_38","unstructured":"Krizhevsky, A., and Hinton, G. (2009). Learning Multiple Layers of Features from Tiny Images, University of Toronto. Technical Report 0."},{"key":"ref_39","unstructured":"Li, F.F., Andreeto, M., Ranzato, M., and Perona, P. (2022). Caltech 101, CaltechDATA."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Fenton, M., McDermott, J., Fagan, D., Forstenlechner, S., O\u2019Neill, M., and Hemberg, E. (2017). PonyGE2: Grammatical Evolution in Python. arXiv.","DOI":"10.1145\/3067695.3082469"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1016\/j.neucom.2017.12.049","article-title":"Evolutionary Convolutional Neural Networks: An Application to Handwriting Recognition","volume":"283","author":"Baldominos","year":"2018","journal-title":"Neurocomput."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/7\/319\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:03:37Z","timestamp":1760126617000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/7\/319"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,29]]},"references-count":41,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,7]]}},"alternative-id":["a16070319"],"URL":"https:\/\/doi.org\/10.3390\/a16070319","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,29]]}}}