{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T01:20:49Z","timestamp":1780363249385,"version":"3.54.1"},"reference-count":54,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2021,6,12]],"date-time":"2021-06-12T00:00:00Z","timestamp":1623456000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2018R1A5A7059549"],"award-info":[{"award-number":["2018R1A5A7059549"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2020R1A2C1014037"],"award-info":[{"award-number":["2020R1A2C1014037"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003052","name":"Ministry of Trade, Industry and Energy","doi-asserted-by":"publisher","award":["10077553"],"award-info":[{"award-number":["10077553"]}],"id":[{"id":"10.13039\/501100003052","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>As the performance of devices that conduct large-scale computations has been rapidly improved, various deep learning models have been successfully utilized in various applications. Particularly, convolution neural networks (CNN) have shown remarkable performance in image processing tasks such as image classification and segmentation. Accordingly, more stable and robust optimization methods are required to effectively train them. However, the traditional optimizers used in deep learning still have unsatisfactory training performance for the models with many layers and weights. Accordingly, in this paper, we propose a new Adam-based hybrid optimization method called HyAdamC for training CNNs effectively. HyAdamC uses three new velocity control functions to adjust its search strength carefully in term of initial, short, and long-term velocities. Moreover, HyAdamC utilizes an adaptive coefficient computation method to prevent that a search direction determined by the first momentum is distorted by any outlier gradients. Then, these are combined into one hybrid method. In our experiments, HyAdamC showed not only notable test accuracies but also significantly stable and robust optimization abilities when training various CNN models. Furthermore, we also found that HyAdamC could be applied into not only image classification and image segmentation tasks.<\/jats:p>","DOI":"10.3390\/s21124054","type":"journal-article","created":{"date-parts":[[2021,6,14]],"date-time":"2021-06-14T22:25:46Z","timestamp":1623709546000},"page":"4054","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["HyAdamC: A New Adam-Based Hybrid Optimization Algorithm for Convolution Neural Networks"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1044-3089","authenticated-orcid":false,"given":"Kyung-Soo","family":"Kim","sequence":"first","affiliation":[{"name":"Center for Computational Social Science, Hanyang University, Seoul 04763, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9042-0599","authenticated-orcid":false,"given":"Yong-Suk","family":"Choi","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Hanyang University, Seoul 04763, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2777","DOI":"10.1007\/s10489-020-01943-6","article-title":"Deep neural network to detect COVID-19: One architecture for both CT Scans and Chest X-rays","volume":"51","author":"Mukherjee","year":"2021","journal-title":"Appl. Intell."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Irfan, M., Iftikhar, M.A., Yasin, S., Draz, U., Ali, T., Hussain, S., Bukhari, S., Alwadie, A.S., Rahman, S., and Glowacz, A. (2021). Role of Hybrid Deep Neural Networks (HDNNs), Computed Tomography, and Chest X-rays for the Detection of COVID-19. Int. J. Environ. Res. Public Health, 18.","DOI":"10.3390\/ijerph18063056"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"108566","DOI":"10.1016\/j.measurement.2020.108566","article-title":"Mechanical fault time series prediction by using EFMSAE-LSTM neural network","volume":"173","author":"Guo","year":"2021","journal-title":"Measurement"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Namasudra, S., Dhamodharavadhani, S., and Rathipriya, R. (2021). Nonlinear Neural Network Based Forecasting Model for Predicting COVID-19 Cases. Neural Process. Lett.","DOI":"10.1007\/s11063-021-10495-w"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Hong, T., Choi, J.A., Lim, K., and Kim, P. (2021). Enhancing Personalized Ads Using Interest Category Classification of SNS Users Based on Deep Neural Networks. Sensors, 21.","DOI":"10.3390\/s21010199"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"106545","DOI":"10.1016\/j.knosys.2020.106545","article-title":"A deep learning based algorithm for multi-criteria recommender systems","volume":"211","author":"Shambour","year":"2021","journal-title":"Knowl. Based Syst."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zgank, A. (2021). IoT-Based Bee Swarm Activity Acoustic Classification Using Deep Neural Networks. Sensors, 21.","DOI":"10.3390\/s21030676"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.patrec.2021.01.030","article-title":"Hierarchical deep neural network for mental stress state detection using IoT based biomarkers","volume":"145","author":"Kumar","year":"2021","journal-title":"Pattern Recognit. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., and Gao, J. (2021). Deep Learning\u2013Based Text Classification: A Comprehensive Review. ACM Comput. Surv., 54.","DOI":"10.1145\/3439726"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.ins.2020.08.037","article-title":"DFM: A parameter-shared deep fused model for knowledge base question answering","volume":"547","author":"Zhou","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going Deeper With Convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_12","unstructured":"Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18\u201323). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_16","unstructured":"Lechevallier, Y., and Saporta, G. (2010). Large-Scale Machine Learning with Stochastic Gradient Descent, Physica-Verlag HD."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/S0893-6080(98)00116-6","article-title":"On the momentum term in gradient descent learning algorithms","volume":"12","author":"Qian","year":"1999","journal-title":"Neural Netw."},{"key":"ref_18","unstructured":"Kingma, D.P., and Ba, J. (2017). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_19","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Vol. 1. No. 2."},{"key":"ref_20","first-page":"2121","article-title":"Adaptive subgradient methods for online learning and stochastic optimization","volume":"12","author":"Duchi","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_21","unstructured":"Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Adaptive Methods for Nonconvex Optimization. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_22","unstructured":"Bernstein, J., Vahdat, A., Yue, Y., and Liu, M.Y. (2021). On the distance between two neural networks and the stability of learning. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"4500","DOI":"10.1109\/TNNLS.2019.2955777","article-title":"diffGrad: An Optimization Method for Convolutional Neural Networks","volume":"31","author":"Dubey","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_24","unstructured":"Ilboudo, W.E.L., Kobayashi, T., and Sugimoto, K. (2020). Robust Stochastic Gradient Descent With Student-t Distribution Based First-Order Momentum. IEEE Trans. Neural Netw. Learn. Syst., 1\u201314."},{"key":"ref_25","unstructured":"Brownlee, J. (2018). Better Deep Learning: Train Faster, Reduce Overfitting, and Make Better Predictions, Machine Learning Mastery."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"022022","DOI":"10.1088\/1742-6596\/1168\/2\/022022","article-title":"An Overview of Overfitting and its Solutions","volume":"1168","author":"Ying","year":"2019","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1007\/s40305-020-00309-6","article-title":"Optimization for Deep Learning: An Overview","volume":"8","author":"Sun","year":"2020","journal-title":"J. Oper. Res. Soc. China"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3668","DOI":"10.1109\/TCYB.2019.2950779","article-title":"A Survey of Optimization Methods From a Machine Learning Perspective","volume":"50","author":"Sun","year":"2020","journal-title":"IEEE Trans. Cybern."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Xu, P., Roosta, F., and Mahoney, M.W. (2020, January 7\u20139). Second-order Optimization for Non-convex Machine Learning: An Empirical Study. Proceedings of the 2020 SIAM International Conference on Data Mining (SDM), Cincinnati, OH, USA.","DOI":"10.1137\/1.9781611976236.23"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Lesage-Landry, A., Taylor, J.A., and Shames, I. (2020). Second-order Online Nonconvex Optimization. IEEE Trans. Autom. Control., 1.","DOI":"10.1016\/j.automatica.2019.108771"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Nguyen, H.T., Lee, E.H., and Lee, S. (2020). Study on the classification performance of underwater sonar image classification based on convolutional neural networks for detecting a submerged human body. Sensors, 20.","DOI":"10.3390\/s20010094"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wang, W., Liang, D., Chen, Q., Iwamoto, Y., Han, X.H., Zhang, Q., Hu, H., Lin, L., and Chen, Y.W. (2020). Medical image classification using deep learning. Deep Learning in Healthcare, Springer.","DOI":"10.1007\/978-3-030-32606-7_3"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.ins.2020.05.013","article-title":"A GPU-based residual network for medical image classification in smart medicine","volume":"536","author":"Zhang","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1007\/s13246-020-00865-4","article-title":"Covid-19: Automatic detection from x-ray images utilizing transfer learning with convolutional neural networks","volume":"43","author":"Apostolopoulos","year":"2020","journal-title":"Phys. Eng. Sci. Med."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"107164","DOI":"10.1016\/j.patcog.2019.107164","article-title":"Semi-supervised robust deep neural networks for multi-label image classification","volume":"100","author":"Cevikalp","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_36","first-page":"1","article-title":"Neural networks for machine learning lecture 6a overview of mini-batch gradient descent","volume":"14","author":"Hinton","year":"2012","journal-title":"Cited"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"348","DOI":"10.1002\/wics.13","article-title":"Conjugate gradient method","volume":"1","author":"Nazareth","year":"2009","journal-title":"WIREs Comput. Stat."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1016\/0009-2614(85)80574-1","article-title":"A Broyden\u2014Fletcher\u2014Goldfarb\u2014Shanno optimization procedure for molecular geometries","volume":"122","author":"Head","year":"1985","journal-title":"Chem. Phys. Lett."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1007\/BF01589116","article-title":"On the limited memory BFGS method for large scale optimization","volume":"45","author":"Liu","year":"1989","journal-title":"Math. Program."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"718","DOI":"10.1109\/TEVC.2018.2885075","article-title":"Comprehensive Learning Particle Swarm Optimization Algorithm With Local Search for Multimodal Functions","volume":"23","author":"Cao","year":"2019","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1814","DOI":"10.1109\/TASE.2020.2979079","article-title":"Hybrid Scatter Search Algorithm for Optimal and Energy-Efficient Steelmaking-Continuous Casting","volume":"17","author":"Tan","year":"2020","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Demidova, L.A., and Gorchakov, A.V. (2020). Research and Study of the Hybrid Algorithms Based on the Collective Behavior of Fish Schools and Classical Optimization Methods. Algorithms, 13.","DOI":"10.3390\/a13040085"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2352","DOI":"10.1162\/neco_a_00990","article-title":"Deep convolutional neural networks for image classification: A comprehensive review","volume":"29","author":"Rawat","year":"2017","journal-title":"Neural Comput."},{"key":"ref_44","unstructured":"Liu, L., Jiang, H., He, P., Chen, W., Liu, X., Gao, J., and Han, J. (2020). On the Variance of the Adaptive Learning Rate and Beyond. arXiv."},{"key":"ref_45","unstructured":"Reddi, S.J., Kale, S., and Kumar, S. (2019). On the Convergence of Adam and Beyond. arXiv."},{"key":"ref_46","unstructured":"Zinkevich, M. (2003, January 21\u201324). Online convex programming and generalized infinitesimal gradient ascent. Proceedings of the 20th International Conference on Machine Learning (ICML-03), Washington, DC, USA."},{"key":"ref_47","unstructured":"Zeiler, M.D. (2012). Adadelta: An adaptive learning rate method. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1109\/ICNN.1993.298623","article-title":"A direct adaptive method for faster backpropagation learning: The RPROP algorithm","volume":"1","author":"Riedmiller","year":"1993","journal-title":"IEEE Int. Conf. Neural Netw."},{"key":"ref_49","unstructured":"Krizhevsky, A. (2021, May 01). Learning Multiple Layers of Features from Tiny Images. Available online: https:\/\/www.cs.toronto.edu\/~kriz\/learning-features-2009-TR.pdf."},{"key":"ref_50","unstructured":"Hu, Y., Huber, A., Anumula, J., and Liu, S.C. (2019). Overcoming the vanishing gradient problem in plain recurrent networks. arXiv."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pbio.1000502","article-title":"An Integrated Micro- and Macroarchitectural Analysis of the Drosophila Brain by Computer-Assisted Serial Section Electron Microscopy","volume":"8","author":"Cardona","year":"2010","journal-title":"PLoS Biol."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"114249","DOI":"10.1016\/j.eswa.2020.114249","article-title":"A Recurrent Neural Network based deep learning model for offline signature verification and recognition system","volume":"168","author":"Ghosh","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_54","unstructured":"Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative Adversarial Networks. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/12\/4054\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:13:40Z","timestamp":1760163220000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/12\/4054"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,12]]},"references-count":54,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["s21124054"],"URL":"https:\/\/doi.org\/10.3390\/s21124054","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,12]]}}}