{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,10]],"date-time":"2025-09-10T22:41:56Z","timestamp":1757544116097,"version":"3.41.2"},"reference-count":32,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T00:00:00Z","timestamp":1616457600000},"content-version":"vor","delay-in-days":81,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2017YFB1303203","JX12413673"],"award-info":[{"award-number":["2017YFB1303203","JX12413673"]}]},{"name":"Postgraduate Research and Practice Innovation Program of Jiangsu Province","award":["2017YFB1303203","JX12413673"],"award-info":[{"award-number":["2017YFB1303203","JX12413673"]}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Computational Intelligence and Neuroscience"],"published-print":{"date-parts":[[2021,1]]},"abstract":"<jats:p>Convolutional neural networks (CNNs) are effective models for image classification and recognition. Gradient descent optimization (GD) is the basic algorithm for CNN model optimization. Since GD appeared, a series of improved algorithms have been derived. Among these algorithms, adaptive moment estimation (Adam) has been widely recognized. However, local changes are ignored in Adam to some extent. In this paper, we introduce an adaptive learning rate factor based on current and recent gradients. According to this factor, we can dynamically adjust the learning rate of each independent parameter to adaptively adjust the global convergence process. We use the factor to adjust the learning rate for each parameter. The convergence of the proposed algorithm is proven by using the regret bound approach of the online learning framework. In the experimental section, comparisons are conducted between the proposed algorithm and other existing algorithms, such as AdaGrad, RMSprop, Adam, diffGrad, and AdaHMG, on test functions and the MNIST dataset. The results show that Adam and RMSprop combined with our algorithm can not only find the global minimum faster in the experiment using the test function but also have a better convergence curve and higher test set accuracy in experiments using datasets. Our algorithm is a supplement to the existing gradient descent algorithms, which can be combined with many other existing gradient descent algorithms to improve the efficiency of iteration, speed up the convergence of the cost function, and improve the final recognition rate.<\/jats:p>","DOI":"10.1155\/2021\/6671830","type":"journal-article","created":{"date-parts":[[2021,3,24]],"date-time":"2021-03-24T01:50:14Z","timestamp":1616550614000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Gradient\u2010Sensitive Optimization for Convolutional Neural Networks"],"prefix":"10.1155","volume":"2021","author":[{"given":"Zhipeng","family":"Liu","sequence":"first","affiliation":[]},{"given":"Rui","family":"Feng","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8471-1138","authenticated-orcid":false,"given":"Xiuhan","family":"Li","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9702-6316","authenticated-orcid":false,"given":"Wei","family":"Wang","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5283-5008","authenticated-orcid":false,"given":"Xiaoling","family":"Wu","sequence":"additional","affiliation":[]}],"member":"311","published-online":{"date-parts":[[2021,3,23]]},"reference":[{"key":"e_1_2_13_1_2","doi-asserted-by":"publisher","DOI":"10.1109\/tpami.2015.2439281"},{"key":"e_1_2_13_2_2","doi-asserted-by":"crossref","unstructured":"FarooqA. AnwarS. M. AwaisM. andRehmanS. A deep CNN based multi-class classification of Alzheimer\u2019s disease using MRI Proceedings of the 2017 IEEE International Conference on Imaging Systems and Techniques (IST) October 2017 Beijing China IEEE.","DOI":"10.1109\/IST.2017.8261460"},{"key":"e_1_2_13_3_2","unstructured":"ReddyS. P. T. KarriS. T. DubeyS. R. andMukherjeeS. Spontaneous facial micro-expression recognition using 3d spatiotemporal convolutional neural networks 2019 https:\/\/arxiv.org\/abs\/1904.01390."},{"key":"e_1_2_13_4_2","doi-asserted-by":"crossref","unstructured":"RehmanS. U. TuS. HuangY. andYangZ. Face recognition: a novel un-supervised convolutional neural network method Proceedings of the 2016 IEEE International Conference of Online Analysis and Computing Science (ICOACS) May 2016 Chongqing China IEEE.","DOI":"10.1109\/ICOACS.2016.7563066"},{"key":"e_1_2_13_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/tpami.2016.2577031"},{"key":"e_1_2_13_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3065386"},{"key":"e_1_2_13_7_2","unstructured":"BashaS. H. S. GhoshS. BabuK. K. DubeyS. R. PulabaigariV. andMukherjeeS. RCCNet: An efficient convolutional neural network for histological routine colon cancer nuclei classification 2018 https:\/\/arxiv.org\/pdf\/1810.02797."},{"key":"e_1_2_13_8_2","doi-asserted-by":"crossref","unstructured":"LecunY. KavukcuogluK. andFarabetC. M. Convolutional networks and applications in vision Proceedings of 2010 IEEE International Symposium on Circuits and Systems May 2010 Paris France IEEE.","DOI":"10.1109\/ISCAS.2010.5537907"},{"key":"e_1_2_13_9_2","doi-asserted-by":"publisher","DOI":"10.3233\/aic-170739"},{"key":"e_1_2_13_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2019.06.084"},{"key":"e_1_2_13_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-45720-8_50"},{"key":"e_1_2_13_12_2","unstructured":"RuderS. An overview of gradient descent optimization algorithms 2016 https:\/\/arxiv.org\/abs\/1609.04747."},{"key":"e_1_2_13_13_2","doi-asserted-by":"publisher","DOI":"10.1162\/089976698300017746"},{"key":"e_1_2_13_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/0041-5553(64)90137-5"},{"key":"e_1_2_13_15_2","unstructured":"SutskeverI. MartensJ. DahlG. andHintonG. On the importance of initialization and momentum in deep learning Proceedings of the 30 th International Conference on Machine Learning June 2013 Atlanta Georgia USA 1139\u20131147."},{"key":"e_1_2_13_16_2","unstructured":"NesterovB. Y. A method of solving a convex programming problem with convergence rate O 2010."},{"key":"e_1_2_13_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-010-0434-y"},{"key":"e_1_2_13_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2011.2146788"},{"volume-title":"Deep Learning","year":"2016","author":"Goodfellow I.","key":"e_1_2_13_19_2"},{"key":"e_1_2_13_20_2","unstructured":"ZeilerM. D. ADADELTA: an adaptive learning rate method 2016 https:\/\/arxiv.org\/abs\/1212.5701."},{"key":"e_1_2_13_21_2","unstructured":"KingmaD.andBaJ. Adam: a method for stochastic optimization 2014 https:\/\/arxiv.org\/abs\/1412.6980."},{"key":"e_1_2_13_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/access.2019.2897639"},{"key":"e_1_2_13_23_2","unstructured":"DozatT. Incorporating nesterov momentum into adam 2016."},{"key":"e_1_2_13_24_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics9111809"},{"key":"e_1_2_13_25_2","doi-asserted-by":"publisher","DOI":"10.3390\/e20040290"},{"key":"e_1_2_13_26_2","article-title":"An optimization method for convolutional neural networks","volume":"31","author":"Dubey S. R.","year":"2019","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_2_13_27_2","article-title":"Estimating an optimal learning rate for a deep neural network","volume":"5","author":"Surmenok P.","year":"2017","journal-title":"Towards Data Science"},{"key":"e_1_2_13_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-007-5016-8"},{"key":"e_1_2_13_29_2","article-title":"Gradient-based learning applied to document recognition","volume":"88","author":"Lecun Y.","year":"1998","journal-title":"Proceedings of the IEEE"},{"key":"e_1_2_13_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2012.2211477"},{"key":"e_1_2_13_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623612"},{"key":"e_1_2_13_32_2","unstructured":"LoshchilovI.andHutterF. Online batch selection for faster training of neural networks 2015 https:\/\/arxiv.org\/abs\/1511.06343."}],"container-title":["Computational Intelligence and Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2021\/6671830.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2021\/6671830.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/2021\/6671830","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T12:02:26Z","timestamp":1722945746000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1155\/2021\/6671830"}},"subtitle":[],"editor":[{"given":"Jussi","family":"Tohka","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2021,1]]},"references-count":32,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,1]]}},"alternative-id":["10.1155\/2021\/6671830"],"URL":"https:\/\/doi.org\/10.1155\/2021\/6671830","archive":["Portico"],"relation":{},"ISSN":["1687-5265","1687-5273"],"issn-type":[{"type":"print","value":"1687-5265"},{"type":"electronic","value":"1687-5273"}],"subject":[],"published":{"date-parts":[[2021,1]]},"assertion":[{"value":"2020-10-19","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-02-17","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-03-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"6671830"}}