{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T19:01:21Z","timestamp":1774551681855,"version":"3.50.1"},"reference-count":31,"publisher":"MIT Press - Journals","issue":"7","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[1999,10,1]]},"abstract":"<jats:p> This article focuses on gradient-based backpropagation algorithms that use either a common adaptive learning rate for all weights or an individual adaptive learning rate for each weight and apply the Goldstein\/Armijo line search. The learning-rate adaptation is based on descent techniques and estimates of the local Lipschitz constant that are obtained without additional error function and gradient evaluations. The proposed algorithms improve the backpropagation training in terms of both convergence rate and convergence characteristics, such as stable learning and robustness to oscillations. Simulations are conducted to compare and evaluate the convergence behavior of these gradient-based training algorithms with several popular training methods. <\/jats:p>","DOI":"10.1162\/089976699300016223","type":"journal-article","created":{"date-parts":[[2002,7,27]],"date-time":"2002-07-27T11:55:01Z","timestamp":1027770901000},"page":"1769-1796","source":"Crossref","is-referenced-by-count":104,"title":["Improving the Convergence of the Backpropagation Algorithm Using Learning Rate Adaptation Methods"],"prefix":"10.1162","volume":"11","author":[{"given":"G. D.","family":"Magoulas","sequence":"first","affiliation":[{"name":"Department of Informatics, University of Athens, GR-157.71, Athens, Greece University of Patras Artificial Intelligence Research Center (UPAIRC), University of Patras, GR-261.10 Patras, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"M. N.","family":"Vrahatis","sequence":"additional","affiliation":[{"name":"Department of Mathematics, University of Patras, GR-261.10, Patras, Greece University of Patras Artificial Intelligence Research Center (UPAIRC), University of Patras, GR-261.10 Patras, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"G. S.","family":"Androulakis","sequence":"additional","affiliation":[{"name":"Department of Mathematics, University of Patras, GR-261.10, Patras, Greece University of Patras Artificial Intelligence Research Center (UPAIRC), University of Patras, GR-261.10 Patras, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"p_1","first-page":"877","volume":"9","author":"Altman M.","year":"1961","journal-title":"Bull. Acad. Polon. Sci. Ser. Sci. Math. Astronom. Phys."},{"key":"p_2","doi-asserted-by":"publisher","DOI":"10.2140\/pjm.1966.16.1"},{"key":"p_3","first-page":"331","volume":"3","author":"Battiti R.","year":"1989","journal-title":"Complex Systems"},{"key":"p_4","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1992.4.2.141"},{"key":"p_6","doi-asserted-by":"publisher","DOI":"10.1093\/qjmam\/2.4.460"},{"key":"p_7","first-page":"536","volume":"25","author":"Cauchy A.","year":"1847","journal-title":"Comp. Rend. Acad. Sci. Paris"},{"key":"p_8","doi-asserted-by":"publisher","DOI":"10.1016\/0885-2308(87)90009-X"},{"key":"p_11","doi-asserted-by":"publisher","DOI":"10.1137\/1019005"},{"key":"p_14","first-page":"661","volume":"2","author":"Fakotakis N.","year":"1996","journal-title":"Proceedings of the IEEE International Conference on Acoustic Speech and Signal Processing"},{"key":"p_16","first-page":"581","volume":"81","author":"Fisher W.","year":"1987","journal-title":"Journal of Acoustical Society of America, Suppl. A"},{"key":"p_17","doi-asserted-by":"publisher","DOI":"10.1007\/BF01386306"},{"key":"p_18","doi-asserted-by":"publisher","DOI":"10.1109\/34.107014"},{"key":"p_19","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(91)90032-Z"},{"key":"p_20","doi-asserted-by":"publisher","DOI":"10.1109\/72.143374"},{"key":"p_21","doi-asserted-by":"publisher","DOI":"10.1109\/21.364864"},{"key":"p_22","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(88)90003-2"},{"key":"p_26","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80116-9"},{"key":"p_27","doi-asserted-by":"publisher","DOI":"10.1088\/0954-898X_2_1_007"},{"key":"p_29","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(96)00052-4"},{"key":"p_31","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80056-5"},{"key":"p_32","first-page":"199","author":"Nocedal J.","year":"1991","journal-title":"Acta Numerica"},{"key":"p_34","first-page":"593","volume":"2","author":"Parker D. B.","year":"1987","journal-title":"Proceedings of the IEEE International Conference on Neural Networks"},{"key":"p_35","doi-asserted-by":"publisher","DOI":"10.1109\/72.286921"},{"key":"p_40","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(91)90006-Q"},{"key":"p_45","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-52255-7_32"},{"key":"p_48","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(94)90052-3"},{"key":"p_49","doi-asserted-by":"publisher","DOI":"10.1007\/BF00332914"},{"key":"p_50","first-page":"619","volume":"2","author":"Watrous R. L.","year":"1987","journal-title":"Proceedings of the IEEE International Conference on Neural Networks"},{"key":"p_51","doi-asserted-by":"publisher","DOI":"10.1109\/72.165592"},{"key":"p_52","doi-asserted-by":"publisher","DOI":"10.1137\/1011036"},{"key":"p_53","doi-asserted-by":"publisher","DOI":"10.1137\/1013035"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/089976699300016223","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:23:21Z","timestamp":1615584201000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/11\/7\/1769-1796\/6320"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1999,10,1]]},"references-count":31,"journal-issue":{"issue":"7","published-print":{"date-parts":[[1999,10,1]]}},"alternative-id":["10.1162\/089976699300016223"],"URL":"https:\/\/doi.org\/10.1162\/089976699300016223","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[1999,10,1]]}}}