{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,16]],"date-time":"2025-05-16T23:43:41Z","timestamp":1747439021283},"reference-count":12,"publisher":"MIT Press - Journals","issue":"8","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[1998,11,1]]},"abstract":"<jats:p> We analyze online gradient descent learning from finite training sets at noninfinitesimal learning rates \u03b7. Exact results are obtained for the time-dependent generalization error of a simple model system: a linear network with a large number of weights N, trained on p = \u03b1N examples. This allows us to study in detail the effects of finite training set size \u03b1 on, for example, the optimal choice of learning rate \u03b7. We also compare online and offline learning, for respective optimal settings of \u03b7 at given final learning time. Online learning turns out to be much more robust to input bias and actually outperforms offline learning when such bias is present; for unbiased inputs, online and offline learning perform almost equally well. <\/jats:p>","DOI":"10.1162\/089976698300017034","type":"journal-article","created":{"date-parts":[[2002,7,27]],"date-time":"2002-07-27T11:55:01Z","timestamp":1027770901000},"page":"2201-2217","source":"Crossref","is-referenced-by-count":8,"title":["Online Learning from Finite Training Sets and Robustness to Input Bias"],"prefix":"10.1162","volume":"10","author":[{"given":"Peter","family":"Sollich","sequence":"first","affiliation":[{"name":"Department of Physics, University of Edinburgh, Edinburgh EH9 3JZ, U.K."}]},{"given":"David","family":"Barber","sequence":"additional","affiliation":[{"name":"Real World Computing Partnership Theoretical Foundation, SNN, University of Nijmegen, 6525 EZ Nijmegen, The Netherlands"}]}],"member":"281","reference":[{"key":"p_1","doi-asserted-by":"publisher","DOI":"10.1088\/0305-4470\/28\/3\/018"},{"key":"p_3","doi-asserted-by":"publisher","DOI":"10.1088\/0305-4470\/22\/12\/016"},{"key":"p_4","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.44.2718"},{"key":"p_5","doi-asserted-by":"publisher","DOI":"10.1109\/72.508935"},{"key":"p_6","doi-asserted-by":"publisher","DOI":"10.1088\/0305-4470\/25\/5\/020"},{"key":"p_7","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.66.2396"},{"key":"p_8","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1991.3.2.226"},{"key":"p_9","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.52.4225"},{"key":"p_10","doi-asserted-by":"publisher","DOI":"10.1088\/0305-4470\/27\/23\/020"},{"key":"p_11","doi-asserted-by":"publisher","DOI":"10.1088\/0305-4470\/28\/21\/016"},{"key":"p_12","doi-asserted-by":"publisher","DOI":"10.1209\/epl\/i1997-00271-3"},{"key":"p_15","doi-asserted-by":"publisher","DOI":"10.1088\/0305-4470\/26\/13\/021"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/089976698300017034","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:51:57Z","timestamp":1615585917000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/10\/8\/2201-2217\/6219"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1998,11,1]]},"references-count":12,"journal-issue":{"issue":"8","published-print":{"date-parts":[[1998,11,1]]}},"alternative-id":["10.1162\/089976698300017034"],"URL":"https:\/\/doi.org\/10.1162\/089976698300017034","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[1998,11,1]]}}}