{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T18:39:13Z","timestamp":1787337553884,"version":"build-2736575974"},"reference-count":41,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Comput."],"published-print":{"date-parts":[[2005,1]]},"abstract":"<jats:p>Kernel-based linear-threshold algorithms, such as support vector machines and Perceptron-like algorithms, are among the best available techniques for solving pattern classification problems. In this paper, we describe an extension of the classical Perceptron algorithm, called second-order Perceptron, and analyze its performance within the mistake bound model of on-line learning. The bound achieved by our algorithm depends on the sensitivity to second-order data information and is the best known mistake bound for (efficient) kernel-based linear-threshold classifiers to date. This mistake bound, which strictly generalizes the well-known Perceptron bound, is expressed in terms of the eigenvalues of the empirical data correlation matrix and depends on a parameter controlling the sensitivity of the algorithm to the distribution of these eigenvalues. Since the optimal setting of this parameter is not known a priori, we also analyze two variants of the second-order Perceptron algorithm: one that adaptively sets the value of the parameter in terms of the number of mistakes made so far, and one that is parameterless, based on pseudoinverses.<\/jats:p>","DOI":"10.1137\/s0097539703432542","type":"journal-article","created":{"date-parts":[[2005,3,28]],"date-time":"2005-03-28T21:00:23Z","timestamp":1112043623000},"page":"640-668","source":"Crossref","is-referenced-by-count":100,"title":["A Second-Order Perceptron Algorithm"],"prefix":"10.1137","volume":"34","author":[{"given":"Nicol\u00f2","family":"Cesa-Bianchi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alex","family":"Conconi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Claudio","family":"Gentile","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2006,7,27]]},"reference":[{"key":"R1","first-page":"1307","volume":"25","author":"Azerman M.","year":"1964","journal-title":"Avtomat. i Telemeh."},{"key":"R2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00116828"},{"key":"R3","doi-asserted-by":"publisher","DOI":"10.1006\/jcss.2001.1795"},{"key":"R4","doi-asserted-by":"crossref","unstructured":"Peter Auer, Manfred Warmuth, Tracking the best disjunction, IEEE Comput. Soc. Press, Los Alamitos, CA, 1995, 312\u2013321MR1619093","DOI":"10.1109\/SFCS.1995.492487"},{"key":"R5","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010896012157"},{"key":"R6","volume-title":"Generalized inverses: theory and applications","author":"Ben\u2010Israel Adi","year":"1980"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.1103\/RevModPhys.34.123"},{"key":"R8","volume-title":"Online computation and competitive analysis","author":"Borodin Allan","year":"1998"},{"key":"R9","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2004.833339"},{"key":"R10","doi-asserted-by":"publisher","DOI":"10.1145\/258128.258179"},{"key":"R11","doi-asserted-by":"publisher","DOI":"10.1007\/BF00115301"},{"key":"R12","doi-asserted-by":"crossref","unstructured":"N. Cristianini and J. Shawe\u2010Taylor,\n                      An Introduction to Support Vector Machines\n                      , Cambridge University Press, Cambridge, UK, 2001.","DOI":"10.1017\/CBO9780511801389"},{"key":"R13","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-0711-5"},{"key":"R14","volume-title":"Pattern classification","author":"Duda Richard","year":"2001"},{"key":"R15","doi-asserted-by":"publisher","DOI":"10.1006\/jcss.2001.1798"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.1023\/A:1021825927902"},{"key":"R17","first-page":"213","volume":"2","author":"Gentile Claudio","year":"2002","journal-title":"J. Mach. Learn. Res."},{"key":"R18","doi-asserted-by":"publisher","DOI":"10.1023\/A:1026319107706"},{"key":"R19","unstructured":"C. Gentile and M. Warmuth,\n                      Linear hinge loss and average margin\n                      , in Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems 10, MIT Press, Cambridge, MA, 1999, pp. 225\u2013231."},{"key":"R20","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010844028087"},{"key":"R21","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007424614876"},{"key":"R22","first-page":"281","volume":"1","author":"Herbster Mark","year":"2001","journal-title":"J. Mach. Learn. Res."},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.1080\/00401706.1970.10488634"},{"key":"R24","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511810817"},{"key":"R25","doi-asserted-by":"publisher","DOI":"10.1023\/A:1017938623079"},{"key":"R26","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(97)00039-8"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.1023\/A:1012435301888"},{"key":"R28","doi-asserted-by":"publisher","DOI":"10.1007\/BF00116827"},{"key":"R29","doi-asserted-by":"crossref","unstructured":"N. Littlestone,\n                      Redundant noisy attributes, attribute errors, and linear threshold learning using Winnow\n                      , in Proceedings of the 4th Annual Workshop on Computational Learning Theory, Morgan\u2010Kaufmann, San Mateo, CA, 1991, pp. 147\u2013156.","DOI":"10.1016\/B978-1-55860-213-7.50017-1"},{"key":"R30","doi-asserted-by":"publisher","DOI":"10.1006\/inco.1994.1009"},{"key":"R31","volume-title":"Introduction to linear algebra","author":"Minc Henryk","year":"1965"},{"key":"R32","unstructured":"A. Novikoff, On convergence proofs for perceptrons, Polytechnic Press of Polytechnic Inst. of Brooklyn, Brooklyn, N.Y., 1963, 615\u2013622MR0175722"},{"key":"R33","unstructured":"R. Rifkin, G. Yeo, and T. Poggio,\n                      Regularized least squares classification\n                      , in Advances in Learning Theory: Methods, Model and Applications, NATO Sci. Ser. III Comput. Systems Sci. 190, IOS Press, Amsterdam, 2003, pp. 131\u2013153."},{"key":"R34","doi-asserted-by":"publisher","DOI":"10.1037\/h0042519"},{"key":"R35","unstructured":"G. Saunders, A. Gammerman, and V. Vovk,\n                      Ridge regression learning algorithm in dual variables\n                      , in Proceedings of the 15th International Conference on Machine Learning, Morgan\u2010Kaufmann, San Francisco, CA, 1998, pp. 515\u2013521."},{"key":"R36","doi-asserted-by":"crossref","unstructured":"B. Sch\u00f6lkopf and A. Smola,\n                      Learning with Kernels\n                      , MIT Press, Cambridge, MA, 2002.","DOI":"10.7551\/mitpress\/4175.001.0001"},{"key":"R37","doi-asserted-by":"crossref","unstructured":"J. A. K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle,\n                      Least Squares Support Vector Machines\n                      , World Scientific, Singapore, 2002.","DOI":"10.1142\/5089"},{"key":"R38","unstructured":"V. Vapnik,\n                      Statistical Learning Theory\n                      , John Wiley and Sons, New York, 1998."},{"key":"R39","doi-asserted-by":"crossref","unstructured":"V. Vovk,\n                      Aggregating strategies\n                      , in Proceedings of the 3rd Annual Workshop on Computational Learning Theory, Morgan\u2010Kaufmann, San Mateo, CA, 1990, pp. 372\u2013383.","DOI":"10.1016\/B978-1-55860-146-8.50032-1"},{"key":"R40","doi-asserted-by":"publisher","DOI":"10.1111\/j.1751-5823.2001.tb00457.x"},{"key":"R41","doi-asserted-by":"publisher","DOI":"10.1109\/18.945262"}],"container-title":["SIAM Journal on Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/S0097539703432542","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T18:16:56Z","timestamp":1787336216000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/S0097539703432542"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2005,1]]},"references-count":41,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2005,1]]}},"alternative-id":["10.1137\/S0097539703432542"],"URL":"https:\/\/doi.org\/10.1137\/s0097539703432542","relation":{},"ISSN":["0097-5397","1095-7111"],"issn-type":[{"value":"0097-5397","type":"print"},{"value":"1095-7111","type":"electronic"}],"subject":[],"published":{"date-parts":[[2005,1]]}}}