{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T08:48:35Z","timestamp":1771231715770,"version":"3.50.1"},"reference-count":40,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2017,2,21]],"date-time":"2017-02-21T00:00:00Z","timestamp":1487635200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Support vector machine (SVM) is one of the most successful learning methods for solving classi\ufb01cation problems. Despite its popularity, SVM has the serious drawback that it is sensitive to outliers in training samples. The penalty on misclassi\ufb01cation is de\ufb01ned by a convex loss called the hinge loss, and the unboundedness of the convex loss causes the sensitivity to outliers. To deal with outliers, robust SVMs have been proposed by replacing the convex loss with a non-convex bounded loss called the ramp loss. In this paper, we study the breakdown point of robust SVMs. The breakdown point is a robustness measure that is the largest amount of contamination such that the estimated classi\ufb01er still gives information about the non-contaminated data. The main contribution of this paper is to show an exact evaluation of the breakdown point of robust SVMs. For learning parameters such as the regularization parameter, we derive a simple formula that guarantees the robustness of the classi\ufb01er. When the learning parameters are determined with a grid search using cross-validation, our formula works to reduce the number of candidate search points. Furthermore, the theoretical \ufb01ndings are con\ufb01rmed in numerical experiments. We show that the statistical properties of robust SVMs are well explained by a theoretical analysis of the breakdown point.<\/jats:p>","DOI":"10.3390\/e19020083","type":"journal-article","created":{"date-parts":[[2017,2,22]],"date-time":"2017-02-22T11:40:52Z","timestamp":1487763652000},"page":"83","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Breakdown Point of Robust Support Vector Machines"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6878-5850","authenticated-orcid":false,"given":"Takafumi","family":"Kanamori","sequence":"first","affiliation":[{"name":"Department of Computer Science and Mathematical Informatics, Nagoya University, Nagoya 464-8601, Japan"},{"name":"RIKEN Center for Advanced Intelligence Project, Tokyo 103-0027, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuhei","family":"Fujiwara","sequence":"additional","affiliation":[{"name":"TOPGATE Co. Ltd., Bunkyo-ku, Tokyo 113-0033, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Akiko","family":"Takeda","sequence":"additional","affiliation":[{"name":"Institute of Statistical Mathematics, Tokyo 190-8562, Japan"},{"name":"RIKEN Center for Advanced Intelligence Project, Tokyo 103-0027, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,2,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-vector networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sch\u00f6lkopf, B., and Smola, A.J. (2002). Learning with Kernels, MIT Press.","DOI":"10.7551\/mitpress\/4175.001.0001"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Berlinet, A., and Thomas-Agnan, C. (2004). Reproducing Kernel Hilbert Spaces in Probability and Statistics, Kluwer Academic.","DOI":"10.1007\/978-1-4419-9096-9"},{"key":"ref_4","unstructured":"Perez-Cruz, F., Weston, J., Hermann, D.J.L., and Sch\u00f6lkopf, B. (2003). Advances in Learning Theory: Methods, Models and Applications 190, IOS Press."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1207","DOI":"10.1162\/089976600300015565","article-title":"New support vector algorithms","volume":"12","author":"Smola","year":"2000","journal-title":"Neural Comput."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1023\/A:1018628609742","article-title":"Least squares support vector machine classifiers","volume":"9","author":"Suykens","year":"1999","journal-title":"Neural Process. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1198\/016214505000000907","article-title":"Convexity, classification, and risk bounds","volume":"101","author":"Bartlett","year":"2006","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_8","first-page":"67","article-title":"On the influence of the kernel on the consistency of support vector machines","volume":"2","author":"Steinwart","year":"2001","journal-title":"J. Mach. Learn. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1214\/aos\/1079120130","article-title":"Statistical behavior and consistency of classification methods based on convex risk minimization","volume":"32","author":"Zhang","year":"2004","journal-title":"Ann. Stat."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"724","DOI":"10.1198\/016214503000000639","article-title":"On \u03c8-learning","volume":"98","author":"Shen","year":"2003","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_11","unstructured":"Yu, Y., Yang, M., Xu, L., White, M., and Schuurmans, D. (2010). Neural Information Processing Systems, MIT Press."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Collobert, R., Sinz, F., Weston, J., and Bottou, L. (2006, January 25\u201329). Trading Convexity for Scalability. Proceedings of the ICML06, 23rd International Conference on Machine Learning, Pittsburgh, PA, USA.","DOI":"10.1145\/1143844.1143870"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"974","DOI":"10.1198\/016214507000000617","article-title":"Robust truncated hinge loss support vector machines","volume":"102","author":"Wu","year":"2007","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_14","unstructured":"Yu, Y., Aslan, \u00d6., and Schuurmans, D. (2012). Advances in Neural Information Processing Systems 25, Curran Associates, Inc."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1217","DOI":"10.1162\/NECO_a_00837","article-title":"Robust Support Vector Machines for Classification with Nonconvex and Smooth Losses","volume":"28","author":"Feng","year":"2016","journal-title":"Neural Comput."},{"key":"ref_16","unstructured":"Tsyurmasto, P., Uryasev, S., and Gotoh, J. (2013). Support Vector Classification with Positive Homogeneous Risk Functionals, Department of Industrial and Systems Engineering, University of Florida. Technical Report, Research Report 2013-4."},{"key":"ref_17","unstructured":"Xu, L., Crammer, K., and Schuurmans, D. (2006, January 16\u201320). Robust Support Vector Machine Training Via Convex Outlier Ablation. Proceedings of the AAAI, Boston, MA, USA."},{"key":"ref_18","unstructured":"Fujiwara, S., Takeda, A., and Kanamori, T. (2014). DC Algorithm for Extended Robust Support Vector Machine, The University of Tokyo. Technical Report METR 2014\u201338."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2541","DOI":"10.1162\/NECO_a_00647","article-title":"Extended robust support vector machine based on financial risk minimization","volume":"26","author":"Takeda","year":"2014","journal-title":"Neural Comput."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Maronna, R., Martin, R.D., and Yohai, V. (2006). Robust Statistics: Theory and Methods, Wiley.","DOI":"10.1002\/0470010940"},{"key":"ref_21","first-page":"1651","article-title":"Boosting the margin: A new explanation for the effectiveness of voting methods","volume":"26","author":"Schapire","year":"1998","journal-title":"Ann. Stat."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/0022-247X(71)90184-3","article-title":"Some results on Tchebycheffian spline functions","volume":"33","author":"Kimeldorf","year":"1971","journal-title":"J. Math. Anal. Appl."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wahba, G. (1999). Advances in Kernel Methods, MIT Press. Chapter Support Vector Machines, Reproducing Kernel Hilbert Spaces, and Randomized GACV.","DOI":"10.7551\/mitpress\/1130.003.0009"},{"key":"ref_24","first-page":"1056","article-title":"\u03bd-Support Vector Machine as Conditional Value-at-Risk Minimization","volume":"Volume 307","author":"Cohen","year":"2008","journal-title":"Proceedings of the ICML, ACM International Conference Proceeding Series"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1443","DOI":"10.1016\/S0378-4266(02)00271-6","article-title":"Conditional value-at-risk for general loss distributions","volume":"26","author":"Rockafellar","year":"2002","journal-title":"J. Bank. Financ."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Boyd, S., and Vandenberghe, L. (2004). Convex Optimization, Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"},{"key":"ref_27","first-page":"289","article-title":"Convex analysis approach to d.c. programming: Theory, algorithms and applications","volume":"22","author":"Dinh","year":"1997","journal-title":"Acta Math. Vietnam."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1162\/08997660360581958","article-title":"The concave-convex procedure","volume":"15","author":"Yuille","year":"2003","journal-title":"Neural Comput."},{"key":"ref_29","unstructured":"Solla, S.A., Leen, T.K., and M\u00fcller, K.-R. (2000). Advances in Neural Information Processing Systems 12, MIT Press."},{"key":"ref_30","first-page":"1461","article-title":"Conjugate relation between loss functions and uncertainty sets in classification problems","volume":"14","author":"Kanamori","year":"2013","journal-title":"J. Mach. Learn. Res."},{"key":"ref_31","unstructured":"Langford, J., and Pineau, J. (July, January 26). A Unified Robust Classification Model. Proceedings of the 29th International Conference on Machine Learning (ICML-12), ICML\u201912, Edinburgh, Scotland."},{"key":"ref_32","unstructured":"Bertsekas, D., Nedic, A., and Ozdaglar, A. (2003). Convex Analysis and Optimization, Athena Scientific."},{"key":"ref_33","unstructured":"Donoho, D., and Huber, P. (1983). A Festschrift for Erich L. Lehmann, CRC Press."},{"key":"ref_34","unstructured":"Hampel, F.R., Rousseeuw, P.J., Ronchetti, E.M., and Stahel, W.A. (1986). Robust Statistics. The Approach Based on Influence Functions, John Wiley and Sons, Inc."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Huber, P.J., and Ronchetti, E.M. (2009). Robust Statistics, Wiley. [2nd ed.].","DOI":"10.1002\/9780470434697"},{"key":"ref_36","first-page":"1007","article-title":"On robustness properties of convex risk minimization methods for pattern recognition","volume":"5","author":"Christmann","year":"2004","journal-title":"J. Mach. Learn. Res."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s10479-004-5022-1","article-title":"The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems","volume":"133","author":"Dinh","year":"2005","journal-title":"Ann. Oper. Res."},{"key":"ref_38","unstructured":"R Core Team (2014). R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1274","DOI":"10.1109\/TPAMI.2003.1233901","article-title":"On the optimal parameter choice for \u03bd-support vector machines","volume":"25","author":"Steinwart","year":"2003","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1080\/10618600.2012.680866","article-title":"Adaptively weighted large margin classifiers","volume":"22","author":"Wu","year":"2013","journal-title":"J. Comput. Graph. Stat."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/2\/83\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:28:50Z","timestamp":1760207330000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/2\/83"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,2,21]]},"references-count":40,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2017,2]]}},"alternative-id":["e19020083"],"URL":"https:\/\/doi.org\/10.3390\/e19020083","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,2,21]]}}}