{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T17:51:30Z","timestamp":1740160290313,"version":"3.37.3"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2021,3,14]],"date-time":"2021-03-14T00:00:00Z","timestamp":1615680000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,14]],"date-time":"2021-03-14T00:00:00Z","timestamp":1615680000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61972100"],"award-info":[{"award-number":["61972100"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int. J. Mach. Learn. &amp; Cyber."],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>A classification problem aims at constructing a best classifier with the smallest risk. When the sample size approaches infinity, the learning algorithms for a classification problem are characterized by an asymptotical property, i.e., universal consistency. It plays a crucial role in measuring the construction of classification rules. A universal consistent algorithm ensures that the larger the sample size of the algorithm is, the more accurately the distribution of the samples could be reconstructed. Support vector machines (SVMs) are regarded as one of the most important models in binary classification problems. How to effectively extend SVMs to twin support vector machines (TWSVMs) so as to improve performance of classification has gained increasing interest in many research areas recently. Many variants for TWSVMs have been proposed and used in practice. Thus in this paper, we focus on the universal consistency of TWSVMs in a binary classification setting. We first give a general framework for TWSVM classifiers that unifies most of the variants of TWSVMs for binary classification problems. Based on it, we then investigate the universal consistency of TWSVMs. To do this, we give some useful definitions of risk, Bayes risk and universal consistency for TWSVMs. Theoretical results indicate that universal consistency is valid for various TWSVM classifiers under some certain conditions, including covering number, localized covering number and stability. For applications of our general framework, several variants of TWSVMs are considered.<\/jats:p>","DOI":"10.1007\/s13042-021-01281-0","type":"journal-article","created":{"date-parts":[[2021,3,14]],"date-time":"2021-03-14T11:02:21Z","timestamp":1615719741000},"page":"1867-1877","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Universal consistency of twin support vector machines"],"prefix":"10.1007","volume":"12","author":[{"given":"Weixia","family":"Xu","sequence":"first","affiliation":[]},{"given":"Dingjiang","family":"Huang","sequence":"additional","affiliation":[]},{"given":"Shuigeng","family":"Zhou","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,3,14]]},"reference":[{"issue":"3","key":"1281_CR1","doi-asserted-by":"publisher","first-page":"631","DOI":"10.1137\/19M125649X","volume":"2","author":"J Berner","year":"2020","unstructured":"Berner J, Grohs P, Jentzen A (2020) Analysis of the generalization error: empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of black-scholes partial differential equations. SIAM J Math Data Sci 2(3):631\u2013657","journal-title":"SIAM J Math Data Sci"},{"issue":"1","key":"1281_CR2","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1016\/0021-9045(81)90030-7","volume":"31","author":"WR Bloom","year":"1981","unstructured":"Bloom WR, Elliott D (1981) The modulus of continuity of the remainder in the approximation of Lipschitz functions. J Approx Theory 31(1):59\u201366","journal-title":"J Approx Theory"},{"key":"1281_CR3","first-page":"196","volume-title":"Advances in neural information processing systems","author":"O Bousquet","year":"2000","unstructured":"Bousquet O, Elisseeff A (2000) Algorithmic stability and generalization performance. In: Leen TK, Dietterich TG, Tresp V (eds) Advances in neural information processing systems, vol 13. MIT Press, Denver, pp 196\u2013202"},{"issue":"6","key":"1281_CR4","doi-asserted-by":"publisher","first-page":"2507","DOI":"10.1214\/15-AOS1350","volume":"43","author":"CT Brownlees","year":"2015","unstructured":"Brownlees CT, Joly E, Lugosi G (2015) Empirical risk minimization for heavy-tailed losses. Ann Stat 43(6):2507\u20132536","journal-title":"Ann Stat"},{"issue":"11","key":"1281_CR5","first-page":"2435","volume":"7","author":"DR Chen","year":"2006","unstructured":"Chen DR, Sun T (2006) Consistency of multiclass empirical risk minimization methods based on convex loss. J Mach Learn Res 7(11):2435\u20132447","journal-title":"J Mach Learn Res"},{"key":"1281_CR6","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-0711-5","volume-title":"A probabilistic theory of pattern recognition, stochastic modelling and applied probability","author":"L Devroye","year":"1996","unstructured":"Devroye L, Gy\u00f6rfi L, Lugosi G (1996) A probabilistic theory of pattern recognition, stochastic modelling and applied probability, vol 31. Springer, Berlin"},{"key":"1281_CR7","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1016\/j.neucom.2018.06.061","volume":"315","author":"F Dumpert","year":"2018","unstructured":"Dumpert F, Andreas C (2018) Universal consistency and robustness of localized support vector machines. Neurocomputing 315:96\u2013106","journal-title":"Neurocomputing"},{"key":"1281_CR8","unstructured":"Fathony R, Behpour S, Zhang X, Ziebart BD (2018) Efficient and consistent adversarial bipartite matching. In: Dy JG, Krause A (eds) Proceedings of the 35th international conference on machine learning, ICML 2018, vol 80. PMLR, Stockholm, pp 1457\u20131466"},{"key":"1281_CR9","first-page":"223","volume-title":"Recent advances of neural network models and applications, WIRN 2013","author":"M Fiasche","year":"2014","unstructured":"Fiasche M (2014) SVM tree for personalized transductive learning in bioinformatics classification problems. In: Bassis S, Esposito A, Morabito FC (eds) Recent advances of neural network models and applications, WIRN 2013, vol 26. Springer, Salerno, pp 223\u2013231"},{"key":"1281_CR10","first-page":"11","volume-title":"Breakthroughs in statistics. Springer series in statistics (perspectives in statistics)","author":"RA Fisher","year":"1992","unstructured":"Fisher RA (1992) On the mathematical foundations of theoretical statistics. In: Kotz S, Johnson NL (eds) Breakthroughs in statistics. Springer series in statistics (perspectives in statistics). Springer, Berlin, pp 11\u201344"},{"issue":"11","key":"1281_CR11","doi-asserted-by":"publisher","first-page":"7153","DOI":"10.1007\/s00521-018-3551-9","volume":"31","author":"D Gupta","year":"2019","unstructured":"Gupta D, Richhariya B, Borah P (2019) A fuzzy twin support vector machine based on information entropy for class imbalance learning. Neural Comput Appl 31(11):7153\u20137164","journal-title":"Neural Comput Appl"},{"key":"1281_CR12","unstructured":"Gy\u00f6rfi L, Weiss R (2020) Universal consistency and rates of convergence of multiclass prototype algorithms in metric spaces. arXiv:2010.00636"},{"issue":"5","key":"1281_CR13","doi-asserted-by":"publisher","first-page":"905","DOI":"10.1109\/TPAMI.2007.1068","volume":"29","author":"RJ Khemchandani","year":"2007","unstructured":"Khemchandani RJ, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905\u2013910","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"13","key":"1281_CR14","doi-asserted-by":"publisher","first-page":"1842","DOI":"10.1016\/j.patrec.2008.05.016","volume":"29","author":"MA Kumar","year":"2008","unstructured":"Kumar MA, Gopal M (2008) Application of smoothing technique on twin support vector machines. Pattern Recognit Lett 29(13):1842\u20131848","journal-title":"Pattern Recognit Lett"},{"issue":"11","key":"1281_CR15","doi-asserted-by":"publisher","first-page":"1437","DOI":"10.1016\/j.patrec.2010.02.015","volume":"31","author":"MA Kumar","year":"2010","unstructured":"Kumar MA, Gopal M (2010) A comparison study on multiple binary-class svm methods for unilabel text categorization. Pattern Recognit Lett 31(11):1437\u20131444","journal-title":"Pattern Recognit Lett"},{"key":"1281_CR16","doi-asserted-by":"publisher","first-page":"1132","DOI":"10.1016\/j.neucom.2015.05.010","volume":"168","author":"X Liu","year":"2015","unstructured":"Liu X, Wan A (2015) Universal consistency of extreme learning machine for RBFNs case. Neurocomputing 168:1132\u20131137","journal-title":"Neurocomputing"},{"key":"1281_CR17","unstructured":"Liu Y (2007) Fisher consistency of multicategory support vector machines. In: Meila M, Shen X(eds) Proceedings of the eleventh international conference on artificial intelligence and statistics, AISTATS 2007, vol 2, San Juan, Puerto Rico, pp 291\u2013298"},{"issue":"1","key":"1281_CR18","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1109\/TPAMI.2006.17","volume":"28","author":"OL Mangasarian","year":"2006","unstructured":"Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69\u201374","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"1","key":"1281_CR19","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1016\/j.patcog.2012.06.019","volume":"46","author":"Z Qi","year":"2013","unstructured":"Qi Z, Tian Y, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recognit 46(1):305\u2013316","journal-title":"Pattern Recognit"},{"issue":"7","key":"1281_CR20","doi-asserted-by":"publisher","first-page":"1627","DOI":"10.1007\/s00521-012-0894-5","volume":"22","author":"Y Shao","year":"2013","unstructured":"Shao Y, Deng N (2013) A novel margin-based twin support vector machine with unity norm hyperplanes. Neural Comput Appl 22(7):1627\u20131635","journal-title":"Neural Comput Appl"},{"issue":"6","key":"1281_CR21","doi-asserted-by":"publisher","first-page":"962","DOI":"10.1109\/TNN.2011.2130540","volume":"22","author":"Y Shao","year":"2011","unstructured":"Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962\u2013968","journal-title":"IEEE Trans Neural Netw"},{"key":"1281_CR22","doi-asserted-by":"publisher","first-page":"276","DOI":"10.1016\/j.knosys.2014.10.011","volume":"73","author":"YH Shao","year":"2015","unstructured":"Shao YH, Chen WJ, Wang Z, Li CN, Deng NY (2015) Weighted linear loss twin support vector machine for large-scale classification. Knowl Based Syst 73:276\u2013288","journal-title":"Knowl Based Syst"},{"key":"1281_CR23","doi-asserted-by":"crossref","unstructured":"Singh G, Chhabra I (2018) Effective and fast face recognition system using complementary OC-LBP and HOG feature descriptors with SVM classifier. J Inf Technol Res 91\u2013110","DOI":"10.4018\/JITR.2018010106"},{"issue":"1","key":"1281_CR24","first-page":"67","volume":"2","author":"I Steinwart","year":"2001","unstructured":"Steinwart I (2001) On the influence of the kernel on the consistency of support vector machines. J Mach Learn Res 2(1):67\u201393","journal-title":"J Mach Learn Res"},{"issue":"1","key":"1281_CR25","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1109\/TIT.2004.839514","volume":"51","author":"I Steinwart","year":"2005","unstructured":"Steinwart I (2005) Consistency of support vector machines and other regularized kernel classifiers. IEEE Trans Inf Theory 51(1):128\u2013142","journal-title":"IEEE Trans Inf Theory"},{"key":"1281_CR26","first-page":"831","volume-title":"Advances in neural information processing systems","author":"VN Vapnik","year":"1991","unstructured":"Vapnik VN (1991) Principles of risk minimization for learning theory. In: Moody JE, Hanson SJ, Lippmann R (eds) Advances in neural information processing systems, vol 4. Morgan Kaufmann, Denver, pp 831\u2013838"},{"issue":"2","key":"1281_CR27","doi-asserted-by":"publisher","first-page":"264","DOI":"10.1137\/1116025","volume":"16","author":"VN Vapnik","year":"1971","unstructured":"Vapnik VN, Chervonenkis AY (1971) On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab Appl 16(2):264\u2013280","journal-title":"Theory Probab Appl"},{"issue":"3","key":"1281_CR28","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1137\/1126059","volume":"26","author":"VN Vapnik","year":"1982","unstructured":"Vapnik VN, Chervonenkis AY (1982) Necessary and sufficient conditions for the uniform convergence of the means to their expectations. Theory Probab Appl 26(3):532\u2013553","journal-title":"Theory Probab Appl"},{"issue":"3","key":"1281_CR29","first-page":"283","volume":"1","author":"VN Vapnik","year":"1991","unstructured":"Vapnik VN, Chervonenkis AY (1991) The necessary and sufficient conditions for consistency in the empirical risk minimization method. Pattern Recognit Image Anal 1(3):283\u2013305","journal-title":"Pattern Recognit Image Anal"},{"key":"1281_CR30","doi-asserted-by":"publisher","first-page":"1503","DOI":"10.1007\/s13042-018-0829-2","volume":"10","author":"W Xu","year":"2019","unstructured":"Xu W, Huang D, Zhou S (2019) Statistical learning with group invariance: problem, method and consistency. Int J Mach Learn Cybern 10:1503\u20131511","journal-title":"Int J Mach Learn Cybern"},{"issue":"4","key":"1281_CR31","first-page":"1063","volume":"9","author":"Y Xu","year":"2012","unstructured":"Xu Y, Xi W, Lv X, Guo R (2012) An improved least squares twin support vector machine. J Inf Comput Sci 9(4):1063\u20131071","journal-title":"J Inf Comput Sci"},{"key":"1281_CR32","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1016\/j.patcog.2017.09.035","volume":"74","author":"H Yan","year":"2018","unstructured":"Yan H, Ye Q, Zhang T, Yu DJ, Yuan X, Xu Y, Fu L (2018) Least squares twin bounded support vector machines based on L1-norm distance metric for classification. Pattern Recognit 74:434\u2013447","journal-title":"Pattern Recognit"},{"issue":"3","key":"1281_CR33","doi-asserted-by":"publisher","first-page":"739","DOI":"10.1006\/jcom.2002.0635","volume":"18","author":"D Zhou","year":"2002","unstructured":"Zhou D (2002) The covering number in learning theory. J Complexity 18(3):739\u2013767","journal-title":"J Complexity"}],"container-title":["International Journal of Machine Learning and Cybernetics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13042-021-01281-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s13042-021-01281-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13042-021-01281-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,10]],"date-time":"2021-06-10T04:24:54Z","timestamp":1623299094000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s13042-021-01281-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,14]]},"references-count":33,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["1281"],"URL":"https:\/\/doi.org\/10.1007\/s13042-021-01281-0","relation":{},"ISSN":["1868-8071","1868-808X"],"issn-type":[{"type":"print","value":"1868-8071"},{"type":"electronic","value":"1868-808X"}],"subject":[],"published":{"date-parts":[[2021,3,14]]},"assertion":[{"value":"7 November 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 January 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 March 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}