{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,4,9]],"date-time":"2025-04-09T09:51:41Z","timestamp":1744192301949,"version":"3.37.3"},"reference-count":40,"publisher":"Wiley","license":[{"start":{"date-parts":[[2021,5,29]],"date-time":"2021-05-29T00:00:00Z","timestamp":1622246400000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100007129","name":"Natural Science Foundation of Shandong Province","doi-asserted-by":"publisher","award":["ZR2020MF146","2019JZZY010716","2019GGX101061"],"award-info":[{"award-number":["ZR2020MF146","2019JZZY010716","2019GGX101061"]}],"id":[{"id":"10.13039\/501100007129","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100018532","name":"Major Scientific and Technological Innovation Project of Shandong Province","doi-asserted-by":"crossref","award":["ZR2020MF146","2019JZZY010716","2019GGX101061"],"award-info":[{"award-number":["ZR2020MF146","2019JZZY010716","2019GGX101061"]}],"id":[{"id":"10.13039\/501100018532","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Key R&D Plan of Shandong Province","award":["ZR2020MF146","2019JZZY010716","2019GGX101061"],"award-info":[{"award-number":["ZR2020MF146","2019JZZY010716","2019GGX101061"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Scientific Programming"],"published-print":{"date-parts":[[2021,5,29]]},"abstract":"<jats:p>Logistic regression has been widely used in artificial intelligence and machine learning due to its deep theoretical basis and good practical performance. Its training process aims to solve a large-scale optimization problem characterized by a likelihood function, where the gradient descent approach is the most commonly used. However, when the data size is large, it is very time-consuming because it computes the gradient using all the training data in every iteration. Though this difficulty can be solved by random sampling, the appropriate sampled examples size is difficult to be predetermined and the obtained could be not robust. To overcome this deficiency, we propose a novel algorithm for fast training logistic regression via adaptive sampling. The proposed method decomposes the problem of gradient estimation into several subproblems according to its dimension; then, each subproblem is solved independently by adaptive sampling. Each element of the gradient estimation is obtained by successively sampling a fixed volume training example multiple times until it satisfies its stopping criteria. The final estimation is combined with the results of all the subproblems. It is proved that the obtained gradient estimation is a robust estimation, and it could keep the objective function value decreasing in the iterative calculation. Compared with the representative algorithms using random sampling, the experimental results show that this algorithm obtains comparable classification performance with much less training time.<\/jats:p>","DOI":"10.1155\/2021\/9991859","type":"journal-article","created":{"date-parts":[[2021,5,31]],"date-time":"2021-05-31T01:16:11Z","timestamp":1622423771000},"page":"1-11","source":"Crossref","is-referenced-by-count":3,"title":["Fast Training Logistic Regression via Adaptive Sampling"],"prefix":"10.1155","volume":"2021","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3697-7134","authenticated-orcid":true,"given":"Yunsheng","family":"Song","sequence":"first","affiliation":[{"name":"College of Information Science and Engineering, Shandong Agricultural University, Tai\u2019an 271018, Shandong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1451-518X","authenticated-orcid":true,"given":"Xiaohan","family":"Kong","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Shandong Agricultural University, Tai\u2019an 271018, Shandong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7044-1261","authenticated-orcid":true,"given":"Shuoping","family":"Huang","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Shandong Agricultural University, Tai\u2019an 271018, Shandong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4481-7613","authenticated-orcid":true,"given":"Chao","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Shandong Agricultural University, Tai\u2019an 271018, Shandong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","reference":[{"key":"1","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-63913-0","volume-title":"An Introduction to Machine Learning","author":"M. Kubat","year":"2017"},{"volume-title":"Applied Logistic Regression Analysis","year":"2002","author":"M. Scott","key":"2"},{"key":"3","doi-asserted-by":"publisher","DOI":"10.3390\/s19153400"},{"issue":"2","key":"4","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1016\/j.ejor.2018.02.009","article-title":"A new hybrid classification algorithm for customer churn prediction based on logistic regression and decision trees","volume":"269","author":"A. De Caigny","year":"2018","journal-title":"European Journal of Operational Research"},{"issue":"11","key":"5","doi-asserted-by":"crossref","first-page":"1177","DOI":"10.1080\/10106049.2019.1588393","article-title":"Spatial prediction of landslide susceptibility by combining evidential belief function, logistic regression and logistic model tree","volume":"34","author":"W. Chen","year":"2019","journal-title":"Geocarto International"},{"key":"6","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2020.03.002"},{"key":"7","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2019.02.004"},{"key":"8","doi-asserted-by":"publisher","DOI":"10.3390\/app9010171"},{"key":"9","doi-asserted-by":"publisher","DOI":"10.1186\/s12920-018-0398-y"},{"issue":"21","key":"10","doi-asserted-by":"crossref","first-page":"4051","DOI":"10.1002\/sim.8281","article-title":"The integrated calibration index (ici) and related metrics for quantifying the calibration of logistic regression models","volume":"38","author":"P. C. Austin","year":"2019","journal-title":"Statistics in Medicine"},{"volume-title":"Statistical Learning Method","year":"2012","author":"H. Li","key":"11"},{"issue":"522","key":"12","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1080\/01621459.2017.1292914","article-title":"Optimal subsampling for large sample logistic regression","volume":"113","author":"H.Y. Wang","year":"2018","journal-title":"Journal of the American Statistical Association"},{"article-title":"An overview of gradient descent algorithm optimization in machine learning: application in the ophthalmology field","author":"A. Mustapha","key":"13","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-45183-7_27"},{"issue":"1","key":"14","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/s10107-007-0149-x","article-title":"Primal-dual subgradient methods for convex problems","volume":"120","author":"Y. Nesterov","year":"2009","journal-title":"Mathematical Programming"},{"article-title":"Distributed delayed stochastic optimization","author":"A. Agarwal","key":"15","doi-asserted-by":"crossref","DOI":"10.1109\/CDC.2012.6426626"},{"author":"G. Andrew","key":"16","article-title":"Scalable training of L1-regularized log-linear models"},{"article-title":"Accelerated training of conditional random fields with stochastic gradient methods","author":"S. V. N. Vishwanathan","key":"17","doi-asserted-by":"crossref","DOI":"10.1145\/1143844.1143966"},{"issue":"1","key":"18","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1137\/100808563","article-title":"Accelerated block-coordinate relaxation for regularized optimization","volume":"22","author":"S. Wright","year":"2012","journal-title":"SIAM Journal on Optimization"},{"issue":"2","key":"19","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1023\/A:1014056429969","article-title":"On issues of instance selection","volume":"6","author":"H. Liu","year":"2002","journal-title":"Data Mining and Knowledge Discovery"},{"issue":"1","key":"20","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/S0893-6080(98)00116-6","article-title":"On the momentum term in gradient descent learning algorithms","volume":"12","author":"N. Qian","year":"1999","journal-title":"Neural Networks"},{"key":"21","first-page":"2121","article-title":"Adaptive subgradient methods for online learning and stochastic optimization","volume":"12","author":"J. Duchi","year":"2011","journal-title":"Journal of Machine Learning Research"},{"author":"R. Ranganath","key":"22","article-title":"An adaptive learning rate for stochastic variational inference"},{"author":"D. Kingma","key":"23","article-title":"Adam: a method for stochastic optimization"},{"author":"Y.-A. Ma","key":"24","article-title":"A complete recipe for stochastic gradient mcmc"},{"issue":"1","key":"25","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1007\/s10107-012-0572-5","article-title":"Sample size selection in optimization methods for machine learning","volume":"134","author":"R. Byrd","year":"2012","journal-title":"Mathematical Programming"},{"issue":"3","key":"26","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1214\/aoms\/1177729586","article-title":"A stochastic approximation method","volume":"22","author":"H. Robbins","year":"1951","journal-title":"The Annals of Mathematical Statistics"},{"issue":"6","key":"27","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1007\/s11432-018-9832-y","article-title":"An accelerator for the logistic regression algorithm based on sampling on-demand","volume":"63","author":"J. Liang","year":"2020","journal-title":"Science China Information Sciences"},{"author":"S. Gopal","key":"28","article-title":"Adaptive sampling for sgd by exploiting side information"},{"volume-title":"Operations Research","year":"2013","author":"H. A. Taha","key":"29"},{"issue":"11","key":"30","doi-asserted-by":"crossref","first-page":"1134","DOI":"10.1145\/1968.1972","article-title":"A theory of the learnable","volume":"27","author":"L. Valiant","year":"1984","journal-title":"Communications of the ACM"},{"article-title":"Efficient progressive sampling","author":"F. Provost","key":"31","doi-asserted-by":"crossref","DOI":"10.1145\/312129.312188"},{"key":"32","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-05261-3","volume-title":"Probability Inequalities","author":"Z. Lin","year":"2011"},{"issue":"3","key":"33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1961189.1961199","article-title":"Libsvm: a library for support vector machines","volume":"2","author":"C.-C. Chang","year":"2011","journal-title":"ACM Transactions on Intelligent Systems and Technology"},{"author":"C.-J. Hsieh","key":"34","article-title":"A divide-and-conquer solver for kernel support vector machines"},{"issue":"6","key":"35","doi-asserted-by":"crossref","first-page":"80","DOI":"10.2307\/3001968","article-title":"Individual comparisons by ranking methods","volume":"1","author":"W. Frank","year":"1945","journal-title":"Biometrics Bulletin"},{"key":"36","first-page":"1","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","author":"J. Dem\u0161ar","year":"2006","journal-title":"Journal of Machine Learning Research"},{"key":"37","first-page":"101","article-title":"In defense of one-vs-all classification","volume":"5","author":"R. Ryan","year":"2004","journal-title":"Journal of Machine Learning Research"},{"issue":"5","key":"38","doi-asserted-by":"crossref","first-page":"882","DOI":"10.1109\/TEVC.2020.2968743","article-title":"Variable-size cooperative coevolutionary particle swarm optimization for feature selection on high-dimensional data","volume":"24","author":"X. Song","year":"2020","journal-title":"IEEE Transactions on Evolutionary Computation"},{"issue":"2","key":"39","doi-asserted-by":"crossref","first-page":"874","DOI":"10.1109\/TCYB.2020.3015756","article-title":"Multiobjective particle swarm optimization for feature selection with fuzzy cost","volume":"51","author":"Y. Hu","year":"2021","journal-title":"IEEE Transactions on Cybernetics"},{"key":"40","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2021.3061152"}],"container-title":["Scientific Programming"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/sp\/2021\/9991859.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/sp\/2021\/9991859.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/sp\/2021\/9991859.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,31]],"date-time":"2021-05-31T01:16:38Z","timestamp":1622423798000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.hindawi.com\/journals\/sp\/2021\/9991859\/"}},"subtitle":[],"editor":[{"given":"Pengwei","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2021,5,29]]},"references-count":40,"alternative-id":["9991859","9991859"],"URL":"https:\/\/doi.org\/10.1155\/2021\/9991859","relation":{},"ISSN":["1875-919X","1058-9244"],"issn-type":[{"type":"electronic","value":"1875-919X"},{"type":"print","value":"1058-9244"}],"subject":[],"published":{"date-parts":[[2021,5,29]]}}}