{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T00:41:52Z","timestamp":1759970512996,"version":"build-2065373602"},"reference-count":61,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T00:00:00Z","timestamp":1737417600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["31900710","62403405","31600862","25A520020"],"award-info":[{"award-number":["31900710","62403405","31600862","25A520020"]}]},{"name":"Key Scientific Research Projects of Higher Education Institutions in Henan Province","award":["31900710","62403405","31600862","25A520020"],"award-info":[{"award-number":["31900710","62403405","31600862","25A520020"]}]},{"name":"Nanhu Scholars Program for Young Scholars of Xinyang Normal University","award":["31900710","62403405","31600862","25A520020"],"award-info":[{"award-number":["31900710","62403405","31600862","25A520020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>This paper introduces logistic regression with sparse and smooth regularizations (LR-SS), a novel framework that simultaneously enhances both classification and feature extraction capabilities of standard logistic regression. By incorporating a family of symmetric smoothness constraints into sparse logistic regression, LR-SS uniquely preserves underlying structures inherent in structured data, distinguishing it from existing approaches. Within the minorization\u2013maximization (MM) framework, we develop an efficient optimization algorithm that combines coordinate descent with soft-thresholding techniques. Through extensive experiments on both simulated and real-world datasets, including time series and image data, we demonstrate that LR-SS significantly outperforms conventional sparse logistic regression in classification tasks while providing more interpretable feature extraction. The results highlight LR-SS\u2019s ability to leverage sparse and symmetric smooth regularizations for capturing intrinsic data structures, making it particularly valuable for machine learning applications requiring both predictive accuracy and model interpretability.<\/jats:p>","DOI":"10.3390\/sym17020151","type":"journal-article","created":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T06:53:32Z","timestamp":1737442412000},"page":"151","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Incorporating Symmetric Smooth Regularizations into Sparse Logistic Regression for Classification and Feature Extraction"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3286-073X","authenticated-orcid":false,"given":"Jing","family":"Wang","sequence":"first","affiliation":[{"name":"School of Computer and Information Technology, Xinyang Normal University, Xinyang 464000, China"},{"name":"Henan Key Laboratory of Analysis and Applications of Education Big Data, Xinyang Normal University, Xinyang 464000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiao","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Xinyang Normal University, Xinyang 464000, China"},{"name":"Henan Key Laboratory of Analysis and Applications of Education Big Data, Xinyang Normal University, Xinyang 464000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pengwei","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Xinyang Normal University, Xinyang 464000, China"},{"name":"Henan Key Laboratory of Analysis and Applications of Education Big Data, Xinyang Normal University, Xinyang 464000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jian","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Xinyang Normal University, Xinyang 464000, China"},{"name":"Henan Key Laboratory of Analysis and Applications of Education Big Data, Xinyang Normal University, Xinyang 464000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yaochen","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Xinyang Normal University, Xinyang 464000, China"},{"name":"Henan Key Laboratory of Analysis and Applications of Education Big Data, Xinyang Normal University, Xinyang 464000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Li","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Early-Childhood Education, Nanjing Xiaozhuang University, Nanjing 211171, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,1,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Hosmer, D.W., Lemeshow, S., and Sturdivant, R.X. (2013). Applied Logistic Regression, John Wiley & Sons.","DOI":"10.1002\/9781118548387"},{"key":"ref_2","first-page":"1255","article-title":"A Predictive Model for Liver Disease Progression Based on Logistic Regression Algorithm","volume":"7","author":"Abdalrada","year":"2019","journal-title":"Period. Eng. Nat. Sci. (PEN)"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"S574","DOI":"10.21037\/jtd.2019.01.25","article-title":"Developing Prediction Models for Clinical Use Using Logistic Regression: An Overview","volume":"11","author":"Shipe","year":"2019","journal-title":"J. Thorac. Dis."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.jclinepi.2020.12.018","article-title":"Logistic Regression and Machine Learning Predicted Patient Mortality from Large Sets of Diagnosis Codes Comparably","volume":"133","author":"Cowling","year":"2021","journal-title":"J. Clin. Epidemiol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1111\/j.1540-5907.2009.00426.x","article-title":"A Global Model for Forecasting Political Instability","volume":"54","author":"Goldstone","year":"2010","journal-title":"Am. J. Political Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"104967","DOI":"10.1016\/j.ssci.2020.104967","article-title":"Risk Assessment by Failure Mode and Effects Analysis (FMEA) Using an Interval Number Based Logistic Regression Model","volume":"132","author":"Bhattacharjee","year":"2020","journal-title":"Saf. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kemive\u0161, A., Ran\u0111elovi\u0107, M., Barjaktarovi\u0107, L., \u0110ikanovi\u0107, P., \u010cabarkapa, M., and Ran\u0111elovi\u0107, D. (2024). Identifying Key Indicators for Successful Foreign Direct Investment through Asymmetric Optimization Using Machine Learning. Symmetry, 16.","DOI":"10.3390\/sym16101346"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1561\/2200000013","article-title":"An Introduction to Conditional Random Fields","volume":"4","author":"Sutton","year":"2012","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1109\/TPAMI.2005.127","article-title":"Sparse Multinomial Logistic Regression: Fast Algorithms and Generalization Bounds","volume":"27","author":"Krishnapuram","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_10","unstructured":"Liu, J., Chen, J., and Ye, J. (July, January 28). Large-scale Sparse Logistic Regression. Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France."},{"key":"ref_11","unstructured":"Mohammadi, M., Atashin, A.A., and Tamburri, D.A. (2021). An Efficient Projection Neural Network for \u21131-Regularized Logistic Regression. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2246","DOI":"10.1093\/bioinformatics\/btg308","article-title":"A Simple and Efficient Algorithm for Gene Selection Using Sparse Logistic Regression","volume":"19","author":"Shevade","year":"2003","journal-title":"Bioinformatics"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"i39","DOI":"10.1093\/bioinformatics\/btae243","article-title":"Scalable De Novo Classification of Antibiotic Resistance of Mycobacterium Tuberculosis","volume":"40","author":"Serajian","year":"2024","journal-title":"Bioinformatics"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1016\/j.neuroimage.2009.02.041","article-title":"Interpreting Single Trial Data Using Groupwise Regularisation","volume":"46","author":"Hesse","year":"2009","journal-title":"NeuroImage"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"752","DOI":"10.1016\/j.neuroimage.2010.02.040","article-title":"Sparse Logistic Regression for Whole-Brain Classification of fMRI Data","volume":"51","author":"Ryali","year":"2010","journal-title":"NeuroImage"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"955","DOI":"10.1002\/sim.8454","article-title":"Structured Sparse Logistic Regression with Application to Lung Cancer Prediction Using Breath Volatile Biomarkers","volume":"39","author":"Zhang","year":"2020","journal-title":"Stat. Med."},{"key":"ref_17","unstructured":"Xu, Y., Du, P., Robertson, J., and Senger, R. (2021). Sparse Logistic Regression on Functional Data. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Klimaszewski, J., Sklyar, M., and Korze\u0144, M. (2017, January 3\u20135). Learning \u21131-Penalized Logistic Regressions with Smooth Approximation. Proceedings of the 2017 IEEE International Conference on INnovations in Intelligent SysTems and Applications (INISTA), Gdynia, Poland.","DOI":"10.1109\/INISTA.2017.8001144"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1016\/j.neuroimage.2012.12.062","article-title":"Interpretable Whole-Brain Prediction Analysis with GraphNet","volume":"72","author":"Grosenick","year":"2013","journal-title":"NeuroImage"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1550","DOI":"10.1016\/j.neuroimage.2011.12.085","article-title":"Combining Sparseness and Smoothness Improves Classification Accuracy and Interpretability","volume":"60","author":"Yamagishi","year":"2012","journal-title":"NeuroImage"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1016\/j.neuroimage.2014.03.067","article-title":"Disease Prediction Based on Functional Connectomes Using a Scalable and Spatially-Informed Support Vector Machine","volume":"96","author":"Watanabe","year":"2014","journal-title":"Neuroimage"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1639","DOI":"10.1109\/TBME.2017.2756665","article-title":"Euler Elastica Regularized Logistic Regression for Whole-Brain Decoding of fMRI Data","volume":"65","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1016\/j.neuroimage.2018.09.031","article-title":"Grouped Sparse Bayesian Learning for Voxel Selection in Multivoxel Pattern Analysis of fMRI Data","volume":"184","author":"Wen","year":"2019","journal-title":"NeuroImage"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1016\/j.neucom.2015.03.112","article-title":"Regression and Classification Using Extreme Learning Machine Based on L1-Norm and L2-Norm","volume":"174","author":"Luo","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1007\/s11075-011-9498-x","article-title":"A New Tikhonov Regularization Method","volume":"59","author":"Fuhry","year":"2012","journal-title":"Numer. Algorithms"},{"key":"ref_26","first-page":"1519","article-title":"An Interior-Point Method for Large-Scale \u21131-Regularized Logistic Regression","volume":"8","author":"Koh","year":"2007","journal-title":"J. Mach. Learn. Res."},{"key":"ref_27","unstructured":"Lee, S.I., Lee, H., Abbeel, P., and Ng, A.Y. (2006, January 16\u201320). Efficient L1 Regularized Logistic Regression. Proceedings of the 21st AAAI Conference on Artificial Intelligence, Boston, MA, USA."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression Shrinkage and Selection Via the Lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J. R. Stat. Soc. Ser. B Stat. Methodol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and Variable Selection Via the Elastic Net","volume":"67","author":"Zou","year":"2005","journal-title":"J. R. Stat. Soc. Ser. B Stat. Methodol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Malbasa, V., and Vucetic, S. (2011, January 21\u201324). Spatially Regularized Logistic Regression for Disease Mapping on Large Moving Populations. Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA.","DOI":"10.1145\/2020408.2020609"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Rao, A., Lee, Y., Gass, A., and Monsch, A. (September, January 30). Classification of Alzheimer\u2019s Disease from Structural MRI Using Sparse Logistic Regression with Optional Spatial Regularization. Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA.","DOI":"10.1109\/IEMBS.2011.6091115"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1436","DOI":"10.1214\/009053606000000281","article-title":"High-Dimensional Graphs and Variable Selection with the Lasso","volume":"34","author":"Meinshausen","year":"2006","journal-title":"Ann. Statist."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1093\/biostatistics\/kxm045","article-title":"Sparse Inverse Covariance Estimation with the Graphical Lasso","volume":"9","author":"Friedman","year":"2008","journal-title":"Biostatistics"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1137\/0611030","article-title":"Partitioning Sparse Matrices with Eigenvectors of Graphs","volume":"11","author":"Pothen","year":"1990","journal-title":"SIAM J. Matrix Anal. Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1373","DOI":"10.1162\/089976603321780317","article-title":"Laplacian Eigenmaps for Dimensionality Reduction and Data Representation","volume":"15","author":"Belkin","year":"2003","journal-title":"Neural Comput."},{"key":"ref_36","first-page":"6509","article-title":"NeuroGraph: Benchmarks for Graph Machine Learning in Brain Connectomics","volume":"36","author":"Said","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"102233","DOI":"10.1016\/j.media.2021.102233","article-title":"BrainGNN: Interpretable Brain Graph Neural Network for fMRI Analysis","volume":"74","author":"Li","year":"2021","journal-title":"Med. Image Anal."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Yan, Y., Zhu, J., Duda, M., Solarz, E., Sripada, C., and Koutra, D. (2019, January 4\u20138). GroupINN: Grouping-based Interpretable Neural Network for Classification of Limited, Noisy Brain Data. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330921"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1109\/TMI.2022.3218745","article-title":"BrainGB: A Benchmark for Brain Network Analysis with Graph Neural Networks","volume":"42","author":"Cui","year":"2022","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/0167-2789(92)90242-F","article-title":"Nonlinear Total Variation Based Noise Removal Algorithms","volume":"60","author":"Rudin","year":"1992","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1111\/j.1467-9868.2005.00490.x","article-title":"Sparsity and Smoothness via the Fused Lasso","volume":"67","author":"Tibshirani","year":"2005","journal-title":"J. R. Stat. Soc. Ser. B Stat. Methodol."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Dohmatob, E.D., Gramfort, A., Thirion, B., and Varoquaux, G. (2014, January 4\u20136). Benchmarking Solvers for TV-\u21131 Least-Squares and Logistic Regression in Brain Imaging. Proceedings of the 2014 International Workshop on Pattern Recognition in Neuroimaging, Tubingen, Germany.","DOI":"10.1109\/PRNI.2014.6858516"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1198\/0003130042836","article-title":"A Tutorial on MM Algorithms","volume":"58","author":"Hunter","year":"2004","journal-title":"Am. Stat."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Lange, K. (2016). MM Optimization Algorithms, SIAM.","DOI":"10.1137\/1.9781611974409"},{"key":"ref_45","first-page":"1871","article-title":"LIBLINEAR: A Library for Large Linear Classification","volume":"9","author":"Fan","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v033.i01","article-title":"Regularization Paths for Generalized Linear Models via Coordinate Descent","volume":"33","author":"Friedman","year":"2010","journal-title":"J. Stat. Softw."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1109\/18.382009","article-title":"De-noising by Soft-Thresholding","volume":"41","author":"Donoho","year":"1995","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1016\/S0895-6111(00)00026-4","article-title":"Digital Hand Atlas and Web-based Bone Age Assessment: System Design and Implementation","volume":"24","author":"Cao","year":"2000","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_49","unstructured":"Davis, L.M. (2013). Predictive Modelling of Bone Ageing. [Ph.D. Thesis, University of East Anglia]."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Ratanamahatana, C.A., and Keogh, E. (2005, January 21\u201323). Three Myths about Dynamic Time Warping Data Mining. Proceedings of the 2005 SIAM International Conference on Data Mining, Newport Beach, CA, USA.","DOI":"10.1137\/1.9781611972757.50"},{"key":"ref_51","unstructured":"Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-MNIST: A Novel Image Dataset for Benchmarking Machine Learning Algorithms. arXiv."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based Learning Applied to Document Recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_53","unstructured":"Snoek, J., Larochelle, H., and Adams, R.P. (2012, January 3\u20136). Practical Bayesian Optimization of Machine Learning Algorithms. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1109\/JPROC.2015.2494218","article-title":"Taking the Human out of the Loop: A Review of Bayesian Optimization","volume":"104","author":"Shahriari","year":"2015","journal-title":"Proc. IEEE"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"116659","DOI":"10.1016\/j.eswa.2022.116659","article-title":"Machine Learning Techniques and Data for Stock Market Forecasting: A Literature Review","volume":"197","author":"Kumbure","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_56","first-page":"19","article-title":"An Explanation Framework for Interpretable Credit Scoring","volume":"12","author":"Demajo","year":"2021","journal-title":"Int. J. Artif. Intell. Appl. (IJAIA)"},{"key":"ref_57","unstructured":"Mitchell, T.M., and Mitchell, T.M. (1997). Machine Learning, McGraw-Hill."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"792","DOI":"10.1109\/TCYB.2015.2416274","article-title":"Generalized 2-D principal component analysis by Lp-norm for image analysis","volume":"46","author":"Wang","year":"2015","journal-title":"IEEE Trans. Cybern."},{"key":"ref_59","first-page":"104","article-title":"Generalized Representation-based Classification by Lp-norm for Face Recognition","volume":"51","author":"Wang","year":"2024","journal-title":"IAENG Int. J. Comput. Sci."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Tang, X., Liu, S., Nian, X., Deng, S., Liu, Y., Ye, Q., Li, Y., Li, Y., Yuan, T., and Sun, H. (2024). Improved adaptive regularization for simulated annealing inversion of transient electromagnetic. Sci. Rep., 14.","DOI":"10.1038\/s41598-024-55710-5"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"126196","DOI":"10.1016\/j.eswa.2024.126196","article-title":"Transportation Mode Detection through Spatial Attention-based Transductive Long Short-Term Memory and Off-Policy Feature Selection","volume":"267","author":"Merikhipour","year":"2025","journal-title":"Expert Syst. Appl."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/2\/151\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T10:32:56Z","timestamp":1759919576000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/2\/151"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,21]]},"references-count":61,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["sym17020151"],"URL":"https:\/\/doi.org\/10.3390\/sym17020151","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2025,1,21]]}}}