{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:43:42Z","timestamp":1760244222619,"version":"build-2065373602"},"reference-count":24,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,10,28]],"date-time":"2022-10-28T00:00:00Z","timestamp":1666915200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ontario Graduate Scholarship (OGS)","award":["08.1620.00000.814006"],"award-info":[{"award-number":["08.1620.00000.814006"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>This paper experiments with well known pruning approaches, iterative and one-shot, and presents a new approach to lottery ticket pruning applied to tabular neural networks based on iterative pruning. Our contribution is a standard model for comparison in terms of speed and performance for tabular datasets that often do not get optimized through research. We show leading results in several tabular datasets that can compete with ensemble approaches. We tested on a wide range of datasets with a general improvement over the original (already leading) model in 6 of 8 datasets tested in terms of F1\/RMSE. This includes a total reduction of over 85% of nodes with the additional ability to prune over 98% of nodes with minimal affect to accuracy. The new iterative approach we present will first optimize for lottery ticket quality by selecting an optimal architecture size and weights, then apply the iterative pruning strategy. The new iterative approach shows minimal degradation in accuracy compared to the original iterative approach, but it is capable of pruning models much smaller due to optimal weight pre-selection. Training and inference time improved over 50% and 10%, respectively, and up to 90% and 35%, respectively, for large datasets.<\/jats:p>","DOI":"10.3390\/make4040048","type":"journal-article","created":{"date-parts":[[2022,10,30]],"date-time":"2022-10-30T07:26:50Z","timestamp":1667114810000},"page":"954-967","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Lottery Ticket Structured Node Pruning for Tabular Datasets"],"prefix":"10.3390","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9777-0711","authenticated-orcid":false,"given":"Ryan","family":"Bluteau","sequence":"first","affiliation":[{"name":"Computer Science, University of Windsor, 401 Sunset Ave., Windsor, ON N9B 3P4, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0563-0250","authenticated-orcid":false,"given":"Robin","family":"Gras","sequence":"additional","affiliation":[{"name":"Computer Science, University of Windsor, 401 Sunset Ave., Windsor, ON N9B 3P4, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zachary","family":"Innes","sequence":"additional","affiliation":[{"name":"Computer Science, University of Windsor, 401 Sunset Ave., Windsor, ON N9B 3P4, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mitchel","family":"Paulin","sequence":"additional","affiliation":[{"name":"Computer Science, University of Windsor, 401 Sunset Ave., Windsor, ON N9B 3P4, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,28]]},"reference":[{"key":"ref_1","unstructured":"Frankle, J., and Carbin, M. (2019). The lottery ticket hypothesis: Finding sparse, trainable neural networks (2018). arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Howard, J., and Gugger, S. (2020). Fastai: A Layered API for Deep Learning, Information (2020). Information, 11, Available online: https:\/\/www.mdpi.com\/2078-2489\/11\/2\/108, https:\/\/github.com\/fastai\/fastai.","DOI":"10.3390\/info11020108"},{"key":"ref_3","unstructured":"Morcos, A.S., Yu, H., Paganini, M., and Tian, Y. (2019). One ticket to win them all: Generalizing lottery ticket initializations across datasets and optimizers. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Girish, S., Maiya, S.R., Gupta, K., Chen, H., Davis, L.S., and Shrivastava, A. (2021, January 20\u201325). The lottery ticket hypothesis for object recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00082"},{"key":"ref_5","unstructured":"Chen, T., Sui, Y., Chen, X., Zhang, A., and Wang, Z. (2021, January 18\u201324). A unified lottery ticket hypothesis for graph neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Virtual."},{"key":"ref_6","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_7","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_8","unstructured":"Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020). Language models are few-shot learners. arXiv."},{"key":"ref_9","unstructured":"Chen, T., Frankle, J., Chang, S., Liu, S., Zhang, Y., Wang, Z., and Carbin, M. (2020). The lottery ticket hypothesis for pre-trained bert networks. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Padhi, I., Schiff, Y., Melnyk, I., Rigotti, M., Mroueh, Y., Dognin, P., Ross, J., Nair, R., and Altman, E. (2021, January 6\u201311). Tabular transformers for modeling multivariate time series. Proceedings of the ICASSP 2021\u20142021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.","DOI":"10.1109\/ICASSP39728.2021.9414142"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Gu, K., and Budhkar, A. (2021, January 6). A Package for Learning on Tabular and Text Data with Transformers. Proceedings of the Third Workshop on Multimodal Artificial Intelligence, Mexico City, Mexico.","DOI":"10.18653\/v1\/2021.maiworkshop-1.10"},{"key":"ref_12","unstructured":"Huang, X., Khetan, A., Cvitkovic, M., and Karnin, Z. (2020). Tabtransformer: Tabular data modeling using contextual embeddings. arXiv."},{"key":"ref_13","unstructured":"Cortez, P., and Silva, A.M.G. (2022, October 26). Using Data Mining to Predict Secondary School Student Performance 2008. Available online: https:\/\/hdl.handle.net\/1822\/8024."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e1403","DOI":"10.1002\/widm.1403","article-title":"Review on publicly available datasets for educational data mining","volume":"11","author":"Mihaescu","year":"2021","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1016\/j.dss.2009.05.016","article-title":"Modeling wine preferences by data mining from physicochemical properties","volume":"47","author":"Cortez","year":"2009","journal-title":"Decis. Support Syst."},{"key":"ref_16","first-page":"296","article-title":"Evolutionary data mining with automatic rule generalization","volume":"1","author":"Cattral","year":"2002","journal-title":"Recent Adv. Comput. Comput. Commun."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4308","DOI":"10.1038\/ncomms5308","article-title":"Searching for exotic particles in high-energy physics with deep learning","volume":"5","author":"Baldi","year":"2014","journal-title":"Nat. Commun."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"278","DOI":"10.4236\/ojs.2021.112015","article-title":"Prediction of Wine Quality Using Machine Learning Algorithms","volume":"11","author":"Dahal","year":"2021","journal-title":"Open J. Stat."},{"key":"ref_19","unstructured":"\u015eekero\u011flu, A.G. (2022, October 26). Impacts of Feature Selection Techniques in Machine Learning Algorithms for Cross Selling: A Comprehensive Study for Insurance Industry 2021. Available online: https:\/\/www.researchgate.net\/publication\/353072980_Impacts_of_Feature_Selection_Techniques_in_Machine_Learning_Algorithms_for_Cross_Selling_A_Comprehensive_Study_for_Insurance_Industry\/."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"6669706","DOI":"10.1155\/2021\/6669706","article-title":"An Adaptive Heterogeneous Online Learning Ensemble Classifier for Nonstationary Environments","volume":"2021","author":"Museba","year":"2021","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"619","DOI":"10.1109\/TKDE.2011.58","article-title":"DDD: A new ensemble approach for dealing with concept drift","volume":"24","author":"Minku","year":"2011","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.ins.2013.12.011","article-title":"Combining block-based and online methods in learning ensembles from concept drifting data streams","volume":"265","author":"Brzezinski","year":"2014","journal-title":"Inf. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"438","DOI":"10.2991\/ijcis.11.1.33","article-title":"Active fuzzy weighting ensemble for dealing with concept drift","volume":"11","author":"Dong","year":"2018","journal-title":"Int. J. Comput. Intell. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Azhari, M., Abarda, A., Ettaki, B., Zerouaoui, J., and Dakkon, M. (2021). Using Machine Learning with PySpark and MLib for Solving a Binary Classification Problem: Case of Searching for Exotic Particles. Recent Advances in Intuitionistic Fuzzy Logic Systems and Mathematics, Springer.","DOI":"10.1007\/978-3-030-53929-0_8"}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/4\/4\/48\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:04:36Z","timestamp":1760144676000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/4\/4\/48"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,28]]},"references-count":24,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["make4040048"],"URL":"https:\/\/doi.org\/10.3390\/make4040048","relation":{},"ISSN":["2504-4990"],"issn-type":[{"type":"electronic","value":"2504-4990"}],"subject":[],"published":{"date-parts":[[2022,10,28]]}}}