{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T01:47:54Z","timestamp":1787017674547,"version":"build-2736575974"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2025,12,4]]},"abstract":"<jats:p>Structured data prediction supports key applications in healthcare, finance, e-commerce, among others. As structured data becomes increasingly heterogeneous and complex, there is a growing need for scalable models that can generalize across diverse tables. Traditional machine learning and deep learning models for structured data often have task-specific architectures requiring computationally retraining for a new prediction task. Consequently, recent studies explore pre-training for structured data, which requires no additional training for a new task. While pre-training has achieved great success in NLP and CV, the research of pre-training for structured data remains preliminary because structured data has its unique characteristics, e.g., complicated correlations and dependencies between features. Existing studies adopt a fixed pre-trained model architecture and less attention has been paid to table-specific characteristics. This limits their ability to adapt to new prediction tasks. To address this challenge, we propose AixelNet, a pre-trained model for structured data prediction that supports table-aware adaptation. We first design a meta feature extraction module that summarizes table-level characteristics enabling model customization. Instead of relying on a single model, AixelNet adopts a multi-model framework with multiple base predictors to capture diverse feature interaction patterns across tables. Finally, these base predictor models are dynamically assigned to different tables for prediction based on the extracted table meta features via a designed hypernetwork, enabling flexible and table-aware model composition for different tables. To further improve generalization and efficiency, we design regularization methods to encourage balanced predictor model usage and diversified learned representations, as well as design a sparse update strategy for sparsely updating relevant predictor models during pre-training. Extensive experiments on 20 classification and 20 regression tasks on tables confirm AixelNet's effectiveness in improving predictive performance compared to eight state-of-the-art baselines and demonstrate its efficiency.<\/jats:p>","DOI":"10.1145\/3769814","type":"journal-article","created":{"date-parts":[[2025,12,6]],"date-time":"2025-12-06T04:32:13Z","timestamp":1764995533000},"page":"1-26","source":"Crossref","is-referenced-by-count":2,"title":["AixelNet: A Pre-trained Model with Table-aware Adaptation for Structured Data Prediction"],"prefix":"10.1145","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0109-5668","authenticated-orcid":false,"given":"Liming","family":"Wang","sequence":"first","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0752-9877","authenticated-orcid":false,"given":"Meihui","family":"Zhang","sequence":"additional","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7271-3999","authenticated-orcid":false,"given":"Zhaojing","family":"Luo","sequence":"additional","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,12,5]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"1","article-title":"MFE: Towards reproducible meta-feature extraction","volume":"21","author":"Alcoba\u00e7a Edesio","year":"2020","unstructured":"Edesio Alcoba\u00e7a, Felipe Siqueira, Adriano Rivolli, Lu\u00eds PF Garcia, Jefferson T Oliva, and Andr\u00e9 CPLF De Carvalho. 2020. MFE: Towards reproducible meta-feature extraction. Journal of Machine Learning Research, Vol. 21, 111 (2020), 1-5.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_2_1","volume-title":"International conference on machine learning. 199-207","author":"Bardenet R\u00e9mi","year":"2013","unstructured":"R\u00e9mi Bardenet, M\u00e1ty\u00e1s Brendel, Bal\u00e1zs K\u00e9gl, and Michele Sebag. 2013. Collaborative hyperparameter tuning. In International conference on machine learning. 199-207."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457321"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588700"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.14778\/3685800.3685880"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.14778\/3523210.3523223"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3639329"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.52202\/079017-2368"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467228"},{"key":"e_1_2_1_11_1","volume-title":"Autogluon-tabular: Robust and accurate automl for structured data. arXiv preprint arXiv:2003.06505","author":"Erickson Nick","year":"2020","unstructured":"Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, and Alexander Smola. 2020. Autogluon-tabular: Robust and accurate automl for structured data. arXiv preprint arXiv:2003.06505 (2020)."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3555041.3589409"},{"key":"e_1_2_1_13_1","first-page":"1","article-title":"Auto-sklearn 2.0: hands-free AutoML via meta-learning","volume":"23","author":"Feurer Matthias","year":"2022","unstructured":"Matthias Feurer, Katharina Eggensperger, Stefan Falkner, Marius Lindauer, and Frank Hutter. 2022. Auto-sklearn 2.0: hands-free AutoML via meta-learning. Journal of Machine Learning Research, Vol. 23, 261 (2022), 1-61.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.14778\/3583140.3583165"},{"key":"e_1_2_1_15_1","unstructured":"Google DeepMind. 2025. Gemma 3 12B Instruct. https:\/\/huggingface.co\/google\/gemma-3-12b-it."},{"key":"e_1_2_1_16_1","first-page":"18932","article-title":"Revisiting Deep Learning Models for Tabular Data","author":"Gorishniy Yury","year":"2021","unstructured":"Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. 2021. Revisiting Deep Learning Models for Tabular Data. In Advances in Neural Information Processing Systems. 18932-18943.","journal-title":"Advances in Neural Information Processing Systems."},{"key":"e_1_2_1_17_1","doi-asserted-by":"crossref","unstructured":"L\u00e9o Grinsztajn Edouard Oyallon and Ga\u00ebl Varoquaux. 2022. Why do tree-based models still outperform deep learning on typical tabular data?. In Advances in Neural Information Processing Systems 35.","DOI":"10.52202\/068431-0037"},{"key":"e_1_2_1_18_1","volume-title":"International Conference on Artificial Intelligence and Statistics. 5549-5581","author":"Hegselmann Stefan","year":"2023","unstructured":"Stefan Hegselmann, Alejandro Buendia, Hunter Lang, Monica Agrawal, Xiaoyi Jiang, and David Sontag. 2023. Tabllm: Few-shot classification of tabular data with large language models. In International Conference on Artificial Intelligence and Statistics. 5549-5581."},{"key":"e_1_2_1_19_1","volume-title":"The Eleventh International Conference on Learning Representations.","author":"Hollmann Noah","year":"2023","unstructured":"Noah Hollmann, Samuel M\u00fcller, Katharina Eggensperger, and Frank Hutter. 2023. TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second. In The Eleventh International Conference on Learning Representations."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588918"},{"key":"e_1_2_1_21_1","first-page":"3149","article-title":"LightGBM: a highly efficient gradient boosting decision tree","author":"Ke Guolin","year":"2017","unstructured":"Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. LightGBM: a highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems. 3149-3157.","journal-title":"Advances in Neural Information Processing Systems."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589270"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3517836"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588698"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2019.2916683"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i10.17067"},{"key":"e_1_2_1_27_1","volume-title":"Adaptive Lightweight Regularization Tool for Complex Analytics. In 2018 IEEE 34th International Conference on Data Engineering. 485-496","author":"Luo Zhaojing","year":"2018","unstructured":"Zhaojing Luo, Shaofeng Cai, Jinyang Gao, Meihui Zhang, Kee Yuan Ngiam, Gang Chen, and Wang-Chien Lee. 2018. Adaptive Lightweight Regularization Tool for Complex Analytics. In 2018 IEEE 34th International Conference on Data Engineering. 485-496."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588936"},{"key":"e_1_2_1_29_1","volume-title":"MLCask: Efficient Management of Component Evolution in Collaborative Data Analytics Pipelines. In 37th IEEE International Conference on Data Engineering. IEEE, 1655-1666","author":"Luo Zhaojing","year":"2021","unstructured":"Zhaojing Luo, Sai Ho Yeung, Meihui Zhang, Kaiping Zheng, Lei Zhu, Gang Chen, Feiyi Fan, Qian Lin, Kee Yuan Ngiam, and Beng Chin Ooi. 2021c. MLCask: Efficient Management of Component Evolution in Collaborative Data Analytics Pipelines. In 37th IEEE International Conference on Data Engineering. IEEE, 1655-1666."},{"key":"e_1_2_1_30_1","unstructured":"Meta AI. 2025. Llama 4 Maverick 17B 128E Instruct. https:\/\/huggingface.co\/meta-llama\/Llama-4-Maverick-17B-128E-Instruct."},{"key":"e_1_2_1_31_1","volume-title":"The Twelfth International Conference on Learning Representations.","author":"Puigcerver Joan","year":"2024","unstructured":"Joan Puigcerver, Carlos Riquelme Ruiz, Basil Mustafa, and Neil Houlsby. 2024. From Sparse to Soft Mixtures of Experts. In The Twelfth International Conference on Learning Representations."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467216"},{"key":"e_1_2_1_33_1","volume-title":"The Tenth International Conference on Learning Representations.","author":"Rakotoarison Herilalaina","year":"2022","unstructured":"Herilalaina Rakotoarison, Louisot Milijaona, Andry Rasoanaivo, Mich\u00e8le Sebag, and Marc Schoenauer. 2022. Learning meta-features for automl. In The Tenth International Conference on Learning Representations."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.14778\/3611479.3611525"},{"key":"e_1_2_1_35_1","volume-title":"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In International Conference on Learning Representations.","author":"Shazeer Noam","year":"2017","unstructured":"Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In International Conference on Learning Representations."},{"key":"e_1_2_1_36_1","volume-title":"Saint: Improved neural networks for tabular data via row attention and contrastive pre-training. arXiv preprint arXiv:2106.01342","author":"Somepalli Gowthami","year":"2021","unstructured":"Gowthami Somepalli, Micah Goldblum, Avi Schwarzschild, C Bayan Bruss, and Tom Goldstein. 2021. Saint: Improved neural networks for tabular data via row attention and contrastive pre-training. arXiv preprint arXiv:2106.01342 (2021)."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3357925"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2023.3310478"},{"key":"e_1_2_1_39_1","volume-title":"UniPredict: Large Language Models are Universal Tabular Classifiers. arXiv preprint arXiv:2310.03266","author":"Wang Ruiyu","year":"2023","unstructured":"Ruiyu Wang, Zifeng Wang, and Jimeng Sun. 2023b. UniPredict: Large Language Models are Universal Tabular Classifiers. arXiv preprint arXiv:2310.03266 (2023)."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3715321"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467066"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.14778\/3641204.3641212"},{"key":"e_1_2_1_43_1","volume-title":"Making Pre-trained Language Models Great on Tabular Prediction. In The Twelfth International Conference on Learning Representations.","author":"Yan Jiahuan","year":"2024","unstructured":"Jiahuan Yan, Bo Zheng, Hongxia Xu, Yiheng Zhu, Danny Z. Chen, Jimeng Sun, Jian Wu, and Jintai Chen. 2024. Making Pre-trained Language Models Great on Tabular Prediction. In The Twelfth International Conference on Learning Representations."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2021.3069861"},{"key":"e_1_2_1_45_1","volume-title":"UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Science. In The Twelfth International Conference on Learning Representations.","author":"Yang Yazheng","year":"2024","unstructured":"Yazheng Yang, Yuqi Wang, Guang Liu, Ledell Wu, and Qi Liu. 2024. UniTabE: A Universal Pretraining Protocol for Tabular Foundation Model in Data Science. In The Twelfth International Conference on Learning Representations."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589334.3645707"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2012.2200299"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476311.3476404"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.14778\/3704965.3704985"},{"key":"e_1_2_1_50_1","volume-title":"Applications and Challenges for Large Language Models: From Data Management Perspective. In 2024 IEEE 40th International Conference on Data Engineering. 5530-5541","author":"Zhang Meihui","year":"2024","unstructured":"Meihui Zhang, Zhaoxuan Ji, Zhaojing Luo, Yuncheng Wu, and Chengliang Chai. 2024. Applications and Challenges for Large Language Models: From Data Management Perspective. In 2024 IEEE 40th International Conference on Data Engineering. 5530-5541."},{"key":"e_1_2_1_51_1","volume-title":"Aixel: A Unified, Adaptive and Extensible System for AI-powered Data Analysis. arXiv preprint arXiv:2510.12642","author":"Zhang Meihui","year":"2025","unstructured":"Meihui Zhang, Liming Wang, Chi Zhang, and Zhaojing Luo. 2025. Aixel: A Unified, Adaptive and Extensible System for AI-powered Data Analysis. arXiv preprint arXiv:2510.12642 (2025)."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457281"},{"key":"e_1_2_1_53_1","first-page":"43181","volume-title":"XTab: Cross-table Pretraining for Tabular Transformers. In International Conference on Machine Learning","volume":"202","author":"Zhu Bingzhao","year":"2023","unstructured":"Bingzhao Zhu, Xingjian Shi, Nick Erickson, Mu Li, George Karypis, and Mahsa Shoaran. 2023. XTab: Cross-table Pretraining for Tabular Transformers. In International Conference on Machine Learning, Vol. 202. 43181-43204."}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3769814","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T04:57:32Z","timestamp":1781326652000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3769814"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,4]]},"references-count":53,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,12,4]]}},"alternative-id":["10.1145\/3769814"],"URL":"https:\/\/doi.org\/10.1145\/3769814","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,4]]}}}