{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T15:10:20Z","timestamp":1755789020293,"version":"3.44.0"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,5,29]],"date-time":"2024-05-29T00:00:00Z","timestamp":1716940800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2024,5,29]]},"abstract":"<jats:p>Feature importance scores (FIS) estimation is an important problem in many data-intensive applications. Traditional approaches can be divided into two types; model-specific methods and model-agnostic methods. In this work, we present FeatureLTE, a novel learning-based approach to FIS estimation. For the first time, as we demonstrate through extensive experiments, it is possible to build general-purpose pre-trained models for FIS estimation. Therefore, FIS estimation reduces to prediction outputs from a pre-trained FeatureLTE model. Pre-trained FeatureLTE models enjoy several desired advantages, including accuracy, robustness, efficiency, and evolvability, and FeatureLTE models really begin to shine on large datasets where traditional methods often find themselves unable to scale. We build our pre-trained models for binary classification and regression problems using observations from nearly 1,000 public datasets. We systematically evaluate various design choices of FeatureLTE model construction and carefully design meta features to make sure that they are computationally lightweight. Based on our evaluation, FeatureLTE is on par with the best existing FIS estimators in terms of FIS quality, and achieves up to 339.48x speedup without sacrificing the quality of FIS estimates on large-scale datasets. Finally, we release two pre-trained FeatureLTE models for binary classification and regression problems that are ready to use on almost all tabular datasets, along with the repository of 701 binary classification datasets and 256 regression datasets with pre-computed feature importance scores to promote future research along this direction.<\/jats:p>","DOI":"10.1145\/3654942","type":"journal-article","created":{"date-parts":[[2024,5,30]],"date-time":"2024-05-30T09:44:53Z","timestamp":1717062293000},"page":"1-19","source":"Crossref","is-referenced-by-count":0,"title":["FeatureLTE: Learning to Estimate Feature Importance"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3876-9379","authenticated-orcid":false,"given":"Tianping","family":"Zhang","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-3219-8375","authenticated-orcid":false,"given":"Zhaoyang","family":"Wang","sequence":"additional","affiliation":[{"name":"Ant Group, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-8925-2787","authenticated-orcid":false,"given":"Chen","family":"Qian","sequence":"additional","affiliation":[{"name":"Ant Group, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4650-3925","authenticated-orcid":false,"given":"Jian","family":"Li","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-8329-7192","authenticated-orcid":false,"given":"Yin","family":"Lou","sequence":"additional","affiliation":[{"name":"Ant Group, Redwood City, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,5,30]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2024. Github repo. https:\/\/github.com\/Linfu2023\/learning-to-estimate-feature-importance"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2010.12.160"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chemolab.2010.12.004"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"e_1_2_1_5_1","unstructured":"L. Breiman J.H. Friedman C.J. Stone and R.A. Olshen. 1984. Classification and regression trees. Chapman and Hall\/CRC."},{"key":"e_1_2_1_6_1","unstructured":"C. Burges. 2010. From RankNet to LambdaRank to LambdaMART: An overview. Technical Report MSR-TR-2010--82."},{"key":"e_1_2_1_7_1","doi-asserted-by":"crossref","unstructured":"R. Caruana M. Elhawary A. Munson M. Riedewald D. Sorokina D. Fink W.M. Hochachka and S. Kelling. 2006. Mining citizen science data to predict orevalence of wild bird species. In KDD.","DOI":"10.1145\/1150402.1150527"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compeleceng.2013.11.024"},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","unstructured":"H.-T. Cheng L. Koc J. Harmsen T. Shaked T. Chandra H. Aradhye G. Anderson G. Corrado W. Chai M. Ispir R. Anil Z. Haque L. Hong V. Jain X. Liu and H. Shah. 2016. Wide & deep learning for recommender systems. In DLRS.","DOI":"10.1145\/2988450.2988454"},{"key":"e_1_2_1_10_1","first-page":"1","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","author":"J.","year":"2006","unstructured":"J. Dem?ar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1 (2006), 1--30.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2015.01.069"},{"key":"e_1_2_1_12_1","first-page":"1","article-title":"All models are wrong, but many are useful: Learning a variable's importance by studying an entire class of prediction models simultaneously","volume":"20","author":"Fisher A.","year":"2019","unstructured":"A. Fisher, C. Rudin, and F. Dominici. 2019. All models are wrong, but many are useful: Learning a variable's importance by studying an entire class of prediction models simultaneously. Journal of Machine Learning Research 20, 177 (2019), 1--81.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_13_1","volume-title":"Mar","author":"Forman G.","year":"2003","unstructured":"G. Forman. 2003. An extensive empirical study of feature selection metrics for text classification. Journal of Machine Learning Research 3, Mar (2003), 1289--1305."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1013203451"},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","unstructured":"T.R. Golub D.K. Slonim P. Tamayo C. Huard M. Gaasenbeek J.P. Mesirov H. Coller M.L. Loh J.R. Downing M.A. Caligiuri C.D. Bloomfield and E.S. Lander. 1999. Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring. Science 286 5439 (1999) 531--537.","DOI":"10.1126\/science.286.5439.531"},{"key":"e_1_2_1_16_1","unstructured":"Y. Gorishniy I. Rubachev and A. Babenko. 2022. On embeddings for numerical features in tabular deep learning. NeuraIPS (2022)."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11222-016-9646-1"},{"key":"e_1_2_1_18_1","unstructured":"I. Guyon and A. Elisseeff. 2003. An introduction to variable and feature selection. Journal of Machine Learning Research 3 Mar (2003) 1157--1182."},{"key":"e_1_2_1_19_1","unstructured":"H. Han X. Guo and H. Yu. 2016. Variable selection using mean decrease accuracy and mean decrease gini based on random forest. In ICSESS."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1214\/07-EJS039"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-14-119"},{"key":"e_1_2_1_22_1","volume-title":"Explorekit: Automatic feature generation and selection. In ICDM.","author":"Katz G.","year":"2016","unstructured":"G. Katz, E.C.R. Shin, and D. Song. 2016. Explorekit: Automatic feature generation and selection. In ICDM."},{"key":"e_1_2_1_23_1","unstructured":"G. Ke Q. Meng T. Finley T. Wang W. Chen W Ma Q. Ye and T.-Y. Liu. 2017. LightGBM: A highly efficient gradient boosting decision tree. In NeurIPS."},{"key":"e_1_2_1_24_1","unstructured":"K. Kira and L.A. Rendell. 1992. The feature selection problem: Traditional methods and a new algorithm. In AAAI."},{"key":"e_1_2_1_25_1","doi-asserted-by":"crossref","unstructured":"T. Kraska A. Beutel E.H. Chi J. Dean and N. Polyzotis. 2018. The case for learned index structures. In SIGMOD.","DOI":"10.1145\/3183713.3196909"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v036.i11"},{"key":"e_1_2_1_27_1","unstructured":"K. Li and J. Malik. 2016. Learning to optimize. arXiv preprint arXiv:1606.01885 (2016)."},{"key":"e_1_2_1_28_1","unstructured":"G. Louppe L. Wehenkel A. Sutera and P. Geurts. 2013. Understanding variable importances in forests of randomized trees. In NeurIPS."},{"key":"e_1_2_1_29_1","unstructured":"S. Lundberg G. Erion and S. Lee. 2018. Consistent individualized feature attribution for tree ensembles. arXiv preprint arXiv:1802.03888 (2018)."},{"key":"e_1_2_1_30_1","unstructured":"S. Lundberg and S. Lee. 2017. A unified approach to interpreting model predictions. In NeurIPS."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2010.00740.x"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-10-213"},{"key":"e_1_2_1_33_1","unstructured":"C. Molnar. 2020. Interpretable machine learning. Lulu. com."},{"key":"e_1_2_1_34_1","doi-asserted-by":"crossref","unstructured":"F. Nargesian H. Samulowitz U. Khurana E.B. Khalil and D.S. Turaga. 2017. Learning feature engineering for classification. In IJCAI.","DOI":"10.24963\/ijcai.2017\/352"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2017.01.013"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-8-25"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1214\/009053607000000505"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1996.tb02080.x"},{"key":"e_1_2_1_39_1","unstructured":"H. Vafaie and I.F. Imam. 1994. Feature selection methods: Genetic algorithms vs. greedy-like search. In INFUS."},{"key":"e_1_2_1_40_1","volume-title":"Meta-learning: A survey. arXiv preprint arXiv:1810.03548","author":"Vanschoren J.","year":"2018","unstructured":"J. Vanschoren. 2018. Meta-learning: A survey. arXiv preprint arXiv:1810.03548 (2018)."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/38.3-4.330"},{"key":"e_1_2_1_42_1","volume-title":"OBOE: Collaborative filtering for AutoML model selection. In KDD.","author":"Yang C.","year":"2019","unstructured":"C. Yang, Y. Akimoto, D.W. Kim, and M. Udell. 2019. OBOE: Collaborative filtering for AutoML model selection. In KDD."},{"key":"e_1_2_1_43_1","doi-asserted-by":"crossref","unstructured":"A. Zien N. Kr\u00e4mer S. Sonnenburg and G. R\u00e4tsch. 2009. The feature importance ranking measure. In ECML\/PKDD.","DOI":"10.1007\/978-3-642-04174-7_45"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654942","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3654942","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T14:39:42Z","timestamp":1755787182000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3654942"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,29]]},"references-count":43,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,5,29]]}},"alternative-id":["10.1145\/3654942"],"URL":"https:\/\/doi.org\/10.1145\/3654942","relation":{},"ISSN":["2836-6573"],"issn-type":[{"type":"electronic","value":"2836-6573"}],"subject":[],"published":{"date-parts":[[2024,5,29]]}}}