{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,11]],"date-time":"2025-06-11T20:07:03Z","timestamp":1749672423340,"version":"3.27.0"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"13","license":[{"start":{"date-parts":[[2023,3,29]],"date-time":"2023-03-29T00:00:00Z","timestamp":1680048000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,29]],"date-time":"2023-03-29T00:00:00Z","timestamp":1680048000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2023,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>A new ensemble framework for an interpretable model called linear iterative feature embedding (LIFE) has been developed to achieve high prediction accuracy, easy interpretation, and efficient computation simultaneously. The LIFE algorithm is able to fit a wide single-hidden-layer neural network (NN) accurately with three steps: defining the subsets of a dataset by the linear projections of neural nodes, creating the features from multiple narrow single-hidden-layer NNs trained on the different subsets of the data, combining the features with a linear model. The theoretical rationale behind LIFE is also provided by the connection to the loss ambiguity decomposition of stack ensemble methods. Both simulation and empirical experiments confirm that LIFE consistently outperforms directly trained single-hidden-layer NNs and also outperforms many other benchmark models, including multilayers feed forward neural network (FFNN), Xgboost, and random forest (RF) in many experiments. As a wide single-hidden-layer NN, LIFE is intrinsically interpretable. Meanwhile, both variable importance and global main and interaction effects can be easily created and visualized. In addition, the parallel nature of the base learner building makes LIFE computationally efficient by leveraging parallel computing.<\/jats:p>","DOI":"10.1007\/s00521-023-08204-w","type":"journal-article","created":{"date-parts":[[2023,3,29]],"date-time":"2023-03-29T17:03:16Z","timestamp":1680109396000},"page":"9657-9685","update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Linear iterative feature embedding: an ensemble framework for an interpretable model"],"prefix":"10.1007","volume":"35","author":[{"given":"Agus","family":"Sudjianto","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinwen","family":"Qiu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Miaoqi","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,3,29]]},"reference":[{"key":"8204_CR1","unstructured":"Chen T, He T, Benesty M, Khotilovich V, Tang Y (2015) Xgboost: extreme gradient boosting. R package version 0.4-2, 1\u20134"},{"issue":"1","key":"8204_CR2","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L (2001) Random forests. Mach Learn 45(1):5\u201332","journal-title":"Mach Learn"},{"issue":"10","key":"8204_CR3","doi-asserted-by":"publisher","first-page":"3009","DOI":"10.1016\/j.matcom.2009.01.023","volume":"79","author":"S Kucherenko","year":"2009","unstructured":"Kucherenko S et al (2009) Derivative based global sensitivity measures and their link with global sensitivity indices. Math Comput Simul 79(10):3009\u20133017","journal-title":"Math Comput Simul"},{"issue":"7","key":"8204_CR4","doi-asserted-by":"publisher","first-page":"1212","DOI":"10.1016\/j.cpc.2010.03.006","volume":"181","author":"S Kucherenko","year":"2010","unstructured":"Kucherenko S et al (2010) A new derivative based importance criterion for groups of variables and its link with the global sensitivity indices. Comput Phys Commun 181(7):1212\u20131217","journal-title":"Comput Phys Commun"},{"key":"8204_CR5","unstructured":"Sundararajan M, Taly A, Yan Q (2017) Axiomatic attribution for deep networks. arXiv preprint arXiv:1703.01365"},{"key":"8204_CR6","unstructured":"Ancona M, Ceolini E, \u00d6ztireli C, Gross M (2017) Towards better understanding of gradient-based attribution methods for deep neural networks. arXiv preprint arXiv:1711.06104"},{"issue":"5","key":"8204_CR7","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","volume":"1","author":"C Rudin","year":"2019","unstructured":"Rudin C (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell 1(5):206\u2013215","journal-title":"Nat Mach Intell"},{"key":"8204_CR8","unstructured":"Vaughan J, Sudjianto A, Brahimi E, Chen J, Nair VN (2018) Explainable neural networks based on additive index models. arXiv preprint arXiv:1806.01933"},{"key":"8204_CR9","doi-asserted-by":"crossref","unstructured":"Chen J, Vaughan J, Nair V, Sudjianto A (2020) Adaptive explainable neural networks (axnns). Available at SSRN 3569318","DOI":"10.2139\/ssrn.3569318"},{"issue":"6","key":"8204_CR10","doi-asserted-by":"publisher","first-page":"2610","DOI":"10.1109\/TNNLS.2020.3007259","volume":"32","author":"Z Yang","year":"2020","unstructured":"Yang Z, Zhang A, Sudjianto A (2020) Enhancing explainability of neural networks through architecture constraints. IEEE Trans Neural Netw Learn Syst 32(6):2610\u20132621","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"8204_CR11","unstructured":"Andrienko G, Andrienko N (2001) Constructing parallel coordinates plot for problem solving. In: 1st International Symposium on Smart Graphics, pp. 9\u201314"},{"key":"8204_CR12","unstructured":"Heath D, Kasif S, Salzberg S (1993) Induction of oblique decision trees. In: IJCAI, vol. 1993, pp. 1002\u20131007"},{"issue":"2","key":"8204_CR13","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","volume":"5","author":"DH Wolpert","year":"1992","unstructured":"Wolpert DH (1992) Stacked generalization. Neural Netw 5(2):241\u2013259","journal-title":"Neural Netw"},{"key":"8204_CR14","doi-asserted-by":"crossref","unstructured":"Chen T, Guestrin C (2016) Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining, pp. 785\u2013794","DOI":"10.1145\/2939672.2939785"},{"key":"8204_CR15","first-page":"1","volume":"19","author":"J Qiu","year":"2018","unstructured":"Qiu J, Jammalamadaka SR, Ning N (2018) Multivariate bayesian structural time series model. J Mach Learn Res 19:1\u201333","journal-title":"J Mach Learn Res"},{"issue":"10","key":"8204_CR16","doi-asserted-by":"publisher","first-page":"1061","DOI":"10.1007\/s10472-020-09710-6","volume":"88","author":"J Qiu","year":"2020","unstructured":"Qiu J, Jammalamadaka SR, Ning N (2020) Multivariate time series analysis from a bayesian machine learning perspective. Ann Math Artif Intell 88(10):1061\u20131082","journal-title":"Ann Math Artif Intell"},{"issue":"2","key":"8204_CR17","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1023\/A:1022859003006","volume":"51","author":"LI Kuncheva","year":"2003","unstructured":"Kuncheva LI, Whitaker CJ (2003) Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach Learn 51(2):181\u2013207","journal-title":"Mach Learn"},{"key":"8204_CR18","unstructured":"Brown G, Wyatt JL, Ti\u00a0o P (2005) Managing diversity in regression ensembles. Journal of machine learning research 6(Sep), 1621\u20131650"},{"issue":"4","key":"8204_CR19","doi-asserted-by":"publisher","first-page":"608","DOI":"10.1016\/j.patcog.2005.08.017","volume":"39","author":"M Aksela","year":"2006","unstructured":"Aksela M, Laaksonen J (2006) Using diversity of errors for selecting members of a committee classifier. Pattern Recogn 39(4):608\u2013623","journal-title":"Pattern Recogn"},{"key":"8204_CR20","doi-asserted-by":"crossref","unstructured":"Gacquer D, Delcroix V, Delmotte F, Piechowiak S (2009) On the effectiveness of diversity when training multiple classifier systems. In: European Conference on Symbolic and Quantitative Approaches to Reasoning and Uncertainty, pp. 493\u2013504. Springer","DOI":"10.1007\/978-3-642-02906-6_43"},{"issue":"3","key":"8204_CR21","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1177\/1748301818761132","volume":"12","author":"HK Butler IV","year":"2018","unstructured":"Butler HK IV, Friend MA, Bauer KW Jr, Bihl TJ (2018) The effectiveness of using diversity to select multiple classifier systems with varying classification thresholds. J Algorithm Comput Technol 12(3):187\u2013199","journal-title":"J Algorithm Comput Technol"},{"key":"8204_CR22","first-page":"231","volume":"7","author":"A Krogh","year":"1994","unstructured":"Krogh A, Vedelsby J (1994) Neural network ensembles, cross validation, and active learning. Adv Neural Inf Process Syst 7:231\u2013238","journal-title":"Adv Neural Inf Process Syst"},{"key":"8204_CR23","doi-asserted-by":"crossref","unstructured":"Ueda N, Nakano R (1996) Generalization error of ensemble estimators. In: Proceedings of International Conference on Neural Networks (ICNN\u201996), vol. 1, pp. 90\u201395. IEEE","DOI":"10.1109\/ICNN.1996.548872"},{"issue":"1","key":"8204_CR24","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1016\/j.inffus.2004.04.004","volume":"6","author":"G Brown","year":"2005","unstructured":"Brown G, Wyatt J, Harris R, Yao X (2005) Diversity creation methods: a survey and categorisation. Inf Fusion 6(1):5\u201320","journal-title":"Inf Fusion"},{"key":"8204_CR25","unstructured":"Hansen JV (2000) Combining predictors: Meta machine learning methods and bias\/variance & ambiguity decompositions. PhD thesis, Aarhus University, Computer Science Department"},{"key":"8204_CR26","unstructured":"Zeng M, Liao Y, Li R, Sudjianto A (2020) Local linear approximation algorithm for neural network. Manuscript"},{"key":"8204_CR27","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"8204_CR28","unstructured":"Lecun Y, Cortes C, Burges C (1999) The MNIST Dataset of Handwritten Digits(Images)"},{"key":"8204_CR29","unstructured":"Sudjianto A, Knauth W, Singh R, Yang Z, Zhang A (2020) Unwrapping the black box of deep relu networks: Interpretability, diagnostics, and simplification. arXiv preprint arXiv:2011.04041"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-023-08204-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-023-08204-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-023-08204-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T08:33:42Z","timestamp":1729154022000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-023-08204-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,29]]},"references-count":29,"journal-issue":{"issue":"13","published-print":{"date-parts":[[2023,5]]}},"alternative-id":["8204"],"URL":"https:\/\/doi.org\/10.1007\/s00521-023-08204-w","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"type":"print","value":"0941-0643"},{"type":"electronic","value":"1433-3058"}],"subject":[],"published":{"date-parts":[[2023,3,29]]},"assertion":[{"value":"20 May 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 January 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declaration"}},{"value":"The authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}