{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T12:15:57Z","timestamp":1792844157434,"version":"4.5.0"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,4,18]],"date-time":"2021-04-18T00:00:00Z","timestamp":1618704000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Anhui Province Key Research and Development Plan","award":["201904a05020073"],"award-info":[{"award-number":["201904a05020073"]}]},{"name":"Australian Research Council Discovery (ARC) Projects","award":["DP170101306"],"award-info":[{"award-number":["DP170101306"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61876206"],"award-info":[{"award-number":["61876206"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Key Research and Development Program of China","award":["2020AAA0106100"],"award-info":[{"award-number":["2020AAA0106100"]}]},{"name":"Open Project Foundation of Intelligent Information Processing Key Laboratory of Shanxi Province","award":["CICIP2020003"],"award-info":[{"award-number":["CICIP2020003"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2021,8,31]]},"abstract":"<jats:p>In this article, we aim to develop a unified view of causal and non-causal feature selection methods. The unified view will fill in the gap in the research of the relation between the two types of methods. Based on the Bayesian network framework and information theory, we first show that causal and non-causal feature selection methods share the same objective. That is to find the Markov blanket of a class attribute, the theoretically optimal feature set for classification. We then examine the assumptions made by causal and non-causal feature selection methods when searching for the optimal feature set, and unify the assumptions by mapping them to the restrictions on the structure of the Bayesian network model of the studied problem. We further analyze in detail how the structural assumptions lead to the different levels of approximations employed by the methods in their search, which then result in the approximations in the feature sets found by the methods with respect to the optimal feature set. With the unified view, we can interpret the output of non-causal methods from a causal perspective and derive the error bounds of both types of methods. Finally, we present practical understanding of the relation between causal and non-causal methods using extensive experiments with synthetic data and various types of real-world data.<\/jats:p>","DOI":"10.1145\/3436891","type":"journal-article","created":{"date-parts":[[2021,4,18]],"date-time":"2021-04-18T16:05:45Z","timestamp":1618761945000},"page":"1-46","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":82,"title":["A Unified View of Causal and Non-causal Feature Selection"],"prefix":"10.1145","volume":"15","author":[{"given":"Kui","family":"Yu","sequence":"first","affiliation":[{"name":"Hefei University of Technology, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Liu","sequence":"additional","affiliation":[{"name":"University of South Australia, Mawson Lakes, Adelaide, SA, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiuyong","family":"Li","sequence":"additional","affiliation":[{"name":"University of South Australia, Mawson Lakes, Adelaide, SA, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,4,18]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"International Encyclopedia of Statistical Science","author":"Agresti Alan","unstructured":"Alan Agresti and Maria Kateri . 2011. Categorical data analysis . In International Encyclopedia of Statistical Science . Springer , 206--208. Alan Agresti and Maria Kateri. 2011. Categorical data analysis. In International Encyclopedia of Statistical Science. Springer, 206--208."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/1756006.1756013"},{"key":"e_1_2_2_3_1","volume-title":"Proceedings of the AMIA Annual Symposium.","volume":"2003","author":"Aliferis Constantin F.","year":"2003","unstructured":"Constantin F. Aliferis , Ioannis Tsamardinos , and Alexander Statnikov . 2003 . HITON: A novel Markov blanket algorithm for optimal variable selection . In Proceedings of the AMIA Annual Symposium. Vol. 2003 . American Medical Informatics Association, 21. Constantin F. Aliferis, Ioannis Tsamardinos, and Alexander Statnikov. 2003. HITON: A novel Markov blanket algorithm for optimal variable selection. In Proceedings of the AMIA Annual Symposium. Vol. 2003. American Medical Informatics Association, 21."},{"key":"e_1_2_2_4_1","unstructured":"Kevin Bache and Moshe Lichman. 2013. UCI machine learning repository. Retrieved from http:\/\/archive.ics.uci.edu\/ml.  Kevin Bache and Moshe Lichman. 2013. UCI machine learning repository. Retrieved from http:\/\/archive.ics.uci.edu\/ml."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.62"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/72.298224"},{"key":"e_1_2_2_7_1","volume-title":"Cooper","author":"Beinlich Ingo A.","year":"1989","unstructured":"Ingo A. Beinlich , Henri J. Suermondt , R. Martin Chavez , and Gregory F . Cooper . 1989 . The ALARM Monitoring System: A Case Study with Two Probabilistic Inference Techniques for Belief Networks. Springer . Ingo A. Beinlich, Henri J. Suermondt, R. Martin Chavez, and Gregory F. Cooper. 1989. The ALARM Monitoring System: A Case Study with Two Probabilistic Inference Techniques for Belief Networks. Springer."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007612503587"},{"key":"e_1_2_2_9_1","volume-title":"Proceedings of the 27th International Conference on Machine Learning. 95--102","author":"Bontempi Gianluca","unstructured":"Gianluca Bontempi and Patrick E. Meyer . 2010. Causal filter selection in microarray data . In Proceedings of the 27th International Conference on Machine Learning. 95--102 . Gianluca Bontempi and Patrick E. Meyer. 2010. Causal filter selection in microarray data. In Proceedings of the 27th International Conference on Machine Learning. 95--102."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/3322706.3322714"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/2503308.2188387"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/asq008"},{"key":"e_1_2_2_13_1","volume-title":"Thomas","author":"Cover Thomas M.","year":"2012","unstructured":"Thomas M. Cover and Joy A . Thomas . 2012 . Elements of Information Theory. John Wiley & Sons . Thomas M. Cover and Joy A. Thomas. 2012. Elements of Information Theory. John Wiley & Sons."},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(03)00079-1"},{"key":"e_1_2_2_15_1","volume-title":"Transmission of Information: A Statistical Theory of Communications","author":"Fano R. M.","unstructured":"R. M. Fano . 1961. Transmission of Information: A Statistical Theory of Communications . MIT Press . R. M. Fano. 1961. Transmission of Information: A Statistical Theory of Communications. MIT Press."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/1005332.1044711"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007465528199"},{"key":"e_1_2_2_18_1","volume-title":"Desmarais","author":"Fu Shunkai","year":"2008","unstructured":"Shunkai Fu and Michel C . Desmarais . 2008 . Fast Markov blanket discovery algorithm via local learning within single pass. In Proceedings of the Conference of the Canadian Society for Computational Studies of Intelligence. Springer , 96--107. Shunkai Fu and Michel C. Desmarais. 2008. Fast Markov blanket discovery algorithm via local learning within single pass. In Proceedings of the Conference of the Canadian Society for Computational Studies of Intelligence. Springer, 96--107."},{"key":"e_1_2_2_19_1","volume-title":"Introduction to Statistical Pattern Recognition","author":"Fukunaga Keinosuke","unstructured":"Keinosuke Fukunaga . 2013. Introduction to Statistical Pattern Recognition . Academic Press . Keinosuke Fukunaga. 2013. Introduction to Statistical Pattern Recognition. Academic Press."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2016.2539338"},{"key":"e_1_2_2_21_1","volume-title":"Causal feature selection. Computational Methods of Feature Selection","author":"Guyon Isabelle","unstructured":"Isabelle Guyon , Constantin Aliferis , and Andr\u00e9 Elisseeff . 2007. Causal feature selection. Computational Methods of Feature Selection , H. Liu and H. Motoda (Eds.). CRC Press . Isabelle Guyon, Constantin Aliferis, and Andr\u00e9 Elisseeff. 2007. Causal feature selection. Computational Methods of Feature Selection, H. Liu and H. Motoda (Eds.). CRC Press."},{"key":"e_1_2_2_22_1","volume-title":"Mar","author":"Guyon Isabelle","year":"2003","unstructured":"Isabelle Guyon and Andr\u00e9 Elisseeff . 2003. An introduction to variable and feature selection. The Journal of Machine Learning Research 3 , Mar ( 2003 ), 1157--1182. Isabelle Guyon and Andr\u00e9 Elisseeff. 2003. An introduction to variable and feature selection. The Journal of Machine Learning Research 3, Mar (2003), 1157--1182."},{"key":"e_1_2_2_23_1","volume-title":"An introduction to feature extraction. Feature Extraction","author":"Guyon Isabelle","unstructured":"Isabelle Guyon and Andr\u00e9 Elisseeff . 2006. An introduction to feature extraction. Feature Extraction . Springer , 1--25. Isabelle Guyon and Andr\u00e9 Elisseeff. 2006. An introduction to feature extraction. Feature Extraction. Springer, 1--25."},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1970.1054466"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(97)00043-X"},{"key":"e_1_2_2_26_1","volume-title":"Proceedings of the 13th International Conference on International Conference on Machine Learning. 284--292","author":"Koller Daphne","year":"1995","unstructured":"Daphne Koller and Mehran Sahami . 1995 . Toward optimal feature selection . In Proceedings of the 13th International Conference on International Conference on Machine Learning. 284--292 . Daphne Koller and Mehran Sahami. 1995. Toward optimal feature selection. In Proceedings of the 13th International Conference on International Conference on Machine Learning. 284--292."},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729694"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.3115\/1075527.1075574"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2996357"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/11744023_6"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TETCI.2020.2978238"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3335676"},{"key":"e_1_2_2_33_1","volume-title":"Proceedings of the Advances in Neural Information Processing Systems. 505--511","author":"Margaritis Dimitris","year":"2000","unstructured":"Dimitris Margaritis and Sebastian Thrun . 2000 . Bayesian network induction via local neighborhoods . In Proceedings of the Advances in Neural Information Processing Systems. 505--511 . Dimitris Margaritis and Sebastian Thrun. 2000. Bayesian network induction via local neighborhoods. In Proceedings of the Advances in Neural Information Processing Systems. 505--511."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2008.923858"},{"key":"e_1_2_2_35_1","unstructured":"Judea Pearl. 2014. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann.  Judea Pearl. 2014. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijar.2006.06.008"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2005.159"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1025667309714"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2021.3058954"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/584091.584093"},{"key":"e_1_2_2_41_1","volume-title":"Proceedings of the Advances in Neural Information Processing Systems. 4637--4645","author":"Shishkin Alexander","year":"2016","unstructured":"Alexander Shishkin , Anastasia Bezzubtseva , Alexey Drutsa , Ilia Shishkov , Ekaterina Gladkikh , Gleb Gusev , and Pavel Serdyukov . 2016 . Efficient high-order interaction-aware feature selection based on conditional mutual information . In Proceedings of the Advances in Neural Information Processing Systems. 4637--4645 . Alexander Shishkin, Anastasia Bezzubtseva, Alexey Drutsa, Ilia Shishkov, Ekaterina Gladkikh, Gleb Gusev, and Pavel Serdyukov. 2016. Efficient high-order interaction-aware feature selection based on conditional mutual information. In Proceedings of the Advances in Neural Information Processing Systems. 4637--4645."},{"key":"e_1_2_2_42_1","first-page":"1393","article-title":"Feature selection via dependence maximization","volume":"13","author":"Song Le","year":"2012","unstructured":"Le Song , Alex Smola , Arthur Gretton , Justin Bedo , and Karsten Borgwardt . 2012 . Feature selection via dependence maximization . Journal of Machine Learning Research 13 , 47 (2012), 1393 -- 1434 . Le Song, Alex Smola, Arthur Gretton, Justin Bedo, and Karsten Borgwardt. 2012. Feature selection via dependence maximization. Journal of Machine Learning Research 13, 47 (2012), 1393--1434.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2020.2968743"},{"key":"e_1_2_2_44_1","volume-title":"Prediction, and Search","author":"Spirtes Peter","unstructured":"Peter Spirtes , Clark N. Glymour , and Richard Scheines . 2000. Causation , Prediction, and Search . Vol. 81 . MIT Press . Peter Spirtes, Clark N. Glymour, and Richard Scheines. 2000. Causation, Prediction, and Search. Vol. 81. MIT Press."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1968.1054135"},{"key":"e_1_2_2_46_1","volume-title":"Aliferis","author":"Tsamardinos Ioannis","year":"2003","unstructured":"Ioannis Tsamardinos and Constantin F . Aliferis . 2003 . Towards principled feature selection: Relevancy, filters and wrappers. In Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics. Morgan Kaufmann Publishers . Ioannis Tsamardinos and Constantin F. Aliferis. 2003. Towards principled feature selection: Relevancy, filters and wrappers. In Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics. Morgan Kaufmann Publishers."},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/956750.956838"},{"key":"e_1_2_2_48_1","volume-title":"Proceedings of the FLAIRS Conference.","volume":"2","author":"Tsamardinos Ioannis","year":"2003","unstructured":"Ioannis Tsamardinos , Constantin F. Aliferis , Alexander R. Statnikov , and Er Statnikov . 2003 . Algorithms for large scale Markov blanket discovery . In Proceedings of the FLAIRS Conference. Vol. 2 . 376--380. Ioannis Tsamardinos, Constantin F. Aliferis, Alexander R. Statnikov, and Er Statnikov. 2003. Algorithms for large scale Markov blanket discovery. In Proceedings of the FLAIRS Conference. Vol. 2. 376--380."},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-006-6889-7"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-013-1368-0"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2003.1238356"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2015.11.007"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.4108\/icst.collaboratecom.2012.250689"},{"key":"e_1_2_2_54_1","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.ins.2019.09.010","article-title":"Towards efficient and effective discovery of Markov blankets for feature selection","author":"Wang Hao","year":"2020","unstructured":"Hao Wang , Zhaolong Ling , Kui Yu , and Xindong Wu . 2020 . Towards efficient and effective discovery of Markov blankets for feature selection . Information Sciences 509 , January (2020), 227 -- 242 . Hao Wang, Zhaolong Ling, Kui Yu, and Xindong Wu. 2020. Towards efficient and effective discovery of Markov blankets for feature selection. Information Sciences 509, January (2020), 227--242.","journal-title":"Information Sciences 509"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2017.2650906"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2015.2504420"},{"key":"e_1_2_2_57_1","volume-title":"Proceedings of the Advances in Neural Information Processing Systems. 687--693","author":"Yang Howard Hua","year":"2000","unstructured":"Howard Hua Yang and John Moody . 2000 . Data visualization and feature selection: New algorithms for nongaussian data . In Proceedings of the Advances in Neural Information Processing Systems. 687--693 . Howard Hua Yang and John Moody. 2000. Data visualization and feature selection: New algorithms for nongaussian data. In Proceedings of the Advances in Neural Information Processing Systems. 687--693."},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2005.134"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3409382"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2019.2908373"},{"key":"e_1_2_2_62_1","first-page":"1205","article-title":"Efficient feature selection via analysis of relevance and redundancy","author":"Yu Lei","year":"2004","unstructured":"Lei Yu and Huan Liu . 2004 . Efficient feature selection via analysis of relevance and redundancy . The Journal of Machine Learning Research 5 , December (2004), 1205 -- 1224 . Lei Yu and Huan Liu. 2004. Efficient feature selection via analysis of relevance and redundancy. The Journal of Machine Learning Research 5, December (2004), 1205--1224.","journal-title":"The Journal of Machine Learning Research 5"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2014.2326099"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCBB.2015.2476796"},{"key":"e_1_2_2_65_1","volume-title":"Proceedings of the 32nd International Conference on Neural Information Processing Systems. 9472--9483","author":"Zheng Xun","unstructured":"Xun Zheng , Bryon Aragam , Pradeep K. Ravikumar , and Eric P. Xing . 2018. DAGs with NO TEARS: Continuous optimization for structure learning . In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 9472--9483 . Xun Zheng, Bryon Aragam, Pradeep K. Ravikumar, and Eric P. Xing. 2018. DAGs with NO TEARS: Continuous optimization for structure learning. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 9472--9483."}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3436891","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3436891","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:20Z","timestamp":1750197740000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3436891"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,18]]},"references-count":64,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,8,31]]}},"alternative-id":["10.1145\/3436891"],"URL":"https:\/\/doi.org\/10.1145\/3436891","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,18]]},"assertion":[{"value":"2019-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}