{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T13:58:43Z","timestamp":1784642323072,"version":"3.55.0"},"reference-count":41,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2018,4,10]],"date-time":"2018-04-10T00:00:00Z","timestamp":1523318400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Entropy measures have been a major interest of researchers to measure the information content of a dynamical system. One of the well-known methodologies is sample entropy, which is a model-free approach and can be deployed to measure the information transfer in time series. Sample entropy is based on the conditional entropy where a major concern is the number of past delays in the conditional term. In this study, we deploy a lag-specific conditional entropy to identify the informative past values. Moreover, considering the seasonality structure of data, we propose a clustering-based sample entropy to exploit the temporal information. Clustering-based sample entropy is based on the sample entropy definition while considering the clustering information of the training data and the membership of the test point to the clusters. In this study, we utilize the proposed method for transductive feature selection in black-box weather forecasting and conduct the experiments on minimum and maximum temperature prediction in Brussels for 1\u20136 days ahead. The results reveal that considering the local structure of the data can improve the feature selection performance. In addition, despite the large reduction in the number of features, the performance is competitive with the case of using all features.<\/jats:p>","DOI":"10.3390\/e20040264","type":"journal-article","created":{"date-parts":[[2018,4,10]],"date-time":"2018-04-10T13:06:08Z","timestamp":1523365568000},"page":"264","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Transductive Feature Selection Using Clustering-Based Sample Entropy for Temperature Prediction in Weather Forecasting"],"prefix":"10.3390","volume":"20","author":[{"given":"Zahra","family":"Karevan","sequence":"first","affiliation":[{"name":"ESAT-STADIUS (Department of Electrical Engineering-Stadius Centre for Dynamical Systems, Signal Processing and Data Analytics), KU Leuven, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Johan","family":"Suykens","sequence":"additional","affiliation":[{"name":"ESAT-STADIUS (Department of Electrical Engineering-Stadius Centre for Dynamical Systems, Signal Processing and Data Analytics), KU Leuven, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,4,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"045102","DOI":"10.1103\/PhysRevE.80.045102","article-title":"Entropy measures for networks: Toward an information theory of complex topologies","volume":"80","author":"Anand","year":"2009","journal-title":"Phys. Rev. E"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"4443","DOI":"10.3390\/e16084443","article-title":"Structure of a global network of financial companies based on transfer entropy","volume":"16","author":"Sandoval","year":"2014","journal-title":"Entropy"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"H2039","DOI":"10.1152\/ajpheart.2000.278.6.H2039","article-title":"Physiological time series analysis using approximate entropy and sample entropy","volume":"278","author":"Richman","year":"2000","journal-title":"Am. J. Physiol. Heart Circ. Physiol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2131","DOI":"10.1002\/joc.1357","article-title":"Measurement of climate complexity using sample entropy","volume":"26","author":"Shuangcheng","year":"2006","journal-title":"Int. J. Climatol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4844","DOI":"10.3390\/e15114844","article-title":"Statistical mechanics and information-theoretic perspectives on complexity in the Earth system","volume":"15","author":"Balasis","year":"2013","journal-title":"Entropy"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, Z., Li, Y., Childress, A.R., and Detre, J.A. (2014). Brain entropy mapping using fMRI. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0089948"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1007\/s004220050549","article-title":"Conditional entropy approach for the evaluation of the coupling strength","volume":"81","author":"Porta","year":"1999","journal-title":"Biol. Cybern."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2556","DOI":"10.1109\/TBME.2014.2323131","article-title":"Lag-specific transfer entropy as a tool to assess cardiovascular and cardiorespiratory information transfer","volume":"61","author":"Faes","year":"2014","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1038\/nature14956","article-title":"The quiet revolution of numerical weather prediction","volume":"525","author":"Bauer","year":"2015","journal-title":"Nature"},{"key":"ref_10","first-page":"431","article-title":"Geographically weighted regression","volume":"47","author":"Brunsdon","year":"1998","journal-title":"J. R. Stat. Soc. Ser. D"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"888","DOI":"10.1162\/neco.1992.4.6.888","article-title":"Local learning algorithms","volume":"4","author":"Bottou","year":"1992","journal-title":"Neural Computation"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Karevan, Z., and Suykens, J.A.K. (2016, January 24\u201329). Clustering-based feature selection for black-box weather temperature prediction. Proceedings of the 2016 International Joint Conference on Neural Networks, Vancouver, BC, Canada.","DOI":"10.1109\/IJCNN.2016.7727541"},{"key":"ref_13","unstructured":"Karevan, Z., Feng, Y., and Suykens, J.A.K. (2016, January 27\u201329). Moving Least Squares Support Vector Machines for weather temperature prediction. Proceedings of the European Symposium on Artificial Neural Networks, Brugge, Belgium."},{"key":"ref_14","unstructured":"Hmamouche, Y., Casali, A., and Lakhal, L. (2017, January 21\u201325). Causality based feature selection approach for multivariate time series forecasting. Proceedings of the International Conference on Advances in Databases, Knowledge, and Data Applications, Barcelona, Spain."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Van Dijck, G., and Van Hulle, M.M. (2006, January 10\u201314). Speeding up the wrapper feature subset selection in regression by mutual information relevance and redundancy analysis. Proceedings of the International Conference on Artificial Neural Networks, Athens, Greece.","DOI":"10.1007\/11840817_4"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ram\u0131rez-Gallego, S., Mourino-Tal\u0131n, H., Mart\u0131nez-Rego, D., Bol\u00f3n-Canedo, V., Ben\u0131tez, J.M., Alonso-Betanzos, A., and Herrera, F. (2017). An Information Theory-Based Feature Selection Framework for Big Data under Apache Spark. IEEE Trans. Syst. Man Cybern. Syst.","DOI":"10.1109\/TSMC.2017.2670926"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1016\/j.patcog.2016.08.011","article-title":"An efficient semi-supervised representatives feature selection algorithm based on information theory","volume":"61","author":"Wang","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"303601","DOI":"10.1155\/2012\/303601","article-title":"Causal information approach to partial conditioning in multivariate data sets","volume":"2012","author":"Marinazzo","year":"2012","journal-title":"Comput. Math. Methods Med."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wang, H., Wang, G., Zeng, X., and Peng, S. (2017, January 9\u201310). Online Streaming Feature Selection Based on Conditional Information Entropy. Proceedings of the 2017 IEEE International Conference on Big Knowledge (ICBK), Hefei, China.","DOI":"10.1109\/ICBK.2017.44"},{"key":"ref_20","unstructured":"(2018, April 05). Weather Underground. Available online: www.wunderground.com."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/584091.584093","article-title":"A mathematical theory of communication","volume":"5","author":"Shannon","year":"2001","journal-title":"ACM Sigmob. Mob. Comput. Commun. Rev."},{"key":"ref_22","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, John Wiley & Sons."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"062114","DOI":"10.1103\/PhysRevE.95.062114","article-title":"Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations","volume":"95","author":"Xiong","year":"2017","journal-title":"Phys. Rev. E"},{"key":"ref_24","first-page":"754","article-title":"Entropy per unit time as a metric invariant of automorphisms","volume":"124","author":"Kolmogorov","year":"1959","journal-title":"Dokl. Akad. Nauk SSSR"},{"key":"ref_25","first-page":"768","article-title":"On the Notion of entropy of a dynamical system","volume":"124","author":"Sinai","year":"1959","journal-title":"Dokl. Akad. Nauk SSSR"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"6212","DOI":"10.3390\/e16126212","article-title":"Ordinal patterns, entropy, and EEG","volume":"16","author":"Keller","year":"2014","journal-title":"Entropy"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1080\/02604027.1997.9972649","article-title":"Entropy, information and predictability of evolutionary systems","volume":"50","author":"Ebeling","year":"1997","journal-title":"World Futures J. Gen. Evol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1065","DOI":"10.1214\/aoms\/1177704472","article-title":"On estimation of a probability density function and mode","volume":"33","author":"Parzen","year":"1962","journal-title":"Ann. Math. Stat."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"258701","DOI":"10.1103\/PhysRevLett.108.258701","article-title":"Escaping the curse of dimensionality in estimating multivariate transfer entropy","volume":"108","author":"Runge","year":"2012","journal-title":"Phys. Rev. Lett."},{"key":"ref_30","first-page":"424","article-title":"Investigating causal relations by econometric models and cross-spectral methods","volume":"37","author":"Granger","year":"1969","journal-title":"Econom. J. Econom. Soc."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"113","DOI":"10.3390\/e15010113","article-title":"The relation between Granger causality and directed information theory: A review","volume":"15","author":"Amblard","year":"2012","journal-title":"Entropy"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"051112","DOI":"10.1103\/PhysRevE.83.051112","article-title":"Information-based detection of nonlinear Granger causality in multivariate processes via a nonuniform embedding technique","volume":"83","author":"Faes","year":"2011","journal-title":"Phys. Rev. E"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Langone, R., Mall, R., and Suykens, J.A.K. (2013, January 4\u20139). Soft Kernel Spectral clustering. Proceedings of the International Joint Conference on Neural Networks, Dallas, TX, USA.","DOI":"10.1109\/IJCNN.2013.6706850"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1109\/TPAMI.2008.292","article-title":"Multiway spectral clustering with out-of-sample extensions through weighted kernel PCA","volume":"32","author":"Alzate","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1098\/rsta.1909.0016","article-title":"Functions of positive and negative type, and their connection with the theory of integral equations","volume":"209","author":"Mercer","year":"1909","journal-title":"Philos. Trans. R. Soc. Lond. Ser. A"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1023\/A:1018628609742","article-title":"Least Squares Support Vector Machine Classifiers","volume":"9","author":"Suykens","year":"1999","journal-title":"Neural Process. Lett."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., and Vandewalle, J. (2002). Least Squares Support Vector Machines, World Scientific.","DOI":"10.1142\/5089"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1080\/0020718508961129","article-title":"Input-output parametric models for non-linear systems part I: Deterministic non-linear systems","volume":"41","author":"Leontaritis","year":"1985","journal-title":"Int. J. Control"},{"key":"ref_39","unstructured":"De Brabanter, K., Karsmakers, P., Ojeda, F., Alzate, C., De Brabanter, J., Pelckmans, K., De Moor, B., Vandewalle, J., and Suykens, J.A.K. (2018, April 10). LS-SVMlab Toolbox User\u2019s Guide: Version 1.8. Available online: https:\/\/www.esat.kuleuven.be\/sista\/lssvmlab\/."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"73","author":"Tibshirani","year":"1996","journal-title":"J. R. Stat. Soc. Ser. B (Methodol.)"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1109\/TNN.2010.2087769","article-title":"Approximate confidence and prediction intervals for least squares support vector regression","volume":"22","author":"Suykens","year":"2011","journal-title":"IEEE Trans. Neural Netw."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/4\/264\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:00:12Z","timestamp":1760194812000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/4\/264"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,4,10]]},"references-count":41,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2018,4]]}},"alternative-id":["e20040264"],"URL":"https:\/\/doi.org\/10.3390\/e20040264","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,4,10]]}}}