{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,17]],"date-time":"2026-05-17T22:36:19Z","timestamp":1779057379370,"version":"3.51.4"},"reference-count":36,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2022,11,25]],"date-time":"2022-11-25T00:00:00Z","timestamp":1669334400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006769","name":"Russian Science Foundation","doi-asserted-by":"publisher","award":["22-11-00055"],"award-info":[{"award-number":["22-11-00055"]}],"id":[{"id":"10.13039\/501100006769","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Approximation of entropies of various types using machine learning (ML) regression methods are shown for the first time. The ML models presented in this study define the complexity of the short time series by approximating dissimilar entropy techniques such as Singular value decomposition entropy (SvdEn), Permutation entropy (PermEn), Sample entropy (SampEn) and Neural Network entropy (NNetEn) and their 2D analogies. A new method for calculating SvdEn2D, PermEn2D and SampEn2D for 2D images was tested using the technique of circular kernels. Training and testing datasets on the basis of Sentinel-2 images are presented (two training images and one hundred and ninety-eight testing images). The results of entropy approximation are demonstrated using the example of calculating the 2D entropy of Sentinel-2 images and R2 metric evaluation. The applicability of the method for the short time series with a length from N = 5 to N = 113 elements is shown. A tendency for the R2 metric to decrease with an increase in the length of the time series was found. For SvdEn entropy, the regression accuracy is R2 &gt; 0.99 for N = 5 and R2 &gt; 0.82 for N = 113. The best metrics were observed for the ML_SvdEn2D and ML_NNetEn2D models. The results of the study can be used for fundamental research of entropy approximations of various types using ML regression, as well as for accelerating entropy calculations in remote sensing. The versatility of the model is shown on a synthetic chaotic time series using Planck map and logistic map.<\/jats:p>","DOI":"10.3390\/rs14235983","type":"journal-article","created":{"date-parts":[[2022,11,28]],"date-time":"2022-11-28T07:01:30Z","timestamp":1669618890000},"page":"5983","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Entropy Approximation by Machine Learning Regression: Application for Irregularity Evaluation of Images in Remote Sensing"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9341-1831","authenticated-orcid":false,"given":"Andrei","family":"Velichko","sequence":"first","affiliation":[{"name":"Institute of Physics and Technology, Petrozavodsk State University, 185910 Petrozavodsk, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maksim","family":"Belyaev","sequence":"additional","affiliation":[{"name":"Institute of Physics and Technology, Petrozavodsk State University, 185910 Petrozavodsk, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthias P.","family":"Wagner","sequence":"additional","affiliation":[{"name":"Panopterra, 64293 Darmstadt, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alireza","family":"Taravat","sequence":"additional","affiliation":[{"name":"Deimos Space, Oxford OX110QR, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,25]]},"reference":[{"key":"ref_1","unstructured":"(2022, September 06). History of Entropy\u2014Wikipedia. Available online: https:\/\/en.wikipedia.org\/wiki\/History_of_entropy."},{"key":"ref_2","unstructured":"(2022, September 06). Boltzmann\u2019s Entropy Formula\u2014Wikipedia. Available online: https:\/\/en.wikipedia.org\/wiki\/Boltzmann%27s_entropy_formula#cite_note-2."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A Mathematical Theory of Communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1016\/S0304-3975(98)00075-9","article-title":"On Tables of Random Numbers","volume":"207","author":"Kolmogorov","year":"1998","journal-title":"Theor. Comput. Sci."},{"key":"ref_5","unstructured":"(2022, September 06). Von Neumann Entropy\u2014Wikipedia. Available online: https:\/\/en.wikipedia.org\/wiki\/Von_Neumann_entropy."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Baez, J.C. (2022). R\u00e9nyi Entropy and Free Energy. Entropy, 24.","DOI":"10.3390\/e24050706"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Koutsoyiannis, D., and Sargentis, G.-F. (2021). Entropy and Wealth. Entropy, 23.","DOI":"10.3390\/e23101356"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MGRS.2017.2762307","article-title":"Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources","volume":"5","author":"Zhu","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"e2021RG000742","DOI":"10.1029\/2021RG000742","article-title":"Deep Learning for Geophysics: Current and Future Trends","volume":"59","author":"Yu","year":"2021","journal-title":"Rev. Geophys."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1007\/s40808-019-00686-9","article-title":"Integration of Remote Sensing, GIS, and Shannon\u2019s Entropy Approach to Conduct Trend Analysis of the Dynamics Change in Urban\/Built-up Areas in the Upper Citarum River Basin, West Java, Indonesia","volume":"6","author":"Yulianto","year":"2020","journal-title":"Model. Earth Syst. Environ."},{"key":"ref_11","first-page":"293","article-title":"The Use of GIS, Remote Sensing and Shannon\u2019s Entropy Statistical Techniques to Analyze and Monitor the Spatial and Temporal Patterns of Urbanization and Sprawl in Zarqa City, Jordan","volume":"8","year":"2016","journal-title":"J. Geogr. Inf. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3129","DOI":"10.12785\/amis\/080654","article-title":"Maximum Entropy for Image Segmentation Based on an Adaptive Particle Swarm Optimization","volume":"8","author":"Qi","year":"2014","journal-title":"Appl. Math. Inf. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1109\/TSMC.1973.4309314","article-title":"Textural Features for Image Classification","volume":"SMC-3","author":"Haralick","year":"1973","journal-title":"IEEE Trans. Syst. Man. Cybern."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Gao, T., Zheng, L., Xu, W., Piao, Y., Feng, R., Chen, X., and Zhou, T. (2021). An Automatic Exposure Method of Plane Array Remote Sensing Image Based on Two-Dimensional Entropy. Sensors, 21.","DOI":"10.3390\/s21103306"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1117\/1.3534855","article-title":"Using Image Entropy Maximum for Auto Exposure","volume":"20","author":"Rahman","year":"2011","journal-title":"J. Electron. Imaging"},{"key":"ref_16","unstructured":"Sun, W., Chen, H., Tang, H., and Liu, Y. (2010, January 4\u20136). Unsupervised Image Change Detection Means Based on 2-D Entropy. Proceedings of the the 2nd International Conference on Information Science and Engineering, Hangzhou, China."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1016\/j.image.2019.04.013","article-title":"Two-Dimensional Dispersion Entropy: An Information-Theoretic Method for Irregularity Analysis of Images","volume":"75","author":"Azami","year":"2019","journal-title":"Signal Process. Image Commun."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"45002","DOI":"10.1088\/2057-1976\/2\/4\/045002","article-title":"Two-Dimensional Sample Entropy: Assessing Image Texture through Irregularity","volume":"2","author":"Silva","year":"2016","journal-title":"Biomed. Phys. Eng. Express"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ribeiro, H.V., Zunino, L., Lenzi, E.K., Santoro, P.A., and Mendes, R.S. (2012). Complexity-Entropy Causality Plane as a Complexity Measure for Two-Dimensional Patterns. PLoS ONE, 7.","DOI":"10.1371\/journal.pone.0040689"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"6092","DOI":"10.1088\/1361-6560\/aa75b0","article-title":"The Approximate Entropy Concept Extended to Three Dimensions for Calibrated, Single Parameter Structural Complexity Interrogation of Volumetric Images","volume":"62","author":"Moore","year":"2017","journal-title":"Phys. Med. Biol."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Velichko, A., Wagner, M.P., Taravat, A., Hobbs, B., and Ord, A. (2022). NNetEn2D: Two-Dimensional Neural Network Entropy in Remote Sensing Imagery and Geophysical Mapping. Remote Sens., 14.","DOI":"10.3390\/rs14092166"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1109\/JSTARS.2020.3031918","article-title":"Analysis and Classification of SAR Textures Using Information Theory","volume":"14","author":"Chagas","year":"2021","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1109\/TGRS.2005.861007","article-title":"Unsupervised Change Detection on SAR Images Using Fuzzy Hidden Markov Chains","volume":"44","author":"Carincotte","year":"2006","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"23526","DOI":"10.1117\/1.2957968","article-title":"Change Detection in Synthetic Aperture Radar Images with a Sliding Hidden Markov Chain Model","volume":"2","author":"Bouyahia","year":"2008","journal-title":"J. Appl. Remote Sens."},{"key":"ref_25","unstructured":"Vallat, R. (2022, November 20). AntroPy: Entropy and Complexity of (EEG) Time-Series in Python. Available online: https:\/\/github.com\/raphaelvallat\/antropy."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1137\/1035134","article-title":"On the Early History of the Singular Value Decomposition","volume":"35","author":"Stewart","year":"1993","journal-title":"SIAM Rev."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1007\/s11741-008-0511-3","article-title":"Analysis of Heart Rate Variability Based on Singular Value Decomposition Entropy","volume":"12","author":"Li","year":"2008","journal-title":"J. Shanghai Univ. Engl. Ed."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jelinek, H.F., Donnan, L., and Khandoker, A.H. (2019, January 23\u201327). Singular Value Decomposition Entropy as a Measure of Ankle Dynamics Efficacy in a Y-Balance Test Following Supportive Lower Limb Taping. Proceedings of the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, Berlin, Germany.","DOI":"10.1109\/EMBC.2019.8856655"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Anagnoste, S., and Caraiani, P. (2019). The Impact of Financial and Macroeconomic Shocks on the Entropy of Financial Markets. Entropy, 21.","DOI":"10.3390\/e21030316"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"126337","DOI":"10.1016\/j.physa.2021.126337","article-title":"A Singular Value Decomposition Entropy Approach for Testing Stock Market Efficiency","volume":"583","author":"Rodriguez","year":"2021","journal-title":"Phys. A Stat. Mech. Its Appl."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"623141","DOI":"10.3389\/fevo.2021.623141","article-title":"SVD Entropy Reveals the High Complexity of Ecological Networks","volume":"9","author":"Strydom","year":"2021","journal-title":"Front. Ecol. Evol."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Buisine, J., Bigand, A., Synave, R., Delepoulle, S., and Renaud, C. (2021). Stopping Criterion during Rendering of Computer-Generated Images Based on SVD-Entropy. Entropy, 23.","DOI":"10.3390\/e23010075"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Velichko, A., and Heidari, H. (2021). A Method for Estimating the Entropy of Time Series Using Artificial Neural Networks. Entropy, 23.","DOI":"10.3390\/e23111432"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Velichko, A. (2020). Neural Network for Low-Memory IoT Devices and MNIST Image Recognition Using Kernels Based on Logistic Map. Electronics, 9.","DOI":"10.3390\/electronics9091432"},{"key":"ref_35","unstructured":"LeCun, Y., Cortes, C., and Burges, C. (2018, November 09). MNIST Handwritten Digit Database. Available online: http:\/\/yann.lecun.com\/exdb\/mnist\/."},{"key":"ref_36","unstructured":"Feurer, M., Eggensperger, K., Falkner, S., Lindauer, M., and Hutter, F. (2020). Auto-Sklearn 2.0: Hands-Free AutoML via Meta-Learning. arXiv."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/23\/5983\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:27:05Z","timestamp":1760146025000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/23\/5983"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,25]]},"references-count":36,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["rs14235983"],"URL":"https:\/\/doi.org\/10.3390\/rs14235983","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,25]]}}}