{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T21:01:27Z","timestamp":1772139687834,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2023,11,21]],"date-time":"2023-11-21T00:00:00Z","timestamp":1700524800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004281","name":"National Science Centre and the European Union Framework Programme for Research and Innovation Horizon 2020","doi-asserted-by":"publisher","award":["2022\/45\/P\/ST7\/03450"],"award-info":[{"award-number":["2022\/45\/P\/ST7\/03450"]}],"id":[{"id":"10.13039\/501100004281","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Measurement is a typical way of gathering information about an investigated object, generalized by a finite set of characteristic parameters. The result of each iteration of the measurement is an instance of the class of the investigated object in the form of a set of values of characteristic parameters. An ordered set of instances forms a collection whose dimensionality for a real object is a factor that cannot be ignored. Managing the dimensionality of data collections, as well as classification, regression, and clustering, are fundamental problems for machine learning. Compactification is the approximation of the original data collection by an equivalent collection (with a reduced dimension of characteristic parameters) with the control of accompanying information capacity losses. Related to compactification is the data completeness verifying procedure, which is characteristic of the data reliability assessment. If there are stochastic parameters among the initial data collection characteristic parameters, the compactification procedure becomes more complicated. To take this into account, this study proposes a model of a structured collection of stochastic data defined in terms of relative entropy. The compactification of such a data model is formalized by an iterative procedure aimed at maximizing the relative entropy of sequential implementation of direct and reverse projections of data collections, taking into account the estimates of the probability distribution densities of their attributes. The procedure for approximating the relative entropy function of compactification to reduce the computational complexity of the latter is proposed. To qualitatively assess compactification this study undertakes a formal analysis that uses data collection information capacity and the absolute and relative share of information losses due to compaction as its metrics. Taking into account the semantic connection of compactification and completeness, the proposed metric is also relevant for the task of assessing data reliability. Testing the proposed compactification procedure proved both its stability and efficiency in comparison with previously used analogues, such as the principal component analysis method and the random projection method.<\/jats:p>","DOI":"10.3390\/e25121567","type":"journal-article","created":{"date-parts":[[2023,11,21]],"date-time":"2023-11-21T10:24:13Z","timestamp":1700562253000},"page":"1567","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Small Stochastic Data Compactification Concept Justified in the Entropy Basis"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7624-7072","authenticated-orcid":false,"given":"Viacheslav","family":"Kovtun","sequence":"first","affiliation":[{"name":"Internet of Things Group, Institute of Theoretical and Applied Informatics Polish Academy of Sciences, Ba\u0142tycka 5, 44-100 Gliwice, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9087-0311","authenticated-orcid":false,"given":"Elena","family":"Zaitseva","sequence":"additional","affiliation":[{"name":"Department of Informatics, University of \u017dilina, 010 26 \u017dilina, Slovakia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vitaly","family":"Levashenko","sequence":"additional","affiliation":[{"name":"Department of Informatics, University of \u017dilina, 010 26 \u017dilina, Slovakia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6221-4790","authenticated-orcid":false,"given":"Krzysztof","family":"Grochla","sequence":"additional","affiliation":[{"name":"Internet of Things Group, Institute of Theoretical and Applied Informatics Polish Academy of Sciences, Ba\u0142tycka 5, 44-100 Gliwice, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9139-8987","authenticated-orcid":false,"given":"Oksana","family":"Kovtun","sequence":"additional","affiliation":[{"name":"Department of the Theory and Practice of Translation, Faculty of Foreign Languages, Vasyl\u2019 Stus Donetsk National University, 600-Richchya Str., 21, 21000 Vinnytsia, Ukraine"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,11,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"101933","DOI":"10.1016\/j.is.2021.101933","article-title":"Ripple: An approach to locate k nearest neighbours for location-based services","volume":"105","author":"Biswas","year":"2022","journal-title":"Inf. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"100071","DOI":"10.1016\/j.dajour.2022.100071","article-title":"A comparative analysis of K-Nearest Neighbor, Genetic, Support Vector Machine, Decision Tree, and Long Short Term Memory algorithms in machine learning","volume":"3","author":"Bansal","year":"2022","journal-title":"Decis. Anal. J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2599","DOI":"10.3934\/mbe.2021132","article-title":"Predictive modeling based on small data in clinical medicine: RBF-based additive input-doubling method","volume":"18","author":"Izonin","year":"2021","journal-title":"Math. Biosci. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Izonin, I., Tkachenko, R., Shakhovska, N., and Lotoshynska, N. (2021). The Additive Input-Doubling Method Based on the SVR with Nonlinear Kernels: Small Data Approach. Symmetry, 13.","DOI":"10.3390\/sym13040612"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"103930","DOI":"10.1016\/j.compind.2023.103930","article-title":"A survey on machine learning based analysis of heterogeneous data in industrial automation","volume":"149","author":"Kamm","year":"2023","journal-title":"Comput. Ind."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Tymchenko, O., Havrysh, B., Tymchenko, O.O., Khamula, O., Kovalskyi, B., and Havrysh, K. (2020, January 21\u201325). Person Voice Recognition Methods. Proceedings of the 2020 IEEE Third International Conference on Data Stream Mining & Processing (DSMP), Lviv, Ukraine.","DOI":"10.1109\/DSMP47368.2020.9204023"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Bisikalo, O., Kovtun, O., Kovtun, V., and Vysotska, V. (2020, January 10\u201312). Research of Pareto-Optimal Schemes of Control of Availability of the Information System for Critical Use. Proceedings of the 2020 1st International Workshop on Intelligent Information Technologies & Systems of Information Security (IntelITSIS), Khmelnytskyi, Ukraine. CEUR-WS.","DOI":"10.1109\/DESSERT50317.2020.9125061"},{"key":"ref_8","first-page":"505","article-title":"Mathematical Modeling of the Availability of the Information System for Critical Use to Optimize Control of its Communication Capabilities","volume":"11","author":"Bisikalo","year":"2021","journal-title":"Int. J. Sens. Wirel. Commun. Control."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1904","DOI":"10.1007\/s12555-021-0368-6","article-title":"Modeling of Operation of Information System for Critical Use in the Conditions of Influence of a Complex Certain Negative Factor","volume":"20","author":"Bisikalo","year":"2022","journal-title":"Int. J. Control. Autom. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Bisikalo, O., Bogach, I., and Sholota, V. (2020, January 25\u201329). The Method of Modelling the Mechanism of Random Access Memory of System for Natural Language Processing. Proceedings of the 2020 IEEE 15th International Conference on Advanced Trends in Radioelectronics, Telecommunications and Computer Engineering (TCSET), Lviv-Slavske, Ukraine, NJ, USA.","DOI":"10.1109\/TCSET49122.2020.235477"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Mochurad, L., and Horun, P. (2023). Improvement Technologies for Data Imputation in Bioinformatics. Technologies, 11.","DOI":"10.3390\/technologies11060154"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Stankevich, S., Kozlova, A., Zaitseva, E., and Levashenko, V. (2023, January 20\u201322). Multivariate Risk Assessment of Land Degradation by Remotely Sensed Data. Proceedings of the 2023 International Conference on Information and Digital Technologies (IDT), Zilina, Slovakia.","DOI":"10.1109\/IDT59031.2023.10194398"},{"key":"ref_13","unstructured":"Kharchenko, V., Illiashenko, O., Fesenko, H., and Babeshko, I. (2022). Communications in Computer and Information Science, Springer."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1295526","DOI":"10.3389\/fenvs.2023.1295526","article-title":"A cascade ensemble-learning model for the deployment at the edge: Case on missing IoT data recovery in environmental monitoring systems","volume":"11","author":"Izonin","year":"2023","journal-title":"Front. Environ. Sci."},{"key":"ref_15","unstructured":"Auzinger, W., Obelovska, K., Dronyuk, I., Pelekh, K., and Stolyarchuk, R. (2021). Proceedings of International Conference on Data Science and Applications, Springer."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"110425","DOI":"10.1016\/j.knosys.2023.110425","article-title":"Multi-view clustering guided by unconstrained non-negative matrix factorization","volume":"266","author":"Deng","year":"2023","journal-title":"Knowl.-Based Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"100423","DOI":"10.1016\/j.cosrev.2021.100423","article-title":"A survey on deep matrix factorizations","volume":"42","author":"Gillis","year":"2021","journal-title":"Comput. Sci. Rev."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.tifs.2015.11.010","article-title":"Data-driven recipe completion using machine learning methods","volume":"49","author":"Stock","year":"2016","journal-title":"Trends Food Sci. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"103399","DOI":"10.1016\/j.frl.2022.103399","article-title":"Robust forecasting with scaled independent component analysis","volume":"51","author":"Shu","year":"2023","journal-title":"Finance Res. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"104530","DOI":"10.1016\/j.jedc.2022.104530","article-title":"Identification of Structural VAR Models via Independent Component Analysis: A Performance Evaluation Study","volume":"144","author":"Moneta","year":"2022","journal-title":"J. Econ. Dyn. Control."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"109026","DOI":"10.1016\/j.ymssp.2022.109026","article-title":"Independent component analysis-based arbitrary polynomial chaos method for stochastic analysis of structures under limited observations","volume":"173","author":"Zhang","year":"2022","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.laa.2020.03.001","article-title":"Single-pass randomized algorithms for LU decomposition","volume":"595","author":"HLi","year":"2020","journal-title":"Linear Algebra its Appl."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"105767","DOI":"10.1016\/j.jcta.2023.105767","article-title":"Free fermions and Schur expansions of multi-Schur functions","volume":"198","author":"Iwao","year":"2023","journal-title":"J. Comb. Theory Ser. A"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"102571","DOI":"10.1016\/j.parco.2019.102571","article-title":"LU-Cholesky QR algorithms for thin QR decomposition","volume":"92","author":"Terao","year":"2020","journal-title":"Parallel Comput."},{"key":"ref_25","unstructured":"Trendafilov, N., and Hirose, K. (2023). International Encyclopedia of Education, Elsevier. [4th ed.]."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1016\/j.enganabound.2022.12.004","article-title":"Singular boundary method: A review and computer implementation aspects","volume":"147","author":"Fu","year":"2023","journal-title":"Eng. Anal. Bound. Elements"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"109126","DOI":"10.1016\/j.ress.2023.109126","article-title":"Support vector machine in structural reliability analysis: A review","volume":"233","author":"Roy","year":"2023","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1016\/j.eswa.2007.08.047","article-title":"A new training method for support vector machines: Clustering k-NN support vector machines","volume":"35","author":"Arslan","year":"2008","journal-title":"Expert Syst. Appl."},{"key":"ref_29","first-page":"180","article-title":"Towards an optimal support vector machine classifier using a parallel particle swarm optimization strategy","volume":"239","author":"Chen","year":"2014","journal-title":"Appl. Math. Comput."},{"key":"ref_30","unstructured":"Pineda, S., Morales, J.M., and Wogrin, S. (2023). Encyclopedia of Electrical and Electronic Power Engineering, Elsevier."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"110176","DOI":"10.1016\/j.asoc.2023.110176","article-title":"A comprehensive survey on design and application of autoencoder in deep learning","volume":"138","author":"Li","year":"2023","journal-title":"Appl. Soft Comput."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"108346","DOI":"10.1016\/j.sigpro.2021.108346","article-title":"Deep Architectures for Image Compression: A Critical Review","volume":"191","author":"Mishra","year":"2022","journal-title":"Signal Process."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"109146","DOI":"10.1016\/j.asoc.2022.109146","article-title":"A deep hypersphere approach to high-dimensional anomaly detection","volume":"125","author":"Zheng","year":"2022","journal-title":"Appl. Soft Comput."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Costa, M.C., Macedo, P., and Cruz, J.P. (2022, January 19\u201325). Neagging: An Aggregation Procedure Based on Normalized Entropy. Proceedings of the International Conference Of Numerical Analysis And Applied Mathematics ICNAAM 2020, Crete, Greece.","DOI":"10.1063\/5.0082228"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Bisikalo, O., Kharchenko, V., Kovtun, V., Krak, I., and Pavlov, S. (2023). Parameterization of the Stochastic Model for Evaluating Variable Small Data in the Shannon Entropy Basis. Entropy, 25.","DOI":"10.3390\/e25020184"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"102863","DOI":"10.1016\/j.advengsoft.2020.102863","article-title":"An efficient gradient projection method for structural topology optimization","volume":"149","author":"Zeng","year":"2020","journal-title":"Adv. Eng. Softw."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"107991","DOI":"10.1016\/j.ress.2021.107991","article-title":"Improvement of the cross-entropy method in high dimension for failure probability estimation through a one-dimensional projection without gradient estimation","volume":"216","author":"Morio","year":"2021","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"111160","DOI":"10.1016\/j.measurement.2022.111160","article-title":"Industrial process monitoring based on optimal active relative entropy components","volume":"197","author":"Liu","year":"2022","journal-title":"Measurement"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"124877","DOI":"10.1016\/j.jmaa.2020.124877","article-title":"Matrix trace inequalities related to the Tsallis relative entropies of real order","volume":"498","author":"Fujii","year":"2021","journal-title":"J. Math. Anal. Appl."},{"key":"ref_40","unstructured":"Makarichev, V., and Kharchenko, V. (2021). Radioelectronic and Computer Systems, National Aerospace University-Kharkiv Aviation Institute. no. 4."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4018\/IJCWT.305860","article-title":"Integrated Information Model of an Enterprise and Cybersecurity Management System","volume":"12","author":"Dotsenko","year":"2022","journal-title":"Int. J. Cyber Warf. Terror."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/12\/1567\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:26:02Z","timestamp":1760131562000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/12\/1567"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,21]]},"references-count":41,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["e25121567"],"URL":"https:\/\/doi.org\/10.3390\/e25121567","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,21]]}}}