{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,14]],"date-time":"2025-11-14T07:40:24Z","timestamp":1763106024085,"version":"build-2065373602"},"reference-count":68,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2024,8,21]],"date-time":"2024-08-21T00:00:00Z","timestamp":1724198400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62361146853","62071115","62371129","2024A05"],"award-info":[{"award-number":["62361146853","62071115","62371129","2024A05"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Research Fund of the National Mobile Communications Research Laboratory, Southeast University","award":["62361146853","62071115","62371129","2024A05"],"award-info":[{"award-number":["62361146853","62071115","62371129","2024A05"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The complete characterization of the almost-entropic region yields rate regions for network coding problems. However, this characterization is difficult and open. In this paper, we propose a novel algorithm to determine whether an arbitrary vector in the entropy space is entropic or not, by parameterizing and generating probability mass functions by neural networks. Given a target vector, the algorithm minimizes the normalized distance between the target vector and the generated entropic vector by training the neural network. The algorithm reveals the entropic nature of the target vector, and obtains the underlying distribution, accordingly. The proposed algorithm was further implemented with convolutional neural networks, which naturally fit the structure of joint probability mass functions, and accelerate the algorithm with GPUs. Empirical results demonstrate improved normalized distances and convergence performances compared with prior works. We also conducted optimizations of the Ingleton score and Ingleton violation index, where a new lower bound of the Ingleton violation index was obtained. An inner bound of the almost-entropic region with four random variables was constructed with the proposed method, presenting the current best inner bound measured by the volume ratio. The potential of a computer-aided approach to construct achievable schemes for network coding problems using the proposed method is discussed.<\/jats:p>","DOI":"10.3390\/e26080711","type":"journal-article","created":{"date-parts":[[2024,8,26]],"date-time":"2024-08-26T03:32:01Z","timestamp":1724643121000},"page":"711","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Optimizing Distributions for Associated Entropic Vectors via Generative Convolutional Neural Networks"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-9505-7905","authenticated-orcid":false,"given":"Shuhao","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Information Science and Engineering, Southeast University, Nanjing 211189, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4155-0685","authenticated-orcid":false,"given":"Nan","family":"Liu","sequence":"additional","affiliation":[{"name":"National Mobile Communications Research Laboratory, Southeast University, Nanjing 211189, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4408-6419","authenticated-orcid":false,"given":"Wei","family":"Kang","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Southeast University, Nanjing 211189, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3170-3190","authenticated-orcid":false,"given":"Haim","family":"Permuter","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beersheba 8410501, Israel"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"5625","DOI":"10.1109\/TIT.2012.2201340","article-title":"An implicit characterization of the achievable rate region for acyclic multisource multisink network coding","volume":"58","author":"Yan","year":"2012","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Hassibi, B., and Shadbakht, S. (2007, January 1\u20136). Normalized entropy vectors, network information theory and convex optimization. Proceedings of the 2007 IEEE Information Theory Workshop on Information Theory for Wireless Networks, Bergen, Norway.","DOI":"10.1109\/ITWITWN.2007.4318051"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1982","DOI":"10.1109\/18.641561","article-title":"A non-Shannon-type conditional inequality of information quantities","volume":"43","author":"Zhang","year":"1997","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_4","first-page":"236","article-title":"Piecewise linear conditional information inequality","volume":"52","year":"2005","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chen, Q., and Yeung, R.W. (2012, January 3\u20137). Characterizing the entropy function region via extreme rays. Proceedings of the 2012 IEEE Information Theory Workshop, Lausanne, Switzerland.","DOI":"10.1109\/ITW.2012.6404674"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Tiwari, H., and Thakor, S. (2019, January 25\u201328). On characterization of entropic vectors at the boundary of almost entropic cones. Proceedings of the 2019 IEEE Information Theory Workshop (ITW), Visby, Sweden.","DOI":"10.1109\/ITW44776.2019.8989116"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Thakor, S., and Saleem, D. (2022, January 1\u20139). A Quasi-Uniform Approach to Characterizing the Boundary of the Almost Entropic Region. Proceedings of the 2022 IEEE Information Theory Workshop (ITW), Mumbai, India.","DOI":"10.1109\/ITW54588.2022.9965832"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1440","DOI":"10.1109\/18.681320","article-title":"On characterization of entropy function via information inequalities","volume":"44","author":"Zhang","year":"1998","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4470","DOI":"10.1109\/TIT.2008.928963","article-title":"Dualities between entropy functions and network codes","volume":"54","author":"Chan","year":"2008","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Matus, F. (2007, January 24\u201329). Infinitely many information inequalities. Proceedings of the 2007 IEEE International Symposium on Information Theory, Nice, France.","DOI":"10.1109\/ISIT.2007.4557201"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"7685","DOI":"10.1109\/TIT.2022.3190800","article-title":"The undecidability of conditional affine information inequalities and conditional independence implication with a binary constraint","volume":"68","author":"Li","year":"2022","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3493","DOI":"10.1109\/TIT.2023.3247570","article-title":"Undecidability of network coding, conditional information inequalities, and conditional independence implication","volume":"69","author":"Li","year":"2023","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"442","DOI":"10.1006\/jcss.1999.1677","article-title":"Inequalities for Shannon entropy and Kolmogorov complexity","volume":"60","author":"Hammer","year":"2000","journal-title":"J. Comput. Syst. Sci."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Dougherty, R., Freiling, C., and Zeger, K. (2006, January 9\u201314). Six new non-Shannon information inequalities. Proceedings of the 2006 IEEE International Symposium on Information Theory, Seattle, WA, USA.","DOI":"10.1109\/ISIT.2006.261840"},{"key":"ref_15","unstructured":"Dougherty, R., Freiling, C., and Zeger, K. (2009). Linear rank inequalities on five or more variables. arXiv."},{"key":"ref_16","unstructured":"Dougherty, R., Freiling, C., and Zeger, K. (2011). Non-Shannon information inequalities in four random variables. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Dougherty, R. (July, January 29). Computations of linear rank inequalities on six variables. Proceedings of the 2014 IEEE International Symposium on Information Theory, Honolulu, HI, USA.","DOI":"10.1109\/ISIT.2014.6875348"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hassibi, B., and Shadbakht, S. (2007, January 24\u201329). On a construction of entropic vectors using lattice-generated distributions. Proceedings of the 2007 IEEE International Symposium on Information Theory, Nice, France.","DOI":"10.1109\/ISIT.2007.4557096"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Shadbakht, S., and Hassibi, B. (2010, January 13\u201318). MCMC methods for entropy optimization and nonlinear network coding. Proceedings of the 2010 IEEE International Symposium on Information Theory, Austin, TX, USA.","DOI":"10.1109\/ISIT.2010.5513737"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"6356","DOI":"10.1109\/TIT.2011.2165817","article-title":"A recursive construction of the set of binary entropy vectors and related algorithmic inner bounds for the entropy region","volume":"57","author":"Walsh","year":"2011","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"6007","DOI":"10.1109\/TIT.2016.2601598","article-title":"Entropy region and convolution","volume":"62","author":"Csirmaz","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Liu, Y., and Walsh, J.M. (October, January 29). Non-isomorphic distribution supports for calculating entropic vectors. Proceedings of the 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, IL, USA.","DOI":"10.1109\/ALLERTON.2015.7447064"},{"key":"ref_23","unstructured":"Liu, Y., and Walsh, J.M. (2015). Mapping the region of entropic vectors with support enumeration & information geometry. arXiv."},{"key":"ref_24","unstructured":"Liu, Y. (2016). Extremal Entropy: Information Geometry, Numerical Entropy Mapping, and Machine Learning Application of Associated Conditional Independences. [Ph.D. Thesis, Drexel University]."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Walsh, J.M., and Trofimoff, A.E. (2019, January 25\u201328). On designing probabilistic supports to map the entropy region. Proceedings of the 2019 IEEE Information Theory Workshop (ITW), Visby, Sweden.","DOI":"10.1109\/ITW44776.2019.8989076"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1109\/TCOMM.2020.3030055","article-title":"Inner bounds for the almost entropic region and network code construction","volume":"69","author":"Alam","year":"2020","journal-title":"IEEE Trans. Commun."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"874","DOI":"10.1109\/TCOMM.2020.3040401","article-title":"Recursive algorithm to verify quasi-uniform entropy vectors and its applications","volume":"69","author":"Saleem","year":"2020","journal-title":"IEEE Trans. Commun."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liu, S., and Chen, Q. (2023, January 25\u201330). Entropy functions on two-dimensional faces of polymatroidal region of degree four. Proceedings of the 2023 IEEE International Symposium on Information Theory (ISIT), Taipei, Taiwan.","DOI":"10.1109\/ISIT54713.2023.10206814"},{"key":"ref_29","unstructured":"Chan, H.L., and Yeung, R.W. (July, January 27). A combinatorial approach to information inequalities. Proceedings of the 1999 Information Theory and Networking Workshop (Cat. No. 99EX371), Metsovo, Greece."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1992","DOI":"10.1109\/TIT.2002.1013138","article-title":"On a relation between information inequalities and group theory","volume":"48","author":"Chan","year":"2002","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Chan, T.H. (2007, January 24\u201329). Group characterizable entropy functions. Proceedings of the 2007 IEEE International Symposium on Information Theory, Nice, France.","DOI":"10.1109\/ISIT.2007.4557275"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1109\/TIT.2016.2627530","article-title":"On Ingleton-violating finite groups","volume":"63","author":"Mao","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Boston, N., and Nan, T.T. (2012, January 1\u20135). Large violations of the Ingleton inequality. Proceedings of the 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, IL, USA.","DOI":"10.1109\/Allerton.2012.6483410"},{"key":"ref_34","unstructured":"Nan, T.T. (2015). Entropy Regions and the Four-Atom Conjecture. [Ph.D. Thesis, The University of Wisconsin-Madison]."},{"key":"ref_35","first-page":"916","article-title":"Violations of the Ingleton inequality and revising the four-atom conjecture","volume":"56","author":"Boston","year":"2020","journal-title":"Kybernetika"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1949","DOI":"10.1109\/TIT.2007.896862","article-title":"Networks, matroids, and non-Shannon information inequalities","volume":"53","author":"Dougherty","year":"2007","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1109\/TIT.2016.2628791","article-title":"Multilevel diversity coding systems: Rate regions, codes, computation, & forbidden minors","volume":"63","author":"Li","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"7283","DOI":"10.1109\/TIT.2017.2745620","article-title":"On multi-source networks: Enumeration, rate region computation, and hierarchy","volume":"63","author":"Li","year":"2017","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Chen, Q., Cheng, M., and Bai, B. (2021). Matroidal Entropy Functions: A Quartet of Theories of Information, Matroid, Design, and Coding. Entropy, 23.","DOI":"10.3390\/e23030323"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Chen, Q., Cheng, M., and Bai, B. (2024). Matroidal entropy functions: Constructions, characterizations and representations. IEEE Trans. Inf. Theory.","DOI":"10.1109\/TIT.2024.3355942"},{"key":"ref_41","first-page":"149","article-title":"Representation of matroids","volume":"23","author":"Ingleton","year":"1971","journal-title":"Comb. Math. Its Appl."},{"key":"ref_42","first-page":"429","article-title":"The dealer\u2019s random bits in perfect secret sharing schemes","volume":"32","author":"Csirmaz","year":"1996","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_43","unstructured":"Shadbakht, S. (2011). Entropy Region and Network Information Theory. [Ph.D. Thesis, California Institute of Technology]."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"4777","DOI":"10.1109\/TIT.2023.3264674","article-title":"Neural estimation and optimization of directed information over continuous spaces","volume":"69","author":"Tsur","year":"2023","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Tsur, D., Huleihel, B., and Permuter, H. (2023, January 25\u201330). Rate distortion via constrained estimated mutual information minimization. Proceedings of the 2023 IEEE International Symposium on Information Theory (ISIT), Taipei, Taiwan.","DOI":"10.1109\/ISIT54713.2023.10206867"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1652","DOI":"10.1109\/TIT.2023.3335863","article-title":"Data-driven optimization of directed information over discrete alphabets","volume":"70","author":"Tsur","year":"2023","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1016\/j.neunet.2022.09.021","article-title":"Neural network-based event-triggered data-driven control of disturbed nonlinear systems with quantized input","volume":"156","author":"Wang","year":"2022","journal-title":"Neural Netw."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Wang, M., Zhu, S., Shen, M., Liu, X., and Wen, S. (2024). Fault-Tolerant Synchronization for Memristive Neural Networks with Multiple Actuator Failures. IEEE Trans. Cybern., 1\u201310.","DOI":"10.1109\/TCYB.2024.3363470"},{"key":"ref_49","unstructured":"Ziegler, G.M. (2012). Lectures on Polytopes, Springer Science & Business Media."},{"key":"ref_50","unstructured":"Fukuda, K., and Prodon, A. (1995, January 3\u20135). Double description method revisited. Proceedings of the Franco-Japanese and Franco-Chinese Conference on Combinatorics and Computer Science, Brest, France."},{"key":"ref_51","unstructured":"Fukuda, K. (2024, May 18). cdd, cddplus and cddlib. Available online: https:\/\/people.inf.ethz.ch\/fukudak\/cdd_home\/."},{"key":"ref_52","unstructured":"Troffaes, M. (2024, May 18). pycddlib. Available online: https:\/\/github.com\/mcmtroffaes\/pycddlib."},{"key":"ref_53","first-page":"51","article-title":"The double description method","volume":"2","author":"Motzkin","year":"1953","journal-title":"Contrib. Theory Games"},{"key":"ref_54","unstructured":"Yeung, R.W. (2008). Information Theory and Network Coding, Springer Science & Business Media."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/S0019-9958(78)91063-X","article-title":"Polymatroidal dependence structure of a set of random variables","volume":"39","author":"Fujishige","year":"1978","journal-title":"Inf. Control"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1016\/0893-6080(89)90020-8","article-title":"Multilayer feedforward networks are universal approximators","volume":"2","author":"Hornik","year":"1989","journal-title":"Neural Netw."},{"key":"ref_57","unstructured":"Bertsekas, D.P. (1998). Nonlinear Programming, Athena Scientific."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Shalev-Shwartz, S., and Ben-David, S. (2014). Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press.","DOI":"10.1017\/CBO9781107298019"},{"key":"ref_59","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). Pytorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst., 32."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1017\/S0963548300001644","article-title":"Conditional independences among four random variables I","volume":"4","year":"1995","journal-title":"Comb. Probab. Comput."},{"key":"ref_61","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_62","unstructured":"Alam, S., Thakor, S., and Abbas, S. (2024, May 18). Numerical Data: Inner Bounds for the Almost Entropic Region and Network Code Construction. Available online: https:\/\/github.com\/mdsultanalam\/NumericalDataInnerBoundsNetworkCodeConstruction."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"B\u00fceler, B., Enge, A., and Fukuda, K. (2000). Exact volume computation for polytopes: A practical study. Polytopes\u2014Combinatorics and Computation, Birkh\u00e4user.","DOI":"10.1007\/978-3-0348-8438-9_6"},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1145\/235815.235821","article-title":"The quickhull algorithm for convex hulls","volume":"22","author":"Barber","year":"1996","journal-title":"ACM Trans. Math. Softw. (TOMS)"},{"key":"ref_65","unstructured":"Barber, C.B., Dobkin, D.P., and Huhdanpaa, H. (2024, May 18). Qhull. Available online: http:\/\/www.qhull.org."},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Avis, D. (2000). A Revised Implementation of the Reverse Search Vertex Enumeration Algorithm. Polytopes\u2014Combinatorics and Computation, Birkh\u00e4user.","DOI":"10.1007\/978-3-0348-8438-9_9"},{"key":"ref_67","unstructured":"Avis, D. (2024, May 18). lrs. Available online: http:\/\/cgm.cs.mcgill.ca\/~avis\/C\/lrs.html."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"1924","DOI":"10.1109\/18.641556","article-title":"A framework for linear information inequalities","volume":"43","author":"Yeung","year":"1997","journal-title":"IEEE Trans. Inf. Theory"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/8\/711\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:40:38Z","timestamp":1760110838000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/8\/711"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,21]]},"references-count":68,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["e26080711"],"URL":"https:\/\/doi.org\/10.3390\/e26080711","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2024,8,21]]}}}