{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T07:40:04Z","timestamp":1791445204862,"version":"4.3.3"},"reference-count":83,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2018,7,27]],"date-time":"2018-07-27T00:00:00Z","timestamp":1532649600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CCF-1217880"],"award-info":[{"award-number":["CCF-1217880"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"publisher","award":["W911NF-15-1-0479"],"award-info":[{"award-number":["W911NF-15-1-0479"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Recent work has focused on the problem of nonparametric estimation of information divergence functionals between two continuous random variables. Many existing approaches require either restrictive assumptions about the density support set or difficult calculations at the support set boundary which must be known a priori. The mean squared error (MSE) convergence rate of a leave-one-out kernel density plug-in divergence functional estimator for general bounded density support sets is derived where knowledge of the support boundary, and therefore, the boundary correction is not required. The theory of optimally weighted ensemble estimation is generalized to derive a divergence estimator that achieves the parametric rate when the densities are sufficiently smooth. Guidelines for the tuning parameter selection and the asymptotic distribution of this estimator are provided. Based on the theory, an empirical estimator of R\u00e9nyi-\u03b1 divergence is proposed that greatly outperforms the standard kernel density plug-in estimator in terms of mean squared error, especially in high dimensions. The estimator is shown to be robust to the choice of tuning parameters. We show extensive simulation results that verify the theoretical results of our paper. Finally, we apply the proposed estimator to estimate the bounds on the Bayes error rate of a cell classification problem.<\/jats:p>","DOI":"10.3390\/e20080560","type":"journal-article","created":{"date-parts":[[2018,7,27]],"date-time":"2018-07-27T12:20:03Z","timestamp":1532694003000},"page":"560","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Ensemble Estimation of Information Divergence \u2020"],"prefix":"10.3390","volume":"20","author":[{"given":"Kevin R.","family":"Moon","sequence":"first","affiliation":[{"name":"Genetics Department and Applied Math Program, Yale University, New Haven, CT 06520, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kumar","family":"Sricharan","sequence":"additional","affiliation":[{"name":"Intuit Inc., Mountain View, CA 94043, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristjan","family":"Greenewald","sequence":"additional","affiliation":[{"name":"IBM Research, Cambridge, MA 02142, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2531-9670","authenticated-orcid":false,"suffix":"III","given":"Alfred O.","family":"Hero","sequence":"additional","affiliation":[{"name":"Electrical Engineering and Computer Science Department, University of Michigan, Ann Arbor, MI 48109, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,7,27]]},"reference":[{"key":"ref_1","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, John Wiley & Sons."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1109\/34.476017","article-title":"Arbitrarily tight upper and lower bounds on the Bayesian probability of error","volume":"18","author":"Diep","year":"1996","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1109\/34.273728","article-title":"A tight upper bound on the Bayesian probability of error","volume":"16","author":"Hashlamoun","year":"1994","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Moon, K., Delouille, V., and Hero, A.O. (2015, January 9\u201312). Meta learning of bounds on the Bayes classifier error. Proceedings of the 2015 IEEE Signal Processing and Signal Processing Education Workshop (SP\/SPE), Salt Lake City, UT, USA.","DOI":"10.1109\/DSP-SPE.2015.7369520"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1214\/aoms\/1177729330","article-title":"A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations","volume":"23","author":"Chernoff","year":"1952","journal-title":"Ann. Math. Stat."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1109\/TSP.2015.2477805","article-title":"Empirically Estimable Classification Bounds Based on a New Divergence Measure","volume":"64","author":"Berisha","year":"2016","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_7","unstructured":"Moon, K.R., and Hero, A.O. (2014, January 8\u201313). Multivariate f-Divergence Estimation With Confidence. Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS 2014), Montreal, QC, Canada."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Gliske, S.V., Moon, K.R., Stacey, W.C., and Hero, A.O. (2016, January 20\u201325). The intrinsic value of HFO features as a biomarker of epileptic activity. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.","DOI":"10.1109\/ICASSP.2016.7472887"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Loh, P.-L. (2017). On Lower Bounds for Statistical Learning Theory. Entropy, 19.","DOI":"10.3390\/e19110617"},{"key":"ref_10","unstructured":"P\u00f3czos, B., and Schneider, J.G. (2011, January 11\u201313). On the estimation of alpha-divergences. Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA."},{"key":"ref_11","unstructured":"Oliva, J., P\u00f3czos, B., and Schneider, J. (2013, January 16\u201321). Distribution to distribution regression. Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_12","unstructured":"Szab\u00f3, Z., Gretton, A., P\u00f3czos, B., and Sriperumbudur, B. (2015, January 9\u201312). Two-stage sampled learning theory on distributions. In Proceeding of The 18th International Conference on Artificial Intelligence and Statistics, San Diego, CA, USA."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"A3","DOI":"10.1051\/swsc\/2015043","article-title":"Image patch analysis of sunspots and active regions. II. Clustering via matrix factorization","volume":"6","author":"Moon","year":"2016","journal-title":"J. Space Weather Space Clim."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"A2","DOI":"10.1051\/swsc\/2015044","article-title":"Image patch analysis of sunspots and active regions. I. Intrinsic dimension and correlation analysis","volume":"6","author":"Moon","year":"2016","journal-title":"J. Space Weather Space Clim."},{"key":"ref_15","first-page":"1265","article-title":"A divisive information theoretic feature clustering algorithm for text classification","volume":"3","author":"Dhillon","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_16","first-page":"1705","article-title":"Clustering with Bregman divergences","volume":"6","author":"Banerjee","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Lewi, J., Butera, R., and Paninski, L. (2006, January 4\u20139). Real-time adaptive information-theoretic optimization of neurophysiology experiments. Proceedings of the 19th International Conference on Neural Information Processing Systems (NIPS 2006), Vancouver, BC, Canada.","DOI":"10.7551\/mitpress\/7503.003.0112"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1318","DOI":"10.1109\/36.477187","article-title":"An extension of the Jeffreys-Matusita distance to multiclass cases for feature selection","volume":"33","author":"Bruzzone","year":"1995","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_19","unstructured":"Guorong, X., Peiqi, C., and Minhui, W. (1996, January 25\u201329). Bhattacharyya distance feature selection. Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1233","DOI":"10.1080\/02664763.2013.864262","article-title":"Variable selection via penalized minimum \u03c6-divergence estimation in logistic regression","volume":"41","author":"Sakate","year":"2014","journal-title":"J. Appl. Stat."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1109\/97.923043","article-title":"Blind source separation using Renyi\u2019s mutual information","volume":"8","author":"Hild","year":"2001","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1859","DOI":"10.1162\/089976602760128045","article-title":"Robust blind source separation by beta divergence","volume":"14","author":"Mihoko","year":"2002","journal-title":"Neural Comput."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1109\/TMI.2010.2086464","article-title":"Total Bregman divergence and its applications to DTI analysis","volume":"30","author":"Vemuri","year":"2011","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Hamza, A.B., and Krim, H. (2003, January 7\u20139). Image registration and segmentation by maximizing the Jensen-R\u00e9nyi divergence. Proceedings of the 4th International Workshop Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR 2003), Lisbon, Portugal.","DOI":"10.1007\/978-3-540-45063-4_10"},{"key":"ref_25","unstructured":"Liu, G., Xia, G., Yang, W., and Xue, N. (2014, January 13\u201318). SAR image segmentation via non-local active contours. Proceedings of the 2014 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Quebec City, QC, Canada."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Korzhik, V., and Fedyanin, I. (2015, January 3\u20135). Steganographic applications of the nearest-neighbor approach to Kullback-Leibler divergence estimation. Proceedings of the 2015 Third International Conference on Digital Information, Networking, and Wireless Communications (DINWC), Moscow, Russia.","DOI":"10.1109\/DINWC.2015.7054231"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1016\/j.sigpro.2012.09.003","article-title":"Divergence measures for statistical data processing\u2013An annotated bibliography","volume":"93","author":"Basseville","year":"2013","journal-title":"Signal Process."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3390\/e12061532","article-title":"Families of alpha-beta-and gamma-divergences: Flexible and robust measures of similarities","volume":"12","author":"Cichocki","year":"2010","journal-title":"Entropy"},{"key":"ref_29","first-page":"299","article-title":"Information-type measures of difference of probability distributions and indirect observations","volume":"2","author":"Csiszar","year":"1967","journal-title":"Stud. Sci. Math. Hungar."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1111\/j.2517-6161.1966.tb00626.x","article-title":"A general class of coefficients of divergence of one distribution from another","volume":"28","author":"Ali","year":"1966","journal-title":"J. R. Stat. Soc. Ser. B Stat. Methodol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_32","unstructured":"R\u00e9nyi, A. (July, January 20). On measures of entropy and information. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1515\/crll.1909.136.210","article-title":"Neue Begr\u00fcndung der Theorie quadratischer Formen von unendlichvielen Ver\u00e4nderlichen","volume":"136","author":"Hellinger","year":"1909","journal-title":"J. Rein. Angew. Math."},{"key":"ref_34","first-page":"401","article-title":"On a measure of divergence between two multinomial populations","volume":"7","author":"Bhattacharyya","year":"1946","journal-title":"Indian J. Stat."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Silva, J.F., and Parada, P.A. (2012, January 1\u20136). Shannon entropy convergence results in the countable infinite case. Proceedings of the 2012 IEEE International Symposium on Information Theory Proceedings (ISIT), Cambridge, MA, USA.","DOI":"10.1109\/ISIT.2012.6283535"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1002\/rsa.10019","article-title":"Convergence properties of functional estimates for discrete distributions","volume":"19","author":"Antos","year":"2001","journal-title":"Random Struct. Algorithms"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Valiant, G., and Valiant, P. (2011, January 6\u20138). Estimating the unseen: An n\/log (n)-sample estimator for entropy and support size, shown optimal via new CLTs. Proceedings of the 43rd Annual ACM Symposium on Theory of Computing, San Jose, CA, USA.","DOI":"10.1145\/1993636.1993727"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2835","DOI":"10.1109\/TIT.2015.2412945","article-title":"Minimax estimation of functionals of discrete distributions","volume":"61","author":"Jiao","year":"2015","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"6774","DOI":"10.1109\/TIT.2017.2733537","article-title":"Maximum likelihood estimation of functionals of discrete distributions","volume":"63","author":"Jiao","year":"2017","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Valiant, G., and Valiant, P. (2011, January 22\u201325). The power of linear estimators. Proceedings of the 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science (FOCS), Palm Springs, CA, USA.","DOI":"10.1109\/FOCS.2011.81"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1162\/089976603321780272","article-title":"Estimation of entropy and mutual information","volume":"15","author":"Paninski","year":"2003","journal-title":"Neural Comput."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2200","DOI":"10.1109\/TIT.2004.833360","article-title":"Estimating entropy on m bins given fewer than m samples","volume":"50","author":"Paninski","year":"2004","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Alba-Fern\u00e1ndez, M.V., Jim\u00e9nez-Gamero, M.D., and Ariza-L\u00f3pez, F.J. (2018). Minimum Penalized \u03d5-Divergence Estimation under Model Misspecification. Entropy, 20.","DOI":"10.3390\/e20050329"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1109\/18.30996","article-title":"Entropy expressions and their estimators for multivariate distributions","volume":"35","author":"Ahmed","year":"1989","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1016\/j.jmva.2003.10.003","article-title":"Estimation of the entropy of a multivariate normal distribution","volume":"92","author":"Misra","year":"2005","journal-title":"J. Multivar. Anal."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"818","DOI":"10.3390\/e12040818","article-title":"Parametric Bayesian estimation of differential entropy and relative entropy","volume":"12","author":"Gupta","year":"2010","journal-title":"Entropy"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"650","DOI":"10.3390\/e13030650","article-title":"K-nearest neighbor based consistent entropy estimation for hyperspherical distributions","volume":"13","author":"Li","year":"2011","journal-title":"Entropy"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2392","DOI":"10.1109\/TIT.2009.2016060","article-title":"Divergence estimation for multidimensional densities via k-nearest-neighbor distances","volume":"55","author":"Wang","year":"2009","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1315","DOI":"10.1109\/18.761290","article-title":"Estimation of the information by an adaptive partitioning of the observation space","volume":"45","author":"Darbellay","year":"1999","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"3180","DOI":"10.1016\/j.jspi.2010.04.011","article-title":"Information divergence estimation based on data-dependent partitions","volume":"140","author":"Silva","year":"2010","journal-title":"J. Stat. Plan. Inference"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"2089","DOI":"10.1016\/j.jspi.2013.08.007","article-title":"Information dependency: Strong consistency of Darbellay\u2013Vajda partition estimators","volume":"143","author":"Le","year":"2013","journal-title":"J. Stat. Plan. Inference"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"3064","DOI":"10.1109\/TIT.2005.853314","article-title":"Divergence estimation of continuous distributions based on data-dependent partitions","volume":"51","author":"Wang","year":"2005","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1109\/MSP.2002.1028355","article-title":"Applications of entropic spanning graphs","volume":"19","author":"Hero","year":"2002","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Moon, K.R., and Hero, A.O. (July, January 29). Ensemble estimation of multivariate f-divergence. Proceedings of the 2014 IEEE International Symposium on Information Theory (ISIT), Honolulu, HI, USA.","DOI":"10.1109\/ISIT.2014.6874854"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Moon, K.R., Sricharan, K., Greenewald, K., and Hero, A.O. (2016, January 10\u201315). Improving convergence of divergence functional ensemble estimators. Proceedings of the 2016 IEEE International Symposium on Information Theory (ISIT), Barcelona, Spain.","DOI":"10.1109\/ISIT.2016.7541476"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"5847","DOI":"10.1109\/TIT.2010.2068870","article-title":"Estimating divergence functionals and the likelihood ratio by convex risk minimization","volume":"56","author":"Nguyen","year":"2010","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_57","unstructured":"Krishnamurthy, A., Kandasamy, K., Poczos, B., and Wasserman, L. (2014, January 21\u201326). Nonparametric Estimation of Renyi Divergence and Friends. Proceedings of the 31st International Conference on Machine Learning, Beijing, China."},{"key":"ref_58","unstructured":"Singh, S., and P\u00f3czos, B. (2014, January 21\u201326). Generalized exponential concentration inequality for R\u00e9nyi divergence estimation. Proceedings of the 31st International Conference on Machine Learning (ICML-14), Beijing, China."},{"key":"ref_59","unstructured":"Singh, S., and P\u00f3czos, B. (2014, January 8\u201313). Exponential Concentration of a Density Functional Estimator. Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS 2014), Montreal, QC, Canada."},{"key":"ref_60","unstructured":"Kandasamy, K., Krishnamurthy, A., Poczos, B., Wasserman, L., and Robins, J. (2015, January 7\u201312). Nonparametric von Mises Estimators for Entropies, Divergences and Mutual Informations. Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS 2015), Montreal, QC, Canada."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"H\u00e4rdle, W. (1990). Applied Nonparametric Regression, Cambridge University Press.","DOI":"10.1017\/CCOL0521382483"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1080\/02331889508802500","article-title":"Asymptotic normality of L1-error in density estimation","volume":"26","author":"Berlinet","year":"1995","journal-title":"Statistics"},{"key":"ref_63","first-page":"3","article-title":"Asymptotic normality of relative entropy in multivariate density estimation","volume":"41","author":"Berlinet","year":"1997","journal-title":"Publ. l\u2019Inst. Stat. l\u2019Univ. Paris"},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"1071","DOI":"10.1214\/aos\/1176342558","article-title":"On some global measures of the deviations of density function estimates","volume":"1","author":"Bickel","year":"1973","journal-title":"Ann. Stat."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"4374","DOI":"10.1109\/TIT.2013.2251456","article-title":"Ensemble estimators for multivariate entropy estimation","volume":"59","author":"Sricharan","year":"2013","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_66","unstructured":"Berrett, T.B., Samworth, R.J., and Yuan, M. (2017). Efficient multivariate entropy estimation via k-nearest neighbour distances. arXiv."},{"key":"ref_67","first-page":"9","article-title":"Sample estimate of the entropy of a random vector","volume":"23","author":"Kozachenko","year":"1987","journal-title":"Probl. Peredachi Inf."},{"key":"ref_68","unstructured":"Hansen, B.E. (2009). Lecture Notes on Nonparametrics."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"1229","DOI":"10.3390\/e13071229","article-title":"On accuracy of PDF divergence estimators and their applicability to representative data sampling","volume":"13","author":"Budka","year":"2011","journal-title":"Entropy"},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1214\/aos\/1176345462","article-title":"The jackknife estimate of variance","volume":"9","author":"Efron","year":"1981","journal-title":"Ann. Stat."},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Wisler, A., Moon, K., and Berisha, V. (2018, January 15\u201320). Direct ensemble estimation of density functionals. Proceedings of the 2018 International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Calgary, AB, Canada.","DOI":"10.1109\/ICASSP.2018.8462308"},{"key":"ref_72","unstructured":"Moon, K.R., Sricharan, K., Greenewald, K., and Hero, A.O. (2016). Nonparametric Ensemble Estimation of Distributional Functionals. arXiv."},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"1663","DOI":"10.1016\/j.cell.2015.11.013","article-title":"Transcriptional heterogeneity and lineage commitment in myeloid progenitors","volume":"163","author":"Paul","year":"2015","journal-title":"Cell"},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1093\/nar\/28.1.27","article-title":"KEGG: Kyoto encyclopedia of genes and genomes","volume":"28","author":"Kanehisa","year":"2000","journal-title":"Nucleic Acids Res."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"D457","DOI":"10.1093\/nar\/gkv1070","article-title":"KEGG as a reference resource for gene and protein annotation","volume":"44","author":"Kanehisa","year":"2015","journal-title":"Nucleic Acids Res."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"D353","DOI":"10.1093\/nar\/gkw1092","article-title":"KEGG: New perspectives on genomes, pathways, diseases and drugs","volume":"45","author":"Kanehisa","year":"2016","journal-title":"Nucleic Acids Res."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1016\/j.cell.2018.05.061","article-title":"Recovering Gene Interactions from Single-Cell Data Using Data Diffusion","volume":"174","author":"Sharma","year":"2018","journal-title":"Cell"},{"key":"ref_78","doi-asserted-by":"crossref","unstructured":"Moon, K.R., Sricharan, K., and Hero, A.O. (2017). Ensemble Estimation of Distributional Functionals via k-Nearest Neighbors. arXiv.","DOI":"10.1109\/ISIT.2017.8007086"},{"key":"ref_79","doi-asserted-by":"crossref","unstructured":"Durrett, R. (2010). Probability: Theory and Examples, Cambridge University Press.","DOI":"10.1017\/CBO9780511779398"},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Gut, A. (2012). Probability: A Graduate Course, Springer.","DOI":"10.1007\/978-1-4614-4708-5"},{"key":"ref_81","unstructured":"Munkres, J. (2000). Topology, Prentice Hall."},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Evans, L.C. (2010). Partial Differential Equations, American Mathematical Society.","DOI":"10.1090\/gsm\/019"},{"key":"ref_83","doi-asserted-by":"crossref","unstructured":"Gilbarg, D., and Trudinger, N.S. (2001). Elliptic Partial Differential Equations of Second Order, Springer.","DOI":"10.1007\/978-3-642-61798-0"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/8\/560\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:14:52Z","timestamp":1760195692000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/8\/560"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,27]]},"references-count":83,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2018,8]]}},"alternative-id":["e20080560"],"URL":"https:\/\/doi.org\/10.3390\/e20080560","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,27]]}}}