{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T00:22:05Z","timestamp":1791418925259,"version":"4.3.3"},"reference-count":126,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2015,6,12]],"date-time":"2015-06-12T00:00:00Z","timestamp":1434067200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The main content of this review article is first to review the main inference tools using Bayes rule, the maximum entropy principle (MEP), information theory, relative entropy and the Kullback\u2013Leibler (KL) divergence, Fisher information and its corresponding geometries. For each of these tools, the precise context of their use is described. The second part of the paper is focused on the ways these tools have been used in data, signal and image processing and in the inverse problems, which arise in different physical sciences and engineering applications. A few examples of the applications are described: entropy in independent components analysis (ICA) and in blind source separation, Fisher information in data model selection, different maximum entropy-based methods in time series spectral estimation and in linear inverse problems and, finally, the Bayesian inference for general inverse problems. Some original materials concerning the approximate Bayesian computation (ABC) and, in particular, the variational Bayesian approximation (VBA) methods are also presented. VBA is used for proposing an alternative Bayesian computational tool to the classical Markov chain Monte Carlo (MCMC) methods. We will also see that VBA englobes joint maximum a posteriori (MAP), as well as the different expectation-maximization (EM) algorithms as particular cases.<\/jats:p>","DOI":"10.3390\/e17063989","type":"journal-article","created":{"date-parts":[[2015,6,12]],"date-time":"2015-06-12T10:56:00Z","timestamp":1434106560000},"page":"3989-4027","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":38,"title":["Entropy, Information Theory, Information Geometry and Bayesian Inference in Data, Signal and Image Processing and Inverse Problems"],"prefix":"10.3390","volume":"17","author":[{"given":"Ali","family":"Mohammad-Djafari","sequence":"first","affiliation":[{"name":"Laboratoire des Signaux et Syst\u00e8mes, UMR 8506 CNRS-SUPELEC-UNIV PARIS SUD, SUPELEC, Plateau de Moulon, 3 rue Juliot-Curie, 91192 Gif-sur-Yvette, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2015,6,12]]},"reference":[{"key":"ref_1","first-page":"43","article-title":"Bayesian or Laplacian inference, entropy and information theory and information geometry in data and signal processing","volume":"1641","year":"2014","journal-title":"AIP Conf. Proc"},{"key":"ref_2","first-page":"370","article-title":"An Essay toward Solving a Problem in the Doctrine of Chances","volume":"53","author":"Bayes","year":"1763","journal-title":"Philos. Trans"},{"key":"ref_3","first-page":"621","article-title":"M\u00e9moire sur la probabilit\u00e9 des causes par les \u00e9v\u00e8nements","volume":"6","year":"1774","journal-title":"M\u00e9moires de l\u2019Academie Royale des Sciences Present\u00e9s par Divers Savan"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A Mathematical Theory of Communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J"},{"key":"ref_5","unstructured":"Hadamard, J. (1908). M\u00e9moire sur le probl\u00e8me d\u2019analyse relatif \u00e0 l\u2019\u00e9quilibre des plaques \u00e9lastiques encastr\u00e9es, Imprimerie nationale."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1103\/PhysRev.106.620","article-title":"Information Theory Statistical Mechanics","volume":"106","author":"Jaynes","year":"1957","journal-title":"Phys. Rev"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1103\/PhysRev.108.171","article-title":"Information Theory and Statistical Mechanics II","volume":"108","author":"Jaynes","year":"1957","journal-title":"Phys. Rev"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1109\/TSSC.1968.300117","article-title":"Prior Probabilities","volume":"4","author":"Jaynes","year":"1968","journal-title":"IEEE Trans. Syst. Sci. Cybern"},{"key":"ref_9","unstructured":"Kullback, S. (1959). Information Theory and Statistics, Wiley."},{"key":"ref_10","first-page":"309","article-title":"On the mathematical foundations of theoretical statistics","volume":"222","author":"Fisher","year":"1922","journal-title":"Philos. Trans. R. Stat. Soc. A"},{"key":"ref_11","first-page":"81","article-title":"Information and accuracy attainable in the estimation of statistical parameters","volume":"37","author":"Rao","year":"1945","journal-title":"Bull. Culcutta Math. Soc"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Chapelle, O., Sch\u00f6lkopf, B., and Zien, A. (2006). Semi-supervised Learning, MIT press.","DOI":"10.7551\/mitpress\/9780262033589.001.0001"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1109\/18.61115","article-title":"Divergence Measures Based on the Shannon Entropy","volume":"37","author":"Lin","year":"1991","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1007\/s00440-004-0344-0","article-title":"Fisher Information Inequalities and the Central Limit Theorem","volume":"129","author":"Johnson","year":"2004","journal-title":"Probab. Theory Relat. Fields"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Berger, J. (1985). Statistical Decision Theory and Bayesian Analysis, Springer-Verlag. [2nd ed].","DOI":"10.1007\/978-1-4757-4286-2"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Gelman, A., Carlin, J.B., Stern, H.S., and Rubin, D.B. (2003). Bayesian Data Analysis, Chapman and Hall\/CRC. [2nd ed].","DOI":"10.1201\/9780429258480"},{"key":"ref_17","unstructured":"Fischer, R., Preuss, R., and Toussaint, U.V. Bayesian Inference and Maximum Entropy Methods in Science and Engineering."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1087","DOI":"10.1063\/1.1699114","article-title":"Equation of State Calculations by Fast Computing Machines","volume":"21","author":"Metropolis","year":"1953","journal-title":"J. Chem. Phys"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1093\/biomet\/57.1.97","article-title":"Monte Carlo Sampling Methods using Markov Chains and their Applications","volume":"57","author":"Hastings","year":"1970","journal-title":"Biometrika"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1080\/01621459.1990.10476213","article-title":"Sampling-Based Approaches to Calculating Marginal Densities","volume":"85","author":"Gelfand","year":"1990","journal-title":"J. Am. Stat. Assoc"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Gilks, W.R., Richardson, S., and Spiegelhalter, D.J. (1996). Markov Chain Monte Carlo in Practice, Chapman and Hall.","DOI":"10.1201\/b14835"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Gilks, W.R., Richardson, S., and Spiegelhalter, D.J. (1996). Markov Chain Monte Carlo in Practice, Chapman and Hall.","DOI":"10.1201\/b14835"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gilks, W.R., Richardson, S., and Spiegelhalter, D.J. (1996). Markov Chain Monte Carlo in Practice, Chapman and Hall.","DOI":"10.1201\/b14835"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Tanner, M.A. (1996). Tools for Statistical Inference: Methods for the Exploration of Posterior Distributions and Likelihood Functions, Springer.","DOI":"10.1007\/978-1-4612-4024-2"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Djuri\u0107, P.M., and Godsill, S.J. (2002). Special Issue on Monte Carlo Methods for Statistical Signal Processing, IEEE.","DOI":"10.1109\/TSP.2002.978373"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1020281327116","article-title":"An Introduction to MCMC for Machine Learning","volume":"50","author":"Andrieu","year":"2003","journal-title":"Mach. Learn"},{"key":"ref_27","unstructured":"Clausius, R. (1850). On the Motive Power of Heat, and on the Laws Which Can be Deduced From it for the Theory of Heat, Poggendorff\u2019s Annalen der Physick, LXXIX, Dover Reprint."},{"key":"ref_28","unstructured":"Caticha, A. Maximum Entropy, fluctuations and priors."},{"key":"ref_29","unstructured":"Giffin, A., and Caticha, A. Updating Probabilities with Data and Moments."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"046127","DOI":"10.1103\/PhysRevE.70.046127","article-title":"Maximum Entropy and Bayesian Data Analysis: Entropic Priors Distributions","volume":"70","author":"Caticha","year":"2004","journal-title":"Phys. Rev. E"},{"key":"ref_31","unstructured":"Krishnaiah, P.R. (1977). Applications of Statistics, North-Holland."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1016\/0021-9991(79)90102-5","article-title":"An Algorithm for Finding the Distribution of Maximal Entropy","volume":"30","author":"Agmon","year":"1979","journal-title":"J. Comput. Phys"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Smith, C.R., and Grandy, W.T. (1985). Maximum-Entropy and Bayesian Methods in Inverse Problems, Springer.","DOI":"10.1007\/978-94-017-2221-6"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1137\/0329017","article-title":"Duality relationships for entropy-like minimization problems","volume":"29","author":"Borwein","year":"1991","journal-title":"SIAM J. Control Optim"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/0024-3795(80)90171-8","article-title":"On some Methods for Entropy Maximization and Matrix Scaling","volume":"34","author":"Elfwing","year":"1980","journal-title":"Linear Algebra Appl"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1007\/BF01588310","article-title":"A note on Solution of Large Sparse Maximum Entropy Problems with Linear Equality Constraints","volume":"18","author":"Eriksson","year":"1980","journal-title":"Math. Program"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1007\/BF01584236","article-title":"Entropy in linear programs","volume":"21","author":"Erlander","year":"1981","journal-title":"Math. Program"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1109\/PROC.1982.12425","article-title":"On the Rationale of Maximum-Entropy Methods","volume":"70","author":"Jaynes","year":"1982","journal-title":"Proc. IEEE"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1109\/TIT.1981.1056373","article-title":"Properties of Cross-Entropy Minimization","volume":"27","author":"Shore","year":"1981","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_40","first-page":"87","article-title":"Maximum d\u2019entropie et probl\u00e8mes inverses en imagerie","volume":"11","year":"1994","journal-title":"Traitement Signal"},{"key":"ref_41","unstructured":"Bercher, J. (1995). D\u00e9veloppement de crit\u00e8res de nature entropique pour la r\u00e9solution des probl\u00e8mes inverses lin\u00e9aires. [Ph.D. Thesis, Universit\u00e9 de Paris\u2013Sud]."},{"key":"ref_42","unstructured":"Le Besnerais, G. (1993). Ph.D. Thesis, Universit\u00e9 de Paris-Sud."},{"key":"ref_43","unstructured":"Caticha, A., and Giffin, A. Updating Probabilities."},{"key":"ref_44","unstructured":"Caticha, A. Entropic Inference."},{"key":"ref_45","unstructured":"Costa, S.I.R., Santos, S.A., and Strapasson, J.E. (2012). Fisher information distance: A geometrical reading, arXiv: 1210.2354."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1109\/18.481776","article-title":"Fisher Information and Stochastic Complexity","volume":"42","author":"Rissanen","year":"1996","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1007\/978-94-010-1848-7_28","article-title":"On Fisher\u2019s amount of information for location family","volume":"3","author":"Shimizu","year":"1975","journal-title":"A Modern Course on Statistical Distributions in Scientific Work"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2048","DOI":"10.1109\/TIT.2009.2018176","article-title":"Sided and Symmetrized Bregman Centroids","volume":"55","author":"Nielsen","year":"2009","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Rasmussen, C.E., and Williams, C.K.I. (2006). Gaussian Processes for Machine Learning, MIT Press.","DOI":"10.7551\/mitpress\/3206.001.0001"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/MASSP.1984.1162243","article-title":"Linear prediction, entropy and signal analysis","volume":"1","author":"Schroeder","year":"1984","journal-title":"IEEE ASSP Mag"},{"key":"ref_51","first-page":"36","article-title":"A Statistical Method for Estimation of Speech Spectral Density and Formant Frequencies","volume":"53-A","author":"Itakura","year":"1970","journal-title":"Electron. Commun. Jpn"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Kitagawa, G., and Gersch, W. (1996). Smoothness Priors Analysis of Time Series, Springer.","DOI":"10.1007\/978-1-4612-0761-0"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Rue, H., and Held, L. (2005). Gaussian Markov Random Fields: Theory and Applications, CRC Press.","DOI":"10.1201\/9780203492024"},{"key":"ref_54","unstructured":"Amari, S., Cichocki, A., and Yang, H.H. A new learning algorithm for blind source separation. 757\u2013763."},{"key":"ref_55","unstructured":"Amari, S. Neural learning in structured parameter spaces\u2014Natural Riemannian gradient. 127\u2013133."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1162\/089976698300017746","article-title":"Natural gradient works efficiently in learning","volume":"10","author":"Amari","year":"1998","journal-title":"Neural Comput"},{"key":"ref_57","unstructured":"Knuth, K.H. (1998). Bayesian source separation and localization. SPIE Proc, 3459."},{"key":"ref_58","unstructured":"Cardoso, J.-F., and Loubaton, P. A Bayesian approach to source separation. 283\u2013288."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"803","DOI":"10.1162\/089976699300016458","article-title":"Independent Factor Analysis","volume":"11","author":"Attias","year":"1999","journal-title":"Neural Comput"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Mohammad-Djafari, A. A Bayesian approach to source separation. 221\u2013244.","DOI":"10.1063\/1.1381859"},{"key":"ref_61","unstructured":"Choudrey, R.A., and Roberts, S. Variational Bayesian Mixture of Independent Component Analysers for Finding Self-Similar Areas in Images. 107\u2013112."},{"key":"ref_62","first-page":"41","article-title":"Bayesian Model Assessment in Factor Analysis","volume":"14","author":"Lopes","year":"2004","journal-title":"Statsinica"},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Ichir, M., and Mohammad-Djafari, A. Bayesian Blind Source Separation of Positive Non Stationary Sources. 493\u2013500.","DOI":"10.1063\/1.1835248"},{"key":"ref_64","unstructured":"Mohammad-Djafari, A. Bayesian Source Separation: Beyond PCA and ICA."},{"key":"ref_65","unstructured":"Comon, P., and Jutten, C. (2010). Handbook of Blind Source Separation: Independent Component Analysis and Applications, Academic Press."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1093\/biomet\/asm018","article-title":"Model selection and estimation in the Gaussian graphical model","volume":"94","author":"Yuan","year":"2007","journal-title":"Biometrika"},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/S0165-1684(00)00187-0","article-title":"Markov Chain Monte Carlo methods with Applications to Signal Processing","volume":"81","author":"Fitzgerald","year":"2001","journal-title":"Signal Process"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1109\/TASSP.1986.1164851","article-title":"Information theory measures with application to model identification","volume":"34","author":"Matsuoka","year":"1986","journal-title":"IEEE Trans. Acoust. Speech Signal Process"},{"key":"ref_69","unstructured":"Bretthorst, G.L. (1989). Maximum Entropy and Bayesian Methods, Springer."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1111\/j.2517-6161.1994.tb01996.x","article-title":"Bayesian model choice: Asymptotics and exact calculations","volume":"56","author":"Gelfand","year":"1994","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_71","unstructured":"Mohammad-Djafari, A. Model selection for inverse problems: Best choice of basis function and model order selection."},{"key":"ref_72","unstructured":"Babu, G.J., and Feigelson, E.D. Current Challenges in Bayesian Model Choice. 71, 224\u2013240."},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1007\/s11222-011-9233-4","article-title":"Block clustering with collapsed latent block models","volume":"22","author":"Wyse","year":"2012","journal-title":"Stat. Comput"},{"key":"ref_74","doi-asserted-by":"crossref","unstructured":"Giovannelli, J.F., and Giremus, A. Bayesian noise model selection and system identification based on approximation of the evidence. 125\u2013128.","DOI":"10.1109\/SSP.2014.6884591"},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1109\/TAC.1974.1100705","article-title":"A new look at the statistical model identification","volume":"AC-19","author":"Akaike","year":"1974","journal-title":"IEEE Trans. Automat. Control"},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1007\/BF02532269","article-title":"Power spectrum estimation through autoregressive model fitting","volume":"21","author":"Akaike","year":"1969","journal-title":"Ann. Inst. Stat. Math"},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"1176","DOI":"10.1109\/TASSP.1984.1164467","article-title":"Jaynes\u2019 principle and maximum entropy spectral estimation","volume":"32","author":"Farrier","year":"1984","journal-title":"IEEE Trans. Acoust. Speech Signal Process"},{"key":"ref_78","unstructured":"Wax, M. (1985). Detection and Estimation of Superimposed Signals. [Ph.D. Thesis, Standford University]."},{"key":"ref_79","unstructured":"Burg, J.P. Maximum Entropy Spectral Analysis."},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"1029","DOI":"10.1109\/PROC.1982.12431","article-title":"Multidimensional spectral estimation","volume":"70","author":"McClellan","year":"1982","journal-title":"Proc. IEEE"},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"880","DOI":"10.1109\/TASSP.1982.1163967","article-title":"Multidimensional MEM spectral estimation","volume":"30","author":"Lang","year":"1982","journal-title":"IEEE Trans. Acoust. Speech Signal Process"},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1109\/TASSP.1984.1164296","article-title":"Which is Better Entropy Expression for Speech Processing:-SlogS or logS?","volume":"ASSP-32","author":"Johnson","year":"1984","journal-title":"IEEE Trans. Acoust. Speech Signal Process"},{"key":"ref_83","doi-asserted-by":"crossref","unstructured":"Wester, R., Tummala, M., and Therrien, C. Multidimensional Autoregressive Spectral Estimation Using Iterative Methods. 1.","DOI":"10.1109\/ACSSC.1990.523376"},{"key":"ref_84","first-page":"153","article-title":"Nouvelle pr\u00e9sentation de la m\u00e9thode du maximum d\u2019entropie","volume":"7","author":"Picinbono","year":"1990","journal-title":"Traitement Signal"},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1137\/0801014","article-title":"Convergence of best entropy estimates","volume":"1","author":"Borwein","year":"1991","journal-title":"SIAM J. Optim"},{"key":"ref_86","unstructured":"Mohammad-Djafari, A. (2010). Inverse Problems in Vision and 3D Tomography, Wiley."},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1051\/rphysap:01987002202015300","article-title":"Tomographie de diffraction and synth\u00e8se de Fourier \u00e0 maximum d\u2019entropie","volume":"22","author":"Demoment","year":"1987","journal-title":"Rev. Phys. Appl. (Paris)"},{"key":"ref_88","doi-asserted-by":"crossref","unstructured":"F\u00e9ron, O., Chama, Z., and Mohammad-Djafari, A. Reconstruction of piecewise homogeneous images from partial knowledge of their Fourier transform. 68\u201375.","DOI":"10.1063\/1.1835199"},{"key":"ref_89","doi-asserted-by":"crossref","first-page":"2265","DOI":"10.1109\/TIP.2010.2047902","article-title":"Joint NDT Image Restoration and Segmentation Using Gauss\u2013Markov\u2013Potts Prior Models and Variational Bayesian Computation","volume":"19","author":"Ayasso","year":"2010","journal-title":"IEEE Trans. Image Process"},{"key":"ref_90","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1080\/09500340903564702","article-title":"Bayesian inversion for optical diffraction tomography","volume":"57","author":"Ayasso","year":"2010","journal-title":"J. Mod. Opt"},{"key":"ref_91","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1016\/0734-189X(83)90108-1","article-title":"Image Restoration by a Powerful Maximum Entropy Method","volume":"23","author":"Burch","year":"1983","journal-title":"Comput. Vis. Graph. Image Process"},{"key":"ref_92","first-page":"646","article-title":"Maximum entropy method in image processing","volume":"131","author":"Gull","year":"1984","journal-title":"IEE Proc. F"},{"key":"ref_93","doi-asserted-by":"crossref","unstructured":"Skilling, J. (1989). Maximum Entropy and Bayesian Methods, Springer.","DOI":"10.1007\/978-94-015-7860-8"},{"key":"ref_94","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1109\/18.50370","article-title":"General entropy criteria for inverse problems with application to data compression, pattern classification and cluster analysis","volume":"36","author":"Jones","year":"1990","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Macaulay, V.A., and Buck, B. (1989). Linear inversion by the method of maximum entropy. Inverse Probl, 5.","DOI":"10.1088\/0266-5611\/5\/5\/013"},{"key":"ref_96","doi-asserted-by":"crossref","first-page":"3177","DOI":"10.1016\/j.jspi.2006.07.016","article-title":"Approximate Bayesian inference for hierarchical Gaussian Markov random field models","volume":"137","author":"Rue","year":"2007","journal-title":"J. Stat. Plan. Inference"},{"key":"ref_97","unstructured":"Wilkinson, R. (2009). Approximate Bayesian computation (ABC) gives exact results under the assumption of model error, arXiv:0811.3355."},{"key":"ref_98","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1111\/j.1467-9868.2008.00700.x","article-title":"Approximate Bayesian Inference for Latent Gaussian Models Using Integrated Nested Laplace Approximations","volume":"71","author":"Rue","year":"2009","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_99","doi-asserted-by":"crossref","unstructured":"Fearnhead, P., and Prangle, D. (2011). Constructing Summary Statistics for Approximate Bayesian Computation: Semi-automatic ABC, arxiv:1004.1112v2.","DOI":"10.1038\/npre.2011.5959.1"},{"key":"ref_100","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.jmp.2012.02.005","article-title":"A tutorial on approximate Bayesian computation","volume":"56","author":"Turner","year":"2012","journal-title":"J. Math. Psych"},{"key":"ref_101","doi-asserted-by":"crossref","first-page":"448","DOI":"10.1162\/neco.1992.4.3.448","article-title":"A Practical Bayesian Framework for Backpropagation Networks","volume":"4","author":"MacKay","year":"1992","journal-title":"Neural Comput"},{"key":"ref_102","doi-asserted-by":"crossref","unstructured":"Mohammad-Djafari, A. Variational Bayesian Approximation for Linear Inverse Problems with a hierarchical prior models. 8085, 669\u2013676.","DOI":"10.1007\/978-3-642-40020-9_74"},{"key":"ref_103","doi-asserted-by":"crossref","first-page":"2222","DOI":"10.1109\/TSP.2004.831119","article-title":"A Variational Approach For Bayesian Blind Image Deconvolution","volume":"52","author":"Likas","year":"2004","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_104","first-page":"793","article-title":"Variational Bayesian learning of directed graphical models with hidden variables","volume":"1","author":"Beal","year":"2006","journal-title":"Bayesian Stat"},{"key":"ref_105","doi-asserted-by":"crossref","first-page":"1948","DOI":"10.1109\/TPAMI.2006.238","article-title":"Bayesian Gaussian Process Classification with the EM-EP Algorithm","volume":"28","author":"Kim","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"ref_106","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1023\/A:1007665907178","article-title":"An introduction to variational methods for graphical models","volume":"37","author":"Jordan","year":"2006","journal-title":"Mach. Learn"},{"key":"ref_107","doi-asserted-by":"crossref","first-page":"824","DOI":"10.1109\/TIP.2006.891045","article-title":"Combining Monte Carlo and Mean-Field-Like Methods for Inference in Hidden Markov Random Fields","volume":"16","author":"Forbes","year":"2007","journal-title":"IEEE Trans. Image Process"},{"key":"ref_108","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum Likelihood from incomplete data via the EM algorithm","volume":"39","author":"Dempster","year":"1977","journal-title":"J. R. Stat. Soc. (B)"},{"key":"ref_109","doi-asserted-by":"crossref","first-page":"892","DOI":"10.1109\/PROC.1987.13825","article-title":"The Role of Likelihood and Entropy in Incomplete-Data Problems: Applications to Estimating Point-Process Intensities and Toeplitz Constrained Covariances","volume":"75","author":"Miller","year":"1987","journal-title":"Proc. IEEE"},{"key":"ref_110","unstructured":"Williams, C. Information geometry of Prior Selection. AIP Conference Proceedings 570."},{"key":"ref_111","unstructured":"Mohammad-Djafari, A. (2009). Approche variationnelle pour le calcul bay\u00e9sien dans les probl\u00e9mes inverses en imagerie, arXiv:0904.4148."},{"key":"ref_112","unstructured":"Beal, M. (2003). Ph.D. Thesis, Gatsby Computational Neuroscience Unit, University College London."},{"key":"ref_113","first-page":"661","article-title":"Variational message passing","volume":"6","author":"Winn","year":"2005","journal-title":"J. Mach. Learn. Res"},{"key":"ref_114","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1109\/TFUZZ.2008.924317","article-title":"Factor Analysis Latent Subspace Modeling and Robust Fuzzy Clustering Using t-DistributionsClassification of binary random Patterns","volume":"17","author":"Chatzis","year":"2009","journal-title":"IEEE Trans. Fuzzy Syst"},{"key":"ref_115","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1198\/016214508000000337","article-title":"The Bayesian Lasso","volume":"103","author":"Park","year":"2008","journal-title":"J. Am. Stat. Assoc"},{"key":"ref_116","unstructured":"Censor, Y., Jiang, M., and Louis, A.K. (2008). Mathematical Methods in Biomedical Imaging and Intensity-Modulated Radiation Therapy (IMRT), Edizioni della Normale."},{"key":"ref_117","doi-asserted-by":"crossref","unstructured":"Mohammad-Djafari, A., and Ayasso, H. Variational Bayes and mean field approximations for Markov field unsupervised estimation. 1\u20136.","DOI":"10.1109\/MLSP.2009.5306261"},{"key":"ref_118","first-page":"211","article-title":"Sparse Bayesian learning and the relevance vector machine","volume":"1","author":"Tipping","year":"2001","journal-title":"J. Mach. Learn. Res"},{"key":"ref_119","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1109\/LSP.2009.2037532","article-title":"Tree-Structured Compressive Sensing With Variational Bayesian Analysis","volume":"17","author":"He","year":"2010","journal-title":"IEEE Signal Process. Lett"},{"key":"ref_120","doi-asserted-by":"crossref","unstructured":"Fraysse, A., and Rodet, T. (2011, January 28\u201330). A gradient-like variational Bayesian algorithm, Nice France.","DOI":"10.1109\/SSP.2011.5967772"},{"key":"ref_121","doi-asserted-by":"crossref","first-page":"741","DOI":"10.1214\/13-BA818","article-title":"On Numerical Aspects of Bayesian Model Selection in High and Ultrahigh-dimensional Settings","volume":"8","author":"Johnson","year":"2013","journal-title":"Bayesian Anal"},{"key":"ref_122","doi-asserted-by":"crossref","first-page":"548","DOI":"10.1063\/1.4906021","article-title":"Estimating the periodic components of a biomedical signal through inverse problem modeling and Bayesian inference with sparsity enforcing prior","volume":"1641","author":"Dumitru","year":"2015","journal-title":"AIP Conf. Proc"},{"key":"ref_123","doi-asserted-by":"crossref","first-page":"556","DOI":"10.1063\/1.4906022","article-title":"Bayesian 3D X-ray computed tomography image reconstruction with a scaled Gaussian mixture prior model","volume":"1641","author":"Wang","year":"2015","journal-title":"AIP Conf. Proc"},{"key":"ref_124","unstructured":"Mohammad-Djafari, A. Bayesian Blind Deconvolution of Images Comparing JMAP, EM and VBA with a Student-t a priori Model. 98\u2013103."},{"key":"ref_125","unstructured":"Su, F., and Mohammad-Djafari, A. An Hierarchical Markov Random Field Model for Bayesian Blind Image Separation."},{"key":"ref_126","unstructured":"Su, F., Cai, S., and Mohammad-Djafari, A. Bayesian blind separation of mixed text patterns. 1373\u20131378."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/6\/3989\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:47:48Z","timestamp":1760215668000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/6\/3989"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,6,12]]},"references-count":126,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2015,6]]}},"alternative-id":["e17063989"],"URL":"https:\/\/doi.org\/10.3390\/e17063989","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,6,12]]}}}