{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:34:08Z","timestamp":1760243648088,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2012,5,10]],"date-time":"2012-05-10T00:00:00Z","timestamp":1336608000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this paper we examine an Information-Theoretic method for solving noisy linear inverse estimation problems which encompasses under a single framework a whole class of estimation methods. Under this framework, the prior information about the unknown parameters (when such information exists), and constraints on the parameters can be incorporated in the statement of the problem. The method builds on the basics of the maximum entropy principle and consists of transforming the original problem into an estimation of a probability density on an appropriate space naturally associated with the statement of the problem. This estimation method is generic in the sense that it provides a framework for analyzing non-normal models, it is easy to implement and is suitable for all types of inverse problems such as small and or ill-conditioned, noisy data. First order approximation, large sample properties and convergence in distribution are developed as well. Analytical examples, statistics for model comparisons and evaluations, that are inherent to this method, are discussed and complemented with explicit examples.<\/jats:p>","DOI":"10.3390\/e14050892","type":"journal-article","created":{"date-parts":[[2012,5,10]],"date-time":"2012-05-10T11:11:15Z","timestamp":1336648275000},"page":"892-923","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["An Entropic Estimator for Linear Inverse Problems"],"prefix":"10.3390","volume":"14","author":[{"given":"Amos","family":"Golan","sequence":"first","affiliation":[{"name":"Department of Economics, Info-Metrics Institute, American University, 4400 Massachusetts Ave., Washington, DC 20016, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Henryk","family":"Gzyl","sequence":"additional","affiliation":[{"name":"Centro de Finanzas, IESA, Caracas 1010, Venezuela"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2012,5,10]]},"reference":[{"key":"ref_1","unstructured":"Golan, A., Judge, G.G., and Miller, D. (1996). Maximum Entropy Econometrics: Robust Estimation with Limited Data, John Wiley & Sons."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Gzyl, H., and Vel\u00e1squez, Y. (2011). Linear Inverse Problems: The Maximum Entropy Connection, World Scientific Publishers.","DOI":"10.1142\/9789814338783"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1103\/PhysRev.106.620","article-title":"Information theory and statistical mechanics","volume":"106","author":"Jaynes","year":"1957","journal-title":"Phys. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1103\/PhysRev.108.171","article-title":"Information theory and statistical mechanics II","volume":"108","author":"Jaynes","year":"1957","journal-title":"Phys. Rev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell System Technical. J."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1725","DOI":"10.1214\/aos\/1176348368","article-title":"Empirical likelihood for linear models","volume":"19","author":"Owen","year":"1991","journal-title":"Ann. Stat."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Owen, A. (2001). Empirical Likelihood, Chapman & Hall\/CRC.","DOI":"10.1201\/9781420036152"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1214\/aos\/1176325370","article-title":"Empirical likelihood and general estimating equations","volume":"22","author":"Qin","year":"1994","journal-title":"Ann. Stat."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1111\/j.0013-0133.1997.174.x","article-title":"Alternative semi parametric likelihood approaches to GMM estimations","volume":"107","author":"Smith","year":"1997","journal-title":"Econ. J."},{"key":"ref_10","unstructured":"Newey, W.K., and Smith, R.J. (2002). Higher order properties of GMM and Generalized empirical likelihood estimators, Unpublished work."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"861","DOI":"10.2307\/2171942","article-title":"An information-theoretic alternative to generalized method of moment estimation","volume":"66","author":"Kitamura","year":"1997","journal-title":"Econometrica"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"333","DOI":"10.2307\/2998561","article-title":"Information-theoretic approaches to inference in moment condition models","volume":"66","author":"Imbens","year":"1998","journal-title":"Econometrica"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lee, J.C., Zellner, A., and Johnson, W.O. (1996). Prediction and Modeling Honoring Seymour Geisser, Springer Verlag.","DOI":"10.1007\/978-1-4612-2414-3"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1108\/S0731-9053(1997)0000012005","article-title":"The Bayesian Method of Moments (BMOM): Theory and applications","volume":"Volume 12","author":"Fomby","year":"1997","journal-title":"Advances in Econometrics"},{"key":"ref_15","first-page":"1","article-title":"Further results on the Bayesian method of moments analysis of multiple regression model","volume":"107","author":"Zellner","year":"2001","journal-title":"Int. Econ. Rev."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1214\/aos\/1034276632","article-title":"Bayesian methods and maximum entropy for ill-posed inverse problems","volume":"25","author":"Gamboa","year":"1997","journal-title":"Ann. Stat."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Erickson, G., and Ryckert, J. (1998). Maximum Entropy and Bayesian Studies, Kluwer.","DOI":"10.1007\/978-94-011-5028-6"},{"key":"ref_18","first-page":"249","article-title":"A generalized maxentropic inversion procedure for noisy data","volume":"127","author":"Golan","year":"2002","journal-title":"Appl. Math. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1080\/00401706.1970.10488634","article-title":"Ridge regression: Biased estimation for non-orthogonal problems","volume":"1","author":"Hoerl","year":"1970","journal-title":"Technometrics"},{"key":"ref_20","first-page":"502","article-title":"A statistical perspective on ill-posed inverse problems","volume":"1","year":"1986","journal-title":"Stat. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1080\/00401706.1995.10484371","article-title":"Better subset regression using the nonnegative garrote","volume":"37","author":"Breiman","year":"1995","journal-title":"Technometrics"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"141","DOI":"10.2307\/1402932","article-title":"Common structures of smoothing techniques in statistics","volume":"53","author":"Titterington","year":"1985","journal-title":"Int. Stat. Rev."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1111\/j.2517-6161.1992.tb01864.x","article-title":"Maximum entropy and the nearly black object","volume":"54","author":"Donoho","year":"1992","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1565","DOI":"10.1109\/18.771159","article-title":"A new look at entropy for solving linear inverse problems","volume":"45","author":"Besnerais","year":"1999","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1007\/BF02607055","article-title":"Regularization methods in statistics","volume":"15","author":"Bickel","year":"2006","journal-title":"Test"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/0800000004","article-title":"Information and entropy econometrics\u2014A review and synthesis","volume":"2","author":"Golan","year":"2008","journal-title":"Found. Trends Econometrics"},{"key":"ref_28","unstructured":"Fomby, T.B., and Hill, R.C. (1997). Advances in Econometrics, JAI Press."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Golan, A. (2002). Special Issue on Information and Entropy Econometrics (Journal of Econometrics), Elsevier.","DOI":"10.1016\/S0304-4076(01)00110-5"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Golan, A., and Kitamura, Y. (2007). Special Issue on Information and Entropy Econometrics: A Volume in Honor of Arnold Zellner (Journal of Econometrics), Elsevier.","DOI":"10.1016\/j.jeconom.2006.05.001"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Mynbayev, K.T. (2011). Short-Memory Linear Processes and Econometric Applications, John Wiley & Sons.","DOI":"10.1002\/9781118007686"},{"key":"ref_32","unstructured":"Asher, R.C., Borchers, B., and Thurber, C.A. (2003). Parameter Estimation and Inverse Problems, Elsevier."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0304-4076(01)00110-5","article-title":"Information and entropy econometrics\u2014Editor\u2019s view","volume":"107","author":"Golan","year":"2002","journal-title":"J. Econom."},{"key":"ref_34","unstructured":"Kullback, S. (1959). Information Theory and Statistics, John Wiley & Sons."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1111\/j.2517-6161.1960.tb00361.x","article-title":"Estimation of parameters in time-series regression models","volume":"22","author":"Durbin","year":"1960","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_36","unstructured":"Mittelhammer, R., Judge, G., and Miller, D. (2000). Econometric Foundations, Cambridge Univ. Press."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Bertero, M., and Boccacci, P. (1998). Introduction to Inverse Problems in Imaging, CRC Press.","DOI":"10.1887\/0750304359"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1080\/00031305.1988.10475585","article-title":"Optimal information processing and Bayes theorem","volume":"42","author":"Zellner","year":"1988","journal-title":"Am. Stat."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/S0304-4076(01)00112-9","article-title":"Information processing and Bayesian analysis","volume":"107","author":"Zellner","year":"2002","journal-title":"J. Econom."},{"key":"ref_40","unstructured":"Lee, J.C., Johnson, W.D., and Zellner, A. (1994). Modeling and Prediction, Springer."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/0304-4076(95)01768-2","article-title":"Models, prior information, and Bayesian analysis","volume":"75","author":"Zellner","year":"1996","journal-title":"J. Econom."},{"key":"ref_42","unstructured":"Zellner, A. (1997). Bayesian Analysis in Econometrics and Statistics: The Zellner View and Papers, Edward Elgar Publishing Ltd."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Kotz, S., Kozubowski, T., and Podg\u00f3rski, K. (2001). The Laplace Distribution and Generalizations, Birkhauser.","DOI":"10.1007\/978-1-4612-0173-1"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1080\/00031305.1994.10476030","article-title":"The three sigma rule","volume":"48","author":"Pukelsheim","year":"1994","journal-title":"Am. Stat."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/14\/5\/892\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:50:13Z","timestamp":1760219413000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/14\/5\/892"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,5,10]]},"references-count":44,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2012,5]]}},"alternative-id":["e14050892"],"URL":"https:\/\/doi.org\/10.3390\/e14050892","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2012,5,10]]}}}