{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T03:23:29Z","timestamp":1740108209074,"version":"3.37.3"},"reference-count":50,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2023,3,16]],"date-time":"2023-03-16T00:00:00Z","timestamp":1678924800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,16]],"date-time":"2023-03-16T00:00:00Z","timestamp":1678924800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Physikalisch-Technische Bundesanstalt (PTB)"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Stat"],"published-print":{"date-parts":[[2024,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The Bayesian approach to solving inverse problems relies on the choice of a prior. This critical ingredient allows expert knowledge or physical constraints to be formulated in a probabilistic fashion and plays an important role for the success of the inference. Recently, Bayesian inverse problems were solved using generative models as highly informative priors. Generative models are a popular tool in machine learning to generate data whose properties closely resemble those of a given database. Typically, the generated distribution of data is embedded in a low-dimensional manifold. For the inverse problem, a generative model is trained on a database that reflects the properties of the sought solution, such as typical structures of the tissue in the human brain in magnetic resonance imaging. The inference is carried out in the low-dimensional manifold determined by the generative model that strongly reduces the dimensionality of the inverse problem. However, this procedure produces a posterior that does not admit a Lebesgue density in the actual variables and the accuracy attained can strongly depend on the quality of the generative model. For linear Gaussian models, we explore an alternative Bayesian inference based on probabilistic generative models; this inference is carried out in the original high-dimensional space. A Laplace approximation is employed to analytically derive the prior probability density function required, which is induced by the generative model. Properties of the resulting inference are investigated. Specifically, we show that derived Bayes estimates are consistent, in contrast to the approach in which the low-dimensional manifold of the generative model is employed. The MNIST data set is used to design numerical experiments that confirm our theoretical findings. It is shown that the approach proposed can be advantageous when the information contained in the data is high and a simple heuristic is considered for the detection of this case. Finally, the pros and cons of both approaches are discussed.<\/jats:p>","DOI":"10.1007\/s00180-023-01345-5","type":"journal-article","created":{"date-parts":[[2023,3,16]],"date-time":"2023-03-16T12:03:02Z","timestamp":1678968182000},"page":"1321-1349","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Generative models and Bayesian inversion using Laplace approximation"],"prefix":"10.1007","volume":"39","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0648-1936","authenticated-orcid":false,"given":"Manuel","family":"Marschall","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6871-8903","authenticated-orcid":false,"given":"Gerd","family":"W\u00fcbbeler","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5081-2742","authenticated-orcid":false,"given":"Franko","family":"Schm\u00e4hling","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0113-3713","authenticated-orcid":false,"given":"Clemens","family":"Elster","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,3,16]]},"reference":[{"key":"1345_CR1","unstructured":"Adler J, \u00d6ktem O (2018) Deep bayesian inversion. arXiv preprint arXiv:1811.05910"},{"key":"1345_CR2","doi-asserted-by":"crossref","unstructured":"Albert A, Strano E, Kaur J, Gonz\u00e1lez M (2018) Modeling urbanization patterns with generative adversarial networks. In: IGARSS 2018\u20132018 IEEE international geoscience and remote sensing symposium. IEEE, pp 2095\u20132098","DOI":"10.1109\/IGARSS.2018.8518032"},{"key":"1345_CR3","volume-title":"Digital image restoration","author":"HC Andrews","year":"1977","unstructured":"Andrews HC, Hunt BR (1977) Digital image restoration. Prentice-Hall, Hoboken"},{"key":"1345_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1017\/S0962492919000059","volume":"28","author":"S Arridge","year":"2019","unstructured":"Arridge S, Maass P, \u00d6ktem O, Sch\u00f6nlieb C-B (2019) Solving inverse problems using data-driven models. Acta Numer 28:1\u2013174","journal-title":"Acta Numer"},{"key":"1345_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.sigpro.2020.107729","volume":"177","author":"Y Bai","year":"2020","unstructured":"Bai Y, Chen W, Chen J, Guo W (2020) Deep learning methods for solving linear inverse problems: research directions and paradigms. Signal Process 177:107729","journal-title":"Signal Process"},{"key":"1345_CR6","doi-asserted-by":"crossref","unstructured":"Bhadra S, Zhou W, Anastasio MA (2020) Medical image reconstruction with image-adaptive priors learned by use of generative adversarial networks. In: Medical imaging 2020: physics of medical imaging, vol 11312. International Society for Optics and Photonics, p 113120V","DOI":"10.1117\/12.2549750"},{"issue":"3","key":"1345_CR7","doi-asserted-by":"publisher","DOI":"10.1088\/0266-5611\/24\/3\/034009","volume":"24","author":"N Bissantz","year":"2008","unstructured":"Bissantz N, Holzmann H (2008) Statistical inference for inverse problems. Inverse Probl 24(3):034009","journal-title":"Inverse Probl"},{"key":"1345_CR8","unstructured":"Bora A, Jalal A, Price E, Dimakis AG (2017) Compressed sensing using generative models. In: International conference on machine learning. PMLR, pp 537\u2013546"},{"issue":"13","key":"1345_CR9","doi-asserted-by":"publisher","first-page":"4207","DOI":"10.1523\/JNEUROSCI.16-13-04207.1996","volume":"16","author":"GM Boynton","year":"1996","unstructured":"Boynton GM, Engel SA, Glover GH, Heeger DJ (1996) Linear systems analysis of functional magnetic resonance imaging in human v1. J Neurosci 16(13):4207\u20134221","journal-title":"J Neurosci"},{"key":"1345_CR10","unstructured":"Burgess CP, Higgins I, Pal A, Matthey L, Watters N, Desjardins G, Lerchner A (2018) Understanding disentangling in $$\\beta$$-vae. arXiv preprint arXiv:1804.03599"},{"issue":"252","key":"1345_CR11","first-page":"2","volume":"18","author":"L Calatroni","year":"2017","unstructured":"Calatroni L, Cao C, De Los Reyes JC, Sch\u00f6nlieb C-B, Valkonen T (2017) Bilevel approaches for learning of variational imaging models. Variat Methods Imag Geometr Control 18(252):2","journal-title":"Variat Methods Imag Geometr Control"},{"key":"1345_CR12","doi-asserted-by":"publisher","first-page":"14 985","DOI":"10.1109\/ACCESS.2018.2886814","volume":"7","author":"Y-J Cao","year":"2018","unstructured":"Cao Y-J, Jia L-L, Chen Y-X, Lin N, Yang C, Zhang B, Liu Z, Li X-X, Dai H-H (2018) Recent advances of generative adversarial networks in computer vision. IEEE Access 7:14 985-15 006","journal-title":"IEEE Access"},{"issue":"6","key":"1345_CR13","doi-asserted-by":"publisher","first-page":"1659","DOI":"10.1137\/S0036142997320413","volume":"36","author":"AS Carasso","year":"1999","unstructured":"Carasso AS (1999) Linear and nonlinear image deblurring: a documented study. SIAM J Numer Anal 36(6):1659\u20131689","journal-title":"SIAM J Numer Anal"},{"issue":"2","key":"1345_CR14","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1080\/00031305.1985.10479400","volume":"39","author":"G Casella","year":"1985","unstructured":"Casella G (1985) An introduction to empirical bayes data analysis. Am Statist 39(2):83\u201387","journal-title":"Am Statist"},{"issue":"6","key":"1345_CR15","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1109\/MSP.2012.2211477","volume":"29","author":"L Deng","year":"2012","unstructured":"Deng L (2012) The MNIST database of handwritten digit images for machine learning research. IEEE Signal Process Mag 29(6):141\u2013142","journal-title":"IEEE Signal Process Mag"},{"key":"1345_CR16","unstructured":"Dillon JV, Langmore I, Tran D, Brevdo E, Vasudevan S, Moore D, Patton B, Alemi A, Hoffman M, Saurous RA (2017) Tensorflow distributions. arXiv preprint arXiv:1711.10604"},{"key":"1345_CR17","doi-asserted-by":"publisher","DOI":"10.1002\/9781118625590","volume-title":"Applied regression analysis","author":"NR Draper","year":"1998","unstructured":"Draper NR, Smith H (1998) Applied regression analysis. Wiley, Hoboken"},{"key":"1345_CR18","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-009-1740-8","volume-title":"Regularization of inverse problems","author":"HW Engl","year":"1996","unstructured":"Engl HW, Hanke M, Neubauer A (1996) Regularization of inverse problems. Springer Science & Business Media, New York"},{"issue":"925","key":"1345_CR19","doi-asserted-by":"publisher","first-page":"306","DOI":"10.1086\/670067","volume":"125","author":"D Foreman-Mackey","year":"2013","unstructured":"Foreman-Mackey D, Hogg DW, Lang D, Goodman J (2013) emcee: The MCMC hammer. Publ Astron Soc Pacific 125(925):306","journal-title":"Publ Astron Soc Pacific"},{"key":"1345_CR20","doi-asserted-by":"publisher","DOI":"10.1201\/9780429258411","volume-title":"Bayesian data analysis","author":"A Gelman","year":"1995","unstructured":"Gelman A, Carlin JB, Stern HS, Rubin DB (1995) Bayesian data analysis. Chapman and Hall\/CRC, Boca raton"},{"issue":"2","key":"1345_CR21","doi-asserted-by":"publisher","first-page":"822","DOI":"10.1137\/21M140225X","volume":"15","author":"M Gonz\u00e1lez","year":"2022","unstructured":"Gonz\u00e1lez M, Almansa A, Tan P (2022) Solving inverse problems by joint posterior maximization with autoencoding prior. SIAM J Imag Sci 15(2):822\u2013859","journal-title":"SIAM J Imag Sci"},{"key":"1345_CR22","unstructured":"Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inform Process Syst 27"},{"issue":"2","key":"1345_CR23","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1039\/C9SC04026A","volume":"11","author":"R-R Griffiths","year":"2020","unstructured":"Griffiths R-R, Hern\u00e1ndez-Lobato JM (2020) Constrained Bayesian optimization for automatic chemical design using variational autoencoders. Chem Sci 11(2):577\u2013586","journal-title":"Chem Sci"},{"issue":"2","key":"1345_CR24","doi-asserted-by":"publisher","first-page":"892","DOI":"10.1137\/21M1406313","volume":"15","author":"M Holden","year":"2022","unstructured":"Holden M, Pereyra M, Zygalakis KC (2022) Bayesian imaging with data-driven priors encoded by neural networks. SIAM J Imag Sci 15(2):892\u2013924","journal-title":"SIAM J Imag Sci"},{"issue":"1","key":"1345_CR25","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1021\/acs.jcim.9b00694","volume":"60","author":"SH Hong","year":"2019","unstructured":"Hong SH, Ryu S, Lim J, Kim WY (2019) Molecular generative model based on an adversarially regularized autoencoder. J Chem Inform Model 60(1):29\u201336","journal-title":"J Chem Inform Model"},{"issue":"04","key":"1345_CR26","first-page":"3121","volume":"34","author":"SA Hussein","year":"2020","unstructured":"Hussein SA, Tirer T, Giryes R (2020) Image-adaptive gan based reconstruction. Proc AAAI Conf Artif Intell 34(04):3121\u20133129","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"1345_CR27","doi-asserted-by":"publisher","unstructured":"Jiang Z, Zhang S, Turnadge C, Xu T (2019) Combining autoencoder neural network and bayesian inversion algorithms to estimate heterogeneous fracture permeability in enhanced geothermal reservoirs. Earth and Space Science Open Archive, p. 19. [Online]. Available: https:\/\/doi.org\/10.1002\/essoar.10501256.1","DOI":"10.1002\/essoar.10501256.1"},{"key":"1345_CR28","volume-title":"Statistical and computational inverse problems","author":"J Kaipio","year":"2006","unstructured":"Kaipio J, Somersalo E (2006) Statistical and computational inverse problems, vol 160. Springer Science & Business Media, New York"},{"key":"1345_CR29","doi-asserted-by":"crossref","unstructured":"Karras T, Laine S, Aittala M, Hellsten J, Lehtinen J, Aila T (2020) Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 8110\u20138119","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"1345_CR30","doi-asserted-by":"crossref","unstructured":"Kingma DP, Welling M (2019) An introduction to variational autoencoders. arXiv preprint arXiv:1906.02691","DOI":"10.1561\/9781680836233"},{"issue":"13","key":"1345_CR31","doi-asserted-by":"publisher","first-page":"135003","DOI":"10.1088\/1361-6560\/ab990e","volume":"65","author":"A Kofler","year":"2020","unstructured":"Kofler A, Haltmeier M, Schaeffter T, Kachelrie\u00df M, Dewey M, Wald C, Kolbitsch C (2020) Neural networks-based regularization for large-scale medical image reconstruction. Phys Med Biol 65(13):135003. https:\/\/doi.org\/10.1088\/1361-6560\/ab990e","journal-title":"Phys Med Biol"},{"issue":"3","key":"1345_CR32","first-page":"699","volume":"9","author":"K-J Lee","year":"2014","unstructured":"Lee K-J, Jones GL, Caffo BS, Bassett SS (2014) Spatial Bayesian variable selection models on functional magnetic resonance imaging time-series data. Bayesian Anal (Online) 9(3):699","journal-title":"Bayesian Anal (Online)"},{"key":"1345_CR33","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1016\/j.ins.2018.12.057","volume":"482","author":"Y Li","year":"2019","unstructured":"Li Y, Pan Q, Wang S, Peng H, Yang T, Cambria E (2019) Disentangled variational auto-encoder for semi-supervised learning. Inform Sci 482:73\u201385","journal-title":"Inform Sci"},{"issue":"15","key":"1345_CR34","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2101344118","volume":"118","author":"Q Liu","year":"2021","unstructured":"Liu Q, Xu J, Jiang R, Wong WH (2021) Density estimation using deep generative neural networks. Proc Natl Acad Sci 118(15):e2101344118","journal-title":"Proc Natl Acad Sci"},{"key":"1345_CR35","unstructured":"MATLAB (2021) version 9.11.0 (R2021b). The MathWorks Inc, Natick"},{"issue":"381","key":"1345_CR36","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1080\/01621459.1983.10477920","volume":"78","author":"CN Morris","year":"1983","unstructured":"Morris CN (1983) Parametric empirical bayes inference: theory and applications. J Am Statist Assoc 78(381):47\u201355","journal-title":"J Am Statist Assoc"},{"key":"1345_CR37","doi-asserted-by":"crossref","unstructured":"M\u00fccke NT, Sanderse B, Boht\u00e9 S, Oosterlee CW (2021) Markov chain generative adversarial neural networks for solving bayesian inverse problems in physics applications. arXiv preprint arXiv:2111.12408","DOI":"10.2139\/ssrn.3991779"},{"issue":"10","key":"1345_CR38","doi-asserted-by":"publisher","first-page":"1216","DOI":"10.3390\/electronics10101216","volume":"10","author":"S-W Park","year":"2021","unstructured":"Park S-W, Ko J-S, Huh J-H, Kim J-C (2021) Review on generative adversarial networks: focusing on computer vision and its applications. Electronics 10(10):1216","journal-title":"Electronics"},{"key":"1345_CR39","unstructured":"Richard M, Chang MY-S (2001) Fast digital image inpainting. In: Appeared in the proceedings of the international conference on visualization, imaging and image processing (VIIP 2001), Marbella, Spain, pp 106\u2013107"},{"key":"1345_CR40","volume-title":"The Bayesian choice: from decision-theoretic foundations to computational implementation","author":"CP Robert","year":"2007","unstructured":"Robert CP et al (2007) The Bayesian choice: from decision-theoretic foundations to computational implementation. Springer, Heidelberg"},{"issue":"2","key":"1345_CR41","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1007\/s10208-016-9340-x","volume":"18","author":"D Rudolf","year":"2018","unstructured":"Rudolf D, Sprungk B (2018) On a generalization of the preconditioned crank-Nicolson metropolis algorithm. Found Comput Math 18(2):309\u2013343","journal-title":"Found Comput Math"},{"key":"1345_CR42","doi-asserted-by":"publisher","DOI":"10.1201\/9780203492024","volume-title":"Gaussian Markov random fields: theory and applications","author":"H Rue","year":"2005","unstructured":"Rue H, Held L (2005) Gaussian Markov random fields: theory and applications. CRC Press, Boca raton"},{"issue":"1","key":"1345_CR43","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1109\/TASLP.2017.2761547","volume":"26","author":"Y Saito","year":"2017","unstructured":"Saito Y, Takamichi S, Saruwatari H (2017) Statistical parametric speech synthesis incorporating generative adversarial networks. IEEE\/ACM Trans Audio Speech Lang Process 26(1):84\u201396","journal-title":"IEEE\/ACM Trans Audio Speech Lang Process"},{"issue":"1","key":"1345_CR44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-019-0197-0","volume":"6","author":"C Shorten","year":"2019","unstructured":"Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1\u201348","journal-title":"J Big Data"},{"issue":"478","key":"1345_CR45","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1198\/016214506000001031","volume":"102","author":"M Smith","year":"2007","unstructured":"Smith M, Fahrmeir L (2007) Spatial Bayesian variable selection with application to functional magnetic resonance imaging. J Am Statist Assoc 102(478):417\u2013431","journal-title":"J Am Statist Assoc"},{"key":"1345_CR46","doi-asserted-by":"crossref","unstructured":"Sood R, Topiwala B, Choutagunta K, Sood R, Rusu M (2018) An application of generative adversarial networks for super resolution medical imaging. In: 2018 17th IEEE international conference on machine learning and applications (ICMLA). IEEE, pp 326\u2013331","DOI":"10.1109\/ICMLA.2018.00055"},{"key":"1345_CR47","first-page":"11259","volume":"33","author":"A Tripp","year":"2020","unstructured":"Tripp A, Daxberger E, Hern\u00e1ndez-Lobato JM (2020) Sample-efficient optimization in the latent space of deep generative models via weighted retraining. Adv Neural Inform Process Syst 33:11259\u201311272","journal-title":"Adv Neural Inform Process Syst"},{"key":"1345_CR48","doi-asserted-by":"crossref","unstructured":"Wang H, Qin Z, Wan T (2018) Text generation based on generative adversarial nets with latent variables. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, pp 92\u2013103","DOI":"10.1007\/978-3-319-93037-4_8"},{"issue":"10","key":"1345_CR49","first-page":"1433","volume":"23","author":"C Yangjie","year":"2018","unstructured":"Yangjie C, Lili J, Yongxia C, Nan L, Xuexiang L (2018) Review of computer vision based on generative adversarial networks. J Image Graph 23(10):1433\u20131449","journal-title":"J Image Graph"},{"key":"1345_CR50","doi-asserted-by":"publisher","first-page":"101552","DOI":"10.1016\/j.media.2019.101552","volume":"58","author":"X Yi","year":"2019","unstructured":"Yi X, Walia E, Babyn P (2019) Generative adversarial network in medical imaging: a review. Med Image Anal 58:101552","journal-title":"Med Image Anal"}],"container-title":["Computational Statistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00180-023-01345-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00180-023-01345-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00180-023-01345-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,6]],"date-time":"2024-04-06T13:02:44Z","timestamp":1712408564000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00180-023-01345-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,16]]},"references-count":50,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,5]]}},"alternative-id":["1345"],"URL":"https:\/\/doi.org\/10.1007\/s00180-023-01345-5","relation":{},"ISSN":["0943-4062","1613-9658"],"issn-type":[{"type":"print","value":"0943-4062"},{"type":"electronic","value":"1613-9658"}],"subject":[],"published":{"date-parts":[[2023,3,16]]},"assertion":[{"value":"16 March 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 March 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}