{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T16:00:39Z","timestamp":1776182439527,"version":"3.50.1"},"reference-count":43,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2021,2,25]],"date-time":"2021-02-25T00:00:00Z","timestamp":1614211200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1718991"],"award-info":[{"award-number":["1718991"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01-EB026955"],"award-info":[{"award-number":["R01-EB026955"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Intel INRC","award":["00"],"award-info":[{"award-number":["00"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Markov Chain Monte Carlo (MCMC) methods sample from unnormalized probability distributions and offer guarantees of exact sampling. However, in the continuous case, unfavorable geometry of the target distribution can greatly limit the efficiency of MCMC methods. Augmenting samplers with neural networks can potentially improve their efficiency. Previous neural network-based samplers were trained with objectives that either did not explicitly encourage exploration, or contained a term that encouraged exploration but only for well structured distributions. Here we propose to maximize proposal entropy for adapting the proposal to distributions of any shape. To optimize proposal entropy directly, we devised a neural network MCMC sampler that has a flexible and tractable proposal distribution. Specifically, our network architecture utilizes the gradient of the target distribution for generating proposals. Our model achieved significantly higher efficiency than previous neural network MCMC techniques in a variety of sampling tasks, sometimes by more than an order magnitude. Further, the sampler was demonstrated through the training of a convergent energy-based model of natural images. The adaptive sampler achieved unbiased sampling with significantly higher proposal entropy than a Langevin dynamics sample. The trained sampler also achieved better sample quality.<\/jats:p>","DOI":"10.3390\/e23030269","type":"journal-article","created":{"date-parts":[[2021,2,25]],"date-time":"2021-02-25T05:20:08Z","timestamp":1614230408000},"page":"269","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["A Neural Network MCMC Sampler That Maximizes Proposal Entropy"],"prefix":"10.3390","volume":"23","author":[{"given":"Zengyi","family":"Li","sequence":"first","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience, Berkeley, CA 94720, USA"},{"name":"Department of Physics, University of California Berkeley, Berkeley, CA 94720, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8930-3512","authenticated-orcid":false,"given":"Yubei","family":"Chen","sequence":"additional","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience, Berkeley, CA 94720, USA"},{"name":"Berkeley AI Research, University of California Berkeley, Berkeley, CA 94720, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Friedrich T.","family":"Sommer","sequence":"additional","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience, Berkeley, CA 94720, USA"},{"name":"Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA 94720, USA"},{"name":"Neuromorphic Computing Group, Intel Labs, 2200 Mission College Blvd., Santa Clara, CA 95054-1549, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"eaaw1147","DOI":"10.1126\/science.aaw1147","article-title":"Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning","volume":"365","author":"Olsson","year":"2019","journal-title":"Science"},{"key":"ref_2","unstructured":"Nijkamp, E., Hill, M., Han, T., Zhu, S.C., and Wu, Y.N. (2019, January 7\u201312). On the Anatomy of MCMC-based Maximum Likelihood Learning of Energy-Based Models. Proceedings of the Conference on Artificial Intelligence (AAAI), New York, NY, USA."},{"key":"ref_3","unstructured":"Neal, R.M. (1993). Probabilistic Inference Using Markov Chain Monte Carlo Methods, Department of Computer Science, University of Toronto."},{"key":"ref_4","first-page":"2","article-title":"MCMC using Hamiltonian dynamics","volume":"2","author":"Neal","year":"2011","journal-title":"Handb. Markov Chain Monte Carlo"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1007\/s11222-019-09885-x","article-title":"Modified Hamiltonian Monte Carlo for Bayesian Inference","volume":"30","author":"Akhmatskaya","year":"2020","journal-title":"Stat. Comput."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1501","DOI":"10.3150\/12-BEJ414","article-title":"Optimal tuning of the hybrid Monte Carlo algorithm","volume":"19","author":"Beskos","year":"2013","journal-title":"Bernoulli"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2257","DOI":"10.3150\/16-BEJ810","article-title":"The geometric foundations of hamiltonian monte carlo","volume":"23","author":"Betancourt","year":"2017","journal-title":"Bernoulli"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1111\/j.1467-9868.2010.00765.x","article-title":"Riemann manifold langevin and hamiltonian monte carlo methods","volume":"73","author":"Girolami","year":"2011","journal-title":"J. R. Stat. Soc. Ser. B (Stat. Methodol.)"},{"key":"ref_9","unstructured":"Song, J., Zhao, S., and Ermon, S. (2017, January 4\u20139). A-nice-mc: Adversarial training for mcmc. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_10","unstructured":"Levy, D., Hoffman, M.D., and Sohl-Dickstein, J. (May, January 30). Generalizing Hamiltonian Monte Carlo with Neural Networks. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Gu, M., Sun, S., and Liu, Y. (2019). Dynamical Sampling with Langevin Normalization Flows. Entropy, 21.","DOI":"10.3390\/e21111096"},{"key":"ref_12","unstructured":"Hoffman, M., Sountsov, P., Dillon, J.V., Langmore, I., Tran, D., and Vasudevan, S. (2019). Neutra-lizing bad geometry in hamiltonian monte carlo using neural transport. arXiv."},{"key":"ref_13","unstructured":"Nijkamp, E., Gao, R., Sountsov, P., Vasudevan, S., Pang, B., Zhu, S.C., and Wu, Y.N. (2020). Learning Energy-based Model with Flow-based Backbone by Neural Transport MCMC. arXiv."},{"key":"ref_14","unstructured":"Titsias, M., and Dellaportas, P. (2019, January 8\u201314). Gradient-based Adaptive Markov Chain Monte Carlo. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1093\/biomet\/57.1.97","article-title":"Monte Carlo sampling methods using Markov chains and their applications","volume":"57","author":"Hastings","year":"1970","journal-title":"Biometrika"},{"key":"ref_16","unstructured":"Sohl-Dickstein, J., Mudigonda, M., and DeWeese, M.R. (2014). Hamiltonian Monte Carlo without detailed balance. arXiv."},{"key":"ref_17","unstructured":"Dinh, L., Sohl-Dickstein, J., and Bengio, S. (2016). Density estimation using real nvp. arXiv."},{"key":"ref_18","unstructured":"Kobyzev, I., Prince, S., and Brubaker, M.A. (2019). Normalizing flows: Introduction and ideas. arXiv."},{"key":"ref_19","unstructured":"Papamakarios, G., Nalisnick, E., Rezende, D.J., Mohamed, S., and Lakshminarayanan, B. (2019). Normalizing flows for probabilistic modeling and inference. arXiv."},{"key":"ref_20","unstructured":"Spanbauer, S., Freer, C., and Mansinghka, V. (2020). Deep Involutive Generative Models for Neural MCMC. arXiv."},{"key":"ref_21","unstructured":"Dinh, L., Krueger, D., and Bengio, Y. (2014). Nice: Non-linear independent components estimation. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Marzouk, Y., Moselhy, T., Parno, M., and Spantini, A. (2016). An introduction to sampling via measure transport. arXiv.","DOI":"10.1007\/978-3-319-11259-6_23-1"},{"key":"ref_23","unstructured":"Langmore, I., Dikovsky, M., Geraedts, S., Norgaard, P., and Von Behren, R. (2019). A Condition Number for Hamiltonian Monte Carlo. arXiv."},{"key":"ref_24","unstructured":"Salimans, T., Kingma, D., and Welling, M. (2015, January 7\u20139). Markov chain monte carlo and variational inference: Bridging the gap. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_25","unstructured":"Zhang, Y., Hern\u00e1ndez-Lobato, J.M., and Ghahramani, Z. (2018). Ergodic measure preserving flows. arXiv."},{"key":"ref_26","first-page":"201243","article-title":"A geometric fuzzy-based approach for airport clustering","volume":"2014","author":"Postorino","year":"2014","journal-title":"Adv. Fuzzy Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Tkachenko, R., Izonin, I., Kryvinska, N., Dronyuk, I., and Zub, K. (2020). An approach towards increasing prediction accuracy for the recovery of missing IoT data based on the GRNN-SGTM ensemble. Sensors, 20.","DOI":"10.3390\/s20092625"},{"key":"ref_28","unstructured":"Neklyudov, K., Egorov, E., Shvechikov, P., and Vetrov, D. (2018). Metropolis-hastings view on variational inference and adversarial training. arXiv."},{"key":"ref_29","unstructured":"Thin, A., Kotelevskii, N., Durmus, A., Panov, M., and Moulines, E. (2020, January 12\u201318). Metropolized Flow: From Invertible Flow to MCMC. Proceedings of the ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, virtual event."},{"key":"ref_30","first-page":"343","article-title":"Adaptively scaling the Metropolis algorithm using expected squared jumped distance","volume":"20","author":"Pasarica","year":"2010","journal-title":"Stat. Sin."},{"key":"ref_31","unstructured":"Poole, B., Ozair, S., Oord, A.V.d., Alemi, A.A., and Tucker, G. (2019). On variational bounds of mutual information. arXiv."},{"key":"ref_32","unstructured":"Song, J., and Ermon, S. (2019). Understanding the limitations of variational mutual information estimators. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1214\/aos\/1056562461","article-title":"Slice sampling","volume":"31","author":"Neal","year":"2003","journal-title":"Ann. Stat."},{"key":"ref_34","first-page":"1593","article-title":"The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo","volume":"15","author":"Hoffman","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Betancourt, M. (2013). A general metric for Riemannian manifold Hamiltonian Monte Carlo. Lecture Notes in Computer Science, Proceedings of the International Conference on Geometric Science of Information, Paris, France, 28\u201330 August 2013, Springer.","DOI":"10.1007\/978-3-642-40020-9_35"},{"key":"ref_36","unstructured":"Xie, J., Lu, Y., Zhu, S.C., and Wu, Y. (2016, January 19\u201324). A theory of generative convnet. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_37","unstructured":"Du, Y., and Mordatch, I. (2019, January 8\u201314). Implicit generation and generalization in energy-based models. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Tieleman, T. (2008, January 5\u20139). Training restricted Boltzmann machines using approximations to the likelihood gradient. Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland.","DOI":"10.1145\/1390156.1390290"},{"key":"ref_39","unstructured":"Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. (2017, January 4\u20139). Gans trained by a two time-scale update rule converge to a local nash equilibrium. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_40","unstructured":"Hoffman, M.D. (2017, January 6\u201311). Learning deep latent Gaussian models with Markov chain Monte Carlo. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_41","unstructured":"Che, T., Zhang, R., Sohl-Dickstein, J., Larochelle, H., Paull, L., Cao, Y., and Bengio, Y. (2020). Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling. arXiv."},{"key":"ref_42","unstructured":"Yu, L., Song, Y., Song, J., and Ermon, S. (2020, January 12\u201318). Training Deep Energy-Based Models with f-Divergence Minimization. Proceedings of the International Conference on Machine Learning, Virtual Event, Vienna, Austria."},{"key":"ref_43","unstructured":"Grathwohl, W., Wang, K.C., Jacobsen, J.H., Duvenaud, D., Norouzi, M., and Swersky, K. (2019, January 6\u20139). Your classifier is secretly an energy based model and you should treat it like one. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/3\/269\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:28:12Z","timestamp":1760160492000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/3\/269"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,25]]},"references-count":43,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["e23030269"],"URL":"https:\/\/doi.org\/10.3390\/e23030269","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,25]]}}}