{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T20:24:38Z","timestamp":1773779078764,"version":"3.50.1"},"reference-count":38,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T00:00:00Z","timestamp":1679616000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Shanghai Municipal Project","award":["20511100900"],"award-info":[{"award-number":["20511100900"]}]},{"name":"Shanghai Municipal Project","award":["62076096"],"award-info":[{"award-number":["62076096"]}]},{"name":"Shanghai Municipal Project","award":["62006078"],"award-info":[{"award-number":["62006078"]}]},{"name":"Shanghai Municipal Project","award":["ZF1213"],"award-info":[{"award-number":["ZF1213"]}]},{"name":"Shanghai Municipal Project","award":["22ZR1421700"],"award-info":[{"award-number":["22ZR1421700"]}]},{"DOI":"10.13039\/501100001809","name":"NSFC Projects","doi-asserted-by":"publisher","award":["20511100900"],"award-info":[{"award-number":["20511100900"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"NSFC Projects","doi-asserted-by":"publisher","award":["62076096"],"award-info":[{"award-number":["62076096"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"NSFC Projects","doi-asserted-by":"publisher","award":["62006078"],"award-info":[{"award-number":["62006078"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"NSFC Projects","doi-asserted-by":"publisher","award":["ZF1213"],"award-info":[{"award-number":["ZF1213"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"NSFC Projects","doi-asserted-by":"publisher","award":["22ZR1421700"],"award-info":[{"award-number":["22ZR1421700"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Shanghai Knowledge Service Platform Project","award":["20511100900"],"award-info":[{"award-number":["20511100900"]}]},{"name":"Shanghai Knowledge Service Platform Project","award":["62076096"],"award-info":[{"award-number":["62076096"]}]},{"name":"Shanghai Knowledge Service Platform Project","award":["62006078"],"award-info":[{"award-number":["62006078"]}]},{"name":"Shanghai Knowledge Service Platform Project","award":["ZF1213"],"award-info":[{"award-number":["ZF1213"]}]},{"name":"Shanghai Knowledge Service Platform Project","award":["22ZR1421700"],"award-info":[{"award-number":["22ZR1421700"]}]},{"name":"STCSM Project","award":["20511100900"],"award-info":[{"award-number":["20511100900"]}]},{"name":"STCSM Project","award":["62076096"],"award-info":[{"award-number":["62076096"]}]},{"name":"STCSM Project","award":["62006078"],"award-info":[{"award-number":["62006078"]}]},{"name":"STCSM Project","award":["ZF1213"],"award-info":[{"award-number":["ZF1213"]}]},{"name":"STCSM Project","award":["22ZR1421700"],"award-info":[{"award-number":["22ZR1421700"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["20511100900"],"award-info":[{"award-number":["20511100900"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["62076096"],"award-info":[{"award-number":["62076096"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["62006078"],"award-info":[{"award-number":["62006078"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["ZF1213"],"award-info":[{"award-number":["ZF1213"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["22ZR1421700"],"award-info":[{"award-number":["22ZR1421700"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The Hamiltonian Monte Carlo (HMC) sampling algorithm exploits Hamiltonian dynamics to construct efficient Markov Chain Monte Carlo (MCMC), which has become increasingly popular in machine learning and statistics. Since HMC uses the gradient information of the target distribution, it can explore the state space much more efficiently than random-walk proposals, but may suffer from high autocorrelation. In this paper, we propose Langevin Hamiltonian Monte Carlo (LHMC) to reduce the autocorrelation of the samples. Probabilistic inference involving multi-modal distributions is very difficult for dynamics-based MCMC samplers, which is easily trapped in the mode far away from other modes. To tackle this issue, we further propose a variational hybrid Monte Carlo (VHMC) which uses a variational distribution to explore the phase space and find new modes, and it is capable of sampling from multi-modal distributions effectively. A formal proof is provided that shows that the proposed method can converge to target distributions. Both synthetic and real datasets are used to evaluate its properties and performance. The experimental results verify the theory and show superior performance in multi-modal sampling.<\/jats:p>","DOI":"10.3390\/e25040560","type":"journal-article","created":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T08:43:03Z","timestamp":1679647383000},"page":"560","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Variational Hybrid Monte Carlo for Efficient Multi-Modal Data Sampling"],"prefix":"10.3390","volume":"25","author":[{"given":"Shiliang","family":"Sun","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, East China Normal University, Shanghai 200062, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0158-5330","authenticated-orcid":false,"given":"Jing","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, East China Normal University, Shanghai 200062, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Minghao","family":"Gu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, East China Normal University, Shanghai 200062, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shanhu","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, East China Normal University, Shanghai 200062, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,24]]},"reference":[{"key":"ref_1","unstructured":"Sun, S., and Zhao, J. (2020). Pattern Recognition and Machine Learning, Tsinghua University Press."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/0370-2693(87)91197-X","article-title":"Hybrid Monte Carlo","volume":"195","author":"Duane","year":"1987","journal-title":"Phys. Lett. B"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Brooks, S., Gelman, A., Jones, G., and Meng, X. (2011). Handbook of Markov chain Monte Carlo, CRC Press.","DOI":"10.1201\/b10905"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1214\/aos\/1056562461","article-title":"Slice sampling","volume":"31","author":"Neal","year":"2003","journal-title":"Ann. Stat."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2797","DOI":"10.1007\/s00180-013-0429-2","article-title":"On the flexibility of the design of multiple try Metropolis schemes","volume":"28","author":"Martino","year":"2013","journal-title":"Comput. Stat."},{"key":"ref_6","unstructured":"Wang, Z., Mohamed, S., and Freitas, N. (2013, January 16\u201321). Adaptive Hamiltonian and Riemann manifold Monte Carlo. Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_7","first-page":"1593","article-title":"The No-U-turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo","volume":"15","author":"Hoffman","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_8","unstructured":"Neal, R.M. (1993). Probabilistic Inference Using Markov Chain Monte Carlo Methods, Department of Computer Science, University of Toronto."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1080\/01621459.2000.10474285","article-title":"Computational and inferential difficulties with mixture posterior distributions","volume":"95","author":"Celeux","year":"2000","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1023\/A:1008923215028","article-title":"Annealed importance sampling","volume":"11","author":"Neal","year":"2001","journal-title":"Stat. Comput."},{"key":"ref_11","unstructured":"Rudoy, D., and Wolfe, P.J. (November, January 29). Monte Carlo methods for multi-modal distributions. Proceedings of the Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1111\/j.1467-9868.2010.00765.x","article-title":"Riemann manifold Langevin and Hamiltonian Monte Carlo methods","volume":"73","author":"Girolami","year":"2011","journal-title":"J. R. Stat. Soc. Ser."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1126\/science.220.4598.671","article-title":"Optimization by simulated annealing","volume":"220","author":"Kirkpatrick","year":"1983","journal-title":"Science"},{"key":"ref_14","unstructured":"Sminchisescu, C., and Welling, M. (2007, January 21\u201324). Generalized darting Monte Carlo. Proceedings of the Artificial Intelligence and Statistics, San Juan, Puerto Rico."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Lan, S., Streets, J., and Shahbaba, B. (2014, January 27\u201331). Wormhole Hamiltonian Monte Carlo. Proceedings of the Association for the Advancement of Artificial Intelligence, Qu\u00e9bec City, QC, Canada.","DOI":"10.1609\/aaai.v28i1.9006"},{"key":"ref_16","unstructured":"Tripuraneni, N., Rowland, M., Ghahramani, Z., and Turner, R. (2017, January 6\u201311). Magnetic Hamiltonian Monte Carlo. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2607","DOI":"10.1103\/PhysRevLett.57.2607","article-title":"Replica Monte Carlo simulation of spin glasses","volume":"57","author":"Swendsen","year":"1986","journal-title":"Phys. Rev. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"6994","DOI":"10.1063\/1.1358861","article-title":"Smart darting Monte Carlo","volume":"114","author":"Andricioaei","year":"2001","journal-title":"J. Chem. Phys."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1016\/0021-9991(77)90121-8","article-title":"Nonphysical sampling distributions in Monte Carlo free-energy estimation: Umbrella sampling","volume":"23","author":"Torrie","year":"1977","journal-title":"J. Comput. Phys."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational inference: A review for statisticians","volume":"112","author":"Blei","year":"2017","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"720","DOI":"10.1162\/neco.1992.4.5.720","article-title":"The evidence framework applied to classification networks","volume":"4","author":"MacKay","year":"1992","journal-title":"Neural Comput."},{"key":"ref_22","first-page":"723","article-title":"A kernel two-sample test","volume":"13","author":"Gretton","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_23","unstructured":"Zhang, Y., Wang, X., Chen, C., Henao, R., Fan, K., and Carin, L. (2016, January 5\u201310). Towards unifying Hamiltonian Monte Carlo and slice sampling. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_24","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Leimkuhler, B., and Reich, S. (2004). Simulating Hamiltonian Dynamics, Cambridge University Press.","DOI":"10.1017\/CBO9780511614118"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1016\/0009-2614(84)80098-6","article-title":"Stochastic boundary conditions for molecular dynamics simulations of ST2 water","volume":"105","author":"Karplus","year":"1984","journal-title":"Chem. Phys. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1601","DOI":"10.1137\/060677148","article-title":"Accurate stationary densities with partitioned numerical methods for stochastic differential equations","volume":"47","author":"Burrage","year":"2009","journal-title":"SIAM J. Numer. Anal."},{"key":"ref_28","unstructured":"Milstein, G.N., and Tretyakov, M.V. (2013). Stochastic Numerics for Mathematical Physics, Springer."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1023\/A:1023562417138","article-title":"Langevin Diffusions and Metropolis-Hastings Algorithms","volume":"4","author":"Roberts","year":"2002","journal-title":"Methodol. Comput. Appl. Probab."},{"key":"ref_30","unstructured":"Kloeden, P.E., and Platen, E. (2013). Numerical Solution of Stochastic Differential Equations, Springer."},{"key":"ref_31","first-page":"34","article-title":"Rational construction of stochastic numerical methods for molecular sampling","volume":"2013","author":"Leimkuhler","year":"2012","journal-title":"Appl. Math. Res. Express"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1063\/1.4802990","article-title":"Robust and efficient configurational molecular sampling via Langevin dynamics","volume":"138","author":"Leimkuhler","year":"2013","journal-title":"J. Chem. Phys."},{"key":"ref_33","unstructured":"Ahn, S., Chen, Y., and Welling, M. (May, January 29). Distributed and adaptive darting Monte Carlo through regenerations. Proceedings of the Artificial Intelligence and Statistics, Scottsdale, AZ, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Livingstone, S., Betancourt, M., Byrne, S., and Girolami, M. (2016). On the geometric ergodicity of Hamiltonian Monte Carlo. arXiv.","DOI":"10.3150\/16-BEJ810"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1093\/bioinformatics\/btl242","article-title":"Integrating structured biological data by kernel maximum mean discrepancy","volume":"22","author":"Borgwardt","year":"2006","journal-title":"Bioinformatics"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Freedman, D.A. (2009). Statistical Models: Theory and Practice, Cambridge University Press.","DOI":"10.1017\/CBO9780511815867"},{"key":"ref_37","unstructured":"Asuncion, A., and Newman, D. (2013). UCI Machine Learning Repository, University of California."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1148\/radiology.148.3.6878708","article-title":"A method of comparing the areas under receiver operating characteristic curves derived from the same cases","volume":"148","author":"Hanley","year":"1983","journal-title":"Radiology"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/4\/560\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:02:31Z","timestamp":1760122951000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/4\/560"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,24]]},"references-count":38,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["e25040560"],"URL":"https:\/\/doi.org\/10.3390\/e25040560","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,24]]}}}