{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:57:22Z","timestamp":1760241442782,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2018,4,4]],"date-time":"2018-04-04T00:00:00Z","timestamp":1522800000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>The machine learning techniques for Markov random fields are fundamental in various fields involving pattern recognition, image processing, sparse modeling, and earth science, and a Boltzmann machine is one of the most important models in Markov random fields. However, the inference and learning problems in the Boltzmann machine are NP-hard. The investigation of an effective learning algorithm for the Boltzmann machine is one of the most important challenges in the field of statistical machine learning. In this paper, we study Boltzmann machine learning based on the (first-order) spatial Monte Carlo integration method, referred to as the 1-SMCI learning method, which was proposed in the author\u2019s previous paper. In the first part of this paper, we compare the method with the maximum pseudo-likelihood estimation (MPLE) method using a theoretical and a numerical approaches, and show the 1-SMCI learning method is more effective than the MPLE. In the latter part, we compare the 1-SMCI learning method with other effective methods, ratio matching and minimum probability flow, using a numerical experiment, and show the 1-SMCI learning method outperforms them.<\/jats:p>","DOI":"10.3390\/a11040042","type":"journal-article","created":{"date-parts":[[2018,4,4]],"date-time":"2018-04-04T12:42:57Z","timestamp":1522845777000},"page":"42","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Learning Algorithm of Boltzmann Machine Based on Spatial Monte Carlo Integration Method"],"prefix":"10.3390","volume":"11","author":[{"given":"Muneki","family":"Yasuda","sequence":"first","affiliation":[{"name":"Graduate School of Science and Engineering, Yamagata University, Yamagata 992-8510, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,4,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1527","DOI":"10.1162\/neco.2006.18.7.1527","article-title":"A fast learning algorithm for deep belief net","volume":"18","author":"Hinton","year":"2006","journal-title":"Neural Comput."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1967","DOI":"10.1162\/NECO_a_00311","article-title":"An Efficient Learning Procedure for Deep Boltzmann Machines","volume":"24","author":"Salakhutdinov","year":"2012","journal-title":"Neural Comput."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Blake, A., Kohli, P., and Rother, C. (2011). Markov Random Fields for Vision and Image Processing, The MIT Press.","DOI":"10.7551\/mitpress\/8579.001.0001"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Rish, I., and Grabarnik, G. (2014). Sparse Modeling: Theory, Algorithms, and Applications, CRC Press.","DOI":"10.1201\/b17758"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1186\/1880-5981-66-5","article-title":"Markov random field modeling for mapping geofluid distributions from seismic velocity structures","volume":"66","author":"Kuwatani","year":"2014","journal-title":"Earth Planets Space"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"042137","DOI":"10.1103\/PhysRevE.90.042137","article-title":"Markov-random-field modeling for linear seismic tomography","volume":"90","author":"Kuwatani","year":"2014","journal-title":"Phys. Rev. E"},{"key":"ref_7","first-page":"147","article-title":"A learning algorithm for Boltzmann machines","volume":"9","author":"Ackley","year":"1985","journal-title":"Cogn. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000001","article-title":"Graphical Models, Exponential Families, and Variational Inference","volume":"1","author":"Wainwright","year":"2008","journal-title":"Found. Trends Mach. Learn."},{"key":"ref_9","unstructured":"Murphy, K.P. (2013). Machine Learning: A Probabilistic Perspective, MIT Press."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1162\/089976698300017386","article-title":"Efficient Learning in Boltzmann Machines Using Linear Response Theory","volume":"10","author":"Kappen","year":"1998","journal-title":"Neural Comput."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2302","DOI":"10.1103\/PhysRevE.58.2302","article-title":"Mean-field theory of Boltzmann machine learning","volume":"58","author":"Tanaka","year":"1998","journal-title":"Phys. Rev. E"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3130","DOI":"10.1162\/neco.2009.08-08-844","article-title":"Approximate Learning Algorithm in Boltzmann Machines","volume":"21","author":"Yasuda","year":"2009","journal-title":"Neural Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"055001","DOI":"10.1088\/1751-8113\/42\/5\/055001","article-title":"Small-correlation expansions for the inverse Ising problem","volume":"42","author":"Sessak","year":"2009","journal-title":"J. Phys. A Math. Theor."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"P09020","DOI":"10.1088\/1742-5468\/2013\/09\/P09020","article-title":"Approximate inverse Ising models close to a Bethe reference point","volume":"2013","author":"Furtlehner","year":"2013","journal-title":"J. Stat. Mech. Theor. Exp."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"22","DOI":"10.3389\/neuro.10.022.2009","article-title":"Statistical physics of pairwise probability models","volume":"3","author":"Roudi","year":"2009","journal-title":"Front. Comput. Neurosci."},{"key":"ref_16","first-page":"179","article-title":"Statistical Analysis of Non-Lattice Data","volume":"24","author":"Besag","year":"1975","journal-title":"J. R. Stat. Soc. D"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"090201","DOI":"10.1103\/PhysRevLett.108.090201","article-title":"Inverse Ising Inference Using All the Data","volume":"108","author":"Aurell","year":"2012","journal-title":"Phys. Rev. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1771","DOI":"10.1162\/089976602760128018","article-title":"Training products of experts by minimizing contrastive divergence","volume":"8","author":"Hinton","year":"2002","journal-title":"Neural Comput."},{"key":"ref_19","first-page":"695","article-title":"Estimation of non-normalized statistical models using score matching","volume":"6","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_20","unstructured":"Sohl-Dickstein, J., Battaglino, P.B., and DeWeese, M.R. (\u2013, January 28). Minimum Probability Flow Learning. Proceedings of the 28th International Conference on International Conference on Machine Learning (ICML\u201911), Bellevue, WA, USA."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"220601","DOI":"10.1103\/PhysRevLett.107.220601","article-title":"New Method for Parameter Estimation in Probabilistic Models: Minimum Probability Flow","volume":"107","author":"Battaglino","year":"2011","journal-title":"Phys. Rev. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"034001","DOI":"10.7566\/JPSJ.84.034001","article-title":"Monte Carlo Integration Using Spatial Structure of Markov Random Field","volume":"84","author":"Yasuda","year":"2015","journal-title":"J. Phys. Soc. Japan"},{"key":"ref_23","unstructured":"Lehmann, E.L., and Casella, G. (1998). Theory of Point Estimation, Springer."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2283","DOI":"10.1162\/neco.2006.18.10.2283","article-title":"Consistency of Pseudo likelihood Estimation of Fully Visible Boltzmann Machines","volume":"18","year":"2006","journal-title":"Neural Comput."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1090\/conm\/080\/999014","article-title":"Composite Likelihood Methods","volume":"80","author":"Lindsay","year":"1988","journal-title":"Contemporary Math."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1214\/aoap\/1177005877","article-title":"Pseudolikelihood for Exponential Family Models of Spatial Point Processes","volume":"1","author":"Jensen","year":"1991","journal-title":"Ann. Appl. Probab."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1016\/0031-9163(65)90978-9","article-title":"Generalized Exact Formula for the Correlations of the Ising Model and Other Classical Systems","volume":"19","author":"Suzuki","year":"1965","journal-title":"Phys. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Tieleman, T. (2008, January 5\u20139). Training Restricted Boltzmann Machines Using Approximations to the Likelihood Gradient. Proceedings of the 25th International Conference on Machine Learning (ICML), Helsinki, Finland.","DOI":"10.1145\/1390156.1390290"},{"key":"ref_29","unstructured":"Salakhutdinov, R., and Hinton, G.E. (2009, January 16\u201318). Deep Boltzmann Machines. Proceedings of the 12th International Conference on Artificial Intelligence and Statistics (AISTATS 2009), Clearwater Beach, FL, USA."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1109\/TNSRE.2016.2519350","article-title":"Discriminative Feature Extraction via Multivariate Linear Regression for SSVEP-Based BCI","volume":"24","author":"Wang","year":"2016","journal-title":"IEEE Trans. Neural Syst. Rehabilitat. Eng."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2256","DOI":"10.1109\/TNNLS.2015.2476656","article-title":"Sparse Bayesian Classification of EEG for Brain\u2013Computer Interface","volume":"27","author":"Zhang","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1016\/j.eswa.2017.12.015","article-title":"Multi-kernel extreme learning machine for EEG classification in brain-computer interfaces","volume":"96","author":"Zhang","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1750039","DOI":"10.1142\/S0129065717500393","article-title":"A Novel Multilayer Correlation Maximization Model for Improving CCA-Based Frequency Recognition in SSVEP Brain\u2013Computer Interface","volume":"28","author":"Jiao","year":"2018","journal-title":"Int. J. Neural Syst."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/11\/4\/42\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:59:39Z","timestamp":1760194779000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/11\/4\/42"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,4,4]]},"references-count":33,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2018,4]]}},"alternative-id":["a11040042"],"URL":"https:\/\/doi.org\/10.3390\/a11040042","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2018,4,4]]}}}