{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T09:21:24Z","timestamp":1778145684500,"version":"3.51.4"},"reference-count":48,"publisher":"MIT Press - Journals","issue":"8","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2012,8]]},"abstract":"<jats:p> We present a new learning algorithm for Boltzmann machines that contain many layers of hidden variables. Data-dependent statistics are estimated using a variational approximation that tends to focus on a single mode, and data-independent statistics are estimated using persistent Markov chains. The use of two quite different techniques for estimating the two types of statistic that enter into the gradient of the log likelihood makes it practical to learn Boltzmann machines with multiple hidden layers and millions of parameters. The learning can be made more efficient by using a layer-by-layer pretraining phase that initializes the weights sensibly. The pretraining also allows the variational inference to be initialized sensibly with a single bottom-up pass. We present results on the MNIST and NORB data sets showing that deep Boltzmann machines learn very good generative models of handwritten digits and 3D objects. We also show that the features discovered by deep Boltzmann machines are a very effective way to initialize the hidden layers of feedforward neural nets, which are then discriminatively fine-tuned. <\/jats:p>","DOI":"10.1162\/neco_a_00311","type":"journal-article","created":{"date-parts":[[2012,4,17]],"date-time":"2012-04-17T21:33:42Z","timestamp":1334698422000},"page":"1967-2006","source":"Crossref","is-referenced-by-count":373,"title":["An Efficient Learning Procedure for Deep Boltzmann Machines"],"prefix":"10.1162","volume":"24","author":[{"given":"Ruslan","family":"Salakhutdinov","sequence":"first","affiliation":[{"name":"Department of Statistics, University of Toronto, Toronto, Ontario M5S 3G3, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Geoffrey","family":"Hinton","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Toronto, Toronto, Ontario M5S 3G3, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1561\/2200000006"},{"key":"B2","first-page":"153","volume-title":"Advances in neural information processing systems, 11","author":"Bengio Y.","year":"2007"},{"key":"B3","volume-title":"Large-scale kernel machines","author":"Bengio Y.","year":"2007"},{"key":"B5","first-page":"469","volume-title":"Advances in neural information processing systems, 23","author":"Dahl G. E.","year":"2010"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1023\/A:1012454411458"},{"key":"B7","first-page":"145","volume-title":"Proceedings of the 13th International Workshop on AI and Statistics","author":"Desjardins G.","year":"2010"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.1984.4767596"},{"key":"B10","first-page":"513","volume-title":"Advances in neural information processing systems, 17","author":"Goldberger J.","year":"2004"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1162\/089976602760128018"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1016\/S0079-6123(06)65034-6"},{"key":"B13","volume-title":"A practical guide to training restricted Boltzmann machines","author":"Hinton G. E.","year":"2010"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2006.18.7.1527"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1126\/science.1127647"},{"key":"B16","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Hinton G. E.","year":"1983"},{"key":"B17","first-page":"3","volume-title":"Advances in neural information processing systems, 6","author":"Hinton G. E.","year":"1994"},{"key":"B18","volume-title":"Learning in graphical models","author":"Jordan M. I.","year":"1999"},{"key":"B19","volume-title":"Advances in neural information processing systems","volume":"10","author":"Kappen H. J.","year":"1998"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1126\/science.220.4598.671"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2004.1315150"},{"key":"B22","first-page":"481","volume-title":"Proc. Int. Conf. on Independent Component Analysis","author":"Marks T. K.","year":"2001"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1109\/TASL.2011.2109382"},{"key":"B24","first-page":"1137","volume-title":"Advances in neural information processing systems, 21","author":"Murray I.","year":"2009"},{"key":"B25","first-page":"1145","volume-title":"Advances in neural information processing systems, 21","author":"Nair V.","year":"2009"},{"key":"B26","first-page":"807","volume-title":"Proc. 27th International Conference on Machine Learning","author":"Nair V.","year":"2010"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(92)90065-6"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008923215028"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-011-5014-9_12"},{"key":"B30","first-page":"1121","volume-title":"Advances in neural information processing systems","volume":"20","author":"Osindero S.","year":"2008"},{"key":"B31","first-page":"995","volume":"1","author":"Peterson C.","year":"1987","journal-title":"Complex Systems"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2007.383157"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729586"},{"key":"B35","first-page":"1598","volume-title":"Advances in neural information processing systems, 23","author":"Salakhutdinov R. R.","year":"2009"},{"key":"B36","first-page":"412","volume-title":"Proceedings of the International Conference on Artificial Intelligence and Statistics","volume":"11","author":"Salakhutdinov R. R.","year":"2007"},{"key":"B37","first-page":"448","volume-title":"Proceedings of the International Conference on Artificial Intelligence and Statistics","volume":"12","author":"Salakhutdinov R. R.","year":"2009"},{"key":"B38","first-page":"1607","volume-title":"Advances in neural information processing systems, 22","author":"Salakhutdinov R. R.","year":"2009"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273596"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390266"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0700622104"},{"key":"B42","first-page":"194","volume-title":"Parallel distributed processing, Vol. 1: Foundations","author":"Smolensky P.","year":"1986"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390290"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553506"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390294"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553517"},{"key":"B47","first-page":"1481","volume-title":"Advances in neural information processing systems, 17","author":"Welling M.","year":"2005"},{"key":"B48","volume-title":"An analysis of contrastive divergence learning in Gaussian Boltzmann machines","author":"Williams C.","year":"2002"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1007\/BF00341287"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1080\/17442509908834179"},{"key":"B51","first-page":"1593","volume-title":"Advances in neural information processing systems","volume":"17","author":"Yuille A. L.","year":"2004"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/NECO_a_00311","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:39:24Z","timestamp":1615585164000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/24\/8\/1967-2006\/7793"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,8]]},"references-count":48,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2012,8]]}},"alternative-id":["10.1162\/NECO_a_00311"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_00311","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,8]]}}}