{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,23]],"date-time":"2025-10-23T20:51:32Z","timestamp":1761252692789},"reference-count":12,"publisher":"MIT Press - Journals","issue":"8","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2011,8]]},"abstract":"<jats:p> Two recently proposed learning algorithms, herding and fast persistent contrastive divergence (FPCD), share the following interesting characteristic: they exploit changes in the model parameters while sampling in order to escape modes and mix better during the sampling process that is part of the learning algorithm. We justify such approaches as ways to escape modes while keeping approximately the same asymptotic distribution of the Markov chain. In that spirit, we extend FPCD using an idea borrowed from Herding in order to obtain a pure sampling algorithm, which we call the rates-FPCD sampler. Interestingly, this sampler can improve the model as we collect more samples, since it optimizes a lower bound on the log likelihood of the training data. We provide empirical evidence that this new algorithm displays substantially better and more robust mixing than Gibbs sampling. <\/jats:p>","DOI":"10.1162\/neco_a_00158","type":"journal-article","created":{"date-parts":[[2011,4,26]],"date-time":"2011-04-26T20:21:23Z","timestamp":1303849283000},"page":"2058-2073","source":"Crossref","is-referenced-by-count":38,"title":["Quickly Generating Representative Samples from an RBM-Derived Process"],"prefix":"10.1162","volume":"23","author":[{"given":"Olivier","family":"Breuleux","sequence":"first","affiliation":[{"name":"D\u00e9partment Informatique et Recherche Operationnelle, University of Montreal, Montreal, Quebec H3T 1J8, Canada"}]},{"given":"Yoshua","family":"Bengio","sequence":"additional","affiliation":[{"name":"D\u00e9partment Informatique et Recherche Operationnelle, University of Montreal, Montreal, Quebec H3T 1J8, Canada"}]},{"given":"Pascal","family":"Vincent","sequence":"additional","affiliation":[{"name":"D\u00e9partment Informatique et Recherche Operationnelle, University of Montreal, Montreal, Quebec H3T 1J8, Canada"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2008.11-07-647"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1007\/978-93-86279-38-5"},{"key":"B3","volume-title":"Unlearning for better mixing","author":"Breuleux O.","year":"2010"},{"key":"B4","first-page":"145","volume-title":"Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2010)","author":"Desjardins G.","year":"2010"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1049\/cp:19991075"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2006.18.7.1527"},{"key":"B8","volume-title":"Boltzmann machines: Constraint satisfaction networks that learn","author":"Hinton G. E.","year":"1984"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390290"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553506"},{"key":"B11","volume-title":"Proceedings of the 25th Conference in Uncertainty in Artificial Intelligence (UAI\u201909)","author":"Welling M","year":"2009"},{"key":"B12","volume-title":"Proceedings of the 26th International Conference on Machine Learning","author":"Welling M","year":"2009"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1080\/17442509908834179"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/NECO_a_00158","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:38:51Z","timestamp":1615585131000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/23\/8\/2058-2073\/7675"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,8]]},"references-count":12,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2011,8]]}},"alternative-id":["10.1162\/NECO_a_00158"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_00158","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,8]]}}}