{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,4,17]],"date-time":"2025-04-17T06:32:25Z","timestamp":1744871545619},"reference-count":38,"publisher":"MIT Press","issue":"4","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,3,18]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Capsule networks (see Hinton et al., 2018) aim to encode knowledge of and reason about the relationship between an object and its parts. In this letter, we specify a generative model for such data and derive a variational algorithm for inferring the transformation of each model object in a scene and the assignments of observed parts to the objects. We derive a learning algorithm for the object models, based on variational expectation maximization (Jordan et al., 1999). We also study an alternative inference algorithm based on the RANSAC method of Fischler and Bolles (1981). We apply these inference methods to data generated from multiple geometric objects like squares and triangles (\u201cconstellations\u201d) and data from a parts-based model of faces. Recent work by Kosiorek et al. (2019) has used amortized inference via stacked capsule autoencoders to tackle this problem; our results show that we significantly outperform them where we can make comparisons (on the constellations data).<\/jats:p>","DOI":"10.1162\/neco_a_01564","type":"journal-article","created":{"date-parts":[[2023,2,6]],"date-time":"2023-02-06T23:25:57Z","timestamp":1675725957000},"page":"727-761","update-policy":"http:\/\/dx.doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":2,"title":["Inference and Learning for Generative Capsule Models"],"prefix":"10.1162","volume":"35","author":[{"given":"Alfredo","family":"Nazabal","sequence":"first","affiliation":[{"name":"Amazon Development Centre Scotland, Edinburgh EH1 3EG, U.K. alfrena@amazon.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nikolaos","family":"Tsagkas","sequence":"additional","affiliation":[{"name":"School of Informatics, University of Edinburgh, Edinburgh EH8 9AB, U.K. n.tsagkas@ed.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christopher K. I.","family":"Williams","sequence":"additional","affiliation":[{"name":"School of Informatics, University of Edinburgh, Edinburgh EH8 9AB, U.K."},{"name":"Alan Turing Institute, London NW1 2DB, U.K. c.k.i.williams@ed.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2023,3,18]]},"reference":[{"key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1007\/BF00055803","article-title":"Paraperspective \u2261 affine","volume":"19","author":"Basri","year":"1996","journal-title":"International Journal of Computer Vision"},{"key":"2023032023153486700_","doi-asserted-by":"crossref","DOI":"10.1145\/1553374.1553380","article-title":"Curriculum learning","volume-title":"Proc. 26th International Conference on Machine Learning","author":"Bengio","year":"2009"},{"key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1037\/0033-295X.94.2.115","article-title":"Recognition-by-components: A theory of human image understanding","volume":"94","author":"Biederman","year":"1987","journal-title":"Psychological Review"},{"key":"2023032023153486700_","volume-title":"Pattern recognition and machine learning","author":"Bishop","year":"2006"},{"key":"2023032023153486700_","author":"De Sousa Ribeiro","year":"2022","journal-title":"Learning with capsules: A survey"},{"key":"2023032023153486700_","first-page":"3749","article-title":"Capsule routing via variational Bayes","volume-title":"Proc. Thirty-Fourth AAAI Conference on Artificial Intelligence","author":"De Sousa Ribeiro","year":"2020"},{"key":"2023032023153486700_","first-page":"6490","article-title":"Introducing routing uncertainty in capsule networks","volume-title":"Advances in neural information processing systems","author":"De Sousa Ribeiro","year":"2020"},{"issue":"9","key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"1627","DOI":"10.1109\/TPAMI.2009.167","article-title":"Object detection with discriminatively trained part based models","volume":"32","author":"Felzenszwalb","year":"2009","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2023032023153486700_","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2003.1211479","article-title":"Object class recognition by unsupervised scale-invariant learning","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition","author":"Fergus","year":"2003"},{"issue":"6","key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Communications of the ACM"},{"issue":"1","key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1109\/T-C.1973.223602","article-title":"The representation and matching of pictoral structures","volume":"22","author":"Fischler","year":"1973","journal-title":"IEEE Transactions on Computers"},{"key":"2023032023153486700_","first-page":"463","article-title":"Learning to parse images","volume-title":"Advances in neural information processing systems","author":"Hinton","year":"2000"},{"key":"2023032023153486700_","article-title":"Transforming autoencoders","volume-title":"Proceedings of the International Conference on Artificial Neural Networks","author":"Hinton","year":"2011"},{"key":"2023032023153486700_","article-title":"Matrix capsules with EM routing","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Hinton","year":"2018"},{"issue":"1","key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1007\/BF01908075","article-title":"Comparing partitions","volume":"2","author":"Hubert","year":"1985","journal-title":"Journal of Classification"},{"key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1023\/A:1007665907178","article-title":"An introduction to variational methods for graphical models","volume":"37","author":"Jordan","year":"1999","journal-title":"Machine Learning"},{"key":"2023032023153486700_","first-page":"15512","article-title":"Stacked capsule autoencoders","volume-title":"Advances in neural information processing systems","author":"Kosiorek","year":"2019"},{"key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"788","DOI":"10.1038\/44565","article-title":"Learning the parts of objects by non-negative matrix factorization","volume":"401","author":"Lee","year":"1999","journal-title":"Nature"},{"key":"2023032023153486700_","first-page":"3744","article-title":"Set Transformer: A framework for attention-based permutation-invariant neural networks","volume-title":"Proc. of the 36th International Conference on Machine Learning","author":"Lee","year":"2019"},{"key":"2023032023153486700_","first-page":"281","article-title":"Capsule generative models","volume-title":"Proceedings of the International Conference on Artificial Neural Networks","author":"Li","year":"2019"},{"key":"2023032023153486700_","doi-asserted-by":"crossref","DOI":"10.1109\/IJCNN48605.2020.9206662","article-title":"Capsule deep generative model that forms parse trees","volume-title":"Proceedings of the 2020 International Joint Conference on Neural Networks","author":"Li","year":"2020"},{"key":"2023032023153486700_","volume-title":"Multivariate analysis","author":"Mardia","year":"1979"},{"key":"2023032023153486700_","article-title":"Iterative amortized inference","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Marino","year":"2018"},{"key":"2023032023153486700_","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1007\/978-3-540-45167-9_14","article-title":"Comparing clusterings by the variation of information","volume-title":"Learning theory and kernel machines","author":"Meil\u0103","year":"2003"},{"key":"2023032023153486700_","first-page":"1","article-title":"Sinkhorn = permutation variational marginal inference","volume-title":"Symposium on Advances in Approximate Bayesian Inference","author":"Mena","year":"2020"},{"key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"871","DOI":"10.1007\/s00180-019-00901-2","article-title":"Computing expectations and marginal likelihoods for permutations","volume":"35","author":"Powell","year":"2019","journal-title":"Computational Statistics"},{"issue":"336","key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"846","DOI":"10.1080\/01621459.1971.10482356","article-title":"Objective criteria for the evaluation of clustering methods","volume":"66","author":"Rand","year":"1971","journal-title":"Journal of the American Statistical Association"},{"issue":"1","key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1038\/4580","article-title":"Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects","volume":"2","author":"Rao","year":"1999","journal-title":"Nature Neurosci."},{"issue":"6","key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"592","DOI":"10.1109\/34.506410","article-title":"Using generative models for handwritten digit recognition","volume":"18","author":"Revow","year":"1996","journal-title":"IEEE Trans. on Pattern Analysis and Machine Intelligence"},{"key":"2023032023153486700_","first-page":"2369","article-title":"Learning parts-based representations of data","volume":"7","author":"Ross","year":"2006","journal-title":"Journal of Machine Learning Research"},{"key":"2023032023153486700_","first-page":"3856","article-title":"Dynamic routing between capsules","volume-title":"Advances in neural information processing systems","author":"Sabour","year":"2017"},{"issue":"2","key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"343","DOI":"10.2140\/pjm.1967.21.343","article-title":"Concerning nonnegative matrices and doubly stochastic matrices","volume":"21","author":"Sinkhorn","year":"1967","journal-title":"Pacific Journal of Mathematics"},{"key":"2023032023153486700_","author":"Smith","year":"2021","journal-title":"Capsule networks: A generative probabilistic perspective"},{"issue":"7","key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1109\/TPAMI.2003.1206515","article-title":"Image modelling with position-encoding dynamic trees","volume":"25","author":"Storkey","year":"2003","journal-title":"IEEE Trans Pattern Analysis and Machine Intelligence"},{"key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"1321","DOI":"10.1098\/rsta.1998.0224","article-title":"Geometric motion segmentation and model selection","volume":"356","author":"Torr","year":"1998","journal-title":"Philosophical Trans. of the Royal Society A"},{"key":"2023032023153486700_","doi-asserted-by":"crossref","DOI":"10.1109\/ISPA.2001.938625","article-title":"Detecting planar homographies in an image pair","volume-title":"Proceedings of the 2nd International Symposium on Image and Signal Processing and Analysis","author":"Vincent","year":"2001"},{"key":"2023032023153486700_","author":"Williams","year":"2019","journal-title":"Autoencoders and probabilistic inference with missing data: An exact solution for the factor analysis case"},{"issue":"4","key":"2023032023153486700_","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1109\/99.641604","article-title":"Geometric hashing: An overview","volume":"4","author":"Wolfson","year":"1997","journal-title":"IEEE Computational Science and Engineering"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/4\/727\/2075269\/neco_a_01564.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/4\/727\/2075269\/neco_a_01564.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T05:22:46Z","timestamp":1679635366000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/35\/4\/727\/114734\/Inference-and-Learning-for-Generative-Capsule"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,18]]},"references-count":38,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,3,18]]},"published-print":{"date-parts":[[2023,3,18]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01564","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,4]]},"published":{"date-parts":[[2023,3,18]]}}}