{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T22:32:06Z","timestamp":1777761126107,"version":"3.51.4"},"reference-count":92,"publisher":"Emerald","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2007,8,20]]},"abstract":"<jats:p>This exploratory paper quests for a stochastic and context sensitive grammar of images. The grammar should achieve the following four objectives and thus serves as a unified framework of representation, learning, and recognition for a large number of object categories. (i) The grammar represents both the hierarchical decompositions from scenes, to objects, parts, primitives and pixels by terminal and non-terminal nodes and the contexts for spatial and functional relations by horizontal links between the nodes. It formulates each object category as the set of all possible valid configurations produced by the grammar. (ii) The grammar is embodied in a simple And\u2013Or graph representation where each Or-node points to alternative sub-configurations and an And-node is decomposed into a number of components. This representation supports recursive top-down\/bottom-up procedures for image parsing under the Bayesian framework and make it convenient to scale up in complexity. Given an input image, the image parsing task constructs a most probable parse graph on-the-fly as the output interpretation and this parse graph is a subgraph of the And\u2013Or graph after making choice on the Or-nodes. (iii) A probabilistic model is defined on this And\u2013Or graph representation to account for the natural occurrence frequency of objects and parts as well as their relations. This model is learned from a relatively small training set per category and then sampled to synthesize a large number of configurations to cover novel object instances in the test set. This generalization capability is mostly missing in discriminative machine learning methods and can largely improve recognition performance in experiments. (iv) To fill the well-known semantic gap between symbols and raw signals, the grammar includes a series of visual dictionaries and organizes them through graph composition. At the bottom-level the dictionary is a set of image primitives each having a number of anchor points with open bonds to link with other primitives. These primitives can be combined to form larger and larger graph structures for parts and objects. The ambiguities in inferring local primitives shall be resolved through top-down computation using larger structures. Finally these primitives forms a primal sketch representation which will generate the input image with every pixels explained. The proposal grammar integrates three prominent representations in the literature: stochastic grammars for composition, Markov (or graphical) models for contexts, and sparse coding with primitives (wavelets). It also combines the structure-based and appearance based methods in the vision literature. Finally the paper presents three case studies to illustrate the proposed grammar.<\/jats:p>","DOI":"10.1561\/0600000018","type":"journal-article","created":{"date-parts":[[2007,9,6]],"date-time":"2007-09-06T03:38:17Z","timestamp":1189049897000},"page":"259-362","source":"Crossref","is-referenced-by-count":292,"title":["A Stochastic Grammar of Images"],"prefix":"10.1108","volume":"2","author":[{"given":"Song-Chun","family":"Zhu","sequence":"first","affiliation":[{"name":"University of California , Los Angeles,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Mumford","sequence":"additional","affiliation":[{"name":"Brown University ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","published-online":{"date-parts":[[2007,8,20]]},"reference":[{"issue":"4","key":"2026032615065336600_ref001","first-page":"597","article-title":"Stochastic attribute-value grammars","volume":"23","author":"Abney","year":"1997","journal-title":"Computational Linguistics"},{"key":"2026032615065336600_ref002","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-65371-1","volume-title":"Branching Processes","author":"Athreya","year":"1972"},{"issue":"8","key":"2026032615065336600_ref003","doi-asserted-by":"crossref","first-page":"1239","DOI":"10.1109\/TPAMI.2005.161","article-title":"Generalizing Swendsen-Wang to sampling arbitrary posterior probabilities","volume":"27","author":"Barbu","year":"2005","journal-title":"IEEE Transactions on PAMI"},{"key":"2026032615065336600_ref004","volume-title":"Tech. Report","author":"Barnard","year":"2005"},{"key":"2026032615065336600_ref005","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1037\/0033-295X.94.2.115","article-title":"Recognition-by-components: A theory of human image understanding","volume":"94","author":"Biederman","year":"1987","journal-title":"Psychological Review"},{"key":"2026032615065336600_ref006","volume-title":"Advances in Neural Information Processing Systems 9","author":"Bienenstock","year":"1998"},{"key":"2026032615065336600_ref007","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1214\/009053605000000174","article-title":"Sequential testing designs for pattern recognition","volume":"33","author":"Blanchard","year":"2005","journal-title":"Annals of Statistics"},{"key":"2026032615065336600_ref008","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1016\/0022-5193(73)90175-6","article-title":"Biological shape and visual science","volume":"38","author":"Blum","year":"1973","journal-title":"Journal of Theoretical Biology"},{"key":"2026032615065336600_ref009","volume-title":"Proceedings of IEEE Conference on Pattern Recognition and Computer Vision","author":"Chen","year":"2006"},{"issue":"2","key":"2026032615065336600_ref010","first-page":"299","article-title":"Estimation of probabilistic context free grammar","volume":"24","author":"Chi","year":"1998","journal-title":"Computational Linguistics"},{"key":"2026032615065336600_ref011","doi-asserted-by":"crossref","DOI":"10.1515\/9783112316009","volume-title":"Syntactic Structures","author":"Chomsky","year":"1957"},{"issue":"1","key":"2026032615065336600_ref012","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1006\/cviu.1995.1004","article-title":"Active appearance models-their training and applications","volume":"61","author":"Cootes","year":"1995","journal-title":"Computer Vision and Image Understanding"},{"key":"2026032615065336600_ref013","doi-asserted-by":"crossref","first-page":"886","DOI":"10.1109\/78.668544","article-title":"Wavelet based statistical signal processing using hidden Markov models","volume":"46","author":"Crouse","year":"1998","journal-title":"IEEE Transactions on Signal Processing"},{"issue":"2","key":"2026032615065336600_ref014","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1016\/1049-9660(92)90013-S","article-title":"From volumes to views: An approach to 3D object recognition","volume":"55","author":"Dickinson","year":"1992","journal-title":"CVGIP: Image Understanding"},{"key":"2026032615065336600_ref015","doi-asserted-by":"crossref","first-page":"2435","DOI":"10.1109\/18.720544","article-title":"Data compression and harmonic analysis","volume":"6","author":"Donoho","year":"1998","journal-title":"IEEE Transactions on Information Theory"},{"key":"2026032615065336600_ref016","article-title":"Learning generative visual models from few training examples: An incremental Bayesian approach tested on 100 ob ject categories","volume-title":"Workshop on Generative Model Based Vision","author":"Fei-Fei","year":"2004"},{"issue":"4","key":"2026032615065336600_ref017","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1109\/TPAMI.2006.79","article-title":"One-Shot learning of object categories","volume":"28","author":"Fei-Fei","year":"2006","journal-title":"IEEE Transactions on PAMI"},{"issue":"1","key":"2026032615065336600_ref018","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1167\/7.1.10","article-title":"What do we perceive in a glance of a real-world scene?","volume":"7","author":"Fei-Fei","year":"2007","journal-title":"Journal of Vision"},{"key":"2026032615065336600_ref019","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1109\/T-C.1973.223602","article-title":"The representation and matching of pictorial structures","volume":"C-22","author":"Fischler","year":"1973","journal-title":"IEEE Transactions on Computer"},{"key":"2026032615065336600_ref020","doi-asserted-by":"crossref","first-page":"8092","DOI":"10.1073\/pnas.0731829100","article-title":"Mixed markov models","volume":"100","author":"Fridman","year":"2003","journal-title":"Proceedings of Natural Academy of Science USA"},{"issue":"2","key":"2026032615065336600_ref021","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1214\/aos\/1016218223","article-title":"Additive logistic regression: A statistical view of boosting","volume":"38","author":"Friedman","year":"2000","journal-title":"Annals of Statistics"},{"key":"2026032615065336600_ref022","volume-title":"Syntactic Pattern Recognition and Applications","author":"Fu","year":"1982"},{"key":"2026032615065336600_ref023","first-page":"716","article-title":"Texture segmentation by multiscale aggregation of filter responses and shape elements","volume-title":"Proceedings of ICCV, Nice","author":"Galun","year":"2003"},{"key":"2026032615065336600_ref024","volume-title":"Proceedings of the 6th International Conference on EMMCVPR","author":"Gao","year":"2007"},{"key":"2026032615065336600_ref025","volume-title":"Technical Report","author":"Gao","year":"2006"},{"key":"2026032615065336600_ref026","first-page":"12075","volume-title":"Int\u2019l Encyc. of the Social and Behavioral Sciences","author":"Geman","year":"2002"},{"key":"2026032615065336600_ref027","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1090\/qam\/1939008","article-title":"Composition systems","volume":"60","author":"Geman","year":"2002","journal-title":"Quarterly of Applied Mathematics"},{"key":"2026032615065336600_ref028","volume-title":"General Pattern Theory","author":"Grenander","year":"1993"},{"key":"2026032615065336600_ref029","article-title":"The Caltech 256","volume-title":"Technical Report","author":"Griffin","year":"2006"},{"issue":"1","key":"2026032615065336600_ref030","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1023023207396","article-title":"Modeling visual patterns by integrating descriptive and generative models","volume":"53","author":"Guo","year":"2003","journal-title":"IJCV"},{"key":"2026032615065336600_ref031","article-title":"Primal sketch: Integrating texture and structure","volume-title":"Proceedings of International Conference on Computer Vision","author":"Guo","year":"2003"},{"key":"2026032615065336600_ref032","volume-title":"Proceedings of International Conference on Computer Vision","author":"Han","year":"2005"},{"key":"2026032615065336600_ref033","article-title":"Visions: A computer system for interpreting scenes","volume-title":"Computer Vision Systems","author":"Hanson","year":"1978"},{"key":"2026032615065336600_ref034","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1109\/TPAMI.1984.4767505","article-title":"Compact region extraction using weighted pixel linking in a pyramid","volume":"6","author":"Hong","year":"1984","journal-title":"IEEE Transactions on PAMI"},{"key":"2026032615065336600_ref035","author":"Huang"},{"key":"2026032615065336600_ref036","volume-title":"Proceedings of IEEE Conference on Computer Vision and Pattern Recognition","author":"Jin","year":"2006"},{"key":"2026032615065336600_ref037","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1038\/290091a0","article-title":"Textons, the elements of eexture perception, and their interactions","volume":"290","author":"Julesz","year":"1981","journal-title":"Nature"},{"key":"2026032615065336600_ref038","doi-asserted-by":"crossref","DOI":"10.1023\/A:1012460413855","article-title":"Saliency, scale and image description","volume-title":"International Journal of Computer Vision","author":"Kadir","year":"2001"},{"key":"2026032615065336600_ref039","volume-title":"Organization in Vision","author":"Kanisza","year":"1979"},{"key":"2026032615065336600_ref040","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2001.990574","article-title":"Generic model abstraction from examples","volume-title":"CVPR","author":"Keselman","year":"2001"},{"key":"2026032615065336600_ref041","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/BF01451741","article-title":"Shapes, shocks and deformations I","volume":"15","author":"Kimia","year":"1995","journal-title":"Interantional Journal of Computer Vision"},{"issue":"1\/2","key":"2026032615065336600_ref042","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1023\/A:1023705401078","article-title":"The nonlinear statistics of high-contrast patches in natural images","volume":"54","author":"Lee","year":"2003","journal-title":"IJCV"},{"key":"2026032615065336600_ref043","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1016\/0004-3702(88)90039-2","article-title":"A process grammar for shape","volume":"34","author":"Leyton","year":"1988","journal-title":"Artificial Intelligence"},{"key":"2026032615065336600_ref044","volume-title":"Proceedings of International Conference on Computer Vision","author":"Lin","year":"2007"},{"key":"2026032615065336600_ref045","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-6465-9","volume-title":"Scale-Space Theory in Computer Vision","author":"Lindeberg","year":"1994"},{"key":"2026032615065336600_ref046","first-page":"134","volume-title":"Monte Carlo Strategies in Scientific Computing","author":"Liu","year":"2001"},{"key":"2026032615065336600_ref047","doi-asserted-by":"crossref","first-page":"3397","DOI":"10.1109\/78.258082","article-title":"Matching pursuit in a time-frequency dictionary","volume":"41","author":"Mallat","year":"1993","journal-title":"IEEE Transactions on Signal Processing"},{"key":"2026032615065336600_ref048","volume-title":"Image Models (and Their Speech Model cousins)","author":"Mark","year":"1994"},{"key":"2026032615065336600_ref049","volume-title":"Vision","author":"Marr","year":"1983"},{"key":"2026032615065336600_ref050","article-title":"A database of human segmented natural images and its application to evaluating segmentation algorithms","volume-title":"ICCV","author":"Martin","year":"2001"},{"key":"2026032615065336600_ref051","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/BF01421486","article-title":"Visual learning and recognition of 3-D objects from appearance","volume":"14","author":"Murase","year":"1995","journal-title":"International Journal of Computer Vision"},{"key":"2026032615065336600_ref052","article-title":"Graphical model for recognizing scenes and objects","volume-title":"Proceedings of NIPS","author":"Murphy","year":"2003"},{"key":"2026032615065336600_ref053","doi-asserted-by":"crossref","DOI":"10.1007\/3-540-56484-5","article-title":"Filtering, segmentation and depth","volume":"662","author":"Nitzberg","year":"1993","journal-title":"Springer Lecture Notes in Computer Science"},{"key":"2026032615065336600_ref054","volume-title":"Knowledge-Based Interpretation of Outdoor Natural Color Scenes","author":"Ohta","year":"1985"},{"key":"2026032615065336600_ref055","first-page":"752","volume-title":"Proceedings of 4th International Joint Conference on Pattern Recognition","author":"Ohta","year":"1978"},{"key":"2026032615065336600_ref056","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1038\/381607a0","article-title":"Emergence of simple-cell receptive field properties by learning a sparse code for natural images","volume":"381","author":"Olshausen","year":"1996","journal-title":"Nature"},{"key":"2026032615065336600_ref057","article-title":"Learning compositional categorization method","volume-title":"Proceedings of European Conference on Computer Vision","author":"Ommer","year":"2006"},{"key":"2026032615065336600_ref058","volume-title":"Heuristics: Intelligent Search Strategies for Computer Problem Solving","author":"Pearl","year":"1984"},{"key":"2026032615065336600_ref059","author":"Porway","year":"2007"},{"key":"2026032615065336600_ref060","author":"Rekers","year":"1995"},{"key":"2026032615065336600_ref061","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/S0959-4388(02)00304-5","article-title":"Neural mechanisms of object recognition","volume":"12","author":"Riesenhuber","year":"2002","journal-title":"Current Opinion in Neurobiology"},{"key":"2026032615065336600_ref062","article-title":"LabelMe: A database and web-based tool for image annotation","volume-title":"MIT AI Lab Memo AIM-2005-025","author":"Russel","year":"2005"},{"key":"2026032615065336600_ref063","article-title":"The boosting approach to machine learning: An overview","volume-title":"MSRI Workshop on nonlinear Estimation and Classification","author":"Schapire","year":"2002"},{"issue":"5","key":"2026032615065336600_ref064","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1109\/TPAMI.2004.1273924","article-title":"Recognition of shapes by editing their shock graphs","volume":"26","author":"Sebastian","year":"2004","journal-title":"IEEE Transactions on PAMI"},{"key":"2026032615065336600_ref065","doi-asserted-by":"crossref","first-page":"1695","DOI":"10.1098\/rstb.2002.1161","article-title":"The role of thalamus in the flow of information to cortex","volume":"357","author":"Sherman","year":"2002","journal-title":"Philosophical Transactions of Royal Society London (Biology)"},{"key":"2026032615065336600_ref066","article-title":"Visual learning with implicit and explicit manifolds","volume-title":"IEEE Conference on CVPR","author":"Shi","year":"2007"},{"issue":"3","key":"2026032615065336600_ref067","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1109\/34.368189","article-title":"Parts of visual form: Computational aspects","volume":"17","author":"Siddiqi","year":"1995","journal-title":"IEEE Transactions on PAMI"},{"issue":"1","key":"2026032615065336600_ref068","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1023\/A:1008102926703","article-title":"Shock graphs and shape matching","volume":"35","author":"Siddiqi","year":"1999","journal-title":"IJCV"},{"issue":"2","key":"2026032615065336600_ref069","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1109\/18.119725","article-title":"Shiftable milti-scale transforms","volume":"38","author":"Simoncelli","year":"1992","journal-title":"IEEE Transactions on Information Theory"},{"key":"2026032615065336600_ref070","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1038\/381520a0","article-title":"Speed of processing in the human visual system","volume":"381","author":"Thorpe","year":"1996","journal-title":"Nature"},{"key":"2026032615065336600_ref071","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2006.116","article-title":"Extracting subimages of an unknown category from a set of images","volume-title":"CVPR","author":"Todorovic","year":"2006"},{"issue":"2","key":"2026032615065336600_ref072","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s11263-005-6642-x","article-title":"Image parsing: Unifying segmentation, detection, and recognition","volume":"63","author":"Tu","year":"2005","journal-title":"International Journal of Computer Vision"},{"key":"2026032615065336600_ref073","article-title":"Image segmentation by data-driven Markov chain Monte Carlo","volume-title":"IEEE Transactions on PAMI","author":"Tu","year":"2002"},{"issue":"2","key":"2026032615065336600_ref074","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1007\/s11263-006-6995-9","article-title":"Parsing images into regions, curves and curve groups","volume":"69","author":"Tu","year":"2006","journal-title":"International Journal of Computer Vision"},{"key":"2026032615065336600_ref075","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/jocn.1991.3.1.71","article-title":"Eigenfaces for recognition","volume":"3","author":"Turk","year":"1991","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2026032615065336600_ref076","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/0010-0277(84)90023-4","article-title":"Visual routine","volume":"18","author":"Ullman","year":"1984","journal-title":"Cognition"},{"key":"2026032615065336600_ref077","volume-title":"Proceedings of 4th International Workshop on Visual Form","author":"Ullman","year":"2001"},{"key":"2026032615065336600_ref078","first-page":"511","article-title":"Rapid object detection using a boosted cascade of simple features","volume-title":"CVPR","author":"Viola","year":"2001"},{"issue":"10","key":"2026032615065336600_ref079","doi-asserted-by":"crossref","first-page":"3033","DOI":"10.1109\/TIP.2006.877496","article-title":"Hierarchical stochastic image grammars for classification and segmentation","volume":"15","author":"Wang","year":"2006","journal-title":"IEEE Transactions on Image Processing"},{"key":"2026032615065336600_ref080","volume-title":"International Conference on Computer Vision","author":"Wang","year":"2005"},{"key":"2026032615065336600_ref081","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2000.854754","article-title":"Towards automatic discovery of object categories","volume-title":"IEEE Conference on CVPR","author":"Weber","year":"2000"},{"key":"2026032615065336600_ref082","volume-title":"International Joint Conference on AI","author":"Witkin","year":"1983"},{"key":"2026032615065336600_ref083","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2007.383034","article-title":"Compositional boosting for computing hierarchical image structures","volume-title":"IEEE Conference on CVPR","author":"Wu","year":"2007"},{"key":"2026032615065336600_ref084","article-title":"From information scaling laws of natural images to regimes of statistical models","volume-title":"Quarterly of Applied Mathematics","author":"Wu","year":"2007"},{"key":"2026032615065336600_ref085","volume-title":"Proceedings of IEEE Conference on CVPR","author":"Xu","year":"2005"},{"key":"2026032615065336600_ref086","author":"Xu","year":"2007"},{"key":"2026032615065336600_ref087","volume-title":"6th International Conference on EMMCVPR","author":"Yao","year":"2007"},{"issue":"11","key":"2026032615065336600_ref088","article-title":"Embedding Gestalt laws in Markov random fields","volume":"21","author":"Zhu","year":"1999","journal-title":"IEEE Transactions on PAMI"},{"issue":"6","key":"2026032615065336600_ref089","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1109\/TPAMI.2003.1201820","article-title":"Statistical modeling and conceptualization of visual patterns","volume":"25","author":"Zhu","year":"2003","journal-title":"IEEE Transactions on PAMI"},{"issue":"8","key":"2026032615065336600_ref090","doi-asserted-by":"crossref","first-page":"1627","DOI":"10.1162\/neco.1997.9.8.1627","article-title":"Minimax entropy principle and its applications to texture modeling","volume":"9","author":"Zhu","year":"1997","journal-title":"Neural Computation"},{"key":"2026032615065336600_ref091","first-page":"187","article-title":"Forms: A flexible object recognition and modeling system","volume":"20","author":"Zhu","year":"1996","journal-title":"Interantional Journal of Computer Vision"},{"key":"2026032615065336600_ref092","article-title":"Integrating top-down\/bottom-up for object recognition by data-driven Markov chain Monte Carlo","volume-title":"CVPR","author":"Zhu","year":"2000"}],"container-title":["Foundations and Trends\u00ae in Computer Graphics and Vision"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/ftcgv\/article-pdf\/2\/4\/259\/10901034\/0600000018en.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/www.emerald.com\/ftcgv\/article-pdf\/2\/4\/259\/10901034\/0600000018en.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T14:09:11Z","timestamp":1777471751000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.emerald.com\/ftcgv\/article\/2\/4\/259\/1320819\/A-Stochastic-Grammar-of-Images"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2007,8,20]]},"references-count":92,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2007,8,20]]}},"URL":"https:\/\/doi.org\/10.1561\/0600000018","relation":{},"ISSN":["1572-2740","1572-2759"],"issn-type":[{"value":"1572-2740","type":"print"},{"value":"1572-2759","type":"electronic"}],"subject":[],"published":{"date-parts":[[2007,8,20]]}}}