{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T13:20:38Z","timestamp":1777555238132,"version":"3.51.4"},"reference-count":58,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2018,12,21]],"date-time":"2018-12-21T00:00:00Z","timestamp":1545350400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"University of Oulu","award":["24000692"],"award-info":[{"award-number":["24000692"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Single-level principal component analysis (PCA) and multi-level PCA (mPCA) methods are applied here to a set of (2D frontal) facial images from a group of 80 Finnish subjects (34 male; 46 female) with two different facial expressions (smiling and neutral) per subject. Inspection of eigenvalues gives insight into the importance of different factors affecting shapes, including: biological sex, facial expression (neutral versus smiling), and all other variations. Biological sex and facial expression are shown to be reflected in those components at appropriate levels of the mPCA model. Dynamic 3D shape data for all phases of a smile made up a second dataset sampled from 60 adult British subjects (31 male; 29 female). Modes of variation reflected the act of smiling at the correct level of the mPCA model. Seven phases of the dynamic smiles are identified: rest pre-smile, onset 1 (acceleration), onset 2 (deceleration), apex, offset 1 (acceleration), offset 2 (deceleration), and rest post-smile. A clear cycle is observed in standardized scores at an appropriate level for mPCA and in single-level PCA. mPCA can be used to study static shapes and images, as well as dynamic changes in shape. It gave us much insight into the question \u201cwhat\u2019s in a smile?\u201d.<\/jats:p>","DOI":"10.3390\/jimaging5010002","type":"journal-article","created":{"date-parts":[[2018,12,21]],"date-time":"2018-12-21T12:14:49Z","timestamp":1545394489000},"page":"2","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["What\u2019s in a Smile? Initial Analyses of Dynamic Changes in Facial Shape and Appearance"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0662-1927","authenticated-orcid":false,"given":"Damian J. J.","family":"Farnell","sequence":"first","affiliation":[{"name":"School of Dentistry, Cardiff University, Heath Park, Cardiff CF14 4XY, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jennifer","family":"Galloway","sequence":"additional","affiliation":[{"name":"School of Dentistry, Cardiff University, Heath Park, Cardiff CF14 4XY, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexei I.","family":"Zhurov","sequence":"additional","affiliation":[{"name":"School of Dentistry, Cardiff University, Heath Park, Cardiff CF14 4XY, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stephen","family":"Richmond","sequence":"additional","affiliation":[{"name":"School of Dentistry, Cardiff University, Heath Park, Cardiff CF14 4XY, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Marshall","sequence":"additional","affiliation":[{"name":"School of Computer Science and Informatics, Cardiff University, Cardiff CF24 3AA, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul L.","family":"Rosin","sequence":"additional","affiliation":[{"name":"School of Computer Science and Informatics, Cardiff University, Cardiff CF24 3AA, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Khtam","family":"Al-Meyah","sequence":"additional","affiliation":[{"name":"School of Computer Science and Informatics, Cardiff University, Cardiff CF24 3AA, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pertti","family":"Pirttiniemi","sequence":"additional","affiliation":[{"name":"Research Unit of Oral Health Sciences, Faculty of Medicine, University of Oulu, FI-90014 Oulu, Finland"},{"name":"Medical Research Center Oulu (MRC Oulu), Oulu University Hospital, FI-90014 Oulu, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2375-5222","authenticated-orcid":false,"given":"Raija","family":"L\u00e4hdesm\u00e4ki","sequence":"additional","affiliation":[{"name":"Research Unit of Oral Health Sciences, Faculty of Medicine, University of Oulu, FI-90014 Oulu, Finland"},{"name":"Medical Research Center Oulu (MRC Oulu), Oulu University Hospital, FI-90014 Oulu, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,12,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/S0889-5406(03)00306-8","article-title":"Dynamic smile visualization and quantification: Part 1. Evolution of the concept and dynamic records for smile capture","volume":"124","author":"Sarver","year":"2003","journal-title":"Am. J. Orthod. Dentofac. Orthop."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1016\/S0889-5406(03)00307-X","article-title":"Dynamic smile visualization and quantification: Part 2. Smile analysis and treatment strategies","volume":"124","author":"Sarver","year":"2003","journal-title":"Am. J. Orthod. Dentofac. Orthop."},{"key":"ref_3","first-page":"9","article-title":"The esthetics of the smile: A review of some recent studies","volume":"12","author":"Dong","year":"1999","journal-title":"Int. J. Prosthodont."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"907","DOI":"10.2466\/pr0.1998.83.3.907","article-title":"Sex differences over age groups in self-posed smiling in photographs","volume":"83","author":"Otta","year":"1998","journal-title":"Psychol. Rep."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"631","DOI":"10.2319\/042515-284.1","article-title":"Incisor display during speech and smile: Age and gender correlations","volume":"86","author":"Drummond","year":"2015","journal-title":"Angle Orthod."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"90","DOI":"10.2319\/040112-268.1","article-title":"Dynamics of a smile in different age groups","volume":"83","author":"Chetan","year":"2012","journal-title":"Angle Orthod."},{"key":"ref_7","unstructured":"Dibeklio\u011flu, H., Gevers, T., Salah, A.A., and Valenti, R. (November, January 29). A smile can reveal your age: Enabling facial dynamics in age estimation. Proceedings of the 20th ACM International Conference on Multimedia, Nara, Japan."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1928","DOI":"10.1109\/TIP.2015.2412377","article-title":"Combining facial dynamics with appearance for age estimation","volume":"24","author":"Alnajar","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1111\/j.1601-6343.2006.00341.x","article-title":"A new method for the 3D measurement of postoperative swelling following orthognathic surgery","volume":"9","author":"Kau","year":"2006","journal-title":"Orthod. Craniofac. Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1093\/ejo\/cjs018","article-title":"Three-dimensional evaluation of facial morphology in children aged 5\u20136 years with a Class III malocclusion","volume":"36","author":"Krneta","year":"2012","journal-title":"Eur. J. Orthod."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"e002910","DOI":"10.1136\/bmjopen-2013-002910","article-title":"A population-based cross-sectional study of the association between facial morphology and cardiometabolic risk factors in adolescence","volume":"3","author":"Djordjevic","year":"2013","journal-title":"BMJ Open"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1111\/j.1601-6343.2011.01539.x","article-title":"Statistical modeling of lip movement in the clinical context","volume":"15","author":"Popat","year":"2012","journal-title":"Orthod. Craniofac. Res."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/ejo\/cjt075","article-title":"Comparison between landmark and surface-based three-dimensional analyses of facial asymmetry in adults","volume":"37","author":"Alqattan","year":"2013","journal-title":"Eur. J. Orthod."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e009027","DOI":"10.1136\/bmjopen-2015-009027","article-title":"The influence of snoring, mouth breathing and apnoea on facial morphology in late childhood: A three-dimensional study","volume":"5","author":"Richmond","year":"2015","journal-title":"BMJ Open"},{"key":"ref_15","unstructured":"Vandeventer, J. (2015). 4D (3D Dynamic) Statistical Models of Conversational Expressions and the Synthesis of Highly-Realistic 4D Facial Expression Sequences. [Ph.D. Thesis, Cardiff University]."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Vandeventer, J., Graser, L., Rychlowska, M., Rosin, P.L., and Marshall, D. (2015). Towards 4D coupled models of conversational facial expression interactions. Proceedings of the British Machine Vision Conference, BMVA Press.","DOI":"10.5244\/C.29.142"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1016\/j.procs.2017.08.037","article-title":"4D Analysis of Facial Ageing Using Dynamic Features","volume":"112","author":"Marshall","year":"2017","journal-title":"Procedia Comput. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"478","DOI":"10.1016\/j.ajhg.2011.12.021","article-title":"Genome-wide Association Study of Three\u2013Dimensional Facial Morphology Identifies a Variant in PAX3 Associated with Nasion Position","volume":"90","author":"Paternoster","year":"2012","journal-title":"Am. J. Hum. Genet."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3807","DOI":"10.1093\/hmg\/ddt231","article-title":"Genome-wide association study of primary tooth eruption identifies pleiotropic loci associated with height and craniofacial distances","volume":"22","author":"Fatemifar","year":"2013","journal-title":"Hum. Mol. Gen."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1016\/j.fsigen.2014.08.008","article-title":"Toward DNA-based facial composites: Preliminary results and validation","volume":"13","author":"Claes","year":"2014","journal-title":"Forensic. Sci. Int. Genet."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Djordjevic, J., Zhurov, A.I., Richmond, S., and Visigen Consortium (2016). Genetic and Environmental Contributions to Facial Morphological Variation: A 3D Population-Based Twin Study. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0162250"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Lecron, F., Boisvert, J., Benjelloun, M., Labelle, H., and Mahmoudi, S. (2012, January 2\u20135). Multilevel statistical shape models: A new framework for modeling hierarchical structures. Proceedings of the 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI), Barcelona, Spain.","DOI":"10.1109\/ISBI.2012.6235797"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.cmpb.2016.01.005","article-title":"Multilevel principal component analysis (mPCA) in shape analysis: A feasibility study in medical and dental imaging","volume":"129","author":"Farnell","year":"2016","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Farnell, D.J.J., Galloway, J., Zhurov, A., Richmond, S., Perttiniemi, P., and Katic, V. (2017). Initial results of multilevel principal components analysis of facial shape. Annual Conference on Medical Image Understanding and Analysis, Springer.","DOI":"10.1007\/978-3-319-60964-5_59"},{"key":"ref_25","unstructured":"Farnell, D.J.J., Galloway, J., Zhurov, A., Richmond, S., Perttiniemi, P., and Katic, V. An Initial Exploration of Ethnicity, Sex, and Subject Variation on Facial Shape, in preparation."},{"key":"ref_26","first-page":"177","article-title":"What\u2019s in a Smile? Initial Results of Multilevel Principal Components Analysis of Facial Shape and Image Texture","volume":"Volume 894","author":"Farnell","year":"2018","journal-title":"Annual Conference on Medical Image Understanding and Analysis"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1016\/0262-8856(94)90060-4","article-title":"Use of Active Shape Models for Locating Structure in Medical Images","volume":"12","author":"Cootes","year":"1994","journal-title":"Image Vis. Comput."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1006\/cviu.1995.1004","article-title":"Active Shape Models\u2014Their Training and Application","volume":"61","author":"Cootes","year":"1995","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1016\/0262-8856(96)01097-9","article-title":"Active shape models and the shape approximation problem","volume":"12","author":"Hill","year":"1996","journal-title":"Image Vis. Comput."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1016\/S0262-8856(98)00175-9","article-title":"A mixture model for representing shape variation","volume":"17","author":"Cootes","year":"1999","journal-title":"Image Vis. Comput."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1109\/TITB.2006.888704","article-title":"Detecting reduced bone mineral density from dental radiographs using statistical shape models","volume":"11","author":"Allen","year":"2007","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Edwards, G.J., Lanitis, A., Taylor, C.J., and Cootes, T. (1996, January 9\u201312). Statistical Models of Face Images: Improving Specificity. Proceedings of the British Machine Vision Conference, Edinburgh, UK.","DOI":"10.5244\/C.10.14"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1267","DOI":"10.1098\/rstb.1997.0109","article-title":"Model-based interpretation of complex and variable images","volume":"352","author":"Taylor","year":"1997","journal-title":"Philos. Trans. R. Soc. Lond. Biol."},{"key":"ref_34","first-page":"581","article-title":"Face recognition using active appearance models","volume":"Volume 1407","author":"Burkhardt","year":"1998","journal-title":"Computer Vision"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1109\/34.927467","article-title":"Active appearance models","volume":"23","author":"Cootes","year":"2001","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"S133","DOI":"10.1259\/bjr\/20343922","article-title":"Anatomical statistical models and their role in feature extraction","volume":"77","author":"Cootes","year":"2004","journal-title":"Br. J. Radiol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1023\/B:VISI.0000029666.37597.d3","article-title":"Active Appearance Models Revisited","volume":"60","author":"Matthews","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.imavis.2015.06.010","article-title":"RSILC: Rotation- and Scale-Invariant, Line-based Color-aware descriptor","volume":"42","author":"Candemir","year":"2005","journal-title":"Image Vision Comput."},{"key":"ref_39","unstructured":"Doersch, C. (arXiv, 2016). Tutorial on variational autoencoders, arXiv."},{"key":"ref_40","unstructured":"Pu, Y., Gan, Z., Henao, R., Yuan, X., Li, C., Stevens, A., and Carin, L. (2016). Variational autoencoder for deep learning of images, labels and captions. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"022140","DOI":"10.1103\/PhysRevE.96.022140","article-title":"Unsupervised learning of phase transitions: From principal component analysis to variational autoencoders","volume":"96","author":"Wetzel","year":"2017","journal-title":"Phys. Rev. E."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1724","DOI":"10.1364\/JOSAA.14.001724","article-title":"Discriminant analysis for recognition of human face images","volume":"14","author":"Etemad","year":"1997","journal-title":"J. Opt. Soc. Am. A"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1109\/TPAMI.2005.58","article-title":"Locally Linear Discriminant Analysis for Multimodally Distributed Classes for Face Recognition with a Single Model Image","volume":"27","author":"Kim","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Dibeklioglu, H., Valenti, R., Salah, A.A., and Gevers, T. (2010, January 25\u201329). Eyes do not lie: Spontaneous versus posed smiles. Proceedings of the 18th ACM International Conference on Multimedia, Firenze, Italy.","DOI":"10.1145\/1873951.1874056"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.jchb.2012.02.002","article-title":"Surface facial modeling and allometry in relation to sexual dimorphism","volume":"63","author":"Bigoni","year":"2012","journal-title":"HOMO"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1111\/j.1601-6343.2008.00428.x","article-title":"A three-dimensional look for facial differences between males and females in a British-Caucasian sample aged 15 \u00bd years old","volume":"11","author":"Toma","year":"2008","journal-title":"Orthol. Craniofac. Res."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1080\/00401706.1999.10485670","article-title":"A fast algorithm for the minimum covariance determinant estimator","volume":"41","author":"Rousseeuw","year":"1999","journal-title":"Technometrics"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1198\/004017002188618509","article-title":"Robust estimates of location and dispersion for high dimensional datasets","volume":"50","author":"Maronna","year":"2002","journal-title":"Technometrics"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/S0167-9473(03)00119-1","article-title":"A resistant estimator of multivariate location and dispersion","volume":"46","author":"Olive","year":"2004","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_50","first-page":"11","article-title":"Robust principal component analysis?","volume":"58","author":"Li","year":"2011","journal-title":"JACM"},{"key":"ref_51","unstructured":"Wright, J., Ganesh, A., Rao, S., Peng, Y., and Ma, Y. (2009). Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1728","DOI":"10.1093\/bioinformatics\/bth158","article-title":"Robust PCA and classification in biosciences","volume":"20","author":"Hubert","year":"2004","journal-title":"Bioinformatics"},{"key":"ref_53","unstructured":"Andersen, R. (2008). Modern Methods for Robust Regression, Sage Publications. Quantitative Applications in the Social Sciences. 152."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Godambe, V.P. (1991). Estimating Functions, Clarendon Press.","DOI":"10.1093\/oso\/9780198522287.001.0001"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.dsp.2014.05.013","article-title":"Gaussian mixture model based estimation of the neutral face shape for emotion recognition","volume":"32","author":"Ulukaya","year":"2014","journal-title":"Digit. Signal Process."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1016\/j.neuroimage.2011.02.046","article-title":"A Bayesian model of shape and appearance for subcortical brain segmentation","volume":"56","author":"Patenaude","year":"2011","journal-title":"Neuroimage"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/j.cviu.2005.09.012","article-title":"Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories","volume":"106","author":"Li","year":"2007","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"1307","DOI":"10.1007\/s13042-017-0645-0","article-title":"Agreeing to disagree: Active learning with noisy labels without crowdsourcing","volume":"9","author":"Bouguelia","year":"2018","journal-title":"Int. J. Mach. Learn. Cybern."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/5\/1\/2\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:35:36Z","timestamp":1760196936000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/5\/1\/2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,12,21]]},"references-count":58,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,1]]}},"alternative-id":["jimaging5010002"],"URL":"https:\/\/doi.org\/10.3390\/jimaging5010002","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints201811.0416.v1","asserted-by":"object"}]},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,12,21]]}}}