{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T04:55:08Z","timestamp":1780635308076,"version":"3.54.1"},"reference-count":54,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2018,10,15]],"date-time":"2018-10-15T00:00:00Z","timestamp":1539561600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Micro-expressions are brief spontaneous facial expressions that appear on a face when a person conceals an emotion, making them different to normal facial expressions in subtlety and duration. Currently, emotion classes within the CASME II dataset (Chinese Academy of Sciences Micro-expression II) are based on Action Units and self-reports, creating conflicts during machine learning training. We will show that classifying expressions using Action Units, instead of predicted emotion, removes the potential bias of human reporting. The proposed classes are tested using LBP-TOP (Local Binary Patterns from Three Orthogonal Planes), HOOF (Histograms of Oriented Optical Flow) and HOG 3D (3D Histogram of Oriented Gradient) feature descriptors. The experiments are evaluated on two benchmark FACS (Facial Action Coding System) coded datasets: CASME II and SAMM (A Spontaneous Micro-Facial Movement). The best result achieves 86.35% accuracy when classifying the proposed 5 classes on CASME II using HOG 3D, outperforming the result of the state-of-the-art 5-class emotional-based classification in CASME II. Results indicate that classification based on Action Units provides an objective method to improve micro-expression recognition.<\/jats:p>","DOI":"10.3390\/jimaging4100119","type":"journal-article","created":{"date-parts":[[2018,10,15]],"date-time":"2018-10-15T03:43:01Z","timestamp":1539574981000},"page":"119","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":105,"title":["Objective Classes for Micro-Facial Expression Recognition"],"prefix":"10.3390","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6496-0209","authenticated-orcid":false,"given":"Adrian K.","family":"Davison","sequence":"first","affiliation":[{"name":"Centre for Imaging Sciences, University of Manchester, Manchester M13 9PL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9013-7083","authenticated-orcid":false,"given":"Walied","family":"Merghani","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Sudan University of Science and Technology, Khartoum 11111, Sudan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7681-4287","authenticated-orcid":false,"given":"Moi Hoon","family":"Yap","sequence":"additional","affiliation":[{"name":"School of Computing, Mathematics and Digital Technology, Manchester Metropolitan University, Manchester M15 6BH, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,10,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ekman, P. (2004). Emotions Revealed: Understanding Faces and Feelings, Phoenix.","DOI":"10.1136\/sbmj.0405184"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Martin, C.W. (2009). Lie Catching and Microexpressions. The Philosophy of Deception, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780195327939.001.0001"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"925","DOI":"10.1037\/0022-3514.94.6.925","article-title":"Culture, emotion regulation, and adjustment","volume":"94","author":"Matsumoto","year":"2008","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1631\/jzus.B1100063","article-title":"Effects of the duration of expressions on the recognition of microexpressions","volume":"13","author":"Shen","year":"2012","journal-title":"J. Zhejiang Univ. Sci. B"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s10919-013-0159-8","article-title":"How Fast are the Leaked Facial Expressions: The Duration of Micro-Expressions","volume":"37","author":"Yan","year":"2013","journal-title":"J. Nonverbal Behav."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1080\/00332747.1969.11023575","article-title":"Nonverbal leakage and clues to deception","volume":"32","author":"Ekman","year":"1969","journal-title":"Psychiatry"},{"key":"ref_7","unstructured":"Ekman, P. (2001). Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage, Norton."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ekman, P., and Rosenberg, E.L. (2005). What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS), Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780195179644.001.0001"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Frank, M.G., Maccario, C.J., and Govindaraju, V.l. (2009). Behavior and Security. Protecting Airline Passengers in the Age of Terrorism, Greenwood Pub. Group.","DOI":"10.5040\/9798216002246.ch-005"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/0031-3203(95)00067-4","article-title":"A comparative study of texture measures with classification based on featured distributions","volume":"29","author":"Ojala","year":"1996","journal-title":"Pattern Recognit."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1109\/TPAMI.2002.1017623","article-title":"Multiresolution gray-scale and rotation invariant texture classification with local binary patterns","volume":"24","author":"Ojala","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1109\/TPAMI.2007.1110","article-title":"Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions","volume":"29","author":"Zhao","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_13","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, USA."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Chaudhry, R., Ravichandran, A., Hager, G., and Vidal, R. (2009, January 20\u201325). Histograms of oriented optical flow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206821"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1007\/s10979-008-9166-4","article-title":"Police lie detection accuracy: The effect of lie scenario","volume":"33","author":"Frank","year":"2009","journal-title":"Law Hum. Behav."},{"key":"ref_16","unstructured":"Frank, M., Herbasz, M., Sinuk, K., Keller, A.M., Kurylo, A., and Nolan, C. (2009). I See How You Feel: Training Laypeople and Professionals to Recognize Fleeting Emotions, International Communication Association."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Yap, M.H., Ugail, H., and Zwiggelaar, R. (2013, January 22\u201326). A database for facial behavioural analysis. Proceedings of the 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Shanghai, China.","DOI":"10.1109\/FG.2013.6553803"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1918","DOI":"10.1212\/WNL.42.10.1918","article-title":"Localization of emotional and volitional facial paresis","volume":"42","author":"Hopf","year":"1992","journal-title":"Neurology"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Cohn, J.F., Kruez, T.S., Matthews, I., Yang, Y., Nguyen, M.H., Padilla, M.T., Zhou, F., and De La Torre, F. (2009, January 10\u201312). Detecting depression from facial actions and vocal prosody. Proceedings of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops (ACII 2009), Amsterdam, The Netherlands.","DOI":"10.1109\/ACII.2009.5349358"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1485","DOI":"10.9734\/BJAST\/2014\/6369","article-title":"Facial Behavioral Analysis: A Case Study in Deception Detection","volume":"4","author":"Yap","year":"2014","journal-title":"Br. J. Appl. Sci. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, X., Pfister, T., Huang, X., Zhao, G., and Pietik\u00e4inen, M. (2013, January 22\u201326). A Spontaneous Micro-expression Database: Inducement, Collection and Baseline. Proceedings of the 10th IEEE International Conference on Automatic Face and Gesture Recognition, Shanghai, China.","DOI":"10.1109\/FG.2013.6553717"},{"key":"ref_22","unstructured":"Yan, W.J., Wu, Q., Liu, Y.J., Wang, S.J., and Fu, X. (2013, January 22\u201326). CASME Database: A dataset of spontaneous micro-expressions collected from neutralized faces. Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition, Shanghai, China."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yan, W.J., Li, X., Wang, S.J., Zhao, G., Liu, Y.J., Chen, Y.H., and Fu, X. (2014). CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0086041"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1109\/TAFFC.2016.2573832","article-title":"SAMM: A Spontaneous Micro-Facial Movement Dataset","volume":"9","author":"Davison","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Polikovsky, S., Kameda, Y., and Ohta, Y. (2009, January 3). Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor. Proceedings of the 3rd International Conference on Imaging for Crime Detection and Prevention (ICDP 2009), London, UK.","DOI":"10.1049\/ic.2009.0244"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Shreve, M., Godavarthy, S., Goldgof, D., and Sarkar, S. (2011, January 21\u201325). Macro- and micro-expression spotting in long videos using spatio-temporal strain. Proceedings of the 2011 IEEE International Conference on Automatic Face Gesture Recognition and Workshops (FG 2011), Santa Barbara, CA, USA.","DOI":"10.1109\/FG.2011.5771451"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ekman, P., and Friesen, W.V. (1978). Facial Action Coding System: A Technique for the Measurement of Facial Movement, Consulting Psychologists Press.","DOI":"10.1037\/t27734-000"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.neucom.2014.01.029","article-title":"For micro-expression recognition: Database and suggestions","volume":"136","author":"Yan","year":"2014","journal-title":"Neurocomputing"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ekman, P., and Friesen, W.V. (1978). Facial Action Coding System: Investigator\u2019s Guide, Consulting Psychologists Press.","DOI":"10.1037\/t27734-000"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Davison, A.K., Yap, M.H., and Lansley, C. (2015, January 9\u201312). Micro-Facial Movement Detection Using Individualised Baselines and Histogram-Based Descriptors. Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Kowloon, China.","DOI":"10.1109\/SMC.2015.326"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-Vector Networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_32","unstructured":"Davison, A.K., Yap, M.H., Costen, N., Tan, K., Lansley, C., and Leightley, D. (2014). Micro-facial Movements: An Investigation on Spatio-Temporal Descriptors. 13th European Conference on Computer Vision (ECCV), Springer."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Huang, X., Wang, S.J., Zhao, G., and Piteikainen, M. (2015, January 7\u201313). Facial Micro-Expression Recognition Using Spatiotemporal Local Binary Pattern With Integral Projection. Proceedings of the The IEEE International Conference on Computer Vision (ICCV) Workshops, Santiago, Chile.","DOI":"10.1109\/ICCVW.2015.10"},{"key":"ref_34","unstructured":"Huang, X., Wang, S., Liu, X., Zhao, G., Feng, X., and Pietikainen, M. (arXiv, 2016). Spontaneous Facial Micro-Expression Recognition using Discriminative Spatiotemporal Local Binary Pattern with an Improved Integral Projection, arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1109\/TAFFC.2015.2485205","article-title":"A Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression Recognition","volume":"7","author":"Liu","year":"2016","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_36","unstructured":"Li, X., Hong, X., Moilanen, A., Huang, X., Pfister, T., Zhao, G., and Pietik\u00e4inen, M. (arXiv, 2015). Reading Hidden Emotions: Spontaneous Micro-expression Spotting and Recognition, arXiv."},{"key":"ref_37","unstructured":"Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., and Culotta, A. (2009). Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization. Advances in Neural Information Processing Systems 22, Curran Associates, Inc."},{"key":"ref_38","unstructured":"Wang, S.J., Yan, W.J., Zhao, G., Fu, X., and Zhou, C.G. (2014). Micro-Expression Recognition Using Robust Principal Component Analysis and Local Spatiotemporal Directional Features. Workshop at the European Conference on Computer Vision, Springer."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"6034","DOI":"10.1109\/TIP.2015.2496314","article-title":"Micro-Expression Recognition Using Color Spaces","volume":"24","author":"Wang","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1016\/j.neucom.2015.10.096","article-title":"Spontaneous facial micro-expression analysis using Spatiotemporal Completed Local Quantized Patterns","volume":"175","author":"Huang","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1016\/j.neucom.2016.05.083","article-title":"Sparse tensor canonical correlation analysis for micro-expression recognition","volume":"214","author":"Wang","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Liong, S.T., See, J., Phan, R.C.W., and Wong, K. (arXiv, 2016). Less is More: Micro-expression Recognition from Video using Apex Frame, arXiv.","DOI":"10.1109\/APSIPA.2017.8282090"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1109\/TAFFC.2016.2518162","article-title":"Microexpression Identification and Categorization Using a Facial Dynamics Map","volume":"8","author":"Xu","year":"2017","journal-title":"IEEE Trans. Affect. Computi."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1016\/j.neucom.2016.12.034","article-title":"A main directional maximal difference analysis for spotting facial movements from long-term videos","volume":"230","author":"Wang","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1007\/BF01115465","article-title":"Measuring facial movement","volume":"1","author":"Ekman","year":"1976","journal-title":"Environ. Psychol. Nonverbal Behav."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Platt, J.C. (1999). Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods, The MIT Press.","DOI":"10.7551\/mitpress\/1130.003.0016"},{"key":"ref_47","unstructured":"Davison, A.K., Lansley, C., Ng, C.C., Tan, K., and Yap, M.H. (arXiv, 2016). Objective Micro-Facial Movement Detection Using FACS-Based Regions and Baseline Evaluation, arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1079","DOI":"10.1109\/ACCESS.2015.2455871","article-title":"Wrinkle detection using hessian line tracking","volume":"3","author":"Ng","year":"2015","journal-title":"IEEE Access"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Wang, Y., See, J., Phan, R.C.W., and Oh, Y.H. (2015). Efficient Spatio-Temporal Local Binary Patterns for Spontaneous Facial Micro-Expression Recognition. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0124674"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000006","article-title":"Learning Deep Architectures for AI","volume":"2","author":"Bengio","year":"2009","journal-title":"Found. Trends Mach. Learn."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1561\/2000000039","article-title":"Deep Learning: Methods and Applications","volume":"7","author":"Deng","year":"2014","journal-title":"Found. Trends Signal Process."},{"key":"ref_52","first-page":"479","article-title":"Facial Skin Classification Using Convolutional Neural Networks","volume":"Volume 10317","author":"Alarifi","year":"2017","journal-title":"Proceedings of the 14th International Conference on Image Analysis and Recognition, ICIAR 2017"},{"key":"ref_53","unstructured":"Soomro, K., Zamir, A.R., and Shah, M. (arXiv, 2012). UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild, arXiv."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., and Fei-Fei, L. (2014, January 23\u201328). Large-scale Video Classification with Convolutional Neural Networks. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.223"}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/4\/10\/119\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:25:36Z","timestamp":1760196336000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/4\/10\/119"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,10,15]]},"references-count":54,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2018,10]]}},"alternative-id":["jimaging4100119"],"URL":"https:\/\/doi.org\/10.3390\/jimaging4100119","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,10,15]]}}}