{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T09:13:08Z","timestamp":1760346788739,"version":"build-2065373602"},"reference-count":50,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2019,4,5]],"date-time":"2019-04-05T00:00:00Z","timestamp":1554422400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Micro-expression is a spontaneous emotional representation that is not controlled by logic. A micro-expression is both transitory (short duration) and subtle (small intensity), so it is difficult to detect in people. Micro-expression detection is widely used in the fields of psychological analysis, criminal justice and human-computer interaction. Additionally, like traditional facial expressions, micro-expressions also have local muscle movement. Psychologists have shown micro-expressions have necessary morphological patches (NMPs), which are triggered by emotion. Furthermore, the objective of this paper is to sort and filter these NMPs and extract features from NMPs to train classifiers to recognize micro-expressions. Firstly, we use the optical flow method to compare the on-set frame and the apex frame of the micro-expression sequences. By doing this, we could find facial active patches. Secondly, to find the NMPs of micro-expressions, this study calculates the local binary pattern from three orthogonal planes (LBP-TOP) operators and cascades them with optical flow histograms to form the fusion features of the active patches. Finally, a random forest feature selection (RFFS) algorithm is used to identify the NMPs and to characterize them via support vector machine (SVM) classifier. We evaluated the proposed method on two popular publicly available databases: CASME II and SMIC. Results show that NMPs are statistically determined and contribute to significant discriminant ability instead of holistic utilization of all facial regions.<\/jats:p>","DOI":"10.3390\/sym11040497","type":"journal-article","created":{"date-parts":[[2019,4,5]],"date-time":"2019-04-05T11:36:01Z","timestamp":1554464161000},"page":"497","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["An Improved Micro-Expression Recognition Method Based on Necessary Morphological Patches"],"prefix":"10.3390","volume":"11","author":[{"given":"Yue","family":"Zhao","sequence":"first","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, Xi\u2019an 710129, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiancheng","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, Xi\u2019an 710129, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,4,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1080\/00332747.1969.11023575","article-title":"Detecting deception from emotional and unemotional cues","volume":"32","author":"Ekman","year":"1969","journal-title":"Psychiatry"},{"key":"ref_2","unstructured":"Ekman, P. (2019, January 20). Mett. Micro Expression Training Tool. Available online: https:\/\/www.paulekman.com\/product\/micro-facial-expressions-training-tool\/."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ekman, P., and Friesen, W. (1978). Facial Action Coding System: A Technique for the Measurement of Facial Movement, Consulting Psychologists Press.","DOI":"10.1037\/t27734-000"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1037\/h0030377","article-title":"Constants across cultures in the face and emotion","volume":"17","author":"Ekman","year":"1971","journal-title":"J. Personal. Soc. Psychol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Gottschalk, L.A., and Auerbach, A.H. (1966). Micro-momentary facial expressions as indicators of ego mechanisms in psychotherapy. Methods of Research in Psychotherapy, Springer.","DOI":"10.1007\/978-1-4684-6045-2"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1007\/s10919-008-0057-7","article-title":"Detecting deception from emotional and unemotional cues","volume":"33","author":"Warren","year":"2009","journal-title":"J. Nonverbal Behav."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ekman, P. (2009). Lie catching and microexpressions. The Philosophy of Deception, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780195327939.003.0008"},{"key":"ref_8","unstructured":"Ekman, P. (2009). Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage, W. W. Norton and Company."},{"key":"ref_9","first-page":"63","article-title":"Facial Expressions of Emotion: An Old Controversy and New Findings","volume":"1273","author":"Ekman","year":"1992","journal-title":"Philos. Trans. R. Soc."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1109\/TPAMI.2007.1110","article-title":"Dynamic texture recognition using local binary patterns with an application to facial expressions","volume":"6","author":"Zhao","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Pflster, T., Li, X.B., Zhao, G.Y., and Pietikainen, M. (2011, January 6\u201313). Recognising spontaneous facial micro-expressions. Proceedings of the 2011 IEEE International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126401"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"C\u0103leanu, C.-D. (2013, January 23\u201325). Face expression recognition: A brief overview of the last decade. Proceedings of the IEEE 8th International Symposium on Applied Computational Intelligence and Informatics (SACI), Timisoara, Romania.","DOI":"10.1109\/SACI.2013.6608958"},{"key":"ref_13","unstructured":"Li, S., and Deng, W. (2018, January 18\u201322). Deep Facial Expression Recognition: A Survey. Proceedings of the International Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA."},{"key":"ref_14","unstructured":"Kulkarni, K., and Corneanu, C. (2018). Automatic recognition of facial displays of unfelt emotions. IEEE Trans. Affect. Comput."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"26391","DOI":"10.1109\/ACCESS.2018.2831927","article-title":"Dominant and complementary emotion recognition from still images of faces","volume":"6","author":"Guo","year":"2018","journal-title":"IEEE Access."},{"key":"ref_16","unstructured":"Loob, C., Rasti, P., and L\u00fcsi, I. (June, January 30). Dominant and complementary multi-emotional facial expression recognition using c-support vector classification. Proceedings of the 12th IEEE International Conference on Automatic Face & Gesture Recognition, Washington, DC, USA."},{"key":"ref_17","unstructured":"Otberdout, N., Kacem, A., Daoudi, M., and Ballihi, L. (2018). Deep Covariance Descriptors for Facial Expression Recognition. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wan, J., Escalera, S., and Anbarjafari, G. (2017, January 22\u201329). Results and Analysis of ChaLearn LAP Multi-modal Isolated and Continuous Gesture Recognition, and Real Versus Fake Expressed Emotions Challenges. Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW), Venice, Italy.","DOI":"10.1109\/ICCVW.2017.377"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/TAFFC.2016.2593719","article-title":"Facial Expression Recognition in Video with Multiple Feature Fusion","volume":"9","author":"Chen","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Shreve, M., Godavarthy, S., Goldgof, D., and Sarkar, S. (2011, January 21\u201325). Macroand micro-expression spotting in long videos using spatiotemporal strain. Proceedings of the 2011 IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, Santa Barbara, CA, USA.","DOI":"10.1109\/FG.2011.5771451"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Huang, X.H., Zhao, G.Y., Hong, X.P., Pietikainen, M., and Zheng, W.M. (2013). Texture description with completed local quantized patterns. Image Analysis, Springer.","DOI":"10.1007\/978-3-642-38886-6_1"},{"key":"ref_22","unstructured":"Wang, Y.D., See, J., and Phan, P.C.W. (2014, January 1\u20135). LBP with six intersection points: Reducing redundant information in LBPTOP for micro-expression recognition. Proceedings of the 12th Conference on Computer Vision, Singapore."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, Y., Li, Y., Ma, X., and Song, R. (2017). Facial Expression Recognition with Fusion Features Extracted from Salient Facial Areas. Sensors, 17.","DOI":"10.20944\/preprints201701.0102.v1"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TAFFC.2014.2386334","article-title":"Automatic Facial Expression Recognition Using Features of Salient Facial Patches","volume":"1","author":"Happy","year":"2015","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Liu, Y., Cao, Y., Li, Y., Liu, M., and Song, R. (2016, January 6\u201310). Facial Expression Recognition with PCA and LBP Features Extracting from Active Facial Patches. Proceedings of the IEEE International Conference on Real-time Computer and Robotics, Angkor Wat, Cambodia.","DOI":"10.1109\/RCAR.2016.7784056"},{"key":"ref_26","first-page":"772","article-title":"Automatic Facial Expression Recognition based on the Salient Facial Patches","volume":"2","author":"Rejila","year":"2016","journal-title":"Int. J. Sci. Technol. Eng."},{"key":"ref_27","unstructured":"Zhu, X., and Ramanan, D. (2012, January 16\u201321). Face detection, pose estimation, and landmark localization in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1145\/212094.212141","article-title":"The computation of optical flow","volume":"3","author":"Beauchemin","year":"1995","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1006\/cviu.1995.1004","article-title":"Active shape models-their training and application","volume":"1","author":"Cootes","year":"1995","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_30","unstructured":"Lucas, B.D., and Kanade, T. (1981, January 24\u201328). An iterative image registration technique with an application to stereo vision. Proceedings of the International Joint Conference on Artificial Intelligence, Vancouver, BC, Canada."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/0004-3702(81)90024-2","article-title":"Determining optical flow","volume":"17","author":"Horn","year":"1981","journal-title":"Artif. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ren, X., and Malik, J. (2003, January 13\u201316). Learning a classification model for segmentation. Proceedings of the Ninth IEEE International Conference on Computer Vision, Nice, France.","DOI":"10.1109\/ICCV.2003.1238308"},{"key":"ref_33","first-page":"888","article-title":"Normalized cuts and image segmentation","volume":"8","author":"Shi","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","first-page":"603","article-title":"Mean shift: A robust approach toward feature space analysis","volume":"5","author":"Comaniciu","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_35","first-page":"1","article-title":"CASME II: An improved spontaneous microexpression database and the baseline evaluation","volume":"1","author":"Yan","year":"2014","journal-title":"PLoS ONE"},{"key":"ref_36","unstructured":"Yan, W.J., Wu, Q., Liu, Y.J., Wang, S.J., and Fu, X.L. (2013, January 22\u201326). CASME database: A dataset of spontaneous micro-expressions collected from neutralized faces. Proceedings of the 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, Shanghai, China."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1109\/TAFFC.2017.2654440","article-title":"CAS(ME)2: A database for spontaneous macro-expression and micro-expression spotting and recognition","volume":"9","author":"Qu","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zhou, Z.H., Zhao, G.Y., and Pietikainen, M. (2011, January 20\u201325). Towards a practical lipreading system. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995345"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Wang, S., Yan, W., Li, X., Zhao, G., and Fu, X. (2014, January 24\u201328). Micro-expression recognition using dynamic textures on tensor independent color space. Proceedings of the 22nd International Conference on Pattern Recognition, Stockholm, Sweden.","DOI":"10.1109\/ICPR.2014.800"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Mayya, V., Pai, R., and Pai, M. (2016, January 11\u201312). Combining temporal interpolation and DCNN for faster recognition of micro-expressions in video sequences. Proceedings of the International Conference on Advances in Computing, Ghaziabad, India.","DOI":"10.1109\/ICACCI.2016.7732128"},{"key":"ref_41","unstructured":"Patel, D., Hong, X., and Zhao, G. (2016, January 11\u201312). Selective deep features for micro-expression. Proceedings of the International Conference on Advances in Computing, Ghaziabad, India."},{"key":"ref_42","first-page":"563","article-title":"Towards reading hidden emotions: A comparative study of spontaneous micro-expression spotting and recognition methods","volume":"99","author":"Li","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_43","first-page":"254","article-title":"Micro-expression identification and categorization using a facial dynamics map","volume":"9","author":"Xu","year":"2016","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1745","DOI":"10.3389\/fpsyg.2017.01745","article-title":"Dual Temporal Scale Convolutional Neural Network for Micro-Expression Recognition","volume":"8","author":"Peng","year":"2017","journal-title":"Front. Psychol."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Peng, M., Wu, Z., and Zhang, Z. (2018, January 15\u201319). From Macro to Micro Expression Recognition: Deep Learning on Small Datasets Using Transfer Learning. Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition, Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00103"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1007\/s11265-017-1276-0","article-title":"Hybrid Facial Regions Extraction for Micro-Expression Recognition System","volume":"90","author":"Liong","year":"2018","journal-title":"J. Signal Process. Syst."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"3160","DOI":"10.1109\/TMM.2018.2820321","article-title":"Learning from Hierarchical Spatiotemporal Descriptors for Micro-Expression Recognition","volume":"20","author":"Zong","year":"2018","journal-title":"IEEE Trans. Multimedia"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.image.2018.05.014","article-title":"Motion descriptors for micro-expression recognition","volume":"67","author":"Lu","year":"2018","journal-title":"Signal Process. Image Commun."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Liu, Y., Li, B., and Lai, Y. (2018). Sparse MDMO: Learning a Discriminative Feature for Spontaneous Micro-Expression Recognition. IEEE Trans. Affect. Comput., 1\u201318.","DOI":"10.1109\/TAFFC.2018.2854166"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Liong, S., and Wong, K. (2017, January 12\u201315). Micro-expression recognition using apex frame with phase information. Proceedings of the 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, Kuala Lumpur, Malaysia.","DOI":"10.1109\/APSIPA.2017.8282090"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/11\/4\/497\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:43:14Z","timestamp":1760186594000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/11\/4\/497"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,5]]},"references-count":50,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2019,4]]}},"alternative-id":["sym11040497"],"URL":"https:\/\/doi.org\/10.3390\/sym11040497","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2019,4,5]]}}}