{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T23:21:20Z","timestamp":1775085680859,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2022,6,5]],"date-time":"2022-06-05T00:00:00Z","timestamp":1654387200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Natural Science Foundation of Fujian Province of China","award":["2019J01001"],"award-info":[{"award-number":["2019J01001"]}]},{"name":"Natural Science Foundation of Fujian Province of China","award":["3502Z20203002"],"award-info":[{"award-number":["3502Z20203002"]}]},{"name":"Industry-University-Research Project of Xiamen City","award":["2019J01001"],"award-info":[{"award-number":["2019J01001"]}]},{"name":"Industry-University-Research Project of Xiamen City","award":["3502Z20203002"],"award-info":[{"award-number":["3502Z20203002"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Micro-expressions are rapid and subtle facial movements. Different from ordinary facial expressions in our daily life, micro-expressions are very difficult to detect and recognize. In recent years, due to a wide range of potential applications in many domains, micro-expression recognition has aroused extensive attention from computer vision. Because available micro-expression datasets are very small, deep neural network models with a huge number of parameters are prone to over-fitting. In this article, we propose an OF-PCANet+ method for micro-expression recognition, in which we design a spatiotemporal feature learning strategy based on shallow PCANet+ model, and we incorporate optical flow sequence stacking with the PCANet+ network to learn discriminative spatiotemporal features. We conduct comprehensive experiments on publicly available SMIC and CASME2 datasets. The results show that our lightweight model obviously outperforms popular hand-crafted methods and also achieves comparable performances with deep learning based methods, such as 3D-FCNN and ELRCN.<\/jats:p>","DOI":"10.3390\/s22114296","type":"journal-article","created":{"date-parts":[[2022,6,5]],"date-time":"2022-06-05T10:47:11Z","timestamp":1654426031000},"page":"4296","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Micro-Expression Recognition Based on Optical Flow and PCANet+"],"prefix":"10.3390","volume":"22","author":[{"given":"Shiqi","family":"Wang","sequence":"first","affiliation":[{"name":"School of Informatics, Xiamen University, Xiamen 361005, China"}]},{"given":"Suen","family":"Guan","sequence":"additional","affiliation":[{"name":"School of Informatics, Xiamen University, Xiamen 361005, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1769-8057","authenticated-orcid":false,"given":"Hui","family":"Lin","sequence":"additional","affiliation":[{"name":"School of Informatics, Xiamen University, Xiamen 361005, China"}]},{"given":"Jianming","family":"Huang","sequence":"additional","affiliation":[{"name":"Center for Digital Media Computing and Software Engineering, Xiamen University, Xiamen 361005, China"}]},{"given":"Fei","family":"Long","sequence":"additional","affiliation":[{"name":"School of Informatics, Xiamen University, Xiamen 361005, China"},{"name":"Center for Digital Media Computing and Software Engineering, Xiamen University, Xiamen 361005, China"}]},{"given":"Junfeng","family":"Yao","sequence":"additional","affiliation":[{"name":"School of Informatics, Xiamen University, Xiamen 361005, China"},{"name":"Center for Digital Media Computing and Software Engineering, Xiamen University, Xiamen 361005, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Bhushan, B. (2015). Study of Facial Micro-expressions in Psychology. Understanding Facial Expressions in Communication: Cross-Cultural and Multidisciplinary Perspectives, Springer.","DOI":"10.1007\/978-81-322-1934-7_13"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1631\/jzus.B1100063","article-title":"Effects of the duration of expressions on the recognition of microexpressions","volume":"13","author":"Shen","year":"2012","journal-title":"J. Zhejiang Univ. Sci. B"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s10919-013-0159-8","article-title":"How Fast are the Leaked Facial Expressions: The Duration of Micro-Expressions","volume":"37","author":"Yan","year":"2013","journal-title":"J. Nonverbal Behav."},{"key":"ref_4","first-page":"333","article-title":"Facial Microexpression Recognition: A Survey","volume":"43","author":"Xu","year":"2017","journal-title":"Acta Autom. Sin."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1080\/00332747.1969.11023575","article-title":"Nonverbal Leakage and Clues to Deception","volume":"32","author":"Ekman","year":"1969","journal-title":"Psychiatry"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Martin, C. (2009). Lie Catching and Micro Expressions. The Philosophy of Deception, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780195327939.001.0001"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1007\/s10979-008-9166-4","article-title":"Police Lie Detection Accuracy: The Effect of Lie Scenario","volume":"33","author":"Frank","year":"2009","journal-title":"Law Hum. Behav."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1548","DOI":"10.1109\/TPAMI.2016.2515606","article-title":"Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications","volume":"38","author":"Corneanu","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1109\/TIP.2020.3035042","article-title":"Joint Local and Global Information Learning with Single Apex Frame Detection for Micro-Expression Recognition","volume":"30","author":"Li","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1007\/s10919-008-0057-7","article-title":"Detecting Deception from Emotional and Unemotional Cues","volume":"33","author":"Warren","year":"2009","journal-title":"J. Nonverbal Behav."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.neucom.2014.01.029","article-title":"For micro-expression recognition: Database and suggestions","volume":"136","author":"Yan","year":"2014","journal-title":"Neurocomputing"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Li, X., Pfister, T., Huang, X., Zhao, G., and Pietik\u00e4inen, M. (2013, January 22\u201326). A Spontaneous Micro-expression Database: Inducement, collection and baseline. Proceedings of the 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, Shanghai, China.","DOI":"10.1109\/FG.2013.6553717"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Pfister, T., Li, X., Zhao, G., and Pietik\u00e4inen, M. (2011, January 6\u201313). Recognising spontaneous facial micro-expressions. Proceedings of the International Conference on Computer Vision, Washington, DC, USA.","DOI":"10.1109\/ICCV.2011.6126401"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1109\/TPAMI.2007.1110","article-title":"Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions","volume":"29","author":"Zhao","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yan, W.J., Li, X., Wang, S.J., Zhao, G., Liu, Y.J., Chen, Y.H., and Fu, X. (2014). CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0086041"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Huang, X., Zhao, G., Hong, X., Pietik\u00e4inen, M., and Zheng, W. (2013, January 17\u201320). Texture Description with Completed Local Quantized Patterns. Proceedings of the Scandinavian Conference on Image Analysis, Espoo, Finland.","DOI":"10.1007\/978-3-642-38886-6_1"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wang, Y., See, J., Phan, R.C.W., and Oh, Y.H. (2014, January 1\u20135). LBP with Six Intersection Points: Reducing Redundant Information in LBP-TOP for Micro-expression Recognition. Proceedings of the Asia Conference on Computer Vision 2014, Singapore.","DOI":"10.1007\/978-3-319-16865-4_34"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Huang, X., Wang, S., Zhao, G., and Piteikainen, M. (2015, January 7\u201313). Facial Micro-Expression Recognition Using Spatiotemporal Local Binary Pattern with Integral Projection. Proceedings of the IEEE International Conference on Computer Vision Workshop, Santiago, Chile.","DOI":"10.1109\/ICCVW.2015.10"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3160","DOI":"10.1109\/TMM.2018.2820321","article-title":"Learning From Hierarchical Spatiotemporal Descriptors for Micro-Expression Recognition","volume":"20","author":"Zong","year":"2018","journal-title":"IEEE Trans. Multimed."},{"key":"ref_20","unstructured":"Lu, Z., Luo, Z., Zheng, H., Chen, J., and Li, W. (2014, January 1\u20135). A Delaunay-Based Temporal Coding Model for Micro-expression Recognition. Proceedings of the Asia Conference on Computer Vision, Singapore."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1109\/TAFFC.2016.2518162","article-title":"Microexpression Identification and Categorization Using a Facial Dynamics Map","volume":"8","author":"Xu","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1109\/TAFFC.2015.2485205","article-title":"A Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression Recognition","volume":"7","author":"Liu","year":"2016","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1016\/j.image.2016.06.004","article-title":"Spontaneous subtle expression detection and recognition based on facial strain","volume":"47","author":"Liong","year":"2016","journal-title":"Signal Process. Image Commun."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Kim, D.H., Baddar, W.J., and Ro, Y.M. (2016, January 15\u201319). Micro-Expression Recognition with Expression-State Constrained Spatio-Temporal Feature Representations. Proceedings of the 24th ACM International Conference on Multimedia, New York, NY, USA.","DOI":"10.1145\/2964284.2967247"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1745","DOI":"10.3389\/fpsyg.2017.01745","article-title":"Dual Temporal Scale Convolutional Neural Network for Micro-Expression Recognition","volume":"8","author":"Peng","year":"2017","journal-title":"Front. Psychol."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Li, Y., Huang, X., and Zhao, G. (2018, January 7\u201310). Can Micro-Expression be Recognized Based on Single Apex Frame?. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.","DOI":"10.1109\/ICIP.2018.8451376"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Khor, H.Q., See, J., Phan, R.C.W., and Lin, W. (2018, January 15\u201319). Enriched Long-Term Recurrent Convolutional Network for Facial Micro-Expression Recognition. Proceedings of the 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00105"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1007\/s10044-018-0757-5","article-title":"Micro-expression recognition based on 3D flow convolutional neural network","volume":"22","author":"Li","year":"2019","journal-title":"Pattern Anal. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1581","DOI":"10.1109\/LSP.2017.2749763","article-title":"Stacking PCANet+: An Overly Simplified ConvNets Baseline for Face Recognition","volume":"24","author":"Low","year":"2017","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"5017","DOI":"10.1109\/TIP.2015.2475625","article-title":"PCANet: A Simple Deep Learning Baseline for Image Classification","volume":"24","author":"Chan","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Garg, R., Roussos, A., and Agapito, L. (2011, January 25\u201327). Robust Trajectory-Space TV-L1 Optical Flow for Non-rigid Sequences. Proceedings of the Energy Minimization Methods in Computer Vision and Pattern Recognition, Petersburg, Russia.","DOI":"10.1007\/978-3-642-23094-3_22"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/11\/4296\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:24:44Z","timestamp":1760138684000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/11\/4296"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,5]]},"references-count":31,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["s22114296"],"URL":"https:\/\/doi.org\/10.3390\/s22114296","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,5]]}}}