{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T04:26:41Z","timestamp":1781238401700,"version":"3.54.1"},"reference-count":35,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2020,7,29]],"date-time":"2020-07-29T00:00:00Z","timestamp":1595980800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>A micro-expression is defined as an uncontrollable muscular movement shown on the face of humans when one is trying to conceal or repress his true emotions. Many researchers have applied the deep learning framework to micro-expression recognition in recent years. However, few have introduced the human visual attention mechanism to micro-expression recognition. In this study, we propose a three-dimensional (3D) spatiotemporal convolutional neural network with the convolutional block attention module (CBAM) for micro-expression recognition. First image sequences were input to a medium-sized convolutional neural network (CNN) to extract visual features. Afterwards, it learned to allocate the feature weights in an adaptive manner with the help of a convolutional block attention module. The method was testified in spontaneous micro-expression databases (Chinese Academy of Sciences Micro-expression II (CASME II), Spontaneous Micro-expression Database (SMIC)). The experimental results show that the 3D CNN with convolutional block attention module outperformed other algorithms in micro-expression recognition.<\/jats:p>","DOI":"10.3390\/info11080380","type":"journal-article","created":{"date-parts":[[2020,7,29]],"date-time":"2020-07-29T07:31:45Z","timestamp":1596007905000},"page":"380","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":70,"title":["Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression Recognition"],"prefix":"10.3390","volume":"11","author":[{"given":"Boyu","family":"Chen","sequence":"first","affiliation":[{"name":"School of Electronic and Information Engineering, Southwest University, Chongqing 400715, China"},{"name":"Chongqing Key Laboratory of Non-Linear Circuit and Intelligent Information Processing, Southwest University, Chongqing 400715, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhihao","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Southwest University, Chongqing 400715, China"},{"name":"Chongqing Key Laboratory of Non-Linear Circuit and Intelligent Information Processing, Southwest University, Chongqing 400715, China"},{"name":"Institute of Psychology, CAS, Beijing 100101, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nian","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Southwest University, Chongqing 400715, China"},{"name":"Chongqing Key Laboratory of Non-Linear Circuit and Intelligent Information Processing, Southwest University, Chongqing 400715, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yang","family":"Tan","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Southwest University, Chongqing 400715, China"},{"name":"Chongqing Key Laboratory of Non-Linear Circuit and Intelligent Information Processing, Southwest University, Chongqing 400715, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinyu","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Southwest University, Chongqing 400715, China"},{"name":"Chongqing Key Laboratory of Non-Linear Circuit and Intelligent Information Processing, Southwest University, Chongqing 400715, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3805-4138","authenticated-orcid":false,"given":"Tong","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Southwest University, Chongqing 400715, China"},{"name":"Chongqing Key Laboratory of Non-Linear Circuit and Intelligent Information Processing, Southwest University, Chongqing 400715, China"},{"name":"Institute of Psychology, CAS, Beijing 100101, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,7,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1196\/annals.1280.010","article-title":"Darwin, deception, and facial expression","volume":"1000","author":"Ekman","year":"2003","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"ref_2","unstructured":"Ekman, P. (2003). Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life, Henry Holt and Company."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s10919-013-0159-8","article-title":"How fast are the leaked facial expressions: The duration of micro-expressions","volume":"37","author":"Yan","year":"2013","journal-title":"J. Nonverbal. Behav."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"508","DOI":"10.1111\/j.1467-9280.2008.02116.x","article-title":"Reading between the lies: Identifying concealed and falsified emotions in universal facial expressions","volume":"19","author":"Porter","year":"2008","journal-title":"Psychol. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1527","DOI":"10.1162\/neco.2006.18.7.1527","article-title":"A fast learning algorithm for deep belief nets","volume":"18","author":"Hinton","year":"2006","journal-title":"Neural. Comput."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1618","DOI":"10.1109\/TIP.2019.2912358","article-title":"Learnet: Dynamic imaging network for micro expression recognition","volume":"29","author":"Verma","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1111","DOI":"10.1109\/TMM.2020.2980722","article-title":"Spatiotemporal recurrent convolutional networks for recognizing spontaneous micro-expressions (vol 22, pg 626, 2020)","volume":"22","author":"Xia","year":"2020","journal-title":"IEEE Trans. Multimed."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.neucom.2018.05.107","article-title":"Micro-expression recognition with small sample size by transferring long-term convolutional neural network","volume":"312","author":"Wang","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Polikovsky, S., Kameda, Y., and Ohta, Y. (2009, January 3). Facial micro-expressions recognition using high speed camera and 3D-gradient descriptor. Proceedings of the 3rd International Conference on Imaging for Crime Detection and Prevention (ICDP 2009), London, UK.","DOI":"10.1049\/ic.2009.0244"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1109\/TPAMI.2007.1110","article-title":"Dynamic texture recognition using local binary patterns with an application to facial expressions","volume":"29","author":"Zhao","year":"2007","journal-title":"IEEE Trans. Pattern Anal."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Pfister, T., Li, X., Zhao, G., and Pietikainen, M. (2011, January 6\u201313). Recognising Spontaneous Facial Micro-Expressions. Proceedings of the International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126401"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, S., Yan, W., Li, X., Zhao, G., and Fu, X. (2014, January 24\u201328). Micro-expression recognition using dynamic textures on tensor independent color space. Proceedings of the International Conference on Pattern Recognition, Stockholm, Sweden.","DOI":"10.1109\/ICPR.2014.800"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1109\/TAFFC.2015.2485205","article-title":"A main directional mean optical flow feature for spontaneous micro-expression recognition","volume":"7","author":"Liu","year":"2016","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going Deeper with Convolutions. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_15","unstructured":"Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_18","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1745","DOI":"10.3389\/fpsyg.2017.01745","article-title":"Dual temporal scale convolutional neural network for micro-expression recognition","volume":"8","author":"Peng","year":"2017","journal-title":"Front. Psychol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1007\/s10044-018-0757-5","article-title":"Micro-expression recognition based on 3D flow convolutional neural network","volume":"22","author":"Li","year":"2019","journal-title":"Pattern Anal. Appl."},{"key":"ref_21","unstructured":"Reddy, S.P.T., Karri, S.T., Dubey, S.R., and Mukherjee, S. (2019, January 14\u201319). Spontaneous facial micro-expression recognition using 3D spatiotemporal convolutional neural networks. Proceedings of the 2019 International Joint Conference on Neural Networks, Budapest, Hungary."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Peng, M., Wu, Z., Zhang, Z., and Chen, T. (2018, January 15\u201319). From macro to micro expression recognition: Deep learning on small datasets using transfer learning. Proceedings of the IEEE International Conference on Automatic Face Gesture Recognition, Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00103"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1109\/TPAMI.2012.59","article-title":"3D convolutional neural networks for human action recognition","volume":"35","author":"Ji","year":"2013","journal-title":"IEEE Trans. Pattern Anal."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J., and Kweon, I.S. (2018, January 8\u201314). Cbam: Convolutional block attention module. Proceedings of the 15th European Conference on Computer Vision, Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Peng, M., Wang, C., Chen, T., and Liu, G. (2016). Nirfacenet: A convolutional neural network for near-infrared face identification. Inf. Int. Interdiscip. J., 7.","DOI":"10.3390\/info7040061"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yan, W.J., Li, X.B., Wang, S.J., Zhao, G.Y., Liu, Y.J., Chen, Y.H., and Fu, X.L. (2014). Casme ii: An improved spontaneous micro-expression database and the baseline evaluation. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0086041"},{"key":"ref_27","unstructured":"Yan, W., Wu, Q., Liu, Y., Wang, S., and Fu, X. (2013, January 22\u201326). Casme database: A dataset of spontaneous micro-expressions collected from neutralized faces. Proceedings of the IEEE International Conference on Automatic Face Gesture Recognition, Shanghai, China."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.image.2019.02.005","article-title":"Off-apexnet on micro-expression recognition system","volume":"74","author":"Gan","year":"2019","journal-title":"Signal Process. Image Commun."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Liong, S., Gan, Y.S., See, J., Khor, H., and Huang, Y. (2019, January 14\u201318). Shallow triple stream three-dimensional cnn (ststnet) for micro-expression recognition. Proceedings of the IEEE International Conference on Automatic Face Gesture Recognition, Lille, France.","DOI":"10.1109\/FG.2019.8756567"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Li, X., Pfister, T., Huang, X., Zhao, G., and Pietikainen, M. (2013, January 22\u201326). A spontaneous micro-expression database: Inducement, collection and baseline. Proceedings of the IEEE International Conference on Automatic Face Gesture Recognition, Shanghai, China.","DOI":"10.1109\/FG.2013.6553717"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Asthana, A., Zafeiriou, S., Cheng, S., and Pantic, M. (2013, January 23\u201328). Robust Discriminative Response Map Fitting with Constrained Local Models. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.442"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2185520.2185561","article-title":"Eulerian video magnification for revealing subtle changes in the world","volume":"31","author":"Wu","year":"2012","journal-title":"ACM Trans. Graph."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Smolic, A., Muller, K., Dix, K., Merkle, P., Kauff, P., and Wiegand, T. (2008, January 12\u201315). Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems. Proceedings of the International Conference on Image Processing, San Diego, CA, USA.","DOI":"10.1109\/ICIP.2008.4712288"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1587\/transinf.2018EDP7153","article-title":"Combining 3D convolutional neural networks with transfer learning by supervised pre-training for facial micro-expression recognition","volume":"102","author":"ZHI","year":"2019","journal-title":"IEICE Trans. Inf. Syst."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/8\/380\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:52:29Z","timestamp":1760176349000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/8\/380"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,29]]},"references-count":35,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,8]]}},"alternative-id":["info11080380"],"URL":"https:\/\/doi.org\/10.3390\/info11080380","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,29]]}}}