{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:15:29Z","timestamp":1760235329126,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,8,11]],"date-time":"2021-08-11T00:00:00Z","timestamp":1628640000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Long video datasets of facial macro- and micro-expressions remains in strong demand with the current dominance of data-hungry deep learning methods. There are limited methods of generating long videos which contain micro-expressions. Moreover, there is a lack of performance metrics to quantify the generated data. To address the research gaps, we introduce a new approach to generate synthetic long videos and recommend assessment methods to inspect dataset quality. For synthetic long video generation, we use the state-of-the-art generative adversarial network style transfer method\u2014StarGANv2. Using StarGANv2 pre-trained on the CelebA dataset, we transfer the style of a reference image from SAMM long videos (a facial micro- and macro-expression long video dataset) onto a source image of the FFHQ dataset to generate a synthetic dataset (SAMM-SYNTH). We evaluate SAMM-SYNTH by conducting an analysis based on the facial action units detected by OpenFace. For quantitative measurement, our findings show high correlation on two Action Units (AUs), i.e., AU12 and AU6, of the original and synthetic data with a Pearson\u2019s correlation of 0.74 and 0.72, respectively. This is further supported by evaluation method proposed by OpenFace on those AUs, which also have high scores of 0.85 and 0.59. Additionally, optical flow is used to visually compare the original facial movements and the transferred facial movements. With this article, we publish our dataset to enable future research and to increase the data pool of micro-expressions research, especially in the spotting task.<\/jats:p>","DOI":"10.3390\/jimaging7080142","type":"journal-article","created":{"date-parts":[[2021,8,11]],"date-time":"2021-08-11T08:35:52Z","timestamp":1628670952000},"page":"142","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Synthesising Facial Macro- and Micro-Expressions Using Reference Guided Style Transfer"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2251-9308","authenticated-orcid":false,"given":"Chuin Hong","family":"Yap","sequence":"first","affiliation":[{"name":"Department of Computing and Mathematics, Manchester Metropolitan University, Manchester M15 6BH, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6883-6515","authenticated-orcid":false,"given":"Ryan","family":"Cunningham","sequence":"additional","affiliation":[{"name":"Department of Computing and Mathematics, Manchester Metropolitan University, Manchester M15 6BH, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6496-0209","authenticated-orcid":false,"given":"Adrian K.","family":"Davison","sequence":"additional","affiliation":[{"name":"Faculty of Biology, Medicine and Health, The University of Manchester, Manchester M13 9PL, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7681-4287","authenticated-orcid":false,"given":"Moi Hoon","family":"Yap","sequence":"additional","affiliation":[{"name":"Department of Computing and Mathematics, Manchester Metropolitan University, Manchester M15 6BH, UK"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s10919-013-0159-8","article-title":"How fast are the leaked facial expressions: The duration of micro-expressions","volume":"37","author":"Yan","year":"2013","journal-title":"J. Nonverbal Behav."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1196\/annals.1280.010","article-title":"Darwin, deception, and facial expression","volume":"1000","author":"Ekman","year":"2003","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"ref_3","first-page":"140","article-title":"Emotions revealed: Recognising facial expressions","volume":"12","author":"Ekman","year":"2004","journal-title":"Stud. BMJ"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1037\/0003-066X.48.4.384","article-title":"Facial expression and emotion","volume":"48","author":"Ekman","year":"1993","journal-title":"Am. Psychol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Li, J., Soladie, C., Seguier, R., Wang, S.J., and Yap, M.H. (2019, January 14\u201318). Spotting micro-expressions on long videos sequences. Proceedings of the 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), Lille, France.","DOI":"10.1109\/FG.2019.8756626"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, J., Wang, S., Yap, M.H., See, J., Hong, X., and Li, X. (2020, January 16\u201320). MEGC2020-The Third Facial Micro-Expression Grand Challenge. Proceedings of the 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)(FG), Buenos Aires, Argentina.","DOI":"10.1109\/FG47880.2020.00035"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"See, J., Yap, M.H., Li, J., Hong, X., and Wang, S.J. (2019, January 14\u201318). Megc 2019\u2014The second facial micro-expressions grand challenge. Proceedings of the 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), Lille, France.","DOI":"10.1109\/FG.2019.8756611"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yap, C.H., Kendrick, C., and Yap, M.H. (2020, January 18\u201322). SAMM Long Videos: A Spontaneous Facial Micro-and Macro-Expressions Dataset. Proceedings of the 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020) (FG), Buenos Aires, Argentina.","DOI":"10.1109\/FG47880.2020.00029"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1109\/TAFFC.2017.2654440","article-title":"CAS (ME)\u02c6 2: A Database for Spontaneous Macro-expression and Micro-expression Spotting and Recognition","volume":"9","author":"Qu","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ekman, R. (1997). What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS), Oxford University Press.","DOI":"10.1093\/oso\/9780195104462.001.0001"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/j.jneumeth.2011.06.023","article-title":"Automated facial action coding system for dynamic analysis of facial expressions in neuropsychiatric disorders","volume":"200","author":"Hamm","year":"2011","journal-title":"J. Neurosci. Methods"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1037\/npe0000028","article-title":"Automated facial coding: Validation of basic emotions and FACS AUs in FaceReader","volume":"7","author":"Lewinski","year":"2014","journal-title":"J. Neurosci. Psychol. Econ."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on image data augmentation for deep learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J. Big Data"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sucholutsky, I., and Schonlau, M. (2020). \u2018Less Than One\u2019-Shot Learning: Learning N Classes From M<N Samples. arXiv.","DOI":"10.1609\/aaai.v35i11.17171"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gatys, L.A., Ecker, A.S., and Bethge, M. (2015). A neural algorithm of artistic style. arXiv.","DOI":"10.1167\/16.12.326"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Li, C., and Wand, M. (2016, January 27\u201330). Combining markov random fields and convolutional neural networks for image synthesis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.272"},{"key":"ref_17","unstructured":"Champandard, A.J. (2016). Semantic style transfer and turning two-bit doodles into fine artworks. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2601097.2601137","article-title":"Style transfer for headshot portraits","volume":"33","author":"Shih","year":"2014","journal-title":"ACM Trans. Graph."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Pumarola, A., Agudo, A., Martinez, A.M., Sanfeliu, A., and Moreno-Noguer, F. (2018, January 8\u201316). Ganimation: Anatomically-aware facial animation from a single image. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01249-6_50"},{"key":"ref_20","unstructured":"Siarohin, A., Lathuili\u00e8re, S., Tulyakov, S., Ricci, E., and Sebe, N. (2020). First order motion model for image animation. arXiv."},{"key":"ref_21","unstructured":"Vondrick, C., Pirsiavash, H., and Torralba, A. (2016, January 16\u201321). Generating videos with scene dynamics. Proceedings of the 30th International Conference on Neural Information Processing Systems, Kyoto, Japan."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Saito, M., Matsumoto, E., and Saito, S. (2017, January 22\u201329). Temporal generative adversarial nets with singular value clipping. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.308"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Tulyakov, S., Liu, M.Y., Yang, X., and Kautz, J. (2018, January 18\u201323). Mocogan: Decomposing motion and content for video generation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00165"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Xie, H.X., Lo, L., Shuai, H.H., and Cheng, W.H. (2020, January 12\u201316). AU-assisted Graph Attention Convolutional Network for Micro-Expression Recognition. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.","DOI":"10.1145\/3394171.3414012"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Yan, W.J., Li, X., Wang, S.J., Zhao, G., Liu, Y.J., Chen, Y.H., and Fu, X. (2014). CASME II: An improved spontaneous micro-expression database and the baseline evaluation. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0086041"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1109\/TAFFC.2016.2573832","article-title":"SAMM: A Spontaneous Micro-Facial Movement Dataset","volume":"9","author":"Davison","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Choi, Y., Uh, Y., Yoo, J., and Ha, J.W. (2020, January 14\u201319). Stargan v2: Diverse image synthesis for multiple domains. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00821"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Karras, T., Laine, S., and Aila, T. (2019, January 16\u201320). A style-based generator architecture for generative adversarial networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00453"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Baltrusaitis, T., Zadeh, A., Lim, Y.C., and Morency, L.P. (2018, January 15\u201319). Openface 2.0: Facial behavior analysis toolkit. Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00019"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Liu, Z., Luo, P., Wang, X., and Tang, X. (2015, January 7\u201313). Deep learning face attributes in the wild. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.425"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., and Choo, J. (2018, January 18\u201323). Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00916"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zadeh, A., Chong Lim, Y., Baltrusaitis, T., and Morency, L.P. (2017, January 22\u201329). Convolutional experts constrained local model for 3d facial landmark detection. Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, Italy.","DOI":"10.1109\/ICCVW.2017.296"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1627","DOI":"10.1021\/ac60214a047","article-title":"Smoothing and differentiation of data by simplified least squares procedures","volume":"36","author":"Savitzky","year":"1964","journal-title":"Anal. Chem."},{"key":"ref_34","unstructured":"Pisani, R.P. (2007). Statistics (International Student Edition), WW Norton & Company. [4th ed.]."},{"key":"ref_35","first-page":"1","article-title":"On the \u2018probable error\u2019 of a coefficient of correlation deduced from a small sample","volume":"1","author":"Fisher","year":"1921","journal-title":"Metron"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1109\/T-AFFC.2013.4","article-title":"Disfa: A spontaneous facial action intensity database","volume":"4","author":"Mavadati","year":"2013","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Farneb\u00e4ck, G. (2003). Two-frame motion estimation based on polynomial expansion. Scandinavian Conference on Image Analysis, Springer.","DOI":"10.1007\/3-540-45103-X_50"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1128","DOI":"10.3389\/fpsyg.2018.01128","article-title":"A survey of automatic facial micro-expression analysis: Databases, methods, and challenges","volume":"9","author":"Oh","year":"2018","journal-title":"Front. Psychol."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1789","DOI":"10.1109\/TIFS.2012.2214212","article-title":"Face recognition performance: Role of demographic information","volume":"7","author":"Klare","year":"2012","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_40","unstructured":"Buolamwini, J., and Gebru, T. (2018, January 23\u201324). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of the Conference on Fairness, Accountability and Transparency, PMLR, New York, NY, USA."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Garcia, R.V., Wandzik, L., Grabner, L., and Krueger, J. (2019, January 4\u20137). The harms of demographic bias in deep face recognition research. Proceedings of the 2019 International Conference on Biometrics (ICB), Crete, Greece.","DOI":"10.1109\/ICB45273.2019.8987334"}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/7\/8\/142\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:44:12Z","timestamp":1760165052000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/7\/8\/142"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,11]]},"references-count":41,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["jimaging7080142"],"URL":"https:\/\/doi.org\/10.3390\/jimaging7080142","relation":{},"ISSN":["2313-433X"],"issn-type":[{"type":"electronic","value":"2313-433X"}],"subject":[],"published":{"date-parts":[[2021,8,11]]}}}