{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T14:31:06Z","timestamp":1773930666689,"version":"3.50.1"},"reference-count":91,"publisher":"Association for Computing Machinery (ACM)","issue":"CHI PLAY","license":[{"start":{"date-parts":[[2021,10,5]],"date-time":"2021-10-05T00:00:00Z","timestamp":1633392000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Postdoc Network ?Cognitive Con- flicts During Media Use? funded by the competition fund of the Leibniz Association (SAW16) within the framework of the ?Pact for Research and Innovation?"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,10,5]]},"abstract":"<jats:p>Users' emotional engagement in a task is important for performance and motivation. Non-intrusive, computerized process measures of engagement have the potential to provide fine-grained access to underlying affective states and processes. Thus, the current work brings together subjective measures (questionnaires) and objective process measures (facial expressions and head movements) of emotions to examine users' emotional engagement with respect to the absence or presence of game-elements. In particular, we randomly assigned 156 adult participants to either a spatial working memory task with or without game elements present, while their faces and head movements were recorded with a webcam during task execution. Positive and negative emotions were assessed before the task and twice during task execution using conventional questionnaires. We additionally examined whether perceived subjective effort, assumed to inherit a substantial affective component, manifests at a bodily expressive level alongside positive and negative emotions. Importantly, we explored the relationship between subjective and objective measures of emotions across the two tasks versions. We found a series of action units and head movements associated with the subjective experience of emotions as well as to subjective effort. Impacted by game elements, these associations often fit intuitively or lined up with findings from literature. As did a linear increase of blink (action unit 45) intensity relate to participants performing the task without game elements, presumably indicating disengagement in the more tedious task variant. On other occasions, associations between subjective and objective measures seemed indiscriminative or even contraindicated. Additionally, facial and bodily reactions and the resulting subjective-objective correspondences were rather consistent within, but not between the two task versions. Our work therefore both gains detailed access to automated emotion recognition and promotes its feasibility within research of game elements while highlighting the individuality and context dependency of emotional expressions.<\/jats:p>","DOI":"10.1145\/3474667","type":"journal-article","created":{"date-parts":[[2021,10,6]],"date-time":"2021-10-06T22:59:48Z","timestamp":1633561188000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Facial and Bodily Expressions of Emotional Engagement"],"prefix":"10.1145","volume":"5","author":[{"given":"Simon","family":"Greipl","sequence":"first","affiliation":[{"name":"Ludwig-Maximilians-University of Munich, Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katharina","family":"Bernecker","sequence":"additional","affiliation":[{"name":"University of Zurich, Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Manuel","family":"Ninaus","sequence":"additional","affiliation":[{"name":"University of Innsbruck, Innsbruck, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,10,6]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1002\/pits.20303"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.adolescence.2008.06.007"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2020.106542"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978--3--319-07221-0_5"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2009.5349579"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2015.11.003"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.psych.50.1.191"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0030737"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1177\/0962280219842890"},{"key":"e_1_2_1_10_1","volume-title":"The psychology of optimal experience New York","author":"Csikszentmihalyi Mihaly","unstructured":"Mihaly Csikszentmihalyi. 1990. The psychology of optimal experience New York. Harper & Row."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0025737"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1080\/00461520.2017.1281747"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2011.10.001"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1080\/02699939208411068"},{"key":"e_1_2_1_15_1","volume-title":"Facial action coding system (FACS). Hum. Face","author":"Ekman Paul","year":"2002","unstructured":"Paul Ekman. 2002. Facial action coding system (FACS). Hum. Face (2002)."},{"key":"e_1_2_1_16_1","first-page":"5","article-title":"Facial action coding system: a technique for the measurement of facial movement","volume":"3","author":"Ekman Paul","year":"1978","unstructured":"Paul Ekman and E. Friesen. 1978. Facial action coding system: a technique for the measurement of facial movement. Palo Alto 3, 2 (1978), 5.","journal-title":"Palo Alto"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8535.00229"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/wcs.1514"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.3102\/00346543074001059"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2016.02.002"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v033.i01"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1037\/0003-066X.43.5.349"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1038\/nrn1872"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1002\/bdm.1755"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10648-019-09514-z"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.95.3.524"},{"key":"e_1_2_1_27_1","volume-title":"Analyzing Posture and Affect in Task-Oriented Tutoring. In FLAIRS Conference, 438--443","author":"Grafsgaard Joseph F.","unstructured":"Joseph F. Grafsgaard, Kristy Elizabeth Boyer, Eric N. Wiebe, and James C. Lester. 2012. Analyzing Posture and Affect in Task-Oriented Tutoring. In FLAIRS Conference, 438--443."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2013.33"},{"key":"e_1_2_1_29_1","volume-title":"Eric N. Wiebe, and James Lester.","author":"Grafsgaard Joseph","year":"2013","unstructured":"Joseph Grafsgaard, Joseph B. Wiggins, Kristy Elizabeth Boyer, Eric N. Wiebe, and James Lester. 2013. Automatically recognizing facial expression: Predicting engagement and frustration. In Educational Data Mining 2013."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jsp.2007.07.001"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJTEL.2020.10028417"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1002\/9781118910566.ch14"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2015.09.005"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10964-010--9579--5"},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the workshop on Facial and Bodily Expression for Control and Adaptation of Games ECAG, Citeseer, 11--19","author":"Wouter","unstructured":"Wouter M. van den Hoogen, Wijnand A. IJsselsteijn, and Yvonne AW de Kort. 2008. Exploring behavioral expressions of player experience in digital games. In Proceedings of the workshop on Facial and Bodily Expression for Control and Adaptation of Games ECAG, Citeseer, 11--19."},{"key":"e_1_2_1_36_1","doi-asserted-by":"crossref","unstructured":"Rob Hyndman Yanfei Kang Pablo Montero-Manso Thiyanga Talagala Earo Wang Yangzhuoran Yang and Mitchell O'Hara-Wild. 2020. tsfeatures: Time Series Feature Extraction. Retrieved from https:\/\/CRAN.R-project.org\/package=tsfeatures","DOI":"10.32614\/CRAN.package.tsfeatures"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1080\/02699931.2021.1877118"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2018.01.007"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0801268105"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1177\/0956797610384745"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICALT.2001.943850"},{"key":"e_1_2_1_42_1","volume-title":"Diagn.-Gottingen- 42","author":"Krohne Heinz W.","year":"1996","unstructured":"Heinz W. Krohne, Boris Egloff, Carl-Walter Kohlmann, and Anja Tausch. 1996. Untersuchungen mit einer deutschen version der\" positive and negative affect schedule\"(PANAS). Diagn.-Gottingen- 42, (1996), 139--156."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1177\/1046878114563660"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1177\/1046878118774385"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.2224\/sbp.7054"},{"key":"e_1_2_1_46_1","volume-title":"Retrieved","author":"Li Shan","year":"2018","unstructured":"Shan Li and Weihong Deng. 2018. Deep Facial Expression Recognition: A Survey. ArXiv180408348 Cs (April 2018). Retrieved January 24, 2019 from http:\/\/arxiv.org\/abs\/1804.08348"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2011.5771418"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.31234\/osf.io"},{"key":"e_1_2_1_49_1","volume-title":"Retrieved","author":"Makantasis Konstantinos","year":"2021","unstructured":"Konstantinos Makantasis, Antonios Liapis, and Georgios N. Yannakakis. 2021. The Pixels and Sounds of Emotion: General-Purpose Representations of Arousal in Games. ArXiv210110706 Cs (February 2021). Retrieved February 17, 2021 from http:\/\/arxiv.org\/abs\/2101.10706"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2006.11.011"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2017.2731763"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1177\/0018720816673782"},{"key":"e_1_2_1_53_1","volume-title":"Proceedings of the Annual Meeting of the Cognitive Science Society.","author":"McDaniel Bethany","year":"2007","unstructured":"Bethany McDaniel, Sidney D'Mello, Brandon King, Patrick Chipman, Kristy Tapp, and Art Graesser. 2007. Facial features for affective state detection in learning environments. In Proceedings of the Annual Meeting of the Cognitive Science Society."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2010.5543833"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2015.08.048"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.88.2.203"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/978--3-030--15569--8_8"},{"key":"e_1_2_1_58_1","unstructured":"Fred M. Newmann (Ed.). 1992. Student engagement and achievement in American secondary schools. Teachers College Press New York."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2019.103641"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/TLT.2020.3031644"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.17083\/ijsg.v2i1.60"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2015.02.016"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-018-01193-y"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1007\/978--1--4614--2018--7_12"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2013.02.006"},{"key":"e_1_2_1_66_1","volume-title":"R: A Language and Environment for Statistical Computing","author":"Team R Core","year":"2020","unstructured":"R Core Team. 2020. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from https:\/\/www.R-project.org\/"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","unstructured":"M. Ramzan H. U. Khan S. M. Awan A. Ismail M. Ilyas and A. Mahmood. 2019. A Survey on State-of-the-Art Drowsiness Detection Techniques. IEEE Access 7 (2019) 61904--61919. DOI:https:\/\/doi.org\/10.1109\/ACCESS.2019.2914373","DOI":"10.1109\/ACCESS.2019.2914373"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2017.01303"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0077714"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10648-019-09498-w"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1007\/978--3--319-07638--6_34"},{"key":"e_1_2_1_72_1","volume-title":"The art of game design: a book of lenses","author":"Schell Jesse","unstructured":"Jesse Schell. 2015. The art of game design: a book of lenses (Second edition ed.). CRC Press, Boca Raton."},{"key":"e_1_2_1_73_1","volume-title":"Contemporary motivation research: From global to local perspectives","author":"Schnotz Wolfgang","unstructured":"Wolfgang Schnotz, Stefan Fries, and Holger Horz. 2009. Motivational aspects of cognitive load theory. In Contemporary motivation research: From global to local perspectives. Hogrefe & Huber Publishers, Ashland, OH, US, 69--96."},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","unstructured":"Y. A. Sekhavat S. Roohi H. Sakian Mohammadi and G. N. Yannakakis. 2020. Play with One's Feelings: A Study on Emotion Awareness for Player Experience. IEEE Trans. Games (2020) 1--1. DOI:https:\/\/doi.org\/10.1109\/TG.2020.3003324","DOI":"10.1109\/TG.2020.3003324"},{"key":"e_1_2_1_75_1","volume-title":"Casenhiser","author":"Shanker Stuart G.","year":"2013","unstructured":"Stuart G. Shanker and Devin M. Casenhiser. 2013. Reducing the effort in effortful control. In A Wittgensteinian Perspective on the use of Conceptual Analysis in Psychology. Springer, 214--232."},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijpsycho.2019.07.003"},{"key":"e_1_2_1_77_1","volume-title":"Retrieved","author":"Steele James","year":"2020","unstructured":"James Steele. 2020. What is (perception of) effort? Objective and subjective effort during task performance. PsyArXiv (June 2020). Retrieved January 30, 2021 from https:\/\/pure.solent.ac.uk\/en\/publications\/what-is-perception-of-effort-objective-and-subjective-effort-duri"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445230"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0026609"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2006.262464"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2011.5946910"},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1099-0992(1998110)28:6<879::AID-EJSP901>3.0.CO;2-W"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1111\/cdev.12138"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.3102\/0002831209361209"},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022--3514.54.6.1063"},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2008.4563182"},{"key":"e_1_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2016.00157"},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0134816"},{"key":"e_1_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJLT.2009.028804"},{"key":"e_1_2_1_90_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2018.2879512"},{"key":"e_1_2_1_91_1","doi-asserted-by":"publisher","DOI":"10.1177\/154193128502900808"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3474667","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3474667","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:42Z","timestamp":1750191522000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3474667"}},"subtitle":["How Dynamic Measures Reflect the Use of Game Elements and Subjective Experience of Emotions and Effort"],"short-title":[],"issued":{"date-parts":[[2021,10,5]]},"references-count":91,"journal-issue":{"issue":"CHI PLAY","published-print":{"date-parts":[[2021,10,5]]}},"alternative-id":["10.1145\/3474667"],"URL":"https:\/\/doi.org\/10.1145\/3474667","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,5]]},"assertion":[{"value":"2021-10-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}