{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:24:25Z","timestamp":1750220665908,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":38,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,10,21]],"date-time":"2020-10-21T00:00:00Z","timestamp":1603238400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"U.S. Army Research Laboratory","award":["#W911NF-13-2-0008"],"award-info":[{"award-number":["#W911NF-13-2-0008"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,10,21]]},"DOI":"10.1145\/3382507.3418892","type":"proceedings-article","created":{"date-parts":[[2020,10,22]],"date-time":"2020-10-22T10:04:34Z","timestamp":1603361074000},"page":"134-143","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Enhancing Affect Detection in Game-Based Learning Environments with Multimodal Conditional Generative Modeling"],"prefix":"10.1145","author":[{"given":"Nathan","family":"Henderson","sequence":"first","affiliation":[{"name":"North Carolina State University, Raleigh, NC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wookhee","family":"Min","sequence":"additional","affiliation":[{"name":"North Carolina State University, Raleigh, NC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan","family":"Rowe","sequence":"additional","affiliation":[{"name":"North Carolina State University, Raleigh, NC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James","family":"Lester","sequence":"additional","affiliation":[{"name":"North Carolina State University, Raleigh, NC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,10,22]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.5555\/3305381.3305405"},{"volume-title":"The Oxford Handbook of Affective Computing, Rafael Calvo, Sidney D'Mello","author":"Baker Ryan S.","key":"e_1_3_2_2_2_1","unstructured":"Ryan S. Baker , Jaclyn Ocumpaugh , and Rafael Calvo . 2015. Interaction-Based Affect Detection in Educational Software . In The Oxford Handbook of Affective Computing, Rafael Calvo, Sidney D'Mello , Jonathan Gratch and Arvid Kappas (eds.). Oxford University Press , 233--245. Ryan S. Baker, Jaclyn Ocumpaugh, and Rafael Calvo. 2015. Interaction-Based Affect Detection in Educational Software. In The Oxford Handbook of Affective Computing, Rafael Calvo, Sidney D'Mello, Jonathan Gratch and Arvid Kappas (eds.). Oxford University Press, 233--245."},{"key":"e_1_3_2_2_3_1","volume-title":"Multimodal machine learning: A survey and taxonomy","author":"Ahuja Chaitanya","year":"2018","unstructured":"Chaitanya Ahuja , and Louis-Philippe Morency . 2018. Multimodal machine learning: A survey and taxonomy . IEEE Trans. Pattern Anal. Mach. Intell . 41, 2 ( 2018 ). 423--443. Chaitanya Ahuja, and Louis-Philippe Morency. 2018. Multimodal machine learning: A survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41, 2 (2018). 423--443."},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818346.2820739"},{"key":"e_1_3_2_2_5_1","volume-title":"Proceedings of the 25th International Joint Conference on Artificial Intelligence. 4125--4129","author":"Bosch Nigel","year":"2016","unstructured":"Nigel Bosch , Sidney D'Mello , Ryan S. Baker , Valerie Shute , Matthew Ventura , Lubin Wang , and Weinan Zhao . 2016 . Detecting student emotions in computer-enabled classrooms . In Proceedings of the 25th International Joint Conference on Artificial Intelligence. 4125--4129 . Nigel Bosch, Sidney D'Mello, Ryan S. Baker, Valerie Shute, Matthew Ventura, Lubin Wang, and Weinan Zhao. 2016. Detecting student emotions in computer-enabled classrooms. In Proceedings of the 25th International Joint Conference on Artificial Intelligence. 4125--4129."},{"key":"e_1_3_2_2_6_1","volume-title":"Heffernan","author":"Botelho Anthony F.","year":"2017","unstructured":"Anthony F. Botelho , Ryan S. Baker , and Neil T . Heffernan . 2017 . Improving sensor-free affect detection using deep learning. In Proceedings of the International Conference on Artificial Intelligence in Education. Springer , Cham, 40--51. Anthony F. Botelho, Ryan S. Baker, and Neil T. Heffernan. 2017. Improving sensor-free affect detection using deep learning. In Proceedings of the International Conference on Artificial Intelligence in Education. Springer, Cham, 40--51."},{"key":"e_1_3_2_2_7_1","volume-title":"Maria Vald\u00e9s Hern\u00e1ndez, Joanna Wardlaw, and Daniel Rueckert.","author":"Bowles Christopher","year":"2018","unstructured":"Christopher Bowles , Liang Chen , Ricardo Guerrero , Paul Bentley , Roger Gunn , Alexander Hammers , David Alexander Dickie , Maria Vald\u00e9s Hern\u00e1ndez, Joanna Wardlaw, and Daniel Rueckert. 2018 . GAN augmentation: Augmenting training data using generative adversarial networks. arXiv preprint arXiv:1810.10863. Christopher Bowles, Liang Chen, Ricardo Guerrero, Paul Bentley, Roger Gunn, Alexander Hammers, David Alexander Dickie, Maria Vald\u00e9s Hern\u00e1ndez, Joanna Wardlaw, and Daniel Rueckert. 2018. GAN augmentation: Augmenting training data using generative adversarial networks. arXiv preprint arXiv:1810.10863."},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242969.3264986"},{"key":"e_1_3_2_2_9_1","volume-title":"Artif. Intell. Res. 16","author":"Chawla Nitesh V.","year":"2002","unstructured":"Nitesh V. Chawla , Kevin W. Bowyer , Lawrence O. Hall , and W. Philip Kegelmeyer . 2002. SMOTE: Synthetic minority over-sampling technique. J . Artif. Intell. Res. 16 , ( 2002 ). 321--357. Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. 2002. SMOTE: Synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, (2002). 321--357."},{"key":"e_1_3_2_2_10_1","volume-title":"A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 1","author":"Cohen Jacob","year":"1960","unstructured":"Jacob Cohen . 1960. A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 1 ( 1960 ). 37--46. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 1 (1960). 37--46."},{"key":"e_1_3_2_2_11_1","volume-title":"A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. J. Educ. Psychol. 105, 4","author":"D'Mello Sidney","year":"2013","unstructured":"Sidney D'Mello . 2013. A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. J. Educ. Psychol. 105, 4 ( 2013 ). 1082--1099. Sidney D'Mello. 2013. A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. J. Educ. Psychol. 105, 4 (2013). 1082--1099."},{"key":"e_1_3_2_2_12_1","volume-title":"A review and meta-analysis of multimodal affect detection systems. ACM Comput. Surv. 47, 3","author":"D'Mello Sidney","year":"2014","unstructured":"Sidney D'Mello and Jacqueline Kory . 2014. A review and meta-analysis of multimodal affect detection systems. ACM Comput. Surv. 47, 3 ( 2014 ). 43--79. Sidney D'Mello and Jacqueline Kory. 2014. A review and meta-analysis of multimodal affect detection systems. ACM Comput. Surv. 47, 3 (2014). 43--79."},{"key":"e_1_3_2_2_13_1","unstructured":"Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2670--2680.  Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2670--2680."},{"key":"e_1_3_2_2_14_1","volume-title":"International Conference of the Florida Artificial Intelligence Research Society. 438--443","author":"Grafsgaard Joseph","year":"2012","unstructured":"Joseph Grafsgaard , Kristy Boyer , Eric Wiebe , and James Lester . 2012 . Analyzing posture and affect in task-oriented tutoring . In International Conference of the Florida Artificial Intelligence Research Society. 438--443 . Joseph Grafsgaard, Kristy Boyer, Eric Wiebe, and James Lester. 2012. Analyzing posture and affect in task-oriented tutoring. In International Conference of the Florida Artificial Intelligence Research Society. 438--443."},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2663204.2663264"},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2019.8925538"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-23204-7_13"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-52237-7_19"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273601"},{"key":"e_1_3_2_2_20_1","volume-title":"Proceedings of the 21st International Conference on Multimodal Interaction. 35--39","author":"Lee Min Kyu","year":"2019","unstructured":"Min Kyu Lee , Dong Yoon Choi , and Byung Cheol Song . 2019 . Facial expression recognition via relation-based conditional generative adversarial network . In Proceedings of the 21st International Conference on Multimodal Interaction. 35--39 . Min Kyu Lee, Dong Yoon Choi, and Byung Cheol Song. 2019. Facial expression recognition via relation-based conditional generative adversarial network. In Proceedings of the 21st International Conference on Multimodal Interaction. 35--39."},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2017.2679538"},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00593"},{"key":"e_1_3_2_2_23_1","unstructured":"Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. In arXiv preprint arXiv:1411.1784.  Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. In arXiv preprint arXiv:1411.1784."},{"key":"e_1_3_2_2_24_1","volume-title":"Rodrigo","author":"Ocumpaugh Jaclyn","year":"2015","unstructured":"Jaclyn Ocumpaugh , Ryan S. Baker , and Mercedes T . Rodrigo . 2015 . Baker Rodrigo Ocumpaugh Monitoring Protocol (BROMP) 2.0 Technical and Training Manual . Jaclyn Ocumpaugh, Ryan S. Baker, and Mercedes T. Rodrigo. 2015. Baker Rodrigo Ocumpaugh Monitoring Protocol (BROMP) 2.0 Technical and Training Manual."},{"key":"e_1_3_2_2_25_1","first-page":"1","article-title":"2013. Considering alternate futures to classify off-task behavior as emotion self-regulation: A supervised learning approach","volume":"5","author":"Sabourin Jennifer L.","year":"2013","unstructured":"Jennifer L. Sabourin , Jonathan P. Rowe , Bradford W. Mott , and James C. Lester . 2013. Considering alternate futures to classify off-task behavior as emotion self-regulation: A supervised learning approach . J. Educ. Data Min. 5 , 1 ( 2013 ). 9--38. Jennifer L. Sabourin, Jonathan P. Rowe, Bradford W. Mott, and James C. Lester. 2013. Considering alternate futures to classify off-task behavior as emotion self-regulation: A supervised learning approach. J. Educ. Data Min. 5, 1 (2013). 9--38.","journal-title":"J. Educ. Data Min."},{"key":"e_1_3_2_2_26_1","unstructured":"Ruslan Salakhutdinov and Andriy Mnih. 2008. Probabilistic matrix factorization. Adv. Neural Inf. Process. Syst. (2008). 1257--1264.  Ruslan Salakhutdinov and Andriy Mnih. 2008. Probabilistic matrix factorization. Adv. Neural Inf. Process. Syst. (2008). 1257--1264."},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2017.8257992"},{"key":"e_1_3_2_2_28_1","unstructured":"Kihyuk Sohn Xinchen Yan and Honglak Lee. 2015. Learning structured output representation using deep conditional generative models. Adv. Neural Inf. Process. Syst. (2015). 3483--3491.  Kihyuk Sohn Xinchen Yan and Honglak Lee. 2015. Learning structured output representation using deep conditional generative models. Adv. Neural Inf. Process. Syst. (2015). 3483--3491."},{"key":"e_1_3_2_2_29_1","first-page":"2","article-title":"2018. Special issue on the Generalized Intelligent Framework for Tutoring (GIFT): Creating a stable and flexible platform for innovations","volume":"28","author":"Sottilare Robert A.","year":"2018","unstructured":"Robert A. Sottilare , Ryan S. Baker , Arthur C. Graesser , and James C. Lester . 2018. Special issue on the Generalized Intelligent Framework for Tutoring (GIFT): Creating a stable and flexible platform for innovations in AIED Research. Int. J. Artif. Intell. Educ. 28 , 2 ( 2018 ). Robert A. Sottilare, Ryan S. Baker, Arthur C. Graesser, and James C. Lester. 2018. Special issue on the Generalized Intelligent Framework for Tutoring (GIFT): Creating a stable and flexible platform for innovations in AIED Research. Int. J. Artif. Intell. Educ. 28, 2 (2018).","journal-title":"AIED Research. Int. J. Artif. Intell. Educ."},{"key":"e_1_3_2_2_30_1","volume-title":"Proceedings of the Interservice\/Industry Training, Simulation and Education Conference. 1--11","author":"Spain Randall","year":"2019","unstructured":"Randall Spain , Jonathan Rowe , Benjamin Goldberg , Robert Pokorny , and James Lester . 2019 . Enhancing learning outcomes through adaptive remediation with GIFT . In Proceedings of the Interservice\/Industry Training, Simulation and Education Conference. 1--11 . Randall Spain, Jonathan Rowe, Benjamin Goldberg, Robert Pokorny, and James Lester. 2019. Enhancing learning outcomes through adaptive remediation with GIFT. In Proceedings of the Interservice\/Industry Training, Simulation and Education Conference. 1--11."},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242969.3264984"},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"crossref","unstructured":"Kim-Han Thung Pew-Thian Yap and Dinggang Shen. 2017. Multi-stage diagnosis of Alzheimer's disease with incomplete multimodal data via multi-task deep learning. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. 160--168.  Kim-Han Thung Pew-Thian Yap and Dinggang Shen. 2017. Multi-stage diagnosis of Alzheimer's disease with incomplete multimodal data via multi-task deep learning. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. 160--168.","DOI":"10.1007\/978-3-319-67558-9_19"},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818346.2820737"},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3340555.3353739"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242969.3264981"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-37734-2_6"},{"key":"e_1_3_2_2_37_1","volume-title":"Proceedings of the 35th International Conference on Machine Learning.","author":"Yoon Jinsung","year":"2018","unstructured":"Jinsung Yoon , James Jordon , and Mihaela Van Der Schaar . 2018 . GAIN: Missing data imputation using generative adversarial nets . In Proceedings of the 35th International Conference on Machine Learning. Jinsung Yoon, James Jordon, and Mihaela Van Der Schaar. 2018. GAIN: Missing data imputation using generative adversarial nets. In Proceedings of the 35th International Conference on Machine Learning."},{"volume-title":"Proceedings of the IEEE International Conference on Computer Vision. 5907--5915","author":"Zhang Han","key":"e_1_3_2_2_38_1","unstructured":"Han Zhang , Tao Xu , Hongsheng Li , Shaoting Zhang , Xiaogang Wang , Xiaolei Huang , and Dimitris N. Metaxas . 2017. StackGAN++: Realistic image synthesis with stacked generative adversarial networks . In Proceedings of the IEEE International Conference on Computer Vision. 5907--5915 . Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N. Metaxas. 2017. StackGAN++: Realistic image synthesis with stacked generative adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision. 5907--5915."}],"event":{"name":"ICMI '20: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"],"location":"Virtual Event Netherlands","acronym":"ICMI '20"},"container-title":["Proceedings of the 2020 International Conference on Multimodal Interaction"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3382507.3418892","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3382507.3418892","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3382507.3418892","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:49Z","timestamp":1750197769000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3382507.3418892"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,21]]},"references-count":38,"alternative-id":["10.1145\/3382507.3418892","10.1145\/3382507"],"URL":"https:\/\/doi.org\/10.1145\/3382507.3418892","relation":{},"subject":[],"published":{"date-parts":[[2020,10,21]]},"assertion":[{"value":"2020-10-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}