{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T23:07:11Z","timestamp":1784156831609,"version":"3.55.0"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2022,9,7]],"date-time":"2022-09-07T00:00:00Z","timestamp":1662508800000},"content-version":"vor","delay-in-days":1,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["1842693"],"award-info":[{"award-number":["1842693"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2022,9,6]]},"abstract":"<jats:p>When in front of a classroom, a skilled teacher can read the room, identifying when students are engaged, frustrated, distracted, etc. In recent years we have seen significant changes in the traditional classroom, with virtual classes becoming a normal learning environment. Reasons for this change are the increased popularity of Massive Open Online Courses (MOOCs) and the disruptions imposed by the ongoing COVID-19 pandemic. However, it is difficult for teachers to read the room in these virtual classrooms, and researchers have begun to look at using sensors to provide feedback to help inform teaching practices. The study presented here sought to ground classroom sensor data in the form of electrodermal activities (EDA) captured using a wrist-worn sensing platform (Empatica E4), with observations about students' emotional engagement in the class. We collected a dataset from eleven students over eight lectures in college-level computer science classes. We trained human annotators who provided ground truth information about student engagement based on in-class observations. Inspired by related work in the field, we implemented an automated data analysis framework, which we used to explore momentary assessments of student engagement in classrooms. Our findings surprised us because we found no significant correlation between the sensor data and our trained observers' data. In this paper, we present our study and framework for automated engagement assessment, and report on our findings that indicate some of the challenges in deploying current technology for real-world, automated momentary assessment of student engagement in the classroom. We offer reflections on our findings and discuss ways forward toward an automated reading the room approach.<\/jats:p>","DOI":"10.1145\/3550328","type":"journal-article","created":{"date-parts":[[2022,9,7]],"date-time":"2022-09-07T14:54:27Z","timestamp":1662562467000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Reading the Room"],"prefix":"10.1145","volume":"6","author":[{"given":"Betsy","family":"DiSalvo","sequence":"first","affiliation":[{"name":"School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dheeraj","family":"Bandaru","sequence":"additional","affiliation":[{"name":"School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qiaosi","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hong","family":"Li","sequence":"additional","affiliation":[{"name":"School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Pl\u00f6tz","sequence":"additional","affiliation":[{"name":"School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,9,7]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Sasirekha Anbusegaran. 2021. Unobtrusive Assessment Of Student Engagement Levels In Online Classroom Environment Using Emotion Analysis. (2021)."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1002\/pits.20303"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2020.02.075"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10639-019-10004-6"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-019-09254-3"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1186\/s41239-020-00187-1"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDMW.2011.178"},{"key":"e_1_2_1_8_1","volume-title":"Electrodermal activity","author":"Boucsein Wolfram","unstructured":"Wolfram Boucsein. 2012. Electrodermal activity. Springer Science & Business Media."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/0005-7916(94)90063-9"},{"key":"e_1_2_1_10_1","volume-title":"Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The journal of the learning sciences 2, 2","author":"Brown Ann L","year":"1992","unstructured":"Ann L Brown. 1992. Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The journal of the learning sciences 2, 2 (1992), 141--178."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2499621"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3003397.3003409"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2012.110"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/SMARTCOMP.2017.7947008"},{"key":"e_1_2_1_15_1","volume-title":"Jaime Guixeres Provinciale, Jose M Aus\u00edn, and Mariano Alca\u00f1iz Raya.","author":"Granero Adrian Colomer","year":"2016","unstructured":"Adrian Colomer Granero, F\u00e9lix Fuentes-Hurtado, Valery Naranjo Ornedo, Jaime Guixeres Provinciale, Jose M Aus\u00edn, and Mariano Alca\u00f1iz Raya. 2016. A comparison of physiological signal analysis techniques and classifiers for automatic emotional evaluation of audiovisual contents. Frontiers in computational neuroscience 10 (2016), 74."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/79.911197"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2883851.2883931"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.23919\/CISTI.2017.7975712"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3242969.3264993"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3264913"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3329189.3329216"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0032674"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2682899"},{"key":"e_1_2_1_24_1","volume-title":"James Van Haneghan","author":"Duraisamy Prakash","year":"2019","unstructured":"Prakash Duraisamy, James Van Haneghan, William Blackwell, Steve Jackson, G Murugesan, and KS Tamilselvan. 2019. Classroom engagement evaluation using computer vision techniques. In Pattern Recognition and Tracking XXX, Vol. 10995. International Society for Optics and Photonics, 109950R."},{"key":"e_1_2_1_25_1","volume-title":"International Conference on Artificial Intelligence in Education","author":"Emerson Andrew","unstructured":"Andrew Emerson, Nathan Henderson, Jonathan Rowe, Wookhee Min, Seung Lee, James Minogue, and James Lester. 2020. Investigating Visitor Engagement in Interactive Science Museum Exhibits with Multimodal Bayesian Hierarchical Models. In International Conference on Artificial Intelligence in Education. Springer, 165--176."},{"key":"e_1_2_1_26_1","volume-title":"Handbook of research on student engagement","author":"Fredricks Jennifer A","unstructured":"Jennifer A Fredricks and Wendy McColskey. 2012. The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. In Handbook of research on student engagement. Springer, 763--782."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0205828"},{"key":"e_1_2_1_28_1","first-page":"1","article-title":"n-Gage: Predicting in-class Emotional, Behavioural and Cognitive Engagement in the Wild","volume":"4","author":"Gao Nan","year":"2020","unstructured":"Nan Gao, Wei Shao, Mohammad Saiedur Rahaman, and Flora D Salim. 2020. n-Gage: Predicting in-class Emotional, Behavioural and Cognitive Engagement in the Wild. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1--26.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.4108\/icst.mobihealth.2014.257418"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3267305.3267693"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3314400"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397316"},{"key":"e_1_2_1_33_1","volume-title":"Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems. \"O'Reilly Media","author":"G\u00e9ron Aur\u00e9lien","unstructured":"Aur\u00e9lien G\u00e9ron. 2019. Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems. \"O'Reilly Media, Inc.\"."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2019.102384"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1037\/e572172013-042"},{"key":"e_1_2_1_36_1","first-page":"797","article-title":"cvxEDA: A convex optimization approach to electrodermal activity processing","volume":"63","author":"Greco Alberto","year":"2015","unstructured":"Alberto Greco, Gaetano Valenza, Antonio Lanata, Enzo Pasquale Scilingo, and Luca Citi. 2015. cvxEDA: A convex optimization approach to electrodermal activity processing. IEEE Transactions on Biomedical Engineering 63, 4 (2015), 797--804.","journal-title":"IEEE Transactions on Biomedical Engineering"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3449086"},{"key":"e_1_2_1_38_1","volume-title":"Proc. Joint Int. Conf. Artificial Intelligence (IJCAI).","author":"Hammerla Nils Y","year":"2016","unstructured":"Nils Y Hammerla, Shane Halloran, and Thomas Pl\u00f6tz. 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables. In Proc. Joint Int. Conf. Artificial Intelligence (IJCAI)."},{"key":"e_1_2_1_39_1","volume-title":"Mixed emotions: Teachers' perceptions of their interactions with students. Teaching and teacher education 16, 8","author":"Hargreaves Andy","year":"2000","unstructured":"Andy Hargreaves. 2000. Mixed emotions: Teachers' perceptions of their interactions with students. Teaching and teacher education 16, 8 (2000), 811--826."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025669"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISWC.1998.729528"},{"key":"e_1_2_1_42_1","volume-title":"Accessing emotion patterns from affective interactions using electrodermal activity. In 2013 humaine association conference on affective computing and intelligent interaction","author":"Henriques Rui","unstructured":"Rui Henriques, Ana Paiva, and Claudia Antunes. 2013. Accessing emotion patterns from affective interactions using electrodermal activity. In 2013 humaine association conference on affective computing and intelligent interaction. IEEE, 43--48."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2632048.2636065"},{"key":"e_1_2_1_44_1","volume-title":"An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics","author":"Richard Landis J","year":"1977","unstructured":"J Richard Landis and Gary G Koch. 1977. An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics (1977), 363--374."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.14786\/flr.v7i2.403"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979210"},{"key":"e_1_2_1_47_1","volume-title":"Motivational techniques of expert human tutors: Lessons for the design of computer-based tutors. Computers as cognitive tools 1993","author":"Lepper Mark R","year":"1993","unstructured":"Mark R Lepper, Maria Woolverton, Donna L Mumme, and J Gurtner. 1993. Motivational techniques of expert human tutors: Lessons for the design of computer-based tutors. Computers as cognitive tools 1993 (1993), 75--105."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.iheduc.2020.100727"},{"key":"e_1_2_1_49_1","volume-title":"A Learning Attention Monitoring System via Photoplethysmogram Using Wearable Wrist Devices. Artificial Intelligence Supported Educational Technologies","author":"Li Qing","year":"2020","unstructured":"Qing Li, Yuan Ren, Tianyu Wei, Chengcheng Wang, Zhi Liu, and Jieyu Yue. 2020. A Learning Attention Monitoring System via Photoplethysmogram Using Wearable Wrist Devices. Artificial Intelligence Supported Educational Technologies (2020), 133."},{"key":"e_1_2_1_50_1","volume-title":"Experiencing SAX: a novel symbolic representation of time series. Data Mining and knowledge discovery 15, 2","author":"Lin Jessica","year":"2007","unstructured":"Jessica Lin, Eamonn Keogh, Li Wei, and Stefano Lonardi. 2007. Experiencing SAX: a novel symbolic representation of time series. Data Mining and knowledge discovery 15, 2 (2007), 107--144."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1469-8986.1971.tb00501.x"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10648-006-9032-1"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3303772.3303821"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10639-020-10273-6"},{"key":"e_1_2_1_55_1","volume-title":"Reimagining higher education during and post-COVID-19: Challenges and opportunities. Journal of Adult and Continuing Education","author":"Neuwirth Lorenz S","year":"2020","unstructured":"Lorenz S Neuwirth, Svetlana Jovi\u0107, and B Runi Mukherji. 2020. Reimagining higher education during and post-COVID-19: Challenges and opportunities. Journal of Adult and Continuing Education (2020), 1477971420947738."},{"key":"e_1_2_1_56_1","unstructured":"Jaclyn Ocumpaugh. 2015. Baker Rodrigo Ocumpaugh monitoring protocol (BROMP) 2.0 technical and training manual. Technical Report. Technical Report. New York NY: Teachers College Columbia University ...."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.3390\/s16010115"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2003.817122"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3027385.3027430"},{"key":"e_1_2_1_60_1","volume-title":"Emotion in education","author":"Pekrun Reinhard","unstructured":"Reinhard Pekrun, Anne C Frenzel, Thomas Goetz, and Raymond P Perry. 2007. The control-value theory of achievement emotions: An integrative approach to emotions in education. In Emotion in education. Elsevier, 13--36."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.3102\/00028312021003487"},{"key":"e_1_2_1_62_1","volume-title":"Adaptation of tutoring to students' emotions in emotionally intelligent tutoring systems. In 2013 second international conference on e-learning and E-technologies in education (ICEEE)","author":"Petrovica Sintija","unstructured":"Sintija Petrovica. 2013. Adaptation of tutoring to students' emotions in emotionally intelligent tutoring systems. In 2013 second international conference on e-learning and E-technologies in education (ICEEE). IEEE, 131--136."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-19773-9_37"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-91464-0_15"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.5555\/265013"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3459666"},{"key":"e_1_2_1_67_1","volume-title":"A wearable sensor for unobtrusive, long-term assessment of electrodermal activity","author":"Poh Ming-Zher","year":"2010","unstructured":"Ming-Zher Poh, Nicholas C Swenson, and Rosalind W Picard. 2010. A wearable sensor for unobtrusive, long-term assessment of electrodermal activity. IEEE transactions on Biomedical engineering 57, 5 (2010), 1243--1252."},{"key":"e_1_2_1_68_1","volume-title":"AIED Workshops.","author":"Rowe Jonathan P","year":"2015","unstructured":"Jonathan P Rowe, Bradford W Mott, and James C Lester. 2015. Opportunities and Challenges in Generalizable Sensor-Based Affect Recognition for Learning.. In AIED Workshops."},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijedro.2021.100079"},{"key":"e_1_2_1_70_1","doi-asserted-by":"crossref","unstructured":"Gale M Sinatra Benjamin C Heddy and Doug Lombardi. 2015. The challenges of defining and measuring student engagement in science. 13 pages.","DOI":"10.1080\/00461520.2014.1002924"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3139513.3139514"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1080\/00958964.2020.1727404"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386527.3405917"},{"key":"e_1_2_1_74_1","doi-asserted-by":"crossref","unstructured":"James E Willis III and Viktoria Alane Strunk. 2015. Ethical responsibilities of preserving academicians in an age of mechanized learning: Balancing the demands of educating at capacity and preserving human interactivity. In Rethinking machine ethics in the age of ubiquitous technology. IGI Global 166--195.","DOI":"10.4018\/978-1-4666-8592-5.ch009"},{"key":"e_1_2_1_75_1","volume-title":"Intelligent Tutor Systems Addressing Student Disengagement: Adding Formative Reappraisal to Enhance Engagement and Learning","author":"Xeriland Timothy J","unstructured":"Timothy J Xeriland. 2018. Intelligent Tutor Systems Addressing Student Disengagement: Adding Formative Reappraisal to Enhance Engagement and Learning. Michigan State University."}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550328","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3550328","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3550328","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:42:31Z","timestamp":1752468151000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550328"}},"subtitle":["Automated, Momentary Assessment of Student Engagement in the Classroom: Are We There Yet?"],"short-title":[],"issued":{"date-parts":[[2022,9,6]]},"references-count":75,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,9,6]]}},"alternative-id":["10.1145\/3550328"],"URL":"https:\/\/doi.org\/10.1145\/3550328","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,6]]},"assertion":[{"value":"2022-09-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}