{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T16:51:28Z","timestamp":1789750288208,"version":"4.0.1"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"EICS","license":[{"start":{"date-parts":[[2023,6,14]],"date-time":"2023-06-14T00:00:00Z","timestamp":1686700800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2023,6,14]]},"abstract":"<jats:p>Developing learning analytics dashboards (LADs) is a growing research interest as online learning tools have become more accessible in K-12 and higher education settings. This paper reports our multimodal classroom engagement data analysis and dashboard design process and the resulting engagement dashboard. Our work stems from the importance of monitoring classroom engagement, which refers to students' active physical and cognitive involvement in learning that influences their motivation and success in a given course. To monitor this vital facade of learning, we developed an engagement dashboard using an iterative and user-centered process. We first created a multimodal machine learning model that utilizes face and pose features obtained from recent deep learning models. Then, we created a dashboard where users can view their engagement over time and discover their learning\/teaching patterns. Finally, we conducted user studies with undergraduate and graduate-level participants to obtain feedback on our dashboard design. Our paper makes three contributions by (1) presenting a student-centric, open-source dashboard, (2) demonstrating a baseline architecture for engagement analysis using our open-access data, and (3) presenting user insights and design takeaways to inspire future LADs. We expect our research to guide the development of tools for novice teacher education, student self-evaluation, and engagement evaluation in crowded classrooms.<\/jats:p>","DOI":"10.1145\/3593240","type":"journal-article","created":{"date-parts":[[2023,6,19]],"date-time":"2023-06-19T14:10:41Z","timestamp":1687183841000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Developing a Multimodal Classroom Engagement Analysis Dashboard for Higher-Education"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4415-0516","authenticated-orcid":false,"given":"Alpay","family":"Sabuncuoglu","sequence":"first","affiliation":[{"name":"UNVEST R&amp;D Center, Istanbul, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1524-1646","authenticated-orcid":false,"given":"T. Metin","family":"Sezgin","sequence":"additional","affiliation":[{"name":"Koc University, Istanbul, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,6,19]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.471"},{"key":"e_1_2_1_2_1","volume-title":"Privacy-Preserving Data Aggregation at Scale","author":"Applebaum Benny","unstructured":"Benny Applebaum , Haakon Ringberg , Michael J. Freedman , Matthew Caesar , and Jennifer Rexford. 2010. Collaborative , Privacy-Preserving Data Aggregation at Scale . In Privacy Enhancing Technologies, Mikhail J. Atallah and Nicholas J. Hopper (Eds.). Springer Berlin Heidelberg , Berlin, Heidelberg , 56--74. Benny Applebaum, Haakon Ringberg, Michael J. Freedman, Matthew Caesar, and Jennifer Rexford. 2010. Collaborative, Privacy-Preserving Data Aggregation at Scale. In Privacy Enhancing Technologies, Mikhail J. Atallah and Nicholas J. Hopper (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 56--74."},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the Seventh International Learning Analytics & Knowledge Conference. ACM, Vancouver British Columbia Canada, 66--69","author":"Kimberly","unstructured":"Kimberly E. Arnold and Niall Sclater. 2017. Student perceptions of their privacy in leaning analytics applications . In Proceedings of the Seventh International Learning Analytics & Knowledge Conference. ACM, Vancouver British Columbia Canada, 66--69 . https:\/\/doi.org\/10.1145\/3027385.3027392 10.1145\/3027385.3027392 Kimberly E. Arnold and Niall Sclater. 2017. Student perceptions of their privacy in leaning analytics applications. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference. ACM, Vancouver British Columbia Canada, 66--69. https:\/\/doi.org\/10.1145\/3027385.3027392"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2015.7284869"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1080\/2159676X.2019.1628806"},{"key":"e_1_2_1_8_1","unstructured":"Zhe Cao Gines Hidalgo Tomas Simon Shih-En Wei and Yaser Sheikh. 2019. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. arXiv:1812.08008 [cs] http:\/\/arxiv.org\/abs\/1812.08008  Zhe Cao Gines Hidalgo Tomas Simon Shih-En Wei and Yaser Sheikh. 2019. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. arXiv:1812.08008 [cs] http:\/\/arxiv.org\/abs\/1812.08008"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2883851.2883893"},{"key":"e_1_2_1_10_1","volume-title":"RMPE: Regional Multi-person Pose Estimation. arXiv:1612.00137 [cs] http:\/\/arxiv.org\/abs\/1612.00137 version: 5.","author":"Fang Hao-Shu","year":"2018","unstructured":"Hao-Shu Fang , Shuqin Xie , Yu-Wing Tai , and Cewu Lu . 2018 . RMPE: Regional Multi-person Pose Estimation. arXiv:1612.00137 [cs] http:\/\/arxiv.org\/abs\/1612.00137 version: 5. Hao-Shu Fang, Shuqin Xie, Yu-Wing Tai, and Cewu Lu. 2018. RMPE: Regional Multi-person Pose Estimation. arXiv:1612.00137 [cs] http:\/\/arxiv.org\/abs\/1612.00137 version: 5."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3506860.3506895"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.3102\/00346543074001059"},{"key":"e_1_2_1_13_1","volume-title":"Wei Shao, and Flora D Salim.","author":"Gao Nan","year":"2021","unstructured":"Nan Gao , Mohammad Saiedur Rahaman , Wei Shao, and Flora D Salim. 2021 . Investigating the Reliability of Self- Report Data in the Wild : The Quest for Ground Truth. In Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers (UbiComp '21). Association for Computing Machinery , New York, NY, USA, 237--242. https: \/\/doi.org\/10.1145\/3460418.3479338 10.1145\/3460418.3479338 Nan Gao, Mohammad Saiedur Rahaman, Wei Shao, and Flora D Salim. 2021. Investigating the Reliability of Self- Report Data in the Wild: The Quest for Ground Truth. In Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers (UbiComp '21). Association for Computing Machinery, New York, NY, USA, 237--242. https: \/\/doi.org\/10.1145\/3460418.3479338"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2018.3620965"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1080\/1475939X.2013.802995"},{"key":"e_1_2_1_16_1","volume-title":"Introduction to Sensitivity Analysis","author":"Iooss Bertrand","unstructured":"Bertrand Iooss and Andrea Saltelli . 2017. Introduction to Sensitivity Analysis . Springer International Publishing . 1103--1122 pages. https:\/\/doi.org\/10.1007\/978--3--319--12385--1_31 10.1007\/978--3--319--12385--1_31 Bertrand Iooss and Andrea Saltelli. 2017. Introduction to Sensitivity Analysis. Springer International Publishing. 1103--1122 pages. https:\/\/doi.org\/10.1007\/978--3--319--12385--1_31"},{"key":"#cr-split#-e_1_2_1_17_1.1","unstructured":"Yury Kartynnik Artsiom Ablavatski Ivan Grishchenko and Matthias Grundmann. 2019. Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs. https:\/\/doi.org\/10.48550\/ARXIV.1907.06724 10.48550\/ARXIV.1907.06724"},{"key":"#cr-split#-e_1_2_1_17_1.2","unstructured":"Yury Kartynnik Artsiom Ablavatski Ivan Grishchenko and Matthias Grundmann. 2019. Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs. https:\/\/doi.org\/10.48550\/ARXIV.1907.06724"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3375462.3375506"},{"key":"e_1_2_1_19_1","unstructured":"Susie Lu and Elijah Meeks. 2022. Viz Palette. https:\/\/projects.susielu.com\/viz-palette  Susie Lu and Elijah Meeks. 2022. Viz Palette. https:\/\/projects.susielu.com\/viz-palette"},{"key":"e_1_2_1_20_1","volume-title":"Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition. (July","author":"Luo Cheng","year":"2022","unstructured":"Cheng Luo , Siyang Song , Weicheng Xie , Linlin Shen , and Hatice Gunes . 2022. Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition. (July 2022 ), 1239--1246. https:\/\/doi.org\/10.24963\/ ijcai.2022\/173 Cheng Luo, Siyang Song, Weicheng Xie, Linlin Shen, and Hatice Gunes. 2022. Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition. (July 2022), 1239--1246. https:\/\/doi.org\/10.24963\/ ijcai.2022\/173"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723576"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TLT.2019.2916802"},{"key":"e_1_2_1_23_1","doi-asserted-by":"crossref","unstructured":"Kasia Muldner Michael Wixon Dovan Rai Winslow Burleson Beverly Woolf and Ivon Arroyo. 2015. Exploring the Impact of a Learning Dashboard on Student Affect Cristina Conati Neil Heffernan Antonija Mitrovic and M. Felisa  Kasia Muldner Michael Wixon Dovan Rai Winslow Burleson Beverly Woolf and Ivon Arroyo. 2015. Exploring the Impact of a Learning Dashboard on Student Affect Cristina Conati Neil Heffernan Antonija Mitrovic and M. Felisa","DOI":"10.1007\/978-3-319-19773-9_31"},{"key":"e_1_2_1_24_1","unstructured":"Tanya Nazaretsky Carmel Bar Michal Walter and Giora Alexandron. 2018. Empowering Teachers with AI: Co- Designing a Learning Analytics Tool for Personalized Instruction in the Science Classroom. (2018) 16.  Tanya Nazaretsky Carmel Bar Michal Walter and Giora Alexandron. 2018. Empowering Teachers with AI: Co- Designing a Learning Analytics Tool for Personalized Instruction in the Science Classroom. (2018) 16."},{"key":"e_1_2_1_25_1","unstructured":"Xuecheng Nie Jiashi Feng Junliang Xing and Shuicheng Yan. 2017. Generative Partition Networks for Multi-Person Pose Estimation. arXiv:1705.07422 [cs] http:\/\/arxiv.org\/abs\/1705.07422 version: 2.  Xuecheng Nie Jiashi Feng Junliang Xing and Shuicheng Yan. 2017. Generative Partition Networks for Multi-Person Pose Estimation. arXiv:1705.07422 [cs] http:\/\/arxiv.org\/abs\/1705.07422 version: 2."},{"key":"e_1_2_1_26_1","unstructured":"Xuecheng Nie Jianfeng Zhang Shuicheng Yan and Jiashi Feng. 2019. Single-Stage Multi-Person Pose Machines. arXiv:1908.09220 [cs] http:\/\/arxiv.org\/abs\/1908.09220 version: 1.  Xuecheng Nie Jianfeng Zhang Shuicheng Yan and Jiashi Feng. 2019. Single-Stage Multi-Person Pose Machines. arXiv:1908.09220 [cs] http:\/\/arxiv.org\/abs\/1908.09220 version: 1."},{"key":"e_1_2_1_27_1","unstructured":"Harsha Nori Samuel Jenkins Paul Koch and Rich Caruana. 2019. InterpretML: A Unified Framework for Machine Learning Interpretability. arXiv:1909.09223 [cs stat] http:\/\/arxiv.org\/abs\/1909.09223  Harsha Nori Samuel Jenkins Paul Koch and Rich Caruana. 2019. InterpretML: A Unified Framework for Machine Learning Interpretability. arXiv:1909.09223 [cs stat] http:\/\/arxiv.org\/abs\/1909.09223"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3506860.3506885"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723576.2723585"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3027385.3027406"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1186\/s41239-021-00284-9"},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Marco Tulio Ribeiro Sameer Singh and Carlos Guestrin. 2016. \"Why Should I Trust You?\": Explaining the Predictions of Any Classifier. arXiv:1602.04938 [cs stat] http:\/\/arxiv.org\/abs\/1602.04938  Marco Tulio Ribeiro Sameer Singh and Carlos Guestrin. 2016. \"Why Should I Trust You?\": Explaining the Predictions of Any Classifier. arXiv:1602.04938 [cs stat] http:\/\/arxiv.org\/abs\/1602.04938","DOI":"10.18653\/v1\/N16-3020"},{"key":"e_1_2_1_34_1","unstructured":"Alpay Sabuncuoglu and T. Metin Sezgin. 2023. Multimodal Group Activity Dataset for Classroom Engagement Level Prediction. arXiv:cs.HC\/2304.08901  Alpay Sabuncuoglu and T. Metin Sezgin. 2023. Multimodal Group Activity Dataset for Classroom Engagement Level Prediction. arXiv:cs.HC\/2304.08901"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TLT.2016.2599522"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvlc.2018.11.002"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491140.3528292"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3303772.3303796"},{"key":"e_1_2_1_40_1","volume-title":"DATUS: Dashboard Assessment Usability Model: A case study with student dashboards. mastersthesis.","author":"da Silva Antunes Rita Sofia","year":"2020","unstructured":"Rita Sofia da Silva Antunes . 2020 . DATUS: Dashboard Assessment Usability Model: A case study with student dashboards. mastersthesis. Rita Sofia da Silva Antunes. 2020. DATUS: Dashboard Assessment Usability Model: A case study with student dashboards. mastersthesis."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvcir.2021.103055"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1086\/663283"},{"key":"e_1_2_1_43_1","unstructured":"Tensorflow. 2023. movenet\/multipose\/lightning. https:\/\/tfhub.dev\/google\/movenet\/multipose\/lightning\/1  Tensorflow. 2023. movenet\/multipose\/lightning. https:\/\/tfhub.dev\/google\/movenet\/multipose\/lightning\/1"},{"key":"e_1_2_1_44_1","first-page":"19","article-title":"Recognizing Action Units for Facial Expression Analysis","volume":"23","author":"Kanade Takeo","year":"2001","unstructured":"Ying-li Tian, Takeo Kanade , and Jeffrey F Cohn . 2001 . Recognizing Action Units for Facial Expression Analysis . IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 23 , 2 (2001), 19 . Ying-li Tian, Takeo Kanade, and Jeffrey F Cohn. 2001. Recognizing Action Units for Facial Expression Analysis. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 23, 2 (2001), 19.","journal-title":"IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE"},{"key":"e_1_2_1_45_1","first-page":"6","article-title":"Learning dashboards: an overview and future research opportunities","volume":"18","author":"Verbert Katrien","year":"2014","unstructured":"Katrien Verbert , Sten Govaerts , Erik Duval , Jose Luis Santos , Frans Van Assche , Gonzalo Parra , and Joris Klerkx . 2014 . Learning dashboards: an overview and future research opportunities . Personal and Ubiquitous Computing 18 , 6 (Aug. 2014), 1499--1514. https:\/\/doi.org\/10.1007\/s00779-013-0751--2 10.1007\/s00779-013-0751--2 Katrien Verbert, Sten Govaerts, Erik Duval, Jose Luis Santos, Frans Van Assche, Gonzalo Parra, and Joris Klerkx. 2014. Learning dashboards: an overview and future research opportunities. Personal and Ubiquitous Computing 18, 6 (Aug. 2014), 1499--1514. https:\/\/doi.org\/10.1007\/s00779-013-0751--2","journal-title":"Personal and Ubiquitous Computing"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1037\/spq0000050"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2014.2316163"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3506860.3506900"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3506860.3506900"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3593240","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3593240","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T12:37:20Z","timestamp":1750163840000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3593240"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,14]]},"references-count":48,"journal-issue":{"issue":"EICS","published-print":{"date-parts":[[2023,6,14]]}},"alternative-id":["10.1145\/3593240"],"URL":"https:\/\/doi.org\/10.1145\/3593240","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,14]]},"assertion":[{"value":"2023-06-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}