{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T18:37:56Z","timestamp":1787769476852,"version":"build-2784847793"},"reference-count":87,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,8,2]],"date-time":"2024-08-02T00:00:00Z","timestamp":1722556800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Academy Professor Project Emotion AI","award":["336116 and 345122"],"award-info":[{"award-number":["336116 and 345122"]}]},{"name":"University of Oulu and Research Council of Finland Profi 7","award":["352788"],"award-info":[{"award-number":["352788"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62076122"],"award-info":[{"award-number":["62076122"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Finnish Cultural Foundation for North Ostrobothnia Regional Fund","award":["60231712"],"award-info":[{"award-number":["60231712"]}]},{"DOI":"10.13039\/501100008413","name":"Instrumentarium Foundation","doi-asserted-by":"crossref","award":["240016"],"award-info":[{"award-number":["240016"]}],"id":[{"id":"10.13039\/501100008413","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>Socially shared regulation plays a pivotal role in the success of collaborative learning. However, evaluating socially shared regulation of learning (SSRL) proves challenging due to the dynamic and infrequent cognitive and socio-emotional interactions, which constitute the focal point of SSRL. To address this challenge, this article gathers interdisciplinary researchers to establish a multimodal dataset with cognitive and socio-emotional interactions for SSRL study. Firstly, to induce cognitive and socio-emotional interactions, learning science researchers designed a special collaborative learning task with regulatory trigger events among triadic people for the SSRL study. Secondly, this dataset includes various modalities like video, Kinect data, audio, and physiological data (accelerometer, EDA, heart rate) from 81 high school students in 28 groups, offering a comprehensive view of the SSRL process. Thirdly, three-level verbal interaction annotations and nonverbal interactions including facial expression, eye gaze, gesture, and posture are provided, which could further contribute to interdisciplinary fields such as computer science, sociology, and education. In addition, comprehensive analysis verifies the dataset\u2019s effectiveness. As far as we know, this is the first multimodal dataset for studying SSRL among triadic group members.<\/jats:p>","DOI":"10.1145\/3658376","type":"journal-article","created":{"date-parts":[[2024,4,22]],"date-time":"2024-04-22T19:52:31Z","timestamp":1713815551000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Interactions for Socially Shared Regulation in Collaborative Learning: An Interdisciplinary Multimodal Dataset"],"prefix":"10.1145","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4824-2044","authenticated-orcid":false,"given":"Yante","family":"Li","sequence":"first","affiliation":[{"name":"University of Oulu, Oulu, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2157-0080","authenticated-orcid":false,"given":"Yang","family":"Liu","sequence":"additional","affiliation":[{"name":"University of Oulu, Oulu, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0759-9656","authenticated-orcid":false,"given":"Andy","family":"Nguyen","sequence":"additional","affiliation":[{"name":"University of Oulu, Oulu, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5884-8475","authenticated-orcid":false,"given":"Henglin","family":"Shi","sequence":"additional","affiliation":[{"name":"University of Oulu, Oulu, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6125-8182","authenticated-orcid":false,"given":"Eija","family":"Vuorenmaa","sequence":"additional","affiliation":[{"name":"University of Oulu, Oulu, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6223-3668","authenticated-orcid":false,"given":"Sanna","family":"J\u00e4rvel\u00e4","sequence":"additional","affiliation":[{"name":"University of Oulu, Oulu, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3694-206X","authenticated-orcid":false,"given":"Guoying","family":"Zhao","sequence":"additional","affiliation":[{"name":"University of Oulu, Oulu, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,8,2]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.5555\/1734243.1734245"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11409-014-9123-1"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503161.3548363"},{"key":"e_1_3_2_5_2","volume-title":"Collaborative Learning Techniques: A Handbook for College Faculty","author":"Barkley Elizabeth F.","year":"2014","unstructured":"Elizabeth F. Barkley, Claire H. Major, and K. Patricia Cross. 2014. Collaborative Learning Techniques: A Handbook for College Faculty. John Wiley & Sons."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2017.2740062"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/2993148.2993175"},{"key":"e_1_3_2_8_2","volume-title":"Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit","author":"Bird Steven","year":"2009","unstructured":"Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. O\u2019Reilly Media, Inc."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/INCoS.2015.88"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3136755.3136780"},{"key":"e_1_3_2_11_2","first-page":"28","volume-title":"Proceedings of the International Workshop on Machine Learning for Multimodal Interaction","author":"Carletta Jean","year":"2005","unstructured":"Jean Carletta, Simone Ashby, Sebastien Bourban, Mike Flynn, Mael Guillemot, Thomas Hain, Jaroslav Kadlec, Vasilis Karaiskos, Wessel Kraaij, Melissa Kronenthal, Guillaume Lathoud, Mike Lincoln, Agnes Lisowska, Iain McCowan, Wilfried Post, Dennis Reidsma, and Pierre Wellner. 2005. The AMI meeting corpus: A pre-announcement. In Proceedings of the International Workshop on Machine Learning for Multimodal Interaction. Springer, 28\u201339."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2013.2271914"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1075\/gs.7"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2007.07.003"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.177"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICALT58122.2023.00052"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.22318\/cscl2023.280974"},{"key":"e_1_3_2_18_2","first-page":"1580","volume-title":"Proceedings of the E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education","author":"DePree Jeff","year":"2009","unstructured":"Jeff DePree, Stanley Su, and Xuelian Xiao. 2009. Event-triggered rule processing in a collaborative learning environment. In Proceedings of the E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education. Association for the Advancement of Computing in Education (AACE), 1580\u20131586."},{"key":"e_1_3_2_19_2","first-page":"1","article-title":"What do you mean by collaborative learning?","author":"Dillenbourg Pierre","year":"1999","unstructured":"Pierre Dillenbourg. 1999. What do you mean by collaborative learning? In Collaborative-Learning: Cognitive and Computational Approaches, 1\u20139.","journal-title":"Collaborative-Learning: Cognitive and Computational Approaches"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2020.3003243"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cedpsych.2022.102050"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04216-4_16"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.5555\/3134162"},{"key":"e_1_3_2_24_2","first-page":"271","volume-title":"Proceedings of the Conference of the Australasian Society for Computers in Learning in Tertiary Education","author":"Fermelis Jan","year":"2007","unstructured":"Jan Fermelis, Richard Tucker, and Stuart Palmer. 2007. Online self and peer assessment in large, multi-campus, multi-cohort contexts. In Proceedings of the Conference of the Australasian Society for Computers in Learning in Tertiary Education, 271\u2013281."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2009.0142"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1080\/08957347.2016.1209207"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-39112-5_1"},{"key":"e_1_3_2_28_2","first-page":"83","volume-title":"Handbook of Self-Regulation of Learning and Performance","author":"Hadwin Allyson","year":"2018","unstructured":"Allyson Hadwin, Sanna J\u00e4rvel\u00e4, and Mariel Miller. 2018. Self-regulation, co-regulation, and shared regulation in collaborative learning environments. In Handbook of Self-Regulation of Learning and Performance. Routledge, 83\u2013106."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524383.3524388"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2019.04.012"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.lcsi.2020.100437"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1080\/00313831.2019.1623310"},{"key":"e_1_3_2_33_2","first-page":"4333","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference","author":"Jansen Michel-Pierre","year":"2020","unstructured":"Michel-Pierre Jansen, Khiet P. Truong, Dirk K. J. Heylen, and Deniece S. Nazareth. 2020. Introducing MULAI: A multimodal database of laughter during dyadic interactions. In Proceedings of the 12th Language Resources and Evaluation Conference, 4333\u20134342."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1111\/bjet.13325"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2023.107737"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2017.11.004"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2011.15"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.obhdp.2005.08.003"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0747-5632(02)00057-2"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11251-005-1279-2"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2020.2981446"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/FG52635.2021.9666975"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-48890-5_11"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/CADGRAPHICS.2015.19"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cedpsych.2010.09.001"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01049"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2023.3301209"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2022.3215918"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cedpsych.2017.01.009"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1037\/14669-000"},{"key":"e_1_3_2_51_2","volume-title":"Hand and Mind: What Gestures Reveal about Thought","author":"McNeill David","year":"1992","unstructured":"David McNeill. 1992. Hand and Mind: What Gestures Reveal about Thought. University of Chicago Press."},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/AMC.2006.1631755"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2018.2884461"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.lindif.2012.12.005"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2017.2740923"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3172944.3172969"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340555.3353721"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11412-021-09358-2"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1111\/bjet.13280"},{"key":"e_1_3_2_60_2","first-page":"527","volume-title":"Proceedings of International Conferences of Learning Sciences (ICLS)","author":"Nguyen Andy","year":"2022","unstructured":"Andy Nguyen, Sanna J\u00e4rvel\u00e4, Yang Wang, and Carolyn R\u00f3se. 2022. Exploring socially shared regulation with an AI deep learning approach using multimodal data. In Proceedings of International Conferences of Learning Sciences (ICLS), 527\u2013534."},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1080\/00220671.2016.1220359"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACVW52041.2021.00005"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-85729-994-9_8"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/568513.568514"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1080\/10494820.2021.1884886"},{"key":"e_1_3_2_66_2","volume-title":"Hierarchical Linear Models: Applications and Data Analysis Methods","author":"Raudenbush Stephen W.","year":"2002","unstructured":"Stephen W. Raudenbush and Anthony S. Bryk. 2002. Hierarchical Linear Models: Applications and Data Analysis Methods, Vol. 1. Sage."},{"key":"e_1_3_2_67_2","first-page":"1","article-title":"General data protection regulation","volume":"25","author":"Protection Regulation","year":"2018","unstructured":"Protection Regulation. 2018. General data protection regulation. Intouch 25 (2018), 1\u20135.","journal-title":"Intouch"},{"key":"e_1_3_2_68_2","volume-title":"Paper Presented at the International Conference on Educational Data Mining (EDM).","author":"Reilly Joseph M.","year":"2018","unstructured":"Joseph M. Reilly, Milan Ravenell, and Bertrand Schneider. 2018. Exploring collaboration using motion sensors and multi-modal learning analytics. In Paper Presented at the International Conference on Educational Data Mining (EDM). International Educational Data Mining Society."},{"issue":"1","key":"e_1_3_2_69_2","first-page":"355","article-title":"Complexity of social interactions in collaborative learning: The case of online database environment","volume":"6","author":"Rimor Rikki","year":"2010","unstructured":"Rikki Rimor, Yigal Rosen, and Kefaya Naser. 2010. Complexity of social interactions in collaborative learning: The case of online database environment. Interdisciplinary Journal of E-Learning and Learning Objects 6, 1 (2010), 355\u2013365.","journal-title":"Interdisciplinary Journal of E-Learning and Learning Objects"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2009.5459361"},{"key":"e_1_3_2_71_2","unstructured":"Michael S. Ryoo and Jake K. Aggarwal. 2010. UT-Interaction Dataset ICPR Contest on Semantic Description of Human Activities (SDHA). Retrieved from http:\/\/cvrc.ece.utexas.edu\/SDHA2010\/Human_Interaction.html"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12193-012-0101-0"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.86.2.420"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10758-021-09526-1"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2011.25"},{"issue":"1","key":"e_1_3_2_76_2","first-page":"40","article-title":"Supporting social interaction in an intelligent collaborative learning system","volume":"12","author":"Soller Amy","year":"2001","unstructured":"Amy Soller. 2001. Supporting social interaction in an intelligent collaborative learning system. International Journal of Artificial Intelligence in Education 12, 1 (2001), 40\u201362.","journal-title":"International Journal of Artificial Intelligence in Education"},{"key":"e_1_3_2_77_2","volume-title":"Age of Opportunity: Lessons from the New Science of Adolescence","author":"Steinberg Laurence D.","year":"2014","unstructured":"Laurence D. Steinberg. 2014. Age of Opportunity: Lessons from the New Science of Adolescence. Houghton Mifflin Harcourt."},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1515\/semi.2005.2005.156.1"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.2044-8260.1983.tb00597.x"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-020-00280-0"},{"key":"e_1_3_2_81_2","doi-asserted-by":"crossref","unstructured":"Gianmarco Ipinze Tutuianu Yang Liu Ari Alam\u00e4ki and Janne Kauttonen. 2023. Benchmarking deep facial expression recognition: An extensive protocol with balanced dataset in the wild. arXiv:2311.02910.","DOI":"10.1016\/j.engappai.2024.108983"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/T4E.2013.34"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2008.03.001"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1177\/10464964221137524"},{"key":"e_1_3_2_85_2","first-page":"56","volume-title":"Help Seeking in Academic Settings","author":"Webb Noreen M.","year":"2013","unstructured":"Noreen M. Webb, Marsha Ing, Nicole Kersting, and Kariane Mari Nemer. 2013. Help seeking in cooperative learning groups. In Help Seeking in Academic Settings. Routledge, 56\u201399."},{"key":"e_1_3_2_86_2","volume-title":"Proceedings of the Rethinking Learning in the Digital Age: Making the Learning Sciences Count, 13th International Conference of the Learning Sciences (ICLS)","volume":"1","author":"Xie Bin","year":"2018","unstructured":"Bin Xie, Joseph M. Reilly, Yong Li Dich, and Bertrand Schneider. 2018. Augmenting qualitative analyses of collaborative learning groups through multi-modal sensing. In Proceedings of the Rethinking Learning in the Digital Age: Making the Learning Sciences Count, 13th International Conference of the Learning Sciences (ICLS), Vol. 1. J. Kay and R. Luckin (Eds.). International Society of the Learning Sciences, Inc."},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1145\/3328485"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2022.3226473"}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3658376","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3658376","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:44:17Z","timestamp":1750290257000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3658376"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,2]]},"references-count":87,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3658376"],"URL":"https:\/\/doi.org\/10.1145\/3658376","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,2]]},"assertion":[{"value":"2022-12-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-18","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}