{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T00:38:14Z","timestamp":1776472694018,"version":"3.51.2"},"reference-count":35,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T00:00:00Z","timestamp":1604620800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005745","name":"Universidad de Atacama","doi-asserted-by":"publisher","award":["DIUDA-Regular 22386"],"award-info":[{"award-number":["DIUDA-Regular 22386"]}],"id":[{"id":"10.13039\/501100005745","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>While technology has helped improve process efficiency in several domains, it still has an outstanding debt to education. In this article, we introduce NAIRA, a Multimodal Learning Analytics platform that provides Real-Time Feedback to foster collaborative learning activities\u2019 efficiency. NAIRA provides real-time visualizations for students\u2019 verbal interactions when working in groups, allowing teachers to perform precise interventions to ensure learning activities\u2019 correct execution. We present a case study with 24 undergraduate subjects performing a remote collaborative learning activity based on the Jigsaw learning technique within the COVID-19 pandemic context. The main goals of the study are (1) to qualitatively describe how the teacher used NAIRA\u2019s visualizations to perform interventions and (2) to identify quantitative differences in the number and time between students\u2019 spoken interactions among two different stages of the activity, one of them supported by NAIRA\u2019s visualizations. The case study showed that NAIRA allowed the teacher to monitor and facilitate the learning activity\u2019s supervised stage execution, even in a remote learning context, with students working in separate virtual classrooms with their video cameras off. The quantitative comparison of spoken interactions suggests the existence of differences in the distribution between the monitored and unmonitored stages of the activity, with a more homogeneous speaking time distribution in the NAIRA supported stage.<\/jats:p>","DOI":"10.3390\/s20216337","type":"journal-article","created":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T09:03:04Z","timestamp":1604653384000},"page":"6337","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["A Multimodal Real-Time Feedback Platform Based on Spoken Interactions for Remote Active Learning Support"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9130-184X","authenticated-orcid":false,"given":"Hector","family":"Cornide-Reyes","sequence":"first","affiliation":[{"name":"Departamento de Ingenier\u00eda Inform\u00e1tica y Ciencias de la Computaci\u00f3n, Universidad de Atacama, Copiap\u00f3 1534146, Chile"},{"name":"Escuela de Ingenier\u00eda Inform\u00e1tica, Pontificia Universidad Cat\u00f3lica de Valpara\u00edso, Valpara\u00edso 2362807, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4491-0148","authenticated-orcid":false,"given":"Fabi\u00e1n","family":"Riquelme","sequence":"additional","affiliation":[{"name":"Escuela de Ingenier\u00eda Inform\u00e1tica, Universidad de Valpara\u00edso, Valpara\u00edso 2362905, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6780-3754","authenticated-orcid":false,"given":"Diego","family":"Monsalves","sequence":"additional","affiliation":[{"name":"Escuela de Ingenier\u00eda Inform\u00e1tica, Universidad de Valpara\u00edso, Valpara\u00edso 2362905, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3652-4645","authenticated-orcid":false,"given":"Rene","family":"Noel","sequence":"additional","affiliation":[{"name":"Escuela de Ingenier\u00eda Inform\u00e1tica, Universidad de Valpara\u00edso, Valpara\u00edso 2362905, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6384-409X","authenticated-orcid":false,"given":"Cristian","family":"Cechinel","sequence":"additional","affiliation":[{"name":"Centro de Ci\u00eancias, Tecnologias e Sa\u00fade, Universidade Federal de Santa Catarina, Ararangu\u00e1 88906-072, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8711-3109","authenticated-orcid":false,"given":"Rodolfo","family":"Villarroel","sequence":"additional","affiliation":[{"name":"Escuela de Ingenier\u00eda Inform\u00e1tica, Pontificia Universidad Cat\u00f3lica de Valpara\u00edso, Valpara\u00edso 2362807, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6411-0511","authenticated-orcid":false,"given":"Francisco","family":"Ponce","sequence":"additional","affiliation":[{"name":"Escuela de Ingenier\u00eda Inform\u00e1tica, Universidad de Valpara\u00edso, Valpara\u00edso 2362905, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1302-0206","authenticated-orcid":false,"given":"Roberto","family":"Munoz","sequence":"additional","affiliation":[{"name":"Escuela de Ingenier\u00eda Inform\u00e1tica, Universidad de Valpara\u00edso, Valpara\u00edso 2362905, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,11,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1126\/science.1200970","article-title":"The World\u2019s Technological Capacity to Store, Communicate, and Compute Information","volume":"332","author":"Hilbert","year":"2011","journal-title":"Science"},{"key":"ref_2","first-page":"37","article-title":"Innovative Pedagogies in Higher Education to Become Effective Teachers of 21st Century Skills: Unpacking the Learning and Innovations Skills Domain of the New Learning Paradigm","volume":"3","author":"Kivunja","year":"2014","journal-title":"Int. J. High. Educ."},{"key":"ref_3","unstructured":"Lucas, B., Claxton, G., and Hanson, J. (2014). Thinking Like an Engineer: Implications for the Education System, Royal Academy of Engineering. Technical Report."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1080\/00220272.2012.668938","article-title":"A comparative analysis of international frameworks for 21stcentury competences: Implications for national curriculum policies","volume":"44","author":"Voogt","year":"2012","journal-title":"J. Curric. Stud."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"633","DOI":"10.1007\/s10209-019-00683-w","article-title":"Using multimodal learning analytics to study collaboration on discussion groups","volume":"18","author":"Riquelme","year":"2019","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.tele.2017.09.018","article-title":"The impact of Transformational and Authentic leadership on innovation in higher education: The contingent role of knowledge sharing","volume":"35","author":"Elrehail","year":"2018","journal-title":"Telemat. Inform."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1031","DOI":"10.1016\/j.tele.2017.09.014","article-title":"A new educational pattern in response to new technologies and sustainable development. Enlightening ICT skills for youth employability in the European Union","volume":"35","author":"Picatoste","year":"2018","journal-title":"Telemat. Inform."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1002\/j.2168-9830.2004.tb00809.x","article-title":"Does active learning work? A review of the research","volume":"93","author":"Prince","year":"2004","journal-title":"J. Eng. Educ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"8410","DOI":"10.1073\/pnas.1319030111","article-title":"Active learning increases student performance in science, engineering, and mathematics","volume":"111","author":"Freeman","year":"2014","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Nealy, C. (2005). Integrating soft skills through active learning in the management classroom. J. Coll. Teach. Learn. (TLC), 2.","DOI":"10.19030\/tlc.v2i4.1805"},{"key":"ref_11","first-page":"401","article-title":"The Evidence for the Effectiveness of Active Learning","volume":"7","author":"Jain","year":"2014","journal-title":"Orient. J. Comput. Sci. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1177\/0888406408330872","article-title":"Can you hear me now? Evaluation of an online wireless technology to provide real-time feedback to special education teachers-in-training","volume":"32","author":"Rock","year":"2009","journal-title":"Teach. Educ. Spec. Educ."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1177\/0022487105285672","article-title":"The teacher effectiveness movement: How 80 years of essentialist control have shaped the teacher education profession","volume":"57","author":"Imig","year":"2006","journal-title":"J. Teach. Educ."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Barmaki, R., and Hughes, C.E. (2015, January 9\u201313). Providing real-time feedback for student teachers in a virtual rehearsal environment. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, Seattle, WA, USA.","DOI":"10.1145\/2818346.2830604"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Cornide-Reyes, H., No\u00ebl, R., Riquelme, F., Gajardo, M., Cechinel, C., Lean, R.M., Becerra, C., Villarroel, R., and Munoz, R. (2019). Introducing Low-Cost Sensors into the Classroom Settings: Improving the Assessment in Agile Practices with Multimodal Learning Analytics. Sensors, 19.","DOI":"10.3390\/s19153291"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"67783","DOI":"10.1109\/ACCESS.2018.2876801","article-title":"Exploring Collaborative Writing of User Stories With Multimodal Learning Analytics: A Case Study on a Software Engineering Course","volume":"6","author":"Noel","year":"2018","journal-title":"IEEE Access"},{"key":"ref_17","first-page":"149","article-title":"Development of a Software that Supports Multimodal Learning Analytics: A Case Study on Oral Presentations","volume":"24","author":"Munoz","year":"2018","journal-title":"J. Univers. Comput. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"125776","DOI":"10.1109\/ACCESS.2020.3008327","article-title":"Where are You? Exploring Micro-Location in Indoor Learning Environments","volume":"8","author":"Riquelme","year":"2020","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"923","DOI":"10.1007\/s42438-020-00155-y","article-title":"Online university teaching during and after the Covid-19 crisis: Refocusing teacher presence and learning activity","volume":"2","author":"Rapanta","year":"2020","journal-title":"Postdigital Sci. Educ."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chen, T., Peng, L., Yin, X., Rong, J., Yang, J., and Cong, G. (2020). Analysis of User Satisfaction with Online Education Platforms in China during the COVID-19 Pandemic. Healthcare, 8.","DOI":"10.3390\/healthcare8030200"},{"key":"ref_21","first-page":"359","article-title":"Evaluation of Online Teaching Platforms Based on AHP in the Context of COVID-19","volume":"8","author":"Cao","year":"2020","journal-title":"Open J. Soc. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"5","DOI":"10.2307\/j.ctt6wq6fh.6","article-title":"Overview of problem-based learning: Definitions and distinctions","volume":"Volume 9","author":"Savery","year":"2015","journal-title":"Essential Readings in Problem-Based Learning: Exploring and Extending the Legacy of Howard S. Barrows"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1080\/13540602.2016.1203772","article-title":"Preparing teacher-students for twenty-first-century learning practices (PREP 21): A framework for enhancing collaborative problem-solving and strategic learning skills","volume":"23","author":"Ahonen","year":"2017","journal-title":"Teach. Teach."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"220","DOI":"10.18608\/jla.2016.32.11","article-title":"Multimodal Learning Analytics and Education Data Mining: Using computational technologies to measure complex learning tasks","volume":"3","author":"Blikstein","year":"2016","journal-title":"J. Learn. Anal."},{"key":"ref_25","unstructured":"Lang, C., Siemens, G., Wise, A., and Gasevi\u0107, D. (2017). Multimodal Learning Analytics. The Handbook of Learning Analytics, Society for Learning Analytics Research (SoLAR). Chapter 11."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Vrzakova, H., Amon, M.J., Stewart, A., Duran, N.D., and D\u2019Mello, S.K. (2020, January 25\u201327). Focused or stuck together: Multimodal patterns reveal triads\u2019 performance in collaborative problem solving. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, Frankfurt, Germany.","DOI":"10.1145\/3375462.3375467"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sharma, K., Leftheriotis, I., and Giannakos, M. (2020). Utilizing Interactive Surfaces to Enhance Learning, Collaboration and Engagement: Insights from Learners\u2019 Gaze and Speech. Sensors, 20.","DOI":"10.3390\/s20071964"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Vujovic, M., Tassani, S., and Hern\u00e1ndez-Leo, D. (2019, January 16\u201319). Motion Capture as an Instrument in Multimodal Collaborative Learning Analytics. Proceedings of the European Conference on Technology Enhanced Learning, Delft, The Netherlands.","DOI":"10.1007\/978-3-030-29736-7_49"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Praharaj, S., Scheffel, M., Drachsler, H., and Specht, M. (2019, January 16\u201319). Group Coach for Co-located Collaboration. Proceedings of the European Conference on Technology Enhanced Learning, Delft, The Netherlands.","DOI":"10.1007\/978-3-030-29736-7_77"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"MacNeil, S., Kiefer, K., Thompson, B., Takle, D., and Latulipe, C. (2019, January 17\u201319). IneqDetect: A Visual Analytics System to Detect Conversational Inequality and Support Reflection during Active Learning. Proceedings of the ACM Conference on Global Computing Education, Chengdu, China.","DOI":"10.1145\/3300115.3309528"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Chandrasegaran, S., Bryan, C., Shidara, H., Chuang, T.Y., and Ma, K.L. (2019, January 4\u20139). TalkTraces: Real-time capture and visualization of verbal content in meetings. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Scotland, UK.","DOI":"10.1145\/3290605.3300807"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"960","DOI":"10.1016\/j.ejor.2014.11.006","article-title":"Cooperation through social influence","volume":"242","author":"Molinero","year":"2015","journal-title":"Eur. J. Oper. Res."},{"key":"ref_33","first-page":"635","article-title":"Collaborative Learning and the \u201cConversation of Mankind\u201d","volume":"46","author":"Bruffee","year":"1984","journal-title":"Coll. Engl."},{"key":"ref_34","unstructured":"(2020, June 17). The Jigsaw Classroom. Available online: https:\/\/www.jigsaw.org\/."},{"key":"ref_35","first-page":"1","article-title":"Design principles and design patterns","volume":"1","author":"Martin","year":"2000","journal-title":"Object Mentor"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6337\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:30:10Z","timestamp":1760178610000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/21\/6337"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,6]]},"references-count":35,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["s20216337"],"URL":"https:\/\/doi.org\/10.3390\/s20216337","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,6]]}}}