{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T00:27:18Z","timestamp":1774398438512,"version":"3.50.1"},"reference-count":47,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2019,8,7]],"date-time":"2019-08-07T00:00:00Z","timestamp":1565136000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The development of multimodal sensor-based applications designed to support learners with the improvement of their skills is expensive since most of these applications are tailor-made and built from scratch. In this paper, we show how the Presentation Trainer (PT), a multimodal sensor-based application designed to support the development of public speaking skills, can be modularly extended with a Virtual Reality real-time feedback module (VR module), which makes usage of the PT more immersive and comprehensive. The described study consists of a formative evaluation and has two main objectives. Firstly, a technical objective is concerned with the feasibility of extending the PT with an immersive VR Module. Secondly, a user experience objective focuses on the level of satisfaction of interacting with the VR extended PT. To study these objectives, we conducted user tests with 20 participants. Results from our test show the feasibility of modularly extending existing multimodal sensor-based applications, and in terms of learning and user experience, results indicate a positive attitude of the participants towards using the application (PT+VR module).<\/jats:p>","DOI":"10.3390\/s19163457","type":"journal-article","created":{"date-parts":[[2019,8,8]],"date-time":"2019-08-08T11:05:32Z","timestamp":1565262332000},"page":"3457","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Beyond Reality\u2014Extending a Presentation Trainer with an Immersive VR Module"],"prefix":"10.3390","volume":"19","author":[{"given":"Jan","family":"Schneider","sequence":"first","affiliation":[{"name":"DIPF Leibniz Institute for Research and Information in Education, Rostocker Stra\u00dfe 6, D-60323 Frankfurt am Main, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gianluca","family":"Romano","sequence":"additional","affiliation":[{"name":"Goethe University Frankfurt, Theodor-W.-Adorno-Platz, D-60323 Frankfurt am Main, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8407-5314","authenticated-orcid":false,"given":"Hendrik","family":"Drachsler","sequence":"additional","affiliation":[{"name":"DIPF Leibniz Institute for Research and Information in Education, Rostocker Stra\u00dfe 6, D-60323 Frankfurt am Main, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,8,7]]},"reference":[{"key":"ref_1","unstructured":"Bloom, B.S., and Sosniak, L.A. (1985). Developing Talent in Young People, Ballantine Books."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1037\/0021-9010.73.2.327","article-title":"Job experience correlates of job performance","volume":"73","author":"McDaniel","year":"1988","journal-title":"J. Appl. Psychol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ericsson, K.A., Hoffman, R.R., Kozbelt, A., and Williams, A.M. (2018). The Cambridge Handbook of Expertise and Expert Performance, Cambridge University Press.","DOI":"10.1017\/9781316480748"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1207\/s15516709cog1304_1","article-title":"Skill Acquisition and the LISP Tutor","volume":"13","author":"Anderson","year":"1989","journal-title":"Cogn. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Graesser, A.C., Hu, X., and Sottilare, R. (2018). Intelligent tutoring systems. International Handbook of the Learning Sciences, Routledge.","DOI":"10.4324\/9781315617572-24"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"970","DOI":"10.1037\/a0032447","article-title":"A meta-analysis of the effectiveness of intelligent tutoring systems on K\u201312 students\u2019 mathematical learning","volume":"105","author":"Cooper","year":"2013","journal-title":"J. Educ. Psychol."},{"key":"ref_7","unstructured":"Crow, T., Luxton-Reilly, A., and Wuensche, B. (February, January 30). Intelligent tutoring systems for programming education: A systematic review. Proceedings of the 20th Australasian Computing Education Conference, ACM, Brisbane, Queensland, Australia."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"217","DOI":"10.3390\/jsan1030217","article-title":"The Internet of Things, Wearable Computing, Objective Metrics, and the Quantified Self 2.0","volume":"1","author":"Swan","year":"2012","journal-title":"J. Sens. Actuator Netw."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4097","DOI":"10.3390\/s150204097","article-title":"Augmenting the Senses: A Review on Sensor-Based Learning Support","volume":"15","author":"Schneider","year":"2015","journal-title":"Sensors"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Blikstein, P. (2013, January 8\u201313). Multimodal learning analytics. Proceedings of the Third International Conference on Learning Analytics and Knowledge, New York, NY, USA.","DOI":"10.1145\/2460296.2460316"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1111\/jcal.12288","article-title":"From signals to knowledge: A conceptual model for multimodal learning analytics","volume":"34","author":"Schneider","year":"2018","journal-title":"J. Comput. Assist. Learn."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.edurev.2018.07.001","article-title":"Using sensors and augmented reality to train apprentices using recorded expert performance: A systematic literature review","volume":"25","author":"Limbu","year":"2018","journal-title":"Educ. Res. Rev."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Spelmezan, D. (2012, January 21\u201324). An investigation into the use of tactile instructions in snowboarding. Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services, San Francisco, CA, USA.","DOI":"10.1145\/2371574.2371639"},{"key":"ref_14","first-page":"104","article-title":"Musicjacket\u2014Combining motion capture and vibrotactile feedback to teach violin bowing","volume":"60","author":"Schoonderwaldt","year":"2010","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1016\/j.arthro.2015.07.023","article-title":"Effectiveness of Virtual Reality Training in Orthopaedic Surgery","volume":"32","author":"Lonjon","year":"2016","journal-title":"Arthrosc. J. Arthrosc. Relat. Surg."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"e1769","DOI":"10.1002\/rcs.1769","article-title":"A virtual\u2014Reality simulator and force sensation combined catheter operation training system and its preliminary evaluation","volume":"13","author":"Wang","year":"2017","journal-title":"Int. J. Med. Robot. Comput. Assist. Surgery"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hoque, M., Courgeon, M., Martin, J.-C., Mutlu, B., and Picard, R.W. (2013, January 8\u201312). MACH: My automated conversation coach. Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing-UbiComp, Zurich, Switzerland.","DOI":"10.1145\/2493432.2493502"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Alexandersson, J., Aretoulaki, M., Campbell, N., Gardner, M., Girenko, A., Klakow, D., Koryzis, D., Petukhova, V., Specht, M., and Spiliotopoulos, D. Metalogue: A Multiperspective Multimodal Dialogue System with Metacognitive Abilities for Highly Adaptive and Flexible Dialogue Management. Proceedings of the 2014 International Conference on Intelligent Environments, 365\u2013368.","DOI":"10.1109\/IE.2014.67"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Praharaj, S., Scheffel, M., Drachsler, H., and Specht, M. (2018). Multimodal Analytics for Real-Time Feedback in Co-located Collaboration. European Conference on Technology Enhanced Learning, Springer.","DOI":"10.1007\/978-3-319-98572-5_15"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Worsley, M. (2018, January 7\u20139). (Dis) engagement matters: Identifying efficacious learning practices with multimodal learning analytics. Proceedings of the 8th International Conference on Learning Analytics and Knowledge, Sydney, Australia.","DOI":"10.1145\/3170358.3170420"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Dermody, F., Sutherland, A., Zhang, Z., Cohen, P., Bohus, D., Horaud, R., and Meng, H. (2015, January 9\u201313). A Multimodal System for Public Speaking with Real Time Feedback. 2015 ACM In Proceedings of International Conference on Multimodal Interaction, Seattle, WA, USA.","DOI":"10.1145\/2818346.2823295"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ochoa, X., Dom\u00ednguez, F., Guam\u00e1n, B., Maya, R., Falcones, G., and Castells, J. (2018, January 7\u20139). The RAP system: Automatic feedback of oral presentation skills using multimodal analysis and low-cost sensors. Proceedings of the 8th International Conference on Learning Analytics and Knowledge, Sydney, Australia.","DOI":"10.1145\/3170358.3170406"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1109\/TLT.2016.2627043","article-title":"Can You Help Me with My Pitch? Studying a Tool for Real-Time Automated Feedback","volume":"9","author":"Schneider","year":"2016","journal-title":"IEEE Trans. Learn. Technol."},{"key":"ref_24","first-page":"745","article-title":"Feedback research revisited","volume":"2","author":"Mory","year":"2004","journal-title":"Handb. Res. Educ. Commun. Technol."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Schneider, J., B\u00f6rner, D., Van Rosmalen, P., and Specht, M. (2017, January 24\u201329). Do You Know What Your Nonverbal Behavior Communicates? Studying a Self-reflection Module for the Presentation Trainer. Proceedings of the International Conference on Immersive Learning, Coimbra, Portugal.","DOI":"10.1007\/978-3-319-60633-0_8"},{"key":"ref_26","unstructured":"Limbu, B., Schneider, J., Klemke, R., and Specht, M. (2018, January 23\u201325). Augmentation of practice with expert performance data: Presenting a calligraphy use case. Proceedings of the 3rd International Conference on Smart Learning Ecosystem and Regional Development-The Interplay of Data, Technology, Place and People, Aalborg, Denmark."},{"key":"ref_27","first-page":"108","article-title":"From AR to Expertise: A User Study of an Augmented Reality Training to Support Expertise Development","volume":"24","author":"Limbu","year":"2018","journal-title":"J. Univers. Comput. Sci."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wen, R., Nguyen, B.P., Chng, C.-B., and Chui, C.-K. (2013, January 5\u20136). In Situ spatial AR surgical planning using projector-Kinect system. Proceedings of the Fourth Symposium on Information and Communication Technology-SoICT \u201913, Danang, Vietnam.","DOI":"10.1145\/2542050.2542060"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Caballero, A.R., and Niguidula, J.D. (2018, January 23\u201329). Disaster Risk Management and Emergency Preparedness: A Case-Driven Training Simulation Using Immersive Virtual Reality. Proceedings of the 4th International Conference on Human-Computer Interaction and User Experience in Indonesia, CHIuXiD\u201918, Yogyakarta, Indonesia.","DOI":"10.1145\/3205946.3205950"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Molka-Danielsen, J., Prasolova-F, E., Fominykh, M., and Lamb, K. (2018, January 4\u20137). Use of a Collaborative Virtual Reality Simulation for Multi-Professional Training in Emergency Management Communications. Proceedings of the 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), Wollongong, Australia.","DOI":"10.1109\/TALE.2018.8615147"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1016\/j.gaitpost.2009.03.002","article-title":"Gait & Posture Young and Elderly","volume":"30","author":"Verhoeff","year":"2009","journal-title":"Gait Posture"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Schneider, J., Di Mitri, D., Limbu, B., and Drachsler, H. (2018). Multimodal learning hub: A tool for capturing customizable multimodal learning experiences. European Conference on Technology Enhanced Learning, Springer.","DOI":"10.1007\/978-3-319-98572-5_4"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Di Mitri, D., Schneider, J., Klemke, R., Specht, M., and Drachsler, H. (2019, January 4\u20138). Read Between the Lines: An Annotation Tool for Multimodal Data for Learning. Proceedings of the 9th International Conference on Learning Analytics & Knowledge, ACM, Tempe, Arizona.","DOI":"10.1145\/3303772.3303776"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders (DSM-5\u00ae), American Psychiatric Pub.","DOI":"10.1176\/appi.books.9780890425596"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1089\/109493102321018187","article-title":"Brief Virtual Reality Therapy for Public Speaking Anxiety","volume":"5","author":"Harris","year":"2002","journal-title":"CyberPsychol. Behav."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.compedu.2019.02.006","article-title":"Fostering oral presentation competence through a virtual reality-based task for delivering feedback","volume":"134","author":"Gulikers","year":"2019","journal-title":"Comput. Educ."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1111\/jcal.12175","article-title":"Presentation Trainer: What experts and computers can tell about your nonverbal communication","volume":"33","author":"Schneider","year":"2017","journal-title":"J. Comput. Assist. Learn."},{"key":"ref_38","unstructured":"Schneider, J. (2019, August 05). Learning Hub. In GitHub Repository. Available online: https:\/\/github.com\/janschneiderou\/LearningHub\/blob\/master\/README.md."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"16","DOI":"10.3102\/0013189X11428813","article-title":"Design-based research: A decade of progress in education research?","volume":"41","author":"Anderson","year":"2012","journal-title":"Educ. Res."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1111\/j.1365-2923.1989.tb00884.x","article-title":"The effectiveness of immediate feedback during the Objective Structured Clinical Examination","volume":"23","author":"Hodder","year":"1989","journal-title":"Med. Educ."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1080\/03634520009379224","article-title":"Public speaking performance improvement as a function of information processing in immediate and delayed feedback interventions","volume":"49","author":"King","year":"2000","journal-title":"Commun. Educ."},{"key":"ref_42","first-page":"159","article-title":"A comparison of immediate feedback delivered via an FM headset versus delayed feedback on the inappropriate verbalizations of a student with ADHD","volume":"25","author":"Price","year":"2002","journal-title":"Educ. Treat. Child."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1177\/0888406411401919","article-title":"Effects of immediate feedback delivered via webcam and bug-in-ear technology on preservice teacher performance","volume":"35","author":"Scheeler","year":"2012","journal-title":"Teach. Educ. Spec. Educ."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Worsley, M., Chiluiza, K., Grafsgaard, J.F., and Ochoa, X. (2015, January 9\u201313). 2015 Multimodal Learning and Analytics Grand Challenge. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, Seattle, WA, USA.","DOI":"10.1145\/2818346.2829995"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"3315","DOI":"10.1109\/TIM.2013.2272859","article-title":"Toward a Mathematical Model for Quality of Experience Evaluation of Haptic Applications","volume":"62","author":"Hamam","year":"2013","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Barmaki, R., and Hughes, C.E. (2015, January 9\u201313). Providing real-time feedback for student teachers in a virtual rehearsal environment. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, Seattle, WA, USA.","DOI":"10.1145\/2818346.2830604"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1007\/s11042-012-0990-7","article-title":"Effect of kinesthetic and tactile haptic feedback on the quality of experience of edutainment applications","volume":"67","author":"Hamam","year":"2013","journal-title":"Multimed. Tools Appl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/16\/3457\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:09:25Z","timestamp":1760188165000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/16\/3457"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,8,7]]},"references-count":47,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2019,8]]}},"alternative-id":["s19163457"],"URL":"https:\/\/doi.org\/10.3390\/s19163457","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,8,7]]}}}