{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T08:46:31Z","timestamp":1771231591935,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2019,10,1]],"date-time":"2019-10-01T00:00:00Z","timestamp":1569888000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Innovation for Defence Excellence and Security (IDEaS) Program","award":["W7714-186568"],"award-info":[{"award-number":["W7714-186568"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Simulation-based training has been proven to be a highly effective pedagogical strategy. However, misalignment between the participant\u2019s level of expertise and the difficulty of the simulation has been shown to have significant negative impact on learning outcomes. To ensure that learning outcomes are achieved, we propose a novel framework for adaptive simulation with the goal of identifying the level of expertise of the learner, and dynamically modulating the simulation complexity to match the learner\u2019s capability. To facilitate the development of this framework, we investigate the classification of expertise using biological signals monitored through wearable sensors. Trauma simulations were developed in which electrocardiogram (ECG) and galvanic skin response (GSR) signals of both novice and expert trauma responders were collected. These signals were then utilized to classify the responders\u2019 expertise, successive to feature extraction and selection, using a number of machine learning methods. The results show the feasibility of utilizing these bio-signals for multimodal expertise classification to be used in adaptive simulation applications.<\/jats:p>","DOI":"10.3390\/s19194270","type":"journal-article","created":{"date-parts":[[2019,10,1]],"date-time":"2019-10-01T11:11:16Z","timestamp":1569928276000},"page":"4270","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Toward Dynamically Adaptive Simulation: Multimodal Classification of User Expertise Using Wearable Devices"],"prefix":"10.3390","volume":"19","author":[{"given":"Kyle","family":"Ross","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pritam","family":"Sarkar","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dirk","family":"Rodenburg","sequence":"additional","affiliation":[{"name":"Faculty of Engineering and Applied Sciences, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0210-2698","authenticated-orcid":false,"given":"Aaron","family":"Ruberto","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Kingston Health Sciences Centre, Kingston, ON K7L 2V7, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul","family":"Hungler","sequence":"additional","affiliation":[{"name":"Department of Chemical Engineering, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adam","family":"Szulewski","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Kingston Health Sciences Centre, Kingston, ON K7L 2V7, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Howes","sequence":"additional","affiliation":[{"name":"Department of Critical Care Medicine, Queen\u2019s University, Kingston, ON K7L 2V7, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7128-0220","authenticated-orcid":false,"given":"Ali","family":"Etemad","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e1511","DOI":"10.3109\/0142159X.2013.818632","article-title":"Simulation in healthcare education: A best evidence practical guide. AMEE Guide No. 82","volume":"35","author":"Motola","year":"2013","journal-title":"Med. Teach."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1046\/j.1365-2923.37.s1.6.x","article-title":"Patient simulation for training basic and advanced clinical skills","volume":"37","author":"Good","year":"2003","journal-title":"Med. Educ."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Susan, G. (2009). Simulation Techniques to Bridge the Gap between Novice and Competent Healthcare Professionals. Online J. Issues Nursing, 14.","DOI":"10.3912\/OJIN.Vol14No02Man03"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"i34","DOI":"10.1136\/qshc.2009.038562","article-title":"Training and simulation for patient safety","volume":"19","author":"Aggarwal","year":"2010","journal-title":"BMJ Qual. Saf."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1007\/s11251-012-9213-x","article-title":"Realism, authenticity, and learning in healthcare simulations: Rules of relevance and irrelevance as interactive achievements","volume":"40","author":"Rystedt","year":"2012","journal-title":"Instr. Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1177\/0018720812442086","article-title":"Sensitivity of Physiological Measures for Detecting Systematic Variations in Cognitive Demand from a Working Memory Task: An On-Road Study across Three Age Groups","volume":"54","author":"Mehler","year":"2012","journal-title":"Hum. Factors"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1007\/s10648-007-9054-3","article-title":"Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction","volume":"19","author":"Kalyuga","year":"2007","journal-title":"Edu. Psychol. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1016\/0959-4752(94)90003-5","article-title":"Cognitive load theory, learning difficulty, and instructional design","volume":"4","author":"Sweller","year":"1994","journal-title":"Learn. Instr."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1097\/SIH.0000000000000097","article-title":"Cognitive Load Theory for the Design of Medical Simulations","volume":"10","author":"Fraser","year":"2015","journal-title":"Simu. Healthc."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1207\/s15516709cog1202_4","article-title":"Cognitive load during problem solving: Effects on learning","volume":"12","author":"Sweller","year":"1988","journal-title":"Cogn. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1016\/j.ijmedinf.2006.09.019","article-title":"The multitasking clinician: Decision-making and cognitive demand during and after team handoffs in emergency care","volume":"76","author":"Laxmisan","year":"2007","journal-title":"Int. J. Med. Inf."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1111\/j.1365-2923.2009.03498.x","article-title":"Cognitive load theory in health professional education: Design principles and strategies","volume":"44","author":"Sweller","year":"2010","journal-title":"Med. Edu."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Rodenburg, D., Hungler, P., Etemad, S.A., Howes, D., Szulewski, A., and Mclellan, J. (2018, January 15\u201317). Dynamically adaptive simulation based on expertise and cognitive load. Proceedings of the 2018 IEEE Games, Entertainment, Media Conference, Galway, Ireland.","DOI":"10.1109\/GEM.2018.8587618"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1207\/S15326985EP3801_4","article-title":"The Expertise Reversal Effect","volume":"38","author":"Kalyuga","year":"2003","journal-title":"Educ. Psychol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"419","DOI":"10.2466\/pms.1994.79.1.419","article-title":"Measurement of Cognitive Load in Instructional Research","volume":"79","author":"Paas","year":"1994","journal-title":"Percept. Motor Skills"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1016\/j.ergon.2005.04.005","article-title":"Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic","volume":"35","author":"Ryu","year":"2005","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sarkar, P., Ross, K., Ruberto, A., Rodenburg, D., Hungler, P., and Etemad, A. (2019). Classification of Cognitive Load and Expertise for Adaptive Simulation Using Deep Multitask Learning. arXiv.","DOI":"10.1109\/ACII.2019.8925507"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Oschlies-Strobel, A., Gruss, S., Jerg-Bretzke, L., Walter, S., and Hazer-Rau, D. (2017, January 11\u201313). Preliminary classification of cognitive load states in a human machine interaction scenario. Proceedings of the 2017 International Conference on Companion Technology, Ulm, Germany.","DOI":"10.1109\/COMPANION.2017.8287084"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Saitis, C., Parvez, M.Z., and Kalimeri, K. (2018). Cognitive Load Assessment from EEG and Peripheral Biosignals for the Design of Visually Impaired Mobility Aids. Wirel. Commun. Mob. Comput., 1\u20139.","DOI":"10.1155\/2018\/8971206"},{"key":"ref_20","unstructured":"Selye, H. (1950). The Physiology and Pathology of Exposure to Stress, ACTA Publications."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Conway, D., Dick, I., Li, Z., Wang, Y., and Chen, F. (2013). The Effect of Stress on Cognitive Load Measurement. Human-Computer Interaction\u2013INTERACT 2013, Springer.","DOI":"10.1007\/978-3-642-40498-6_58"},{"key":"ref_22","unstructured":"(2018, April 04). Advanced Training in Emergency Care Procedures. SimMan\u00ae 3G. Available online: https:\/\/www.laerdal.com\/ca\/products\/simulation-training\/emergency-care-trauma\/simman-3g\/."},{"key":"ref_23","unstructured":"(2018, April 04). Work Smarter with the Ultimate Mixed Reality Device. Available online: https:\/\/www.microsoft.com\/en-us\/hololens."},{"key":"ref_24","unstructured":"(2018, April 04). Consensys ECG Development Kits. Available online: http:\/\/www.shimmersensing.com\/products\/ecg-development-kit."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1109\/TBME.1985.325532","article-title":"A Real-Time QRS Detection Algorithm","volume":"BME-32","author":"Pan","year":"1985","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"7312","DOI":"10.24297\/ijct.v17i2.7616","article-title":"Performance Comparison of Ann Classifiers for Sleep Apnea Detection Based on Ecg Signal Analysis Using Hilbert Transform","volume":"17","author":"Bali","year":"2018","journal-title":"Int. J. Comput. Technol."},{"key":"ref_27","first-page":"1017","article-title":"A Guide for Analysing EDA & Skin Conductance Responses for Psychological Experiments","volume":"49","author":"Braithwaite","year":"2013","journal-title":"Psychophysiology"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Plataniotis, K., Hatzinakos, D., and Lee, J. (September, January 21). ECG Biometric Recognition without Fiducial Detection. Proceedings of the 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference, Baltimore, MD, USA.","DOI":"10.1109\/BCC.2006.4341628"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1111\/j.1542-474X.1996.tb00275.x","article-title":"Heart Rate Variability: Standards of Measurement, Physiological Interpretation, and Clinical Use: Task Force of the European Society of Cardiology and the North American Society for Pacing and Electrophysiology","volume":"1","author":"Malik","year":"1996","journal-title":"Ann. Noninvasive Electrocardiol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1109\/TITS.2005.848368","article-title":"Detecting stress during real-world driving tasks using physiological sensors","volume":"6","author":"Healey","year":"2005","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1007\/BF00648343","article-title":"Least-squares frequency analysis of unequally spaced data","volume":"39","author":"Lomb","year":"1976","journal-title":"Astrophys. Space Sci."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Calvo, R.A., Brown, I., and Scheding, S. (2009, January 1\u20134). Effect of Experimental Factors on the Recognition of Affective Mental States through Physiological Measures. Proceedings of the AI 2009: Advances in Artificial Intelligence. 22nd Australasian Joint Conference, Melbourne, Australia.","DOI":"10.1007\/978-3-642-10439-8_7"},{"key":"ref_33","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J. R. Stat. Soc."},{"key":"ref_35","unstructured":"Tang, J., Alelyani, S., and Liu, H. (2014). Feature selection for classification: A review. Data Classif., 37\u201364."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-vector networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1109\/TGE.1977.6498972","article-title":"The decision tree classifier: Design and potential","volume":"15","author":"Swain","year":"1977","journal-title":"IEEE Trans. Geosci. Electron."},{"key":"ref_38","unstructured":"Breiman, L. (2019, July 16). RANDOM FORESTS\u2013RANDOM FEATURES. Available online: http:\/\/citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.367.9714&rep=rep1&type=pdf."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/TIT.1967.1053964","article-title":"Nearest neighbor pattern classification","volume":"IT-13","author":"Cover","year":"1967","journal-title":"IEEE Trans. Inf. Theory"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/19\/4270\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:26:47Z","timestamp":1760189207000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/19\/4270"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,1]]},"references-count":39,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2019,10]]}},"alternative-id":["s19194270"],"URL":"https:\/\/doi.org\/10.3390\/s19194270","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,1]]}}}