{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T15:28:42Z","timestamp":1777735722543,"version":"3.51.4"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2019,12,21]],"date-time":"2019-12-21T00:00:00Z","timestamp":1576886400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,12,21]],"date-time":"2019-12-21T00:00:00Z","timestamp":1576886400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000287","name":"Royal Academy of Engineering","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000287","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Virtual Reality"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Virtual reality (VR) simulation offers significant potential for human factors training as it provides a novel approach which enables training in environments that are otherwise dangerous, impractical or expensive to simulate. While VR training has been adopted in many environments, such as heavy industry, surgery and aviation, there remains an inadequate understanding of how virtual simulations impact cognitive factors. One such factor, which needs careful consideration during the design of VR simulations, is the degree of mental or cognitive load experienced during training. This study aimed to validate a newly developed measure of workload, based on existing instruments (e.g. the NASA-TLX), but tailored to the specific demands placed on users of simulated environments. While participants completed a VR puzzle game, a series of experimental manipulations of workload were used to assess the sensitivity of the new instrument. The manipulations affected the questionnaire subscales (mental demands; physical demands; temporal demands; frustration; task complexity; situational stress; distraction; perceptual strain; task control; presence) as predicted in all cases (<jats:italic>p<\/jats:italic>s\u2009&lt;\u2009.05), except for presence, which displayed little relationship with other aspects of task load. The scale was also found to have good convergent validity with an alternate index of task load. The findings support the sensitivity of the new instrument for assessing task load in virtual reality. Overall, this study contributes to the understanding of mental workload in simulated environments and provides a practical tool for use in both future research and applications in the field.<\/jats:p>","DOI":"10.1007\/s10055-019-00422-9","type":"journal-article","created":{"date-parts":[[2020,2,3]],"date-time":"2020-02-03T14:18:14Z","timestamp":1580739494000},"page":"557-566","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":128,"title":["Development and validation of a simulation workload measure: the simulation task load index (SIM-TLX)"],"prefix":"10.1007","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3880-3856","authenticated-orcid":false,"given":"David","family":"Harris","sequence":"first","affiliation":[]},{"given":"Mark","family":"Wilson","sequence":"additional","affiliation":[]},{"given":"Samuel","family":"Vine","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,12,21]]},"reference":[{"issue":"1","key":"422_CR1","doi-asserted-by":"publisher","first-page":"29","DOI":"10.3233\/NRE-2009-0497","volume":"25","author":"SV Adamovich","year":"2009","unstructured":"Adamovich SV, Fluet GG, Tunik E, Merians AS (2009) Sensorimotor training in virtual reality: a review. NeuroRehabilitation 25(1):29\u201344. https:\/\/doi.org\/10.3233\/NRE-2009-0497","journal-title":"NeuroRehabilitation"},{"key":"422_CR2","unstructured":"Bates D, M\u00e4chler M, Bolker B, Walker S (2014) Fitting linear mixed-effects models using lme4. Retrieved from http:\/\/arxiv.org\/abs\/1406.5823"},{"issue":"2","key":"422_CR3","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1016\/j.ejogrb.2013.03.017","volume":"169","author":"R Bharathan","year":"2013","unstructured":"Bharathan R, Vali S, Setchell T, Miskry T, Darzi A, Aggarwal R (2013) Psychomotor skills and cognitive load training on a virtual reality laparoscopic simulator for tubal surgery is effective. Eur J Obstet Gynecol Reprod Biol 169(2):347\u2013352. https:\/\/doi.org\/10.1016\/j.ejogrb.2013.03.017","journal-title":"Eur J Obstet Gynecol Reprod Biol"},{"issue":"1","key":"422_CR4","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1207\/S15326985EP3801_7","volume":"38","author":"R Brunken","year":"2003","unstructured":"Brunken R, Plass JL, Leutner D (2003) Direct measurement of cognitive load in multimedia learning. Educ Psychol 38(1):53\u201361. https:\/\/doi.org\/10.1207\/S15326985EP3801_7","journal-title":"Educ Psychol"},{"key":"422_CR5","doi-asserted-by":"publisher","DOI":"10.1162\/105474603322955950","volume-title":"Virtual reality technology","author":"GC Burdea","year":"2003","unstructured":"Burdea GC, Coiffet P (2003) Virtual reality technology. Wiley, Hoboken"},{"issue":"2","key":"422_CR6","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1037\/h0046016","volume":"56","author":"DT Campbell","year":"1959","unstructured":"Campbell DT, Fiske DW (1959) Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull 56(2):81\u2013105","journal-title":"Psychol Bull"},{"issue":"11","key":"422_CR7","doi-asserted-by":"publisher","first-page":"977","DOI":"10.1016\/j.ergon.2008.01.012","volume":"38","author":"A DiDomenico","year":"2008","unstructured":"DiDomenico A, Nussbaum MA (2008) Interactive effects of physical and mental workload on subjective workload assessment. Int J Ind Ergon 38(11):977\u2013983. https:\/\/doi.org\/10.1016\/j.ergon.2008.01.012","journal-title":"Int J Ind Ergon"},{"issue":"4","key":"422_CR8","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1080\/1463922X.2011.624647","volume":"15","author":"PS Dunston","year":"2014","unstructured":"Dunston PS, Proctor RW, Wang X (2014) Challenges in evaluating skill transfer from construction equipment simulators. Theor Issues Ergon Sci 15(4):354\u2013375. https:\/\/doi.org\/10.1080\/1463922X.2011.624647","journal-title":"Theor Issues Ergon Sci"},{"key":"422_CR9","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-019-06887-8","author":"JG Frederiksen","year":"2019","unstructured":"Frederiksen JG, S\u00f8rensen SMD, Konge L, Svendsen MBS, Nobel-J\u00f8rgensen M, Bjerrum F, Andersen SAW (2019) Cognitive load and performance in immersive virtual reality versus conventional virtual reality simulation training of laparoscopic surgery: a randomized trial. Surg Endosc. https:\/\/doi.org\/10.1007\/s00464-019-06887-8","journal-title":"Surg Endosc"},{"key":"422_CR10","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2017.02183","author":"R Gray","year":"2017","unstructured":"Gray R (2017) Transfer of training from virtual to real baseball batting. Front Psychol. https:\/\/doi.org\/10.3389\/fpsyg.2017.02183","journal-title":"Front Psychol"},{"issue":"2","key":"422_CR11","doi-asserted-by":"publisher","first-page":"212","DOI":"10.3200\/JMBR.36.2.212-224","volume":"36","author":"MA Guadagnoli","year":"2004","unstructured":"Guadagnoli MA, Lee TD (2004) Challenge point: a framework for conceptualizing the effects of various practice conditions in motor learning. J Mot Behav 36(2):212\u2013224. https:\/\/doi.org\/10.3200\/JMBR.36.2.212-224","journal-title":"J Mot Behav"},{"key":"422_CR12","doi-asserted-by":"publisher","unstructured":"Hart SG, Staveland LE (1988) Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Advances in psychology, vol 52, pp 139\u2013183. https:\/\/doi.org\/10.1016\/S0166-4115(08)62386-9","DOI":"10.1016\/S0166-4115(08)62386-9"},{"issue":"3","key":"422_CR13","doi-asserted-by":"publisher","first-page":"1397","DOI":"10.1007\/s00464-017-5821-5","volume":"32","author":"DA Hashimoto","year":"2018","unstructured":"Hashimoto DA, Petrusa E, Phitayakorn R, Valle C, Casey B, Gee D (2018) A proficiency-based virtual reality endoscopy curriculum improves performance on the fundamentals of endoscopic surgery examination. Surg Endosc 32(3):1397\u20131404. https:\/\/doi.org\/10.1007\/s00464-017-5821-5","journal-title":"Surg Endosc"},{"issue":"1","key":"422_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/S0959-4752(01)00014-7","volume":"12","author":"PA Kirschner","year":"2002","unstructured":"Kirschner PA (2002) Cognitive load theory: implications of cognitive load theory on the design of learning. Learn Instr 12(1):1\u201310. https:\/\/doi.org\/10.1016\/S0959-4752(01)00014-7","journal-title":"Learn Instr"},{"issue":"3","key":"422_CR15","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1016\/j.cpr.2004.04.001","volume":"24","author":"M Krijn","year":"2004","unstructured":"Krijn M, Emmelkamp PMG, Olafsson RP, Biemond R (2004) Virtual reality exposure therapy of anxiety disorders: a review. Clin Psychol Rev 24(3):259\u2013281. https:\/\/doi.org\/10.1016\/j.cpr.2004.04.001","journal-title":"Clin Psychol Rev"},{"issue":"8","key":"422_CR16","doi-asserted-by":"publisher","first-page":"1060","DOI":"10.1080\/00140139.2015.1122234","volume":"59","author":"SJ Lackey","year":"2016","unstructured":"Lackey SJ, Salcedo JN, Szalma JL, Hancock PA (2016) The stress and workload of virtual reality training: the effects of presence, immersion and flow. Ergonomics 59(8):1060\u20131072. https:\/\/doi.org\/10.1080\/00140139.2015.1122234","journal-title":"Ergonomics"},{"issue":"1","key":"422_CR17","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1080\/02701367.1989.10607413","volume":"60","author":"E McAuley","year":"1989","unstructured":"McAuley E, Duncan T, Tammen VV (1989) Psychometric properties of the intrinsic motivation inventory in a competitive sport setting: a confirmatory factor analysis. Res Q Exerc Sport 60(1):48\u201358. https:\/\/doi.org\/10.1080\/02701367.1989.10607413","journal-title":"Res Q Exerc Sport"},{"key":"422_CR18","first-page":"309","volume-title":"Le Traite de la realite Virtuelle","author":"D Mestre","year":"2006","unstructured":"Mestre D, Fuchs P, Berthoz A, Vercher JL (2006) Immersion et pr\u00e9sence. In: Fuchs P, Moreau G, Berthoz A, Vercher JL (eds) Le Traite de la realite Virtuelle. Ecole Des Mines de Paris, Paris, pp 309\u2013338"},{"key":"422_CR19","doi-asserted-by":"publisher","DOI":"10.1201\/9780203489925-30","volume-title":"Handbook of ergonomics and human factors methods","author":"JM Mulder","year":"2004","unstructured":"Mulder JM, de Waard D, Brookhuis KA (2004) Estimating mental effort using heart rate and heart rate variability. In: Stanton N, Hedge A, Hendrick HW, Brookhuis KA, Salas E (eds) Handbook of ergonomics and human factors methods. Taylor and Francis, London. https:\/\/doi.org\/10.1201\/9780203489925-30"},{"issue":"8","key":"422_CR20","doi-asserted-by":"publisher","first-page":"805","DOI":"10.1111\/medu.12732","volume":"49","author":"LM Naismith","year":"2015","unstructured":"Naismith LM, Cheung JJH, Ringsted C, Cavalcanti RB (2015) Limitations of subjective cognitive load measures in simulation-based procedural training. Med Educ 49(8):805\u2013814. https:\/\/doi.org\/10.1111\/medu.12732","journal-title":"Med Educ"},{"issue":"3","key":"422_CR21","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1007\/s10055-017-0320-5","volume":"22","author":"DL Neumann","year":"2018","unstructured":"Neumann DL, Moffitt RL, Thomas PR, Loveday K, Watling DP, Lombard CL, Antonova S, Tremeer MA (2018) A systematic review of the application of interactive virtual reality to sport. Virtual Real 22(3):183\u2013198. https:\/\/doi.org\/10.1007\/s10055-017-0320-5","journal-title":"Virtual Real"},{"issue":"4","key":"422_CR22","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1049\/iet-its:20080023","volume":"2","author":"A Pauzie","year":"2008","unstructured":"Pauzie A (2008) A method to assess the driver mental workload: the driving activity load index (DALI). IET Intel Transp Syst 2(4):315\u2013322. https:\/\/doi.org\/10.1049\/iet-its:20080023","journal-title":"IET Intel Transp Syst"},{"issue":"1","key":"422_CR23","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1016\/S0959-4752(01)00016-0","volume":"12","author":"E Pollock","year":"2002","unstructured":"Pollock E, Chandler P, Sweller J (2002) Assimilating complex information. Learn Instr 12(1):61\u201386. https:\/\/doi.org\/10.1016\/S0959-4752(01)00016-0","journal-title":"Learn Instr"},{"issue":"1","key":"422_CR24","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1207\/S15326985EP3801_3","volume":"38","author":"A Renkl","year":"2003","unstructured":"Renkl A, Atkinson RK (2003) Structuring the transition from example study to problem solving in cognitive skill acquisition: a cognitive load perspective. Educ Psychol 38(1):15\u201322. https:\/\/doi.org\/10.1207\/S15326985EP3801_3","journal-title":"Educ Psychol"},{"issue":"4","key":"422_CR25","doi-asserted-by":"publisher","first-page":"494","DOI":"10.1080\/001401300184378","volume":"43","author":"FD Rose","year":"2000","unstructured":"Rose FD, Attree EA, Brooks BM, Parslow DM, Penn PR (2000) Training in virtual environments: transfer to real world tasks and equivalence to real task training. Ergonomics 43(4):494\u2013511. https:\/\/doi.org\/10.1080\/001401300184378","journal-title":"Ergonomics"},{"key":"422_CR26","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1016\/j.chb.2017.09.012","volume":"78","author":"D Shin","year":"2018","unstructured":"Shin D (2018) Empathy and embodied experience in virtual environment: to what extent can virtual reality stimulate empathy and embodied experience? Comput Hum Behav 78:64\u201373. https:\/\/doi.org\/10.1016\/j.chb.2017.09.012","journal-title":"Comput Hum Behav"},{"key":"422_CR27","doi-asserted-by":"publisher","first-page":"302","DOI":"10.1016\/j.future.2018.11.041","volume":"94","author":"A Somrak","year":"2019","unstructured":"Somrak A, Humar I, Hossain MS, Alhamid MF, Hossain MA, Guna J (2019) Estimating VR sickness and user experience using different HMD technologies: an evaluation study. Future Gener Comput Syst 94:302\u2013316. https:\/\/doi.org\/10.1016\/j.future.2018.11.041","journal-title":"Future Gener Comput Syst"},{"issue":"4","key":"422_CR28","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1016\/S1071-5819(03)00015-6","volume":"58","author":"KM Stanney","year":"2003","unstructured":"Stanney KM, Mollaghasemi M, Reeves L, Breaux R, Graeber DA (2003) Usability engineering of virtual environments (VEs): identifying multiple criteria that drive effective VE system design. Int J Hum Comput Stud 58(4):447\u2013481. https:\/\/doi.org\/10.1016\/S1071-5819(03)00015-6","journal-title":"Int J Hum Comput Stud"},{"key":"422_CR29","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1109\/VR.2016.7504689","volume":"2016","author":"A Steed","year":"2016","unstructured":"Steed A, Pan Y, Zisch F, Steptoe W (2016) The impact of a self-avatar on cognitive load in immersive virtual reality. IEEE Virtual Real: VR 2016:67\u201376. https:\/\/doi.org\/10.1109\/VR.2016.7504689","journal-title":"IEEE Virtual Real: VR"},{"issue":"02","key":"422_CR30","doi-asserted-by":"publisher","first-page":"41","DOI":"10.4236\/ojmsi.2015.32005","volume":"03","author":"JA Stevens","year":"2015","unstructured":"Stevens JA, Kincaid JP (2015) The relationship between presence and performance in virtual simulation training. Open J Model Simul 03(02):41\u201348. https:\/\/doi.org\/10.4236\/ojmsi.2015.32005","journal-title":"Open J Model Simul"},{"issue":"3","key":"422_CR31","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1097\/01.sla.0000200839.93965.26","volume":"243","author":"LM Sutherland","year":"2006","unstructured":"Sutherland LM, Middleton PF, Anthony A, Hamdorf J, Cregan P, Scott D, Maddern GJ (2006) Surgical simulation. Ann Surg 243(3):291\u2013300. https:\/\/doi.org\/10.1097\/01.sla.0000200839.93965.26","journal-title":"Ann Surg"},{"key":"422_CR32","unstructured":"Sweller J (1999) Instructional design in technical areas. ACER Press, Melbourne, Australia"},{"key":"422_CR33","unstructured":"R Core Team (2017) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. https:\/\/www.R-project.org\/"},{"issue":"5","key":"422_CR34","doi-asserted-by":"publisher","first-page":"497","DOI":"10.1162\/105474600566989","volume":"9","author":"M Usoh","year":"2000","unstructured":"Usoh M, Catena E, Arman S, Slater M (2000) Using presence questionnaires in reality. Presence Teleoperators Virtual Environ 9(5):497\u2013503. https:\/\/doi.org\/10.1162\/105474600566989","journal-title":"Presence Teleoperators Virtual Environ"},{"key":"422_CR35","doi-asserted-by":"crossref","unstructured":"van Buuren S, Groothuis-Oudshoorn K (2011) Mice: multivariate imputation by chained equations in R. Retrieved 10 Dec 2018 from http:\/\/www.jstatsoft.org\/v45\/i03\/","DOI":"10.18637\/jss.v045.i03"},{"issue":"3","key":"422_CR36","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1007\/BF02504799","volume":"53","author":"T van Gog","year":"2005","unstructured":"van Gog T, Ericsson KA, Rikers RMJP, Paas F (2005) Instructional design for advanced learners: establishing connections between the theoretical frameworks of cognitive load and deliberate practice. Educ Tech Res Dev 53(3):73\u201381. https:\/\/doi.org\/10.1007\/BF02504799","journal-title":"Educ Tech Res Dev"},{"key":"422_CR37","doi-asserted-by":"publisher","unstructured":"Wickens CD (1992) Virtual reality and education. In: Proceedings of 1992 IEEE international conference on systems, man, and cybernetics, vol 1, pp 842\u2013847. https:\/\/doi.org\/10.1109\/ICSMC.1992.271688","DOI":"10.1109\/ICSMC.1992.271688"},{"issue":"9","key":"422_CR38","doi-asserted-by":"publisher","first-page":"1961","DOI":"10.1007\/s00268-011-1141-4","volume":"35","author":"MR Wilson","year":"2011","unstructured":"Wilson MR, Poolton JM, Malhotra N, Ngo K, Bright E, Masters RSW (2011) Development and validation of a surgical workload measure: the surgery task load index (SURG-TLX). World J Surg 35(9):1961. https:\/\/doi.org\/10.1007\/s00268-011-1141-4","journal-title":"World J Surg"},{"key":"422_CR39","unstructured":"Zijlstra FRH (1993) Efficiency in work behaviour: a design approach for modern tools. Retrieved 5 Dec 2018 from http:\/\/resolver.tudelft.nl\/uuid:d97a028b-c3dc-4930-b2ab-a7877993a17f"}],"container-title":["Virtual Reality"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10055-019-00422-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10055-019-00422-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10055-019-00422-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,12,20]],"date-time":"2020-12-20T00:23:44Z","timestamp":1608423824000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10055-019-00422-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,21]]},"references-count":39,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["422"],"URL":"https:\/\/doi.org\/10.1007\/s10055-019-00422-9","relation":{},"ISSN":["1359-4338","1434-9957"],"issn-type":[{"value":"1359-4338","type":"print"},{"value":"1434-9957","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12,21]]},"assertion":[{"value":"13 April 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 December 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 December 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}