{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,24]],"date-time":"2026-07-24T23:44:17Z","timestamp":1784936657297,"version":"3.55.0"},"reference-count":60,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,1,29]],"date-time":"2023-01-29T00:00:00Z","timestamp":1674950400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Virtual Worlds"],"abstract":"<jats:p>Cybersickness is a drawback of virtual reality (VR), which also affects the cognitive and motor skills of users. The Simulator Sickness Questionnaire (SSQ) and its variant, the Virtual Reality Sickness Questionnaire (VRSQ), are two tools that measure cybersickness. However, both tools suffer from important limitations which raise concerns about their suitability. Two versions of the Cybersickness in VR Questionnaire (CSQ-VR), a paper-and-pencil and a 3D\u2013VR version, were developed. The validation of the CSQ-VR and a comparison against the SSQ and the VRSQ were performed. Thirty-nine participants were exposed to three rides with linear and angular accelerations in VR. Assessments of cognitive and psychomotor skills were performed at baseline and after each ride. The validity of both versions of the CSQ-VR was confirmed. Notably, CSQ-VR demonstrated substantially better internal consistency than both SSQ and VRSQ. Additionally, CSQ-VR scores had significantly better psychometric properties in detecting a temporary decline in performance due to cybersickness. Pupil size was a significant predictor of cybersickness intensity. In conclusion, the CSQ-VR is a valid assessment of cybersickness with superior psychometric properties to SSQ and VRSQ. The CSQ-VR enables the assessment of cybersickness during VR exposure, and it benefits from examining pupil size, a biomarker of cybersickness.<\/jats:p>","DOI":"10.3390\/virtualworlds2010002","type":"journal-article","created":{"date-parts":[[2023,1,30]],"date-time":"2023-01-30T11:25:47Z","timestamp":1675077947000},"page":"16-35","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":138,"title":["Cybersickness in Virtual Reality Questionnaire (CSQ-VR): A Validation and Comparison against SSQ and VRSQ"],"prefix":"10.3390","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2914-1064","authenticated-orcid":false,"given":"Panagiotis","family":"Kourtesis","sequence":"first","affiliation":[{"name":"Department of Psychology, National and Kapodistrian University of Athens, 15772 Athens, Greece"},{"name":"Department of Psychology, University of Edinburgh, Edinburgh EH8 9YL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Josie","family":"Linnell","sequence":"additional","affiliation":[{"name":"Department of Psychology, University of Edinburgh, Edinburgh EH8 9YL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rayaan","family":"Amir","sequence":"additional","affiliation":[{"name":"Department of Psychology, University of Edinburgh, Edinburgh EH8 9YL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ferran","family":"Argelaguet","sequence":"additional","affiliation":[{"name":"Institut National de Recherche en Sciences et Technologies du Num\u00e9rique (Inria), de l\u2019Universit\u00e9 de Rennes, 35042 Rennes, France"},{"name":"Institut de Recherche en Informatique et Syst\u00e8mes Al\u00e9atoires (IRISA), de Centre National de la Recherche Scientifique (CNRS), 35042 Rennes, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah E.","family":"MacPherson","sequence":"additional","affiliation":[{"name":"Department of Psychology, University of Edinburgh, Edinburgh EH8 9YL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"103778","DOI":"10.1016\/j.compedu.2019.103778","article-title":"A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda","volume":"147","author":"Radianti","year":"2019","journal-title":"Comput. Educ."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"645153","DOI":"10.3389\/frvir.2021.645153","article-title":"A Review on Virtual Reality Skill Training Applications","volume":"2","author":"Xie","year":"2021","journal-title":"Front. Virtual Real."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"100151","DOI":"10.1016\/j.chbr.2021.100151","article-title":"How immersive virtual reality methods may meet the criteria of the National Academy of Neuropsychology and American Academy of Clinical Neuropsychology: A software review of the Virtual Reality Everyday Assessment Lab (VR-EAL)","volume":"4","author":"Kourtesis","year":"2021","journal-title":"Comput. Hum. Behav. Rep."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1146\/annurev-clinpsy-081219-115923","article-title":"Virtual Reality Therapy in Mental Health","volume":"17","author":"Emmelkamp","year":"2021","journal-title":"Annu. Rev. Clin. Psychol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Cruz-Neira, C., Fern\u00e1ndez, M., and Portal\u00e9s, C. (2018). Virtual Reality and Games. Multimodal Technol. Interact., 2.","DOI":"10.3390\/mti2010008"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1007\/s10055-020-00446-6","article-title":"Estimating cybersickness from virtual reality applications","volume":"25","author":"Rebenitsch","year":"2020","journal-title":"Virtual Real."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1138","DOI":"10.1177\/107118139704100292","article-title":"Cybersickness is Not Simulator Sickness","volume":"Volume 41","author":"Stanney","year":"1997","journal-title":"Proceedings of the Human Factors and Ergonomics Society Annual Meeting"},{"key":"ref_8","unstructured":"Blackmore, K., Nesbitt, K., and Smith, S.P. (2014, January 2). A Systematic Review of Cybersickness. Proceedings of the 2014 Conference on Interactive Entertainment, Newcastle, Australia."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1145\/333329.333344","article-title":"A discussion of cybersickness in virtual environments","volume":"32","author":"LaViola","year":"2000","journal-title":"ACM SIGCHI Bull."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.displa.2017.01.002","article-title":"Correlating reaction time and nausea measures with traditional measures of cybersickness","volume":"48","author":"Nesbitt","year":"2017","journal-title":"Displays"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"582156","DOI":"10.3389\/frvir.2021.582156","article-title":"Effects of Linear Visual-Vestibular Conflict on Presence, Perceived Scene Stability and Cybersickness in the Oculus Go and Oculus Quest","volume":"2","author":"Kim","year":"2021","journal-title":"Front. Virtual Real."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"342","DOI":"10.3389\/fnhum.2019.00342","article-title":"Technological Competence Is a Pre-condition for Effective Implementation of Virtual Reality Head Mounted Displays in Human Neuroscience: A Technological Review and Meta-Analysis","volume":"13","author":"Kourtesis","year":"2019","journal-title":"Front. Hum. Neurosci."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"96","DOI":"10.3389\/fnhum.2020.00096","article-title":"Factors Associated with Virtual Reality Sickness in Head-Mounted Displays: A Systematic Review and Meta-Analysis","volume":"14","author":"Saredakis","year":"2020","journal-title":"Front. Hum. Neurosci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"130","DOI":"10.3390\/virtualworlds1020008","article-title":"Virtual Reality Induced Symptoms and Effects: Concerns, Causes, Assessment & Mitigation","volume":"1","author":"Conner","year":"2022","journal-title":"Virtual Worlds"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1177\/0018720809332848","article-title":"Performance and Autonomic Responses during Motion Sickness","volume":"51","author":"Dahlman","year":"2009","journal-title":"Hum. Factors: J. Hum. Factors Ergon. Soc."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"659","DOI":"10.1007\/s10055-021-00535-0","article-title":"Spatial and attentional after effects of virtual reality and relations to cybersickness","volume":"26","author":"Varmaghani","year":"2021","journal-title":"Virtual Real."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/s10055-018-0370-3","article-title":"VR aftereffect and the relation of cybersickness and cognitive performance","volume":"23","author":"Mittelstaedt","year":"2018","journal-title":"Virtual Real."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1016\/j.physbeh.2015.08.043","article-title":"Cybersickness provoked by head-mounted display affects cutaneous vascular tone, heart rate and reaction time","volume":"151","author":"Nalivaiko","year":"2015","journal-title":"Physiol. Behav."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1207\/s15327108ijap0303_3","article-title":"Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness","volume":"3","author":"Kennedy","year":"1993","journal-title":"Int. J. Aviat. Psychol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"102958","DOI":"10.1016\/j.apergo.2019.102958","article-title":"Psychometric evaluation of Simulator Sickness Questionnaire and its variants as a measure of cybersickness in consumer virtual environments","volume":"82","author":"Sevinc","year":"2019","journal-title":"Appl. Ergon."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/j.apergo.2017.12.016","article-title":"Virtual reality sickness questionnaire (VRSQ): Motion sickness measurement index in a virtual reality environment","volume":"69","author":"Kim","year":"2018","journal-title":"Appl. Ergon."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.displa.2007.09.005","article-title":"Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems","volume":"29","author":"Sharples","year":"2008","journal-title":"Displays"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"752","DOI":"10.1038\/nrn3122","article-title":"Virtual reality in neuroscience research and therapy","volume":"12","author":"Bohil","year":"2011","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.displa.2016.11.001","article-title":"Vection and cybersickness generated by head-and-display motion in the Oculus Rift","volume":"46","author":"Palmisano","year":"2017","journal-title":"Displays"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1177\/147078530805000106","article-title":"Do Data Characteristics Change According to the Number of Scale Points Used? An Experiment Using 5-Point, 7-Point and 10-Point Scales","volume":"50","author":"Dawes","year":"2008","journal-title":"Int. J. Mark. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"396","DOI":"10.9734\/BJAST\/2015\/14975","article-title":"Likert Scale: Explored and Explained","volume":"7","author":"Joshi","year":"2015","journal-title":"Br. J. Appl. Sci. Technol."},{"key":"ref_27","first-page":"1","article-title":"What is the best response scale for survey and questionnaire design; review of different lengths of rating scale\/attitude scale\/Likert scale","volume":"8","author":"Taherdoost","year":"2019","journal-title":"Int. J. Acad. Res. Manag."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1177\/0013164411431162","article-title":"Psychological Distance between Categories in the Likert Scale","volume":"72","author":"Wakita","year":"2012","journal-title":"Educ. Psychol. Meas."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"417","DOI":"10.3389\/fnhum.2019.00417","article-title":"Validation of the Virtual Reality Neuroscience Questionnaire: Maximum Duration of Immersive Virtual Reality Sessions without the Presence of Pertinent Adverse Symptomatology","volume":"13","author":"Kourtesis","year":"2019","journal-title":"Front. Hum. Neurosci."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Somrak, A., Poga\u010dnik, M., and Guna, J. (2021). Suitability and Comparison of Questionnaires Assessing Virtual Reality-Induced Symptoms and Effects and User Experience in Virtual Environments. Sensors, 21.","DOI":"10.3390\/s21041185"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S1071-5819(03)00017-X","article-title":"Pupil size variation as an indication of affective processing","volume":"59","author":"Partala","year":"2003","journal-title":"Int. J. Hum.-Comput. Stud."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Schwind, V., Knierim, P., Tasci, C., Franczak, P., Haas, N., and Henze, N. (2017, January 6\u201311). \u201cThese Are Not My Hands!\u201d: Effect of Gender on the Perception of Avatar Hands in Virtual Reality. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, New York, NY, USA.","DOI":"10.1145\/3025453.3025602"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Bebko, A.O., and Troje, N.F. (2020). bmlTUX: Design and Control of Experiments in Virtual Reality and Beyond. I-Perception, 11.","DOI":"10.31234\/osf.io\/arvkf"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"941","DOI":"10.3357\/ASEM.2394.2009","article-title":"Combined Pitch and Roll and Cybersickness in a Virtual Environment","volume":"80","author":"Bonato","year":"2009","journal-title":"Aviat. Space Environ. Med."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Kourtesis, P., Korre, D., Collina, S., Doumas, L.A.A., and MacPherson, S.E. (2020). Guidelines for the Development of Immersive Virtual Reality Software for Cognitive Neuroscience and Neuropsychology: The Development of Virtual Reality Everyday Assessment Lab (VR-EAL), a Neuropsychological Test Battery in Immersive Virtual Reality. Front. Comput. Sci., 1.","DOI":"10.3389\/fcomp.2019.00012"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Wechsler, D. (1939). Wechsler Bellevue Adult Intelligence Scale, Williams & Wilkins.","DOI":"10.1037\/t06871-000"},{"key":"ref_37","first-page":"891","article-title":"Human memory and the medial temporal region of the brain","volume":"34","author":"Corsi","year":"1972","journal-title":"Diss. Abstr. Int."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"258","DOI":"10.3758\/s13428-010-0024-1","article-title":"A free, easy-to-use, computer-based simple and four-choice reaction time programme: The Deary-Liewald reaction time task","volume":"43","author":"Deary","year":"2010","journal-title":"Behav. Res. Methods"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/j.paid.2006.01.012","article-title":"Predicting individual differences in motion sickness susceptibility by questionnaire","volume":"41","author":"Golding","year":"2006","journal-title":"Pers. Individ. Differ."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2023","DOI":"10.1093\/brain\/awf210","article-title":"Effects of monocular viewing and eye dominance on spatial attention","volume":"125","author":"Roth","year":"2002","journal-title":"Brain"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1016\/S2212-5671(15)00123-9","article-title":"How Reliable Are Measurement Scales? External Factors with Indirect Influence on Reliability Estimators","volume":"20","author":"Ursachi","year":"2015","journal-title":"Procedia Econ. Financ."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1136\/eb-2012-100645","article-title":"Principles for high-quality, high-value testing","volume":"18","author":"Power","year":"2013","journal-title":"Evid.-Base. Med."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1177\/070674370705200210","article-title":"What\u2019s under the ROC? An Introduction to Receiver Operating Characteristics Curves","volume":"52","author":"Streiner","year":"2007","journal-title":"Can. J. Psychiatry"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1080\/13854046.2020.1722244","article-title":"American Academy of Clinical Neuropsychology consensus conference statement on uniform labeling of performance test scores","volume":"34","author":"Guilmette","year":"2020","journal-title":"Clin. Neuropsychol."},{"key":"ref_45","unstructured":"(2020, January 01). The Jamovi Project [Computer Software]. Available online: http:\/\/www.jamovi.org."},{"key":"ref_46","unstructured":"R Core Team (2022). A Language and Environment for Statistical Computing, R Foundation for Statistical Computing. Available online: https:\/\/www.R-project.org\/."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"2312","DOI":"10.1080\/02664763.2019.1630372","article-title":"Ordered quantile normalization: A semiparametric transformation built for the cross-validation era","volume":"47","author":"Peterson","year":"2019","journal-title":"J. Appl. Stat."},{"key":"ref_48","unstructured":"Revelle, W. (2018). psych: Procedures for Psychological, Psychometric, and Personality Research, Northwestern University. Available online: https:\/\/CRAN.R-project.org\/package=psych."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Wickham, H. (2016). ggplot2: Elegant Graphics for Data Analysis, Springer-Verlag. Available online: https:\/\/ggplot2.tidyverse.org.","DOI":"10.1007\/978-3-319-24277-4"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"48","DOI":"10.18637\/jss.v067.i01","article-title":"Fitting Linear Mixed-Effects Models Using lme4","volume":"67","author":"Bates","year":"2015","journal-title":"J. Stat. Softw."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"739742","DOI":"10.3389\/fpsyt.2021.739742","article-title":"Arguing in Favor of Revising the Simulator Sickness Questionnaire Factor Structure When Assessing Side Effects Induced by Immersions in Virtual Reality","volume":"12","author":"Bouchard","year":"2021","journal-title":"Front. Psychiatry"},{"key":"ref_52","unstructured":"Iii, W.B.S. (2017). Psychometric Evaluation of the Simulator Sickness Questionnaire as a Measure of Cybersickness. [Ph.D. Thesis, Iowa State University]."},{"key":"ref_53","first-page":"7","article-title":"Psychometric Theory\u201425 Years Ago and Now","volume":"4","author":"Nunnally","year":"1975","journal-title":"Educ. Res."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1017\/S1355617720000764","article-title":"Validation of the Virtual Reality Everyday Assessment Lab (VR-EAL): An Immersive Virtual Reality Neuropsychological Battery with Enhanced Ecological Validity","volume":"27","author":"Kourtesis","year":"2020","journal-title":"J. Int. Neuropsychol. Soc."},{"key":"ref_55","first-page":"137","article-title":"Human-Computer Interaction: Overview on State of the Art","volume":"1","author":"Karray","year":"2008","journal-title":"Int. J. Smart Sens. Intell. Syst."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1783","DOI":"10.1080\/10447318.2020.1828535","article-title":"Identifying Causes of and Solutions for Cybersickness in Immersive Technology: Reformulation of a Research and Development Agenda","volume":"36","author":"Stanney","year":"2020","journal-title":"Int. J. Hum.\u2013Comput. Interact."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Jeong, D., Yoo, S., and Yun, J. (2019, January 23\u201327). Cybersickness Analysis with EEG Using Deep Learning Algorithms. Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan.","DOI":"10.1109\/VR.2019.8798334"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1007\/s10055-021-00517-2","article-title":"Quantifying VR cybersickness using EEG","volume":"26","author":"Krokos","year":"2021","journal-title":"Virtual Real."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Lopes, P., Tian, N., and Boulic, R. (2020, January 16\u201318). Eye Thought You Were Sick! Exploring Eye Behaviors for Cybersickness Detection in VR. Proceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games, New York, NY, USA.","DOI":"10.1145\/3424636.3426906"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Islam, R., Desai, K., and Quarles, J. (2021, January 4\u20138). Cybersickness Prediction from Integrated HMD\u2019s Sensors: A Multimodal Deep Fusion Approach using Eye-tracking and Head-tracking Data. Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Bari, Italy.","DOI":"10.1109\/ISMAR52148.2021.00017"}],"container-title":["Virtual Worlds"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2813-2084\/2\/1\/2\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:19:24Z","timestamp":1760120364000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2813-2084\/2\/1\/2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,29]]},"references-count":60,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["virtualworlds2010002"],"URL":"https:\/\/doi.org\/10.3390\/virtualworlds2010002","relation":{"has-preprint":[{"id-type":"doi","id":"10.36227\/techrxiv.21973640","asserted-by":"object"},{"id-type":"doi","id":"10.31234\/osf.io\/3fwvd","asserted-by":"object"},{"id-type":"doi","id":"10.36227\/techrxiv.21973640.v1","asserted-by":"object"}],"is-referenced-by":[{"id-type":"doi","id":"10.1007\/s10055-025-01300-3","asserted-by":"object"}]},"ISSN":["2813-2084"],"issn-type":[{"value":"2813-2084","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,29]]}}}