{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T12:59:32Z","timestamp":1781873972884,"version":"3.54.5"},"reference-count":37,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,2,8]],"date-time":"2021-02-08T00:00:00Z","timestamp":1612742400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Although virtual reality (VR) has already achieved technological maturity, there are still some significant drawbacks for technology acceptance and broader user adoption, presenting research challenges. Thus, there is a need for standard, reliable, and quick assessment tools for Virtual Reality-Induced Symptoms and Effects (VRISE) and user experience in VR Assessing VRISE and user experience could be time consuming, especially when using objective physiological measures. In this study, we have reviewed, compared, and performed a suitability assessment of existing standard measures for evaluating VRISE and user experience in VR We have developed a first-person VR game with different scenes and different conditions. For assessing VRISE symptoms, we have used the Simulator Sickness Questionnaire (SSQ) and Fast Motion Sickness Score (FMS). For assessing user experience, we have used the short version of the User Experience Questionnaire (UEQ-S). We have also used a novel Virtual Reality Neuroscience Questionnaire (VRNQ) for assessing VRISE and user experience aspects. The result has shown that FMS and VRNQ (VRISE section) are suitable for quick assessment of VRISE and that VRNQ (User experience section) is suitable for assessing user experience. The advantage of FMS and VRNQ questionnaires is that they are shorter to fulfill and easier to understand. FMS also enables to record the VRISE levels during the virtual experience and thus capturing its trend over time. Another advantage of the VRNQ is that it also provides the minimum and parsimonious cut-offs to appraise the suitability of VR software, which we have confirmed in our study to be adequate.<\/jats:p>","DOI":"10.3390\/s21041185","type":"journal-article","created":{"date-parts":[[2021,2,10]],"date-time":"2021-02-10T04:33:46Z","timestamp":1612931626000},"page":"1185","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":65,"title":["Suitability and Comparison of Questionnaires Assessing Virtual Reality-Induced Symptoms and Effects and User Experience in Virtual Environments"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3483-237X","authenticated-orcid":false,"given":"Andrej","family":"Somrak","sequence":"first","affiliation":[{"name":"Faculty of Electrical Engineering, University of Ljubljana, Tr\u017ea\u0161ka CESTA 25, 1000 Ljubljana, Slovenia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matev\u017e","family":"Poga\u010dnik","sequence":"additional","affiliation":[{"name":"Faculty of Electrical Engineering, University of Ljubljana, Tr\u017ea\u0161ka CESTA 25, 1000 Ljubljana, Slovenia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5161-7751","authenticated-orcid":false,"given":"Jo\u017ee","family":"Guna","sequence":"additional","affiliation":[{"name":"Faculty of Electrical Engineering, University of Ljubljana, Tr\u017ea\u0161ka CESTA 25, 1000 Ljubljana, Slovenia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,8]]},"reference":[{"key":"ref_1","unstructured":"(2020, October 30). Gartner Hype Cycle for Emerging Technologies. Available online: https:\/\/www.gartner.com\/en\/newsroom\/press-releases\/2018-08-20-gartner-identifies-five-emerging-technology-trends-that-will-blur-the-lines-between-human-and-machine."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1207\/s15327108ijap0303_3","article-title":"Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness","volume":"3","author":"Kennedy","year":"1993","journal-title":"Int. J. Aviat. Psychol."},{"key":"ref_3","unstructured":"Stone, W.B. (2017). Psychometric Evaluation of the Simulator Sickness Questionnaire as a Measure of Cybersickness. [Ph.D. Thesis, Iowa State University, Digital Repository]."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Hirzle, T., Cordts, M., Rukzio, E., Gugenheimer, J., and Bulling, A. (2021, January 8\u201313). A Critical Assessment of the Use of SSQ as a Measure of General Discomfort in VR Head-Mounted Displays. Proceedings of the 2021 ACM CHI Virtual Conference on Human Factors in Computing Systems - CHI, Yokohama, Japan.","DOI":"10.1145\/3411764.3445361"},{"key":"ref_5","unstructured":"Wolpe, J., and Lazarus, A.A. (1966). Behavior Therapy Techniques: A Guide to the Treatment of Neuroses, Pergamon Press."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1016\/j.future.2018.08.049","article-title":"Influence of Video Content Type on Users\u2019 Virtual Reality Sickness Perception and Physiological Response","volume":"91","author":"Guna","year":"2019","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1016\/j.future.2018.11.041","article-title":"Estimating VR Sickness and User Experience Using Different HMD Technologies: An Evaluation Study","volume":"94","author":"Somrak","year":"2019","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1007\/s10055-017-0329-9","article-title":"Self-reported Discomfort When Using Commercially Targeted Virtual Reality Equipment in Discomfort Distraction","volume":"22","author":"Boylan","year":"2018","journal-title":"Virtual Real."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1007\/s10055-015-0266-4","article-title":"Simulator Sickness Incidence and Susceptibility during Neck Motion-Controlled Virtual Reality Tasks","volume":"19","author":"Treleaven","year":"2015","journal-title":"Virtual Real."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1016\/0022-3999(95)00638-9","article-title":"Assessment of the Multiple Dimensions of Nausea: The Nausea Profile (NP)","volume":"40","author":"Muth","year":"1996","journal-title":"J. Psychosom. Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1177\/0018720811403736","article-title":"Validating an Efficient Method to Quantify Motion Sickness","volume":"53","author":"Keshavarz","year":"2011","journal-title":"Hum. Factors"},{"key":"ref_12","first-page":"573","article-title":"Assessing User Experience in Virtual Reality\u2014A Comparison of Different Measurements","volume":"Volume 10918","author":"Marcus","year":"2018","journal-title":"Design, User Experience, and Usability: Theory and Practice"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Law, E.L.-C., Br\u00fchlmann, F., and Mekler, E.D. (2018, January 28\u201331). Systematic Review and Validation of the Game Experience Questionnaire (GEQ)\u2014Implications for Citation and Reporting Practice. Proceedings of the Annual Symposium on Computer-Human Interaction in Play Extended Abstracts\u2014CHI PLAY \u201918, Mebourne, Australia.","DOI":"10.1145\/3242671.3242683"},{"key":"ref_14","unstructured":"Allison, D., Tummel, C., Richert, A., and Isenhardt, I. (July, January 27). Towards Measuring User Experience, Activation and Task Performance in Immersive Virtual Learning Environments for Students. Proceedings of the International Conference on Immersive Learning, Santa Barbara, CA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wienrich, C., Weidner, C.K., Schatto, C., Obremski, D., and Israel, J.H. (2018, January 5\u20137). A Virtual Nose as a Rest-Frame\u2014The Impact on Simulator Sickness and Game Experience. Proceedings of the 2018 10th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), W\u00fcrzburg, Germany.","DOI":"10.1109\/VS-Games.2018.8493408"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"103","DOI":"10.9781\/ijimai.2017.09.001","article-title":"Design and Evaluation of a Short Version of the User Experience Questionnaire (UEQ-S)","volume":"4","author":"Schrepp","year":"2017","journal-title":"IJIMAI"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"417","DOI":"10.3389\/fnhum.2019.00417","article-title":"Validation of the Virtual Reality Neuroscience Questionnaire: Maximum Duration of Immersive Virtual Reality Sessions Without the Presence of Pertinent Adverse Symptomatology","volume":"13","author":"Kourtesis","year":"2019","journal-title":"Front. Hum. Neurosci."},{"key":"ref_18","unstructured":"Prothero, J.D., and Parker, D.E. (2003). A Unified Approach to Presence and Motion Sickness. Virtual and adaptive Environments: Applications, Implications, and Human Performance Issues, CRC Press."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Davis, S., Nesbitt, K., and Nalivaiko, E. (2014, January 2\u20133). A Systematic Review of Cybersickness. Proceedings of the 2014 Conference on Interactive Entertainment\u2014IE2014, Newcastle, NSW, Australia.","DOI":"10.1145\/2677758.2677780"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1145\/333329.333344","article-title":"A Discussion of Cybersickness in Virtual Environments","volume":"32","author":"LaViola","year":"2000","journal-title":"SIGCHI Bull."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1163\/22134808-20181293","article-title":"Cybersickness: A Multisensory Integration Perspective","volume":"30","author":"Gallagher","year":"2018","journal-title":"Multisens. Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1658","DOI":"10.1080\/10447318.2020.1778351","article-title":"Virtual Reality Sickness: A Review of Causes and Measurements","volume":"36","author":"Chang","year":"2020","journal-title":"Int. J. Hum. Comput. Interact."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"4","DOI":"10.3389\/frobt.2020.00004","article-title":"Virtual Reality Is Sexist: But It Does Not Have to Be","volume":"7","author":"Stanney","year":"2020","journal-title":"Front. Robot. AI"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Saredakis, D., Szpak, A., Birckhead, B., Keage, H.A., and Loetscher, T. (2020). Factors Associated with Virtual Reality Sickness in Head-Mounted Displays: A Systematic Review and Meta-Analysis. Front. Hum. Neurosci., 34.","DOI":"10.31234\/osf.io\/7u4hn"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1016\/j.displa.2007.09.009","article-title":"Characteristics of Habituation to Motion in a Virtual Environment","volume":"29","author":"Howarth","year":"2008","journal-title":"Displays"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Wright, W.G. (2014). Using Virtual Reality to Augment Perception, Enhance Sensorimotor Adaptation, and Change Our Minds. Front. Syst. Neurosci., 8.","DOI":"10.3389\/fnsys.2014.00056"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1007\/978-3-540-89350-9_6","article-title":"Construction and Evaluation of a User Experience Questionnaire","volume":"Volume 5298","author":"Holzinger","year":"2008","journal-title":"HCI and Usability for Education and Work"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2018\/7827286","article-title":"Towards a Model of User Experience in Immersive Virtual Environments","volume":"2018","author":"Christmann","year":"2018","journal-title":"Adv. Hum. Comput. Interact."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1080\/00207590701396674","article-title":"Usability, Aesthetics and Emotions in Human\u2013Technology Interaction","volume":"42","author":"Mahlke","year":"2007","journal-title":"Int. J. Psychol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.displa.2017.01.002","article-title":"Correlating Reaction Time and Nausea Measures with Traditional Measures of Cybersickness","volume":"48","author":"Nesbitt","year":"2017","journal-title":"Displays"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1007\/s10015-019-00536-y","article-title":"Emotional Evaluation for Pictures Displayed with Small FOV Telescope Environment in Virtual Reality Headset","volume":"24","author":"Yamaguchi","year":"2019","journal-title":"Artif. Life Robot."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"34","DOI":"10.4028\/www.scientific.net\/AEF.10.34","article-title":"A Review on Cybersickness and Usability in Virtual Environments","volume":"10","author":"Mousavi","year":"2013","journal-title":"AEF"},{"key":"ref_33","unstructured":"Cedergren, J.E. (2018). Evaluating the User Experience and Usability of Virtual Reality Locomotion Techniques. [Master\u2019s Thesis, University of Oslo]."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1177\/1046878119831420","article-title":"Gaming in Virtual Reality: What Changes in Terms of Usability, Emotional Response and Sense of Presence Compared to Non-Immersive Video Games?","volume":"50","author":"Pallavicini","year":"2019","journal-title":"Simul. Gaming"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Chen, F., Lin, Y.-C., Chien, J.-W., and Tsai, C.-E. (2016, January 15\u201316). Virtual Reality for Digital User Experience and Interactive Learning Based on User Satisfaction: A Pilot Study. Proceedings of the 2016 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA.","DOI":"10.1109\/CSCI.2016.0077"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Hou, G., Dong, H., and Yang, Y. (2017, January 22\u201324). Developing a Virtual Reality Game User Experience Test Method Based on EEG Signals. Proceedings of the 2017 5th International Conference on Enterprise Systems (ES), Beijing, China.","DOI":"10.1109\/ES.2017.45"},{"key":"ref_37","unstructured":"(2020, December 12). Unity Real-Time Development Platform\u20143D, 2D VR & AR Engine. Available online: https:\/\/unity.com\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1185\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:21:17Z","timestamp":1760160077000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1185"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,8]]},"references-count":37,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["s21041185"],"URL":"https:\/\/doi.org\/10.3390\/s21041185","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,8]]}}}