{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T21:29:00Z","timestamp":1772141340037,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2021,8,31]],"date-time":"2021-08-31T00:00:00Z","timestamp":1630368000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Over the last few decades, several studies have shown the feasibility, acceptability, and efficacy of VR-based instruments in the early evaluation of executive dysfunction (ED) in psychiatric and neurologic conditions. Due to the negative impact of ED in everyday functioning, identifying innovative strategies for evaluating ED allows clinicians to detect executive impairment early and minimize its effects. This work aimed to test the usability and user experience (UX) of EXecutive-functions Innovative Tool 360\u00b0 (EXIT 360\u00b0), a 360\u00b0-based tool for assessing ED. Seventy-six healthy subjects underwent an evaluation that involved (1) usability assessment using System Usability Scale and (2) evaluation of UX using the ICT-Sense of Presence and UX Questionnaire. Results showed a satisfactory level of usability (mean = 75.9 \u00b1 12.8), with good scores for usability and learnability. As regards UX, EXIT 360\u00b0 showed an absence of negative effects (mean = 1.79 \u00b1 0.95) and high scores in ecological validity (mean = 4.32 \u00b1 0.54) and engagement (mean = 3.76 \u00b1 0.56). Moreover, it obtained good scores in efficiency (mean = 1.84 \u00b1 0.84), originality (mean = 2.49 \u00b1 0.71), and attractiveness (mean = 1.93 \u00b1 0.98). Interestingly, demographic characteristics and technological expertise had no impact on the performance (p &gt; 0.05). Overall, EXIT 360\u00b0 appeared to be a usable, learn-to-use, engaging, and creative tool with irrelevant negative effects. Further studies will be conducted to evaluate these aspects in the clinical population.<\/jats:p>","DOI":"10.3390\/s21175867","type":"journal-article","created":{"date-parts":[[2021,8,31]],"date-time":"2021-08-31T22:58:15Z","timestamp":1630450695000},"page":"5867","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["EXecutive-Functions Innovative Tool (EXIT 360\u00b0): A Usability and User Experience Study of an Original 360\u00b0-Based Assessment Instrument"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8237-6164","authenticated-orcid":false,"given":"Francesca","family":"Borgnis","sequence":"first","affiliation":[{"name":"IRCCS Fondazione Don Carlo Gnocchi ONLUS, 20148 Milan, Italy"},{"name":"Department of Psychology, Universit\u00e0 Cattolica del Sacro Cuore, 20123 Milan, Italy"}]},{"given":"Francesca","family":"Baglio","sequence":"additional","affiliation":[{"name":"IRCCS Fondazione Don Carlo Gnocchi ONLUS, 20148 Milan, Italy"}]},{"given":"Elisa","family":"Pedroli","sequence":"additional","affiliation":[{"name":"Applied Technology for Neuro-Psychology Lab., Istituto Auxologico Italiano, Istituto di Ricovero e Cura a Carattere Scientifico, 20149 Milan, Italy"},{"name":"Faculty of Psychology, eCampus University, 22060 Novedrate, Italy"}]},{"given":"Federica","family":"Rossetto","sequence":"additional","affiliation":[{"name":"IRCCS Fondazione Don Carlo Gnocchi ONLUS, 20148 Milan, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0849-3984","authenticated-orcid":false,"given":"Sara","family":"Isernia","sequence":"additional","affiliation":[{"name":"IRCCS Fondazione Don Carlo Gnocchi ONLUS, 20148 Milan, Italy"}]},{"given":"Lidia","family":"Uccellatore","sequence":"additional","affiliation":[{"name":"Department of Psychology, Universit\u00e0 Cattolica del Sacro Cuore, 20123 Milan, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3657-106X","authenticated-orcid":false,"given":"Giuseppe","family":"Riva","sequence":"additional","affiliation":[{"name":"Department of Psychology, Universit\u00e0 Cattolica del Sacro Cuore, 20123 Milan, Italy"},{"name":"Applied Technology for Neuro-Psychology Lab., Istituto Auxologico Italiano, Istituto di Ricovero e Cura a Carattere Scientifico, 20149 Milan, Italy"}]},{"given":"Pietro","family":"Cipresso","sequence":"additional","affiliation":[{"name":"Applied Technology for Neuro-Psychology Lab., Istituto Auxologico Italiano, Istituto di Ricovero e Cura a Carattere Scientifico, 20149 Milan, Italy"},{"name":"Department of Psychology, Universit\u00e0 degli Studi di Torino, 10124 Turin, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1146\/annurev-psych-113011-143750","article-title":"Executive functions","volume":"64","author":"Diamond","year":"2013","journal-title":"Annu. Rev. Psychol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1016\/j.acn.2007.08.010","article-title":"Assessment of executive functions: Review of instruments and identification of critical issues","volume":"23","author":"Chan","year":"2008","journal-title":"Arch. Clin. Neuropsychol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1080\/09602011.2015.1074083","article-title":"An exploration of the ecological validity of the virtual action planning\u2013supermarket (VAP-S) with people with schizophrenia","volume":"28","author":"Aubin","year":"2018","journal-title":"Neuropsychol. Rehabil."},{"key":"ref_4","first-page":"1","article-title":"Attentional profile of adolescents with ADHD in virtual-reality dual execution tasks: A pilot study","volume":"9","author":"Climent","year":"2020","journal-title":"Appl. Neuropsychol. Child"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"721","DOI":"10.3233\/NRE-172183","article-title":"Application of virtual environments in a multi-disciplinary day neurorehabilitation program to improve executive functioning using the Stroop task","volume":"41","author":"Dahdah","year":"2017","journal-title":"NeuroRehabilitation"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/fnbeh.2014.00405","article-title":"Virtual multiple errands test (VMET): A virtual reality-based tool to detect early executive functions deficit in parkinson\u2019s disease","volume":"8","author":"Cipresso","year":"2014","journal-title":"Front. Behav. Neurosci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1016\/j.schres.2009.09.015","article-title":"Using virtual reality to evaluate executive functioning among persons with schizophrenia: A validity study","volume":"115","author":"Josman","year":"2009","journal-title":"Schizophr. Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1017\/S1355617707070178","article-title":"Cognitive rehabilitation in the elderly: Effects on strategic behavior in relation to goal management","volume":"13","author":"Levine","year":"2007","journal-title":"J. Int. Neuropsychol. Soc. JINS"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1080\/13803395.2012.740002","article-title":"Validity of the virtual reality stroop task (VRST) in active duty military","volume":"35","author":"Armstrong","year":"2013","journal-title":"J. Clin. Exp. Neuropsychol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/fnhum.2015.00660","article-title":"Virtual reality for enhanced ecological validity and experimental control in the clinical, affective and social neurosciences","volume":"9","author":"Parsons","year":"2015","journal-title":"Front. Hum. Neurosci."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-017-16121-x","article-title":"Picture interpretation test (PIT) 360\u00b0: An innovative measure of executive functions","volume":"7","author":"Serino","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-019-42201-1","article-title":"An ecological measure to screen executive functioning in ms: The picture interpretation test (PIT) 360\u00b0","volume":"9","author":"Realdon","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"808","DOI":"10.1080\/09602011.2015.1109523","article-title":"A virtual shopping task for the assessment of executive functions: Validity for people with stroke","volume":"27","author":"Weiss","year":"2017","journal-title":"Neuropsychol. Rehabil."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1023\/B:NERV.0000009483.91468.fb","article-title":"The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills","volume":"13","author":"Chaytor","year":"2003","journal-title":"Neuropsychol. Rev."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/fnins.2021.622095","article-title":"A simple and effective way to study executive functions by using 360\u00b0 videos","volume":"15","author":"Borgnis","year":"2021","journal-title":"Front. Neurosci."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Tuena, C., Pedroli, E., Trimarchi, P.D., Gallucci, A., Chiappini, M., Goulene, K., Gaggioli, A., Riva, G., Lattanzio, F., and Giunco, F. (2020). Usability issues of clinical and research applications of virtual reality in older people: A systematic review. Front. Hum. Neurosci., 14.","DOI":"10.3389\/fnhum.2020.00093"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1207","DOI":"10.1080\/00140139.2020.1774080","article-title":"Usability, user experience and accessibility: Towards an integrative model","volume":"63","author":"Sauer","year":"2020","journal-title":"Ergonomics"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Pedroli, E., Greci, L., Colombo, D., Serino, S., Cipresso, P., Arlati, S., Mondellini, M., Boilini, L., Giussani, V., and Goulene, K. (2018). Characteristics, usability, and users experience of a system combining cognitive and physical therapy in a virtual environment: Positive bike. Sensors, 18.","DOI":"10.3390\/s18072343"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Pedroli, E., Cipresso, P., Serino, S., Albani, G., and Riva, G. (2013, January 5\u20138). A virtual reality test for the assessment of cognitive deficits: Usability and perspectives. Proceedings of the 3rd International Workshop on Pervasive Computing Paradigms for Mental Health, Venice, Italy.","DOI":"10.4108\/pervasivehealth.2013.252359"},{"key":"ref_20","unstructured":"(2021, July 29). Semantic Scholar. Available online: https:\/\/www.semanticscholar.org\/paper\/Ergonomic-requirements-for-office-work-with-visual\/17e53f6fa806ea573e4ee0904fbb12c3a527d5f0."},{"key":"ref_21","unstructured":"This British Standard, and UK implementation of EN ISO (2010). Ergonomics of Human-System Interaction, British Standard Institution."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1037\/0090-5550.46.3.296","article-title":"The application of virtual reality technology in rehabilitation","volume":"46","author":"Schultheis","year":"2001","journal-title":"Rehabil. Psychol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1037\/pri0000028","article-title":"Virtual reality for psychological assessment in clinical practice","volume":"1","author":"Parsons","year":"2016","journal-title":"Pract. Innov."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"695","DOI":"10.1111\/j.1532-5415.2005.53221.x","article-title":"The montreal cognitive assessment, moca: A brief screening tool for mild cognitive impairment","volume":"53","author":"Nasreddine","year":"2005","journal-title":"J. Am. Geriatr. Soc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1007\/s10072-014-1995-y","article-title":"Normative data for the montreal cognitive assessment in an italian population sample","volume":"36","author":"Santangelo","year":"2015","journal-title":"Neurol. Sci."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Borgnis, F., Baglio, F., Pedroli, E., Rossetto, F., Meloni, M., Riva, G., and Cipresso, P. (2021). EXIT 360\u00b0\u2014executive-functions innovative tool 360\u00b0\u2014a simple and effective way to study executive functions in parkinson\u2019s disease by using 360\u00b0 videos. Appl. Sci., 11.","DOI":"10.3390\/app11156791"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1878","DOI":"10.3389\/fpsyg.2018.01878","article-title":"New trends in episodic memory assessment: Immersive 360 ecological videos","volume":"9","author":"Serino","year":"2018","journal-title":"Front. Psychol."},{"key":"ref_28","unstructured":"Brooke, J. (1986). System Usability Scale (SUS): A Quick-And-Dirty Method Of System Evaluation User Information, Digital Equipment Co Ltd."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lewis, J.R., and Sauro, J. (2009). The Factor Structure Of The System Usability Scale. Proceedings of the International Conference on Human Centered Design, Springer.","DOI":"10.1007\/978-3-642-02806-9_12"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1024\/1010-0652.19.3.159","article-title":"Motivation, flow-erleben und lernleistung in universit\u00e4ren lernsettings","volume":"19","author":"Engeser","year":"2005","journal-title":"Z. P\u00e4dagogische Psychol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1111\/j.1467-6494.1994.tb00797.x","article-title":"Facilitating internalization: The self-determination theory perspective","volume":"62","author":"Deci","year":"1994","journal-title":"J. Personal."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1162\/105474601300343612","article-title":"A cross-media presence questionnaire: The ITC-sense of presence inventory","volume":"10","author":"Lessiter","year":"2001","journal-title":"Presence Teleoperators Virtual Environ."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Laugwitz, B., Held, T., and Schrepp, M. (2008). Construction and evaluation of a user experience questionnaire. Symposium of the Austrian HCI and Usability Engineering Group, Springer.","DOI":"10.1007\/978-3-540-89350-9_6"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Schrepp, M., Hinderks, A., and Thomaschewski, J. (2014). Applying the User Experience Questionnaire (UEQ) in Different Evaluation Scenarios. Proceedings of the International Conference of Design, User Experience, and Usability, Springer.","DOI":"10.1007\/978-3-319-07668-3_37"},{"key":"ref_35","first-page":"88","article-title":"Design and validation of a framework for the creation of user experience questionnaires","volume":"5","author":"Schrepp","year":"2019","journal-title":"Int. J. Interact. Multimed. Artif. Intell."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1080\/10447310802205776","article-title":"An empirical evaluation of the system usability scale","volume":"24","author":"Bangor","year":"2008","journal-title":"Int. J. Hum.\u2013Comput. Interact."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/BF02310555","article-title":"Coefficient alpha and the internal structure of tests","volume":"16","author":"Cronbach","year":"1951","journal-title":"Psychometrika"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"752","DOI":"10.1038\/nrn3122","article-title":"Virtual reality in neuroscience research and therapy","volume":"12","author":"Bohil","year":"2011","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_39","first-page":"40","article-title":"Construction of a benchmark for the user experience questionnaire (UEQ)","volume":"4","author":"Schrepp","year":"2017","journal-title":"Int. J. Interact. Multim. Artif. Intell."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5867\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:53:32Z","timestamp":1760165612000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5867"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,31]]},"references-count":39,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21175867"],"URL":"https:\/\/doi.org\/10.3390\/s21175867","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,31]]}}}