{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T20:59:16Z","timestamp":1790629156485,"version":"4.1.0"},"reference-count":37,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Educ"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Extended reality technologies, including virtual reality (VR), augmented reality, and mixed reality, are increasingly used in medical education to create immersive and interactive learning environments. As these modalities expand, validated instruments are needed to measure learners\u2019 experiences accurately. The Immersive Technology Evaluation Measure (ITEM) is a multidomain questionnaire assessing immersion, motivation, cognitive load, usability, and debriefing. Although cognitive interviewing informed its original development, less is known about how ITEM questions function when used in a different linguistic and educational context.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to evaluate the clarity, comprehensibility, and response processes of ITEM among native Arabic-speaking medical students enrolled in an English-medium medical program. We also sought to identify linguistic, referential, and contextual sources of comprehension difficulty and use these findings to inform proposed item-level refinements.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We conducted a qualitative cognitive interviewing study with 10 third-year and fourth-year medical students at United Arab Emirates University following VR-based clinical skills activities. Using concurrent think-aloud and verbal probing techniques, participants explained how they interpreted each ITEM question and arrived at their responses. Interview transcripts, recordings, and interviewer notes were independently reviewed using a descriptive, item-focused approach. Participant feedback was examined for recurring comprehension problems, and proposed item-level decisions were reviewed through research team consensus. Item-level saturation was reached after 8 interviews and confirmed with 2 additional interviews.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Participants identified comprehension or response-process problems in 47.5% (19\/40) of ITEM questions. Difficulties occurred across all 5 domains and were most frequent in immersion (6\/9, 66.7%) and usability (6\/10, 60%), followed by debriefing (4\/5, 80%), motivation (2\/10, 20%), and cognitive load (1\/6, 16.7%). Common difficulties involved nonspecific references such as \u201cactivity\u201d and \u201ctechnology,\u201d unfamiliar or abstract terminology, negative wording, and unclear temporal or contextual framing. For example, \u201cconcern\u201d was interpreted by some participants as worry rather than focus, and \u201cmentally demanding\u201d was interpreted in relation to mental health rather than cognitive effort. Cognitive interviewing informed different item-level decisions rather than a uniform revision: 11 problematic questions received proposed wording or contextual revisions, 2 received presentation-only modifications, and 6 were retained unchanged where clarification could introduce a meaning not clearly established in the original item. Six additional questions received limited contextual or terminology-standardizing changes for consistency. No items were removed, and the original 5-domain structure was retained.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Cognitive interviewing identified response-process difficulties that were not apparent from the questionnaire wording alone and provided a systematic basis for determining when clarification was warranted and when the original wording should be preserved. The findings extend response-process evidence for ITEM and illustrate the value of examining established educational measures in new linguistic and educational settings. The proposed refinements provide a foundation for further cognitive testing and psychometric evaluation across immersive learning contexts.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/95904","type":"journal-article","created":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T20:09:53Z","timestamp":1790626193000},"page":"e95904-e95904","source":"Crossref","is-referenced-by-count":0,"title":["Refining the Immersive Technology Evaluation Measure for Virtual Reality Clinical Skills Among Native Arabic-Speaking Medical Students: Qualitative Study"],"prefix":"10.2196","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5981-7133","authenticated-orcid":false,"given":"Afaf Sulaiman","family":"Alblooshi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-4944-4165","authenticated-orcid":false,"given":"Falah Mohammed","family":"Almarzooqi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5723-4970","authenticated-orcid":false,"given":"Taleb M","family":"Almansoori","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-4718-3016","authenticated-orcid":false,"given":"Anmol","family":"Punn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5906-9228","authenticated-orcid":false,"given":"Marwa Gaffar","family":"Alameen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5434-9705","authenticated-orcid":false,"given":"Alexander","family":"Kieu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7835-8681","authenticated-orcid":false,"given":"Faten Abdullah","family":"AlRadini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,9,28]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","DOI":"10.1186\/s12909-024-05777-5","article-title":"Reviewing the current state of virtual reality integration in medical education - a scoping review","volume":"24","author":"Mergen","journal-title":"BMC Med Educ"},{"key":"R2","doi-asserted-by":"publisher","DOI":"10.3389\/fdgth.2024.1365345","article-title":"Virtual reality and augmented reality in medical education: an umbrella review","volume":"6","author":"Tene","journal-title":"Front Digit Health"},{"issue":"19","key":"R3","doi-asserted-by":"publisher","first-page":"8520","DOI":"10.3390\/su16198520","article-title":"Effectiveness of virtual reality in healthcare education: systematic review and meta-analysis","volume":"16","author":"Sung","journal-title":"Sustainability"},{"issue":"1","key":"R4","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1007\/s10639-022-11167-5","article-title":"Systematic literature review and bibliometric analysis on virtual reality and education","volume":"28","author":"Rojas-S\u00e1nchez","journal-title":"Educ Inf Technol"},{"key":"R5","doi-asserted-by":"crossref","unstructured":"Forrest K McKimm J . Healthcare Simulation at a Glance. 1st ed. John Wiley & Sons; 2019. ISBN: 9781118871843","DOI":"10.1002\/9781119604020"},{"issue":"2","key":"R6","doi-asserted-by":"publisher","DOI":"10.2196\/30082","article-title":"Learning outcomes of immersive technologies in health care student education: systematic review of the literature","volume":"24","author":"Ryan","journal-title":"J Med Internet Res"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.1016\/j.cexr.2023.100027","article-title":"Cognitive interviewing as a method to inform questionnaire design and validity - Immersive Technology Evaluation Measure (ITEM) for healthcare education","volume":"2","author":"Jacobs","journal-title":"Comput Educ X Real"},{"key":"R8","doi-asserted-by":"crossref","unstructured":"Brooke J . SUS: a 'quick and dirty' usability scale. In: Jordan PW Thomas B McClelland IL Weerdmeester B , editors. Usability Evaluation in Industry. CRC Press; 1996. [doi: 10.1201\/9781498710411]","DOI":"10.1201\/9781498710411"},{"issue":"3","key":"R9","doi-asserted-by":"publisher","first-page":"319","DOI":"10.2307\/249008","article-title":"Perceived usefulness, perceived ease of use, and user acceptance of information technology","volume":"13","author":"Davis","journal-title":"MIS Q"},{"issue":"3","key":"R10","doi-asserted-by":"publisher","first-page":"266","DOI":"10.1162\/105474601300343603","article-title":"The experience of presence: factor analytic insights","volume":"10","author":"Schubert","journal-title":"Presence Teleoperators Virtual Environ"},{"issue":"9","key":"R11","doi-asserted-by":"publisher","first-page":"904","DOI":"10.1177\/154193120605000909","article-title":"NASA-task load index (NASA-TLX); 20 years later","volume":"50","author":"Hart","journal-title":"Proc Hum Factors Ergon Soc Annu Meet"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1080\/02701367.1989.10607413","article-title":"Psychometric properties of the intrinsic motivation inventory in a competitive sport setting: a confirmatory factor analysis","volume":"60","author":"McAuley","journal-title":"Res Q Exerc Sport"},{"issue":"9","key":"R13","doi-asserted-by":"publisher","first-page":"641","DOI":"10.1016\/j.ijhcs.2008.04.004","article-title":"Measuring and defining the experience of immersion in games","volume":"66","author":"Jennett","journal-title":"Int J Hum Comput Stud"},{"issue":"3","key":"R14","doi-asserted-by":"publisher","first-page":"163","DOI":"10.30476\/JAMP.2022.95226.1632","article-title":"Developing measures of immersion and motivation for learning technologies in healthcare simulation: a pilot study","volume":"10","author":"Jacobs","journal-title":"J Adv Med Educ Prof"},{"issue":"3","key":"R15","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1037\/0022-3514.43.3.450","article-title":"Control and information in the intrapersonal sphere: an extension of cognitive evaluation theory","volume":"43","author":"Ryan","journal-title":"J Pers Soc Psychol"},{"issue":"2","key":"R16","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1097\/SIH.0000000000000072","article-title":"Promoting Excellence and Reflective Learning in Simulation (PEARLS): development and rationale for a blended approach to health care simulation debriefing","volume":"10","author":"Eppich","journal-title":"Simul Healthc"},{"issue":"4","key":"R17","doi-asserted-by":"publisher","first-page":"295","DOI":"10.3109\/01421599709034208","article-title":"Development and validation of the Dundee Ready Education Environment Measure (DREEM)","volume":"19","author":"Roff","journal-title":"Med Teach"},{"issue":"2","key":"R18","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1097\/00001888-197802000-00003","article-title":"Measuring the medical school learning environment","volume":"53","author":"Marshall","journal-title":"J Med Educ"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1080\/03075079712331381121","article-title":"The development, validation and application of the Course Experience Questionnaire","volume":"22","author":"Wilson","journal-title":"Stud High Educ"},{"key":"R20","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6920-14-44","article-title":"Development of the Clinical Learning Evaluation Questionnaire for undergraduate clinical education: factor structure, validity, and reliability study","volume":"14","author":"AlHaqwi","journal-title":"BMC Med Educ"},{"key":"R21","doi-asserted-by":"crossref","unstructured":"Tourangeau R Rips LJ Rasinski K . The Psychology of Survey Response. Cambridge University Press; 2000. [doi: 10.1017\/CBO9780511819322]","DOI":"10.1017\/CBO9780511819322"},{"key":"R22","unstructured":"Willis GB . Cognitive Interviewing: A Tool for Improving Questionnaire Design. SAGE Publications; 2004. ISBN: 9780761928034"},{"issue":"3","key":"R23","doi-asserted-by":"publisher","first-page":"353","DOI":"10.4300\/JGME-D-13-00154.1","article-title":"What do our respondents think we\u2019re asking? Using cognitive interviewing to improve medical education surveys","volume":"5","author":"Willis","journal-title":"J Grad Med Educ"},{"key":"R24","unstructured":"Tourangeau R . Cognitive sciences and survey methods. In: Jabine TB Straf ML Tanur JM Tourangeau R , editors. Cognitive Aspects of Survey Methodology: Building a Bridge between Disciplines. National Academies Press; 1984:73-100. ISBN: 9780309077842"},{"issue":"2","key":"R25","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1093\/poq\/nfm006","article-title":"Research synthesis: the practice of cognitive interviewing","volume":"71","author":"Beatty","journal-title":"Public Opin Q"},{"issue":"3","key":"R26","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1037\/0033-295X.87.3.215","article-title":"Verbal reports as data","volume":"87","author":"Ericsson","journal-title":"Psychol Rev"},{"key":"R27","doi-asserted-by":"crossref","unstructured":"Forsyth BH Lessler JT . Cognitive laboratory methods: a taxonomy. In: Biemer PP Groves RM Lyberg LE Mathiowetz NA Sudman S , editors. Measurement Errors in Surveys. John Wiley & Sons; 2004:393-418. [doi: 10.1002\/9781118150382]","DOI":"10.1002\/9781118150382.ch20"},{"issue":"3","key":"R28","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1027\/1614-2241\/a000075","article-title":"The Cognitive Interviewing Reporting Framework (CIRF): towards the harmonization of cognitive testing reports","volume":"9","author":"Boeije","journal-title":"Methodology"},{"issue":"4","key":"R29","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1177\/1525822X11416092","article-title":"Cross-cultural cognitive interviewing: seeking comparability and enhancing understanding","volume":"23","author":"Willis","journal-title":"Field Methods"},{"issue":"6","key":"R30","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1177\/0269216315608348","article-title":"Discovering the hidden benefits of cognitive interviewing in two languages: the first phase of a validation study of the Integrated Palliative care Outcome Scale","volume":"30","author":"Schildmann","journal-title":"Palliat Med"},{"issue":"4","key":"R31","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1177\/1525822X16630262","article-title":"Influence of English-language proficiency on the cognitive processing of survey questions","volume":"28","author":"Park","journal-title":"Field Methods"},{"issue":"1","key":"R32","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1093\/jssam\/smz045","article-title":"Language proficiency among respondents: implications for data quality in a longitudinal face-to-face survey","volume":"9","author":"Wenz","journal-title":"J Surv Stat Methodol"},{"issue":"1","key":"R33","doi-asserted-by":"publisher","DOI":"10.1186\/s12909-022-03986-4","article-title":"Promoting physician-patient language concordance in undergraduate medical education: a peer assisted learning approach","volume":"23","author":"Ismaiel","journal-title":"BMC Med Educ"},{"issue":"10","key":"R34","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.46777","article-title":"Language barriers in understanding healthcare information: Arabic-speaking students\u2019 comprehension of diabetic questionnaires in Arabic and English languages","volume":"15","author":"Gazzaz","journal-title":"Cureus"},{"issue":"2","key":"R35","doi-asserted-by":"publisher","first-page":"377","DOI":"10.1016\/j.jbi.2008.08.010","article-title":"Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support","volume":"42","author":"Harris","journal-title":"J Biomed Inform"},{"key":"R36","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2019.103208","article-title":"The REDCap consortium: building an international community of software platform partners","volume":"95","author":"Harris","journal-title":"J Biomed Inform"},{"issue":"9","key":"R37","doi-asserted-by":"publisher","first-page":"1289","DOI":"10.1177\/0193945916678212","article-title":"Questionnaire adapting: little changes mean a lot","volume":"39","author":"Sousa","journal-title":"West J Nurs Res"}],"container-title":["JMIR Medical Education"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T20:10:01Z","timestamp":1790626201000},"score":1,"resource":{"primary":{"URL":"https:\/\/mededu.jmir.org\/2026\/1\/e95904"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,28]]},"references-count":37,"URL":"https:\/\/doi.org\/10.2196\/95904","relation":{},"ISSN":["2369-3762"],"issn-type":[{"value":"2369-3762","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,28]]},"article-number":"v12i1e95904"}}