{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:28:39Z","timestamp":1760236119470,"version":"build-2065373602"},"reference-count":32,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2021,10,22]],"date-time":"2021-10-22T00:00:00Z","timestamp":1634860800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","award":["RGPIN-2018-05917"],"award-info":[{"award-number":["RGPIN-2018-05917"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Direct ophthalmoscopy (DO) is a medical procedure whereby a health professional, using a direct ophthalmoscope, examines the eye fundus. DO skills are in decline due to the use of interactive diagnostic equipment and insufficient practice with the direct ophthalmoscope. To address the loss of DO skills, physical and computer-based simulators have been developed to offer additional training. Among the computer-based simulations, virtual and augmented reality (VR and AR, respectively) allow simulated immersive and interactive scenarios with eye fundus conditions that are difficult to replicate in the classroom. VR and AR require employing 3D user interfaces (3DUIs) to perform the virtual eye examination. Using a combination of a between-subjects and within-subjects paradigm with two groups of five participants, this paper builds upon a previous preliminary usability study that compared the use of the HTC Vive controller, the Valve Index controller, and the Microsoft HoloLens 1 hand gesticulation interaction methods when performing a virtual direct ophthalmoscopy eye examination. The work described in this paper extends our prior work by considering the interactions with the Oculus Quest controller and Oculus Quest hand-tracking system to perform a virtual direct ophthalmoscopy eye examination while allowing us to compare these methods without our prior interaction techniques. Ultimately, this helps us develop a greater understanding of usability effects for virtual DO examinations and virtual reality in general. Although the number of participants was limited, n = 5 for Stage 1 (including the HTC Vive controller, the Valve Index controller, and the Microsoft HoloLens hand gesticulations), and n = 13 for Stage 2 (including the Oculus Quest controller and the Oculus Quest hand tracking), given the COVID-19 restrictions, our initial results comparing VR and AR 3D user interactions for direct ophthalmoscopy are consistent with our previous preliminary study where the physical controllers resulted in higher usability scores, while the Oculus Quest\u2019s more accurate hand motion capture resulted in higher usability when compared to the Microsoft HoloLens hand gesticulation.<\/jats:p>","DOI":"10.3390\/mti5110066","type":"journal-article","created":{"date-parts":[[2021,10,24]],"date-time":"2021-10-24T22:07:11Z","timestamp":1635113231000},"page":"66","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Virtual and Augmented Reality Direct Ophthalmoscopy Tool: A Comparison between Interactions Methods"],"prefix":"10.3390","volume":"5","author":[{"given":"Michael","family":"Chan","sequence":"first","affiliation":[{"name":"Software and Informatics Research Centre, Ontario Tech University, Oshawa, ON L1G 0C5, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6728-3848","authenticated-orcid":false,"given":"Alvaro","family":"Uribe-Quevedo","sequence":"additional","affiliation":[{"name":"Software and Informatics Research Centre, Ontario Tech University, Oshawa, ON L1G 0C5, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0434-3847","authenticated-orcid":false,"given":"Bill","family":"Kapralos","sequence":"additional","affiliation":[{"name":"Software and Informatics Research Centre, Ontario Tech University, Oshawa, ON L1G 0C5, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2969-0012","authenticated-orcid":false,"given":"Michael","family":"Jenkin","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering & Computer Science, York University, Toronto, ON M3J 1P3, Canada"}]},{"given":"Norman","family":"Jaimes","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, Universidad Militar Nueva Granada, Bogot\u00e1 110231, Colombia"}]},{"given":"Kamen","family":"Kanev","sequence":"additional","affiliation":[{"name":"Research Institute of Electronics, Shizuoka University, Hamamatsu 432-8011, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"338","DOI":"10.2298\/SARH170609142P","article-title":"The Application of Simulation in Medical Education\u2014Our Experience \u201cFrom Improvisation to Simulation\u201d","volume":"146","author":"Pavlovic","year":"2017","journal-title":"Srpski Arhiv Za Celokupno Lekarstvo"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1177\/1046878109355683","article-title":"Developing high-fidelity health care simulation scenarios: A guide for educators and professionals","volume":"42","author":"Alinier","year":"2011","journal-title":"Simul. Gaming"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1109\/TG.2020.3003315","article-title":"The anesthesia crisis scenario builder for authoring anesthesia crisis-based simulations","volume":"12","author":"Wilcocks","year":"2020","journal-title":"IEEE Trans. Games"},{"key":"ref_4","first-page":"1","article-title":"An Experimental Training Support Framework for Eye Fundus Examination Skill Development","volume":"Volume 63","author":"Nguyen","year":"2017","journal-title":"Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1212\/CPJ.0000000000000115","article-title":"The Demise of Direct Ophthalmoscopy","volume":"5","author":"Mackay","year":"2015","journal-title":"Neurol. Clin. Pract."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1093\/intqhc\/mzh065","article-title":"Fundus Anomalies: What the Pediatrician\u2019s Eye Can\u2019t See","volume":"16","author":"Morad","year":"2004","journal-title":"Int. J. Qual. Health Care"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Yan, H. (2018). Anatomy and Examination in Ocular Trauma, Springer.","DOI":"10.1007\/978-981-13-0068-4"},{"key":"ref_8","first-page":"375","article-title":"Addressing the \u2019Forgotten Art of Fundoscopy\u2019: Evaluation of a Novel Teaching Ophthalmoscope","volume":"30","author":"Schulz","year":"2016","journal-title":"Eye (Basingstoke)"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1842","DOI":"10.1016\/j.ophtha.2018.08.032","article-title":"Ophthalmology objectives for medical students: Revisiting what every graduating medical student should know","volume":"125","author":"Graubart","year":"2018","journal-title":"Ophthalmology"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"435","DOI":"10.2147\/AMEP.S108041","article-title":"Ophthalmoscopy Simulation: Advances in Training and Practice for Medical Students and Young Ophthalmologists","volume":"8","author":"Ricci","year":"2017","journal-title":"Adv. Med. Educ. Pract."},{"key":"ref_11","unstructured":"Acosta, D., Gu, D., Uribe-Quevedo, A., Kanev, K., Jenkin, M., Kapralos, B., and Jaimes, N. (2018). Mobile e-Training Tools for Augmented Reality Eye Fundus Examination. Interactive Mobile Communication, Technologies and Learning, Hamilton, Ontario, Canada, 11\u201312 October 2018, Springer."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Codd-Downey, R., Shewaga, R., Uribe-Quevedo, A., Kapralos, B., Kanev, K., and Jenkin, M. (2016, January 15\u201318). A Novel Tabletop and Tablet-Based Display System to Support Learner-Centric Ophthalmic Anatomy Education. Proceedings of the International Conference on Augmented Reality, Virtual Reality and Computer Graphics, Lecce, Italy.","DOI":"10.1007\/978-3-319-40651-0_1"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chan, M., Uribe-Quevedo, A., Kapralos, B., Jenkin, M., Kanev, K., and Jaimes, N. (2021). A Review of Virtual Reality-Based Eye Examination Simulators. Recent Advances in Technologies for Inclusive Well-Being: Virtual Patients, Gamification and Simulation, Springer.","DOI":"10.1007\/978-3-030-59608-8_6"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e66","DOI":"10.1055\/s-0041-1726349","article-title":"Comparing EYEsi Virtual Reality Simulator and Traditional Teaching Methods for Direct Ophthalmoscopy: Students\u2019 Perspectives at Indiana University School of Medicine","volume":"13","author":"Tso","year":"2021","journal-title":"J. Acad. Ophthalmol."},{"key":"ref_15","first-page":"44","article-title":"Evaluation of a virtual reality simulator for learning direct ophthalmoscopy in student teaching","volume":"117","author":"Boden","year":"2020","journal-title":"Der Ophthalmol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"617","DOI":"10.1016\/j.ophtha.2011.09.013","article-title":"Quality of Nonmydriatic Digital Fundus Photography Obtained by Nurse Practitioners in the Emergency Department: The FOTO-ED Study","volume":"119","author":"Lamirel","year":"2011","journal-title":"Ophtha"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"S47","DOI":"10.1097\/00001888-200010001-00015","article-title":"A controlled trial of an interactive, web-based virtual reality program for teaching physical diagnosis skills to medical students","volume":"75","author":"Grundman","year":"2000","journal-title":"Acad. Med."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Soto, C., Vargas, M., Uribe-Quevedo, A., Jaimes, N., and Kapralos, B. (2015, January 19\u201320). AR Stereoscopic 3D Human Eye Examination App. Proceedings of the 2015 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL), Thessalonik\u0113, Greece.","DOI":"10.1109\/IMCTL.2015.7359594"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"36","DOI":"10.5935\/0004-2749.20150010","article-title":"Evaluation of a Simulation Tool in Ophthalmology: Application in Teaching Funduscopy","volume":"78","author":"Androwiki","year":"2015","journal-title":"Arq. Bras. Oftalmol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1016\/j.ajo.2004.04.009","article-title":"A Simple Device for Teaching Direct Ophthalmoscopy to Primary Care Practitioners","volume":"138","author":"Chung","year":"2004","journal-title":"Am. J. Ophthalmol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e021","DOI":"10.1055\/s-0034-1396412","article-title":"The use and evaluation of an inexpensive eye model in direct ophthalmoscopy training","volume":"7","author":"Wu","year":"2014","journal-title":"J. Acad. Ophthalmol."},{"key":"ref_22","first-page":"1056","article-title":"Teaching Ophthalmoscopy To Medical Students (The TOTeMS Study)","volume":"125","author":"Kelly","year":"2013","journal-title":"Natl. Inst. Health"},{"key":"ref_23","first-page":"37","article-title":"An Eye Model for Practicing Ocular Exam Skills","volume":"21","author":"Kim","year":"2020","journal-title":"West. J. Emerg. Med. Integr. Emerg. Care Popul. Health"},{"key":"ref_24","unstructured":"Nguyen, M., Acosta, D., Gu, D., Quevedo, A.U., Kapralos, B., Jenkin, M., and Kanev, K. (2017). Virtual Eye Fundus Examination. Cureus J. Med. Sci., 1. Available online: https:\/\/www.cureus.com\/posters\/1312-virtual-eye-fundus-examination."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Chan, M., Uribe-Quevedo, A., Kapralos, B., Jaimes, N., Jenkin, M., and Kanev, K. (2020, January 12\u201314). A Preliminary Usability Comparison of Augmented and Virtual Reality User Interactions for Direct Ophthalmoscopy. Proceedings of the 2020 IEEE 8th International Conference on Serious Games and Applications for Health (SeGAH), Vancouver, BC, Canada.","DOI":"10.1109\/SeGAH49190.2020.9201804"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e9","DOI":"10.1056\/NEJMvcm1308125","article-title":"Examination of the retina","volume":"373","author":"Liu","year":"2015","journal-title":"N. Engl. J. Med."},{"key":"ref_27","unstructured":"Alvr-Org, P. (2021, October 20). ALVR: Stream VR Games from Your PC to Your Headset via Wi-Fi. Available online: https:\/\/reposhub.com\/cpp\/game-development\/alvr-org-ALVR.html."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Drew, M.R., Falcone, B., and Baccus, W.L. (2018). What Does the System Usability Scale (SUS) Measure? Validation Using Think Aloud Verbalization and Behavioral Metrics, Springer. DUXU 2018: Design, User Experience, and Usability: Theory and Practice.","DOI":"10.1007\/978-3-319-91797-9_25"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.iccn.2018.01.004","article-title":"The NASA Task Load Index as a measure of overall workload among neonatal, paediatric and adult intensive care nurses","volume":"46","author":"Mara","year":"2018","journal-title":"Intensive Crit. Care Nurs."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Hart, S.G. (2006, January 1). NASA-task load index (NASA-TLX); 20 years later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Los Angeles, CA, USA.","DOI":"10.1037\/e577632012-009"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Nielsen, J., and Landauer, T.K. (1993, January 24\u201329). A mathematical model of the finding of usability problems. Proceedings of the INTERACT\u201993 and CHI\u201993 Conference on Human Factors in Computing Systems, Amsterdam, The Netherlands.","DOI":"10.1145\/169059.169166"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1149","DOI":"10.3758\/BRM.41.4.1149","article-title":"Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses","volume":"41","author":"Faul","year":"2009","journal-title":"Behav. Res. Methods"}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/5\/11\/66\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:20:53Z","timestamp":1760167253000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/5\/11\/66"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,22]]},"references-count":32,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["mti5110066"],"URL":"https:\/\/doi.org\/10.3390\/mti5110066","relation":{},"ISSN":["2414-4088"],"issn-type":[{"type":"electronic","value":"2414-4088"}],"subject":[],"published":{"date-parts":[[2021,10,22]]}}}