{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T19:00:20Z","timestamp":1763578820090,"version":"3.37.3"},"reference-count":58,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2021,5,13]],"date-time":"2021-05-13T00:00:00Z","timestamp":1620864000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100007567","name":"City University of Hong Kong","doi-asserted-by":"publisher","award":["6000703"],"award-info":[{"award-number":["6000703"]}],"id":[{"id":"10.13039\/100007567","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,5,13]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The integration of building information modeling (BIM) and virtual reality (VR) has attracted increasing attention in the architecture engineering and construction (AEC) industry. Despite the rapid emergence of BIM-based VR applications, no evaluation framework specialized for the technologies exists in the AEC industry. After extensively reviewing existing studies and interviewing experts, the research team proposed an evaluation framework for BIM-based VR applications, which consists of 3 stages, 5 areas, 14 criteria, and 29 metrics and focuses on the design phase of the projects. To assess the usefulness of the framework, the team applied it to five BIM-based VR applications using a BIM-based design project for an educational building in Hong Kong. The team also interviewed experts to discuss the comprehensiveness of the framework. The results show that the framework provides consistent results for comprehensive evaluation criteria and metrics in a quantitative and flexible manner. Further research that considers additional criteria and\/or metrics related to other phases (e.g. construction phase) is required to extend the framework to the whole lifecycle.<\/jats:p>","DOI":"10.1093\/jcde\/qwab022","type":"journal-article","created":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T12:22:18Z","timestamp":1616502138000},"page":"910-922","source":"Crossref","is-referenced-by-count":22,"title":["Evaluation framework for BIM-based VR applications in design phase"],"prefix":"10.1093","volume":"8","author":[{"given":"Jung In","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Architecture and Civil Engineering, City University of Hong Kong, Tat Chee Avenue, Kowloon 999077, Hong Kong SAR"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1649-8280","authenticated-orcid":false,"given":"Sining","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Architecture and Civil Engineering, City University of Hong Kong, Tat Chee Avenue, Kowloon 999077, Hong Kong SAR"}]},{"given":"Xingbin","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Water Conservancy and Civil Engineering, South China Agricultural University, Tianhe District, Guangzhou, Guangdong 510642, China"}]},{"given":"Calvin","family":"Keung","sequence":"additional","affiliation":[{"name":"Department of Architecture and Civil Engineering, City University of Hong Kong, Tat Chee Avenue, Kowloon 999077, Hong Kong SAR"}]},{"given":"Minjae","family":"Suh","sequence":"additional","affiliation":[{"name":"Department of Engineering Technology, Sam Houston State University, Huntsville, TX 77341, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7887-1055","authenticated-orcid":false,"given":"Tae Wan","family":"Kim","sequence":"additional","affiliation":[{"name":"Division of Architecture and Urban Design, Incheon National University, 119 Academy-ro, Yeonsu-gu, Incheon 22012, South Korea"}]}],"member":"286","published-online":{"date-parts":[[2021,5,13]]},"reference":[{"key":"2021070811150888900_bib1","first-page":"127","article-title":"Interoperability of building information - Myth of reality","volume-title":"Proceedings of the 6th European Conference on Product and Process Modelling - EWork and EBusiness in Architecture, Engineering and Construction (ECPPM 2006)","author":"Aranda-Mena","year":"2006"},{"key":"2021070811150888900_bib2","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1145\/91385.91409","article-title":"Reality built for two: A virtual reality tool","volume-title":"Proceedings of the 1990 Symposium on Interactive 3D Graphics","author":"Blanchard","year":"1990"},{"issue":"2","key":"2021070811150888900_bib3","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1162\/105474699566107","article-title":"A conceptual model of the sense of presence in virtual environments","volume":"8","author":"Bystrom","year":"1999","journal-title":"Teleoperators and Virtual Environments"},{"issue":"1","key":"2021070811150888900_bib4","doi-asserted-by":"crossref","first-page":"60","DOI":"10.18178\/ijimt.2018.9.1.788","article-title":"Lean, BIM and augmented reality applied in the design and construction phase: A literature review","volume":"9","author":"Calderon-Hernandez","year":"2018","journal-title":"International Journal of Innovation, Management and Technology"},{"key":"2021070811150888900_bib5","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1109\/VRAIS.1993.380753","article-title":"DIVE-a multi-user virtual reality system","volume-title":"Proceedings of IEEE Virtual Reality Annual International Symposium","author":"Carlsson","year":"1993"},{"issue":"4","key":"2021070811150888900_bib6","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1016\/j.culher.2010.04.001","article-title":"Beyond virtual museums: Experiencing immersive virtual reality in real museums","volume":"11","author":"Carrozzino","year":"2010","journal-title":"Journal of Cultural Heritage"},{"issue":"17","key":"2021070811150888900_bib3_23_1618919540146","doi-asserted-by":"crossref","first-page":"1658","DOI":"10.1080\/10447318.2020.1778351","article-title":"Virtual reality sickness: a review of causes and measurements","volume":"36","author":"Chang","year":"2020","journal-title":"International Journal of Human\u2013Computer Interaction"},{"volume-title":"Integrating BIM and Virtual Reality \u2013 The Next Big Thing","year":"2016","author":"Cherian","key":"2021070811150888900_bib7"},{"key":"2021070811150888900_bib8","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/j.autcon.2017.10.009","article-title":"Zero latency: Real-time synchronization of BIM data in virtual reality for collaborative decision-making","volume":"85","author":"Du","year":"2018","journal-title":"Automation in Construction"},{"key":"2021070811150888900_bib9","first-page":"1","article-title":"Simultaneous data exchange between BIM and VR for collaborative decision making","volume-title":"Congress on Computing in Civil Engineering, Proceedings","author":"Du","year":"2017"},{"year":"2016","author":"EnscapeTM - Real-Time Rendering for Revit, SketchUp, Rhino & ArchiCad","key":"2021070811150888900_bib10"},{"issue":"4","key":"2021070811150888900_bib11","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1016\/j.aei.2012.06.003","article-title":"An approach to assessing virtual environments for synchronous and remote collaborative design","volume":"26","author":"Germani","year":"2012","journal-title":"Advanced Engineering Informatics"},{"key":"2021070811150888900_bib12","first-page":"1325","article-title":"Metrics-based approach for VR technology evaluation in styling product design","volume-title":"Proceedings of the ASME Design Engineering Technical Conference, 49026","author":"Germani","year":"2009"},{"issue":"9","key":"2021070811150888900_bib13","doi-asserted-by":"crossref","first-page":"e0184682","DOI":"10.1371\/journal.pone.0184682","article-title":"Evaluation of a conceptual framework for predicting navigation performance in virtual reality","volume":"12","author":"Gr\u00fcbel","year":"2017","journal-title":"Plos One"},{"issue":"3","key":"2021070811150888900_bib14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3301423","article-title":"Automated usability evaluation of virtual reality applications","volume":"26","author":"Harms","year":"2019","journal-title":"ACM Transactions on Computer-Human Interaction"},{"issue":"2","key":"2021070811150888900_bib15","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.jcde.2018.06.001","article-title":"BIM adoption across the Chinese AEC industries: An extended BIM adoption model","volume":"6","author":"Herr","year":"2019","journal-title":"Journal of Computational Design and Engineering"},{"key":"2021070811150888900_bib16","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1016\/j.autcon.2015.03.020","article-title":"Immersive virtual environments versus physical built environments: A benchmarking study for building design and user-built environment explorations","volume":"54","author":"Heydarian","year":"2015","journal-title":"Automation in Construction"},{"key":"2021070811150888900_bib18","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1145\/351006.351018","article-title":"Collaboration and learning within immersive virtual reality","volume-title":"Proceedings of the Third International Conference on Collaborative Virtual Environments","author":"Jackson","year":"2000"},{"issue":"3","key":"2021070811150888900_bib19","doi-asserted-by":"crossref","first-page":"04016100","DOI":"10.1061\/(ASCE)CO.1943-7862.0001233","article-title":"VDC scorecard: Formulation, application, and validation","volume":"143","author":"Kam","year":"2017","journal-title":"Journal of Construction Engineering and Management"},{"key":"2021070811150888900_bib20","first-page":"111","article-title":"Virtual object manipulation on a table-top AR environment","volume-title":"Proceedings - IEEE and ACM International Symposium on Augmented Reality, ISAR 2000","author":"Kato","year":"2000"},{"issue":"4","key":"2021070811150888900_bib21","doi-asserted-by":"crossref","first-page":"e16724","DOI":"10.2196\/16724","article-title":"Multidimensional evaluation of virtual reality paradigms in clinical neuropsychology: Application of the VR-check framework","volume":"22","author":"Krohn","year":"2020","journal-title":"Journal of Medical Internet Research"},{"issue":"18","key":"2021070811150888900_bib22","doi-asserted-by":"crossref","first-page":"1154","DOI":"10.1177\/154193129403801802","article-title":"Side effects and aftereffects of immersion in virtual environments","volume":"38","author":"Lampton","year":"1994","journal-title":"Proceedings of the Human Factors and Ergonomics Society"},{"key":"2021070811150888900_bib4_868_1618919546128","first-page":"1","article-title":"MSVT: A virtual reality-based multimodal scientific visualization tool","author":"LaViola","year":"2000,","journal-title":"Proceedings of the third IASTED international conference on computer graphics and imaging"},{"key":"2021070811150888900_bib23","doi-asserted-by":"crossref","first-page":"498","DOI":"10.1016\/j.autcon.2011.11.009","article-title":"Visualizing safety assessment by integrating the use of game technology","volume":"22","author":"Li","year":"2012","journal-title":"Automation in Construction"},{"volume-title":"Virtually usable: A review of virtual reality usability evaluation methods","year":"2016","author":"Martens","key":"2021070811150888900_bib24"},{"key":"2021070811150888900_bib25","article-title":"The three pillars of virtual reality?","volume-title":"Investigating the Roles of Immersion, Presence, and Interactivity. Proceedings of the 51st Hawaii International Conference on System Sciences","author":"M\u00fctterlein","year":"2018"},{"issue":"2.21","key":"2021070811150888900_bib26","doi-asserted-by":"crossref","first-page":"156","DOI":"10.14419\/ijet.v7i2.21.11858","article-title":"Virtual reality (VR) & augmented reality (AR) technologies for tourism and hospitality industry","volume":"7","author":"Nayyar","year":"2018","journal-title":"International Journal of Engineering & Technology"},{"issue":"4","key":"2021070811150888900_bib27","doi-asserted-by":"crossref","first-page":"B130","DOI":"10.1364\/JOSAA.35.00B130","article-title":"Correlation between perception of color, shadows, and surface textures and the realism of a scene in virtual reality","volume":"35","author":"Pardo","year":"2018","journal-title":"JOSA A"},{"issue":"1","key":"2021070811150888900_bib28","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1007\/BF02009710","article-title":"An investigation into nausea and other side-effects of head-coupled immersive virtual reality","volume":"1","author":"Regan","year":"1995","journal-title":"Virtual Reality"},{"issue":"6","key":"2021070811150888900_bib29","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1007\/978-1-4899-0038-8_9","article-title":"Some effects of using virtual reality technology","volume":"65","author":"Regan","year":"1997","journal-title":"Virtual Reality, Training's Future"},{"issue":"1","key":"2021070811150888900_bib30","doi-asserted-by":"crossref","first-page":"18","DOI":"10.2174\/1875323X01002010018","article-title":"Virtual reality technology used in civil engineering education","volume":"2","author":"Sampaio","year":"2010","journal-title":"The Open Virtual Reality Journal"},{"key":"2021070811150888900_bib31","first-page":"59","article-title":"Enhancing BIM methodology with VR technology","author":"Sampaio","year":"2018","journal-title":"State of the Art Virtual Reality and Augmented Reality Knowhow"},{"key":"2021070811150888900_bib32","first-page":"121","article-title":"Management of building supported on virtual interactive models: Construction planning and preventive maintenance","volume":"17","author":"Sampaio","year":"2012","journal-title":"Electronic Journal of Information Technology in Construction"},{"issue":"2","key":"2021070811150888900_bib33","first-page":"1","article-title":"Evaluation of the design of the aedem programme for emotional education in secondary education","volume":"22","author":"S\u00e1nchez\u00a0Calleja","year":"2016","journal-title":"Revista Electronica de Investigacion y Evaluacion Educativa"},{"issue":"Special Issue","key":"2021070811150888900_bib34","doi-asserted-by":"crossref","first-page":"254","DOI":"10.3763\/aedm.2010.IDDS3","article-title":"Tool for benchmarking BIM performance of design, engineering and construction firms in the Netherlands","volume":"6","author":"Sebastian","year":"2010","journal-title":"Architectural Engineering and Design Management"},{"issue":"3","key":"2021070811150888900_bib35","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1682\/JRRD.2015.03.0045","article-title":"Usability evaluation of low-cost virtual reality hand and arm rehabilitation games","volume":"53","author":"Seo","year":"2016","journal-title":"Journal of Rehabilitation Research and Development"},{"key":"2021070811150888900_bib36","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1016\/j.proeng.2016.04.029","article-title":"A multiuser shared virtual environment for facility management","volume":"145","author":"Shi","year":"2016","journal-title":"Procedia Engineering"},{"issue":"6","key":"2021070811150888900_bib37","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1093\/jcde\/qwaa057","article-title":"Applying BIM to support dispute avoidance in managing multi-owned buildings","volume":"7","author":"Shin","year":"2020","journal-title":"Journal of Computational Design and Engineering"},{"issue":"2","key":"2021070811150888900_bib38","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1007\/s11831-019-09386-0","article-title":"Recent tools and techniques of BIM-based virtual reality: A systematic review","volume":"28","author":"Sidani","year":"2019","journal-title":"Archives of Computational Methods in Engineering"},{"issue":"5","key":"2021070811150888900_bib39","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1093\/jcde\/qwaa040","article-title":"I walk, therefore I am: A multidimensional study on the influence of the locomotion method upon presence in virtual reality","volume":"7","author":"Soler-Dom\u00ednguez","year":"2020","journal-title":"Journal of Computational Design and Engineering"},{"key":"2021070811150888900_bib1_486_1618919526847","first-page":"125","article-title":"On the narrative structure of Virtual Reality walkthroughs, Proceedings of the 9th International Conference on Computer Aided Architectural Design Futures, Eindhoven, Netherlands","author":"Stappers","year":"2001"},{"key":"2021070811150888900_bib40","doi-asserted-by":"crossref","first-page":"65","DOI":"10.4018\/978-1-60566-928-1.ch004","article-title":"Building information modelling maturity matrix","volume-title":"Handbook of research on building information modeling and construction informatics: Concepts and technologies","author":"Succar","year":"2010"},{"issue":"2","key":"2021070811150888900_bib41","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/j.jcde.2018.05.006","article-title":"Navigation modes, operation methods, observation scales and background options in UI design for high learning performance in VR-based architectural applications","volume":"6","author":"Sun","year":"2019","journal-title":"Journal of Computational Design and Engineering"},{"issue":"6","key":"2021070811150888900_bib42","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1080\/014492900750052679","article-title":"Evaluating the usability of virtual reality user interfaces","volume":"19","author":"Sutcliffe","year":"2000","journal-title":"Behaviour and Information Technology, Behaviour & Information Technology"},{"key":"2021070811150888900_bib43","doi-asserted-by":"crossref","first-page":"130883","DOI":"10.1109\/ACCESS.2019.2940073","article-title":"Beyond feeling sick: The visual and cognitive aftereffects of virtual reality","volume":"7","author":"Szpak","year":"2019","journal-title":"IEEE Access"},{"key":"2021070811150888900_bib44","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.autcon.2013.03.004","article-title":"Location tracking and data visualization technology to advance construction ironworkers\u2019 education and training in safety and productivity","volume":"35","author":"Teizer","year":"2013","journal-title":"Automation in Construction"},{"key":"2021070811150888900_bib2_764_1618919532245","first-page":"25","article-title":"Virtual reality in construction: a review. Engineering computational technology","author":"Thabet","year":"2002","journal-title":"Civil-Comp press"},{"key":"2021070811150888900_bib45","doi-asserted-by":"crossref","DOI":"10.1109\/HICSS.2003.1174239","article-title":"The gender factor in virtual reality navigation and wayfinding","volume-title":"Proceedings of the 36th Annual Hawaii International Conference on System Sciences","author":"Vila","year":"2003"},{"issue":"2","key":"2021070811150888900_bib46","doi-asserted-by":"crossref","first-page":"136","DOI":"10.3724\/SP.J.2096-5796.2019.0008","article-title":"Haptic display for virtual reality: progress and challenges","volume":"1","author":"Wang","year":"2019","journal-title":"Virtual Reality & Intelligent Hardware"},{"issue":"3","key":"2021070811150888900_bib47","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1016\/j.autcon.2005.06.002","article-title":"Compatibility issues in augmented reality systems for AEC: An experimental prototype study","volume":"15","author":"Wang","year":"2006","journal-title":"Automation in Construction"},{"key":"2021070811150888900_bib48","first-page":"573","article-title":"Assessing user experience in virtual reality \u2013 A comparison of different measurements","volume-title":"Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)","author":"Wienrich","year":"2018"},{"issue":"12","key":"2021070811150888900_bib49","doi-asserted-by":"crossref","first-page":"3516","DOI":"10.1109\/TCSVT.2018.2886277","article-title":"Assessing visual quality of omnidirectional videos","volume":"29","author":"Xu","year":"2018","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"2021070811150888900_bib50","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1109\/ISMAR.2015.12","article-title":"A framework to evaluate omnidirectional video coding schemes","volume-title":"2015 IEEE International Symposium on Mixed and Augmented Reality","author":"Yu","year":"2015"},{"issue":"1","key":"2021070811150888900_bib51","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40327-018-0065-6","article-title":"Virtual reality-integrated workflow in BIM-enabled projects collaboration and design review: A case study","volume":"6","author":"Zaker","year":"2018","journal-title":"Visualization in Engineering"},{"key":"2021070811150888900_bib52","first-page":"99700C","article-title":"Quality metric for spherical panoramic video","volume-title":"Optics and Photonics for Information Processing X","author":"Zakharchenko","year":"2016"},{"issue":"2","key":"2021070811150888900_bib53","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1007\/s11633-005-0144-8","article-title":"A hierarchical framework for visualising and simulating supply chains in virtual environments","volume":"2","author":"Zhang","year":"2005","journal-title":"International Journal of Automation and Computing"},{"issue":"2","key":"2021070811150888900_bib54","doi-asserted-by":"crossref","first-page":"229","DOI":"10.2319\/051511-333.1","article-title":"Validity and reliability of a method for assessment of cervical vertebral maturation","volume":"82","author":"Zhao","year":"2012","journal-title":"The Angle Orthodontist"},{"issue":"5","key":"2021070811150888900_bib55","doi-asserted-by":"crossref","first-page":"04017028","DOI":"10.1061\/(ASCE)CP.1943-5487.0000679","article-title":"Emotional Response\u2013Based approach for assessing the sense of presence of subjects in virtual building evacuation studies","volume":"31","author":"Zou","year":"2017","journal-title":"Journal of Computing in Civil Engineering"}],"container-title":["Journal of Computational Design and Engineering"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jcde\/article-pdf\/8\/3\/910\/37931344\/qwab022.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jcde\/article-pdf\/8\/3\/910\/37931344\/qwab022.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,8]],"date-time":"2021-07-08T19:59:17Z","timestamp":1625774357000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jcde\/article\/8\/3\/910\/6275212"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,13]]},"references-count":58,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,5,13]]}},"URL":"https:\/\/doi.org\/10.1093\/jcde\/qwab022","relation":{},"ISSN":["2288-5048"],"issn-type":[{"type":"electronic","value":"2288-5048"}],"subject":[],"published-other":{"date-parts":[[2021,6]]},"published":{"date-parts":[[2021,5,13]]}}}