{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,31]],"date-time":"2026-01-31T10:16:36Z","timestamp":1769854596832,"version":"3.49.0"},"reference-count":45,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,4,10]],"date-time":"2024-04-10T00:00:00Z","timestamp":1712707200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Virtual Real."],"abstract":"<jats:p>Virtual reality (VR) has become an important interactive visualization tool for various industrial processes including facility inspection and maintenance. The capability of a VR application to present users with realistic simulations of complex systems and immerse them in, for example, inaccessible remote environments is often essential for using VR in real-world industrial domains. While many VR solutions have already been developed to support virtual facility inspection, previous systems provide immersive visualizations only with limited realism, because the real-world conditions of facilities are often difficult to reconstruct with accurate meshes and point clouds or typically too time-consuming to be consistently updated in computer-aided design (CAD) software toolkits. In this work, we present <jats:italic>Magic NeRF Lens<\/jats:italic>, a VR framework that supports immersive photorealistic visualizations of complex industrial facilities leveraging the recent advancement of neural radiance fields (NeRF). We introduce a data fusion technique to merge a NeRF model with the polygonal representation of it\u2019s corresponding CAD model, which optimizes VR NeRF rendering through magic-lens-style interactions while introducing a novel industrial visualization design that can support practical tasks such as facility maintenance planning and redesign. We systematically benchmarked the performance of our framework, investigated users\u2019 perceptions of the magic-lens-style visualization design through a visual search experiment to derive design insights, and performed an empirical evaluation of our system through expert reviews. To support further research and development of customized VR NeRF applications, the source code of the toolkit was made openly available.<\/jats:p>","DOI":"10.3389\/frvir.2024.1377245","type":"journal-article","created":{"date-parts":[[2024,4,10]],"date-time":"2024-04-10T04:24:40Z","timestamp":1712723080000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Magic NeRF lens: interactive fusion of neural radiance fields for virtual facility inspection"],"prefix":"10.3389","volume":"5","author":[{"given":"Ke","family":"Li","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Susanne","family":"Schmidt","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tim","family":"Rolff","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Reinhard","family":"Bacher","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wim","family":"Leemans","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Frank","family":"Steinicke","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,4,10]]},"reference":[{"key":"B1","volume-title":"Realityengine graphics","author":"Akeley","year":"2000"},{"key":"B2","article-title":"Silicon mirage: the art and science of virtual reality","author":"Aukstakalnis","year":"1992"},{"key":"B3","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1186\/2213-7459-1-3","article-title":"High-precision vision-based mobile augmented reality system for context-aware architectural, engineering, construction and facility management (aec\/fm) applications","volume":"1","author":"Bae","year":"2013","journal-title":"Vis. Eng."},{"key":"B4","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1109\/ISMAR.2012.6402557","article-title":"A hand-held ar magic lens with user-perspective rendering","volume-title":"2012 IEEE international symposium on mixed and augmented reality (ISMAR)","author":"Bar\u010devi\u0107","year":"2012"},{"key":"B5","doi-asserted-by":"crossref","first-page":"5835","DOI":"10.1109\/ICCV48922.2021.00580","article-title":"Mip-nerf: a multiscale representation for anti-aliasing neural radiance fields","volume-title":"2021 IEEE\/CVF international conference on computer vision (ICCV)","author":"Barron","year":"2021"},{"key":"B6","doi-asserted-by":"publisher","first-page":"586","DOI":"10.1117\/12.57955","article-title":"Method for registration of 3-d shapes","volume":"1611","author":"Besl","year":"1992","journal-title":"Sens. fusion IV control paradigms data Struct. (Spie)"},{"key":"B7","doi-asserted-by":"crossref","DOI":"10.1145\/166117.166126","article-title":"Toolglass and magic lenses: the see-through interface","volume-title":"Proceedings of the 20th annual conference on Computer graphics and interactive techniques","author":"Bier","year":"1993"},{"key":"B8","article-title":"Sus: a \u201cquick and dirty\u201d usability scale","volume-title":"Usability evaluation in industry 18","author":"Brooke","year":"1996"},{"key":"B9","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1109\/mcg.2006.84","article-title":"Magic lenses for augmented virtual environments","volume":"26","author":"Brown","year":"2006","journal-title":"IEEE Comput. Graph. Appl."},{"key":"B10","doi-asserted-by":"publisher","first-page":"486","DOI":"10.3390\/app10020486","article-title":"Programming of industrial robots using virtual reality and digital twins","volume":"10","author":"Burghardt","year":"2020","journal-title":"Appl. Sci."},{"key":"B11","doi-asserted-by":"crossref","DOI":"10.1145\/3056540.3076193","article-title":"The design space of augmented and virtual reality applications for assistive environments in manufacturing: a visual approach","volume-title":"Proceedings of the 10th international conference on PErvasive technologies related to assistive environments","author":"B\u00fcttner","year":"2017"},{"key":"B12","volume-title":"Tensorf: tensorial radiance fields","author":"Chen","year":"2022"},{"key":"B13","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR52729.2023.01590","article-title":"Mobilenerf: exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures","volume-title":"The conference on computer vision and pattern recognition (CVPR)","author":"Chen","year":"2023"},{"key":"B14","article-title":"Virtual reality for industrial applications","volume-title":"Computer graphics: systems and applications","author":"Dai","year":"1997"},{"key":"B15","first-page":"76","article-title":"Marwin: a mobile autonomous robot for maintenance and inspection","volume-title":"Proc. 16th int. Conf. On accelerator and large experimental Physics control systems (ICALEPCS\u201917)","author":"Dehne","year":"2017"},{"key":"B16","doi-asserted-by":"publisher","first-page":"3854","DOI":"10.1109\/tvcg.2022.3203102","article-title":"Fov-nerf: foveated neural radiance fields for virtual reality","volume":"28","author":"Deng","year":"2021","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B17","doi-asserted-by":"publisher","first-page":"37506","DOI":"10.1109\/access.2018.2849572","article-title":"Cerntauro: a modular architecture for robotic inspection and telemanipulation in harsh and semi-structured environments","volume":"6","author":"di Castro","year":"2018","journal-title":"IEEE Access"},{"key":"B18","doi-asserted-by":"publisher","first-page":"935","DOI":"10.1007\/s12008-019-00538-x","article-title":"Development of virtual reality support to factory layout planning","volume":"13","author":"Gong","year":"2019","journal-title":"Int. J. Interact. Des. Manuf. (IJIDeM)"},{"key":"B19","volume-title":"Instruct-nerf2nerf: editing 3d scenes with instructions","author":"Haque","year":"2023"},{"key":"B20","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1016\/S0166-4115(08)62386-9","article-title":"Development of nasa-tlx (task load index): results of empirical and theoretical research","volume":"52","author":"Hart","year":"1988","journal-title":"Adv. Psychol."},{"key":"B21","doi-asserted-by":"crossref","DOI":"10.1145\/3585499","article-title":"Nerfshop: interactive editing of neural radiance fields","volume-title":"Proceedings of the ACM on computer Graphics and interactive techniques 6","author":"Jambon","year":"2023"},{"key":"B22","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1207\/s15327108ijap0303_3","article-title":"Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness","volume":"3","author":"Kennedy","year":"1993","journal-title":"Int. J. Aviat. Psychol."},{"key":"B23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3592433","article-title":"3d Gaussian splatting for real-time radiance field rendering","volume":"42","author":"Kerbl","year":"2023","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"B24","volume-title":"Control-nerf: editable feature volumes for scene rendering and manipulation","author":"Lazova","year":"2022"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1145\/3145690.3145697","article-title":"Estimating the simulator sickness in immersive virtual reality with optical flow analysis","author":"Lee","year":"2017","journal-title":"SIGGRAPH Asia 2017 Posters"},{"key":"B26","first-page":"739","article-title":"Bringing instant neural graphics primitives to immersive virtual reality","volume-title":"2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)","author":"Li","year":""},{"key":"B27","first-page":"44","article-title":"Mixed reality tunneling effects for stereoscopic untethered video-see-through head-mounted displays","volume-title":"2022 IEEE international symposium on mixed and augmented reality (ISMAR)","author":"Li","year":""},{"key":"B28","volume-title":"Steernerf: accelerating nerf rendering via smooth viewpoint trajectory","author":"Li","year":""},{"key":"B29","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-58452-8_24","article-title":"Nerf: representing scenes as neural radiance fields for view synthesis","volume-title":"European conference on computer vision","author":"Mildenhall","year":"2020"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3528223.3530127","article-title":"Instant neural graphics primitives with a multiresolution hash encoding","volume":"41","author":"M\u00fcller","year":"2022","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"B31","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1001\/jama.1937.02780030070033","article-title":"Topography of the layer of rods and cones in the human retina","volume":"108","author":"Osterberg","year":"1937","journal-title":"J. Am. Med. Assoc."},{"key":"B32","doi-asserted-by":"crossref","DOI":"10.1145\/1128923.1128935","article-title":"Towards the use of cad models in vr applications","volume-title":"Vrcia \u201906","author":"Raposo","year":"2006"},{"key":"B33","doi-asserted-by":"publisher","first-page":"1104","DOI":"10.3390\/rs3061104","article-title":"Heritage recording and 3d modeling with photogrammetry and 3d scanning","volume":"3","author":"Remondino","year":"2011","journal-title":"Remote. Sens."},{"key":"B34","doi-asserted-by":"crossref","DOI":"10.1063\/5.0127406","article-title":"The implementation for virtual regulatory inspection in nuclear facility during the covid-19 pandemic in Indonesia","volume-title":"The international conference on advanced material and technology (icamt) 2021","author":"Sarmita","year":"2022"},{"key":"B35","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2016.445","article-title":"Structure-from-motion revisited","volume-title":"Conference on computer vision and pattern recognition (CVPR)","author":"Sch\u00f6nberger","year":"2016"},{"key":"B36","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1026\/\/1617-6383.15.2.69","article-title":"The sense of presence in virtual environments: a three-component scale measuring spatial presence, involvement, and realness","volume":"15","author":"Schubert","year":"2003","journal-title":"Z. f\u00fcr Medien."},{"key":"B37","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1109\/VR.2001.913806","article-title":"Simulator sickness and presence in a high fov virtual environment","volume-title":"Proceedings IEEE virtual reality 2001","author":"Seay","year":"2001"},{"key":"B38","doi-asserted-by":"crossref","first-page":"8238","DOI":"10.1109\/CVPR52688.2022.00807","article-title":"Block-nerf: scalable large scene neural view synthesis","volume-title":"2022 IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","author":"Tancik","year":"2022"},{"key":"B39","volume-title":"Fourier features let networks learn high frequency functions in low dimensional domains","author":"Tancik","year":"2020"},{"key":"B40","volume-title":"Nerfstudio: a modular framework for neural radiance field development","author":"Tancik","year":"2023"},{"key":"B41","article-title":"A survey on interactive lenses in visualization","volume-title":"Eurographics conference on visualization","author":"Tominski","year":"2014"},{"key":"B42","doi-asserted-by":"crossref","DOI":"10.1145\/237091.237098","article-title":"3d magic lenses","volume-title":"ACM symposium on user interface software and technology","author":"Viega","year":"1996"},{"key":"B43","doi-asserted-by":"publisher","first-page":"367","DOI":"10.1109\/VISUAL.2005.1532818","article-title":"The magic volume lens: an interactive focus+context technique for volume rendering","volume":"2005","author":"Wang","year":"2005","journal-title":"Vis. 05. IEEE Vis."},{"key":"B44","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/j.optlaseng.2018.02.017","article-title":"High-speed 3d shape measurement with structured light methods: a review","volume":"106","author":"Zhang","year":"2018","journal-title":"Opt. Lasers Eng."},{"key":"B45","doi-asserted-by":"publisher","first-page":"625","DOI":"10.1111\/cgf.13386","article-title":"State of the art on 3d reconstruction with rgb-d cameras","volume":"37","author":"Zollh\u00f6fer","year":"2018","journal-title":"Comput. Graph. Forum"}],"container-title":["Frontiers in Virtual Reality"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frvir.2024.1377245\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,10]],"date-time":"2024-04-10T04:24:56Z","timestamp":1712723096000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frvir.2024.1377245\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,10]]},"references-count":45,"alternative-id":["10.3389\/frvir.2024.1377245"],"URL":"https:\/\/doi.org\/10.3389\/frvir.2024.1377245","relation":{},"ISSN":["2673-4192"],"issn-type":[{"value":"2673-4192","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,10]]},"article-number":"1377245"}}