{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T17:43:20Z","timestamp":1760118200293,"version":"build-2065373602"},"reference-count":40,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2024,12,27]],"date-time":"2024-12-27T00:00:00Z","timestamp":1735257600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006465","name":"Culture, Sports and Tourism R&amp;D Program","doi-asserted-by":"publisher","award":["RS-2023-00227648"],"award-info":[{"award-number":["RS-2023-00227648"]}],"id":[{"id":"10.13039\/501100006465","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Paranasal sinus surgery, a common treatment for chronic rhinosinusitis, requires exceptional precision due to the proximity of critical anatomical structures. To ensure accurate instrument control and clear visualization of the surgical site, surgeons utilize computer-aided surgical navigation (CSN). A key component of CSN is the registration process, which is traditionally reliant on manual or marker-based techniques. However, there is a growing shift toward marker-less registration methods. In previous work, we investigated a mesh-based registration approach using a Mixed Reality Head-Mounted Display (MR-HMD), specifically the Microsoft HoloLens 2. However, this method faced limitations, including depth holes and invalid values. These issues stemmed from the device\u2019s low-resolution camera specifications and the 3D projection steps required to upscale RGB camera spaces. In this study, we propose a novel automated marker-less registration method leveraging Neural Radiance Field (NeRF) technology with an MR-HMD. To address insufficient depth information in the previous approach, we utilize rendered-depth images generated by the trained NeRF model. We evaluated our method against two other techniques, including prior mesh-based registration, using a facial phantom and three participants. The results demonstrate our proposed method achieves at least a 0.873 mm (12%) improvement in registration accuracy compared to others.<\/jats:p>","DOI":"10.3390\/computers14010005","type":"journal-article","created":{"date-parts":[[2024,12,27]],"date-time":"2024-12-27T02:59:27Z","timestamp":1735268367000},"page":"5","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["An Automated Marker-Less Registration Approach Using Neural Radiance Fields for Potential Use in Mixed Reality-Based Computer-Aided Surgical Navigation of Paranasal Sinus"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2064-5124","authenticated-orcid":false,"given":"Suhyeon","family":"Kim","sequence":"first","affiliation":[{"name":"School of Computing, Gachon University, Seongnam 13120, Republic of Korea"}]},{"given":"Hyeonji","family":"Kim","sequence":"additional","affiliation":[{"name":"School of Computing, Gachon University, Seongnam 13120, Republic of Korea"}]},{"given":"Younhyun","family":"Jung","sequence":"additional","affiliation":[{"name":"School of Computing, Gachon University, Seongnam 13120, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Albu, S. (2020). Chronic Rhinosinusitis\u2014An Update on Epidemiology, Pathogenesis and Management. J. Clin. Med., 9.","DOI":"10.3390\/jcm9072285"},{"key":"ref_2","first-page":"370","article-title":"Chronic Rhinosinusitis","volume":"108","author":"Keating","year":"2023","journal-title":"Am. Fam. Physician"},{"key":"ref_3","first-page":"Doc08","article-title":"Comprehensive Review on Endonasal Endoscopic Sinus Surgery","volume":"14","author":"Weber","year":"2015","journal-title":"GMS Curr. Top. Otorhinolaryngol. Head Neck Surg."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1586\/17434440.2.4.395","article-title":"Computer-Aided Surgery of the Paranasal Sinuses and Skull Base","volume":"2","author":"Wise","year":"2005","journal-title":"Expert Rev. Med. Devices"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Duan, X., Xie, D., Zhang, R., Li, X., Sun, J., Qian, C., Song, X., and Li, C. (2023). A Novel Robotic Bronchoscope System for Navigation and Biopsy of Pulmonary Lesions. Cyborg Bionic Syst., 4.","DOI":"10.34133\/cbsystems.0013"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Fan, Y., Xu, L., Liu, S., Li, J., Xia, J., Qin, X., Li, Y., Gao, T., and Tang, X. (2024). The State-of-the-Art and Perspectives of Laser Ablation for Tumor Treatment. Cyborg Bionic Syst., 5.","DOI":"10.34133\/cbsystems.0062"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Han, Z., Tian, H., Han, X., Wu, J., Zhang, W., Li, C., Qiu, L., Duan, X., and Tian, W. (2024). A Respiratory Motion Prediction Method Based on LSTM-AE with Attention Mechanism for Spine Surgery. Cyborg Bionic Syst., 5.","DOI":"10.34133\/cbsystems.0063"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1038\/s41467-023-44385-7","article-title":"AI Co-Pilot Bronchoscope Robot","volume":"15","author":"Zhang","year":"2024","journal-title":"Nat. Commun."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/s12070-019-01773-0","article-title":"Role of Image Guided Navigation in Endoscopic Surgery of Paranasal Sinuses: A Comparative Study","volume":"72","author":"Singh","year":"2020","journal-title":"Indian J. Otolaryngol. Head Neck Surg."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"259","DOI":"10.4055\/cios.2011.3.4.259","article-title":"Computer Assisted Navigation in Knee Arthroplasty","volume":"3","author":"Bae","year":"2011","journal-title":"Clin. Orthop. Surg."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1109\/10.1353","article-title":"An Articulated Neurosurgical Navigation System Using MRI and CT Images","volume":"35","author":"Kosugi","year":"1988","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Grauvogel, T.D., Engelskirchen, P., Semper-Hogg, W., Grauvogel, J., and Laszig, R. (2017). Navigation Accuracy after Automatic-and Hybrid-Surface Registration in Sinus and Skull Base Surgery. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0180975"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1109\/TMI.2006.889748","article-title":"Fast and Accurate Automatic Registration for MR-Guided Procedures Using Active Microcoils","volume":"26","author":"Krueger","year":"2007","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1016\/j.patrec.2008.11.001","article-title":"Automatic Localization of the Center of Fiducial Markers in 3D CT\/MRI Images for Image-Guided Neurosurgery","volume":"30","author":"Wang","year":"2009","journal-title":"Pattern Recognit. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"944","DOI":"10.1007\/s00464-009-0568-2","article-title":"An Effective Point-Based Registration Tool for Surgical Navigation","volume":"24","author":"Hong","year":"2010","journal-title":"Surg. Endosc."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1016\/j.infrared.2016.03.011","article-title":"Real-Time Automatic Registration in Optical Surgical Navigation","volume":"76","author":"Lin","year":"2016","journal-title":"Infrared Phys. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1016\/j.ijom.2019.09.015","article-title":"Application of an Open Position Splint Integrated with a Reference Frame and Registration Markers for Mandibular Navigation Surgery","volume":"49","author":"Yamamoto","year":"2020","journal-title":"Int. J. Oral Maxillofac. Surg."},{"key":"ref_18","unstructured":"Wang, M., and Song, Z. (2021). Surgical Navigation. Computer-Aided Oral and Maxillofacial Surgery, Elsevier."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"101910","DOI":"10.1118\/1.4895847","article-title":"A New Markerless Patient-to-Image Registration Method Using a Portable 3D Scanner","volume":"41","author":"Fan","year":"2014","journal-title":"Med. Phys."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1002\/rcs.1626","article-title":"A Robust Automated Markerless Registration Framework for Neurosurgery Navigation","volume":"11","author":"Jiang","year":"2015","journal-title":"Int. J. Med. Robot. Comput. Assist. Surg."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1259\/dmfr\/21358271","article-title":"Marker-Free Registration for the Accurate Integration of CT Images and the Subject\u2019s Anatomy During Navigation Surgery of the Maxillary Sinus","volume":"41","author":"Kang","year":"2012","journal-title":"Dentomaxillofac. Radiol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"5480","DOI":"10.1002\/cam4.3189","article-title":"The Clinical Application Value of Mixed-Reality-Assisted Surgical Navigation for Laparoscopic Nephrectomy","volume":"9","author":"Li","year":"2020","journal-title":"Cancer Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1007\/s11548-020-02302-z","article-title":"Mixed Reality as a Novel Tool for Diagnostic and Surgical Navigation in Orthopaedics","volume":"16","author":"Teatini","year":"2021","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Lee, S., Jung, H., Lee, E., Jung, Y., and Kim, S.T. (2021). A Preliminary Work: Mixed Reality-Integrated Computer-Aided Surgical Navigation System for Paranasal Sinus Surgery Using Microsoft HoloLens 2. Advances in Computer Graphics: 38th Computer Graphics International Conference, CGI 2021, Virtual Event, 6\u201310 September 2021, Springer. Proceedings.","DOI":"10.1007\/978-3-030-89029-2_47"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kim, S., An, H., Song, M., Lee, S., Jung, H., Kim, S., and Jung, Y. (2023). Automated Marker-Less Patient-to-Preoperative Medical Image Registration Approach Using RGB-D Images and Facial Landmarks for Potential Use in Computed-Aided Surgical Navigation of the Paranasal Sinus. Advances in Computer Graphics: 40th Computer Graphics International Conference, CGI 2023, Shanghai, China, 28 August\u20131 September 2023, Springer. Proceedings, Part IV.","DOI":"10.1007\/978-3-031-50078-7_11"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1145\/3503250","article-title":"Nerf: Representing Scenes as Neural Radiance Fields for View Synthesis","volume":"65","author":"Mildenhall","year":"2021","journal-title":"Commun. ACM"},{"key":"ref_27","unstructured":"Microsoft (2024, November 15). Microsoft Hololens 2 Docs. Available online: https:\/\/www.microsoft.com\/it-it\/hololens."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Gsaxner, C., Pepe, A., Wallner, J., Schmalstieg, D., and Egger, J. (2019, January 13\u201317). Markerless Image-to-Face Registration for Untethered Augmented Reality in Head and Neck Surgery. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China.","DOI":"10.1007\/978-3-030-32254-0_27"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Groenenberg, A., Brouwers, L., Bemelman, M., Maal, T.J., Heyligers, J.M., and Louwerse, M.M. (2024). Feasibility and Accuracy of a Real-Time Depth-Based Markerless Navigation Method for Hologram-Guided Surgery. BMC Digit. Health, 2.","DOI":"10.1186\/s44247-024-00067-y"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1008","DOI":"10.1007\/s10278-019-00272-6","article-title":"A Marker-Less Registration Approach for Mixed Reality\u2013Aided Maxillofacial Surgery: A Pilot Evaluation","volume":"32","author":"Pepe","year":"2019","journal-title":"J. Digit. Imaging"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Sun, K., Xiao, B., Liu, D., and Wang, J. (2019, January 15\u201320). Deep High-Resolution Representation Learning for Human Pose Estimation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00584"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"3349","DOI":"10.1109\/TPAMI.2020.2983686","article-title":"Deep High-Resolution Representation Learning for Visual Recognition","volume":"43","author":"Wang","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1149","DOI":"10.1109\/TPAMI.2012.205","article-title":"Local Evidence Aggregation for Regression-Based Facial Point Detection","volume":"35","author":"Martinez","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Sun, Y., Wang, X., and Tang, X. (2013, January 23\u201328). Deep Convolutional Network Cascade for Facial Point Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.446"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1137\/0713009","article-title":"Generalizing the Singular Value Decomposition","volume":"13","year":"1976","journal-title":"SIAM J. Numer. Anal."},{"key":"ref_37","unstructured":"Chetverikov, D., Svirko, D., Stepanov, D., and Krsek, P. (2002, January 11\u201315). The Trimmed Iterative Closest Point Algorithm. Proceedings of the 2002 International Conference on Pattern Recognition, Quebec City, QC, Canada."},{"key":"ref_38","first-page":"1","article-title":"Instant Neural Graphics Primitives with a Multiresolution Hash Encoding","volume":"41","author":"Evans","year":"2022","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_39","unstructured":"Microsoft (2024, November 15). Hololens2forcv. Available online: https:\/\/github.com\/microsoft\/HoloLens2ForCV."},{"key":"ref_40","unstructured":"(2024, November 15). Opencv. Available online: https:\/\/opencv.org\/."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/1\/5\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T17:01:04Z","timestamp":1760115664000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/1\/5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,27]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,1]]}},"alternative-id":["computers14010005"],"URL":"https:\/\/doi.org\/10.3390\/computers14010005","relation":{},"ISSN":["2073-431X"],"issn-type":[{"type":"electronic","value":"2073-431X"}],"subject":[],"published":{"date-parts":[[2024,12,27]]}}}