{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T11:27:18Z","timestamp":1789644438901,"version":"4.0.0"},"reference-count":39,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,5,20]],"date-time":"2023-05-20T00:00:00Z","timestamp":1684540800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and Technology","award":["111-2221-E-182-020"],"award-info":[{"award-number":["111-2221-E-182-020"]}]},{"name":"Ministry of Science and Technology","award":["CMRPD2M0021"],"award-info":[{"award-number":["CMRPD2M0021"]}]},{"name":"Ministry of Science and Technology","award":["BMRPB46"],"award-info":[{"award-number":["BMRPB46"]}]},{"DOI":"10.13039\/501100004606","name":"CGMH project","doi-asserted-by":"publisher","award":["111-2221-E-182-020"],"award-info":[{"award-number":["111-2221-E-182-020"]}],"id":[{"id":"10.13039\/501100004606","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004606","name":"CGMH project","doi-asserted-by":"publisher","award":["CMRPD2M0021"],"award-info":[{"award-number":["CMRPD2M0021"]}],"id":[{"id":"10.13039\/501100004606","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004606","name":"CGMH project","doi-asserted-by":"publisher","award":["BMRPB46"],"award-info":[{"award-number":["BMRPB46"]}],"id":[{"id":"10.13039\/501100004606","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Bioengineering"],"abstract":"<jats:p>Most current surgical navigation methods rely on optical navigators with images displayed on an external screen. However, minimizing distractions during surgery is critical and the spatial information displayed in this arrangement is non-intuitive. Previous studies have proposed combining optical navigation systems with augmented reality (AR) to provide surgeons with intuitive imaging during surgery, through the use of planar and three-dimensional imagery. However, these studies have mainly focused on visual aids and have paid relatively little attention to real surgical guidance aids. Moreover, the use of augmented reality reduces system stability and accuracy, and optical navigation systems are costly. Therefore, this paper proposed an augmented reality surgical navigation system based on image positioning that achieves the desired system advantages with low cost, high stability, and high accuracy. This system also provides intuitive guidance for the surgical target point, entry point, and trajectory. Once the surgeon uses the navigation stick to indicate the position of the surgical entry point, the connection between the surgical target and the surgical entry point is immediately displayed on the AR device (tablet or HoloLens glasses), and a dynamic auxiliary line is shown to assist with incision angle and depth. Clinical trials were conducted for EVD (extra-ventricular drainage) surgery, and surgeons confirmed the system\u2019s overall benefit. A \u201cvirtual object automatic scanning\u201d method is proposed to achieve a high accuracy of 1 \u00b1 0.1 mm for the AR-based system. Furthermore, a deep learning-based U-Net segmentation network is incorporated to enable automatic identification of the hydrocephalus location by the system. The system achieves improved recognition accuracy, sensitivity, and specificity of 99.93%, 93.85%, and 95.73%, respectively, representing a significant improvement from previous studies.<\/jats:p>","DOI":"10.3390\/bioengineering10050617","type":"journal-article","created":{"date-parts":[[2023,5,22]],"date-time":"2023-05-22T02:28:42Z","timestamp":1684722522000},"page":"617","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Augmented Reality Surgical Navigation System Integrated with Deep Learning"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8182-1676","authenticated-orcid":false,"given":"Shin-Yan","family":"Chiou","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, College of Engineering, Chang Gung University, Kwei-Shan, Taoyuan 333, Taiwan"},{"name":"Department of Nuclear Medicine, Linkou Chang Gung Memorial Hospital, Taoyuan 333, Taiwan"},{"name":"Department of Neurosurgery, Keelung Chang Gung Memorial Hospital, Keelung 204, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Li-Sheng","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, College of Engineering, Chang Gung University, Kwei-Shan, Taoyuan 333, Taiwan"},{"name":"Department of Electrical and Electronic Engineering, College of Engineering, Yonsei University, Seodaemun-gu, Seoul 03722, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chia-Wei","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, College of Engineering, Chang Gung University, Kwei-Shan, Taoyuan 333, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dong-Hyun","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronic Engineering, College of Engineering, Yonsei University, Seodaemun-gu, Seoul 03722, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1548-965X","authenticated-orcid":false,"given":"Mohammed A.","family":"Al-masni","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence, College of Software & Convergence Technology, Daeyang AI Center, Sejong University, Seoul 05006, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3512-7215","authenticated-orcid":false,"given":"Hao-Li","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Taiwan University, Taipei 106, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3505-2196","authenticated-orcid":false,"given":"Kuo-Chen","family":"Wei","sequence":"additional","affiliation":[{"name":"New Taipei City Tucheng Hospital, Tao-Yuan, Tucheng, New Taipei City 236, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiun-Lin","family":"Yan","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Keelung Chang Gung Memorial Hospital, Keelung 204, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pin-Yuan","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Keelung Chang Gung Memorial Hospital, Keelung 204, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.compbiomed.2019.03.012","article-title":"Augmented and virtual reality in dental medicine: A systematic review","volume":"108","author":"Joda","year":"2019","journal-title":"Comput. Biol. Med."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"206","DOI":"10.3171\/2014.9.JNS141001","article-title":"Augmented reality\u2013guided neurosurgery: Accuracy and intraoperative application of an image projection technique","volume":"123","author":"Tabrizi","year":"2015","journal-title":"J. Neurosurg."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.jbi.2015.04.003","article-title":"Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display","volume":"55","author":"Chen","year":"2015","journal-title":"J. Biomed. Inform."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lai, M., Skyrman, S., Shan, C., Babic, D., Homan, R., Edstr\u00f6m, E., Persson, O., Urstr\u00f6m, G., Elmi-Terander, A., and Hendriks, B. (2020). Fusion of augmented reality imaging with the endoscopic view for endonasal skull base surgery; A novel application for surgical navigation based on intraoperative cone beam computed tomography and optical tracking. PLoS ONE, 15.","DOI":"10.1371\/journal.pone.0229454"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1007\/s11548-013-0828-4","article-title":"Mobile augmented reality for computer-assisted percutaneous nephrolithotomy","volume":"8","author":"Rassweiler","year":"2013","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1295","DOI":"10.1109\/TBME.2014.2301191","article-title":"Augmented reality navigation with automatic marker-free image registration using 3-D image overlay for dental surgery","volume":"61","author":"Wang","year":"2014","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/j.jamcollsurg.2016.06.392","article-title":"Augmented reality navigation surgery facilitates laparoscopic rescue of failed portal vein embolization","volume":"223","author":"Conrad","year":"2016","journal-title":"J. Am. Coll. Surg."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1159\/000354816","article-title":"Easy-to-use augmented reality neuronavigation using a wireless tablet PC","volume":"92","author":"Deng","year":"2014","journal-title":"Stereotact. Funct. Neurosurg."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1049\/htl.2018.5079","article-title":"Augmenting Microsoft\u2019s HoloLens with vuforia tracking for neuronavigation","volume":"5","author":"Frantz","year":"2018","journal-title":"Healthc. Technol. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1093\/ons\/opz094","article-title":"Clinical accuracy of holographic navigation using point-based registration on augmented-reality glasses","volume":"17","author":"Vandoormaal","year":"2019","journal-title":"Oper. Neurosurg."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1055\/s-0039-1677911","article-title":"Artificial intelligence in clinical decision support: A focused literature survey","volume":"28","author":"Montani","year":"2019","journal-title":"Yearb. Med. Inform."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1007\/s13304-018-0567-8","article-title":"Augmented reality in open surgery","volume":"70","author":"Fida","year":"2018","journal-title":"Updat. Surg."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1409","DOI":"10.1109\/TPAMI.2011.239","article-title":"Tracking-learning-detection","volume":"34","author":"Kalal","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"104366","DOI":"10.1016\/j.compbiomed.2021.104366","article-title":"Impact of augmented-reality improvement in ablation catheter navigation as assessed by virtual-heart simulations of ventricular tachycardia ablation","volume":"133","author":"Prakosa","year":"2021","journal-title":"Comput. Biol. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"104402","DOI":"10.1016\/j.compbiomed.2021.104402","article-title":"Augmented reality based navigation for distal interlocking of intramedullary nails utilizing Microsoft HoloLens 2","volume":"133","author":"Tu","year":"2021","journal-title":"Comput. Biol. Med."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.inffus.2022.12.010","article-title":"Diagnosis of brain diseases in fusion of neuroimaging modalities using deep learning: A review","volume":"93","author":"Shoeibi","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Shoeibi, A., Ghassemi, N., Khodatars, M., Moridian, P., Khosravi, A., Zare, A., Gorriz, J.M., Chale-Chale, A.H., Khadem, A., and Acharya, U.R. (2022). Automatic diagnosis of schizophrenia and attention deficit hyperactivity disorder in rs-fMRI modality using convolutional autoencoder model and interval type-2 fuzzy regression. Cogn. Neurodyn., 1\u201323.","DOI":"10.1007\/s11571-022-09897-w"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"102464","DOI":"10.1016\/j.nicl.2020.102464","article-title":"Automated detection of cerebral microbleeds in MR images: A two-stage deep learning approach","volume":"28","author":"Kim","year":"2020","journal-title":"NeuroImage Clin."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"10191","DOI":"10.1038\/s41598-021-89686-3","article-title":"CMM-Net: Contextual multi-scale multi-level network for efficient biomedical image segmentation","volume":"11","author":"Kim","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_20","first-page":"1","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_21","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_22","unstructured":"Liu, C.S.W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the 8th International Conference on Medical Image Computing and Computer-Assisted Intervention\u2013(MICCAI), Munich, Germany. Part III 18.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Long, L., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2015, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_25","unstructured":"Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., and Darrell, T. (2014, January 21\u201326). Decaf: A deep convolutional activation feature for generic visual recognition. Proceedings of the 31st International Conference on Machine Learning, Beijing, China."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"103547","DOI":"10.1016\/j.compbiomed.2019.103547","article-title":"Comparison of morphometric parameters in prediction of hydrocephalus using random forests","volume":"116","author":"Yigin","year":"2020","journal-title":"Comput. Biol. Med."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"104268","DOI":"10.1016\/j.compbiomed.2021.104268","article-title":"Automatic segmentation and location learning of neonatal cerebral ventricles in 3D ultrasound data combining CNN and CPPN","volume":"131","author":"Martin","year":"2021","journal-title":"Comput. Biol. Med."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.compbiomed.2019.03.014","article-title":"A novel end-to-end brain tumor segmentation method using improved fully convolutional networks","volume":"108","author":"Li","year":"2019","journal-title":"Comput. Biol. Med."},{"key":"ref_29","unstructured":"Westenberger, P. (2008). Avizo\u2014Three-Dimensional Visualization Framework, Geoinformatics 2008\u2014Data to Knowledge."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"923","DOI":"10.1148\/radiol.2017162664","article-title":"Automated critical test findings identification and online notification system using artificial intelligence in imaging","volume":"285","author":"Prevedello","year":"2017","journal-title":"Radiology"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1029","DOI":"10.1007\/s00062-020-00993-0","article-title":"SVM-Based Normal Pressure Hydrocephalus Detection","volume":"31","author":"Rau","year":"2021","journal-title":"Clin. Neuroradiol."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Sahli, H., Mouelhi, A., Sayadi, M., and Rachdi, R. (2018, January 12\u201314). Discriminant textural feature selection and classification for a computerized fetal hydrocephalus detection. Proceedings of the 2018 IEEE International Conference on Image Processing, Applications and Systems (IPAS), Sophia Antipolis, France.","DOI":"10.1109\/IPAS.2018.8708865"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11548-007-0078-4","article-title":"A real-time navigation system for laparoscopic surgery based on three-dimensional ultrasound using magneto-optic hybrid tracking configuration","volume":"2","author":"Konishi","year":"2007","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1007\/s00383-011-3034-x","article-title":"Augmented reality navigation system for laparoscopic splenectomy in children based on preoperative CT image using optical tracking device","volume":"28","author":"Ieiri","year":"2012","journal-title":"Pediatr. Surg. Int."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"e21229","DOI":"10.1097\/MD.0000000000021229","article-title":"Evaluation of an artificial intelligent hydrocephalus diagnosis model based on transfer learning","volume":"99","author":"Duan","year":"2020","journal-title":"Medicine"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1855","DOI":"10.1109\/TBME.2011.2126572","article-title":"A portable image overlay projection device for computer-aided open liver surgery","volume":"58","author":"Gavaghan","year":"2011","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2958","DOI":"10.1007\/s00464-018-6151-y","article-title":"Mobile, real-time, and point-of-care augmented reality is robust, accurate, and feasible: A prospective pilot study","volume":"32","author":"Kenngott","year":"2018","journal-title":"Surg. Endosc."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1049\/htl.2019.0062","article-title":"HoloInjection: Augmented reality support for CT-guided spinal needle injections","volume":"6","author":"Heinrich","year":"2019","journal-title":"Healthc. Technol. Lett."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"756","DOI":"10.1007\/s00270-019-02403-6","article-title":"Smartphone augmented reality CT-based platform for needle insertion guidance: A phantom study","volume":"43","author":"Hecht","year":"2020","journal-title":"Cardiovasc. Interv. Radiol."}],"container-title":["Bioengineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5354\/10\/5\/617\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:38:59Z","timestamp":1760125139000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5354\/10\/5\/617"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,20]]},"references-count":39,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["bioengineering10050617"],"URL":"https:\/\/doi.org\/10.3390\/bioengineering10050617","relation":{},"ISSN":["2306-5354"],"issn-type":[{"value":"2306-5354","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,20]]}}}