{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T20:27:28Z","timestamp":1790713648629,"version":"4.1.0"},"reference-count":49,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,2,8]],"date-time":"2022-02-08T00:00:00Z","timestamp":1644278400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000780","name":"European Commission","doi-asserted-by":"publisher","award":["731974"],"award-info":[{"award-number":["731974"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]},{"name":"CrossLAB Project -  (Departments of Excellence) of the University of Pisa, Laboratory of Augmented Reality","award":["-"],"award-info":[{"award-number":["-"]}]},{"name":"ARTS4.0: Augmented Reality for Tumor Surgery 4.0 \u2013 Project co-financed under Tuscany POR FESR 2014-2020","award":["-"],"award-info":[{"award-number":["-"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>In the context of image-guided surgery, augmented reality (AR) represents a ground-breaking enticing improvement, mostly when paired with wearability in the case of open surgery. Commercially available AR head-mounted displays (HMDs), designed for general purposes, are increasingly used outside their indications to develop surgical guidance applications with the ambition to demonstrate the potential of AR in surgery. The applications proposed in the literature underline the hunger for AR-guidance in the surgical room together with the limitations that hinder commercial HMDs from being the answer to such a need. The medical domain demands specifically developed devices that address, together with ergonomics, the achievement of surgical accuracy objectives and compliance with medical device regulations. In the framework of an EU Horizon2020 project, a hybrid video and optical see-through augmented reality headset paired with a software architecture, both specifically designed to be seamlessly integrated into the surgical workflow, has been developed. In this paper, the overall architecture of the system is described. The developed AR HMD surgical navigation platform was positively tested on seven patients to aid the surgeon while performing Le Fort 1 osteotomy in cranio-maxillofacial surgery, demonstrating the value of the hybrid approach and the safety and usability of the navigation platform.<\/jats:p>","DOI":"10.3390\/info13020081","type":"journal-article","created":{"date-parts":[[2022,2,9]],"date-time":"2022-02-09T21:22:15Z","timestamp":1644441735000},"page":"81","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["Architecture of a Hybrid Video\/Optical See-through Head-Mounted Display-Based Augmented Reality Surgical Navigation Platform"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1004-6964","authenticated-orcid":false,"given":"Marina","family":"Carbone","sequence":"first","affiliation":[{"name":"Department of Information Engineering, University of Pisa, 56126 Pisa, Italy"},{"name":"EndoCAS Center, Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, 56122 Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6773-3741","authenticated-orcid":false,"given":"Fabrizio","family":"Cutolo","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, University of Pisa, 56126 Pisa, Italy"},{"name":"EndoCAS Center, Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, 56122 Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5235-0414","authenticated-orcid":false,"given":"Sara","family":"Condino","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, University of Pisa, 56126 Pisa, Italy"},{"name":"EndoCAS Center, Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, 56122 Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7818-1356","authenticated-orcid":false,"given":"Laura","family":"Cercenelli","sequence":"additional","affiliation":[{"name":"eDIMES Lab\u2014Laboratory of Bioengineering, Department of Experimental, Diagnostic and Specialty Medicine, University of Bologna, 40126 Bologna, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Renzo","family":"D\u2019Amato","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, University of Pisa, 56126 Pisa, Italy"},{"name":"EndoCAS Center, Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, 56122 Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9833-9800","authenticated-orcid":false,"given":"Giovanni","family":"Badiali","sequence":"additional","affiliation":[{"name":"Maxillofacial Surgery Unit, Department of Biomedical and Neuromotor Sciences and S. Orsola-Malpighi Hospital, University of Bologna, 40126 Bologna, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9294-2828","authenticated-orcid":false,"given":"Vincenzo","family":"Ferrari","sequence":"additional","affiliation":[{"name":"Department of Information Engineering, University of Pisa, 56126 Pisa, Italy"},{"name":"EndoCAS Center, Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, 56122 Pisa, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Rolland, J.P., and Cakmakci, O. (2005). The Past, Present, and Future of Head-Mounted Display Designs, SPIE.","DOI":"10.1117\/12.575697"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1162\/pres.1997.6.4.355","article-title":"A Survey of Augmented Reality","volume":"6","author":"Azuma","year":"1997","journal-title":"Presence: Teleoperators Virtual Environ."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Carbone, M., Condino, S., Cutolo, F., Viglialoro, R.M., Kaschke, O., Thomale, U.W., and Ferrari, V. (2018). Proof of Concept: Wearable Augmented Reality Video See-Through Display for Neuro-Endoscopy. AVR 2018, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-95282-6_7"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4574172","DOI":"10.1155\/2017\/4574172","article-title":"Recent Development of Augmented Reality in Surgery: A Review","volume":"2017","author":"Vavra","year":"2017","journal-title":"J. Healthc. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Cutolo, F. (2017). Augmented Reality in Image-Guided Surgery. Encyclopedia of Computer Graphics and Games, Lee, N., Ed., Springer International Publishing.","DOI":"10.1007\/978-3-319-08234-9_78-1"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"9321535","DOI":"10.1155\/2019\/9321535","article-title":"Augmented Reality in Healthcare","volume":"2019","author":"Ferrari","year":"2019","journal-title":"J. Healthc. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Condino, S., Turini, G., Viglialoro, R., Gesi, M., and Ferrari, V. (2019). Wearable Augmented Reality Application for Shoulder Rehabilitation. Electronics, 8.","DOI":"10.3390\/electronics8101178"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1177\/1553350618799552","article-title":"Toward Holographic-Guided Surgery","volume":"26","author":"Meulstee","year":"2019","journal-title":"Surg. Innov."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1186\/s41747-017-0033-2","article-title":"Through the HoloLens\u2122 looking glass: Augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels","volume":"2","author":"Pratt","year":"2018","journal-title":"Eur. Radiol. Exp."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1066","DOI":"10.1097\/PRS.0000000000003802","article-title":"Mixed Reality with HoloLens: Where Virtual Reality Meets Augmented Reality in the Operating Room","volume":"140","author":"Tepper","year":"2017","journal-title":"Plast. Reconstr. Surg."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2473","DOI":"10.1007\/s00405-020-06351-7","article-title":"Augmented reality with HoloLens in parotid surgery: How to assess and to improve accuracy","volume":"278","author":"Scherl","year":"2021","journal-title":"Eur. Arch. Oto-Rhino-Laryngol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"662","DOI":"10.1007\/s11605-020-04519-4","article-title":"Augmented Reality during Open Liver Surgery Using a Markerless Non-rigid Registration System","volume":"25","author":"Golse","year":"2021","journal-title":"J. Gastrointest. Surg."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/MCSE.2020.2972822","article-title":"Augmented and Virtual Reality in Surgery","volume":"22","author":"Desselle","year":"2020","journal-title":"Comput. Sci. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"920","DOI":"10.1002\/alr.22546","article-title":"Application of holographic augmented reality for external approaches to the frontal sinus","volume":"10","author":"Neves","year":"2020","journal-title":"Int. Forum Allergy Rhinol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"999","DOI":"10.1080\/17434440.2019.1693891","article-title":"Are augmented reality headsets in surgery a dead end?","volume":"16","author":"Ferrari","year":"2019","journal-title":"Expert Rev. Med Devices"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1109\/TBME.2019.2914517","article-title":"Perceptual Limits of Optical See-Through Visors for Augmented Reality Guidance of Manual Tasks","volume":"67","author":"Condino","year":"2019","journal-title":"IEEE T Bio-Med. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2228","DOI":"10.1109\/TVCG.2018.2832633","article-title":"Effects of AR Display Context Switching and Focal Distance Switching on Human Performance","volume":"25","author":"Gabbard","year":"2019","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Carbone, M., Piazza, R., and Condino, S. (2020). Commercially Available Head-Mounted Displays Are Unsuitable for Augmented Reality Surgical Guidance: A Call for Focused Research for Surgical Applications. Surg. Innov., 27.","DOI":"10.1177\/1553350620903197"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1143","DOI":"10.1007\/s00464-019-06862-3","article-title":"Effectiveness of the HoloLens mixed-reality headset in minimally invasive surgery: A simulation-based feasibility study","volume":"34","author":"Aydin","year":"2020","journal-title":"Surg. Endosc."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"8851964","DOI":"10.1155\/2020\/8851964","article-title":"Experimental Setup Employed in the Operating Room Based on Virtual and Mixed Reality: Analysis of Pros and Cons in Open Abdomen Surgery","volume":"2020","author":"Galati","year":"2020","journal-title":"J. Healthc. Eng."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/17434440.2021.1865795","article-title":"Device profile of the XVision-spine (XVS) augmented-reality surgical navigation system: Overview of its safety and efficacy","volume":"18","author":"Dibble","year":"2021","journal-title":"Expert Rev. Med Devices"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1162\/105474600566808","article-title":"Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization","volume":"9","author":"Rolland","year":"2000","journal-title":"Presence: Teleoper. Virtual Environ."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"362","DOI":"10.1109\/TBC.2011.2130930","article-title":"Three-Dimensional Displays: A Review and Applications Analysis","volume":"57","author":"Holliman","year":"2011","journal-title":"IEEE T Broadcast"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2649","DOI":"10.1109\/TVCG.2017.2754257","article-title":"A Survey of Calibration Methods for Optical See-Through Head-Mounted Displays","volume":"24","author":"Grubert","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Itoh, Y., and Klinker, G. (2014, January 10\u201312). Performance and sensitivity analysis of INDICA: INteraction-Free DIsplay CAlibration for Optical See-Through Head-Mounted Displays. Proceedings of the 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Munich, Germany.","DOI":"10.1109\/ISMAR.2014.6948424"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"572001","DOI":"10.3389\/frobt.2020.572001","article-title":"Optical See-Through Head-Mounted Displays With Short Focal Distance: Conditions for Mitigating Parallax-Related Registration Error","volume":"7","author":"Cutolo","year":"2020","journal-title":"Front. Robot. AI"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Raimbaud, P., Lopez, M.S.A., Figueroa, P., and Hernandez, J.T. (2020, January 22\u201326). Influence of Depth Cues on Eye Tracking Depth Measurement in Augmented Reality Using the MagicLeap Device. Proceedings of the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Atlanta, GA, USA.","DOI":"10.1109\/VRW50115.2020.00045"},{"key":"ref_28","first-page":"123","article-title":"Perceptual issues in Augmented Reality","volume":"2653","author":"Drascic","year":"1996","journal-title":"Stereosc. Disp. Virtual Real. Syst. III"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Condino, S., Fida, B., Carbone, M., Cercenelli, L., Badiali, G., Ferrari, V., and Cutolo, A.F. (2020). Wearable Augmented Reality Platform for Aiding Complex 3D Trajectory Tracing. Sensors, 20.","DOI":"10.3390\/s20061612"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Cattari, N., Piazza, R., D\u2019Amato, R., Fida, B., Carbone, M., Condino, S., Cutolo, F., and Ferrari, V. (July, January 1). Towards a Wearable Augmented Reality Visor for High-Precision Manual Tasks. Proceedings of the 2020 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Bari, Italy.","DOI":"10.1109\/MeMeA49120.2020.9137253"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"159698","DOI":"10.1109\/ACCESS.2019.2950877","article-title":"Toed-in vs Parallel Displays in Video See-Through Head-Mounted Displays for Close-Up View","volume":"7","author":"Cattari","year":"2019","journal-title":"IEEE Access"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"9","DOI":"10.3390\/technologies6010009","article-title":"Perspective Preserving Solution for Quasi-Orthoscopic Video See-Through HMDs","volume":"6","author":"Cutolo","year":"2018","journal-title":"Technologies"},{"key":"ref_33","unstructured":"State, A., Ackerman, J., Hirota, G., Lee, J., and Fuchs, H. (2001, January 29\u201330). Dynamic virtual convergence for video see-through head-mounted displays: Maintaining maximum stereo overlap throughout a close-range work space. Proceedings of the Proceedings IEEE and ACM International Symposium on Augmented Reality, New York, NY, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Cutolo, F., Fontana, U., Carbone, M., D\u2019Amato, R., and Ferrari, V. (2017, January 9\u201313). Hybrid Video\/Optical See-Through HMD. Proceedings of the 2017 Ieee International Symposium on Mixed and Augmented Reality (Ismar-Adjunct), Nantes, France.","DOI":"10.1109\/ISMAR-Adjunct.2017.31"},{"key":"ref_35","unstructured":"(2022, January 12). OpenCV. Available online: http:\/\/opencv.willowgarage.com\/documentation\/cpp\/index.html."},{"key":"ref_36","unstructured":"(VTK), T.V.T. (2022, January 12). Available online:. Available online: http:\/\/www.vtk.org\/."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"706","DOI":"10.1109\/ACCESS.2019.2962122","article-title":"Software Framework for Customized Augmented Reality Headsets in Medicine","volume":"8","author":"Cutolo","year":"2020","journal-title":"IEEE Access"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Cutolo, F., Cattari, N., Carbone, M., D\u2019Amato, R., and Ferrari, V. (2021, January 4\u20138). Device-Agnostic Augmented Reality Rendering Pipeline for AR in Medicine. Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Bari, Italy.","DOI":"10.1109\/ISMAR-Adjunct54149.2021.00077"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A flexible new technique for camera calibration","volume":"22","author":"Zhang","year":"2000","journal-title":"IEEE T Pattern Anal"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Cutolo, F., Fontana, U., Cattari, N., and Ferrari, V. (2020). Off-Line Camera-Based Calibration for Optical See-Through Head-Mounted Displays. Appl. Sci., 10.","DOI":"10.3390\/app10010193"},{"key":"ref_41","unstructured":"VOSTARS (2022, January 12). VOSTARS at Work in the Surgical Room. Available online: https:\/\/www.youtube.com\/watch?v=N8qZUj4amm0."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Choi, J.-W., and Lee, J.Y. (2021). Update on Orthognathic Surgical Techniques. The Surgery-First Orthognathic Approach, Springer.","DOI":"10.1007\/978-981-15-7541-9"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"925e","DOI":"10.1097\/PRS.0000000000004438","article-title":"Current concepts in orthognathic surgery","volume":"141","author":"Naran","year":"2018","journal-title":"Plast. Reconstr. Surg."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1970","DOI":"10.1016\/j.jcms.2014.09.001","article-title":"Augmented reality as an aid in maxillofacial surgery: Validation of a wearable system allowing maxillary repositioning","volume":"42","author":"Badiali","year":"2014","journal-title":"J. Cranio Maxill. Surg."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Cercenelli, L., Carbone, M., Condino, S., Cutolo, F., Marcelli, E., Tarsitano, A., Marchetti, C., Ferrari, V., and Badiali, G. (2020). The Wearable VOSTARS System for Augmented Reality-Guided Surgery: Preclinical Phantom Evaluation for High-Precision Maxillofacial Tasks. J. Clin. Med., 9.","DOI":"10.3390\/jcm9113562"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1356","DOI":"10.1016\/j.ajic.2018.05.022","article-title":"Evaluation of the bacterial burden of gel nails, standard nail polish, and natural nails on the hands of health care workers","volume":"46","author":"Hewlett","year":"2018","journal-title":"Am. J. Infect. Control."},{"key":"ref_47","unstructured":"Community BO (2022, January 12). Blender\u2014A 3D modelling and rendering package [Internet]. Stichting Blender Foundation, Amsterdam. Available online: http:\/\/www.blender.org."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"616","DOI":"10.1007\/s00464-011-1920-x","article-title":"Value of multidetector computed tomography image segmentation for preoperative planning in general surgery","volume":"26","author":"Ferrari","year":"2012","journal-title":"Surg. Endosc."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1217","DOI":"10.1111\/j.1365-2929.2004.02012.x","article-title":"Likert scales: How to (ab)use them","volume":"38","author":"Jamieson","year":"2004","journal-title":"Med. Educ."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/81\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:16:45Z","timestamp":1760134605000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/81"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,8]]},"references-count":49,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["info13020081"],"URL":"https:\/\/doi.org\/10.3390\/info13020081","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,8]]}}}