{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,8]],"date-time":"2026-03-08T17:11:50Z","timestamp":1772989910406,"version":"3.50.1"},"reference-count":30,"publisher":"Institution of Engineering and Technology (IET)","issue":"1","license":[{"start":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T00:00:00Z","timestamp":1763510400000},"content-version":"vor","delay-in-days":322,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["13GW0236B"],"award-info":[{"award-number":["13GW0236B"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["ietresearch.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Healthcare Tech Letters"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:p>Accurate labels of surgical procedures such as image segmentations or interaction labels are paramount for many of today's medical image computing tasks. Creating a dataset with these labels requires a great deal of manual work and relies on the involvement of medical experts, which is very time\u2010consuming and costly. We propose a pathway for the automatic generation of such labels utilizing the spatial and temporal registration between a patient, the anatomical model, tracked surgical instruments, and the surgeon's view of the patient. These requirements for the automatic generation of labels are identical to the requirements of many navigated and augmented reality (AR) enabled surgeries. The AR system, through 3D registration, has the defining ability to accurately overlay real objects with their virtual counterparts. Our approach collects the complete raw data (e.g. video, tracking data, calibrations etc.) that feeds a live laparoscopic AR system for later analysis. By converting these complete recordings of the surgery into different representations, the AR system generates valuable datasets as mere by\u2010products. Additionally, as our approach does not rely on visual input alone but on additional 3D information, the system can create labels even if the visual input is occluded or a tool interacts with tissue outside of the view of the laparoscopic camera. In this paper, we present a realization of this concept, then evolve this foundational idea into an interactive system that assists users in annotating surgical data. Finally, we gather and analyse feedback from six participants to evaluate the efficacy and user\u2010friendliness of our\u00a0system.<\/jats:p>","DOI":"10.1049\/htl2.70031","type":"journal-article","created":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T10:41:28Z","timestamp":1763548888000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Automatic Annotations by Augmented Reality\u2010Enabled Laparoscopic Surgery"],"prefix":"10.1049","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4896-7918","authenticated-orcid":false,"given":"Alexander","family":"Winkler","sequence":"first","affiliation":[{"name":"Chair for Computer Aided Medical Procedures and Augmented Reality (CAMP) Technical University of Munich (TUM) Munich Germany"},{"name":"Department of General, Visceral, and Transplantation Surgery Hospital of the Ludwig Maximilian University (LMU) Munich Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christian","family":"Heiliger","sequence":"additional","affiliation":[{"name":"Department of General, Visceral, and Transplantation Surgery Hospital of the Ludwig Maximilian University (LMU) Munich Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thomas","family":"Heiliger","sequence":"additional","affiliation":[{"name":"Department of General, Visceral, and Transplantation Surgery Hospital of the Ludwig Maximilian University (LMU) Munich Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ulrich","family":"Eck","sequence":"additional","affiliation":[{"name":"Chair for Computer Aided Medical Procedures and Augmented Reality (CAMP) Technical University of Munich (TUM) Munich Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Konrad","family":"Karcz","sequence":"additional","affiliation":[{"name":"Department of General, Visceral, and Transplantation Surgery Hospital of the Ludwig Maximilian University (LMU) Munich Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nassir","family":"Navab","sequence":"additional","affiliation":[{"name":"Chair for Computer Aided Medical Procedures and Augmented Reality (CAMP) Technical University of Munich (TUM) Munich Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"265","published-online":{"date-parts":[[2025,11,19]]},"reference":[{"key":"e_1_2_12_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2022.102433"},{"key":"e_1_2_12_3_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41597\u2010023\u201002073\u20107"},{"key":"e_1_2_12_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2023.102770"},{"key":"e_1_2_12_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11548\u2010019\u201002108\u20108"},{"key":"e_1_2_12_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464\u2010022\u201009487\u20101"},{"key":"e_1_2_12_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cag.2022.12.006"},{"key":"e_1_2_12_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISIE45552.2021.9576367"},{"key":"e_1_2_12_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102306"},{"key":"e_1_2_12_10_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12938\u2010022\u201001001\u2010x"},{"key":"e_1_2_12_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/JDT.2008.2001575"},{"key":"e_1_2_12_12_1","doi-asserted-by":"publisher","DOI":"10.3390\/jimaging9010004"},{"key":"e_1_2_12_13_1","doi-asserted-by":"crossref","unstructured":"I.Garc\u00eda\u2010Pereira J.Gimeno P.Morillo andP.Casanova\u2010Salas \u201cA Taxonomy of Augmented Reality Annotations \u201d inProceedings of the 15th International Joint Conference on Computer Vision Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) \u2010 GRAPP(INSTICC 2020) 412\u2013419.","DOI":"10.5220\/0009193404120419"},{"key":"e_1_2_12_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2005.06.040"},{"key":"e_1_2_12_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2005.57"},{"key":"e_1_2_12_16_1","doi-asserted-by":"crossref","unstructured":"S.\u2010A.Ahmadi T.Sielhorst R.Stauder M.Horn H.Feussner andN.Navab \u201cRecovery of Surgical Workflow Without Explicit Models \u201d inProceedings of the 9th International Conference on Medical Image Computing and Computer\u2010Assisted Intervention \u2010 Volume Part I ser. MICCAI'06 MICCAI Society (Springer\u2010Verlag 2006) 420\u2013428.","DOI":"10.1007\/11866565_52"},{"key":"e_1_2_12_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11548\u2010019\u201002008\u2010x"},{"key":"e_1_2_12_18_1","doi-asserted-by":"crossref","unstructured":"P. Z.Ramirez C.Paternesi L. D.Luigi L.Lella D. D.Gregorio andL. D.Stefano \u201cShooting Labels: 3D Semantic Labeling by Virtual Reality \u201d in2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)(IEEE 2020) 99\u2013106.","DOI":"10.1109\/AIVR50618.2020.00027"},{"key":"e_1_2_12_19_1","doi-asserted-by":"publisher","DOI":"10.1097\/XCS.0000000000000712"},{"key":"e_1_2_12_20_1","doi-asserted-by":"crossref","unstructured":"D.Pakhomov W.Shen andN.Navab \u201cTowards Unsupervised Learning for Instrument Segmentation in Robotic Surgery With Cycle\u2010Consistent Adversarial Networks \u201d in2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE 2020) 8499\u20138504.","DOI":"10.1109\/IROS45743.2020.9340816"},{"key":"e_1_2_12_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464\u2010022\u201009384\u20107"},{"key":"e_1_2_12_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00381\u2010021\u201005049\u20103"},{"key":"e_1_2_12_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11548\u2010022\u201002809\u20107"},{"key":"e_1_2_12_24_1","doi-asserted-by":"crossref","unstructured":"A.Martin\u2010Gomez U.Eck andN.Navab \u201cVisualization Techniques for Precise Alignment in VR: A Comparative Study \u201d in2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)(IEEE 2019) 735\u2013741 https:\/\/ieeexplore.ieee.org\/abstract\/document\/8798135.","DOI":"10.1109\/VR.2019.8798135"},{"key":"e_1_2_12_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.mri.2012.05.001"},{"key":"e_1_2_12_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11548\u2010018\u20101761\u20103"},{"key":"e_1_2_12_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464\u2010014\u20103504\u2010z"},{"key":"e_1_2_12_28_1","doi-asserted-by":"publisher","DOI":"10.3390\/app13031629"},{"key":"e_1_2_12_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/11795018_9"},{"key":"e_1_2_12_30_1","unstructured":"A.Radford J. W.Kim T.Xu G.Brockman C.Mcleavey andI.Sutskever \u201cRobust Speech Recognition via Large\u2010Scale Weak Supervision \u201d inProceedings of the 40th International Conference on Machine Learning ed.A.Krause E.Brunskill K.Cho B.Engelhardt S.Sabato andJ.Scarlett(ACM 2023) 28492\u201328518."},{"key":"e_1_2_12_31_1","first-page":"189","volume-title":"Usability Evaluation in Industry","author":"Brooke J.","year":"1996"}],"container-title":["Healthcare Technology Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ietresearch.onlinelibrary.wiley.com\/doi\/pdf\/10.1049\/htl2.70031","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/ietresearch.onlinelibrary.wiley.com\/doi\/full-xml\/10.1049\/htl2.70031","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/ietresearch.onlinelibrary.wiley.com\/doi\/pdf\/10.1049\/htl2.70031","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,8]],"date-time":"2026-03-08T12:08:09Z","timestamp":1772971689000},"score":1,"resource":{"primary":{"URL":"https:\/\/ietresearch.onlinelibrary.wiley.com\/doi\/10.1049\/htl2.70031"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1]]},"references-count":30,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["10.1049\/htl2.70031"],"URL":"https:\/\/doi.org\/10.1049\/htl2.70031","archive":["Portico"],"relation":{},"ISSN":["2053-3713","2053-3713"],"issn-type":[{"value":"2053-3713","type":"print"},{"value":"2053-3713","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1]]},"assertion":[{"value":"2025-10-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70031"}}