{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T20:59:08Z","timestamp":1771880348357,"version":"3.50.1"},"reference-count":24,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,8,18]],"date-time":"2025-08-18T00:00:00Z","timestamp":1755475200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,18]],"date-time":"2025-08-18T00:00:00Z","timestamp":1755475200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002341","name":"Research Council of Finland","doi-asserted-by":"crossref","award":["345448"],"award-info":[{"award-number":["345448"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100014438","name":"Business Finland","doi-asserted-by":"crossref","award":["80\/31\/2023"],"award-info":[{"award-number":["80\/31\/2023"]}],"id":[{"id":"10.13039\/501100014438","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>\n                      <jats:bold>Purpose:<\/jats:bold>\n                    <\/jats:title>\n                    <jats:p>Artificial intelligence (AI) achieves high-quality annotations of radiological images, yet often lacks the robustness required in clinical practice. Interactive annotation starts with an AI-generated delineation, allowing radiologists to refine it with feedback, potentially improving precision and reliability. These techniques have been explored in two-dimensional desktop environments, but are not validated by radiologists or integrated with immersive visualization technologies. We used a Virtual Reality (VR) system to determine whether (1) the annotation quality improves when radiologists can edit the AI annotation and (2) whether the extra work done by editing is worthwhile.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>\n                      <jats:bold>Methods:<\/jats:bold>\n                    <\/jats:title>\n                    <jats:p>We evaluated the clinical feasibility of an interactive VR approach to annotate mandibular and mental foramina on segmented 3D mandibular models. Three experienced dentomaxillofacial radiologists reviewed AI-generated annotations and, when needed, refined them at the voxel level in 3D space through click-based interactions until clinical standards were met.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>\n                      <jats:bold>Results:<\/jats:bold>\n                    <\/jats:title>\n                    <jats:p>Our results indicate that integrating expert feedback within an immersive VR environment enhances annotation accuracy, improves clinical usability, and offers valuable insights for developing medical image analysis systems incorporating radiologist input.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>\n                      <jats:bold>Conclusion:<\/jats:bold>\n                    <\/jats:title>\n                    <jats:p>This study is the first to compare the quality of original and interactive AI annotation and to use radiologists\u2019 opinions as the measure. More research is needed for generalization.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s11548-025-03497-9","type":"journal-article","created":{"date-parts":[[2025,8,18]],"date-time":"2025-08-18T04:18:04Z","timestamp":1755490684000},"page":"49-58","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Interactive AI annotation of medical images in a virtual reality environment"],"prefix":"10.1007","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-1738-4965","authenticated-orcid":false,"given":"Lotta","family":"Orsmaa","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3735-5860","authenticated-orcid":false,"given":"Mikko","family":"Saukkoriipi","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1970-1862","authenticated-orcid":false,"given":"Jari","family":"Kangas","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5015-3984","authenticated-orcid":false,"given":"Nastaran","family":"Rasouli","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1503-3078","authenticated-orcid":false,"given":"Jorma","family":"J\u00e4rnstedt","sequence":"additional","affiliation":[]},{"given":"Helena","family":"Mehtonen","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7364-6502","authenticated-orcid":false,"given":"Jaakko","family":"Sahlsten","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6154-2378","authenticated-orcid":false,"given":"Joel","family":"Jaskari","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3805-9687","authenticated-orcid":false,"given":"Kimmo","family":"Kaski","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3276-7866","authenticated-orcid":false,"given":"Roope","family":"Raisamo","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,8,18]]},"reference":[{"issue":"17","key":"3497_CR1","doi-asserted-by":"publisher","first-page":"1237","DOI":"10.1056\/NEJM199304293281706","volume":"328","author":"WC Black","year":"1993","unstructured":"Black WC, Welch HG (1993) Advances in diagnostic imaging and overestimations of disease prevalence and the benefits of therapy. N Engl J Med 328(17):1237\u20131243","journal-title":"N Engl J Med"},{"issue":"12","key":"3497_CR2","doi-asserted-by":"publisher","first-page":"2801","DOI":"10.1097\/CORR.0000000000001398","volume":"478","author":"AML Meesters","year":"2020","unstructured":"Meesters AML, Ten Duis K, Banierink H, Stirler VMA, Wouters PCR, Kraeima J, Vries J-PPM, Witjes MJH, IJpma FFA (2020) What are the interobserver and intraobserver variability of gap and stepoff measurements in acetabular fractures? Clin Orthop Relat Res 478(12):2801\u20132808","journal-title":"Clin Orthop Relat Res"},{"issue":"4","key":"3497_CR3","doi-asserted-by":"publisher","first-page":"1529","DOI":"10.1007\/s10278-024-00981-7","volume":"37","author":"W Yao","year":"2024","unstructured":"Yao W, Bai J, Liao W, Chen Y, Liu M, Xie Y (2024) From CNN to transformer: a review of medical image segmentation models. J Imaging Informat Med 37(4):1529\u20131547","journal-title":"J Imaging Informat Med"},{"issue":"6","key":"3497_CR4","doi-asserted-by":"publisher","first-page":"3576","DOI":"10.1007\/s00330-020-06672-5","volume":"30","author":"MP Recht","year":"2020","unstructured":"Recht MP, Dewey M, Dreyer K, Langlotz C, Niessen W, Prainsack B, Smith JJ (2020) Integrating artificial intelligence into the clinical practice of radiology: challenges and recommendations. Eur Radiol 30(6):3576\u20133584","journal-title":"Eur Radiol"},{"issue":"7","key":"3497_CR5","doi-asserted-by":"publisher","first-page":"1562","DOI":"10.1109\/TMI.2018.2791721","volume":"37","author":"G Wang","year":"2018","unstructured":"Wang G, Li W, Zuluaga MA, Pratt R, Patel PA, Aertsen M, Doel T, David AL, Deprest J, Ourselin S, Vercauteren T (2018) Interactive medical image segmentation using deep learning with image-specific fine tuning. IEEE Trans Med Imaging 37(7):1562\u20131573","journal-title":"IEEE Trans Med Imaging"},{"key":"3497_CR6","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","volume":"58","author":"A Barredo Arrieta","year":"2020","unstructured":"Barredo Arrieta A, D\u00edaz-Rodr\u00edguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, Garcia S, Gil-Lopez S, Molina D, Benjamins R, Chatila R, Herrera F (2020) Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf Fusion 58:82\u2013115","journal-title":"Inf Fusion"},{"issue":"4","key":"3497_CR7","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1007\/s41095-020-0177-5","volume":"6","author":"H Ramadan","year":"2020","unstructured":"Ramadan H, Lachqar C, Tairi H (2020) A survey of recent interactive image segmentation methods. Comput Visual Media 6(4):355\u2013384","journal-title":"Comput Visual Media"},{"key":"3497_CR8","doi-asserted-by":"publisher","unstructured":"Sofiiuk K, Petrov IA, Konushin A (2022) Reviving iterative training with mask guidance for interactive segmentation. In: 2022 IEEE international conference on image processing (ICIP), pp 3141\u20133145. https:\/\/doi.org\/10.1109\/ICIP46576.2022.9897365","DOI":"10.1109\/ICIP46576.2022.9897365"},{"issue":"6","key":"3497_CR9","doi-asserted-by":"publisher","first-page":"775","DOI":"10.1016\/S0097-8493(96)00050-7","volume":"20","author":"KJ Zuiderveld","year":"1996","unstructured":"Zuiderveld KJ, Koning AHJ, Stokking R, Maintz JBA, Appelman FJR, Viergever MA (1996) Multimodality visualization of medical volume data. Comput Graph 20(6):775\u2013791","journal-title":"Comput Graph"},{"issue":"1","key":"3497_CR10","doi-asserted-by":"publisher","first-page":"384","DOI":"10.1186\/s40064-015-1170-9","volume":"4","author":"SAZBS Aluwee","year":"2015","unstructured":"Aluwee SAZBS, Kato H, Zhou X, Hara T, Fujita H, Kanematsu M, Furui T, Yano R, Miyai N, Morishige K-I (2015) Magnetic resonance imaging of uterine fibroids: a preliminary investigation into the usefulness of 3d-rendered images for surgical planning. Springerplus 4(1):384\u2013384","journal-title":"Springerplus"},{"issue":"7","key":"3497_CR11","doi-asserted-by":"publisher","first-page":"1865","DOI":"10.1097\/SCS.0000000000006525","volume":"31","author":"S Zawy Alsofy","year":"2020","unstructured":"Zawy Alsofy S, Sakellaropoulou I, Stroop R (2020) Evaluation of surgical approaches for tumor resection in the deep infratentorial region and impact of virtual reality technique for the surgical planning and strategy. J Craniofac Surg 31(7):1865\u20131869","journal-title":"J Craniofac Surg"},{"issue":"6","key":"3497_CR12","doi-asserted-by":"publisher","first-page":"166","DOI":"10.1007\/s10916-019-1308-3","volume":"43","author":"S Ammanuel","year":"2019","unstructured":"Ammanuel S, Brown I, Uribe J, Rehani B (2019) Creating 3d models from radiologic images for virtual reality medical education modules. J Med Syst 43(6):166\u20133","journal-title":"J Med Syst"},{"key":"3497_CR13","doi-asserted-by":"publisher","unstructured":"Orsmaa L, Kangas J, Rasouli N, Jaskari J, Sahlsten J, Mehtonen H, J\u00e4rnstedt J, Kaski K, Raisamo R (2025) Diving, grabbing and teleporting: Methods for medical 3d image manipulation in vr. Interact Comput. https:\/\/doi.org\/10.1093\/iwc\/iwae061","DOI":"10.1093\/iwc\/iwae061"},{"issue":"12","key":"3497_CR14","doi-asserted-by":"publisher","first-page":"81","DOI":"10.3390\/mti5120081","volume":"5","author":"I Rakkolainen","year":"2021","unstructured":"Rakkolainen I, Farooq A, Kangas J, Hakulinen J, Rantala J, Turunen M, Raisamo R (2021) Technologies for multimodal interaction in extended reality\u2014a scoping review. Multimod Technol Interact 5(12):81","journal-title":"Multimod Technol Interact"},{"key":"3497_CR15","unstructured":"OpenXR: OpenXR toolkit. http:\/\/mbucchia.github.io\/OpenXR-Toolkit\/. Accessed 30 Sep 2023"},{"key":"3497_CR16","unstructured":"Lavik M. Unity volume rendering. https:\/\/github.com\/mlavik1\/UnityVolumeRendering. Accessed 30 Sep 2024"},{"issue":"1","key":"3497_CR17","doi-asserted-by":"publisher","first-page":"18598","DOI":"10.1038\/s41598-022-20605-w","volume":"12","author":"J J\u00e4rnstedt","year":"2022","unstructured":"J\u00e4rnstedt J, Sahlsten J, Jaskari J, Kaski K, Mehtonen H, Lin Z, Hietanen A, Sundqvist O, Varjonen V, Mattila V et al (2022) Comparison of deep learning segmentation and multigrader-annotated mandibular canals of multicenter cbct scans. Sci Rep 12(1):18598","journal-title":"Sci Rep"},{"issue":"1","key":"3497_CR18","doi-asserted-by":"publisher","first-page":"14159","DOI":"10.1038\/s41598-023-40516-8","volume":"13","author":"J J\u00e4rnstedt","year":"2023","unstructured":"J\u00e4rnstedt J, Sahlsten J, Jaskari J, Kaski K, Mehtonen H, Hietanen A, Sundqvist O, Varjonen V, Mattila V, Prapayasatok S et al (2023) Reproducibility analysis of automated deep learning based localisation of mandibular canals on a temporal cbct dataset. Sci Rep 13(1):14159","journal-title":"Sci Rep"},{"key":"3497_CR19","doi-asserted-by":"crossref","unstructured":"Saukkoriipi M, Sahlsten J, Jaskari J, Orsmaa L, Kangas J, Rasouli N, Raisamo R, Hirvonen J, Mehtonen H, J\u00e4rnstedt J, M\u00e4kitie A, Naser M, Fuller C, Kann B, Kaski K (2024) Interactive 3D segmentation for primary gross tumor volume in oropharyngeal cancer. arXiv:2409.06605","DOI":"10.21203\/rs.3.rs-6221538\/v1"},{"key":"3497_CR20","unstructured":"Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7\u20139, 2015, conference track proceedings. arXiv:1412.6980"},{"key":"3497_CR21","doi-asserted-by":"crossref","unstructured":"Lewis JR, Utesch BS, Maher DE (2013) Umux-lite: when there\u2019s no time for the sus. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 2099\u20132102","DOI":"10.1145\/2470654.2481287"},{"issue":"3","key":"3497_CR22","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1191\/1478088706qp063oa","volume":"2","author":"V Braun","year":"2006","unstructured":"Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 2(3):77\u2013101","journal-title":"Qual Res Psychol"},{"key":"3497_CR23","doi-asserted-by":"crossref","unstructured":"Spaanderman DJ, Starmans MPA, Erp GCM, Hanff DF, Sluijter JH, Schut A-RW, Leenders GJLH, Verhoef C, Gr\u00fcnhagen DJ, Niessen WJ, Visser JJ, Klein S (2024) Minimally interactive segmentation of soft-tissue tumors on ct and mri using deep learning. Eur Radiol","DOI":"10.1007\/s00330-024-11167-8"},{"key":"3497_CR24","doi-asserted-by":"publisher","first-page":"100408","DOI":"10.1016\/j.phro.2022.12.005","volume":"25","author":"Z Wei","year":"2023","unstructured":"Wei Z, Ren J, Korreman SS, Nijkamp J (2023) Towards interactive deep-learning for tumour segmentation in head and neck cancer radiotherapy. Phys Imaging Radiat Oncol 25:100408\u2013100408","journal-title":"Phys Imaging Radiat Oncol"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-025-03497-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-025-03497-9","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-025-03497-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T20:04:03Z","timestamp":1771877043000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-025-03497-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,18]]},"references-count":24,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,1]]}},"alternative-id":["3497"],"URL":"https:\/\/doi.org\/10.1007\/s11548-025-03497-9","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,18]]},"assertion":[{"value":"10 January 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 July 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 August 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All authors declare that they have no Conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}