{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,22]],"date-time":"2026-05-22T02:22:53Z","timestamp":1779416573948,"version":"3.53.1"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T00:00:00Z","timestamp":1768348800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T00:00:00Z","timestamp":1768348800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100006260","name":"Technion - Israel Institute of Technology","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100006260","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    <jats:bold>Purpose<\/jats:bold>\n                    Surgical gestures are fundamental components of surgical procedures, encompassing actions such as cutting, suturing, and knot-tying. Gesture recognition plays a pivotal role in the automatic analysis of surgical data. Although recent advancements have improved surgical gesture recognition, much of the existing research relies on simulations or minimally invasive surgery data, failing to capture the complexities of open surgery. In this study, we introduce and employ a new open surgery dataset focused on closing incisions after saphenous vein harvesting.\n                    <jats:bold>Methods<\/jats:bold>\n                    Our goal is to improve gesture recognition accuracy by incorporating tool pose estimation and 3D hand pose predictions of surgeons. We employ MS-TCN++\u00a0\u00a0and LTContext\u00a0\u00a0for gesture recognition, and further enhance performance through an ensemble of models using different modalities\u2013video, tool pose, and hand pose data.\n                  <\/jats:p>\n                  <jats:p>\n                    <jats:bold>Results<\/jats:bold>\n                    The results reveal that using an ensemble model combining all three modalities provides a substantial improvement over video-only approaches, leading to statistically significant gains across multiple evaluation metrics. We further demonstrate that the model can rely solely on hand and tool poses, completely discarding the video input, while still achieving comparable performance. Additionally, we show that an ensemble model using only hand and tool poses produces results that are either: statistically significantly better than using video alone, or not statistically significantly different.\n                  <\/jats:p>\n                  <jats:p>\n                    <jats:bold>Conclusion<\/jats:bold>\n                    This study highlights the effectiveness of integrating multimodal data for surgical gesture recognition. By combining video, hand pose, and tool pose information, our approach achieves higher accuracy and robustness compared to video-only methods. Moreover, the comparable performance of pose-only models suggests a promising, privacy-preserving alternative for surgical data analysis.\n                  <\/jats:p>","DOI":"10.1007\/s11548-025-03564-1","type":"journal-article","created":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T06:46:52Z","timestamp":1768373212000},"page":"805-813","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Enhancing open-surgery gesture recognition using 3D pose estimation"],"prefix":"10.1007","volume":"21","author":[{"given":"Ori","family":"Meiraz","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shlomi","family":"Laufer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Spector","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Itay","family":"Or","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gil","family":"Bolotin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tom","family":"Friedman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,1,14]]},"reference":[{"key":"3564_CR1","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1007\/s00138-010-0298-4","volume":"23","author":"MAR Ahad","year":"2012","unstructured":"Ahad MAR, Tan JK, Kim H, Ishikawa S (2012) Motion history image: its variants and applications. Mach Vis Appl 23:255\u2013281. https:\/\/doi.org\/10.1007\/s00138-010-0298-4","journal-title":"Mach Vis Appl"},{"key":"3564_CR2","doi-asserted-by":"publisher","unstructured":"Bahrami E, Francesca G, Gall J (2023) How much temporal long-term context is needed for action segmentation? In: IEEE International Conference on Computer Vision (ICCV), https:\/\/doi.org\/10.1109\/ICCV51070.2023.00950","DOI":"10.1109\/ICCV51070.2023.00950"},{"key":"3564_CR3","doi-asserted-by":"publisher","unstructured":"Bertasius G, Wang H, Torresani L (2021) Is space-time attention all you need for video understanding? In: ICML, pp. 4, https:\/\/doi.org\/10.48550\/arXiv.2102.05095","DOI":"10.48550\/arXiv.2102.05095"},{"key":"3564_CR4","doi-asserted-by":"crossref","unstructured":"Bkheet E, D\u2019Angelo AL, Goldbraikh A, Laufer S (2023) Using hand pose estimation to automate open surgery training feedback. International Journal of Computer Assisted Radiology and Surgery 18(7):1279\u20131285","DOI":"10.1007\/s11548-023-02947-6"},{"key":"3564_CR5","doi-asserted-by":"publisher","unstructured":"Carreira J, Zisserman A (2017) Quo vadis, action recognition? a new model and the kinetics dataset. In: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 6299\u20136308, https:\/\/doi.org\/10.1109\/CVPR.2017.502","DOI":"10.1109\/CVPR.2017.502"},{"key":"3564_CR6","doi-asserted-by":"publisher","unstructured":"Chen Y, Zhang Z, Yuan C, Li B, Deng Y, Hu W (2021) Channel-wise topology refinement graph convolution for skeleton-based action recognition. In: Proceedings of the IEEE\/CVF international conference on computer vision, pp 13359\u201313368, https:\/\/doi.org\/10.1109\/ICCV48922.2021.01311","DOI":"10.1109\/ICCV48922.2021.01311"},{"key":"3564_CR7","doi-asserted-by":"publisher","unstructured":"Chi Hg, Ha MH, Chi S, Lee SW, Huang Q, Ramani K (2022) Infogcn: Representation learning for human skeleton-based action recognition. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 20186\u201320196, https:\/\/doi.org\/10.1109\/CVPR52688.2022.01955","DOI":"10.1109\/CVPR52688.2022.01955"},{"key":"3564_CR8","doi-asserted-by":"publisher","unstructured":"Czempiel T, Paschali M, Keicher M, Simson W, Feussner H, Kim ST, Navab N (2020) Tecno: Surgical phase recognition with multi-stage temporal convolutional networks. In: Medical Image Computing and Computer Assisted Intervention\u2013MICCAI 2020: 23rd International Conference, Lima, Peru, October 4\u20138, 2020, Proceedings, Part III 23, Springer, pp 343\u2013352, https:\/\/doi.org\/10.1007\/978-3-030-59716-0_33","DOI":"10.1007\/978-3-030-59716-0_33"},{"key":"3564_CR9","doi-asserted-by":"publisher","unstructured":"De\u00a0Rossi G, Roin S, Setti F, Muradore R (2020) A multi-modal learning system for on-line surgical action segmentation. In: 2020 International Symposium on Medical Robotics (ISMR), IEEE, pp 132\u2013138, https:\/\/doi.org\/10.1109\/ISMR48331.2020.9312950","DOI":"10.1109\/ISMR48331.2020.9312950"},{"key":"3564_CR10","doi-asserted-by":"publisher","unstructured":"De\u00a0Smedt Q, Wannous H, Vandeborre JP (2016) Skeleton-based dynamic hand gesture recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, https:\/\/doi.org\/10.1109\/CVPRW.2016.153","DOI":"10.1109\/CVPRW.2016.153"},{"issue":"11","key":"3564_CR11","doi-asserted-by":"publisher","first-page":"1325","DOI":"10.1111\/2041-210X.12584","volume":"7","author":"O Friard","year":"2016","unstructured":"Friard O, Gamba M (2016) Boris: a free, versatile open-source event-logging software for video\/audio coding and live observations. Methods Ecol Evol 7(11):1325\u20131330. https:\/\/doi.org\/10.1111\/2041-210X.12584","journal-title":"Methods Ecol Evol"},{"key":"3564_CR12","unstructured":"Gao Y, Vedula SS, Reiley CE, Ahmidi N, Varadarajan B, Lin HC, Tao L, Zappella L, B\u00e9jar B, Yuh DD, Chen C, Vidal R, Khudanpur S, Hager G (2014) Jhu-isi gesture and skill assessment working set (jigsaws): a surgical activity dataset for human motion modeling. In: MICCAI workshop: M2cai, p\u00a03"},{"key":"3564_CR13","doi-asserted-by":"publisher","unstructured":"Goldbraikh A, Avisdris N, Pugh CM, Laufer S (2022) Bounded future ms-tcn++ for surgical gesture recognition. In: European Conference on Computer Vision, Springer, pp 406\u2013421, https:\/\/doi.org\/10.1007\/978-3-031-25066-8_22","DOI":"10.1007\/978-3-031-25066-8_22"},{"key":"3564_CR14","doi-asserted-by":"publisher","unstructured":"Hamoud I, Srivastav V, Jamal MA, Mutter D, Mohareri O, Padoy N (2025) Multi-view video-pose pretraining for operating room surgical activity recognition. arXiv preprint arXiv:2502.13883https:\/\/doi.org\/10.48550\/arXiv.2502.13883","DOI":"10.48550\/arXiv.2502.13883"},{"key":"3564_CR15","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778, https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"3564_CR16","doi-asserted-by":"publisher","unstructured":"Lea C, Hager GD, Vidal R (2015) An improved model for segmentation and recognition of fine-grained activities with application to surgical training tasks. In: 2015 IEEE winter conference on applications of computer vision, IEEE, pp 1123\u20131129, https:\/\/doi.org\/10.1109\/WACV.2015.154","DOI":"10.1109\/WACV.2015.154"},{"key":"3564_CR17","doi-asserted-by":"publisher","unstructured":"Lea C, Flynn MD, Vidal R, Reiter A, Hager GD (2017) Temporal convolutional networks for action segmentation and detection. In: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 156\u2013165, https:\/\/doi.org\/10.1109\/CVPR.2017.113","DOI":"10.1109\/CVPR.2017.113"},{"issue":"6","key":"3564_CR18","doi-asserted-by":"publisher","first-page":"6647","DOI":"10.1109\/TPAMI.2020.3021756","volume":"45","author":"S Li","year":"2020","unstructured":"Li S, Farha YA, Liu Y, Cheng MM, Gall J (2020) Ms-tcn++: multi-stage temporal convolutional network for action segmentation. IEEE Trans Pattern Anal Mach Intell 45(6):6647\u20136658. https:\/\/doi.org\/10.1109\/TPAMI.2020.3021756","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"3564_CR19","doi-asserted-by":"publisher","DOI":"10.1002\/9781119482260","author":"RJ Little","year":"2019","unstructured":"Little RJ, Rubin DB (2019) Statistical analysis with missing data, vol 793. John Wiley & Sons. https:\/\/doi.org\/10.1002\/9781119482260","journal-title":"John Wiley & Sons"},{"issue":"17","key":"3564_CR20","doi-asserted-by":"publisher","first-page":"8021","DOI":"10.3390\/app14178021","volume":"14","author":"Y Men","year":"2024","unstructured":"Men Y, Luo J, Zhao Z, Wu H, Zhang G, Luo F, Yu M (2024) Research on surgical gesture recognition in open surgery based on fusion of r3d and multi-head attention mechanism. Appl Sci 14(17):8021. https:\/\/doi.org\/10.3390\/app14178021","journal-title":"Appl Sci"},{"key":"3564_CR21","doi-asserted-by":"publisher","unstructured":"\u00d6zsoy E, \u00d6rnek EP, Eck U, Czempiel T, Tombari F, Navab N (2022) 4d-or: Semantic scene graphs for or domain modeling. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp 475\u2013485, https:\/\/doi.org\/10.48550\/arXiv.2203.11937","DOI":"10.48550\/arXiv.2203.11937"},{"key":"3564_CR22","doi-asserted-by":"publisher","unstructured":"Pavlakos G, Shan D, Radosavovic I, Kanazawa A, Fouhey D, Malik J (2024) Reconstructing hands in 3D with transformers. In: CVPR, https:\/\/doi.org\/10.1109\/CVPR52733.2024.00938","DOI":"10.1109\/CVPR52733.2024.00938"},{"key":"3564_CR23","doi-asserted-by":"publisher","unstructured":"Saggio G, Santosuosso GL, Cavallo P, Pinto CA, Petrella M, Giannini F, Di\u00a0Lorenzo N, Lazzaro A, Corona A, D\u2019Auria F, Iezzi L, Gaspari AL (2011) Gesture recognition and classification for surgical skill assessment. In: 2011 IEEE International Symposium on Medical Measurements and Applications, IEEE, pp 662\u2013666, https:\/\/doi.org\/10.1109\/MeMeA.2011.5966681","DOI":"10.1109\/MeMeA.2011.5966681"},{"issue":"5","key":"3564_CR24","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1177\/1553350619853099","volume":"26","author":"TJ Saun","year":"2019","unstructured":"Saun TJ, Zuo KJ, Grantcharov TP (2019) Video technologies for recording open surgery: a systematic review. Surg Innov 26(5):599\u2013612. https:\/\/doi.org\/10.1177\/1553350619853099","journal-title":"Surg Innov"},{"issue":"8","key":"3564_CR25","doi-asserted-by":"publisher","first-page":"1627","DOI":"10.1021\/ac60214a047","volume":"36","author":"A Savitzky","year":"1964","unstructured":"Savitzky A, Golay MJ (1964) Smoothing and differentiation of data by simplified least squares procedures. Anal Chem 36(8):1627\u20131639. https:\/\/doi.org\/10.1021\/ac60214a047","journal-title":"Anal Chem"},{"key":"3564_CR26","doi-asserted-by":"publisher","unstructured":"Sharghi A, Haugerud H, Oh D, Mohareri O (2020) Automatic operating room surgical activity recognition for robot-assisted surgery. In: Medical Image Computing and Computer Assisted Intervention\u2013MICCAI 2020: 23rd International Conference, Lima, Peru, October 4\u20138, 2020, Proceedings, Part III 23, Springer, pp 385\u2013395, https:\/\/doi.org\/10.1007\/978-3-030-59716-0_37","DOI":"10.1007\/978-3-030-59716-0_37"},{"key":"3564_CR27","doi-asserted-by":"publisher","unstructured":"Spektor R, Friedman T, Or I, Bolotin G, Laufer S (2025) Monocular pose estimation of articulated open surgery tools-in the wild. Medical Image Analysis. https:\/\/doi.org\/10.1016\/j.media.2025.103618","DOI":"10.1016\/j.media.2025.103618"},{"key":"3564_CR28","unstructured":"Tan M, Le Q (2021) Efficientnetv2: Smaller models and faster training. In: International conference on machine learning, PMLR, pp 10096\u201310106"},{"issue":"1","key":"3564_CR29","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/TMI.2016.2593957","volume":"36","author":"AP Twinanda","year":"2016","unstructured":"Twinanda AP, Shehata S, Mutter D, Marescaux J, De Mathelin M, Padoy N (2016) Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36(1):86\u201397. https:\/\/doi.org\/10.1109\/TMI.2016.2593957","journal-title":"IEEE Trans Med Imaging"},{"key":"3564_CR30","doi-asserted-by":"publisher","unstructured":"Wei H, Xie R, Cheng H, Feng L, An B, Li Y (2022) Mitigating neural network overconfidence with logit normalization. In: International conference on machine learning, PMLR, pp 23631\u201323644, https:\/\/doi.org\/10.48550\/arXiv.2205.09310","DOI":"10.48550\/arXiv.2205.09310"},{"key":"3564_CR31","doi-asserted-by":"publisher","unstructured":"Wilcoxon F (1992) Individual comparisons by ranking methods. In: Breakthroughs in statistics: Methodology and distribution. Springer, p 196\u2013202, https:\/\/doi.org\/10.1007\/978-1-4612-4380-9_16","DOI":"10.1007\/978-1-4612-4380-9_16"},{"key":"3564_CR32","doi-asserted-by":"crossref","unstructured":"Yi F, Wen H, Jiang T (2021) Asformer: Transformer for action segmentation. In: The British Machine Vision Conference (BMVC)","DOI":"10.5244\/C.35.49"},{"issue":"4","key":"3564_CR33","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1007\/s11548-022-02811-z","volume":"18","author":"B Zhang","year":"2023","unstructured":"Zhang B, Goel B, Sarhan MH, Goel VK, Abukhalil R, Kalesan B, Stottler N, Petculescu S (2023) Surgical workflow recognition with temporal convolution and transformer for action segmentation. Int J Comput Assist Radiol Surg 18(4):785\u2013794. https:\/\/doi.org\/10.1007\/s11548-022-02811-z","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3564_CR34","doi-asserted-by":"publisher","unstructured":"Zhang J, Wang Y, Tang J, Zou J, Fan S (2021) Ms-tcn: A multiscale temporal convolutional network for fault diagnosis in industrial processes. In: 2021 American Control Conference (ACC), IEEE, pp 1601\u20131606, https:\/\/doi.org\/10.23919\/ACC50511.2021.9482728","DOI":"10.23919\/ACC50511.2021.9482728"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-025-03564-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-025-03564-1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-025-03564-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,22]],"date-time":"2026-05-22T01:30:21Z","timestamp":1779413421000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-025-03564-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,14]]},"references-count":34,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,4]]}},"alternative-id":["3564"],"URL":"https:\/\/doi.org\/10.1007\/s11548-025-03564-1","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,14]]},"assertion":[{"value":"28 June 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 December 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 January 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The study was approved by the Rambam Health Care Campus IRB committee.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Informed consent was obtained from all individual participants included in the study.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}}]}}