{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T12:27:25Z","timestamp":1782822445927,"version":"3.54.5"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2023,3,22]],"date-time":"2023-03-22T00:00:00Z","timestamp":1679443200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,22]],"date-time":"2023-03-22T00:00:00Z","timestamp":1679443200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010665","name":"H2020 Marie Sklodowska-Curie Actions","doi-asserted-by":"publisher","award":["813782"],"award-info":[{"award-number":["813782"]}],"id":[{"id":"10.13039\/100010665","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100016308","name":"Bpifrance","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100016308","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Agence nationale de la recherche","award":["ANR-16-CE33-0009"],"award-info":[{"award-number":["ANR-16-CE33-0009"]}]},{"name":"Agence nationale de la recherche","award":["ANR-10-IAHU-02"],"award-info":[{"award-number":["ANR-10-IAHU-02"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Purpose<\/jats:title>\n                <jats:p>Automatic recognition of surgical activities from intraoperative surgical videos is crucial for developing intelligent support systems for computer-assisted interventions. Current state-of-the-art recognition methods are based on deep learning where data augmentation has shown the potential to improve the generalization of these methods. This has spurred work on automated and simplified augmentation strategies for image classification and object detection on datasets of still images. Extending such augmentation methods to videos is not straightforward, as the temporal dimension needs to be considered. Furthermore, surgical videos pose additional challenges as they are composed of multiple, interconnected, and long-duration activities.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>This work proposes a new simplified augmentation method, called <jats:italic>TRandAugment<\/jats:italic>, specifically designed for long surgical videos, that treats each video as an assemble of temporal segments and applies consistent but random transformations to each segment. The proposed augmentation method is used to train an end-to-end spatiotemporal model consisting of a CNN (ResNet50) followed by a TCN.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The effectiveness of the proposed method is demonstrated on two surgical video datasets, namely Bypass40 and CATARACTS, and two tasks, surgical phase and step recognition. <jats:italic>TRandAugment<\/jats:italic> adds a performance boost of 1\u20136% over previous state-of-the-art methods, that uses manually designed augmentations.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>This work presents a simplified and automated augmentation method for long surgical videos. The proposed method has been validated on different datasets and tasks indicating the importance of devising temporal augmentation methods for long surgical videos.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s11548-023-02864-8","type":"journal-article","created":{"date-parts":[[2023,3,22]],"date-time":"2023-03-22T03:03:11Z","timestamp":1679454191000},"page":"1665-1672","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["TRandAugment: temporal random augmentation strategy for surgical activity recognition from videos"],"prefix":"10.1007","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3452-4764","authenticated-orcid":false,"given":"Sanat","family":"Ramesh","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Diego","family":"Dall\u2019Alba","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cristians","family":"Gonzalez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tong","family":"Yu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pietro","family":"Mascagni","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Didier","family":"Mutter","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jacques","family":"Marescaux","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paolo","family":"Fiorini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicolas","family":"Padoy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,3,22]]},"reference":[{"issue":"9","key":"2864_CR1","doi-asserted-by":"publisher","first-page":"691","DOI":"10.1038\/s41551-017-0132-7","volume":"1","author":"L Maier-Hein","year":"2017","unstructured":"Maier-Hein L, Vedula SS, Speidel S, Navab N, Kikinis R, Park A, Eisenmann M, Feussner H, Forestier G, Giannarou S, Hashizume M, Katic D, Kenngott H, Kranzfelder M, Malpani A, M\u00e4rz K, Neumuth T, Padoy N, Pugh C, Schoch N, Stoyanov D, Taylor R, Wagner M, Hager GD, Jannin P (2017) Surgical data science for next-generation interventions. Nat Biomed Eng 1(9):691\u2013696","journal-title":"Nat Biomed Eng"},{"issue":"1","key":"2864_CR2","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1109\/JPROC.2019.2946993","volume":"108","author":"T Vercauteren","year":"2020","unstructured":"Vercauteren T, Unberath M, Padoy N, Navab N (2020) Cai4cai: the rise of contextual artificial intelligence in computer-assisted interventions. Proc IEEE 108(1):198\u2013214","journal-title":"Proc IEEE"},{"issue":"5","key":"2864_CR3","doi-asserted-by":"publisher","first-page":"1681","DOI":"10.1007\/s00464-012-2656-y","volume":"27","author":"M Kranzfelder","year":"2013","unstructured":"Kranzfelder M, Staub C, Fiolka A, Schneider A, Gillen S, Wilhelm D, Friess H, Knoll A, Feussner H (2013) Toward increased autonomy in the surgical OR: needs, requests, and expectations. Surg Endosc 27(5):1681\u20131688","journal-title":"Surg Endosc"},{"issue":"9","key":"2864_CR4","doi-asserted-by":"publisher","first-page":"1427","DOI":"10.1007\/s11548-015-1222-1","volume":"10","author":"D Kati\u0107","year":"2015","unstructured":"Kati\u0107 D, Julliard C, Wekerle A-L, Kenngott H, M\u00fcller-Stich BP, Dillmann R, Speidel S, Jannin P, Gibaud B (2015) LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition. Int J Comput Assist Radiol Surg 10(9):1427\u20131434","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"9","key":"2864_CR5","doi-asserted-by":"publisher","first-page":"4918","DOI":"10.1007\/s00464-021-08578-9","volume":"35","author":"OR Meireles","year":"2021","unstructured":"Meireles OR, Rosman G, Altieri MS, Carin L, Hager G, Madani A, Padoy N, Pugh CM, Sylla P, Ward TM (2021) D.A.H.: SAGES consensus recommendations on an annotation framework for surgical video. Surg Endosc 35(9):4918\u20134929","journal-title":"Surg Endosc"},{"issue":"1","key":"2864_CR6","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/TMI.2016.2593957","volume":"36","author":"AP Twinanda","year":"2017","unstructured":"Twinanda AP, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N (2017) EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36(1):86\u201397","journal-title":"IEEE Trans Med Imaging"},{"key":"2864_CR7","doi-asserted-by":"crossref","unstructured":"Zisimopoulos O, Flouty E, Luengo I, Giataganas P, Nehme J, Chow A, Stoyanov D (2018) DeepPhase: surgical phase recognition in cataracts videos. In: International conference on medical image computing and computer-assisted intervention, pp. 265\u2013272","DOI":"10.1007\/978-3-030-00937-3_31"},{"key":"2864_CR8","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2019.101572","volume":"59","author":"Y Jin","year":"2020","unstructured":"Jin Y, Li H, Dou Q, Chen H, Qin J, Fu C, Heng P (2020) Multi-task recurrent convolutional network with correlation loss for surgical video analysis. Med Image Anal 59:101572","journal-title":"Med Image Anal"},{"key":"2864_CR9","doi-asserted-by":"crossref","unstructured":"Czempiel T, Paschali M, Keicher M, Simson W, Feussner H, Kim ST, Navab N (2020)TeCNO: surgical phase recognition with multi-stage temporal convolutional networks. In: International conference on medical image computing and computer-assisted intervention, pp. 343\u2013352","DOI":"10.1007\/978-3-030-59716-0_33"},{"key":"2864_CR10","doi-asserted-by":"publisher","first-page":"1111","DOI":"10.1007\/s11548-021-02388-z","volume":"16","author":"S Ramesh","year":"2021","unstructured":"Ramesh S, DallAlba D, Gonzalez C, Yu T, Mascagni P, Mutter D, Padoy N (2021) Multi-task temporal convolutional networks for joint recognition of surgical phases and steps in gastric bypass procedures. Int J Comput Assist Radiol Surg 16:1111\u20131119","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"2864_CR11","doi-asserted-by":"crossref","unstructured":"Czempiel T, Paschali M, Ostler D, Kim ST, Busam B, Navab N (2021) OperA: Attention-regularized transformers for surgical phase recognition. In: International conference on medical image computing and computer-assisted intervention, pp. 604\u2013614","DOI":"10.1007\/978-3-030-87202-1_58"},{"key":"2864_CR12","doi-asserted-by":"crossref","unstructured":"Gao X, Jin Y, Long Y, Dou Q, Heng P-A (2021) Trans-SVNet: accurate phase recognition from surgical videos via hybrid embedding aggregation transformer. In: International conference on medical image computing and computer-assisted intervention, pp. 593\u2013603","DOI":"10.1007\/978-3-030-87202-1_57"},{"key":"2864_CR13","unstructured":"Ho D, Liang E, Chen X, Stoica I, Abbeel P (2019) Population Based Augmentation: Efficient Learning of Augmentation Policy Schedules. In: Proceedings of the 36th international conference on machine learning, vol. 97, pp. 2731\u20132741"},{"key":"2864_CR14","doi-asserted-by":"crossref","unstructured":"Cubuk ED, Zoph B, Mane D, Vasudevan V, Le QV (2019) Autoaugment: Learning augmentation strategies from data. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 113\u2013123","DOI":"10.1109\/CVPR.2019.00020"},{"key":"2864_CR15","unstructured":"Lim S, Kim I, Kim T, Kim C, Kim S(2019) Fast autoaugment. Adv Neural Inform Process Syst 32"},{"key":"2864_CR16","doi-asserted-by":"crossref","unstructured":"He K, Gkioxari G, Dollar P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision (ICCV)","DOI":"10.1109\/ICCV.2017.322"},{"key":"2864_CR17","doi-asserted-by":"crossref","unstructured":"Kimata J, Nitta T, Tamaki T (2022) Objectmix: Data augmentation by copy-pasting objects in videos for action recognition. arXiv preprint arXiv:2204.00239","DOI":"10.1145\/3551626.3564941"},{"key":"2864_CR18","doi-asserted-by":"crossref","unstructured":"Fang H-S, Sun J, Wang R, Gou M, Li Y-L, Lu C (2019) Instaboost: boosting instance segmentation via probability map guided copy-pasting. In: Proceedings of the IEEE\/CVF international conference on computer vision, pp. 682\u2013691","DOI":"10.1109\/ICCV.2019.00077"},{"key":"2864_CR19","unstructured":"Ford N, Gilmer J, Carlini N, Cubuk ED (2019) Adversarial examples are a natural consequence of test error in noise. In: International conference on machine learning"},{"key":"2864_CR20","doi-asserted-by":"crossref","unstructured":"Ramesh S, Srivastav V, Alapatt D, Yu T, Murali A, Sestini L, Nwoye CI, Hamoud I, Sharma S, Fleurentin A, Exarchakis G, Karargyris A, Padoy N (2022) Dissecting Self-Supervised Learning Methods for Surgical Computer Vision. arXiv preprint arXiv:2207.00449","DOI":"10.1016\/j.media.2023.102844"},{"key":"2864_CR21","doi-asserted-by":"crossref","unstructured":"Qian R, Meng T, Gong B, Yang M-H, Wang H, Belongie S, Cui Y(2021) Spatiotemporal contrastive video representation learning. In: 2021 IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 6960\u20136970","DOI":"10.1109\/CVPR46437.2021.00689"},{"key":"2864_CR22","doi-asserted-by":"crossref","unstructured":"Pan T, Song Y, Yang T, Jiang W, Liu W (2021) Videomoco: contrastive video representation learning with temporally adversarial examples. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 11205\u201311214","DOI":"10.1109\/CVPR46437.2021.01105"},{"key":"2864_CR23","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102158","volume":"73","author":"X Shi","year":"2021","unstructured":"Shi X, Jin Y, Dou Q, Heng P-A (2021) Semi-supervised learning with progressive unlabeled data excavation for label-efficient surgical workflow recognition. Med Image Anal 73:102158","journal-title":"Med Image Anal"},{"key":"2864_CR24","doi-asserted-by":"crossref","unstructured":"Cubuk ED, Zoph B, Shlens J, Le QV (2020) Randaugment: Practical automated data augmentation with a reduced search space. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition workshops, pp. 702\u2013703","DOI":"10.1109\/CVPRW50498.2020.00359"},{"key":"2864_CR25","doi-asserted-by":"crossref","unstructured":"Kim T, Lee H, Cho M, Lee HS, Cho DH, Lee S (2020) Learning temporally invariant and localizable features via data augmentation for video recognition. In: Computer vision \u2013 ECCV 2020 workshops, pp. 386\u2013403","DOI":"10.1007\/978-3-030-66096-3_27"},{"key":"2864_CR26","doi-asserted-by":"crossref","unstructured":"Gowda SN, Rohrbach M, Keller F, Sevilla-Lara L (2022) Learn2augment: learning to composite videos for data augmentation in action recognition. In: European conference on computer vision, pp. 242\u2013259","DOI":"10.1007\/978-3-031-19821-2_14"},{"key":"2864_CR27","unstructured":"Kim T, Kim J, Shim M, Yun S, Kang M, Wee D, Lee S (2022) Exploring temporally dynamic data augmentation for video recognition. arXiv preprint arXiv:2206.15015"},{"key":"2864_CR28","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1016\/j.media.2018.11.008","volume":"52","author":"HA Hajj","year":"2019","unstructured":"Hajj HA, Lamard M, Conze P-H, Roychowdhury S, Hu X, Mar\u0161alkait\u0117 G, Zisimopoulos O, Dedmari MA, Zhao F, Prellberg J, Sahu M, Galdran A, Ara\u00fajo T, Vo DM, Panda C, Dahiya N, Kondo S, Bian Z, Vahdat A, Bialopetravi\u010dius J, Flouty E, Qiu C, Dill S, Mukhopadhyay A, Costa P, Aresta G, Ramamurthy S, Lee S-W, Campilho A, Zachow S, Xia S, Conjeti S, Stoyanov D, Armaitis J, Heng P-A, Macready WG, Cochener B, Quellec G (2019) CATARACTS: challenge on automatic tool annotation for CATARACT surgery. Med Image Anal 52:24\u201341","journal-title":"Med Image Anal"},{"key":"2864_CR29","unstructured":"Han J, Fang P, Li W, Hong J, Armin MA, Reid I, Petersson L, Li H (2022) You only cut once: boosting data augmentation with a single cut. In: Proceedings of the 39th international conference on machine learning, vol. 162, pp. 8196\u20138212"},{"key":"2864_CR30","doi-asserted-by":"crossref","unstructured":"Zhong Z, Zheng L, Kang G, Li S, Yang Y (2020) Random erasing data augmentation. In: Proceedings of the AAAI conference on artificial intelligence (AAAI)","DOI":"10.1609\/aaai.v34i07.7000"},{"key":"2864_CR31","doi-asserted-by":"crossref","unstructured":"Yun S, Han D, Oh SJ, Chun S, Choe J, Yoo YJ (2019) Cutmix: regularization strategy to train strong classifiers with localizable features. In: 2019 IEEE\/CVF international conference on computer vision (ICCV), 6022\u20136031","DOI":"10.1109\/ICCV.2019.00612"},{"key":"2864_CR32","unstructured":"Hendrycks, D, Mu N, Cubuk ED, Zoph B, Gilmer J, Lakshminarayanan B (2020) AugMix: a simple data processing method to improve robustness and uncertainty. In: Proceedings of the international conference on learning representations (ICLR)"},{"key":"2864_CR33","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision, pp. 630\u2013645","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"2864_CR34","unstructured":"Yu T, Mutter D, Marescaux J, Padoy N (2019) Learning from a tiny dataset of manual annotations: a teacher\/student approach for surgical phase recognition. In: International conference on information processing in computer-assisted interventions (IPCAI)"},{"issue":"21","key":"2864_CR35","doi-asserted-by":"publisher","first-page":"22473","DOI":"10.1007\/s11042-017-4793-8","volume":"76","author":"K Charri\u00e8re","year":"2017","unstructured":"Charri\u00e8re K, Quellec G, Lamard M, Martiano D, Cazuguel G, Coatrieux G, Cochener B (2017) Real-time analysis of cataract surgery videos using statistical models. Multimed Tools Appl 76(21):22473\u201322491","journal-title":"Multimed Tools Appl"},{"key":"2864_CR36","doi-asserted-by":"crossref","unstructured":"Liu B, Wang X, Dixit M, Kwitt R, Vasconcelos N (2018) Feature space transfer for data augmentation. In: CVPR","DOI":"10.1109\/CVPR.2018.00947"},{"key":"2864_CR37","doi-asserted-by":"crossref","unstructured":"Chu P, Bian X, Liu S, Ling H (2020) Feature space augmentation for long-tailed data. In: Computer Vision \u2013 ECCV 2020, pp. 694\u2013710","DOI":"10.1007\/978-3-030-58526-6_41"},{"key":"2864_CR38","doi-asserted-by":"publisher","first-page":"1059","DOI":"10.1007\/s11548-019-01958-6","volume":"14","author":"CI Nwoye","year":"2019","unstructured":"Nwoye CI, Mutter D, Marescaux J, Padoy N (2019) Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos. Int J Comput Assist Radiol Surg 14:1059\u20131067","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"2864_CR39","doi-asserted-by":"crossref","unstructured":"Nwoye CI, Gonzalez C, Yu T, Mascagni P, Mutter D, Marescaux J, Padoy N (2020) Recognition of instrument-tissue interactions in endoscopic videos via action triplets. In: International conference on medical image computing and computer-assisted intervention, pp. 364\u2013374","DOI":"10.1007\/978-3-030-59716-0_35"},{"key":"2864_CR40","unstructured":"Alapatt D, Mascagni P, Vardazaryan A, Garcia A, Okamoto N, Mutter D, Marescaux J, Costamagna G, Dallemagne B, Padoy N (2021) Temporally constrained neural networks (TCNN): A framework for semi-supervised video semantic segmentation. arXiv preprint arXiv:2112.13815"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-023-02864-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-023-02864-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-023-02864-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,8]],"date-time":"2023-09-08T11:13:20Z","timestamp":1694171600000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-023-02864-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,22]]},"references-count":40,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["2864"],"URL":"https:\/\/doi.org\/10.1007\/s11548-023-02864-8","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,22]]},"assertion":[{"value":"5 January 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 March 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The surgical videos were recorded and anonymized following the informed consent of patients in compliance with the local institutional review board (IRB) requirements.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}}]}}