{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T21:10:54Z","timestamp":1775077854450,"version":"3.50.1"},"reference-count":28,"publisher":"Springer Science and Business Media LLC","issue":"11","license":[{"start":{"date-parts":[[2021,8,20]],"date-time":"2021-08-20T00:00:00Z","timestamp":1629417600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,8,20]],"date-time":"2021-08-20T00:00:00Z","timestamp":1629417600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"published-print":{"date-parts":[[2021,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Purpose<\/jats:title>\n                <jats:p>Surgical workflow recognition is a crucial and challenging problem when building a computer-assisted surgery system. Current techniques focus on utilizing a convolutional neural network and a recurrent neural network (CNN\u2013RNN) to solve the surgical workflow recognition problem. In this paper, we attempt to use a deep 3DCNN to solve this problem.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>In order to tackle the surgical workflow recognition problem and the imbalanced data problem, we implement a 3DCNN workflow referred to as I3D-FL-PKF. We utilize focal loss (FL) to train a 3DCNN architecture known as Inflated 3D ConvNet (I3D) for surgical workflow recognition. We use prior knowledge filtering (PKF) to filter the recognition results.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>We evaluate our proposed workflow on a large sleeve gastrectomy surgical video dataset. We show that focal loss can help to address the imbalanced data problem. We show that our PKF can be used to generate smoothed prediction results and improve the overall accuracy. We show that the proposed workflow achieves 84.16% frame-level accuracy and reaches a weighted Jaccard score of 0.7327 which outperforms traditional CNN\u2013RNN design.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>The proposed workflow can obtain consistent and smooth predictions not only within the surgical phases but also for phase transitions. By utilizing focal loss and prior knowledge filtering, our implementation of deep 3DCNN has great potential to solve surgical workflow recognition problems for clinical practice.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s11548-021-02473-3","type":"journal-article","created":{"date-parts":[[2021,8,20]],"date-time":"2021-08-20T13:02:39Z","timestamp":1629464559000},"page":"2029-2036","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["Surgical workflow recognition with 3DCNN for Sleeve Gastrectomy"],"prefix":"10.1007","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1906-2116","authenticated-orcid":false,"given":"Bokai","family":"Zhang","sequence":"first","affiliation":[]},{"given":"Amer","family":"Ghanem","sequence":"additional","affiliation":[]},{"given":"Alexander","family":"Simes","sequence":"additional","affiliation":[]},{"given":"Henry","family":"Choi","sequence":"additional","affiliation":[]},{"given":"Andrew","family":"Yoo","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,8,20]]},"reference":[{"issue":"1","key":"2473_CR1","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/TMI.2016.2593957","volume":"36","author":"AP Twinanda","year":"2016","unstructured":"Twinanda AP, Shehata S, Mutter D, Marescaux J, De Mathelin M, Padoy N (2016) Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36(1):86\u201397","journal-title":"IEEE Trans Med Imaging"},{"key":"2473_CR2","doi-asserted-by":"crossref","unstructured":"Primus M, Putzgruber-Adamitsch D, Taschwer M, M\u00fcnzer B, El-Shabrawi Y, B\u00f6sz\u00f6rmenyi L, Schoeffmann K (2018) Frame-based classification of operation phases in cataract surgery videos. In: Proceedings of the MMM. LNCS, vol 10704. Bangkok, Thailand, pp 241\u2013253","DOI":"10.1007\/978-3-319-73603-7_20"},{"issue":"7","key":"2473_CR3","doi-asserted-by":"publisher","first-page":"8061","DOI":"10.1007\/s11042-017-4699-5","volume":"77","author":"P Stefan","year":"2018","unstructured":"Stefan P, Klaus S (2018) Endonet: a deep architecture for recognition tasks on laparoscopic videos. Multimed Tools Appl 77(7):8061\u20138079","journal-title":"Multimed Tools Appl"},{"key":"2473_CR4","first-page":"1","volume":"34","author":"D Kitaguchi","year":"2019","unstructured":"Kitaguchi D, Takeshita N, Matsuzaki H, Takano H, Owada Y, Enomoto T, Oda T, Miura H, Yamanashi T, Watanabe M, Sato D, Sugomori Y, Hara S, Ito M (2019) Real-time automatic surgical phase recognition in laparoscopic sigmoidectomy using the convolutional neural network-based deep learning approach. Surg Endosc 34:1\u20138","journal-title":"Surg Endosc"},{"key":"2473_CR5","doi-asserted-by":"crossref","unstructured":"Loukas C (2019) Surgical phase recognition of short video shots based on temporal modeling of deep features. In: 12th international joint conference on biomedical engineering systems and technologies: SCITEPRESS, vol 2, pp 21\u201329","DOI":"10.5220\/0007352000210029"},{"issue":"5","key":"2473_CR6","doi-asserted-by":"publisher","first-page":"1114","DOI":"10.1109\/TMI.2017.2787657","volume":"37","author":"Y Jin","year":"2017","unstructured":"Jin Y, Dou Q, Chen H, Yu L, Qin J, Fu C-W, Heng P-A (2017) SV-RCNet: workflow recognition from surgical videos using recurrent convolutional network. IEEE Trans Med Imaging 37(5):1114\u20131126","journal-title":"IEEE Trans Med Imaging"},{"key":"2473_CR7","doi-asserted-by":"crossref","unstructured":"Zisimopoulos O, Flouty E, Luengo I, Giataganas P, Nehme J, Chow A, Stoyanov D (2018) Deepphase: surgical phase recognition in cataracts videos. In: MICCAI international conference on medical image computing and computer-assisted-intervention, pp 265\u2013272","DOI":"10.1007\/978-3-030-00937-3_31"},{"issue":"1","key":"2473_CR8","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1186\/s13640-018-0316-4","volume":"2018","author":"Y Chen","year":"2018","unstructured":"Chen Y, Sun QL, Zhong K (2018) Semi-supervised spatio-temporal CNN for recognition of surgical workflow. J Image Video Proc 2018(1):76","journal-title":"J Image Video Proc"},{"key":"2473_CR9","unstructured":"Yengera G, Mutter D, Marescaux J, Padoy N (2018) Less is more: surgical phase recognition with less annotations through self-supervised pre-training of CNN-LSTM networks. arXiv preprint arXiv:1805.08569"},{"key":"2473_CR10","first-page":"85","volume-title":"OR2.0 context-aware operating theaters","author":"I Funke","year":"2018","unstructured":"Funke I, Jenke A, Mees ST, Weitz J, Speidel S, Bodenstedt S (2018) Temporal coherence-based self-supervised learning for laparoscopic workflow analysis. In: Stoyanov D et al (eds) OR2.0 context-aware operating theaters. Springer, Berlin, pp 85\u201393"},{"key":"2473_CR11","unstructured":"Mondal SS, Sathish R, Sheet D (2019) Multitask learning of temporal connectionism in convolutional networks using a joint distribution loss function to simultaneously identify tools and phase in surgical videos. arXiv preprint arXiv:1905.08315"},{"key":"2473_CR12","doi-asserted-by":"publisher","first-page":"101572","DOI":"10.1016\/j.media.2019.101572","volume":"59","author":"Y Jin","year":"2019","unstructured":"Jin Y, Li H, Dou Q, Chen H, Qin J, Fu C-W, Heng P-A (2019) Multi-task recurrent convolutional network with correlation loss for surgical video analysis. Med Image Anal 59:101572","journal-title":"Med Image Anal"},{"key":"2473_CR13","doi-asserted-by":"crossref","unstructured":"Nakawala H, Bianchi R, Pescatori LE, De Cobelli O, Ferrigno G, De Momi E (2019) Deep-onto network for surgical workflow and context recognition. Int J Comput Assist Radiol Surg 14:685\u2013696","DOI":"10.1007\/s11548-018-1882-8"},{"key":"2473_CR14","doi-asserted-by":"publisher","first-page":"414","DOI":"10.1097\/SLA.0000000000003460","volume":"2703","author":"A Hashimoto Daniel","year":"2019","unstructured":"Hashimoto Daniel A, Guy R, Witkowski Elan R, Caitlin S, Navarrete-Welton Allison J, Rattner David W, Lillemoe Keith D, Rus Daniela L, Meireles Ozanan R (2019) Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy. Ann Surg 2703:414","journal-title":"Ann Surg"},{"key":"2473_CR15","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"2473_CR16","doi-asserted-by":"crossref","unstructured":"Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. pp 2818\u20132826","DOI":"10.1109\/CVPR.2016.308"},{"key":"2473_CR17","first-page":"97","volume-title":"Computer assisted robotic endoscopy","author":"W Chen","year":"2018","unstructured":"Chen W, Feng J, Lu J, Zhou J (2018) Endo3d: online workflow analysis for endoscopic surgeries based on 3d CNN and LSTM. In: Stoyanov D et al (eds) Computer assisted robotic endoscopy. Springer, Cham, pp 97\u2013107"},{"key":"2473_CR18","doi-asserted-by":"crossref","unstructured":"Carreira J, Zisserman A (2017) Quo vadis, action recognition? A new model and the kinetics dataset. pp 6299\u20136308","DOI":"10.1109\/CVPR.2017.502"},{"key":"2473_CR19","doi-asserted-by":"crossref","unstructured":"Lin T-Y, Goyal P, Girshick R, He K, Dollar P (2017) Focal Loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980\u20132988","DOI":"10.1109\/ICCV.2017.324"},{"issue":"supplement1","key":"2473_CR20","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1159\/000198239","volume":"2","author":"M Daskalakis","year":"2009","unstructured":"Daskalakis M, Weiner RA (2009) Sleeve gastrectomy as a single-stage bariatric operation: indications and limitations. Obes Facts 2(supplement1):8\u201310","journal-title":"Obes Facts"},{"key":"2473_CR21","doi-asserted-by":"publisher","first-page":"821","DOI":"10.3748\/wjg.14.821","volume":"14","author":"A Iannelli","year":"2008","unstructured":"Iannelli A, Dainese R, Piche T, Facchiano E, Gugenheim J (2008) Laparoscopic sleeve gastrectomy for morbid obesity. World J Gastroenterol 14:821\u2013827","journal-title":"World J Gastroenterol"},{"key":"2473_CR22","doi-asserted-by":"crossref","unstructured":"van Rutte P, Nienhuijs SW, Jakimowicz JJ, van Montfort G (2017) Identification of technical errors and hazard zones in sleeve gas-trectomy using OCHRA: \u201cOCHRA for sleeve gastrectomy\u201d. Surg Endosc 31(2):561\u2013566","DOI":"10.1007\/s00464-016-4997-4"},{"key":"2473_CR23","doi-asserted-by":"publisher","first-page":"2974","DOI":"10.1007\/s11695-017-2729-z","volume":"27","author":"GH van Ramshorst","year":"2017","unstructured":"van Ramshorst GH, Kaijser MA, Pierie JPEN, van Wagensveld BA (2017) Resident training in bariatric surgery-a national survey in the Netherlands. Obes Surg 27:2974\u20132980","journal-title":"Obes Surg"},{"key":"2473_CR24","doi-asserted-by":"publisher","first-page":"2634","DOI":"10.1007\/s11695-018-3219-7","volume":"28","author":"MA Kaijser","year":"2018","unstructured":"Kaijser MA, van Ramshorst GH, Emous M, Veeger NJGM, van Wagensveld BA, Pierie JEN (2018) A Delphi consensus of the crucial steps in gastric bypass and sleeve gastrectomy procedures in the Netherlands. Obes Surg 28:2634\u20132643","journal-title":"Obes Surg"},{"issue":"2","key":"2473_CR25","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1007\/s00464-016-4997-4","volume":"31","author":"PWJ Van Rutte","year":"2017","unstructured":"Van Rutte PWJ, Nienhuijs SW, Jakimowicz JJ, van Montfort G (2017) Identification of technical errors and hazard zones in sleeve gastrectomy using OCHRA. Surg Endosc 31(2):561\u2013566","journal-title":"Surg Endosc"},{"key":"2473_CR26","unstructured":"Kay W, Carreira J, Simonyan K, Zhang B, Hillier C, Vijayanarasimhan S, Viola F, Green T, Back T, Natsev P, Suleyman M, Zisserman A (2017) The kinetics human action video dataset. arXiv preprint arXiv:1705.06950"},{"key":"2473_CR27","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1613\/jair.953","volume":"16","author":"V Chawla Nitesh","year":"2002","unstructured":"Chawla Nitesh V, Bowyer Kevin W, Hall Lawrence O, Philip KW (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321\u2013357","journal-title":"J Artif Intell Res"},{"key":"2473_CR28","unstructured":"Simon K, Jonathon S, Quoc VL (2019) Do better imagenet models transfer better? In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 2661\u20132671"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-021-02473-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-021-02473-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-021-02473-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,11,12]],"date-time":"2021-11-12T13:11:14Z","timestamp":1636722674000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-021-02473-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,20]]},"references-count":28,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2021,11]]}},"alternative-id":["2473"],"URL":"https:\/\/doi.org\/10.1007\/s11548-021-02473-3","relation":{},"ISSN":["1861-6410","1861-6429"],"issn-type":[{"value":"1861-6410","type":"print"},{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,20]]},"assertion":[{"value":"10 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 August 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 August 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Bokai Zhang, Amer Ghanem, Alexander Simes, Henry Choi, and Andrew Yoo declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"For this type of study, formal consent is not required.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"This article contains statistical data from an internal dataset which is approved to use for research purposes.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}}]}}