{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T11:03:00Z","timestamp":1780398180303,"version":"3.54.1"},"reference-count":30,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T00:00:00Z","timestamp":1709769600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec><jats:title>Objective<\/jats:title><jats:p>Automated surgical step recognition (SSR) using AI has been a catalyst in the \u201cdigitization\u201d of surgery. However, progress has been limited to laparoscopy, with relatively few SSR tools in endoscopic surgery. This study aimed to create a SSR model for transurethral resection of bladder tumors (TURBT), leveraging a novel application of transfer learning to reduce video dataset requirements.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and methods<\/jats:title><jats:p>Retrospective surgical videos of TURBT were manually annotated with the following steps of surgery: primary endoscopic evaluation, resection of bladder tumor, and surface coagulation. Manually annotated videos were then utilized to train a novel AI computer vision algorithm to perform automated video annotation of TURBT surgical video, utilizing a transfer-learning technique to pre-train on laparoscopic procedures. Accuracy of AI SSR was determined by comparison to human annotations as the reference standard.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>A total of 300 full-length TURBT videos (median 23.96\u2009min; IQR 14.13\u201341.31\u2009min) were manually annotated with sequential steps of surgery. One hundred and seventy-nine videos served as a training dataset for algorithm development, 44 for internal validation, and 77 as a separate test cohort for evaluating algorithm accuracy. Overall accuracy of AI video analysis was 89.6%. Model accuracy was highest for the primary endoscopic evaluation step (98.2%) and lowest for the surface coagulation step (82.7%).<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>We developed a fully automated computer vision algorithm for high-accuracy annotation of TURBT surgical videos. This represents the first application of transfer-learning from laparoscopy-based computer vision models into surgical endoscopy, demonstrating the promise of this approach in adapting to new procedure types.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frai.2024.1375482","type":"journal-article","created":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T04:35:22Z","timestamp":1709786122000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Automated surgical step recognition in transurethral bladder tumor resection using artificial intelligence: transfer learning across surgical modalities"],"prefix":"10.3389","volume":"7","author":[{"given":"Ekamjit S.","family":"Deol","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew K.","family":"Tollefson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alenka","family":"Antolin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maya","family":"Zohar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Omri","family":"Bar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Danielle","family":"Ben-Ayoun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lance A.","family":"Mynderse","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Derek J.","family":"Lomas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ross A.","family":"Avant","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adam R.","family":"Miller","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel S.","family":"Elliott","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stephen A.","family":"Boorjian","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tamir","family":"Wolf","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dotan","family":"Asselmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abhinav","family":"Khanna","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,3,7]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"1521","DOI":"10.1007\/s00464-020-08168-1","article-title":"Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis","volume":"35","author":"Anteby","year":"2021","journal-title":"Surg. Endosc."},{"key":"ref2","doi-asserted-by":"publisher","first-page":"22208","DOI":"10.1038\/s41598-020-79173-6","article-title":"Impact of data on generalization of AI for surgical intelligence applications","volume":"10","author":"Bar","year":"2020","journal-title":"Sci. Rep."},{"key":"ref3","doi-asserted-by":"publisher","first-page":"1434","DOI":"10.1056\/NEJMsa1300625","article-title":"Surgical skill and complication rates after bariatric surgery","volume":"369","author":"Birkmeyer","year":"2013","journal-title":"N. Engl. J. Med."},{"key":"ref4","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1159\/000511351","article-title":"Artificial intelligence-assisted surgery: potential and challenges","volume":"36","author":"Bodenstedt","year":"2020","journal-title":"Visceral Med."},{"key":"ref5","doi-asserted-by":"publisher","first-page":"3160","DOI":"10.1007\/s00464-021-08619-3","article-title":"Artificial intelligence-based automated laparoscopic cholecystectomy surgical phase recognition and analysis","volume":"36","author":"Cheng","year":"2022","journal-title":"Surg. Endosc."},{"key":"ref6","doi-asserted-by":"publisher","first-page":"4040","DOI":"10.1007\/s00464-023-09971-2","article-title":"TEsoNet: knowledge transfer in surgical phase recognition from laparoscopic sleeve gastrectomy to the laparoscopic part of Ivor\u2013Lewis esophagectomy","volume":"37","author":"Eckhoff","year":"2023","journal-title":"Surg. Endosc."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"570","DOI":"10.1177\/000313480407000702","article-title":"The operative note as billing documentation: a preliminary report","volume":"70","author":"Flynn","year":"2004","journal-title":"Am. Surg."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"684","DOI":"10.1097\/SLA.0000000000004425","article-title":"Machine learning for surgical phase recognition: a systematic review","volume":"273","author":"Garrow","year":"2021","journal-title":"Ann. Surg."},{"key":"ref9","volume-title":"Deep learning","author":"Goodfellow","year":"2016"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"6150","DOI":"10.1007\/s00464-020-08110-5","article-title":"Deep learning for surgical phase recognition using endoscopic videos","volume":"35","author":"Gu\u00e9don","year":"2021","journal-title":"Surg. Endosc."},{"key":"ref11","doi-asserted-by":"publisher","first-page":"414","DOI":"10.1097\/SLA.0000000000003460","article-title":"Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy","volume":"270","author":"Hashimoto","year":"2019","journal-title":"Ann. Surg."},{"key":"ref12","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1111\/bju.14735","article-title":"A deep-learning model using automated performance metrics and clinical features to predict urinary continence recovery after robot-assisted radical prostatectomy","volume":"124","author":"Hung","year":"2019","journal-title":"BJU Int."},{"key":"ref13","doi-asserted-by":"publisher","first-page":"296","DOI":"10.1016\/j.juro.2017.07.081","article-title":"Development and validation of objective performance metrics for robot-assisted radical prostatectomy: a pilot study","volume":"199","author":"Hung","year":"2018","journal-title":"J. Urol."},{"key":"ref14","doi-asserted-by":"publisher","first-page":"1114","DOI":"10.1109\/TMI.2017.2787657","article-title":"SV-RCNet: workflow recognition from surgical videos using recurrent convolutional network","volume":"37","author":"Jin","year":"2018","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1016\/j.wneu.2022.01.020","article-title":"The uncharted waters of machine and deep learning for surgical phase recognition in neurosurgery","volume":"160","author":"Jumah","year":"2022","journal-title":"World Neurosurg."},{"key":"ref16","doi-asserted-by":"publisher","first-page":"101097ju0000000000003845","DOI":"10.1097\/JU.0000000000003845","article-title":"Automated identification of key steps in robotic-assisted radical prostatectomy using artificial intelligence","author":"Khanna","year":"2024","journal-title":"J. Urol."},{"key":"ref17","doi-asserted-by":"publisher","first-page":"e744","DOI":"10.1097\/JU.0000000000003305.07","article-title":"PD27-07 automated operative reports for robotic radical prostatectomy using an artificial intelligence platform","volume":"209","author":"Khanna","year":"2023","journal-title":"J. Urol."},{"key":"ref18","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1038\/s43856-023-00263-3","article-title":"A multi-institutional study using artificial intelligence to provide reliable and fair feedback to surgeons","volume":"3","author":"Kiyasseh","year":"","journal-title":"Commun. Med."},{"key":"ref19","doi-asserted-by":"publisher","first-page":"780","DOI":"10.1038\/s41551-023-01010-8","article-title":"A vision transformer for decoding surgeon activity from surgical videos","volume":"7","author":"Kiyasseh","year":"","journal-title":"Nat. Biomed. Eng."},{"key":"ref20","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1038\/s41746-022-00738-y","article-title":"Surgical gestures as a method to quantify surgical performance and predict patient outcomes","volume":"5","author":"Ma","year":"2022","journal-title":"npj Digit. Med."},{"key":"ref21","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1038\/s41746-022-00707-5","article-title":"Computer vision in surgery: from potential to clinical value","volume":"5","author":"Mascagni","year":"2022","journal-title":"npj Digit. Med."},{"key":"ref22","doi-asserted-by":"crossref","DOI":"10.1109\/ICCVW54120.2021.00355","article-title":"Video transformer network","author":"Neimark","year":""},{"key":"ref23","article-title":"\u201cTrain one, classify one, teach one\u201d-cross-surgery transfer learning for surgical step recognition","author":"Neimark","year":""},{"key":"ref24","doi-asserted-by":"publisher","first-page":"15","DOI":"10.3390\/medsci8010015","article-title":"Epidemiology of bladder cancer","volume":"8","author":"Saginala","year":"2020","journal-title":"Med. Sci."},{"key":"ref25","volume-title":"Hinman's atlas of urologic surgery E-book","author":"Smith","year":"2016"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"6847","DOI":"10.1245\/s10434-022-11996-1","article-title":"Automated surgical-phase recognition for robot-assisted minimally invasive Esophagectomy using artificial intelligence","volume":"29","author":"Takeuchi","year":"2022","journal-title":"Ann. Surg. Oncol."},{"key":"ref27","doi-asserted-by":"publisher","first-page":"4517","DOI":"10.1007\/s00464-023-09924-9","article-title":"Evaluation of surgical complexity by automated surgical process recognition in robotic distal gastrectomy using artificial intelligence","volume":"37","author":"Takeuchi","year":"2023","journal-title":"Surg. Endosc."},{"key":"ref28","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/TMI.2016.2593957","article-title":"Endonet: a deep architecture for recognition tasks on laparoscopic videos","volume":"36","author":"Twinanda","year":"2017","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"1610","DOI":"10.1111\/j.1464-410X.2010.09387.x","article-title":"Surgery illustrated\u2013surgical atlas: transurethral resection of bladder tumours","volume":"105","author":"Wiesner","year":"2010","journal-title":"BJU Int."},{"key":"ref30","article-title":"Accurate detection of out of body segments in surgical video using semi-supervised learning","author":"Zohar","year":"2020"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1375482\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T04:35:26Z","timestamp":1709786126000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1375482\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,7]]},"references-count":30,"alternative-id":["10.3389\/frai.2024.1375482"],"URL":"https:\/\/doi.org\/10.3389\/frai.2024.1375482","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,7]]},"article-number":"1375482"}}