{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,28]],"date-time":"2026-10-28T17:06:13Z","timestamp":1793207173534,"version":"4.5.0"},"reference-count":35,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"1","license":[{"start":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T00:00:00Z","timestamp":1756857600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,1]]},"abstract":"<jats:sec>\n                    <jats:title>Background:<\/jats:title>\n                    <jats:p>A comprehensive understanding of surgical scenes by computers is a crucial foundation for achieving intelligent surgical assistance and autonomous decision-making. Surgical scene information encompasses coarse-grained data reflecting the overall process and fine-grained details showcasing specific operations. This study aims to construct a standardized, full-grained annotation dataset for laparoscopic radical nephrectomy and develop a deep learning framework for multi-hierarchical granularity integration, providing support for clinical intelligent applications.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and methods:<\/jats:title>\n                    <jats:p>We collected 41 multi-center videos and performed comprehensive annotations, including 8 surgical phases, 17 surgical steps, 6 types of instrument segmentation, and 35 surgical action triplets (SATs). Using this dataset, we designed a lightweight framework that enables simultaneous perception of multi-granularity information through shared feature extraction and task-specific decoders. Based on the comprehensive understanding information, we developed a robust intraoperative bleeding prediction model using 123 combinations of machine learning algorithms and generated an automated postoperative surgical summary report.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>The dataset of this study comprises 141 443 frames with annotations for surgical phases and steps, 8435 frames with instrument segmentation annotations, and 25 305 frames with SATs. Our deep learning framework achieves outstanding performance across tasks of different granularities, with recognition accuracy even surpassing that of six state-of-the-art algorithms developed separately for individual subtasks. Based on full-granularity features, we found that surgical steps involving key anatomical structures such as the renal artery, renal vein, and ureter are critical predictors of bleeding risk. And the post-operative automated reporting system accurately extracts key surgical time points and specific procedural durations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion:<\/jats:title>\n                    <jats:p>We constructed the first fully annotated dataset for laparoscopic radical nephrectomy, and developed a novel framework that achieves bidirectional compensation and enhancement of both coarse and fine-granularity information. This study not only expands the theoretical boundaries of artificial intelligence in surgical procedures, but also lays a solid foundation for advancing intelligent surgery through the \u201cdata\u2013algorithm\u2013application\u201d cycle.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1097\/js9.0000000000003434","type":"journal-article","created":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T13:04:48Z","timestamp":1756904688000},"page":"460-471","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":1,"title":["Deep learning-driven multi-hierarchical granularity integration for surgical scene understanding: experimental study"],"prefix":"10.1097","volume":"112","author":[{"given":"Guangdi","family":"Chu","sequence":"first","affiliation":[{"name":"Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuan","family":"Gao","sequence":"additional","affiliation":[{"name":"Institute of Medical Robotics and Intelligent Systems, Tianjin University, Tianjin, China"},{"name":"Key Laboratory of Mechanism Theory and Equipment Design, Ministry of Education, Tianjin, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Jiao","sequence":"additional","affiliation":[{"name":"Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China"},{"name":"Telemedicine and Equipment Technology Innovation Center, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guipeng","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fengyuan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chengjun","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pengfei","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Urology, The First Affiliated Hospital of Anhui Medical University, Hefei, Anhui, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xu","family":"Han","sequence":"additional","affiliation":[{"name":"Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhiyuan","family":"Mi","sequence":"additional","affiliation":[{"name":"Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jingkai","family":"Wang","sequence":"additional","affiliation":[{"name":"Qingdao Medical College, Qingdao University, Qingdao, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaohui","family":"Ren","sequence":"additional","affiliation":[{"name":"Qingdao Medical College, Qingdao University, Qingdao, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianning","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Urology, The First Affiliated Hospital of Shandong First Medical University & Shandong Provincial Qianfoshan Hospital, Shandong Medicine and Health Key Laboratory of Organ Transplantation and Nephrosis, Shandong institute of Nephrology, Jinan, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuxin","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Mechanism Theory and Equipment Design, Ministry of Education, Tianjin, China"},{"name":"Chongqing University, Chongqing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qian","family":"Dong","sequence":"additional","affiliation":[{"name":"Department of Pediatric Surgery, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haitao","family":"Niu","sequence":"additional","affiliation":[{"name":"Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China"},{"name":"Institute of Medical Robotics and Intelligent Systems, Tianjin University, Tianjin, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianmin","family":"Li","sequence":"additional","affiliation":[{"name":"Institute of Medical Robotics and Intelligent Systems, Tianjin University, Tianjin, China"},{"name":"Key Laboratory of Mechanism Theory and Equipment Design, Ministry of Education, Tianjin, China"},{"name":"Telemedicine and Equipment Technology Innovation Center, The Affiliated Hospital of Qingdao University, Qingdao, Shandong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","published-online":{"date-parts":[[2025,9,3]]},"reference":[{"key":"R1-20260518","first-page":"61\u201375","article-title":"The IDEAL framework for surgical robotics: development, comparative evaluation and long-term monitoring","volume":"30","author":"Marcus","year":"2024","journal-title":"Nat Med"},{"key":"R2-20260518","doi-asserted-by":"crossref","first-page":"eadr5247","DOI":"10.1126\/scirobotics.adr5247","article-title":"Augmented dexterity: how robots can enhance human surgical skills","volume":"9","author":"Goldberg","year":"2024","journal-title":"Sci Rob"},{"key":"R3-20260518","doi-asserted-by":"crossref","first-page":"2271\u201372","DOI":"10.1093\/eurheartj\/ehae095","article-title":"Future cardiovascular healthcare via magnetic resonance imaging-driven robotics","volume":"45","author":"Lloyd","year":"2024","journal-title":"Eur Heart J"},{"key":"R4-20260518","doi-asserted-by":"crossref","first-page":"320\u201325","DOI":"10.1016\/j.eururo.2023.07.015","article-title":"Metaverse surgical planning with three-dimensional virtual models for minimally invasive partial nephrectomy","volume":"85","author":"Checcucci","year":"2024","journal-title":"Eur Urol"},{"key":"R5-20260518","doi-asserted-by":"crossref","first-page":"e2400980","DOI":"10.1002\/advs.202400980","article-title":"Autonomous magnetic navigation in endoscopic image mosaics","volume":"11","author":"Mattille","year":"2024","journal-title":"Adv Sci (Weinh)"},{"key":"R6-20260518","doi-asserted-by":"crossref","first-page":"86\u201391","DOI":"10.1016\/j.eururo.2023.02.024","article-title":"Improving augmented reality through deep learning: real-time instrument delineation in robotic renal surgery","volume":"84","author":"De Backer","year":"2023","journal-title":"Eur Urol"},{"key":"R7-20260518","doi-asserted-by":"crossref","first-page":"107472","DOI":"10.1016\/j.compbiomed.2023.107472","article-title":"Clean visual field reconstruction in robot-assisted laparoscopic surgery based on dynamic prediction","volume":"165","author":"Guan","year":"2023","journal-title":"Comput Biol Med"},{"key":"R8-20260518","first-page":"100082","article-title":"Transparency in the reporting of artificial intelligence\u2013the TITAN guideline","volume":"10","author":"Agha","year":"2025","journal-title":"Premier J Sci"},{"key":"R9-20260518","doi-asserted-by":"crossref","first-page":"171\u201382","DOI":"10.1038\/s41575-022-00701-y","article-title":"Artificial intelligence and automation in endoscopy and surgery","volume":"20","author":"Chadebecq","year":"2023","journal-title":"Nat Rev Gastroenterol Hepatol"},{"key":"R10-20260518","doi-asserted-by":"crossref","first-page":"8459","DOI":"10.1109\/LRA.2024.3445654","article-title":"Graph-based spatial reasoning for tracking landmarks in dynamic laparoscopic environments","volume":"9","author":"Zhang","journal-title":"IEEE Rob Autom Lett"},{"key":"R11-20260518","doi-asserted-by":"crossref","first-page":"3294\u2013306","DOI":"10.1097\/JS9.0000000000001371","article-title":"A real-time augmented reality system integrated with artificial intelligence for skin tumor surgery: experimental study and case series","volume":"110","author":"Huang","year":"2024","journal-title":"Int J Surg"},{"key":"R12-20260518","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1038\/s41746-024-01273-8","article-title":"Artificial intelligence assisted operative anatomy recognition in endoscopic pituitary surgery","volume":"7","author":"Khan","year":"2024","journal-title":"NPJ Digit Med"},{"key":"R13-20260518","doi-asserted-by":"crossref","first-page":"2941\u201352","DOI":"10.1097\/JS9.0000000000000559","article-title":"Multilevel effective surgical workflow recognition in robotic left lateral sectionectomy with deep learning: experimental research","volume":"109","author":"Liu","year":"2023","journal-title":"Int J Surg"},{"key":"R14-20260518","doi-asserted-by":"crossref","first-page":"103366","DOI":"10.1016\/j.media.2024.103366","article-title":"LoViT: long video transformer for surgical phase recognition","volume":"99","author":"Liu","year":"2025","journal-title":"Med Image Anal"},{"key":"R15-20260518","doi-asserted-by":"crossref","first-page":"103026","DOI":"10.1016\/j.media.2023.103026","article-title":"ST-ITEF: Spatio-Temporal Intraoperative Task Estimating Framework to recognize surgical phase and predict instrument path based on multi-object tracking in keratoplasty","volume":"91","author":"Feng","year":"2024","journal-title":"Med Image Anal"},{"key":"R16-20260518","doi-asserted-by":"crossref","first-page":"102888","DOI":"10.1016\/j.media.2023.102888","article-title":"CholecTriplet2022: show me a tool and tell me the triplet \u2013 an endoscopic vision challenge for surgical action triplet detection","volume":"89","author":"Nwoye","year":"2023","journal-title":"Med Image Anal"},{"key":"R17-20260518","doi-asserted-by":"crossref","first-page":"AIoa2300088","DOI":"10.1056\/AIoa2300088","article-title":"Artificial intelligence identifies factors associated with blood loss and surgical experience in cholecystectomy","volume":"1","author":"Aklilu","year":"2024","journal-title":"NEJM AI"},{"key":"R18-20260518","doi-asserted-by":"crossref","first-page":"102803","DOI":"10.1016\/j.media.2023.102803","article-title":"CholecTriplet2021: a benchmark challenge for surgical action triplet recognition","volume":"86","author":"Nwoye","year":"2023","journal-title":"Med Image Anal"},{"key":"R19-20260518","first-page":"100081","article-title":"Revised strengthening the reporting of cohort, cross-sectional and case-control studies in surgery (STROCSS) guideline: an update for the age of Artificial Intelligence","volume":"10","author":"Agha","year":"2025","journal-title":"Premier J Sci"},{"key":"R21-20260518","doi-asserted-by":"crossref","first-page":"41\u201344","DOI":"10.1016\/j.eururo.2022.06.018","article-title":"Application of improved robot-assisted laparoscopic telesurgery with 5G technology in urology","volume":"83","author":"Li","year":"2023","journal-title":"Eur Urol"},{"key":"R22-20260518","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1186\/s12957-019-1580-1","article-title":"The methods and techniques of identifying renal pedicle vessels during retroperitoneal laparoscopic radical and partial nephrectomy","volume":"17","author":"Yang","year":"2019","journal-title":"World J Surg Oncol"},{"key":"R23-20260518","doi-asserted-by":"crossref","first-page":"780\u201396","DOI":"10.1038\/s41551-023-01010-8","article-title":"A vision transformer for decoding surgeon activity from surgical videos","volume":"7","author":"Kiyasseh","year":"2023","journal-title":"Nat Biomed Eng"},{"key":"R24-20260518","doi-asserted-by":"crossref","first-page":"844\u201353","DOI":"10.1016\/j.juro.2009.05.035","article-title":"The RENAL nephrometry score: a comprehensive standardized system for quantitating renal tumor size, location and depth","volume":"182","author":"Kutikov","year":"2009","journal-title":"J Urol"},{"key":"R25-20260518","doi-asserted-by":"crossref","first-page":"786\u201393","DOI":"10.1016\/j.eururo.2009.07.040","article-title":"Preoperative aspects and dimensions used for an anatomical (PADUA) classification of renal tumours in patients who are candidates for nephron-sparing surgery","volume":"56","author":"Ficarra","year":"2009","journal-title":"Eur Urol"},{"key":"R26-20260518","first-page":"11\u201314","article-title":"The learning curve for radical nephrectomy for kidney cancer: implications for surgical training","volume":"49","author":"Larcher","year":"2023","journal-title":"Eur Urol Open Sci"},{"key":"R27-20260518","first-page":"2037\u201357","article-title":"The learning curves of major laparoscopic and robotic procedures in urology: a systematic review","volume":"109","author":"Chahal","year":"2023","journal-title":"Int J Surg"},{"key":"R28-20260518","doi-asserted-by":"crossref","first-page":"482\u201390","DOI":"10.1016\/j.eururo.2024.07.022","article-title":"Feasibility of image-guided navigation with electromagnetic tracking during robot-assisted sentinel node biopsy: a prospective study","volume":"87","author":"Aguilera Saiz","year":"2025","journal-title":"Eur Urol"},{"key":"R29-20260518","doi-asserted-by":"crossref","first-page":"588\u201397","DOI":"10.1016\/j.eururo.2021.10.023","article-title":"Percutaneous kidney puncture with three-dimensional mixed-reality hologram guidance: from preoperative planning to intraoperative navigation","volume":"81","author":"Porpiglia","year":"2022","journal-title":"Eur Urol"},{"key":"R30-20260518","doi-asserted-by":"crossref","first-page":"S1496","DOI":"10.1016\/S0302-2838(22)01096-X","article-title":"Intraoperative mobile-assisted navigation for nephroscopy","volume":"81","author":"Talyshinskii","year":"2022","journal-title":"Eur Urol"},{"key":"R32-20260518","first-page":"264\u201374","article-title":"Concept graph neural networks for surgical video understanding","volume":"43","author":"Ban","year":"2023","journal-title":"IEEE Trans Med Imaging"},{"key":"R33-20260518","first-page":"1628\u201339","article-title":"MT4MTL-KD: a multi-teacher knowledge distillation framework for triplet recognition","volume":"43","author":"Gui","year":"2023","journal-title":"IEEE Trans Med Imaging"},{"key":"R34-20260518","doi-asserted-by":"crossref","first-page":"8818\u201328","DOI":"10.1007\/s00464-023-10375-5","article-title":"A novel high accuracy model for automatic surgical workflow recognition using artificial intelligence in laparoscopic totally extraperitoneal inguinal hernia repair (TEP)","volume":"37","author":"Ortenzi","year":"2023","journal-title":"Surg Endosc"},{"key":"R35-20260518","doi-asserted-by":"crossref","first-page":"1291\u201399","DOI":"10.1007\/s11548-024-03147-6","article-title":"Parameter-efficient framework for surgical action triplet recognition","volume":"19","author":"Li","year":"2024","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"R36-20260518","doi-asserted-by":"crossref","first-page":"4983\u201394","DOI":"10.1109\/JBHI.2023.3299321","article-title":"MT-FiST: a Multi-Task Fine-Grained Spatial-Temporal framework for surgical action triplet recognition","volume":"27","author":"Li","year":"2023","journal-title":"IEEE J Biomed Health Inform"},{"key":"R37-20260518","doi-asserted-by":"crossref","first-page":"6676","DOI":"10.1038\/s41467-023-42451-8","article-title":"Intelligent surgical workflow recognition for endoscopic submucosal dissection with real-time animal study","volume":"14","author":"Cao","year":"2023","journal-title":"Nat Commun"}],"container-title":["International Journal of Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.lww.com\/10.1097\/JS9.0000000000003434","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T13:19:36Z","timestamp":1779110376000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.lww.com\/10.1097\/JS9.0000000000003434"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,3]]},"references-count":35,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026]]}},"URL":"https:\/\/doi.org\/10.1097\/js9.0000000000003434","relation":{},"ISSN":["1743-9159"],"issn-type":[{"value":"1743-9159","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,3]]},"assertion":[{"value":"2025-04-14","name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-27","name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}}]}}