{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T09:05:18Z","timestamp":1773219918567,"version":"3.50.1"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T00:00:00Z","timestamp":1765324800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T00:00:00Z","timestamp":1765324800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100020192","name":"Medtronic Europe","doi-asserted-by":"publisher","award":["Digital Technologies"],"award-info":[{"award-number":["Digital Technologies"]}],"id":[{"id":"10.13039\/100020192","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100020192","name":"Medtronic Europe","doi-asserted-by":"publisher","award":["a Medtronic company"],"award-info":[{"award-number":["a Medtronic company"]}],"id":[{"id":"10.13039\/100020192","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Surg Endosc"],"published-print":{"date-parts":[[2026,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Proficiency-based progression is key to analyzing and improving surgical performance. Objective assessment has demonstrated a direct link between operative performance and outcomes in laparoscopic surgery but not in robotics. There is current research to automate assessment processes with sensor data and machine learning. This requires granular, reliable annotations to train clinically implementable, trusted models, to improve patient safety.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Aim<\/jats:title>\n                    <jats:p>To evaluate objective skill and error tools in robotic rectal cancer surgery, to provide a granular validated dataset from which to train and test deep learning models.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methodology<\/jats:title>\n                    <jats:p>A national, ethically approved, multicentre study, Video Analysis in Minimally Invasive Surgery (VAMIS) (ClinicalTrials.gov NCT05279287), recorded robotic-assisted total mesorectal excision (RTME). Recruited participants were pseudonymised and clinical data were collected. Operations were recorded and uploaded to Touch Surgery\u2122 using the DS1 computer (Digital Technologies, a Medtronic company) and annotated by independent, blinded raters. Objective assessment employed error, Objective Clinical Human Reliability Analysis (OCHRA), Modifiable-Global Evaluative Assessment in Robotic Skills (M-GEARS) and TME performance tools. Correlational and multivariable regression analyses were performed, investigating associations between intraoperative skill and errors with clinical outcomes.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      30 RTME operations were recorded, annotating 538 errors (median 13\/operation). Major consequential errors were significantly associated with complications (\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.031). Weighted error variables, accounting for error severity, were significantly associated with increased odds of prolonged operative time (\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.025). Inter-rater reliability demonstrated an excellent matched error agreement percentage of two raters (mean agreement 90% (range 68\u2013100%), after calibration sessions). OCHRA was significantly correlated with M-GEARS (\n                      <jats:italic>r<\/jats:italic>\n                      \u2009=\u2009\u2212\u00a00.54 to \u2212\u00a00.77,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009&lt;\u20090.001\u20130.002) and the RTME performance tool (\n                      <jats:italic>r<\/jats:italic>\n                      \u2009=\u20090.74,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.007).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>This feasibility study validated the concept that granular error and skill annotations can be objectively measured and associated with clinical outcomes in robotic rectal cancer surgery. This is an important step for larger studies and in aiding the development of deep learning models to predict errors and skill.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s00464-025-12393-x","type":"journal-article","created":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T15:33:12Z","timestamp":1765380792000},"page":"2085-2094","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Association of skill and errors with outcomes in robotic rectal cancer surgery"],"prefix":"10.1007","volume":"40","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7288-3354","authenticated-orcid":false,"given":"M.","family":"Boal","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"C.","family":"Reali","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"R.","family":"Duhoky","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"T.","family":"Gill","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"J.","family":"Khan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"D.","family":"Miskovic","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8498-9175","authenticated-orcid":false,"given":"N.","family":"Francis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,12,10]]},"reference":[{"key":"12393_CR1","unstructured":"World Health Organisation (2023) Colorectal cancer. https:\/\/www.who.int\/news-room\/fact-sheets\/detail\/colorectal-cancer"},{"key":"12393_CR2","doi-asserted-by":"publisher","first-page":"503","DOI":"10.1177\/014107688808100904","volume":"81","author":"RJ Heald Mchir","year":"1988","unstructured":"Heald Mchir RJ (1988) The \u2018Holy Plane\u2019 of rectal surgery. J Royal Soc Med 81:503","journal-title":"J Royal Soc Med"},{"key":"12393_CR3","doi-asserted-by":"publisher","DOI":"10.1093\/bjsopen\/zrae071","author":"\u00c5 Collin","year":"2024","unstructured":"Collin \u00c5, Dahlb\u00e4ck C, Folkesson J, Buchwald P (2024) Total mesorectal excision quality in rectal cancer surgery affects local recurrence rate but not distant recurrence and survival: population-based cohort study. BJS Open. https:\/\/doi.org\/10.1093\/bjsopen\/zrae071","journal-title":"BJS Open"},{"key":"12393_CR4","doi-asserted-by":"publisher","first-page":"821","DOI":"10.1016\/S0140-6736(09)60485-2","volume":"373","author":"P Quirke","year":"2009","unstructured":"Quirke P et al (2009) Effect of the plane of surgery achieved on local recurrence in patients with operable rectal cancer: a prospective study using data from the MRC CR07 and NCIC-CTG CO16 randomised clinical trial. Lancet 373:821\u2013828","journal-title":"Lancet"},{"key":"12393_CR5","doi-asserted-by":"publisher","first-page":"625","DOI":"10.1097\/SLA.0000000000000928","volume":"260","author":"JRT Monson","year":"2014","unstructured":"Monson JRT et al (2014) Failure of evidence-based cancer care in the United States. Ann Surg 260:625\u2013632","journal-title":"Ann Surg"},{"key":"12393_CR6","doi-asserted-by":"publisher","first-page":"590","DOI":"10.1001\/jamasurg.2020.1004","volume":"155","author":"NJ Curtis","year":"2020","unstructured":"Curtis NJ et al (2020) Association of surgical skill assessment with clinical outcomes in cancer surgery. JAMA Surg 155:590\u2013598","journal-title":"JAMA Surg"},{"key":"12393_CR7","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-022-00738-y","author":"R Ma","year":"2022","unstructured":"Ma R et al (2022) Surgical gestures as a method to quantify surgical performance and predict patient outcomes. NPJ Digit Med. https:\/\/doi.org\/10.1038\/s41746-022-00738-y","journal-title":"NPJ Digit Med"},{"key":"12393_CR8","doi-asserted-by":"publisher","first-page":"1434","DOI":"10.1056\/NEJMsa1300625","volume":"369","author":"JD Birkmeyer","year":"2013","unstructured":"Birkmeyer JD et al (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369:1434\u20131442","journal-title":"N Engl J Med"},{"key":"12393_CR9","doi-asserted-by":"publisher","DOI":"10.1093\/bjs\/znad331","author":"MWE Boal","year":"2023","unstructured":"Boal MWE et al (2023) Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review. Br J Surg. https:\/\/doi.org\/10.1093\/bjs\/znad331","journal-title":"Br J Surg"},{"key":"12393_CR10","unstructured":"Xu J et al. (2024) SEDMamba: enhancing selective state space modelling with bottleneck mechanism and fine-to-coarse temporal fusion for efficient error detection in robot-assisted surgery. arXiv. http:\/\/arxiv.org\/abs\/2406.15920"},{"key":"12393_CR11","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1001\/jamasurg.2018.1512","volume":"153","author":"AJ Hung","year":"2018","unstructured":"Hung AJ, Chen J, Gill IS (2018) Automated performance metrics and machine learning algorithms to measure surgeon performance and anticipate clinical outcomes in robotic surgery. JAMA Surg 153:770\u2013771. https:\/\/doi.org\/10.1001\/jamasurg.2018.1512","journal-title":"JAMA Surg"},{"key":"12393_CR12","doi-asserted-by":"publisher","first-page":"1240","DOI":"10.1016\/j.surg.2020.08.016","volume":"169","author":"F Luongo","year":"2021","unstructured":"Luongo F, Hakim R, Nguyen JH, Anandkumar A, Hung AJ (2021) Deep learning-based computer vision to recognize and classify suturing gestures in robot-assisted surgery. Surgery 169:1240\u20131244","journal-title":"Surgery"},{"issue":"5","key":"12393_CR13","doi-asserted-by":"publisher","first-page":"1245","DOI":"10.1016\/j.surg.2020.09.020","volume":"169","author":"AB Chen","year":"2021","unstructured":"Chen AB, Liang S, Nguyen JH, Liu Y, Hung AJ (2021) Machine learning analyses of automated performance metrics during granular sub-stitch phases predict surgeon experience. Surgery 169(5):1245\u20131249. https:\/\/doi.org\/10.1016\/j.surg.2020.09.020","journal-title":"Surgery"},{"key":"12393_CR14","doi-asserted-by":"publisher","DOI":"10.1007\/s11701-024-02062-x","author":"MWE Boal","year":"2024","unstructured":"Boal MWE et al (2024) Development and evaluation of a societal core robotic surgery accreditation curriculum for the UK. J Robot Surg. https:\/\/doi.org\/10.1007\/s11701-024-02062-x","journal-title":"J Robot Surg"},{"key":"12393_CR15","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1007\/s10151-016-1444-4","volume":"20","author":"JD Foster","year":"2016","unstructured":"Foster JD et al (2016) Application of objective clinical human reliability analysis (OCHRA) in assessment of technical performance in laparoscopic rectal cancer surgery. Tech Coloproctol 20:361\u2013367","journal-title":"Tech Coloproctol"},{"issue":"4","key":"12393_CR16","doi-asserted-by":"publisher","first-page":"778","DOI":"10.1097\/SLA.0000000000003452","volume":"273","author":"NJ Curtis","year":"2021","unstructured":"Curtis NJ, Dennison G, Brown CSB, Hewett PJ, Hanna GB, Stevenson ARL, Francis NK (2021) Clinical evaluation of intraoperative near misses in laparoscopic rectal cancer surgery. Ann Surg 273(4):778\u2013784. https:\/\/doi.org\/10.1097\/SLA.0000000000003452","journal-title":"Ann Surg"},{"key":"12393_CR17","doi-asserted-by":"publisher","first-page":"796","DOI":"10.1007\/s00464-011-1955-z","volume":"26","author":"D Miskovic","year":"2012","unstructured":"Miskovic D et al (2012) Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level and other interventional techniques. Surg Endosc 26:796\u2013803","journal-title":"Surg Endosc"},{"key":"12393_CR18","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-023-10510-2","author":"J Gorard","year":"2023","unstructured":"Gorard J, Boal M, Swamynathan V, Ghamrawi W, Francis N (2023) The application of objective clinical human reliability analysis (OCHRA) in the assessment of basic robotic surgical skills. Surg Endosc. https:\/\/doi.org\/10.1007\/s00464-023-10510-2","journal-title":"Surg Endosc"},{"key":"12393_CR19","doi-asserted-by":"publisher","first-page":"4285","DOI":"10.1007\/s00464-020-07918-5","volume":"35","author":"S Puliatti","year":"2021","unstructured":"Puliatti S et al (2021) Development and validation of the objective assessment of robotic suturing and knot tying skills for chicken anastomotic model. Surg Endosc 35:4285\u20134294","journal-title":"Surg Endosc"},{"key":"12393_CR20","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1111\/bju.15311","volume":"128","author":"A Mottrie","year":"2021","unstructured":"Mottrie A et al (2021) Objective assessment of intraoperative skills for robot-assisted radical prostatectomy (RARP): results from the ERUS scientific and educational working groups metrics initiative. BJU Int 128:103\u2013111","journal-title":"BJU Int"},{"key":"12393_CR21","doi-asserted-by":"publisher","first-page":"903","DOI":"10.1111\/codi.14637","volume":"21","author":"M G\u00f3mez Ruiz","year":"2019","unstructured":"G\u00f3mez Ruiz M et al (2019) Expert consensus on a train-the-trainer curriculum for robotic colorectal surgery. Colorectal Dis 21:903\u2013908","journal-title":"Colorectal Dis"},{"key":"12393_CR22","doi-asserted-by":"publisher","DOI":"10.1097\/SLA.0000000000004584","author":"GB Hanna","year":"2020","unstructured":"Hanna GB et al (2020) Laparoscopic colorectal surgery outcomes improved after national training program (LAPCO) for specialists in England. Ann Surg. https:\/\/doi.org\/10.1097\/SLA.0000000000004584","journal-title":"Ann Surg"},{"key":"12393_CR23","doi-asserted-by":"publisher","first-page":"614","DOI":"10.1111\/j.1463-1318.2011.02643.x","volume":"13","author":"MG Coleman","year":"2011","unstructured":"Coleman MG, Hanna GB, Kennedy R (2011) The National Training Programme for Laparoscopic Colorectal Surgery in England: a new training paradigm. Colorectal Dis 13:614\u2013616","journal-title":"Colorectal Dis"},{"key":"12393_CR24","doi-asserted-by":"publisher","first-page":"4918","DOI":"10.1007\/s00464-021-08578-9","volume":"35","author":"OR Meireles","year":"2021","unstructured":"Meireles OR et al (2021) SAGES consensus recommendations on an annotation framework for surgical video the SAGES video annotation for AI working groups. Surg Endosc 35:4918\u20134929","journal-title":"Surg Endosc"},{"key":"12393_CR25","doi-asserted-by":"publisher","first-page":"3822","DOI":"10.1007\/s00464-018-6108-1","volume":"32","author":"NK Francis","year":"2018","unstructured":"Francis NK et al (2018) EAES classification of intraoperative adverse events in laparoscopic surgery. Surg Endosc 32:3822\u20133829","journal-title":"Surg Endosc"}],"container-title":["Surgical Endoscopy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00464-025-12393-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00464-025-12393-x","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00464-025-12393-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T19:35:23Z","timestamp":1773084923000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00464-025-12393-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,10]]},"references-count":25,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,3]]}},"alternative-id":["12393"],"URL":"https:\/\/doi.org\/10.1007\/s00464-025-12393-x","relation":{},"ISSN":["0930-2794","1432-2218"],"issn-type":[{"value":"0930-2794","type":"print"},{"value":"1432-2218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,10]]},"assertion":[{"value":"8 September 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 November 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 December 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"M.Boal, C.Reali, R.Duhoky, T.Gill, J.Khan, D.Miskovic and N.Francis have no conflicts of interest or financial ties to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Disclosures"}},{"value":"The VAMIS study protocol (ClinicalTrials.gov NCT05279287) has Health Research Authority and Health and Care Research Wales (HCRW) approval, as well as local Research Ethics Committee (REC) approval for national implementation (IRAS project ID: 209024).","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}