{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T11:57:31Z","timestamp":1793361451490,"version":"4.5.0"},"reference-count":44,"publisher":"American Medical Association (AMA)","issue":"8","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Surg"],"abstract":"\n Importance<\/jats:title>\n Automatic surgical skill assessment with artificial intelligence (AI) is more objective than manual video review\u2013based skill assessment and can reduce human burden. Standardization of surgical field development is an important aspect of this skill assessment.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n To develop a deep learning model that can recognize the standardized surgical fields in laparoscopic sigmoid colon resection and to evaluate the feasibility of automatic surgical skill assessment based on the concordance of the standardized surgical field development using the proposed deep learning model.<\/jats:p>\n <\/jats:sec>\n \n Design, Setting, and Participants<\/jats:title>\n This retrospective diagnostic study used intraoperative videos of laparoscopic colorectal surgery submitted to the Japan Society for Endoscopic Surgery between August 2016 and November 2017. Data were analyzed from April 2020 to September 2022.<\/jats:p>\n <\/jats:sec>\n \n Interventions<\/jats:title>\n Videos of surgery performed by expert surgeons with Endoscopic Surgical Skill Qualification System (ESSQS) scores higher than 75 were used to construct a deep learning model able to recognize a standardized surgical field and output its similarity to standardized surgical field development as an AI confidence score (AICS). Other videos were extracted as the validation set.<\/jats:p>\n <\/jats:sec>\n \n Main Outcomes and Measures<\/jats:title>\n Videos with scores less than or greater than 2 SDs from the mean were defined as the low- and high-score groups, respectively. The correlation between AICS and ESSQS score and the screening performance using AICS for low- and high-score groups were analyzed.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n The sample included 650 intraoperative videos, 60 of which were used for model construction and 60 for validation. The Spearman rank correlation coefficient between the AICS and ESSQS score was 0.81. The receiver operating characteristic (ROC) curves for the screening of the low- and high-score groups were plotted, and the areas under the ROC curve for the low- and high-score group screening were 0.93 and 0.94, respectively.<\/jats:p>\n <\/jats:sec>\n \n Conclusions and Relevance<\/jats:title>\n The AICS from the developed model strongly correlated with the ESSQS score, demonstrating the model\u2019s feasibility for use as a method of automatic surgical skill assessment. The findings also suggest the feasibility of the proposed model for creating an automated screening system for surgical skills and its potential application to other types of endoscopic procedures.<\/jats:p>\n <\/jats:sec>","DOI":"10.1001\/jamasurg.2023.1131","type":"journal-article","created":{"date-parts":[[2023,6,7]],"date-time":"2023-06-07T11:03:05Z","timestamp":1686135785000},"page":"e231131","source":"Crossref","is-referenced-by-count":93,"title":["Automatic Surgical Skill Assessment System Based on Concordance of Standardized Surgical Field Development Using Artificial Intelligence"],"prefix":"10.1001","volume":"158","author":[{"given":"Takahiro","family":"Igaki","sequence":"first","affiliation":[{"name":"Surgical Device Innovation Office, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"},{"name":"Department of Colorectal Surgery, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"},{"name":"Department of Gastrointestinal Surgery, Tokyo Medical and Dental University Graduate School of Medicine, Yushima, Bunkyo-Ku, Tokyo, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daichi","family":"Kitaguchi","sequence":"additional","affiliation":[{"name":"Surgical Device Innovation Office, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"},{"name":"Department of Colorectal Surgery, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hiroki","family":"Matsuzaki","sequence":"additional","affiliation":[{"name":"Surgical Device Innovation Office, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kei","family":"Nakajima","sequence":"additional","affiliation":[{"name":"Surgical Device Innovation Office, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"},{"name":"Department of Colorectal Surgery, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shigehiro","family":"Kojima","sequence":"additional","affiliation":[{"name":"Surgical Device Innovation Office, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"},{"name":"Department of Colorectal Surgery, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hiro","family":"Hasegawa","sequence":"additional","affiliation":[{"name":"Surgical Device Innovation Office, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"},{"name":"Department of Colorectal Surgery, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nobuyoshi","family":"Takeshita","sequence":"additional","affiliation":[{"name":"Surgical Device Innovation Office, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"},{"name":"Department of Colorectal Surgery, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yusuke","family":"Kinugasa","sequence":"additional","affiliation":[{"name":"Department of Gastrointestinal Surgery, Tokyo Medical and Dental University Graduate School of Medicine, Yushima, Bunkyo-Ku, Tokyo, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Masaaki","family":"Ito","sequence":"additional","affiliation":[{"name":"Surgical Device Innovation Office, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"},{"name":"Department of Colorectal Surgery, National Cancer Center Hospital East, Kashiwanoha, Kashiwa, Chiba, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10","published-online":{"date-parts":[[2023,8,9]]},"reference":[{"issue":"15","key":"soi230023r1","doi-asserted-by":"publisher","first-page":"1434","DOI":"10.1056\/NEJMsa1300625","article-title":"Surgical skill and complication rates after bariatric surgery.","volume":"369","author":"Birkmeyer","year":"2013","journal-title":"N Engl J Med"},{"issue":"10","key":"soi230023r2","doi-asserted-by":"publisher","first-page":"960","DOI":"10.1001\/jamasurg.2020.3007","article-title":"Association between surgeon technical skills and patient outcomes.","volume":"155","author":"Stulberg","year":"2020","journal-title":"JAMA Surg"},{"issue":"2","key":"soi230023r3","first-page":"273","article-title":"Objective Structured Assessment of Technical Skill (OSATS) for surgical residents.","volume":"84","author":"Martin","year":"1997","journal-title":"Br J Surg"},{"issue":"6","key":"soi230023r4","first-page":"403","article-title":"MIST VR: a virtual reality trainer for laparoscopic surgery assesses performance.","volume":"79","author":"Wilson","year":"1997","journal-title":"Ann R Coll Surg Engl"},{"issue":"1","key":"soi230023r5","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1001\/archsurg.136.1.40","article-title":"Reliability of the Advanced Dundee Endoscopic Psychomotor Tester for bimanual tasks.","volume":"136","author":"Francis","year":"2001","journal-title":"Arch Surg"},{"issue":"4","key":"soi230023r6","doi-asserted-by":"publisher","first-page":"640","DOI":"10.1007\/s004640080081","article-title":"Motion analysis.","volume":"16","author":"Smith","year":"2002","journal-title":"Surg Endosc"},{"issue":"1","key":"soi230023r7","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1016\/S0002-9610(02)00891-7","article-title":"The relationship between motion analysis and surgical technical assessments.","volume":"184","author":"Datta","year":"2002","journal-title":"Am J Surg"},{"issue":"1","key":"soi230023r8","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1016\/S0039-6060(03)00155-7","article-title":"Using simulators to assess laparoscopic competence: ready for widespread use?","volume":"135","author":"Feldman","year":"2004","journal-title":"Surgery"},{"issue":"3","key":"soi230023r9","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1001\/archsurg.140.3.293","article-title":"Synchronized video and motion analysis for the assessment of procedures in the operating theater.","volume":"140","author":"Dosis","year":"2005","journal-title":"Arch Surg"},{"issue":"8","key":"soi230023r10","doi-asserted-by":"publisher","first-page":"705","DOI":"10.1007\/s00595-007-3696-0","article-title":"Objective assessment of endoscopic surgical skills by analyzing direction-dependent dexterity using the Hiroshima University Endoscopic Surgical Assessment Device (HUESAD).","volume":"38","author":"Egi","year":"2008","journal-title":"Surg Today"},{"issue":"1","key":"soi230023r11","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1007\/s00464-008-0120-9","article-title":"Simulation in laparoscopic surgery: a concurrent validity study for FLS.","volume":"23","author":"Xeroulis","year":"2009","journal-title":"Surg Endosc"},{"issue":"3","key":"soi230023r12","doi-asserted-by":"publisher","first-page":"142","DOI":"10.3109\/13645706.2011.596547","article-title":"Approaching time is important for assessment of endoscopic surgical skills.","volume":"21","author":"Tokunaga","year":"2012","journal-title":"Minim Invasive Ther Allied Technol"},{"issue":"1","key":"soi230023r13","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/j.amjsurg.2005.04.004","article-title":"A global assessment tool for evaluation of intraoperative laparoscopic skills.","volume":"190","author":"Vassiliou","year":"2005","journal-title":"Am J Surg"},{"issue":"2","key":"soi230023r14","doi-asserted-by":"publisher","first-page":"372","DOI":"10.1097\/SLA.0b013e318160b371","article-title":"Toward feasible, valid, and reliable video-based assessments of technical surgical skills in the operating room.","volume":"247","author":"Aggarwal","year":"2008","journal-title":"Ann Surg"},{"issue":"7","key":"soi230023r15","doi-asserted-by":"publisher","first-page":"972","DOI":"10.1002\/bjs.7115","article-title":"Objective assessment of technical surgical skills.","volume":"97","author":"van Hove","year":"2010","journal-title":"Br J Surg"},{"issue":"2","key":"soi230023r16","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1001\/jamasurg.2019.4917","article-title":"Artificial intelligence and surgical decision-making.","volume":"155","author":"Loftus","year":"2020","journal-title":"JAMA Surg"},{"key":"soi230023r17","doi-asserted-by":"publisher","first-page":"S36","DOI":"10.1016\/j.metabol.2017.01.011","article-title":"Artificial intelligence in medicine.","volume":"69S","author":"Hamet","year":"2017","journal-title":"Metabolism"},{"issue":"7788","key":"soi230023r18","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1038\/s41586-019-1799-6","article-title":"International evaluation of an AI system for breast cancer screening.","volume":"577","author":"McKinney","year":"2020","journal-title":"Nature"},{"issue":"10223","key":"soi230023r19","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1016\/S0140-6736(20)30294-4","article-title":"Adding artificial intelligence to gastrointestinal endoscopy.","volume":"395","author":"Berzin","year":"2020","journal-title":"Lancet"},{"issue":"6","key":"soi230023r20","doi-asserted-by":"publisher","first-page":"2493","DOI":"10.1007\/s00464-020-07659-5","article-title":"Computer-assisted real-time automatic prostate segmentation during TaTME: a single-center feasibility study.","volume":"35","author":"Kitaguchi","year":"2021","journal-title":"Surg Endosc"},{"issue":"12","key":"soi230023r21","doi-asserted-by":"publisher","first-page":"5377","DOI":"10.1007\/s00464-019-07330-8","article-title":"SurgAI: deep learning for computerized laparoscopic image understanding in gynaecology.","volume":"34","author":"Madad Zadeh","year":"2020","journal-title":"Surg Endosc"},{"issue":"3","key":"soi230023r22","doi-asserted-by":"publisher","first-page":"476","DOI":"10.1097\/SLA.0b013e318275b72a","article-title":"Is competency assessment at the specialist level achievable? a study for the national training programme in laparoscopic colorectal surgery in England.","volume":"257","author":"Miskovic","year":"2013","journal-title":"Ann Surg"},{"issue":"7","key":"soi230023r23","doi-asserted-by":"publisher","first-page":"738","DOI":"10.1097\/DCR.0000000000000817","article-title":"The American Society of Colon and Rectal Surgeons assessment tool for performance of laparoscopic colectomy.","volume":"60","author":"Champagne","year":"2017","journal-title":"Dis Colon Rectum"},{"issue":"7","key":"soi230023r24","doi-asserted-by":"publisher","first-page":"590","DOI":"10.1001\/jamasurg.2020.1004","article-title":"Association of surgical skill assessment with clinical outcomes in cancer surgery.","volume":"155","author":"Curtis","year":"2020","journal-title":"JAMA Surg"},{"issue":"1","key":"soi230023r25","first-page":"101","article-title":"An analysis of the problem of biliary injury during laparoscopic cholecystectomy.","volume":"180","author":"Strasberg","year":"1995","journal-title":"J Am Coll Surg"},{"issue":"1","key":"soi230023r26","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1016\/j.jamcollsurg.2010.02.053","article-title":"Rationale and use of the critical view of safety in laparoscopic cholecystectomy.","volume":"211","author":"Strasberg","year":"2010","journal-title":"J Am Coll Surg"},{"issue":"12","key":"soi230023r27","doi-asserted-by":"publisher","first-page":"5021","DOI":"10.1007\/s00464-018-6267-0","article-title":"Laparoscopic right hemicolectomy with CME: standardization using the \u201ccritical view\u201d concept.","volume":"32","author":"Strey","year":"2018","journal-title":"Surg Endosc"},{"issue":"11","key":"soi230023r28","doi-asserted-by":"publisher","first-page":"4924","DOI":"10.1007\/s00464-019-07281-0","article-title":"Real-time automatic surgical phase recognition in laparoscopic sigmoidectomy using the convolutional neural network-based deep learning approach.","volume":"34","author":"Kitaguchi","year":"2020","journal-title":"Surg Endosc"},{"key":"soi230023r29","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1016\/j.ijsu.2020.05.015","article-title":"Automated laparoscopic colorectal surgery workflow recognition using artificial intelligence: experimental research.","volume":"79","author":"Kitaguchi","year":"2020","journal-title":"Int J Surg"},{"issue":"20","key":"soi230023r30","doi-asserted-by":"publisher","first-page":"2191","DOI":"10.1001\/jama.2013.281053","article-title":"World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects.","volume":"310","author":"World Medical Association","year":"2013","journal-title":"JAMA"},{"issue":"6","key":"soi230023r31","doi-asserted-by":"publisher","first-page":"721","DOI":"10.1002\/ags3.v4.6","article-title":"Clinical impact of Endoscopic Surgical Skill Qualification System (ESSQS) by Japan Society for Endoscopic Surgery (JSES) for laparoscopic distal gastrectomy and low anterior resection based on the National Clinical Database (NCD) registry.","volume":"4","author":"Akagi","year":"2020","journal-title":"Ann Gastroenterol Surg"},{"issue":"11","key":"soi230023r32","doi-asserted-by":"publisher","first-page":"6089","DOI":"10.1007\/s00464-020-08102-5","article-title":"Impact of the Endoscopic Surgical Skill Qualification System on the safety of laparoscopic gastrectomy for gastric cancer.","volume":"35","author":"Shibasaki","year":"2021","journal-title":"Surg Endosc"},{"issue":"1","key":"soi230023r33","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1186\/s12893-019-0528-2","article-title":"Usefulness of the endoscopic surgical skill qualification system in laparoscopic colorectal surgery: short-term outcomes: a single-center and retrospective analysis.","volume":"19","author":"Aoyama","year":"2019","journal-title":"BMC Surg"},{"issue":"3","key":"soi230023r34","doi-asserted-by":"publisher","first-page":"486","DOI":"10.1002\/bjs5.50263","article-title":"Impact of technically qualified surgeons on laparoscopic colorectal resection outcomes: results of a propensity score-matching analysis.","volume":"4","author":"Ichikawa","year":"2020","journal-title":"BJS Open"},{"issue":"3","key":"soi230023r36","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1038\/bmt.2012.244","article-title":"Investigation of the freely available easy-to-use software \u2018EZR\u2019 for medical statistics.","volume":"48","author":"Kanda","year":"2013","journal-title":"Bone Marrow Transplant"},{"issue":"9","key":"soi230023r37","doi-asserted-by":"publisher","first-page":"1623","DOI":"10.1007\/s11548-016-1468-2","article-title":"Automated video-based assessment of surgical skills for training and evaluation in medical schools.","volume":"11","author":"Zia","year":"2016","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"3","key":"soi230023r38","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1097\/SLA.0000000000002478","article-title":"Modeling surgical technical skill using expert assessment for automated computer rating.","volume":"269","author":"Azari","year":"2019","journal-title":"Ann Surg"},{"issue":"1","key":"soi230023r39","doi-asserted-by":"publisher","DOI":"10.1177\/2473974X19830635","article-title":"Semiautomated motion tracking for objective skills assessment in otologic surgery: a pilot study.","volume":"3","author":"Kanumuri","year":"2019","journal-title":"OTO Open"},{"key":"soi230023r40","first-page":"2002","article-title":"Surgical tool detection in cataract surgery videos through multi-image fusion inside a convolutional neural network.","volume":"2017","author":"Al Hajj","year":"2017","journal-title":"Annu Int Conf IEEE Eng Med Biol Soc"},{"issue":"5","key":"soi230023r41","doi-asserted-by":"publisher","first-page":"731","DOI":"10.1007\/s11548-018-1735-5","article-title":"Automated surgical skill assessment in RMIS training.","volume":"13","author":"Zia","year":"2018","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"6","key":"soi230023r42","doi-asserted-by":"publisher","first-page":"1964","DOI":"10.3390\/jcm9061964","article-title":"Evaluation of surgical skills during robotic surgery by deep learning-based multiple surgical instrument tracking in training and actual operations.","volume":"9","author":"Lee","year":"2020","journal-title":"J Clin Med"},{"issue":"6","key":"soi230023r43","doi-asserted-by":"publisher","first-page":"1629","DOI":"10.1016\/j.jsurg.2019.06.011","article-title":"Automated methods of technical skill assessment in surgery: a systematic review.","volume":"76","author":"Levin","year":"2019","journal-title":"J Surg Educ"},{"issue":"5","key":"soi230023r44","doi-asserted-by":"publisher","first-page":"955","DOI":"10.1097\/SLA.0000000000004351","article-title":"Artificial intelligence for surgical safety: automatic assessment of the critical view of safety in laparoscopic cholecystectomy using deep learning.","volume":"275","author":"Mascagni","year":"2022","journal-title":"Ann Surg"},{"key":"soi230023r35","doi-asserted-by":"crossref","unstructured":"Chollet? F. Xception: deep learning with depthwise separable convolutions.? ArXiv. Preprint posted online April 4, 2017. doi:10.1109\/CVPR.2017.195","DOI":"10.1109\/CVPR.2017.195"}],"container-title":["JAMA Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamasurgery\/articlepdf\/2805952\/jamasurgery_igaki_2023_oi_230023_1691551386.63513.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,9]],"date-time":"2023-08-09T11:03:26Z","timestamp":1691579006000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamasurgery\/fullarticle\/2805952"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,9]]},"references-count":44,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2023,8,1]]}},"URL":"https:\/\/doi.org\/10.1001\/jamasurg.2023.1131","relation":{},"ISSN":["2168-6254"],"issn-type":[{"value":"2168-6254","type":"print"}],"subject":[],"published":{"date-parts":[[2023,8,9]]}}}