{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T13:25:33Z","timestamp":1792589133762,"version":"4.5.0"},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2023,6,27]],"date-time":"2023-06-27T00:00:00Z","timestamp":1687824000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,6,27]],"date-time":"2023-06-27T00:00:00Z","timestamp":1687824000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100018258","name":"Southern Methodist University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100018258","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Robotic Surg"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>We use machine learning to evaluate surgical skill from videos during the tumor resection and renography steps of a robotic assisted partial nephrectomy (RAPN). This expands previous work using synthetic tissue to include actual surgeries. We investigate cascaded neural networks for predicting surgical proficiency scores (OSATS and GEARS) from RAPN videos recorded from the DaVinci system. The semantic segmentation task generates a mask and tracks the various surgical instruments. The movements from the instruments found via semantic segmentation are processed by a scoring network that regresses (predicts) GEARS and OSATS scoring for each subcategory. Overall, the model performs well for many subcategories such as force sensitivity and knowledge of instruments of GEARS and OSATS scoring, but can suffer from false positives and negatives that would not be expected of human raters. This is mainly attributed to limited training data variability and sparsity.<\/jats:p>","DOI":"10.1007\/s11701-023-01657-0","type":"journal-article","created":{"date-parts":[[2023,6,27]],"date-time":"2023-06-27T03:01:58Z","timestamp":1687834918000},"page":"2323-2330","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Evaluating robotic-assisted partial nephrectomy surgeons with fully convolutional segmentation and multi-task attention networks"],"prefix":"10.1007","volume":"17","author":[{"given":"Yihao","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhongjie","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jessica","family":"Dai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tara N.","family":"Morgan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alaina","family":"Garbens","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hal","family":"Kominsky","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeffrey","family":"Gahan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric C.","family":"Larson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,6,27]]},"reference":[{"issue":"5","key":"1657_CR1","doi-asserted-by":"publisher","first-page":"54","DOI":"10.5772\/60118","volume":"12","author":"D Brian","year":"2015","unstructured":"Brian D (2015) Robotic surgery-a personal view of the past, present and future. Int J Adv Robot Syst 12(5):54","journal-title":"Int J Adv Robot Syst"},{"issue":"6","key":"1657_CR2","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1109\/MSP.2017.2738401","volume":"34","author":"R Dhanesh","year":"2017","unstructured":"Dhanesh R, Taylor Graham W (2017) Deep multimodal learning: a survey on recent advances and trends\u2019. IEEE Signal Process Mag 34(6):96\u2013108","journal-title":"IEEE Signal Process Mag"},{"key":"1657_CR3","unstructured":"Chen Y (2015) Convolutional neural network for sentence classification. Master\u2019s thesis. University of Waterloo"},{"issue":"2","key":"1657_CR4","first-page":"273","volume":"84","author":"JA Martin","year":"1997","unstructured":"Martin JA, Glenn R, Richard R, Helen M, John M, Carol H, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84(2):273\u2013278","journal-title":"Br J Surg"},{"issue":"4","key":"1657_CR5","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1089\/end.2018.0871","volume":"33","author":"A Johnson Brett","year":"2019","unstructured":"Johnson Brett A, Matthew T, Steinberg Ryan L, Matthew K, Bradly M, Gahan Jeffrey C (2019) Design and validation of a lowcost, high-fidelity model for urethrovesical anastomosis in radical prostatectomy. J Endourol 33(4):331\u2013336","journal-title":"J Endourol"},{"issue":"4","key":"1657_CR6","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1007\/s11701-021-01316-2","volume":"16","author":"W Yihao","year":"2022","unstructured":"Yihao W, Jessica D, Morgan Tara N, Mohamed E, Garbens Alaina Q, Xingming SR, Jeffrey G, Larson Eric C (2022) Evaluating robotic-assisted surgery training videos with multi-task convolutional neural networks. J Robot Surg 16(4):917\u2013925. https:\/\/doi.org\/10.1007\/s11701-021-01316-2","journal-title":"J Robot Surg"},{"key":"1657_CR7","first-page":"30","volume-title":"Advances in neural information processing systems","author":"V Ashish","year":"2017","unstructured":"Ashish V, Noam S, Niki P, Jakob U, Llion J, Gomez Aidan N, \u0141ukasz K, Illia P (2017) Attention is all you need. Advances in neural information processing systems. Springer, p 30"},{"key":"1657_CR8","doi-asserted-by":"crossref","unstructured":"Sammit G, Wu Z, Wang Y, Wu Z, Kamata A, Nese J, Larson EC (2022) Automated prosody classification for oral reading fluency with quadratic kappa loss and attentive X-vectors. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 3613\u20133617","DOI":"10.1109\/ICASSP43922.2022.9747391"},{"issue":"1","key":"1657_CR9","first-page":"26","volume":"22","author":"Z Zijian","year":"2017","unstructured":"Zijian Z, Sandrine V, Ying W, Faliang C, Ruijian L (2017) Tracking-by-detection of surgical instruments in minimally invasive surgery via the convolutional neural network deep learning-based method. Comput Assist Surg 22(1):26\u201335","journal-title":"Comput Assist Surg"},{"key":"1657_CR10","first-page":"88","volume-title":"Machine learning for healthcare conference","author":"L Hei","year":"2017","unstructured":"Hei L, Khurshid G, Jia D (2017) Surgeon technical skill assessment using computer vision based analysis. Machine learning for healthcare conference. PMLR, pp 88\u201399"},{"issue":"6","key":"1657_CR11","doi-asserted-by":"publisher","first-page":"1964","DOI":"10.3390\/jcm9061964","volume":"9","author":"D Lee","year":"2020","unstructured":"Lee D, Won YH, Kwon H, Hyoun-Joong K, Eun LK, Chan KH (2020) Evaluation of surgical skills during robotic surgery by deep learning-based multiple surgical instrument tracking in training and actual operations. J Clin Med 9(6):1964","journal-title":"J Clin Med"},{"issue":"4","key":"1657_CR12","first-page":"e505","volume":"203","author":"G Jeffrey","year":"2020","unstructured":"Jeffrey G, Ryan S, Garbens Alaina Q, Xingming LE (2020) MP34-06 machine learning using a multi-task convolutional neural networks can accurately assess robotic skills. J Urol 203(4):e505\u2013e505","journal-title":"J Urol"},{"key":"1657_CR13","doi-asserted-by":"crossref","unstructured":"Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V, Garcia-Rodriguez J (2017) A review on deep learning techniques applied to semantic segmentation. ArXiv preprint arXiv:1704.06857","DOI":"10.1016\/j.asoc.2018.05.018"},{"key":"1657_CR14","doi-asserted-by":"crossref","unstructured":"Wang P, Chen P, Yuan Y, Liu D, Huang Z, Hou X, Cottrell G (2018) Understanding convolution for semantic segmentation. In: IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE, pp 1451\u20131460","DOI":"10.1109\/WACV.2018.00163"},{"issue":"3","key":"1657_CR15","doi-asserted-by":"publisher","first-page":"413","DOI":"10.3390\/jpm13030413","volume":"13","author":"M Giorgia","year":"2023","unstructured":"Giorgia M, Leonardo T, Luca U, Francesco P, Enrico V (2023) A multi-task convolutional neural network for semantic segmentation and event detection in laparoscopic surgery. J Personal Med 13(3):413","journal-title":"J Personal Med"},{"issue":"6","key":"1657_CR16","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1109\/MSP.2017.2738401","volume":"34","author":"R Dhanesh","year":"2017","unstructured":"Dhanesh R, Taylor Graham W (2017) Deep multimodal learning: a survey on recent advances and trends. IEEE Signal Process Mag 34(6):96\u2013108","journal-title":"IEEE Signal Process Mag"},{"key":"1657_CR17","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1016\/j.patrec.2017.05.018","volume":"105","author":"J de La Torre","year":"2018","unstructured":"de La Torre J, Puig D, Valls A (2018) Weighted kappa loss function for multi-class classification of ordinal data in deep learning. Pattern Recognit Lett 105:144\u2013154","journal-title":"Pattern Recognit Lett"},{"issue":"2","key":"1657_CR18","doi-asserted-by":"publisher","first-page":"388","DOI":"10.1016\/j.euf.2022.09.017","volume":"9","author":"F Rui","year":"2023","unstructured":"Rui F, Alberto B, James P, Alexandre M, Ben VC, Vander SJ, Angelo M, Gallagher Anthony G et al (2023) International expert consensus on metric-based characterization of robot-assisted partial nephrectomy. Eur Urol Focus 9(2):388\u2013395","journal-title":"Eur Urol Focus"},{"key":"1657_CR19","unstructured":"Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: CoRR abs\/1505.04597 . arXiv: 1505.04597"},{"key":"1657_CR20","unstructured":"Shen Z, Bello I, Vemulapalli R, Jia X, Chen C-H (2020) Global self-attention networks for image recognition. In: arXiv preprint arXiv:2010.03019"},{"key":"1657_CR21","doi-asserted-by":"crossref","unstructured":"Chen C-FR, Fan Q, Panda R (2021) Crossvit: cross attention multi-scale vision transformer for image classification. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision, pp 357\u2013366","DOI":"10.1109\/ICCV48922.2021.00041"},{"key":"1657_CR22","doi-asserted-by":"crossref","unstructured":"Snyder D, Garcia-Romero D, Sell G, Povey D, Khudanpur S (2018) X-vectors: robust DNN embeddings for speaker recognition. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, pp 5329\u20135333","DOI":"10.1109\/ICASSP.2018.8461375"},{"key":"1657_CR23","doi-asserted-by":"crossref","unstructured":"Okabe K, Koshinaka T, Shinoda K (2018) Attentive statistics pooling for deep speaker embedding. In: arXiv preprint arXiv:1803.10963","DOI":"10.21437\/Interspeech.2018-993"},{"issue":"1","key":"1657_CR24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-018-0162-3","volume":"6","author":"S Connor","year":"2019","unstructured":"Connor S, Khoshgoftaar Taghi M (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1\u201348","journal-title":"J Big Data"},{"key":"1657_CR25","doi-asserted-by":"crossref","unstructured":"He K, Gkioxari G, Doll\u00e1r P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2961\u20132969","DOI":"10.1109\/ICCV.2017.322"},{"issue":"438","key":"1657_CR26","first-page":"548","volume":"92","author":"E Bradley","year":"1997","unstructured":"Bradley E, Robert T (1997) Improvements on cross-validation: the 632+ bootstrap method. J Am Stat Assoc 92(438):548\u2013560","journal-title":"J Am Stat Assoc"}],"container-title":["Journal of Robotic Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11701-023-01657-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11701-023-01657-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11701-023-01657-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,9]],"date-time":"2023-09-09T01:12:16Z","timestamp":1694221936000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11701-023-01657-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,27]]},"references-count":26,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["1657"],"URL":"https:\/\/doi.org\/10.1007\/s11701-023-01657-0","relation":{},"ISSN":["1863-2491"],"issn-type":[{"value":"1863-2491","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,27]]},"assertion":[{"value":"17 May 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 June 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 June 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"IRB approval"}}]}}