{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T15:26:22Z","timestamp":1793892382589,"version":"4.8.0"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T00:00:00Z","timestamp":1773100800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T00:00:00Z","timestamp":1773100800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000070","name":"National Institute of Biomedical Imaging and Bioengineering","doi-asserted-by":"publisher","award":["R01EB029398"],"award-info":[{"award-number":["R01EB029398"]}],"id":[{"id":"10.13039\/100000070","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Robotic Surg"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    This study aimed to classify robot-assisted surgery (RAS) skill levels (inexperienced, competent, and experienced) during performance of vesico-urethral anastomosis (VUA) using multimodal physiological signals. We trained Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) models on data collected from 23 RAS (RAS trainees and experienced surgeons) performing two VUAs on animal tissue. The dataset included 116-channel electroencephalogram (EEG) and 20 eye-tracking signals recorded during two VUA subtasks: (1) needle grasping, positioning, and entry; and (2) needle driving with wrist rotation and suture pull-out. Skill levels were rated by three raters using the Robotic Anastomosis Competency Evaluation (RACE) tool. Hyperparameters of the models were tuned using grid search with group 4-fold cross-validation on 16 participants and final model performance was evaluated on data from 7 unseen (held-out test) participants, repeated over 10 iterations. Weighted F-scores for classifying skill level using EEG and eye-tracking data were 0.84 for subtask 1 and 0.89 for subtask 2. Using paired t-tests, high-density EEG (116 channels) significantly outperformed low-density EEG (32 channels) for subtask 1 (\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.001), with no difference for subtask 2 (\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.15). Adding eye-tracking data significantly improved classification for subtask 2 (\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.001), but not for subtask 1 (\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.5). Multimodal deep learning using EEG and eye-tracking data enabled objective classification of surgical skills during VUA. The benefits of high-density EEG and multimodal integration were task-dependent, underscoring the need to tailor assessment tools to the cognitive and sensorimotor demands of specific surgical subtasks.\n                  <\/jats:p>","DOI":"10.1007\/s11701-026-03290-z","type":"journal-article","created":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T02:33:39Z","timestamp":1773110019000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Multimodal deep learning for objective skill assessment in robot-assisted vesico-urethral anastomosis"],"prefix":"10.1007","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9256-6284","authenticated-orcid":false,"given":"Somayeh B.","family":"Shafiei","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2842-031X","authenticated-orcid":false,"given":"Saeed","family":"Shadpour","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anthony","family":"Dakwar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhaomin","family":"Xu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James L.","family":"Mohler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,3,10]]},"reference":[{"issue":"1","key":"3290_CR1","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1016\/j.juro.2011.09.032","volume":"187","author":"AC Goh","year":"2012","unstructured":"Goh AC et al (2012) Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol 187(1):247\u2013252","journal-title":"J Urol"},{"issue":"1","key":"3290_CR2","first-page":"E10","volume":"13","author":"H Khan","year":"2019","unstructured":"Khan H et al (2019) Use of Robotic Anastomosis Competency Evaluation (RACE) tool for assessment of surgical competency during urethrovesical anastomosis. Can Urol Association J 13(1):E10","journal-title":"Can Urol Association J"},{"issue":"1","key":"3290_CR3","doi-asserted-by":"publisher","first-page":"296","DOI":"10.1016\/j.juro.2017.07.081","volume":"199","author":"AJ Hung","year":"2018","unstructured":"Hung AJ et al (2018) Development and validation of objective performance metrics for robot-assisted radical prostatectomy: a pilot study. J Urol 199(1):296\u2013304","journal-title":"J Urol"},{"key":"3290_CR4","unstructured":"Rosen J et al (2001) Objective laparoscopic skills assessments of surgical residents using Hidden Markov Models based on haptic information and tool\/tissue interactions, in Medicine Meets Virtual Reality. 2001, IOS Press. pp. 417\u2013423"},{"key":"3290_CR5","doi-asserted-by":"crossref","unstructured":"Ahmidi N et al (2010) Surgical task and skill classification from eye tracking and tool motion in minimally invasive surgery. in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer","DOI":"10.1007\/978-3-642-15711-0_37"},{"issue":"1","key":"3290_CR6","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1016\/j.jsurg.2018.05.009","volume":"76","author":"JC Kwong","year":"2019","unstructured":"Kwong JC, Lee JY, Goldenberg MG (2019) Understanding and assessing nontechnical skills in robotic urological surgery: a systematic review and synthesis of the validity evidence. J Surg Educ 76(1):193\u2013200","journal-title":"J Surg Educ"},{"key":"3290_CR7","doi-asserted-by":"crossref","unstructured":"Shafiei SB, Shadpour S, Mohler JL (2024) An Integrated Electroencephalography and Eye-Tracking Analysis Using eXtreme Gradient Boosting for Mental Workload Evaluation in Surgery. Human Factors. p. 00187208241285513","DOI":"10.1177\/00187208241285513"},{"issue":"1","key":"3290_CR8","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1038\/s41514-023-00119-z","volume":"9","author":"S Shadpour","year":"2023","unstructured":"Shadpour S et al (2023) Developing cognitive workload and performance evaluation models using functional brain network analysis. npj Aging 9(1):22","journal-title":"npj Aging"},{"issue":"6","key":"3290_CR9","doi-asserted-by":"publisher","first-page":"2963","DOI":"10.1007\/s11701-023-01722-8","volume":"17","author":"SB Shafiei","year":"2023","unstructured":"Shafiei SB et al (2023) Surgical skill level classification model development using EEG and eye-gaze data and machine learning algorithms. J robotic Surg 17(6):2963\u20132971","journal-title":"J robotic Surg"},{"issue":"11","key":"3290_CR10","doi-asserted-by":"publisher","first-page":"8447","DOI":"10.1007\/s00464-023-10409-y","volume":"37","author":"SB Shafiei","year":"2023","unstructured":"Shafiei SB et al (2023) Performance and learning rate prediction models development in FLS and RAS surgical tasks using electroencephalogram and eye gaze data and machine learning. Surg Endosc 37(11):8447\u20138463","journal-title":"Surg Endosc"},{"issue":"1","key":"3290_CR11","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1038\/s41539-024-00216-y","volume":"9","author":"SB Shafiei","year":"2024","unstructured":"Shafiei SB et al (2024) Development of performance and learning rate evaluation models in robot-assisted surgery using electroencephalography and eye-tracking. npj Sci Learn 9(1):3","journal-title":"npj Sci Learn"},{"issue":"1","key":"3290_CR12","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1038\/s41746-022-00566-0","volume":"5","author":"K Lam","year":"2022","unstructured":"Lam K et al (2022) Machine learning for technical skill assessment in surgery: a systematic review. NPJ Digit Med 5(1):24","journal-title":"NPJ Digit Med"},{"issue":"1","key":"3290_CR13","doi-asserted-by":"publisher","first-page":"430","DOI":"10.1038\/s41398-020-01117-5","volume":"10","author":"SB Shafiei","year":"2020","unstructured":"Shafiei SB et al (2020) Identifying mental health status using deep neural network trained by visual metrics. Translational psychiatry 10(1):430","journal-title":"Translational psychiatry"},{"issue":"5","key":"3290_CR14","doi-asserted-by":"publisher","first-page":"2305","DOI":"10.1007\/s11760-022-02447-1","volume":"17","author":"Y Zhang","year":"2023","unstructured":"Zhang Y, Zhang Y, Wang S (2023) An attention-based hybrid deep learning model for EEG emotion recognition. SIViP 17(5):2305\u20132313","journal-title":"SIViP"},{"issue":"11","key":"3290_CR15","doi-asserted-by":"publisher","first-page":"5406","DOI":"10.1109\/JBHI.2022.3198688","volume":"26","author":"L Feng","year":"2022","unstructured":"Feng L et al (2022) EEG-based emotion recognition using spatial-temporal graph convolutional LSTM with attention mechanism. IEEE J Biomedical Health Inf 26(11):5406\u20135417","journal-title":"IEEE J Biomedical Health Inf"},{"issue":"9","key":"3290_CR16","doi-asserted-by":"publisher","first-page":"858","DOI":"10.1089\/end.2017.0284","volume":"31","author":"MG Goldenberg","year":"2017","unstructured":"Goldenberg MG, Goldenberg L, Grantcharov TP (2017) Surgeon performance predicts early continence after robot-assisted radical prostatectomy. J Endourol 31(9):858\u2013863","journal-title":"J Endourol"},{"issue":"1","key":"3290_CR17","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1016\/j.urology.2014.09.017","volume":"85","author":"SJ Raza","year":"2015","unstructured":"Raza SJ et al (2015) Surgical competency for urethrovesical anastomosis during robot-assisted radical prostatectomy: development and validation of the robotic anastomosis competency evaluation. Urology 85(1):27\u201332","journal-title":"Urology"},{"issue":"1","key":"3290_CR18","first-page":"159174","volume":"33","author":"G Landis JRKoch","year":"1977","unstructured":"Landis JRKoch G (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159174","journal-title":"Biometrics"},{"key":"3290_CR19","unstructured":"Neuro A Advanced Source Analysis (ASA) ANT Neuro: Hengelo, The Netherlands"},{"issue":"3","key":"3290_CR20","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1016\/j.ijpsycho.2015.06.001","volume":"97","author":"J Kayser","year":"2015","unstructured":"Kayser J, Tenke CE (2015) On the benefits of using surface Laplacian (current source density) methodology in electrophysiology. Int J psychophysiology: official J Int Organ Psychophysiol 97(3):171","journal-title":"Int J psychophysiology: official J Int Organ Psychophysiol"},{"issue":"1","key":"3290_CR21","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1016\/j.jneumeth.2007.06.026","volume":"166","author":"R Srinivasan","year":"2007","unstructured":"Srinivasan R et al (2007) EEG and MEG coherence: measures of functional connectivity at distinct spatial scales of neocortical dynamics. J Neurosci Methods 166(1):41\u201352","journal-title":"J Neurosci Methods"},{"key":"3290_CR22","unstructured":"Le Guennec A, Malinowski S, Tavenard R (2016) Data augmentation for time series classification using convolutional neural networks. In ECML\/PKDD workshop on advanced analytics and learning on temporal data"},{"key":"3290_CR23","doi-asserted-by":"crossref","unstructured":"Um TT et al (2017) Data augmentation of wearable sensor data for parkinson\u2019s disease monitoring using convolutional neural networks. In Proceedings of the 19th ACM international conference on multimodal interaction","DOI":"10.1145\/3136755.3136817"},{"issue":"8","key":"3290_CR24","doi-asserted-by":"publisher","first-page":"5618","DOI":"10.1007\/s00464-022-09023-1","volume":"36","author":"CF Justiniano","year":"2022","unstructured":"Justiniano CF et al (2022) Is robotic utilization associated with increased minimally invasive colorectal surgery rates? Surgeon-level evidence. Surg Endosc 36(8):5618\u20135626","journal-title":"Surg Endosc"},{"key":"3290_CR25","doi-asserted-by":"publisher","first-page":"674","DOI":"10.1016\/j.jss.2024.01.028","volume":"296","author":"R Caso","year":"2024","unstructured":"Caso R et al (2024) Comparing robotic, thoracoscopic, and open segmentectomy: a National Cancer Database analysis. J Surg Res 296:674\u2013680","journal-title":"J Surg Res"},{"key":"3290_CR26","unstructured":"Gao Y et al (2014) Jhu-isi gesture and skill assessment working set (jigsaws): A surgical activity dataset for human motion modeling. In MICCAI workshop: M2cai"},{"key":"3290_CR27","doi-asserted-by":"publisher","first-page":"1959","DOI":"10.1007\/s11548-018-1860-1","volume":"13","author":"Z Wang","year":"2018","unstructured":"Wang Z, Majewicz A, Fey (2018) Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery. Int J Comput Assist Radiol Surg 13:1959\u20131970","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"3290_CR28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.cmpb.2019.05.008","volume":"177","author":"XA Nguyen","year":"2019","unstructured":"Nguyen XA et al (2019) Surgical skill levels: Classification and analysis using deep neural network model and motion signals. Comput Methods Programs Biomed 177:1\u20138","journal-title":"Comput Methods Programs Biomed"},{"issue":"12","key":"3290_CR29","doi-asserted-by":"publisher","first-page":"1706","DOI":"10.3390\/brainsci13121706","volume":"13","author":"T Manabe","year":"2023","unstructured":"Manabe T et al (2023) Distinguishing laparoscopic surgery experts from novices using EEG topographic features. Brain Sci 13(12):1706","journal-title":"Brain Sci"},{"issue":"1","key":"3290_CR30","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1016\/j.jss.2014.04.032","volume":"191","author":"T Tien","year":"2014","unstructured":"Tien T et al (2014) Eye tracking for skills assessment and training: a systematic review. J Surg Res 191(1):169\u2013178","journal-title":"J Surg Res"},{"issue":"1","key":"3290_CR31","doi-asserted-by":"publisher","first-page":"14611","DOI":"10.1038\/s41598-024-65648-3","volume":"14","author":"SB Shafiei","year":"2024","unstructured":"Shafiei SB et al (2024) Prediction of Robotic Anastomosis Competency Evaluation (RACE) metrics during vesico-urethral anastomosis using electroencephalography, eye-tracking, and machine learning. Sci Rep 14(1):14611","journal-title":"Sci Rep"},{"issue":"2","key":"3290_CR32","doi-asserted-by":"publisher","first-page":"e292","DOI":"10.1097\/AS9.0000000000000292","volume":"4","author":"SB Shafiei","year":"2023","unstructured":"Shafiei SB et al (2023) Developing surgical skill level classification model using visual metrics and a gradient boosting algorithm. Annals Surg Open 4(2):e292","journal-title":"Annals Surg Open"},{"issue":"8","key":"3290_CR33","doi-asserted-by":"publisher","first-page":"1821","DOI":"10.1016\/j.clinph.2004.03.031","volume":"115","author":"P Luu","year":"2004","unstructured":"Luu P, Tucker DM, Makeig S (2004) Frontal midline theta and the error-related negativity: neurophysiological mechanisms of action regulation. Clin Neurophysiol 115(8):1821\u20131835","journal-title":"Clin Neurophysiol"},{"key":"3290_CR34","doi-asserted-by":"publisher","first-page":"110992","DOI":"10.1016\/j.brainresbull.2024.110992","volume":"214","author":"SB Shafiei","year":"2024","unstructured":"Shafiei SB, Shadpour S, Shafqat A (2024) Mental workload evaluation using weighted phase lag index and coherence features extracted from EEG data. Brain Res Bull 214:110992","journal-title":"Brain Res Bull"},{"issue":"7","key":"3290_CR35","doi-asserted-by":"publisher","first-page":"3561","DOI":"10.21037\/jtd-20-720","volume":"12","author":"BG Soliman","year":"2020","unstructured":"Soliman BG et al (2020) Impact of da Vinci Xi robot in pulmonary resection. J Thorac Disease 12(7):3561","journal-title":"J Thorac Disease"},{"key":"3290_CR36","unstructured":"Choi T-M et al (2025) SurgMLLMBench: A Multimodal Large Language Model Benchmark Dataset for Surgical Scene Understanding. arXiv preprint arXiv: https:\/\/arxiv.org\/abs\/2511.21339"}],"container-title":["Journal of Robotic Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11701-026-03290-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11701-026-03290-z","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11701-026-03290-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T19:42:05Z","timestamp":1781638925000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11701-026-03290-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,10]]},"references-count":36,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["3290"],"URL":"https:\/\/doi.org\/10.1007\/s11701-026-03290-z","relation":{},"ISSN":["1863-2491"],"issn-type":[{"value":"1863-2491","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,10]]},"assertion":[{"value":"25 November 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 February 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 March 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"This study was approved by the Institutional Review Board at Roswell Park Comprehensive Cancer Center (IRB Protocol I-241913). The IRB waived the requirement for written consent; participants received an information sheet and provided verbal consent.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}}],"article-number":"337"}}