{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T21:24:52Z","timestamp":1780694692141,"version":"3.54.1"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2020,1,24]],"date-time":"2020-01-24T00:00:00Z","timestamp":1579824000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,1,24]],"date-time":"2020-01-24T00:00:00Z","timestamp":1579824000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001831","name":"Delft University of Technology","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001831","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Med Syst"],"published-print":{"date-parts":[[2020,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Motion tracking software for assessing laparoscopic surgical proficiency has been proven to be effective in differentiating between expert and novice performances. However, with several indices that can be generated from the software, there is no set threshold that can be used to benchmark performances. The aim of this study was to identify the best possible algorithm that can be used to benchmark expert, intermediate and novice performances for objective evaluation of psychomotor skills. 12 video recordings of various surgeons were collected in a blinded fashion. Data from our previous study of 6 experts and 23 novices was also included in the analysis to determine thresholds for performance. Video recording were analyzed both by the Kinovea 0.8.15 software and a blinded expert observer using the CAT form. Multiple algorithms were tested to accurately identify expert and novice performances. \u00bd\u00a0L\u2009+\u2009<jats:inline-formula><jats:alternatives><jats:tex-math>$$ \\raisebox{1ex}{$1$}\\!\\left\/ \\!\\raisebox{-1ex}{$3$}\\right. $$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><mml:mfrac><mml:mn>1<\/mml:mn><mml:mn>3<\/mml:mn><\/mml:mfrac><\/mml:math><\/jats:alternatives><\/jats:inline-formula> A\u2009+\u2009<jats:inline-formula><jats:alternatives><jats:tex-math>$$ \\raisebox{1ex}{$1$}\\!\\left\/ \\!\\raisebox{-1ex}{$6$}\\right. $$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><mml:mfrac><mml:mn>1<\/mml:mn><mml:mn>6<\/mml:mn><\/mml:mfrac><\/mml:math><\/jats:alternatives><\/jats:inline-formula>\u00a0J scoring of path length, average movement and jerk index respectively resulted in identifying 23\/24 performances. Comparing the algorithm to CAT assessment yielded in a linear regression coefficient R<jats:sup>2<\/jats:sup> of 0.844. The value of motion tracking software in providing objective clinical evaluation and retrospective analysis is evident. Given the prospective use of this tool the algorithm developed in this study proves to be effective in benchmarking performances for psychomotor skills evaluation.<\/jats:p>","DOI":"10.1007\/s10916-020-1525-9","type":"journal-article","created":{"date-parts":[[2020,1,24]],"date-time":"2020-01-24T10:02:50Z","timestamp":1579860170000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Validation of Motion Tracking Software for Evaluation of Surgical Performance in Laparoscopic Cholecystectomy"],"prefix":"10.1007","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2253-0566","authenticated-orcid":false,"given":"Sandeep","family":"Ganni","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sanne M. B. I.","family":"Botden","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Magdalena","family":"Chmarra","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Meng","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard H. M.","family":"Goossens","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jack J.","family":"Jakimowicz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,1,24]]},"reference":[{"issue":"7422","key":"1525_CR1","first-page":"1032","volume":"327","author":"K Moorthy","year":"2003","unstructured":"Moorthy, K., Munz, Y., Sarker, S. K., and Darzi, A., Objective assessment of technical skills in surgery. Br J Surg 327(7422):1032\u20131037, 2003.","journal-title":"Br J Surg"},{"issue":"7","key":"1525_CR2","doi-asserted-by":"publisher","first-page":"972","DOI":"10.1002\/bjs.7115","volume":"97","author":"P. D. van Hove","year":"2010","unstructured":"van Hove, P. D., Tuijthof, G. J. M., Verdaasdonk, E. G. G., Stassen, L. P. S., and Dankelman, J., Objective assessment of technical surgical skills. Br J Surg, 2010. https:\/\/doi.org\/10.1002\/bjs.7115.","journal-title":"British Journal of Surgery"},{"issue":"1","key":"1525_CR3","doi-asserted-by":"publisher","first-page":"e81","DOI":"10.1016\/j.jss.2011.06.034","volume":"171","author":"Ignacio Oropesa","year":"2011","unstructured":"Oropesa et al., Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery. J Surg Res, 2011. https:\/\/doi.org\/10.1016\/j.jss.2011.06.034.","journal-title":"Journal of Surgical Research"},{"key":"1525_CR4","doi-asserted-by":"publisher","first-page":"1693","DOI":"10.1007\/s00464-008-0144-1","volume":"23","author":"SMBI Botden","year":"2008","unstructured":"Botden, S. M. B. I., and Jakimowicz, J. J., What is going on in augmented reality simulation in laparoscopic surgery? Surg Endosc 23:1693\u20131700, 2008. https:\/\/doi.org\/10.1007\/s00464-008-0144-1.","journal-title":"Surg Endosc"},{"key":"1525_CR5","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1007\/s00464-003-9043-7","volume":"18","author":"S Bann","year":"2004","unstructured":"Bann, S., Darzi, A., Munz, Y., Kumar, B. D., and Moorthy, K., Laparoscopic virtual reality and box trainers: Is one superior to the other? Surg Endosc 18:485\u2013494, 2004. https:\/\/doi.org\/10.1007\/s00464-003-9043-7.","journal-title":"Surg Endosc"},{"issue":"9","key":"1525_CR6","doi-asserted-by":"publisher","first-page":"1220","DOI":"10.1007\/s00464-004-2240-1","volume":"19","author":"MP Schijven","year":"2005","unstructured":"Schijven, M. P., Jakimowicz, J. J., Broeders, I. A. M. J., and Tseng, L. N. L., The Eindhoven laparoscopic cholecystectomy training course\u2014Improving operating room performance using virtual reality training: Results from the first E.a.E.S. accredited virtual reality trainings curriculum. Surg Endosc 19(9):1220\u20131226, 2005. https:\/\/doi.org\/10.1007\/s00464-004-2240-1.","journal-title":"Surg Endosc"},{"key":"1525_CR7","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1097\/00000658-200210000-00008","volume":"236","author":"NE Seymour","year":"2002","unstructured":"Seymour, N. E., Gallagher, A. G., Roman, S. A., O\u2019Brien, M. K., Bansal, V. K., Andersen, D. K., and Satava, R. M., Virtual reality training improves operating room performance. Ann Surg 236:458\u2013464, 2002. https:\/\/doi.org\/10.1097\/00000658-200210000-00008.","journal-title":"Ann Surg"},{"issue":"6","key":"1525_CR8","doi-asserted-by":"publisher","first-page":"2451","DOI":"10.1007\/s00464-016-5246-6","volume":"31","author":"S Ganni","year":"2017","unstructured":"Ganni, S., Chmarra, M. K., Goossens, R. H. M., and Jakimowicz, J. J., Self-assessment in laparoscopic surgical skills training: Is it reliable? Surg Endosc 31(6):2451\u20132456, 2017.","journal-title":"Surg Endosc"},{"issue":"1","key":"1525_CR9","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1136\/bmjstel-2018-000411","volume":"6","author":"Aimee K Gardner","year":"2019","unstructured":"Ak, G., and Adbelfattah, K., Getting better all the time? Facilitating accurate team self-assessments through simulation. BMJ Simulation and Technology Enhanced Learning, 2019. https:\/\/doi.org\/10.1136\/bmjstel-2018-000411.","journal-title":"BMJ Simulation and Technology Enhanced Learning"},{"issue":"2","key":"1525_CR10","doi-asserted-by":"publisher","first-page":"527","DOI":"10.1016\/j.jsurg.2017.07.030","volume":"75","author":"Sandeep Ganni","year":"2018","unstructured":"Ganni, S., Botden, S. M. B. I., Schaap, D. P. et al., \u201cReflection-before-practice\u201d improves self-assessment and end-performance in laparoscopic surgical skills training. Journal of Surgical Education, 2017. https:\/\/doi.org\/10.1016\/j.jsurg.2017.07.030.","journal-title":"Journal of Surgical Education"},{"issue":"3","key":"1525_CR11","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1007\/s004640090008","volume":"15","author":"T. P. Grantcharov","year":"2001","unstructured":"Grantcharov, T. P., Rosenberg, J., Pahle, E., and Funch-Jensen, E., Virtual reality computer simulation - an objective method for the evaluation of laparoscopic skills. Surg Endosc, 2001. https:\/\/doi.org\/10.1007\/s004640090008.","journal-title":"Surgical Endoscopy"},{"issue":"6","key":"1525_CR12","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1109\/MCG.2006.125","volume":"26","author":"P Lamata","year":"2006","unstructured":"Lamata, P., Gomez, E. J., Bello, F. et al., Conceptual framework for laparoscopic VR simulators. IEEE Comput Graph Appl 26(6):69\u201379, 2006.","journal-title":"IEEE Comput Graph Appl"},{"key":"1525_CR13","doi-asserted-by":"publisher","first-page":"771","DOI":"10.1007\/s00464-010-1251-3","volume":"25","author":"S Yamaguchi","year":"2010","unstructured":"Yamaguchi, S., Yoshida, D., Kenmotsu, H., Yasunaga, T., Konishi, K., Ieiri, S., Nakashima, H., Tanoue, K., and Hashizume, M., Objective assessment of laparoscopic suturing skills using a motion-tracking system. Surg Endosc 25:771\u2013775, 2010. https:\/\/doi.org\/10.1007\/s00464-010-1251-3.","journal-title":"Surg Endosc"},{"key":"1525_CR14","doi-asserted-by":"publisher","first-page":"299","DOI":"10.1177\/1553350612459808","volume":"20","author":"I Oropesa","year":"2013","unstructured":"Oropesa, I., Chmarra, M. K., S\u00e1nchez-Gonz\u00e1lez, P., Lamata, P., Rodrigues, S. P., Enciso, S., S\u00e1nchez-Margallo, F. M., Jansen, F.-W., Dankelman, J., and G\u00f3mez, E. J., Relevance of motion-related assessment metrics in laparoscopic surgery. Surg Innov 20:299\u2013312, 2013. https:\/\/doi.org\/10.1177\/1553350612459808.","journal-title":"Surg Innov"},{"issue":"3","key":"1525_CR15","doi-asserted-by":"publisher","first-page":"854","DOI":"10.1007\/s00464-012-2524-9","volume":"27","author":"EF Hofstad","year":"2012","unstructured":"Hofstad, E. F., V\u00e5penstad, C., Chmarra, M. K., Lang\u00f8, T., Kuhry, E., and M\u00e5rvik, R., A study of psychomotor skills in minimally invasive surgery: What differentiates expert and nonexpert performance. Surg Endosc 27(3):854\u2013863, 2012. https:\/\/doi.org\/10.1007\/s00464-012-2524-9.","journal-title":"Surg Endosc"},{"issue":"2","key":"1525_CR16","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1016\/j.jsurg.2016.10.006","volume":"74","author":"A Ghasemloonia","year":"2017","unstructured":"Ghasemloonia, A., Maddahi, Y., Zareinia, K., Lama, S., Dort, J. C., and Sutherland, G. R., Surgical skill assessment using motion quality and smoothness. Journal of Surgical Education 74(2):295\u2013305, 2017. https:\/\/doi.org\/10.1016\/j.jsurg.2016.10.006.","journal-title":"Journal of Surgical Education"},{"issue":"6","key":"1525_CR17","doi-asserted-by":"publisher","first-page":"2994","DOI":"10.1007\/s00464-018-6023-5","volume":"32","author":"Sandeep Ganni","year":"2018","unstructured":"Ganni, S., Botden, S. M. B. I., Chmarra, M. K., Goossens, R. H. M., and Jakimowicz, J. J., A software-based tool for video motion tracking in the surgical skills assessment landscape. Surg Endosc, 2018. https:\/\/doi.org\/10.1007\/s00464-018-6023-5.","journal-title":"Surgical Endoscopy"},{"issue":"11","key":"1525_CR18","doi-asserted-by":"publisher","first-page":"3732","DOI":"10.1007\/s00464-019-06667-4","volume":"33","author":"Karl-Friedrich Kowalewski","year":"2019","unstructured":"Kowalewski, K. F., Garrow, C. R., Schmidt, M. W., Benner, L., Muller, B. P., and Nickel, F., Sensor-based machine learning for workflow detection and as key to detect expert level in laparoscopic suturing and knot-tying. Surg Endosc, 2019. https:\/\/doi.org\/10.1007\/s00464-019-06667-4.","journal-title":"Surgical Endoscopy"},{"key":"1525_CR19","doi-asserted-by":"publisher","unstructured":"Baghdadi, A., Hussein, A. A., Ahmed, Y., Cavuoto, L. A., and Guru, K. A., A computer vision technique for automated assessment of surgical performance using surgeon console-feed videos. Int J Comput Assist Radiol Surg, 2018. https:\/\/doi.org\/10.1007\/s11548-1881-9.","DOI":"10.1007\/s11548-1881-9"},{"key":"1525_CR20","doi-asserted-by":"publisher","first-page":"476","DOI":"10.1097\/SLA.0b013e318275b72a","volume":"257","author":"D Miskovic","year":"2013","unstructured":"Miskovic, D., Ni, M., Wyles, S. M., Kennedy, R. H., Francis, N. K., Parvaiz, A., Cunningham, C., Rockall, T. A., Gudgeon, A. M., Coleman, M. G., and Hanna, G. B., Is competency assessment at the specialist level achievable? A study for the national training program in laparoscopic colorectal surgery in England. Ann Surg 257:476\u2013482, 2013.","journal-title":"Ann Surg"},{"issue":"1","key":"1525_CR21","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/j.amjsurg.2005.04.004","volume":"190","author":"Melina C. Vassiliou","year":"2005","unstructured":"Vassilou, M. C., Feldman, L. S., Andrew, C. G., Bergman, S., Leffondre, K., Stanbridge, D., and Fried, G. M., A global assessment tool for evaluation of intraoperative laparoscopic skills. The Americal Journal of Surgery, 2004. https:\/\/doi.org\/10.1016\/j.amjsurg.2005.04.004.","journal-title":"The American Journal of Surgery"},{"key":"1525_CR22","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1002\/bjs.1800840237","volume":"84","author":"JA Martin","year":"1997","unstructured":"Martin, J. A., Regehr, G., Reznick, R. et al., Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:243\u2013278, 1997.","journal-title":"Br J Surg"},{"key":"1525_CR23","doi-asserted-by":"publisher","first-page":"356","DOI":"10.1007\/s00464-010-1190-z","volume":"25","author":"CE Reiley","year":"2011","unstructured":"Reiley, C. E., Lin, H. C., Yuh, D. D. et al., Review of methods for objective surgical skill evaluation. Surg Endosc 25:356, 2011. https:\/\/doi.org\/10.1007\/s00464-010-1190-z.","journal-title":"Surg Endosc"},{"issue":"6","key":"1525_CR24","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1080\/13645700701702135","volume":"16","author":"MK Chmarra","year":"2007","unstructured":"Chmarra, M. K., Grimbergen, C. A., and Dankelman, J., Systems for tracking minimally invasive surgical instruments. Min Inv Ther All Tech 16(6):328\u2013340, 2007.","journal-title":"Min Inv Ther All Tech"},{"issue":"4","key":"1525_CR25","doi-asserted-by":"publisher","first-page":"S171","DOI":"10.1016\/j.jamcollsurg.2017.07.387","volume":"225","author":"Daniel A. Hashimoto","year":"2017","unstructured":"Hashimoto, D. A., Rosman, G., Volkov, M., Rus, D. L., and Meireles, O. R., Artificial intelligence for intraoperative video analysis: Machine Learning\u2019s role in surgical education. J Am Coll Surg, 2017. https:\/\/doi.org\/10.1016\/j.jamcollsurg.2017.07.387.","journal-title":"Journal of the American College of Surgeons"}],"container-title":["Journal of Medical Systems"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-020-1525-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10916-020-1525-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-020-1525-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,1,23]],"date-time":"2021-01-23T01:03:56Z","timestamp":1611363836000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10916-020-1525-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,24]]},"references-count":25,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,3]]}},"alternative-id":["1525"],"URL":"https:\/\/doi.org\/10.1007\/s10916-020-1525-9","relation":{},"ISSN":["0148-5598","1573-689X"],"issn-type":[{"value":"0148-5598","type":"print"},{"value":"1573-689X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1,24]]},"assertion":[{"value":"16 October 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 January 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 January 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with Ethical Standards"}},{"value":"Sandeep Ganni, Sanne MBI Botden, Magdalena K. Chmarra, Meng Li, Richard HM Goossens and Jack J. Jakimowicz have no conflicts of interest or financial ties to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Disclosures"}}],"article-number":"56"}}