{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T22:28:14Z","timestamp":1793140094062,"version":"4.5.0"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T00:00:00Z","timestamp":1773446400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T00:00:00Z","timestamp":1776988800000},"content-version":"vor","delay-in-days":41,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001734","name":"Copenhagen University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001734","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Educ"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Competency-based medical education requires structured training and valid methods for assessing technical proficiency. Virtual reality simulators offer objective, reproducible assessments but often rely on developer-defined metrics lacking transparency and robust validity evidence. This study aimed to collect validity evidence, guided by Messick\u2019s unified validity framework, for a novel, expert-informed, immersive virtual reality simulator designed to assess proficiency among orthopedic physicians for volar plate fixation of distal radius fractures.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Twelve orthopedic residents (novices) and eleven experienced orthopedic surgeons completed one distal radius fracture fixation in the simulator. Performance was evaluated using 32 simulator metrics addressing technical proficiency, imaging accuracy, and procedural errors. Simulator metrics were compared between the two groups individually (secondary outcomes) and combined into a composite total simulator score (primary outcome). Validity evidence was structured according to Messick\u2019s framework. The composite score\u2019s discriminatory ability was assessed using the Contrasting Groups\u2019 Method.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Experienced surgeons significantly outperformed novices on nine of 32 simulator metrics, while the novices scored higher on one. After excluding this construct-irrelevant metric, the remaining 31 metrics demonstrated acceptable reliability (Cronbach\u2019s alpha\u2009=\u20090.79). The total simulator score was significantly different for the two groups: novices scored\u2009\u2212\u20091.6 points (CI -12.3 to 9.2), and experienced surgeons scored 61.9 points (CI 53.6 to 70.3),\n                      <jats:italic>p<\/jats:italic>\n                      &lt; .001. A discriminatory standard for the total simulator score of 34.0 points completely discriminated between novices and experienced surgeons.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>This study provides comprehensive validity evidence \u2013 covering all five sources in Messick\u2019s framework \u2013 for a newly developed, expert-informed, immersive virtual reality simulator for distal radius fracture fixation. The simulator-based assessment reliably discriminated between novices and experienced orthopedic surgeons. These findings support the simulator\u2019s utility for objective skills assessment in distal radius fracture fixation within orthopedic residency training.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12909-026-08983-5","type":"journal-article","created":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T04:55:03Z","timestamp":1773464103000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Assessment of surgical skills on a novel expert-informed immersive virtual reality simulator for distal radius fracture fixation: a cross-sectional validation study"],"prefix":"10.1186","volume":"26","author":[{"given":"Mads Emil","family":"Jacobsen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Monica","family":"Ghidinelli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chitra","family":"Subramaniam","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristoffer Borbjerg","family":"Hare","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leizl Joy","family":"Nayahangan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lars","family":"Konge","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amandus","family":"Gustafsson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,3,14]]},"reference":[{"key":"8983_CR1","doi-asserted-by":"publisher","first-page":"1320","DOI":"10.1302\/0301-620X.98B10.37031","volume":"98\u2013B","author":"MT Nousiainen","year":"2016","unstructured":"Nousiainen MT, McQueen SA, Hall J, Kraemer W, Ferguson P, Marsh JL, et al. Resident education in orthopaedic trauma: the future role of competency-based medical education. Bone Joint J. 2016;98\u2013B:1320\u20135.","journal-title":"Bone Joint J"},{"key":"8983_CR2","doi-asserted-by":"publisher","first-page":"668","DOI":"10.1080\/0142159X.2018.1472754","volume":"40","author":"F Bjerrum","year":"2018","unstructured":"Bjerrum F, Thomsen ASS, Nayahangan LJ, Konge L. Surgical simulation: current practices and future perspectives for technical skills training. Med Teach. 2018;40:668\u201375. https:\/\/doi.org\/10.1080\/0142159X.2018.1472754.","journal-title":"Med Teach"},{"key":"8983_CR3","doi-asserted-by":"publisher","first-page":"126","DOI":"10.1186\/s13018-014-0126-z","volume":"9","author":"ERB Stirling","year":"2014","unstructured":"Stirling ERB, Lewis TL, Ferran NA. Surgical skills simulation in trauma and orthopaedic training. J Orthop Surg Res. 2014;9:126.","journal-title":"J Orthop Surg Res"},{"key":"8983_CR4","doi-asserted-by":"publisher","unstructured":"Lohre R, Bois AJ, Athwal GS, Goel DP. Improved complex skill acquisition by immersive virtual reality training: a randomized controlled trial. J Bone Joint Surg Am. 2020;102. https:\/\/doi.org\/10.2106\/JBJS.19.00982.","DOI":"10.2106\/JBJS.19.00982"},{"key":"8983_CR5","doi-asserted-by":"publisher","first-page":"2170","DOI":"10.1097\/CORR.0000000000001362","volume":"478","author":"MD Orland","year":"2020","unstructured":"Orland MD, Patetta MJ, Wieser M, Kayupov E, Gonzalez MH. Does virtual reality improve procedural completion and accuracy in an intramedullary tibial nail procedure? A randomized control trial. Clin Orthop Relat Res. 2020;478:2170\u20137. https:\/\/doi.org\/10.1097\/CORR.0000000000001362.","journal-title":"Clin Orthop Relat Res"},{"key":"8983_CR6","doi-asserted-by":"publisher","first-page":"807","DOI":"10.1097\/CORR.0000000000002064","volume":"480","author":"KR Vaghela","year":"2022","unstructured":"Vaghela KR, Trockels A, Lee J, Akhtar K. Is the virtual reality fundamentals of arthroscopic surgery training program a valid platform for resident arthroscopy training? Clin Orthop Relat Res. 2022;480:807\u201315. https:\/\/doi.org\/10.1097\/CORR.0000000000002064.","journal-title":"Clin Orthop Relat Res"},{"key":"8983_CR7","doi-asserted-by":"publisher","unstructured":"Li Y, Chan R, Menon MRG, Ryan JF, Mador B, Campbell SM, et al. Validity Evidence for procedure-specific competence assessment tools in orthopaedic surgery: A scoping review. J Am Acad Orthop Surg Glob Res Rev. 2024;8. https:\/\/doi.org\/10.5435\/JAAOSGLOBAL-D-23-00065.","DOI":"10.5435\/JAAOSGLOBAL-D-23-00065"},{"key":"8983_CR8","doi-asserted-by":"publisher","first-page":"e580","DOI":"10.1016\/S2666-7568(21)00172-0","volume":"2","author":"A-M Wu","year":"2021","unstructured":"Wu A-M, Bisignano C, James SL, Abady GG, Abedi A, Abu-Gharbieh E, et al. Global, regional, and national burden of bone fractures in 204 countries and territories, 1990\u20132019: a systematic analysis from the Global Burden of Disease Study 2019. Lancet Heal Longev. 2021;2:e580\u201392.","journal-title":"Lancet Heal Longev"},{"key":"8983_CR9","doi-asserted-by":"publisher","unstructured":"Watson A, Leroux T, Ogilvie-Harris D, Nousiainen M, Ferguson PC, Murnahan L, Dwyer T. Entrustable Professional Activities in Orthopaedics. JB JS Open Access. 2021;6(2):e20.00010. https:\/\/doi.org\/10.2106\/JBJS.OA.20.00010.","DOI":"10.2106\/JBJS.OA.20.00010"},{"key":"8983_CR10","doi-asserted-by":"publisher","first-page":"875","DOI":"10.1016\/j.jhsa.2023.05.012","volume":"48","author":"ME Jacobsen","year":"2023","unstructured":"Jacobsen ME, Nayahangan LJ, Ghidinelli M, Subramaniam C, Hare KB, Konge L, et al. Assessment of technical competence in distal radius fracture fixation by a volar locking plate: a global delphi consensus study. J Hand Surg Am. 2023;48:875\u201385.","journal-title":"J Hand Surg Am"},{"key":"8983_CR11","unstructured":"American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington (DC): American Educational Research Association; 2014."},{"key":"8983_CR12","doi-asserted-by":"publisher","first-page":"1063","DOI":"10.1097\/SLA.0000000000002652","volume":"267","author":"NJ Borgersen","year":"2018","unstructured":"Borgersen NJ, Naur TMH, S\u00f8rensen SMD, Bjerrum F, Konge L, Subhi Y, et al. Gathering validity evidence for surgical simulation: a systematic review. Ann Surg. 2018;267:1063\u20138. https:\/\/doi.org\/10.1097\/SLA.0000000000002652.","journal-title":"Ann Surg"},{"key":"8983_CR13","doi-asserted-by":"publisher","first-page":"960","DOI":"10.3109\/0142159X.2012.703791","volume":"34","author":"R Bloch","year":"2012","unstructured":"Bloch R, Norman G. Generalizability theory for the perplexed: a practical introduction and guide: AMEE Guide 68. Med Teach. 2012;34:960\u201392.","journal-title":"Med Teach"},{"key":"8983_CR14","doi-asserted-by":"crossref","unstructured":"Yudkowsky R, Park Y, Downing S, (editors). Assessment in Health Professions Education. Second Edition. Routledge; 2020.","DOI":"10.4324\/9781315166902"},{"key":"8983_CR15","doi-asserted-by":"publisher","unstructured":"Lohre R, Bois AJ, Pollock JW, Lapner P, McIlquham K, Athwal GS, et al. Effectiveness of immersive virtual reality on orthopedic surgical skills and knowledge acquisition among senior surgical residents: A randomized clinical trial. JAMA Netw Open. 2020;3. https:\/\/doi.org\/10.1001\/JAMANETWORKOPEN.2020.31217.","DOI":"10.1001\/JAMANETWORKOPEN.2020.31217"},{"key":"8983_CR16","doi-asserted-by":"publisher","first-page":"477","DOI":"10.2340\/17453674.2024.41345","volume":"95","author":"M S\u00f8nderup","year":"2024","unstructured":"S\u00f8nderup M, Gustafsson A, Konge L, Jacobsen ME. Intraoperative fluoroscopy skills in distal radius fracture surgery: valid and reliable assessment on a novel immersive virtual reality simulator. Acta Orthop. 2024;95:477\u201384. https:\/\/doi.org\/10.2340\/17453674.2024.41345.","journal-title":"Acta Orthop"},{"key":"8983_CR17","doi-asserted-by":"publisher","unstructured":"Jacobsen ME, Andersen MJ, Hansen CO, Konge L. Testing basic competency in knee arthroscopy using a virtual reality simulator exploring validity and reliability. J Bone Joint Surg Am. 2015;97(9):775-81. https:\/\/doi.org\/10.2106\/JBJS.N.00747.","DOI":"10.2106\/JBJS.N.00747"},{"key":"8983_CR18","doi-asserted-by":"publisher","unstructured":"Jacobsen ME, Gustafsson A, J\u00f8rgensen PG, Park YS, Konge L. Practicing procedural skills is more effective than basic psychomotor training in knee arthroscopy: a randomized study. Orthop J Sports Med. 2021;9(2):2325967120985129. https:\/\/doi.org\/10.1177\/2325967120985129.","DOI":"10.1177\/2325967120985129"},{"key":"8983_CR19","doi-asserted-by":"publisher","first-page":"403","DOI":"10.3109\/17453674.2014.917502","volume":"85","author":"P Pedersen","year":"2014","unstructured":"Pedersen P, Palm H, Ringsted C, Konge L. Virtual-reality simulation to assess performance in hip fracture surgery. Acta Orthop. 2014;85:403\u20137.","journal-title":"Acta Orthop"},{"key":"8983_CR20","unstructured":"Thomas PA, Kern DE, Hughes MT, Chen BY. Curriculum development for medical education: a six-step approach. 3rd edition. Baltimore (MD): Johns Hopkins University Press; 2016."},{"key":"8983_CR21","doi-asserted-by":"publisher","first-page":"E108","DOI":"10.2106\/JBJS.18.01122","volume":"101","author":"A Gustafsson","year":"2019","unstructured":"Gustafsson A, Viberg B, Paltved C, Palm H, Konge L, Nayahangan LJ. Identifying technical procedures in orthopaedic surgery and traumatology that should be integrated in a simulation-based curriculum: a national general needs assessment in Denmark. J Bone Joint Surg Am. 2019;101:E108.","journal-title":"J Bone Joint Surg Am"},{"key":"8983_CR22","doi-asserted-by":"publisher","first-page":"648","DOI":"10.1111\/J.1365-2923.2012.04268.X","volume":"46","author":"R Brydges","year":"2012","unstructured":"Brydges R, Nair P, Ma I, Shanks D, Hatala R. Directed self-regulated learning versus instructor-regulated learning in simulation training. Med Educ. 2012;46:648\u201356. https:\/\/doi.org\/10.1111\/J.1365-2923.2012.04268.X.","journal-title":"Med Educ"},{"key":"8983_CR23","doi-asserted-by":"publisher","first-page":"79","DOI":"10.2106\/JBJS.20.01553","volume":"104","author":"TG Myers","year":"2022","unstructured":"Myers TG, Marsh JL, Nicandri G, Gorczyca J, Pellegrini VD. Contemporary issues in the acquisition of orthopaedic surgical skills during residency: competency-based medical education and simulation. J Bone Jt Surg. 2022;104:79\u201391. https:\/\/doi.org\/10.2106\/JBJS.20.01553.","journal-title":"J Bone Jt Surg"},{"key":"8983_CR24","doi-asserted-by":"publisher","first-page":"e1843","DOI":"10.1016\/J.ASMR.2022.05.009","volume":"4","author":"DP Berthold","year":"2022","unstructured":"Berthold DP, Muench LN, Rupp MC, Siebenlist S, Cote MP, Mazzocca AD, et al. Head-mounted display virtual reality is effective in orthopaedic training: a systematic review. Arthrosc Sport Med Rehabil. 2022;4:e1843\u20139. https:\/\/doi.org\/10.1016\/J.ASMR.2022.05.009.","journal-title":"Arthrosc Sport Med Rehabil"},{"key":"8983_CR25","doi-asserted-by":"publisher","unstructured":"Crockatt WK, Confino JE, Kopydlowski NJ, Jobin CM, Levine WN. Comparing skill acquisition and validity of immersive virtual reality with cadaver laboratory sessions in training for reverse total shoulder arthroplasty. JB JS Open Access 2023;8. https:\/\/doi.org\/10.2106\/JBJS.OA.22.00141.","DOI":"10.2106\/JBJS.OA.22.00141"},{"key":"8983_CR26","doi-asserted-by":"publisher","first-page":"1715","DOI":"10.1016\/J.INJURY.2021.04.045","volume":"52","author":"A Feeley","year":"2021","unstructured":"Feeley A, Feeley I, Merghani K, Sheehan E. A pilot study to evaluate the face & construct validity of an orthopaedic virtual reality simulator. Injury. 2021;52:1715\u201320. https:\/\/doi.org\/10.1016\/J.INJURY.2021.04.045.","journal-title":"Injury"},{"key":"8983_CR27","doi-asserted-by":"publisher","first-page":"1585","DOI":"10.1302\/0301-620X.101B12.BJJ-2019-0643.R1","volume":"101\u2013B","author":"K Logishetty","year":"2019","unstructured":"Logishetty K, Rudran B, Cobb JP. Virtual reality training improves trainee performance in total hip arthroplasty: a randomized controlled trial. Bone Joint J. 2019;101\u2013B:1585\u201392. https:\/\/doi.org\/10.1302\/0301-620X.101B12.BJJ-2019-0643.R1.","journal-title":"Bone Joint J"},{"key":"8983_CR28","doi-asserted-by":"publisher","first-page":"872","DOI":"10.1097\/SLA.0000000000005209","volume":"275","author":"PI Pietersen","year":"2022","unstructured":"Pietersen PI, Bjerrum F, Tolsgaard MG, Konge L, Andersen SAW. Standard setting in simulation-based training of surgical procedures: a systematic review. Ann Surg. 2022;275:872\u201382. https:\/\/doi.org\/10.1097\/SLA.0000000000005209.","journal-title":"Ann Surg"},{"key":"8983_CR29","doi-asserted-by":"publisher","first-page":"783","DOI":"10.3109\/0142159X.2011.611022","volume":"33","author":"LWT Schuwirth","year":"2011","unstructured":"Schuwirth LWT, Van Der Vleuten CPM. General overview of the theories used in assessment: AMEE Guide 57. Med Teach. 2011;33:783\u201397. https:\/\/doi.org\/10.3109\/0142159X.2011.611022.","journal-title":"Med Teach"},{"key":"8983_CR30","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1097\/SLA.0000000000004650","volume":"274","author":"E Mazzone","year":"2021","unstructured":"Mazzone E, Puliatti S, Amato M, Bunting B, Rocco B, Montorsi F, et al. A systematic review and meta-analysis on the impact of proficiency-based progression simulation training on performance outcomes. Ann Surg. 2021;274:281\u20139. https:\/\/doi.org\/10.1097\/SLA.0000000000004650.","journal-title":"Ann Surg"},{"key":"8983_CR31","doi-asserted-by":"publisher","first-page":"2178","DOI":"10.1097\/CORR.0000000000001432","volume":"478","author":"BB Gilmer","year":"2020","unstructured":"Gilmer BB. CORR insights\u00ae: does virtual reality improve procedural completion and accuracy in an intramedullary tibial nail procedure? A randomized control trial. Clin Orthop Relat Res. 2020;478:2178\u20139. https:\/\/doi.org\/10.1097\/CORR.0000000000001432.","journal-title":"Clin Orthop Relat Res"},{"key":"8983_CR32","doi-asserted-by":"publisher","first-page":"348","DOI":"10.1080\/17453674.2019.1607111","volume":"90","author":"A Gustafsson","year":"2019","unstructured":"Gustafsson A, Pedersen P, R\u00f8mer TB, Viberg B, Palm H, Konge L. Hip-fracture osteosynthesis training: exploring learning curves and setting proficiency standards. Acta Orthop. 2019;90:348\u201353.","journal-title":"Acta Orthop"},{"key":"8983_CR33","doi-asserted-by":"publisher","first-page":"275","DOI":"10.2340\/17453674.2024.40812","volume":"95","author":"A Gustafsson","year":"2024","unstructured":"Gustafsson A, R\u00f6lfing JD, Palm H, Viberg B, Grimstrup S, Konge L. Setting proficiency standards for simulation-based mastery learning of short antegrade femoral nail osteosynthesis: a multicenter study. Acta Orthop. 2024;95:275\u201381. https:\/\/doi.org\/10.2340\/17453674.2024.40812.","journal-title":"Acta Orthop"}],"container-title":["BMC Medical Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12909-026-08983-5","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12909-026-08983-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12909-026-08983-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T20:04:11Z","timestamp":1781640251000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1186\/s12909-026-08983-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,14]]},"references-count":33,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["8983"],"URL":"https:\/\/doi.org\/10.1186\/s12909-026-08983-5","relation":{},"ISSN":["1472-6920"],"issn-type":[{"value":"1472-6920","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,14]]},"assertion":[{"value":"7 August 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 March 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 March 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The study was performed in accordance with the declaration of Helsinki. The study protocol was submitted to the regional ethics committee of the Capital Region of Denmark, which deemed the study exempt from ethical approval, as it was an educational study that did not involve any patients (F-23046473). All participants received oral and written information about the study and provided written informed consent prior to participation. Clinical trial number: not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Written informed consent for publication of the photograph (Fig.\u00a01 A) was obtained from the individual depicted. A copy of the signed consent form is available for review by the Editor if requested.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"664"}}