{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T19:00:09Z","timestamp":1790103609454,"version":"4.0.1"},"reference-count":19,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"3","license":[{"start":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T00:00:00Z","timestamp":1782864000000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"abstract":"<jats:sec>\n                    <jats:title>Background:<\/jats:title>\n                    <jats:p>Identifying orthopaedic surgery residents at risk of future professional or practice-related difficulties remains challenging. While knowledge-based assessments such as the Orthopaedic In-Training Examination (OITE) are widely used, they incompletely capture behavioral and professionalism concerns that often underlie significant problems during residency and subsequent practice.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods:<\/jats:title>\n                    <jats:p>We conducted a retrospective, single-institution cohort study of 119 orthopaedic surgery residents who entered training between 1978 and 2013. Institutional resident files were reviewed for documented performance problems in academic, clinical, and professionalism domains, OITE scores, and American Board of Orthopaedic Surgery (ABOS) Part I and Part II examination outcomes. Adverse postresidency indicators were assessed using publicly available data, including board examination failure, licensure status, practice continuity, and persistent negative online presence. Receiver operating characteristic (ROC) curve analysis was used to evaluate the predictive utility of cumulative documented performance problems and OITE scores.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>Forty-two residents (35%) had at least 1 documented performance problem during residency (mean 4.98, range 1-13). Residents with documented problems were significantly more likely to fail ABOS Part I (33% vs. 8%; relative risk [RR] 4.28, p = 0.002) and Part II (24% vs. 4%; RR 6.11, p = 0.004) and to demonstrate persistent negative online presence (19% vs. 5%; RR 3.67, p = 0.026). Having more than 1 adverse postresidency indicator was also more common among residents with documented problems (31% vs. 6%; RR 4.77, p = 0.001). On ROC analysis, 4 or more documented problems predicted failure on ABOS Part I (AUC 0.787) and the presence of adverse postresidency indicators (AUC 0.772), while 3 or more problems best predicted ABOS Part II failure (AUC 0.784). OITE scores demonstrated lower predictive accuracy (AUC 0.696) and lacked a clearly actionable threshold.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions:<\/jats:title>\n                    <jats:p>Cumulative documented performance problems during residency\u2014particularly behavioral and professionalism-related concerns\u2014are associated with increased risk of board examination failure and adverse practice outcomes after graduation. Aggregated longitudinal documentation may provide an early, complementary signal of risk beyond knowledge-based testing.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2106\/jbjs.oa.25.00119","type":"journal-article","created":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T18:00:27Z","timestamp":1790100027000},"update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":0,"title":["Reading the Writing on the Wall"],"prefix":"10.2106","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3439-0089","authenticated-orcid":false,"given":"Austin M.","family":"Beason","sequence":"first","affiliation":[{"name":"Division of Orthopaedic Surgery, Southern Illinois University School of Medicine, Springfield, Illinois"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1880-2859","authenticated-orcid":false,"given":"Nicole K.","family":"Roberts","sequence":"additional","affiliation":[{"name":"Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5870-0718","authenticated-orcid":false,"given":"Youssef","family":"El Bitar","sequence":"additional","affiliation":[{"name":"Division of Orthopaedic Surgery, Southern Illinois University School of Medicine, Springfield, Illinois"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Margaret L.","family":"Boehler","sequence":"additional","affiliation":[{"name":"Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-1015-7602","authenticated-orcid":false,"given":"Cathy J.","family":"Schwind","sequence":"additional","affiliation":[{"name":"Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"D. Gordon","family":"Allan","sequence":"additional","affiliation":[{"name":"Division of Orthopaedic Surgery, Southern Illinois University School of Medicine, Springfield, Illinois"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","published-online":{"date-parts":[[2026,7]]},"reference":[{"issue":"2","key":"R1-20260922","first-page":"e21.00150","article-title":"American board of orthopaedic surgery's initiatives toward competency-based education","volume":"7","author":"Van Heest","year":"2022","journal-title":"JB JS Open Access"},{"issue":"13","key":"R2-20260922","doi-asserted-by":"crossref","first-page":"660","DOI":"10.5435\/JAAOS-D-22-01195","article-title":"Update on educational resources and evaluation tools for orthopaedic surgery residents","volume":"31","author":"Cohn","year":"2023","journal-title":"J Am Acad Orthop Surg."},{"issue":"9","key":"R3-20260922","doi-asserted-by":"crossref","first-page":"1416","DOI":"10.2106\/00004623-200109000-00019","article-title":"The education of future orthopaedists--d\u00e8j\u00e1 vu","volume":"83","author":"Simon","year":"2001","journal-title":"The J Bone Jt Surg Am Vol"},{"issue":"4","key":"R4-20260922","first-page":"e24.00102","article-title":"Professional behavior assessment during residency training: can we identify outliers?","volume":"9","author":"Van Heest","year":"2024","journal-title":"JB JS Open Access"},{"issue":"3","key":"R5-20260922","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1097\/00001888-200403000-00011","article-title":"Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board","volume":"79","author":"Papadakis","year":"2004","journal-title":"Acad Med."},{"issue":"Suppl","key":"R6-20260922","doi-asserted-by":"crossref","first-page":"S17","DOI":"10.1097\/00001888-200510001-00008","article-title":"Domains of unprofessional behavior during medical school associated with future disciplinary action by a state medical board","volume":"80","author":"Teherani","year":"2005","journal-title":"Acad Med."},{"issue":"5","key":"R7-20260922","doi-asserted-by":"crossref","first-page":"e18","DOI":"10.2106\/JBJS.17.01512","article-title":"Resident surgical skills web-based evaluation: a comparison of 2 assessment tools","volume":"101","author":"Van Heest","year":"2019","journal-title":"J Bone Jt Surg."},{"issue":"4","key":"R9-20260922","doi-asserted-by":"crossref","first-page":"e20.00103","DOI":"10.2106\/JBJS.OA.20.00103","article-title":"Use of the behavior assessment tool in 18 pilot residency programs","volume":"5","author":"Armstrong","year":"2020","journal-title":"JB JS Open Access"},{"issue":"6","key":"R10-20260922","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1016\/j.surg.2009.01.019","article-title":"The nature of general surgery resident performance problems","volume":"145","author":"Williams","year":"2009","journal-title":"Surgery"},{"issue":"5","key":"R11-20260922","doi-asserted-by":"crossref","first-page":"1092","DOI":"10.2106\/00004623-200405000-00032","article-title":"Passing the boards: can USMLE and orthopaedic in-Training examination scores predict passage of the ABOS Part-I examination?","volume":"86","author":"Klein","year":"2004","journal-title":"J Bone Jt Surg."},{"issue":"24","key":"R12-20260922","doi-asserted-by":"crossref","first-page":"e1370","DOI":"10.5435\/JAAOS-D-20-01019","article-title":"Do orthopaedic In-Training examination scores predict the likelihood of passing the American board of orthopaedic surgery part I examination? An update with 2014 to 2018 data","volume":"29","author":"Fritz","year":"2021","journal-title":"J Am Acad Orthop Surg."},{"issue":"10","key":"R13-20260922","doi-asserted-by":"crossref","first-page":"542","DOI":"10.5435\/JAAOS-D-24-00544","article-title":"Assessing the linking between the orthopaedic In-Training examination and the American board of orthopaedic surgery part I certification examination","volume":"33","author":"Fritz","year":"2025","journal-title":"J Am Acad Orthop Surg."},{"issue":"9","key":"R14-20260922","doi-asserted-by":"crossref","first-page":"2436","DOI":"10.1007\/s11999-009-0939-y","article-title":"Predictors of success on the American Board of Orthopaedic Surgery examination","volume":"467","author":"Herndon","year":"2009","journal-title":"Clin Orthop Relat Res."},{"issue":"10","key":"R15-20260922","doi-asserted-by":"crossref","first-page":"2797","DOI":"10.1007\/s11999-010-1327-3","article-title":"Do scores of the USMLE step 1 and OITE correlate with the ABOS part I certifying examination?: a multicenter study","volume":"468","author":"Dougherty","year":"2010","journal-title":"Clin Orthop Relat Res."},{"issue":"12","key":"R16-20260922","doi-asserted-by":"crossref","first-page":"e84","DOI":"10.2106\/JBJS.L.00457","article-title":"Utility of AAOS OITE scores in predicting ABOS part I outcomes: AAOS exhibit selection","volume":"95","author":"Swanson","year":"2013","journal-title":"J Bone Jt Surg Am"},{"key":"R17-20260922","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1097\/00003086-198904000-00035","article-title":"Characteristics of successful and unsuccessful applicants to orthopedic residency training programs","volume":"241","author":"Clark","year":"1989","journal-title":"Clin Orthop Relat Res."},{"issue":"9","key":"R18-20260922","doi-asserted-by":"crossref","first-page":"828","DOI":"10.1001\/jamasurg.2019.1738","article-title":"Association of coworker reports about unprofessional behavior by surgeons with surgical complications in their patients","volume":"154","author":"Cooper","year":"2019","journal-title":"JAMA Surg."},{"issue":"6","key":"R19-20260922","doi-asserted-by":"crossref","first-page":"888","DOI":"10.1097\/ACM.0000000000003145","article-title":"Do professionalism lapses in medical school predict problems in residency and clinical practice?","volume":"95","author":"Krupat","year":"2020","journal-title":"Acad Med."},{"issue":"1","key":"R20-20260922","doi-asserted-by":"crossref","first-page":"104","DOI":"10.2106\/JBJS.23.01262","article-title":"AOA critical issues symposium: the disruptive physician: bad apple or toxic tree?","volume":"107","author":"Ramirez","year":"2024","journal-title":"J Bone Jt Surg."}],"container-title":["JBJS Open Access"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.ovid.com\/10.2106\/JBJS.OA.25.00119","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T18:00:30Z","timestamp":1790100030000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ovid.com\/10.2106\/JBJS.OA.25.00119"}},"subtitle":["Resident Evaluation Measures as Predictors of Performance in Practice"],"short-title":[],"issued":{"date-parts":[[2026,7]]},"references-count":19,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026]]}},"URL":"https:\/\/doi.org\/10.2106\/jbjs.oa.25.00119","relation":{},"ISSN":["2472-7245","2472-7245"],"issn-type":[{"value":"2472-7245","type":"electronic"},{"value":"2472-7245","type":"print"}],"subject":[],"published":{"date-parts":[[2026,7]]}}}