{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T19:56:59Z","timestamp":1789588619136,"version":"build-2803163510"},"reference-count":21,"publisher":"American Medical Association (AMA)","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Netw Open"],"abstract":"\n Importance<\/jats:title>\n The ultimate goal of residency education is to train physicians to deliver high-quality patient care. However, residents rarely receive data-driven feedback because resident-level quality measures are lacking.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n To develop and evaluate resident-sensitive quality measures (RSQMs) using electronic health record data to inform graduate medical education.<\/jats:p>\n <\/jats:sec>\n \n Design, Setting, and Participants<\/jats:title>\n This cohort study used call schedules linking senior internal medicine residents to patient admissions during overnight internal medicine call shifts at 5 teaching hospitals in Canada from July 1, 2010, through December 31, 2019. Using clinical practice guidelines, 7 RSQMs related to pneumonia or general care for all internal medicine admissions were developed. To support interpretation, a care variation framework was applied that categorized measures as low value (not recommended), discretionary (context dependent), or evidence based (recommended) to enable comparisons between observed and expected variation. The data were analyzed between March 1, 2024, and February 23, 2026.<\/jats:p>\n <\/jats:sec>\n \n Main Outcomes and Measures<\/jats:title>\n The low-value care RSQM measured potentially inappropriate red cell transfusions (all admissions). Discretionary RSQMs measured use of antibiotics, imaging, or blood work for either pneumonia or all admissions. The evidence-based care RSQM measured ordering of first-line antibiotics for pneumonia. Resident-level variation was assessed using descriptive statistics, including the median proportion of eligible admissions with each RSQM performed and interquartile range.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n The cohort included 132\u202f291 patient admissions (median [IQR] age, 70 [55-83] years; 50.6% male) linked to 793 residents. Residents had a median (IQR) of 187 (89-228) admissions, including a median (IQR) of 18 (10-24) admissions for pneumonia. Potentially inappropriate red cell transfusions occurred in a low proportion of encounters, with little variation (median, 0%; IQR, 0%-0%). Discretionary measures, including use of second-line antibiotics, advanced imaging, chest computed tomography, and serum protein electrophoresis, varied across residents. For pneumonia admissions (n = 13 470), the RSQM for first-line antibiotic use in pneumonia was sensitive to the time windows for included orders, ranging from 22% (3027 admissions) to 76% (10 205 admissions), depending on the cutoffs applied.<\/jats:p>\n <\/jats:sec>\n \n Conclusions and Relevance<\/jats:title>\n This cohort study outlined an approach to developing and evaluating RSQMs using readily available electronic health record data to evaluate internal medicine residents\u2019 quality of care. Although the RSQMs showed potential, their use for inpatient internal medicine may be more appropriate at the program level due to unresolved concerns regarding attribution and statistical reliability.<\/jats:p>\n <\/jats:sec>","DOI":"10.1001\/jamanetworkopen.2026.11700","type":"journal-article","created":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T15:30:25Z","timestamp":1778254225000},"page":"e2611700","source":"Crossref","is-referenced-by-count":1,"title":["Developing Resident-Sensitive Quality Measures for Internal Medicine"],"prefix":"10.1001","volume":"9","author":[{"given":"Brandon","family":"Tang","sequence":"first","affiliation":[{"name":"Division of General Internal Medicine, Department of Medicine, University of Toronto, Toronto, Ontario, Canada"},{"name":"Li Ka Shing Knowledge Institute, Unity Health Toronto, Toronto, Ontario, Canada"},{"name":"Division of General Internal Medicine, Department of Medicine, Unity Health Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew C. L.","family":"Lam","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew","family":"Wankiewicz","sequence":"additional","affiliation":[{"name":"Li Ka Shing Knowledge Institute, Unity Health Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marwa F.","family":"Ismail","sequence":"additional","affiliation":[{"name":"Li Ka Shing Knowledge Institute, Unity Health Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Surain B.","family":"Roberts","sequence":"additional","affiliation":[{"name":"Li Ka Shing Knowledge Institute, Unity Health Toronto, Toronto, Ontario, Canada"},{"name":"Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chang","family":"Liu","sequence":"additional","affiliation":[{"name":"Li Ka Shing Knowledge Institute, Unity Health Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Schumacher","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio"},{"name":"Division of Emergency Medicine, Cincinnati Children\u2019s Hospital Medical Center, Cincinnati, Ohio"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin","family":"Kinnear","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio"},{"name":"Division of Hospital Medicine, Cincinnati Children\u2019s Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yoon Soo","family":"Park","sequence":"additional","affiliation":[{"name":"Department of Medical Education, University of Illinois Chicago"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anjala","family":"Tess","sequence":"additional","affiliation":[{"name":"Department of Medicine, Harvard Medical School, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amol A.","family":"Verma","sequence":"additional","affiliation":[{"name":"Division of General Internal Medicine, Department of Medicine, University of Toronto, Toronto, Ontario, Canada"},{"name":"Li Ka Shing Knowledge Institute, Unity Health Toronto, Toronto, Ontario, Canada"},{"name":"Division of General Internal Medicine, Department of Medicine, Unity Health Toronto, Toronto, Ontario, Canada"},{"name":"Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada"},{"name":"Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fahad","family":"Razak","sequence":"additional","affiliation":[{"name":"Division of General Internal Medicine, Department of Medicine, University of Toronto, Toronto, Ontario, Canada"},{"name":"Li Ka Shing Knowledge Institute, Unity Health Toronto, Toronto, Ontario, Canada"},{"name":"Division of General Internal Medicine, Department of Medicine, Unity Health Toronto, Toronto, Ontario, Canada"},{"name":"Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Pusic","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, Harvard Medical School, Boston, Massachusetts"},{"name":"Research and Education Foundation, American Board of Medical Specialties, Chicago, Illinois"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shiphra","family":"Ginsburg","sequence":"additional","affiliation":[{"name":"Division of Respirology, Department of Medicine, Mount Sinai Hospital, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brian M.","family":"Wong","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Toronto, Toronto, Ontario, Canada"},{"name":"Centre for Quality Improvement and Patient Safety, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada"},{"name":"Division of General Internal Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10","published-online":{"date-parts":[[2026,5,8]]},"reference":[{"issue":"6","key":"zoi260356r1","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1080\/0142159X.2017.1315067","article-title":"A call to action: the controversy of and rationale for competency-based medical education.","volume":"39","author":"Holmboe","year":"2017","journal-title":"Med Teach"},{"issue":"5","key":"zoi260356r2","first-page":"Doc69","article-title":"Competency-based postgraduate medical education: past, present and future.","volume":"34","author":"Ten Cate","year":"2017","journal-title":"GMS J Med Educ"},{"issue":"2","key":"zoi260356r3","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1007\/s10459-015-9614-0","article-title":"Workplace-based assessment: a review of user perceptions and strategies to address the identified shortcomings.","volume":"21","author":"Massie","year":"2016","journal-title":"Adv Health Sci Educ Theory Pract"},{"issue":"7775","key":"zoi260356r4","doi-asserted-by":"publisher","first-page":"c5064","DOI":"10.1136\/bmj.c5064","article-title":"Impact of workplace based assessment on doctors\u2019 education and performance: a systematic review.","volume":"341","author":"Miller","year":"2010","journal-title":"BMJ"},{"issue":"8","key":"zoi260356r5","doi-asserted-by":"publisher","first-page":"1248","DOI":"10.1097\/ACM.0000000000003147","article-title":"Initial implementation of resident-sensitive quality measures in the pediatric emergency department: a wide range of performance.","volume":"95","author":"Schumacher","year":"2020","journal-title":"Acad Med"},{"issue":"4","key":"zoi260356r6","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1080\/21548331.2021.1915657","article-title":"Lessons for hospital care from the first wave of COVID-19 in Ontario, Canada.","volume":"49","author":"Verma","year":"2021","journal-title":"Hosp Pract (1995)"},{"issue":"2","key":"zoi260356r7","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1136\/bmjqs-2019-010425","article-title":"Physician-level variation in clinical outcomes and resource use in inpatient general internal medicine: an observational study.","volume":"30","author":"Verma","year":"2021","journal-title":"BMJ Qual Saf"},{"issue":"4","key":"zoi260356r9","doi-asserted-by":"publisher","first-page":"E842","DOI":"10.9778\/cmajo.20170097","article-title":"Patient characteristics, resource use and outcomes associated with general internal medicine hospital care: the General Medicine Inpatient Initiative (GEMINI) retrospective cohort study.","volume":"5","author":"Verma","year":"2017","journal-title":"CMAJ Open"},{"issue":"3","key":"zoi260356r10","doi-asserted-by":"publisher","first-page":"578","DOI":"10.1093\/jamia\/ocaa225","article-title":"Assessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experience.","volume":"28","author":"Verma","year":"2021","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"zoi260356r11","doi-asserted-by":"publisher","first-page":"835","DOI":"10.1093\/jamia\/ocaf042","article-title":"Optimizing the efficiency and effectiveness of data quality assurance in a multicenter clinical dataset.","volume":"32","author":"Fu","year":"2025","journal-title":"J Am Med Inform Assoc"},{"issue":"11","key":"zoi260356r12","doi-asserted-by":"publisher","first-page":"1326","DOI":"10.1097\/ACM.0000000000005288","article-title":"Uses of electronic health record data to measure the clinical learning environment of graduate medical education trainees: a systematic review.","volume":"98","author":"Lees","year":"2023","journal-title":"Acad Med"},{"issue":"5","key":"zoi260356r13","doi-asserted-by":"publisher","first-page":"1271","DOI":"10.1007\/s11606-020-06320-0","article-title":"Development of resident-sensitive quality measures for inpatient general internal medicine.","volume":"36","author":"Kinnear","year":"2021","journal-title":"J Gen Intern Med"},{"issue":"7370","key":"zoi260356r14","doi-asserted-by":"publisher","first-page":"961","DOI":"10.1136\/bmj.325.7370.961","article-title":"Unwarranted variations in healthcare delivery: implications for academic medical centres.","volume":"325","author":"Wennberg","year":"2002","journal-title":"BMJ"},{"issue":"8","key":"zoi260356r15","doi-asserted-by":"publisher","first-page":"1128","DOI":"10.1097\/ACM.0000000000004667","article-title":"Will any road get you there? examining warranted and unwarranted variation in medical education.","volume":"97","author":"Holmboe","year":"2022","journal-title":"Acad Med"},{"issue":"9","key":"zoi260356r16","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2022-062264","article-title":"Methodology paper for the General Medicine Inpatient Initiative Medical Education Database (GEMINI MedED): a retrospective cohort study of internal medicine resident case-mix, clinical care and patient outcomes.","volume":"12","author":"Lam","year":"2022","journal-title":"BMJ Open"},{"issue":"12","key":"zoi260356r17","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.50768","article-title":"Variation in case exposure during internal medicine residency.","volume":"7","author":"Lam","year":"2024","journal-title":"JAMA Netw Open"},{"key":"zoi260356r18","doi-asserted-by":"publisher","first-page":"1228","DOI":"10.1080\/03610910902859574","article-title":"Using the standardized difference to compare the prevalence of a binary variable between two groups in observational research.","volume":"38","author":"Austin","year":"2009","journal-title":"Commun Stat Simul Comput"},{"issue":"5","key":"zoi260356r19","doi-asserted-by":"publisher","first-page":"E799","DOI":"10.9778\/cmajo.20220193","article-title":"Identifying clusters of coexisting conditions and outcomes among adults admitted to hospital with community-acquired pneumonia: a multicentre cohort study.","volume":"11","author":"Malecki","year":"2023","journal-title":"CMAJ Open"},{"key":"zoi260356r20","doi-asserted-by":"publisher","first-page":"S57","DOI":"10.1097\/ACM.0000000000005610","article-title":"Ambulatory long block: a model of precision education and assessment for internal medicine residents.","volume":"99","author":"Warm","year":"2024","journal-title":"Acad Med"},{"issue":"3","key":"zoi260356r21","doi-asserted-by":"publisher","first-page":"281","DOI":"10.4300\/JGME-D-21-00706.1","article-title":"The reliability of graduate medical education quality of care clinical performance measures.","volume":"14","author":"Kim","year":"2022","journal-title":"J Grad Med Educ"},{"key":"zoi260356r8","unstructured":"MyPractice and OurPractice: general medicine. Health Quality Ontario. Accessed February 1, 2026. https:\/\/www.hqontario.ca\/Quality-Improvement\/Practice-Reports\/MyPractice-General-Medicine"}],"container-title":["JAMA Network Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/articlepdf\/2848775\/tang_2026_oi_260356_1777988995.26792.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T15:30:26Z","timestamp":1778254226000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/fullarticle\/2848775"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,8]]},"references-count":21,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,5,1]]}},"URL":"https:\/\/doi.org\/10.1001\/jamanetworkopen.2026.11700","relation":{},"ISSN":["2574-3805"],"issn-type":[{"value":"2574-3805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,8]]}}}