{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T17:10:00Z","timestamp":1781629800185,"version":"3.54.5"},"reference-count":28,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,1,12]],"date-time":"2026-01-12T00:00:00Z","timestamp":1768176000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,1,28]],"date-time":"2026-01-28T00:00:00Z","timestamp":1769558400000},"content-version":"vor","delay-in-days":16,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Universit\u00e4tsklinikum D\u00fcsseldorf. Anstalt \u00f6ffentlichen Rechts"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Educ"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Outcomes in medical education are as complex as medical reality, ranging from individual skilled, ethical and professional development to improvement in patient care, changes in overall quality of care or changes in policy. There have been several suggestions for reporting outcome levels in medical education research, however none of which have been widely implemented,. All current models have substantial shortcomings. They lack granularity to adequately represent the complexity of medical education and care, particularly when reporting results related to patient care and health policy. Furthermore, none of the models allow for the reporting on narrative-only descriptive reports where no outcomes have yet been formally measured such as project reports.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In a modified double diamond approach, we developed a new, comprehensive model taking into account the strengths and addressing the weaknesses of prior models. The preliminary model incorporated all potential outcomes in medical education without implying a hierarchy. We sent the draft model to a group of raters for testing who discussed, revised, and reviewed it. The group of raters were experienced professionals from medical education and other fields, and represented different countries (Australia, Canada, Colombia, Germany, United States).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The raters agreed that the proposed model was easy to understand and use and had high face validity. All raters agreed on a revised model unanimously. The final comprehensive model \u2013 ROME (Reporting Outcomes in Medical Education) model \u2013 allows for reporting both narrative and quantitative outcomes in medical education literature. We developed a corresponding Excel sheet to help with the assessment of studies as well as two worked examples to illustrate its use.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>The ROME model is easy to use and at the same time comprehensively depicts all potential outcomes potentially addressed in reporting medical education research. It thus provides a classification taxonomy and describes research settings and outcomes more precisely. It serves both as a potential synthesis framework as well as a blueprint guideline for reporting outcomes. This will support researchers in medical education and their and reviewers alike. We outline potential next steps to further validate and advance our model. It will facilitate the routine reporting of outcome levels.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12909-026-08579-z","type":"journal-article","created":{"date-parts":[[2026,1,12]],"date-time":"2026-01-12T13:04:08Z","timestamp":1768223048000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["The reporting outcomes in medical education (ROME) model: proposition of a new framework"],"prefix":"10.1186","volume":"26","author":[{"given":"Janna-Lina","family":"Kerth","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronny","family":"Lehmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeffery W.","family":"Wells","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eleonora","family":"Leopardi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rebecca L.","family":"Morgan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Melissa","family":"Neubacher","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juan Jos\u00e9","family":"Yepes-Nu\u00f1ez","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hans Martin","family":"Bosse","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,1,12]]},"reference":[{"issue":"10","key":"8579_CR1","doi-asserted-by":"publisher","first-page":"955","DOI":"10.1097\/00001888-200410000-00010","volume":"79","author":"FM Chen","year":"2004","unstructured":"Chen FM, Bauchner H, Burstin H. A call for outcomes research in medical education. Acad Med. 2004;79(10):955\u201360.","journal-title":"Acad Med"},{"issue":"9","key":"8579_CR2","doi-asserted-by":"publisher","first-page":"1002","DOI":"10.1001\/jama.298.9.1002","volume":"298","author":"DA Reed","year":"2007","unstructured":"Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;298(9):1002\u20139.","journal-title":"JAMA"},{"issue":"5","key":"8579_CR3","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1097\/00001888-199005000-00003","volume":"65","author":"DG Kassebaum","year":"1990","unstructured":"Kassebaum DG. The measurement of outcomes in the assessment of educational program effectiveness. Acad Med. 1990;65(5):293\u20136.","journal-title":"Acad Med"},{"key":"8579_CR4","doi-asserted-by":"publisher","first-page":"10669","DOI":"10.15766\/mep_2374-8265.10669","volume":"14","author":"ST Li","year":"2018","unstructured":"Li ST, Gusic ME, Vinci RJ, Szilagyi PG, Klein MD. A structured framework and resources to use to get your medical education work published. MedEdPORTAL. 2018;14:10669.","journal-title":"MedEdPORTAL"},{"issue":"8","key":"8579_CR5","doi-asserted-by":"publisher","first-page":"759","DOI":"10.1097\/00001888-200608000-00015","volume":"81","author":"DW Musick","year":"2006","unstructured":"Musick DW. A conceptual model for program evaluation in graduate medical education. Acad Med. 2006;81(8):759\u201365.","journal-title":"Acad Med"},{"issue":"3","key":"8579_CR6","doi-asserted-by":"publisher","first-page":"424","DOI":"10.4300\/JGME-D-14-00339.1","volume":"6","author":"GM Sullivan","year":"2014","unstructured":"Sullivan GM, Simpson D, Cook DA, DeIorio NM, Andolsek K, Opas L, et al. Redefining quality in medical education research: A consumer\u2019s view. J Grad Med Educ. 2014;6(3):424\u20139.","journal-title":"J Grad Med Educ"},{"issue":"5","key":"8579_CR7","doi-asserted-by":"publisher","first-page":"e288","DOI":"10.3109\/0142159X.2012.668637","volume":"34","author":"AW Frye","year":"2012","unstructured":"Frye AW, Hemmer PA. Program evaluation models and related theories: AMEE guide 67. Med Teach. 2012;34(5):e288\u201399.","journal-title":"Med Teach"},{"key":"8579_CR8","first-page":"21","volume":"13","author":"D Kirkpatrick","year":"1959","unstructured":"Kirkpatrick D. Techniques for evaluation training programs. J Am Soc Train Dir. 1959;13:21\u20136.","journal-title":"J Am Soc Train Dir"},{"issue":"1","key":"8579_CR9","first-page":"54","volume":"50","author":"D Kirkpatrick","year":"1995","unstructured":"Kirkpatrick D. Revisiting kirkpatrick\u2019s four-level model. Train Dev. 1995;50(1):54\u20137.","journal-title":"Train Dev"},{"issue":"1","key":"8579_CR10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/chp.20001","volume":"29","author":"DE Moore Jr","year":"2009","unstructured":"Moore DE Jr., Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29(1):1\u201315.","journal-title":"J Contin Educ Health Prof"},{"key":"8579_CR11","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1186\/1478-4491-11-50","volume":"11","author":"G O\u2019Malley","year":"2013","unstructured":"O\u2019Malley G, Perdue T, Petracca F. A framework for outcome-level evaluation of in-service training of health care workers. Hum Resour Health. 2013;11:50.","journal-title":"Hum Resour Health"},{"key":"8579_CR12","unstructured":"Kuh GD, Kinzie J, Buckley JA, Bridges BK, Hayek JC. What Matters to Student Success: A Review of the Literature. Commissioned Report for the National Symposium on Postsecondary Student Success. Spearheading a Dialog on Student Success. National Postsecondary Education Cooperative. Washington DC; 2006."},{"key":"8579_CR13","volume-title":"Studying successfully in human medicine: A comparison of traditional and reformed degree programs at Charit\u00e9 \u2013 Universit\u00e4tsmedizin Berlin","author":"J Bosch","year":"2020","unstructured":"Bosch J. Studying successfully in human medicine: A comparison of traditional and reformed degree programs at Charit\u00e9 \u2013 Universit\u00e4tsmedizin Berlin. Berlin: Freie Universit\u00e4t Berlin; 2020."},{"key":"8579_CR14","doi-asserted-by":"publisher","first-page":"925","DOI":"10.2147\/AMEP.S212858","volume":"10","author":"M Leif","year":"2019","unstructured":"Leif M, Semarad N, Ganesan V, Selting K, Burr J, Svec A, et al. The quality of evidence in preclinical medical education literature: A systematic review. Adv Med Educ Pract. 2019;10:925\u201333.","journal-title":"Adv Med Educ Pract"},{"issue":"4","key":"8579_CR15","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1046\/j.1365-2923.2001.00910.x","volume":"35","author":"JB Prystowsky","year":"2001","unstructured":"Prystowsky JB, Bordage G. An outcomes research perspective on medical education: the predominance of trainee assessment and satisfaction. Med Educ. 2001;35(4):331\u20136.","journal-title":"Med Educ"},{"issue":"1","key":"8579_CR16","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1016\/j.acap.2021.07.007","volume":"22","author":"JL Kerth","year":"2022","unstructured":"Kerth JL, van Treel L, Bosse HM. The use of entrustable professional activities in pediatric postgraduate medical education: A systematic review. Acad Pediatr. 2022;22(1):21\u20138.","journal-title":"Acad Pediatr"},{"issue":"12","key":"8579_CR17","doi-asserted-by":"publisher","first-page":"1176","DOI":"10.1111\/j.1365-2929.2005.02341.x","volume":"39","author":"O ten Cate","year":"2005","unstructured":"ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39(12):1176\u20137.","journal-title":"Med Educ"},{"issue":"2","key":"8579_CR18","doi-asserted-by":"publisher","first-page":"491","DOI":"10.1007\/s10459-022-10098-7","volume":"27","author":"O ten Cate","year":"2022","unstructured":"ten Cate O, Schumacher DJ. Entrustable professional activities versus competencies and skills: exploring why different concepts are often conflated. Adv Health Sci Educ Theory Pract. 2022;27(2):491\u20139.","journal-title":"Adv Health Sci Educ Theory Pract"},{"issue":"2","key":"8579_CR19","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1007\/s10488-010-0319-7","volume":"38","author":"E Proctor","year":"2011","unstructured":"Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65\u201376.","journal-title":"Adm Policy Ment Health"},{"key":"8579_CR20","volume-title":"Implementation research: A synthesis of the literature","author":"DL Fixsen","year":"2005","unstructured":"Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute; 2005."},{"issue":"2","key":"8579_CR21","doi-asserted-by":"publisher","first-page":"340","DOI":"10.1086\/226550","volume":"83","author":"JW Meyer","year":"1977","unstructured":"Meyer JW, Rowan B. Institutionalized organizations: formal structure as myth and ceremony. Am J Sociol. 1977;83(2):340\u201363.","journal-title":"Am J Sociol"},{"issue":"6","key":"8579_CR22","doi-asserted-by":"publisher","first-page":"553","DOI":"10.1080\/01421599978960","volume":"21","author":"RM Harden","year":"1999","unstructured":"Harden RM, Grant J, Buckley G, Hart IR. BEME guide 1: best evidence medical education. Med Teach. 1999;21(6):553\u201362.","journal-title":"Med Teach"},{"key":"8579_CR23","doi-asserted-by":"publisher","first-page":"Article31","DOI":"10.2202\/1557-4679.1275","volume":"6","author":"A Donner","year":"2010","unstructured":"Donner A, Rotondi MA. Sample size requirements for interval Estimation of the kappa statistic for interobserver agreement studies with a binary outcome and multiple raters. Int J Biostat. 2010;6:Article31.","journal-title":"Int J Biostat"},{"issue":"2","key":"8579_CR24","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1097\/00001888-200502000-00001","volume":"80","author":"ME Whitcomb","year":"2005","unstructured":"Whitcomb ME. Using clinical outcomes data to reform medical education. Acad Med. 2005;80(2):117.","journal-title":"Acad Med"},{"key":"8579_CR25","doi-asserted-by":"crossref","unstructured":"Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-174.","DOI":"10.2307\/2529310"},{"issue":"8","key":"8579_CR26","doi-asserted-by":"publisher","first-page":"841","DOI":"10.1097\/ACM.0000000000005730","volume":"99","author":"RH Ellaway","year":"2024","unstructured":"Ellaway RH, O\u2019Brien BC, Sherbino J, Maggio LA, Artino AR Jr., Nimmon L, et al. Is there a problem with evidence in health professions education? Acad Med. 2024;99(8):841\u20138.","journal-title":"Acad Med"},{"issue":"9","key":"8579_CR27","doi-asserted-by":"publisher","first-page":"1012","DOI":"10.1080\/0142159X.2019.1605161","volume":"41","author":"A Thomas","year":"2019","unstructured":"Thomas A, L DG, van der Vleuten C, Chilingaryan G, Amari F, Steinert Y. Use of evidence in health professions education: Attitudes, practices, barriers and supports. Med Teach. 2019;41(9):1012\u201322.","journal-title":"Med Teach"},{"issue":"6","key":"8579_CR28","doi-asserted-by":"publisher","first-page":"362","DOI":"10.1007\/S40037-021-00688-3","volume":"10","author":"A Thomas","year":"2021","unstructured":"Thomas A, Ellaway RH. Rethinking implementation science for health professions education: A manifesto for change. Perspect Med Educ. 2021;10(6):362\u20138.","journal-title":"Perspect Med Educ"}],"container-title":["BMC Medical Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12909-026-08579-z","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12909-026-08579-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12909-026-08579-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T16:37:06Z","timestamp":1781627826000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1186\/s12909-026-08579-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,12]]},"references-count":28,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["8579"],"URL":"https:\/\/doi.org\/10.1186\/s12909-026-08579-z","relation":{},"ISSN":["1472-6920"],"issn-type":[{"value":"1472-6920","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,12]]},"assertion":[{"value":"2 March 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 January 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 January 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"152"}}