{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,7]],"date-time":"2026-11-07T04:24:28Z","timestamp":1794025468087,"version":"4.8.0"},"reference-count":10,"publisher":"Oxford University Press (OUP)","license":[{"start":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T00:00:00Z","timestamp":1791936000000},"content-version":"am","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Problem<\/jats:title>\n                    <jats:p>Analytic reasoning requires learners to integrate knowledge, demonstrate analytical skills, and employ metacognition to avoid cognitive biases. Mechanism Maps (MMs), student-generated concept maps, support this process by requiring students to link clinical findings with pathophysiologic mechanisms. However, broader adoption of MMs has been limited by challenges in assessment: existing scoring approaches emphasize structural features rather than accuracy of reasoning, show variable interrater reliability, and are time-intensive. Importantly, it remains unclear whether MM-based assessments can identify students at risk of difficulty in analytic reasoning before examination performance.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Approach<\/jats:title>\n                    <jats:p>The authors developed a five-domain rubric (comprehensiveness, analytical detail, organization, accuracy, overall quality) to assess MMs. First-year medical students (n\u2009=\u2009134) in a systems-based physiology\/pathophysiology course at Harvard Medical School in February\u2013March 2019 created three MMs for clinical vignettes as ungraded exercises. Each MM was assigned one week before a course assessment.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Outcomes<\/jats:title>\n                    <jats:p>Rater training used an initial set of MMs, after which four trained raters, working in pairs, independently scored the remaining maps. Scoring showed good-to-excellent agreement between raters across most domains, with lowest inter-rater agreement for the accuracy domain. MM scores were associated with later examination performance. For Vignette 2, each 1-point increase in score roughly doubled the odds of scoring in the top three examination quartiles rather than the bottom quartile; this association held even after accounting for quiz performance. Vignette 3 scores showed a similar pattern, though the association was attenuated after adjustment for quiz performance.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Next Steps<\/jats:title>\n                    <jats:p>The multidimensional rubric enabled reliable MM scoring, associated with examination performance, and suggests potential early identification of lower-performing students. Future work should consider prospective validation in additional institutional contexts.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/acamed\/wvag324","type":"journal-article","created":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T10:13:12Z","timestamp":1793873592000},"source":"Crossref","is-referenced-by-count":0,"title":["Developing and evaluating a rubric to assess medical students\u2019 analytic thinking and predict academic risk"],"prefix":"10.1093","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4471-2918","authenticated-orcid":false,"given":"Krisztina","family":"Fischer","sequence":"first","affiliation":[{"name":"Master of Medical Sciences in Medical Education Program, Harvard Medical School , Boston, MA,","place":["United States"]},{"name":"Department of Radiology, Brigham and Women\u2019s Hospital , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Furfaro","sequence":"additional","affiliation":[{"name":"Department of Medicine, Harvard Medical School, Beth Israel Deaconess Medical Center , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5201-180X","authenticated-orcid":false,"given":"Alexander A","family":"Iyer","sequence":"additional","affiliation":[{"name":"Harvard Medical School Program in Medical Education, , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8922-1955","authenticated-orcid":false,"given":"Jeremy B","family":"Richards","sequence":"additional","affiliation":[{"name":"Department of Medical Education, Western Atlantic University School of Medicine , Freeport, Bahamas"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-9277-4655","authenticated-orcid":false,"given":"Johannah","family":"Mitchell","sequence":"additional","affiliation":[{"name":"Carl J. Shapiro Institute for Education and Research, Beth Israel Deaconess Medical Center, Harvard Medical School , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1020-3197","authenticated-orcid":false,"given":"Amy M","family":"Sullivan","sequence":"additional","affiliation":[{"name":"Department of Medicine, Harvard Medical School, Beth Israel Deaconess Medical Center , Boston, MA,","place":["United States"]},{"name":"Carl J. Shapiro Institute for Education and Research, Beth Israel Deaconess Medical Center, Harvard Medical School , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8709-4884","authenticated-orcid":false,"given":"Richard M","family":"Schwartzstein","sequence":"additional","affiliation":[{"name":"Department of Medicine, Harvard Medical School, Beth Israel Deaconess Medical Center , Boston, MA,","place":["United States"]},{"name":"Carl J. Shapiro Institute for Education and Research, Beth Israel Deaconess Medical Center, Harvard Medical School , Boston, MA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,10,14]]},"reference":[{"key":"2026110622463339200_wvag324-B1","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1097\/ACM.0000000000002518","article-title":"Teaching critical thinking: a case for instruction in cognitive biases to reduce diagnostic errors and improve patient safety","volume":"94","author":"Royce","year":"2019","journal-title":"Acad Med"},{"key":"2026110622463339200_wvag324-B2","doi-asserted-by":"publisher","first-page":"1509","DOI":"10.1001\/jama.2025.14901","article-title":"Critical thinking for 21st-century medicine - moving beyond illness scripts","volume":"334","author":"Schwartzstein","year":"2025","journal-title":"JAMA"},{"key":"2026110622463339200_wvag324-B3","doi-asserted-by":"publisher","first-page":"e1478","DOI":"10.3109\/0142159X.2013.785628","article-title":"A randomized pilot study of the use of concept maps to enhance problem-based learning among first-year medical students","volume":"35","author":"Veronese","year":"2013","journal-title":"Med Teach"},{"key":"2026110622463339200_wvag324-B4","doi-asserted-by":"publisher","first-page":"1537","DOI":"10.1007\/s40670-025-02348-1","article-title":"Concept mapping plays a complementary role in optimizing the effectiveness of interactive simulations in medical student learning of bacterial sepsis pathophysiology","volume":"35","author":"Traylor","year":"2025","journal-title":"Med Sci Educ"},{"key":"2026110622463339200_wvag324-B5","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1097\/ACM.0000000000002445","article-title":"Assessing the effectiveness of using mechanistic concept maps in case-based collaborative learning","volume":"94","author":"Fischer","year":"2019","journal-title":"Acad Med"},{"key":"2026110622463339200_wvag324-B6","doi-asserted-by":"publisher","first-page":"629","DOI":"10.1007\/s10459-019-09944-y","article-title":"Investigating the validity of web-enabled mechanistic case diagramming scores to assess students\u2019 integration of foundational and clinical sciences","volume":"25","author":"Ferguson","year":"2020","journal-title":"Adv Health Sci Educ Theory Pract"},{"key":"2026110622463339200_wvag324-B7","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1080\/0142159X.2023.2182176","article-title":"Concept mapping: an aid to teaching and learning: AMEE guide no. 157","volume":"45","author":"Torre","year":"2023","journal-title":"Med Teach"},{"key":"2026110622463339200_wvag324-B8","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1186\/1472-6920-9-19","article-title":"Interrater reliability of the mind map assessment rubric in a cohort of medical students","volume":"9","author":"D\u2019Antoni","year":"2009","journal-title":"BMC Med Educ"},{"key":"2026110622463339200_wvag324-B9","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1097\/CEH.0000000000000086","article-title":"Frame-of-reference training: establishing reliable assessment of teaching effectiveness","volume":"36","author":"Newman","year":"2016","journal-title":"J Contin Educ Health Prof"},{"key":"2026110622463339200_wvag324-B10","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","year":"2016","journal-title":"J Chiropr Med"}],"container-title":["Academic Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/academicmedicine\/advance-article-pdf\/doi\/10.1093\/acamed\/wvag324\/71940363\/wvag324.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/academicmedicine\/advance-article-pdf\/doi\/10.1093\/acamed\/wvag324\/71940363\/wvag324.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,11,7]],"date-time":"2026-11-07T03:46:37Z","timestamp":1794023197000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/academicmedicine\/advance-article\/doi\/10.1093\/acamed\/wvag324\/8916284"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,14]]},"references-count":10,"URL":"https:\/\/doi.org\/10.1093\/acamed\/wvag324","relation":{},"ISSN":["1938-808X"],"issn-type":[{"value":"1938-808X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,14]]},"article-number":"wvag324"}}