{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T19:04:27Z","timestamp":1789585467370,"version":"build-2803163510"},"reference-count":21,"publisher":"Oxford University Press (OUP)","issue":"9","license":[{"start":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T00:00:00Z","timestamp":1781222400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Evaluate whether large language models reason or simply regurgitate training data in clinical diagnosis.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>We audited 2000 clinical case reports from PubMed Central: 1000 from 2021 to 2022 (within training data) and 1000 from 2025 (after training cutoffs). Five frontier LLMs generated diagnoses evaluated by an independent AI judge validated against physician consensus (n\u2009=\u200910\u2009000 evaluations).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Diagnostic accuracy was virtually identical across temporal cohorts (66.8% contaminated vs 66.9% clean), directly contradicting the memorization hypothesis. Lexical similarity was uniformly low (mean ROUGE-L 0.057), and semantic similarity measured by BERTScore showed no memorization signal (F1 0.8182 contaminated vs 0.8195 clean, \u0394 = +0.0013), confirming that models generate novel reasoning rather than regurgitating training data.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>This large-scale audit, using both lexical and semantic similarity metrics, provides compelling evidence that LLMs engage in genuine clinical reasoning rather than regurgitating memorized training data.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Models demonstrated equivalent accuracy on cases they could not have seen during training, suggesting they have internalized generalizable medical knowledge rather than memorizing specific cases.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocag069","type":"journal-article","created":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T11:29:34Z","timestamp":1776770974000},"page":"1617-1626","source":"Crossref","is-referenced-by-count":0,"title":["Reasoning or reciting? A temporal contamination audit of large language models in clinical medicine"],"prefix":"10.1093","volume":"33","author":[{"given":"Alexander P","family":"Sheppert","sequence":"first","affiliation":[{"name":"Internal Medicine, Legacy Health System , Portland, OR 98686,","place":["United States"]},{"name":"Internal Medicine, The Vancouver Clinic , Vancouver, WA 98686,","place":["United States"]},{"name":"Artificial Intelligence, Capitol Technology University , Laurel, MD 20708,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brian","family":"Adams","sequence":"additional","affiliation":[{"name":"Vascular Surgery, Intermountain Healthcare , Salt Lake City, UT 84604,","place":["United States"]},{"name":"Bioinformatics, University of Utah , Salt Lake City, UT 84112,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew D","family":"Sheppert","sequence":"additional","affiliation":[{"name":"Internal Medicine, Legacy Health System , Portland, OR 98686,","place":["United States"]},{"name":"Internal Medicine, The Vancouver Clinic , Vancouver, WA 98686,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sylvia","family":"Riley","sequence":"additional","affiliation":[{"name":"Internal Medicine, Legacy Health System , Portland, OR 98686,","place":["United States"]},{"name":"Internal Medicine, The Vancouver Clinic , Vancouver, WA 98686,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,6,12]]},"reference":[{"key":"2026091614315156100_ocag069-B1","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"2026091614315156100_ocag069-B2","doi-asserted-by":"crossref","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat Med"},{"key":"2026091614315156100_ocag069-B3","year":"2023"},{"key":"2026091614315156100_ocag069-B4","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1038\/s41746-022-00742-2","article-title":"A large language model for electronic health records","volume":"5","author":"Yang","year":"2022","journal-title":"NPJ Digit Med"},{"key":"2026091614315156100_ocag069-B5","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1038\/s41746-023-00879-8","article-title":"The shaky foundations of large language models and foundation models for electronic health records","volume":"6","author":"Wornow","year":"2023","journal-title":"NPJ Digit Med"},{"key":"2026091614315156100_ocag069-B6","year":"2021"},{"key":"2026091614315156100_ocag069-B7"},{"key":"2026091614315156100_ocag069-B8","first-page":"10776"},{"key":"2026091614315156100_ocag069-B9","author":"OpenAI","year":"2023"},{"key":"2026091614315156100_ocag069-B10"},{"key":"2026091614315156100_ocag069-B11","author":"OpenRouter"},{"key":"2026091614315156100_ocag069-B12"},{"key":"2026091614315156100_ocag069-B13"},{"key":"2026091614315156100_ocag069-B14"},{"key":"2026091614315156100_ocag069-B15","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1109\/TSMCC.2011.2134847","article-title":"Understanding plagiarism linguistic patterns, textual features, and detection methods","volume":"42","author":"Alzahrani","year":"2012","journal-title":"IEEE Trans Syst Man Cybern C"},{"key":"2026091614315156100_ocag069-B16","doi-asserted-by":"crossref","first-page":"e45312","DOI":"10.2196\/45312","article-title":"How does ChatGPT perform on the United States Medical Licensing Examination? The implications of large language models for medical education and knowledge assessment","volume":"9","author":"Gilson","year":"2023","journal-title":"JMIR Med Educ."},{"key":"2026091614315156100_ocag069-B17"},{"key":"2026091614315156100_ocag069-B18"},{"key":"2026091614315156100_ocag069-B19","doi-asserted-by":"crossref","first-page":"1952","DOI":"10.1001\/jamainternmed.2013.10081","article-title":"Physicians\u2019 diagnostic accuracy, confidence, and resource requests: a vignette study","volume":"173","author":"Meyer","year":"2013","journal-title":"JAMA Intern Med"},{"key":"2026091614315156100_ocag069-B20","doi-asserted-by":"crossref","first-page":"e64963","DOI":"10.2196\/64963","article-title":"Comparing diagnostic accuracy of clinical professionals and large language models: systematic review and meta-analysis","volume":"13","author":"Shan","year":"2025","journal-title":"JMIR Med Inform"},{"key":"2026091614315156100_ocag069-B21","author":"Ouyang","year":"2022"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/advance-article-pdf\/doi\/10.1093\/jamia\/ocag069\/68516573\/ocag069.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/33\/9\/1617\/68516573\/ocag069.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/33\/9\/1617\/68516573\/ocag069.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T18:32:00Z","timestamp":1789583520000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/33\/9\/1617\/8706785"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,12]]},"references-count":21,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,6,12]]},"published-print":{"date-parts":[[2026,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocag069","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,9]]},"published":{"date-parts":[[2026,6,12]]}}}