{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T23:46:01Z","timestamp":1792021561874,"version":"4.4.0"},"reference-count":8,"publisher":"Georg Thieme Verlag KG","issue":"04","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2026,8]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Background Large language models can return citations whose PubMed Identifier (PMID), digital object identifier (DOI), title, and journal appear plausible but identify different papers. Checking one valid identifier may therefore fail to detect an incorrect citation bundle.<\/jats:p>\n                  <jats:p>Objective This study aimed to measure citation metadata incoherence in responses from five large language model snapshots to an internal medicine prompt benchmark.<\/jats:p>\n                  <jats:p>Methods Five models answered 30 synthetic prompts each, producing 150 model-by-prompt responses. Twenty-six prompts per model requested citations, and four were designed to elicit abstention. A corrected parser extracted 300 citation units. We checked reported PMIDs against PubMed metadata, queried reported DOIs separately, and rechecked DOI strings not found in PubMed through Crossref. Response-level outcomes were primary because citation units were nested within responses. The author reviewed a deterministic stratified sample of 50 units for parser capture and citation identity.<\/jats:p>\n                  <jats:p>Results Among 130 citation-expected responses, 69 contained a DOI that resolved to a different PMID than the PMID supplied by the model, 100 contained at least one automated metadata mismatch, and 28 contained no citation. Those 100 were 69 DOI\u2013PMID, 29 other, and 2 unresolvable. All 20 abstention trap responses abstained. Among 116 units whose reported DOI resolved in PubMed, 99 resolved to a different PMID and 17 to the reported PMID. Human review found correct parser capture in 48 of 50 sampled units. Two section boundary errors were corrected before the final rerun; the primary counts did not change. Human review classified 32 sampled units as incoherent, 7 as coherent, 6 as having an unresolvable PMID, and 5 as containing no citation. Clinical claim support was not assessed.<\/jats:p>\n                  <jats:p>Conclusion Citation identity failures occurred in all five tested model snapshots. Evaluation and clinical evidence workflows should confirm that the PMID, DOI, title, and journal identify the same source before treating a model-supplied citation as evidence.<\/jats:p>","DOI":"10.1055\/a-2972-4974","type":"journal-article","created":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T22:52:41Z","timestamp":1792018361000},"page":"950-954","source":"Crossref","is-referenced-by-count":0,"title":["Do Large Language Models Link to Incorrect Citations? A PubMed and DOI Benchmark"],"prefix":"10.1055","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5022-9124","authenticated-orcid":false,"given":"Goktug","family":"Ozkan","sequence":"additional","affiliation":[{"name":"Maltepe State Hospital, Istanbul, T\u00fcrkiye","place":["Istanbul"],"department":["Department of Internal Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"194","published-online":{"date-parts":[[2026,10,14]]},"reference":[{"key":"ref1","doi-asserted-by":"crossref","first-page":"e54345","DOI":"10.2196\/54345","article-title":"Reference hallucination score for medical artificial intelligence chatbots: development and usability study","volume":"12","author":"F Aljamaan","year":"2024","journal-title":"JMIR Med Inform"},{"issue":"06","key":"ref2","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1097\/SIH.0000000000000877","article-title":"Exploring AI hallucinations of ChatGPT: reference accuracy and citation relevance of ChatGPT models and training conditions","volume":"20","author":"A Cheng","year":"2025","journal-title":"Simul Healthc"},{"key":"ref3","doi-asserted-by":"crossref","first-page":"115334","DOI":"10.1016\/j.psychres.2023.115334","article-title":"ChatGPT and bard exhibit spontaneous citation fabrication during psychiatry literature search","volume":"326","author":"A McGowan","year":"2023","journal-title":"Psychiatry Res"},{"issue":"7\/8","key":"ref4","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.diii.2024.04.003","article-title":"ChatGPT in radiology: a systematic review of performance, pitfalls, and future perspectives","volume":"105","author":"P Keshavarz","year":"2024","journal-title":"Diagn Interv Imaging"},{"issue":"06","key":"ref5","doi-asserted-by":"crossref","first-page":"1071","DOI":"10.1093\/jamia\/ocaf063","article-title":"The emergence of large language models as tools in literature reviews: a large language model-assisted systematic review","volume":"32","author":"D Scherbakov","year":"2025","journal-title":"J Am Med Inform Assoc"},{"issue":"03","key":"ref6","doi-asserted-by":"crossref","first-page":"389","DOI":"10.7326\/ANNALS-24-02189","article-title":"Development of prompt templates for large language model-driven screening in systematic reviews","volume":"178","author":"C Cao","year":"2025","journal-title":"Ann Intern Med"},{"issue":"05","key":"ref7","doi-asserted-by":"crossref","first-page":"ooaf119","DOI":"10.1093\/jamiaopen\/ooaf119","article-title":"Grounded large language models for diagnostic prediction in real-world emergency department settings","volume":"8","author":"A Niset","year":"2025","journal-title":"JAMIA Open"},{"issue":"09","key":"ref8","first-page":"e92571","article-title":"Building a safe and transparent workflow for large language model (LLM)-assisted clinical trials and prediction models: a technical report","volume":"17","author":"J Frutuoso","year":"2025","journal-title":"Cureus"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/a-2972-4974.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T22:52:43Z","timestamp":1792018363000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/a-2972-4974"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8]]},"references-count":8,"journal-issue":{"issue":"04","published-online":{"date-parts":[[2026,8,14]]},"published-print":{"date-parts":[[2026,8]]}},"URL":"https:\/\/doi.org\/10.1055\/a-2972-4974","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8]]}}}