{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T17:20:39Z","timestamp":1781112039558,"version":"3.54.1"},"reference-count":23,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"46","license":[{"start":{"date-parts":[[2025,11,14]],"date-time":"2025-11-14T00:00:00Z","timestamp":1763078400000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,11,14]]},"abstract":"<jats:p>\n                    Chat Generative Pretrained Transformer (ChatGPT), a large language model developed by OpenAI, has shown potential in healthcare communication and patient education. However, its performance in specialized medical domains, such as pituitary adenomas (PAs), remains unclear. Therefore, this study aimed to evaluate the reliability and consistency of ChatGPT in answering PA-related questions. We hypothesized that ChatGPT would demonstrate high reliability in responding to general patient-oriented queries but lower reliability for specialized clinical questions. A total of 256 PA-related questions were collected from patients and families, clinical practice guidelines, and medical question banks. Each question was input into ChatGPT (GPT-4, March 2025 version), and the generated responses were independently reviewed by 2 senior neurosurgeons. Any discrepancies in their assessments were resolved by a third neurosurgeon with over 30 years of clinical experience. Responses were categorized as completely correct, partially correct but usable, partially correct, or incorrect. Responses rated as completely correct or partially correct but usable were considered reliable. Consistency was assessed based on the stability of response quality across similar question types. Comparisons were made by question type (general vs professional) and source using univariate analysis. Among the 256 responses, 143 (55.8%) were completely correct, 68 (26.6%) were partially correct but usable, 19 (7.4%) were partially correct, and 26 (10.2%) were incorrect. Overall, 82.4% of the responses were considered reliable, and 68.4% demonstrated consistency. Reliability was significantly higher for general questions than for professional ones (95.0% vs 78.6%, OR\u2005=\u20055.182, 95% CI: 1.545\u201317.378,\n                    <jats:italic toggle=\"yes\">P<\/jats:italic>\n                    \u2005=\u2005.003), and for guideline-derived questions compared to question bank-derived ones (100.0% vs 75.7%, OR\u2005=\u20051.321, 95% CI: 1.214\u20131.437,\n                    <jats:italic toggle=\"yes\">P<\/jats:italic>\n                    \u2005=\u2005.017). Differences in consistency across subgroups were not statistically significant. ChatGPT exhibits high reliability and moderate consistency in answering PA-related questions, especially for general and guideline-based content. It may serve as a supplementary source of patient information but should not replace professional medical consultation, particularly in complex or surgical contexts. As this study was conducted in an artificial testing environment without validation in real patient consultations, the generalizability of the findings remains limited.\n                  <\/jats:p>","DOI":"10.1097\/md.0000000000045928","type":"journal-article","created":{"date-parts":[[2025,11,14]],"date-time":"2025-11-14T21:01:57Z","timestamp":1763154117000},"page":"e45928","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":2,"title":["Evaluation of ChatGPT as a supplementary tool for pituitary adenomas: An observational study based on simulated consultations"],"prefix":"10.1097","volume":"104","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2604-7363","authenticated-orcid":false,"given":"Yuhui","family":"Chen","sequence":"first","affiliation":[{"name":"Department of Neurosurgery, Fuzong Clinical Medical College of Fujian Medical University, Fuzhou, Fujian, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuyang","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Fuzong Clinical Medical College of Fujian Medical University, Fuzhou, Fujian, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Li","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Fuzong Clinical Medical College of Fujian Medical University, Fuzhou, Fujian, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tianshun","family":"Feng","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Fuzong Clinical Medical College of Fujian Medical University, Fuzhou, Fujian, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4218-0068","authenticated-orcid":false,"given":"Shousen","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Fuzong Clinical Medical College of Fujian Medical University, Fuzhou, Fujian, China."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","reference":[{"key":"R1-20251121","doi-asserted-by":"crossref","first-page":"e230582","DOI":"10.1148\/radiol.230582","article-title":"Performance of ChatGPT on a radiology board-style examination: insights into current strengths and limitations.","volume":"307","author":"Bhayana","year":"2023","journal-title":"Radiology"},{"key":"R2-20251121","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1001\/jama.2023.11440","article-title":"Artificial intelligence in clinical diagnosis: opportunities, challenges, and hype.","volume":"330","author":"Kulkarni","year":"2023","journal-title":"JAMA"},{"key":"R3-20251121","doi-asserted-by":"crossref","first-page":"e2317517","DOI":"10.1001\/jamanetworkopen.2023.17517","article-title":"Evaluating artificial intelligence responses to public health questions.","volume":"6","author":"Ayers","year":"2023","journal-title":"JAMA Netw Open"},{"key":"R4-20251121","doi-asserted-by":"crossref","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models.","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digit Health"},{"key":"R5-20251121","doi-asserted-by":"crossref","first-page":"e31","DOI":"10.1016\/j.bja.2023.04.017","article-title":"Assessment of ChatGPT success with specialty medical knowledge using anaesthesiology board examination practice questions.","volume":"131","author":"Shay","year":"2023","journal-title":"Br J Anaesth"},{"key":"R6-20251121","doi-asserted-by":"crossref","first-page":"1623","DOI":"10.1097\/CORR.0000000000002704","article-title":"Can artificial intelligence pass the american board of orthopaedic surgery examination? Orthopaedic residents versus ChatGPT.","volume":"481","author":"Lum","year":"2023","journal-title":"Clin Orthop Relat Res"},{"key":"R7-20251121","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41523-023-00557-8","article-title":"Large language model (ChatGPT) as a support tool for breast tumor board.","volume":"9","author":"Sorin","year":"2023","journal-title":"npj Breast Cancer"},{"key":"R8-20251121","doi-asserted-by":"crossref","first-page":"904","DOI":"10.3171\/2023.2.JNS23419","article-title":"ChatGPT versus the neurosurgical written boards: a comparative analysis of artificial intelligence\/machine learning performance on neurosurgical board-style questions.","volume":"139","author":"Hopkins","year":"2023","journal-title":"J Neurosurg"},{"key":"R9-20251121","doi-asserted-by":"crossref","first-page":"1090","DOI":"10.1227\/neu.0000000000002551","article-title":"Performance of ChatGPT, GPT-4, and google bard on a neurosurgery oral boards preparation question bank.","volume":"93","author":"Ali","year":"2023","journal-title":"Neurosurgery"},{"key":"R10-20251121","doi-asserted-by":"crossref","first-page":"1386","DOI":"10.1001\/jama.2023.5444","article-title":"Diagnosis and management of pituitary adenomas: a review.","volume":"329","author":"Tritos","year":"2023","journal-title":"JAMA"},{"key":"R11-20251121","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.ando.2020.11.001","article-title":"Multidisciplinary protocol of preoperative and surgical management of patients with pituitary tumors candidates to pituitary surgery.","volume":"82","author":"Araujo-Castro","year":"2021","journal-title":"Ann Endocrinol (Paris)"},{"key":"R12-20251121","doi-asserted-by":"crossref","first-page":"721","DOI":"10.3350\/cmh.2023.0089","article-title":"Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma.","volume":"29","author":"Yeo","year":"2023","journal-title":"Clin Mol Hepatol"},{"key":"R13-20251121","first-page":"e102","volume":"5","year":"2023","journal-title":"ChatGPT: friend or foe?"},{"key":"R14-20251121","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1038\/d41586-023-00288-7","article-title":"ChatGPT: five priorities for research.","volume":"614","author":"van Dis","year":"2023","journal-title":"Nature"},{"key":"R15-20251121","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1093\/clinchem\/hvad058","article-title":"Assessing the accuracy and clinical utility of ChatGPT in laboratory medicine.","volume":"69","author":"Munoz-Zuluaga","year":"2023","journal-title":"Clin Chem"},{"key":"R16-20251121","doi-asserted-by":"crossref","first-page":"407","DOI":"10.3390\/jpm15090407","article-title":"Artificial intelligence in the diagnosis of pediatric rare diseases: from real-world data toward a personalized medicine approach.","volume":"15","author":"Ili\u0107","year":"2025","journal-title":"J Pers Med"},{"key":"R17-20251121","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1186\/s13023-025-03656-w","article-title":"Comparative analysis of large language models on rare disease identification.","volume":"20","author":"Ao","year":"2025","journal-title":"Orphanet J Rare Dis"},{"key":"R18-20251121","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1186\/s13023-025-03655-x","article-title":"Applying artificial intelligence to rare diseases: a literature review highlighting lessons from Fabry disease.","volume":"20","author":"Germain","year":"2025","journal-title":"Orphanet J Rare Dis"},{"key":"R19-20251121","doi-asserted-by":"crossref","first-page":"2106","DOI":"10.1111\/cts.13619","article-title":"Artificial intelligence in rare disease diagnosis and treatment.","volume":"16","author":"Wojtara","year":"2023","journal-title":"Clin Transl Sci"},{"key":"R20-20251121","doi-asserted-by":"crossref","first-page":"894","DOI":"10.1210\/jc.2010-1048","article-title":"Pituitary incidentaloma: an endocrine society clinical practice guideline.","volume":"96","author":"Freda","year":"2011","journal-title":"J Clin Endocrinol Metab"},{"key":"R21-20251121","doi-asserted-by":"crossref","first-page":"G1","DOI":"10.1530\/EJE-17-0796","article-title":"European society of endocrinology clinical practice guidelines for the management of aggressive pituitary tumours and carcinomas.","volume":"178","author":"Raverot","year":"2018","journal-title":"Eur J Endocrinol"},{"key":"R25-20251121","doi-asserted-by":"crossref","first-page":"1790","DOI":"10.1007\/s11695-023-06603-5","article-title":"Assessing the accuracy of responses by the language model ChatGPT to questions regarding bariatric surgery.","volume":"33","author":"Samaan","year":"2023","journal-title":"Obes Surg"},{"key":"R26-20251121","doi-asserted-by":"crossref","first-page":"e230922","DOI":"10.1148\/radiol.230922","article-title":"How AI responds to common lung cancer questions: chatGPT vs google bard.","volume":"307","author":"Rahsepar","year":"2023","journal-title":"Radiology"}],"container-title":["Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.lww.com\/10.1097\/MD.0000000000045928","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,21]],"date-time":"2025-11-21T23:02:28Z","timestamp":1763766148000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.lww.com\/10.1097\/MD.0000000000045928"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,14]]},"references-count":23,"journal-issue":{"issue":"46","published-print":{"date-parts":[[2025]]}},"URL":"https:\/\/doi.org\/10.1097\/md.0000000000045928","relation":{},"ISSN":["0025-7974","1536-5964"],"issn-type":[{"value":"0025-7974","type":"print"},{"value":"1536-5964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,14]]},"assertion":[{"value":"2025-07-30","name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-13","name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}}]}}