{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T18:37:01Z","timestamp":1791484621203,"version":"4.3.4"},"reference-count":13,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"6","content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:sec>\n            <jats:title>Purpose:<\/jats:title>\n            <jats:p>To determine how often ChatGPT is able to provide accurate and comprehensive information regarding clinical vitreoretinal scenarios. To assess the types of sources ChatGPT primarily uses and to determine whether they are hallucinated.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods:<\/jats:title>\n            <jats:p>This was a retrospective cross-sectional study. The authors designed 40 open-ended clinical scenarios across four main topics in vitreoretinal disease. Responses were graded on correctness and comprehensiveness by three blinded retina specialists. The primary outcome was the number of clinical scenarios that ChatGPT answered correctly and comprehensively. Secondary outcomes included theoretical harm to patients, the distribution of the type of references used by the chatbot, and the frequency of hallucinated references.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results:<\/jats:title>\n            <jats:p>In June 2023, ChatGPT answered 83% of clinical scenarios (33\/40) correctly but provided a comprehensive answer in only 52.5% of cases (21\/40). Subgroup analysis demonstrated an average correct score of 86.7% in neovascular age-related macular degeneration, 100% in diabetic retinopathy, 76.7% in retinal vascular disease, and 70% in the surgical domain. There were six incorrect responses with one case (16.7%) of no harm, three cases (50%) of possible harm, and two cases (33.3%) of definitive harm.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion:<\/jats:title>\n            <jats:p>ChatGPT correctly answered more than 80% of complex open-ended vitreoretinal clinical scenarios, with a reduced capability to provide a comprehensive response.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1097\/iae.0000000000004053","type":"journal-article","created":{"date-parts":[[2024,1,25]],"date-time":"2024-01-25T22:00:13Z","timestamp":1706220013000},"page":"954-964","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":16,"title":["PERFORMANCE ASSESSMENT OF AN ARTIFICIAL INTELLIGENCE CHATBOT IN CLINICAL VITREORETINAL SCENARIOS"],"prefix":"10.1097","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-1917-7940","authenticated-orcid":false,"given":"Michael J.","family":"Maywood","sequence":"first","affiliation":[{"name":"Department of Ophthalmology, Corewell Health William Beaumont University Hospital, Royal Oak, Michigan;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ravi","family":"Parikh","sequence":"additional","affiliation":[{"name":"Manhattan Retina and Eye Consultants, New York, New York ;"},{"name":"Department of Ophthalmology, New York University School of Medicine, New York, New York ;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Avnish","family":"Deobhakta","sequence":"additional","affiliation":[{"name":"Icahn School of Medicine of Mount Sinai, New York, New York ;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1880-3982","authenticated-orcid":false,"given":"Tedi","family":"Begaj","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Corewell Health William Beaumont University Hospital, Royal Oak, Michigan;"},{"name":"Associated Retinal Consultants, Royal Oak, Michigan ."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","reference":[{"key":"R1-20250605","doi-asserted-by":"crossref","first-page":"721","DOI":"10.3350\/cmh.2023.0089","article-title":"Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma","volume":"29","author":"Yeo","year":"2023","journal-title":"Clin Mol Hepatol"},{"key":"R2-20250605","doi-asserted-by":"crossref","first-page":"798","DOI":"10.1001\/jamaophthalmol.2023.2754","article-title":"Performance of an upgraded artificial intelligence chatbot for ophthalmic knowledge assessment","volume":"141","author":"Mihalache","year":"2023","journal-title":"JAMA Ophthalmol"},{"key":"R3-20250605","doi-asserted-by":"crossref","first-page":"100324","DOI":"10.1016\/j.xops.2023.100324","article-title":"Evaluating the performance of ChatGPT in ophthalmology: an analysis of its successes and shortcomings","volume":"3","author":"Antaki","year":"2023","journal-title":"Ophthalmol Sci"},{"key":"R4-20250605","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/j.ajo.2023.05.024","article-title":"Performance of generative large language models on ophthalmology board-style questions","volume":"254","author":"Cai","year":"2023","journal-title":"Am J Ophthalmol"},{"key":"R8-20250605","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"R9-20250605","doi-asserted-by":"crossref","first-page":"842","DOI":"10.1001\/jama.2023.1044","article-title":"Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model","volume":"329","author":"Sarraju","year":"2023","journal-title":"JAMA"},{"key":"R10-20250605","doi-asserted-by":"crossref","first-page":"P66","DOI":"10.1016\/j.ophtha.2019.09.025","article-title":"Diabetic retinopathy preferred Practice Pattern\u00ae","volume":"127","author":"Flaxel","year":"2020","journal-title":"Ophthalmology"},{"key":"R11-20250605","doi-asserted-by":"crossref","first-page":"P1","DOI":"10.1016\/j.ophtha.2019.09.024","article-title":"Age-related macular degeneration preferred Practice Pattern\u00ae","volume":"127","author":"Flaxel","year":"2020","journal-title":"Ophthalmology"},{"key":"R12-20250605","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1016\/j.oret.2023.05.022","article-title":"Appropriateness and readability of ChatGPT-4-generated responses for surgical treatment of retinal diseases","volume":"7","author":"Momenaei","year":"2023","journal-title":"Ophthalmol Retina"},{"key":"R13-20250605","doi-asserted-by":"crossref","first-page":"906","DOI":"10.1001\/jamaophthalmol.2023.3314","article-title":"Accuracy of vitreoretinal disease information from an artificial intelligence chatbot","volume":"141","author":"Caranfa","year":"2023","journal-title":"JAMA Ophthalmol"},{"key":"R14-20250605","doi-asserted-by":"crossref","first-page":"1399","DOI":"10.1016\/j.ophtha.2019.01.027","article-title":"Autologous retinal transplant for refractory macular holes: multicenter international collaborative study group","volume":"126","author":"Grewal","year":"2019","journal-title":"Ophthalmology"},{"key":"R15-20250605","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1016\/j.survophthal.2017.03.004","article-title":"Retinal pigment epithelium tears: classification, pathogenesis, predictors, and management","volume":"62","author":"Ersoz","year":"2017","journal-title":"Surv Ophthalmol"},{"key":"R16-20250605","doi-asserted-by":"crossref","first-page":"819","DOI":"10.1001\/jamaophthalmol.2023.3119","article-title":"Evaluation and comparison of ophthalmic scientific abstracts and references by current artificial intelligence chatbots","volume":"141","author":"Hua","year":"2023","journal-title":"JAMA Ophthalmol"}],"container-title":["Retina"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.lww.com\/10.1097\/IAE.0000000000004053","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,5]],"date-time":"2025-06-05T17:02:29Z","timestamp":1749142949000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.lww.com\/10.1097\/IAE.0000000000004053"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6]]},"references-count":13,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024]]}},"URL":"https:\/\/doi.org\/10.1097\/iae.0000000000004053","relation":{},"ISSN":["0275-004X"],"issn-type":[{"value":"0275-004X","type":"print"}],"subject":[],"published":{"date-parts":[[2024,6]]}}}