{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,14]],"date-time":"2026-11-14T19:50:22Z","timestamp":1794685822993,"version":"4.9.1"},"reference-count":31,"publisher":"BMJ","issue":"1","license":[{"start":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T00:00:00Z","timestamp":1729123200000},"content-version":"unspecified","delay-in-days":16,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Open Ophth"],"accepted":{"date-parts":[[2024,10,2]]},"published-print":{"date-parts":[[2024,10]]},"abstract":"<jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>To conduct a head-to-head comparative analysis of cataract surgery patient education material generated by Chat Generative Pre-trained Transformer (ChatGPT-4) and Google Bard.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods and analysis<\/jats:title>\n                  <jats:p>98 frequently asked questions on cataract surgery in English were taken in November 2023 from 5 trustworthy online patient information resources. 59 of these were curated (20 augmented for clarity and 39 duplicates excluded) and categorised into 3 domains: condition (n=15), preparation for surgery (n=21) and recovery after surgery (n=23). They were formulated into input prompts with \u2018prompt engineering\u2019. Using the Patient Education Materials Assessment Tool-Printable (PEMAT-P) Auto-Scoring Form, four ophthalmologists independently graded ChatGPT-4 and Google Bard responses. The readability of responses was evaluated using a Flesch-Kincaid calculator. Responses were also subjectively examined for any inaccurate or harmful information.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Google Bard had a higher mean overall Flesch-Kincaid Level (8.02) compared with ChatGPT-4 (5.75) (p&lt;0.001), also noted across all three domains. ChatGPT-4 had a higher overall PEMAT-P understandability score (85.8%) in comparison to Google Bard (80.9%) (p&lt;0.001), which was also noted in the \u2018preparation for cataract surgery\u2019 (85.2% vs 75.7%; p&lt;0.001) and \u2018recovery after cataract surgery\u2019 (86.5% vs 82.3%; p=0.004) domains. There was no statistically significant difference in overall (42.5% vs 44.2%; p=0.344) or individual domain actionability scores (p&gt;0.10). None of the generated material contained dangerous information.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>In comparison to Google Bard, ChatGPT-4 fared better overall, scoring higher on the PEMAT-P understandability scale and exhibiting more faithfulness to the prompt engineering instruction. Since input prompts might vary from real-world patient searches, follow-up studies with patient participation are required.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1136\/bmjophth-2024-001824","type":"journal-article","created":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T19:45:12Z","timestamp":1729194312000},"page":"e001824","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":37,"title":["Artificial intelligence chatbots as sources of patient education material for cataract surgery: ChatGPT-4 versus Google Bard"],"prefix":"10.1136","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8571-2503","authenticated-orcid":false,"given":"Matthew","family":"Azzopardi","sequence":"first","affiliation":[{"name":"Royal Free London NHS Foundation Trust, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin","family":"Ng","sequence":"additional","affiliation":[{"name":"University of Oxford Christ Church, Oxford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abison","family":"Logeswaran","sequence":"additional","affiliation":[{"name":"Moorfields Eye Hospital NHS Foundation Trust, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Constantinos","family":"Loizou","sequence":"additional","affiliation":[{"name":"Modality Partnership, Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryan Chin Taw","family":"Cheong","sequence":"additional","affiliation":[{"name":"Royal National ENT and Eastman Dental Hospitals, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Prasanth","family":"Gireesh","sequence":"additional","affiliation":[{"name":"Leela Eye Clinic, Chennai, Tamilnadu, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1081-1141","authenticated-orcid":false,"given":"Darren Shu Jeng","family":"Ting","sequence":"additional","affiliation":[{"name":"Academic Unit of Ophthalmology, University of Birmingham Institute of Inflammation and Ageing, Birmingham, UK"},{"name":"Birmingham and Midland Eye Centre, Birmingham, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu Jeat","family":"Chong","sequence":"additional","affiliation":[{"name":"Singapore Eye Research Institute, Singapore"},{"name":"University of Cambridge, Cambridge, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2024,10,17]]},"reference":[{"key":"2025090109480953000_9.1.e001824.1","doi-asserted-by":"publisher","DOI":"10.1609\/AIMAG.V27I4.1911"},{"key":"2025090109480953000_9.1.e001824.2","unstructured":"OpenAI . Introducing ChatGPT. 2022. Available: https:\/\/openai.com\/blog\/chatgpt"},{"key":"2025090109480953000_9.1.e001824.3","unstructured":"Pichai S . An important next step on our AI journey [Google]. 2023. Available: https:\/\/blog.google\/technology\/ai\/bard-google-ai-search-updates\/"},{"key":"2025090109480953000_9.1.e001824.4","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1038\/s41433-023-02619-4","article-title":"ChatGPT in ophthalmology: the dawn of a new era?","volume":"38","author":"Ting","year":"2024","journal-title":"Eye (Lond)"},{"key":"2025090109480953000_9.1.e001824.5","unstructured":"OpenAI . GPT-4 is OpenAI\u2019s most advanced system, producing safer and more useful responses. 2023. Available: https:\/\/openai.com\/gpt-4"},{"key":"2025090109480953000_9.1.e001824.6","doi-asserted-by":"crossref","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat Med"},{"key":"2025090109480953000_9.1.e001824.7","unstructured":"Nori H , King N , McKinney SM , et al . Capabilities of gpt-4 on medical challenge problems. arXiv 2023. doi:10.48550\/arXiv.2303.13375"},{"key":"2025090109480953000_9.1.e001824.8","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pdig.0000341","article-title":"Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study","volume":"3","author":"Thirunavukarasu","year":"2024","journal-title":"PLOS Digit Health"},{"key":"2025090109480953000_9.1.e001824.9","unstructured":"Pichai S , Hassabis D . Introducing Gemini: our largest and most capable AI model [Google]. 2023. Available: https:\/\/blog.google\/technology\/ai\/google-gemini-ai\/#sundar-note"},{"key":"2025090109480953000_9.1.e001824.10","doi-asserted-by":"publisher","DOI":"10.1136\/bjophthalmol-2018-313173"},{"key":"2025090109480953000_9.1.e001824.11","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2017.18152"},{"key":"2025090109480953000_9.1.e001824.12","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1097\/APO.0000000000000394","article-title":"Artificial Intelligence in Cornea, Refractive Surgery, and Cataract: Basic Principles, Clinical Applications, and Future Directions","volume":"10","author":"Rampat","year":"2021","journal-title":"Asia Pac J Ophthalmol (Phila)"},{"key":"2025090109480953000_9.1.e001824.13","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.4103\/ijo.IJO_2119_21","article-title":"Development and validation of an offline deep learning algorithm to detect vitreoretinal abnormalities on ocular ultrasound","volume":"70","author":"Adithya","year":"2022","journal-title":"Indian J Ophthalmol"},{"key":"2025090109480953000_9.1.e001824.14","first-page":"3669","article-title":"A Global View on Output and Outcomes of Cataract Surgery With National Indices of Socioeconomic Development","volume":"58","author":"Wang","year":"2017","journal-title":"Invest Ophthalmol Vis Sci"},{"key":"2025090109480953000_9.1.e001824.15","unstructured":"Moorfields Eye Hospital NHS Trust . Patient information - cataract service 2020. 2023. Available: https:\/\/www.moorfields.nhs.uk\/sites\/default\/files\/Cataract%20service_1.pdf"},{"key":"2025090109480953000_9.1.e001824.16","unstructured":"RNIB, The Royal College of Ophthalmologists . Understanding cataracts. 2022. Available: https:\/\/www.rcophth.ac.uk\/wp-content\/uploads\/2023\/03\/Understanding-Cataracts-2022.pdf"},{"key":"2025090109480953000_9.1.e001824.17","unstructured":"NHS United Kingdom . Cataract surgery. 2021. Available: https:\/\/www.nhs.uk\/conditions\/cataract-surgery"},{"key":"2025090109480953000_9.1.e001824.18","unstructured":"National Eye Institute . Cataract surgery. 2023. Available: https:\/\/www.nei.nih.gov\/learn-about-eye-health\/eye-conditions-and-diseases\/cataracts\/cataract-surgery"},{"key":"2025090109480953000_9.1.e001824.19","unstructured":"Patient (UK registered trademark) . Cataracts. 2023. Available: https:\/\/patient.info\/eye-care\/cataracts"},{"key":"2025090109480953000_9.1.e001824.20","first-page":"463","article-title":"Health literacy: the gap between physicians and patients","volume":"72","author":"Safeer","year":"2005","journal-title":"Am Fam Physician"},{"key":"2025090109480953000_9.1.e001824.21","unstructured":"Good Calculators . Flesch kincaid calculator. n.d. Available: https:\/\/goodcalculators.com\/flesch-kincaid-calculator"},{"key":"2025090109480953000_9.1.e001824.22","unstructured":"Agency for Healthcare Research and Quality . PEMAT for printable materials (PEMAT-P). Rockville, Maryland, 2013. Available: https:\/\/www.ahrq.gov\/health-literacy\/patient-education\/pemat-p.html"},{"key":"2025090109480953000_9.1.e001824.23","unstructured":"Agency for Healthcare Research and Quality . The patient education materials assessment tool (PEMAT) and user\u2019s guide. Rockville, Maryland, 2013. Available: https:\/\/www.ahrq.gov\/health-literacy\/patient-education\/pemat1.html"},{"key":"2025090109480953000_9.1.e001824.24","doi-asserted-by":"crossref","first-page":"985","DOI":"10.1007\/s00405-023-08319-9","article-title":"Artificial intelligence chatbots as sources of patient education material for obstructive sleep apnoea: ChatGPT versus Google Bard","volume":"281","author":"Cheong","year":"2024","journal-title":"Eur Arch Otorhinolaryngol"},{"key":"2025090109480953000_9.1.e001824.25","unstructured":"Peque\u00f1o IV A . Major chatgpt update: AI program no longer restricted to Sept. 2021 knowledge cutoff after internet browser Revamp [Forbes]. 2023. Available: https:\/\/www.forbes.com\/sites\/antoniopequenoiv\/2023\/09\/27\/major-chatgpt-update-ai-program-no-longer-restricted-to-sept-2021-knowledge-cutoff-after-internet-browser-revamp\/?sh=4f65a6c66e01"},{"key":"2025090109480953000_9.1.e001824.26","article-title":"Artificial Hallucinations in ChatGPT: Implications in Scientific Writing","volume":"15","author":"Alkaissi","year":"2023","journal-title":"Cureus"},{"key":"2025090109480953000_9.1.e001824.27","doi-asserted-by":"crossref","first-page":"100024","DOI":"10.1016\/j.nlp.2023.100024","article-title":"An evaluation on large language model outputs: Discourse and memorization","volume":"4","author":"de Wynter","year":"2023","journal-title":"Nat Lang Process J"},{"key":"2025090109480953000_9.1.e001824.28","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1038\/s41586-024-07421-0","article-title":"Detecting hallucinations in large language models using semantic entropy","volume":"630","author":"Farquhar","year":"2024","journal-title":"Nature New Biol"},{"key":"2025090109480953000_9.1.e001824.29","doi-asserted-by":"crossref","unstructured":"Maleki N , Padmanabhan B , Dutta K . AI hallucinations: a misnomer worth clarifying. arXiv 2024. doi:10.48550\/arXiv.2401.06796","DOI":"10.1109\/CAI59869.2024.00033"},{"key":"2025090109480953000_9.1.e001824.30","unstructured":"National Cyber Security Centre . ChatGPT and large language models: what\u2019s the risk. 2023. Available: https:\/\/www.ncsc.gov.uk\/blog-post\/chatgpt-and-large-language-models-whats-the-risk"},{"key":"2025090109480953000_9.1.e001824.31","unstructured":"Shah S . ChatGPT 5 release date: what we know about openai\u2019s next chatbot [The Standard]. 2024. Available: https:\/\/www.standard.co.uk\/news\/tech\/chatgpt-5-release-date-details-openai-chatbot-b1130369.html"}],"container-title":["BMJ Open Ophthalmology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjophth-2024-001824","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,1]],"date-time":"2025-09-01T16:48:20Z","timestamp":1756745300000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmjophth.bmj.com\/lookup\/doi\/10.1136\/bmjophth-2024-001824"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10]]},"references-count":31,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,10,17]]},"published-print":{"date-parts":[[2024,10]]}},"alternative-id":["10.1136\/bmjophth-2024-001824"],"URL":"https:\/\/doi.org\/10.1136\/bmjophth-2024-001824","relation":{},"ISSN":["2397-3269"],"issn-type":[{"value":"2397-3269","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10]]}}}