{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,12]],"date-time":"2026-10-12T22:55:56Z","timestamp":1791845756781,"version":"4.3.4"},"reference-count":38,"publisher":"SLACK, Inc.","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Pediatr Ophthalmol Strabismus"],"published-print":{"date-parts":[[2025,3]]},"abstract":"<jats:sec>\n            <jats:title>Purpose<\/jats:title>\n            <jats:p>To assess the appropriateness and readability of responses provided by four large language models (LLMs) (ChatGPT-4, Claude 3, Gemini, and Microsoft Copilot) to parents' queries pertaining to retinopathy of prematurity (ROP).<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>A total of 60 frequently asked questions were collated and categorized into six distinct sections. The responses generated by the LLMs were evaluated by three experienced ROP specialists to determine their appropriateness and comprehensiveness. Additionally, the readability of the responses was assessed using a range of metrics, including the Flesch-Kincaid Grade Level (FKGL), Gunning Fog (GF) Index, Coleman-Liau (CL) Index, Simple Measure of Gobbledygook (SMOG) Index, and Flesch Reading Ease (FRE) score.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>ChatGPT-4 demonstrated the highest level of appropriateness (100%) and performed exceptionally well in the Likert analysis, scoring 5 points on 96% of questions. The CL Index and FRE scores identified Gemini as the most readable LLM, whereas the GF Index and SMOG Index rated Microsoft Copilot as the most readable. Nevertheless, ChatGPT-4 exhibited the most intricate text structure, with scores of 18.56 on the GF Index, 18.56 on the CL Index, 17.2 on the SMOG Index, and 9.45 on the FRE score. This suggests that the responses demand a college-level comprehension.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusions<\/jats:title>\n            <jats:p>ChatGPT-4 demonstrated higher performance than other LLMs in responding to questions related to ROP; however, its texts were more complex. In terms of readability, Gemini and Microsoft Copilot were found to be more successful.<\/jats:p>\n            <jats:p>\n              <jats:bold>\n                [\n                <jats:italic>J Pediatr Ophthalmol Strabismus<\/jats:italic>\n                . 2025;62(2):84\u201395.]\n              <\/jats:bold>\n            <\/jats:p>\n          <\/jats:sec>","DOI":"10.3928\/01913913-20240911-05","type":"journal-article","created":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T10:31:48Z","timestamp":1730111508000},"page":"84-95","source":"Crossref","is-referenced-by-count":11,"title":["Assessing the Responses of Large Language Models (ChatGPT-4, Claude 3, Gemini, and Microsoft Copilot) to Frequently Asked Questions in Retinopathy of Prematurity: A Study on Readability and Appropriateness"],"prefix":"10.3928","volume":"62","author":[{"given":"Serhat","family":"Ermis","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ece","family":"\u00d6zal","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Murat","family":"Karapapak","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ebrar","family":"Kumanta\u015f","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sad\u0131k Altan","family":"\u00d6zal","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"2308","reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(13)60178-6"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1080\/13548506.2020.1797129"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.2196\/19985"},{"key":"e_1_3_1_5_2","unstructured":"Pew Research Center. Health Online. 2013. https:\/\/www.pewre-search.org\/internet\/2013\/01\/15\/health-online-2013\/. Accessed April 26 2023."},{"key":"e_1_3_1_6_2","unstructured":"Eurostat. Individuals Using the Internet for Seeking Health-Related Information. Accessed on 18 April 2024). https:\/\/ec.europa.eu\/eurostat\/databrowser\/view\/tin00101\/default\/table"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.2752"},{"key":"e_1_3_1_8_2","unstructured":"Open AI. (2024). Accessed: April 13 2024: https:\/\/chat.openai.com"},{"key":"e_1_3_1_9_2","unstructured":"Google Gemini. (2024). Accessed: April 13 2024. https:\/\/gemini.google.com\/app"},{"key":"e_1_3_1_10_2","unstructured":"Microsoft Copilot. (2024). Accessed: April 13 2024. https:\/\/copilot.microsoft.com"},{"key":"e_1_3_1_11_2","unstructured":"Anthropic Claude. (2024). Accessed: April 13 2024. https:\/\/www.anthropic.com\/claude"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ebiom.2023.104770"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ogla.2022.03.004"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.3928\/01913913-20240124-01"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1111\/aos.15661"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1080\/08820538.2024.2326058"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oret.2023.05.022"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10459-010-9222-y"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.3928\/01913913-20240409-01"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1097\/IOP.0000000000002549"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1017\/dmp.2020.151"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.2196\/46939"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41433-023-02563-3"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1227\/neu.0000000000002551"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1148\/rycan.230086"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics13111950"},{"key":"e_1_3_1_27_2","article-title":"ChatGPT sets record for fastest-growing user base\u2013analyst note","author":"Hu K.","year":"2023","unstructured":"Hu K. ChatGPT sets record for fastest-growing user base\u2013analyst note. Reuters. Sect. Technology. February 2, 2023. https:\/\/www.reuters.com\/technology\/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01\/","journal-title":"Reuters. Sect. Technology"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.59960"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.59898"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1089\/thy.2023.0491"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1177\/20552076231224603"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00296-023-05473-5"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.45473"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","unstructured":"Makrygiannakis MA Giannakopoulos K Kaklamanos EG. Evidence-based potential of generative artificial intelligence large language models in orthodontics: a comparative study of ChatGPT Google Bard and Microsoft Bing. Eur J Orthod. 2024;cjae017. 10.1093\/ejo\/cjae017 PMID:38613510","DOI":"10.1093\/ejo\/cjae017"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.soard.2024.03.011"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.3390\/children11060750"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","unstructured":"Seth I Lim B Xie Y . Comparing the efficacy of large language models ChatGPT BARD and Bing AI in providing information on rhinoplasty: an observational study. Aesthet Surg J Open Forum. 2023;5:ojad084. 10.1093\/asjof\/ojad084 PMID:37795257","DOI":"10.1093\/asjof\/ojad084"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1002\/ski2.313"},{"key":"e_1_3_1_39_2","first-page":"127e142","article-title":"Evaluation of the quality of web-site about health using marketing scoring","volume":"14","author":"Letunovska NY","year":"2022","unstructured":"Letunovska NY, Alfawaz K. Evaluation of the quality of web-site about health using marketing scoring. J Inf Technol Manag. 2022;14:127e142.","journal-title":"J Inf Technol Manag"}],"container-title":["Journal of Pediatric Ophthalmology &amp; Strabismus"],"original-title":[],"language":"en","deposited":{"date-parts":[[2025,3,24]],"date-time":"2025-03-24T11:17:51Z","timestamp":1742815071000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.healio.com\/doi\/10.3928\/01913913-20240911-05"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3]]},"references-count":38,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,3]]}},"alternative-id":["10.3928\/01913913-20240911-05"],"URL":"https:\/\/doi.org\/10.3928\/01913913-20240911-05","relation":{},"ISSN":["0191-3913","1938-2405"],"issn-type":[{"value":"0191-3913","type":"print"},{"value":"1938-2405","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3]]}}}