{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,12]],"date-time":"2026-10-12T22:55:58Z","timestamp":1791845758218,"version":"4.3.4"},"reference-count":39,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2025,9,4]],"date-time":"2025-09-04T00:00:00Z","timestamp":1756944000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Int J STD AIDS"],"published-print":{"date-parts":[[2026,2]]},"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>This study compares three large language models (LLMs) in answering common HIV questions, given ongoing concerns about their accuracy and reliability in patient education.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Models answered 63 HIV questions. Accuracy (5-point Likert), readability (Flesch-Kincaid, Gunning Fog, Coleman-Liau), and reliability (DISCERN, EQIP) were assessed.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Claude 3.7 Sonnet showed significantly higher accuracy (4.54 \u00b1 0.44) compared to ChatGPT-4o (4.29 \u00b1 0.49) and Gemini Advanced 2.0 Flash (4.31 \u00b1 0.50) (\n                      <jats:italic toggle=\"yes\">p<\/jats:italic>\n                      &lt; .001). ChatGPT-4o had lower accuracy in disease definition, follow-up, and transmission routes, while Gemini Advanced 2.0 Flash performed poorly in daily life and treatment-related questions. Readability analyses indicated ChatGPT-4o produced the most accessible content according to Flesch-Kincaid and Coleman-Liau indices, whereas Claude 3.7 Sonnet was most comprehensible by Gunning Fog standards. Gemini Advanced 2.0 Flash consistently generated more complex texts across all readability measures (\n                      <jats:italic toggle=\"yes\">p<\/jats:italic>\n                      &lt; .001). Regarding reliability, Claude 3.7 Sonnet achieved \u201cgood\u201d quality on DISCERN, while others were rated \u201cmoderate\u201d (\n                      <jats:italic toggle=\"yes\">p<\/jats:italic>\n                      = .059). On EQIP, Claude 3.7 Sonnet (median 61.8) and ChatGPT-4o (55.3) were classified as \u201cgood quality with minor limitations,\u201d whereas Gemini Advanced 2.0 Flash (41.2) was rated \u201clow quality\u201d (\n                      <jats:italic toggle=\"yes\">p<\/jats:italic>\n                      = .049).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Claude 3.7 Sonnet is preferable for accuracy and reliability, while ChatGPT-4o offers superior readability. Selecting LLMs for HIV education should consider accuracy, readability, and reliability, emphasizing regular assessment of content quality and cultural sensitivity.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/09564624251372369","type":"journal-article","created":{"date-parts":[[2025,9,4]],"date-time":"2025-09-04T10:20:52Z","timestamp":1756981252000},"page":"112-120","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":6,"title":["Artificial intelligence meets HIV education: Comparing three large language models on accuracy, readability, and reliability"],"prefix":"10.1177","volume":"37","author":[{"given":"\u00d6zge","family":"Eren Korkmaz","sequence":"first","affiliation":[{"name":"Izmir Dokuz Eylul University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4955-0789","authenticated-orcid":false,"given":"Burcu","family":"A\u00e7\u0131kal\u0131n Ar\u0131kan","sequence":"additional","affiliation":[{"name":"Izmir Democracy University Buca Seyfi Demirsoy Training Research Hospital"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Selda","family":"Say\u0131n Kutlu","sequence":"additional","affiliation":[{"name":"Denizli Pamukkale University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Figen","family":"Kaptan Aydo\u011fmu\u015f","sequence":"additional","affiliation":[{"name":"Izmir Katip Celebi University Atat\u00fcrk Training and Research Hospital"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nurbanu","family":"Sezak","sequence":"additional","affiliation":[{"name":"Izmir Democracy University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2025,9,4]]},"reference":[{"key":"e_1_3_7_2_2","unstructured":"Data on the size of the HIV epidemic [cited 2025 Apr 26]. Available from: https:\/\/www.who.int\/data\/gho\/data\/themes\/hiv-aids\/data-on-the-size-of-the-hiv-aids-epidemic"},{"key":"e_1_3_7_3_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12955-017-0778-6"},{"key":"e_1_3_7_4_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2016-011453"},{"key":"e_1_3_7_5_2","doi-asserted-by":"publisher","DOI":"10.1080\/20008198.2021.1990527"},{"key":"e_1_3_7_6_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2019-032435"},{"key":"e_1_3_7_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10461-021-03262-4"},{"key":"e_1_3_7_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-49186-4_31"},{"key":"e_1_3_7_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.metabol.2017.01.011"},{"key":"e_1_3_7_10_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0273445"},{"key":"e_1_3_7_11_2","doi-asserted-by":"publisher","DOI":"10.1097\/QAD.0000000000003935"},{"key":"e_1_3_7_12_2","unstructured":"Ma Y Achiche S Tu G et al. The first AI\u2010based chatbot to promote HIV self\u2010management: a mixed methods usability study."},{"key":"e_1_3_7_13_2","doi-asserted-by":"publisher","DOI":"10.1080\/10872981.2023.2246781"},{"issue":"2","key":"e_1_3_7_14_2","first-page":"45","article-title":"The use of AI in enhancing patient communication","volume":"4","author":"Amina NJ","year":"2024","unstructured":"Amina NJ. The use of AI in enhancing patient communication. Res Invent J Sci Exp Sci 2024; 4(2): 45\u201348.","journal-title":"Res Invent J Sci Exp Sci"},{"key":"e_1_3_7_15_2","unstructured":"HIV and AIDS [cited 2025 Jul 23]. Available from: https:\/\/www.who.int\/news-room\/questions-and-answers\/item\/hiv-aids"},{"key":"e_1_3_7_16_2","unstructured":"HIV and AIDS - basic facts | UNAIDS [cited 2025 Jul 23]. Available from: https:\/\/www.unaids.org\/en\/frequently-asked-questions-about-hiv-and-aids"},{"key":"e_1_3_7_17_2","unstructured":"Pozitif Ya\u015fam Derne\u011fi. Pozitif ya\u015fam derne\u011fi. Available from: https:\/\/www.pozitifyasam.org\/"},{"key":"e_1_3_7_18_2","unstructured":"Pozitif. Dernek | Pozitif-iz derne\u011fi | \u0130stanbul. Available from: https:\/\/www.pozitifiz.org"},{"key":"e_1_3_7_19_2","unstructured":"S\u0131k\u00e7a sorulan HIV sorular\u0131. S\u0131k\u00e7a sorulan HIV sorular\u0131 I k\u0131rm\u0131z\u0131 Kurdele \u0130stanbul I HIV bilgisi I. Available from. https:\/\/www.kirmizikurdele.org\/sikca-sorulan-hiv-sorulari"},{"key":"e_1_3_7_20_2","unstructured":"AIDS ve CYBH Derne\u011fi. [cited 2025 Jul 23]. Available from: https:\/\/www.aidsvecinselhastaliklar.com\/bilgilen"},{"key":"e_1_3_7_21_2","doi-asserted-by":"publisher","DOI":"10.3390\/children11060750"},{"key":"e_1_3_7_22_2","doi-asserted-by":"publisher","DOI":"10.1136\/jech.53.2.105"},{"key":"e_1_3_7_23_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1369-7625.2004.00273.x"},{"key":"e_1_3_7_24_2","unstructured":"Readable. Flesch reading ease and the flesch kincaid grade level. Available from: https:\/\/readable.com\/readability\/flesch-reading-ease-flesch-kincaid-grade-level\/"},{"key":"e_1_3_7_25_2","unstructured":"Readable. The gunning Fog index. Available from: https:\/\/readable.com\/readability\/gunning-fog-index\/"},{"key":"e_1_3_7_26_2","unstructured":"Readable. The coleman Liau readability index. Available from: https:\/\/readable.com\/readability\/coleman-liau-readability-index\/"},{"key":"e_1_3_7_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11596-021-2474-3"},{"key":"e_1_3_7_28_2","article-title":"Evaluating AI chatbots for HIV prevention: an assessment of response quality and user tailoring","author":"Fujimoto M","year":"2024","unstructured":"Fujimoto M, Hunter L, McCoy S, et al. Evaluating AI chatbots for HIV prevention: an assessment of response quality and user tailoring. Calif HIVAIDS Policy Res Cent 2024.","journal-title":"Calif HIVAIDS Policy Res Cent"},{"key":"e_1_3_7_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10461-024-04391-2"},{"key":"e_1_3_7_30_2","doi-asserted-by":"publisher","DOI":"10.1111\/hiv.13604"},{"key":"e_1_3_7_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613905.3651077"},{"key":"e_1_3_7_32_2","doi-asserted-by":"publisher","DOI":"10.2196\/52055"},{"key":"e_1_3_7_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.hlpt.2020.06.004"},{"issue":"2","key":"e_1_3_7_34_2","first-page":"171","article-title":"Measuring readability in Turkish","volume":"58","author":"Ate\u015fman E","year":"1997","unstructured":"Ate\u015fman E. Measuring readability in Turkish. AU T\u00f6mer Lang J 1997; 58(2): 171\u2013174.","journal-title":"AU T\u00f6mer Lang J"},{"key":"e_1_3_7_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40615-024-02162-2"},{"key":"e_1_3_7_36_2","doi-asserted-by":"publisher","DOI":"10.51594\/csitrj.v5i6.1199"},{"key":"e_1_3_7_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11904-020-00490-6"},{"key":"e_1_3_7_38_2","doi-asserted-by":"publisher","DOI":"10.1097\/JNC.0000000000000468"},{"key":"e_1_3_7_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11904-023-00681-x"},{"issue":"1","key":"e_1_3_7_40_2","first-page":"89","article-title":"AI in the management of HIV: case study Cameroon","volume":"10","author":"Andigema AS","year":"2023","unstructured":"Andigema AS. AI in the management of HIV: case study Cameroon. Int J Virol AIDS 2023; 10(1): 89.","journal-title":"Int J Virol AIDS"}],"container-title":["International Journal of STD &amp; AIDS"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/09564624251372369","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/09564624251372369","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/09564624251372369","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T22:47:03Z","timestamp":1777675623000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/09564624251372369"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,4]]},"references-count":39,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,2]]}},"alternative-id":["10.1177\/09564624251372369"],"URL":"https:\/\/doi.org\/10.1177\/09564624251372369","relation":{},"ISSN":["0956-4624","1758-1052"],"issn-type":[{"value":"0956-4624","type":"print"},{"value":"1758-1052","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,4]]}}}