{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T09:57:06Z","timestamp":1778234226939,"version":"3.51.4"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,4,3]],"date-time":"2025-04-03T00:00:00Z","timestamp":1743638400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,4,3]],"date-time":"2025-04-03T00:00:00Z","timestamp":1743638400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002668","name":"Abant Izzet Baysal University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100002668","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Med Syst"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Background<\/jats:title>\n            <jats:p>Artificial intelligence (AI) chatbots are increasingly used for medical inquiries, including sensitive topics like sexually transmitted diseases (STDs). However, concerns remain regarding the reliability and readability of the information they provide. This study aimed to assess the reliability and readability of AI chatbots in providing information on STDs. The key objectives were to determine (1) the reliability of STD-related information provided by AI chatbots, and (2) whether the readability of this information meets the recommended standarts for patient education materials.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>Eleven relevant STD-related search queries were identified using Google Trends and entered into four AI chatbots: ChatGPT, Gemini, Perplexity, and Copilot. The reliability of the responses was evaluated using established tools, including DISCERN, EQIP, JAMA, and GQS. Readability was assessed using six widely recognized metrics, such as the Flesch-Kincaid Grade Level and the Gunning Fog Index. The performance of chatbots was statistically compared in terms of reliability and readability.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>The analysis revealed significant differences in reliability across the AI chatbots. Perplexity and Copilot consistently outperformed ChatGPT and Gemini in DISCERN and EQIP scores, suggesting that these two chatbots provided more reliable information. However, results showed that none of the chatbots achieved the 6th-grade readability standard. All the chatbots generated information that was too complex for the general public, especially for individuals with lower health literacy levels.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion<\/jats:title>\n            <jats:p>While Perplexity and Copilot showed better reliability in providing STD-related information, none of the chatbots met the recommended readability benchmarks. These findings highlight the need for future improvements in both the accuracy and accessibility of AI-generated health information, ensuring it can be easily understood by a broader audience.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1007\/s10916-025-02178-z","type":"journal-article","created":{"date-parts":[[2025,4,4]],"date-time":"2025-04-04T16:43:32Z","timestamp":1743785012000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["AI Chatbots as Sources of STD Information: A Study on Reliability and Readability"],"prefix":"10.1007","volume":"49","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7423-4585","authenticated-orcid":false,"given":"H\u00fcseyin Alperen","family":"Y\u0131ld\u0131z","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1454-5672","authenticated-orcid":false,"given":"Emrullah","family":"S\u00f6\u011f\u00fctdelen","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,4,3]]},"reference":[{"key":"2178_CR1","doi-asserted-by":"publisher","unstructured":"Sun, J., Zhang, S., Hou, M., Sun, Q., Cao, F., Zhang, Z., Tang, G., Wang, X., Geng., L., Cui, L., and Chen, Z.-J., Who can help me? Understanding the antecedent and consequence of medical information seeking behavior in the era of bigdata. Front Public Health. 11:1192405, 2023. https:\/\/doi.org\/10.3389\/fpubh.2023.1192405","DOI":"10.3389\/fpubh.2023.1192405"},{"key":"2178_CR2","doi-asserted-by":"publisher","first-page":"795","DOI":"10.1016\/j.amjmed.2019.01.017","volume":"132","author":"N Noorbakhsh-Sabet","year":"2019","unstructured":"Noorbakhsh-Sabet N, Zand R, Zhang Y, Abedi V (2019) Artificial Intelligence Transforms the Future of Health Care. Am J Med 132:795\u2013801. https:\/\/doi.org\/10.1016\/j.amjmed.2019.01.017","journal-title":"Am J Med"},{"key":"2178_CR3","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1146\/annurev-publhealth-040119-094127","volume":"41","author":"B Swire-Thompson","year":"2020","unstructured":"Swire-Thompson B, Lazer D (2020) Public Health and Online Misinformation: Challenges and Recommendations. Annu Rev Public Health 41:433\u2013451. https:\/\/doi.org\/10.1146\/annurev-publhealth-040119-094127","journal-title":"Annu Rev Public Health"},{"key":"2178_CR4","unstructured":"Radfort, A., Narasimhan, K., Salimans, T., and Sutskever, I., (OpenAI Transformer): Improving language understanding by Generative Pre-Training. OpenAI.\u00a0(2018). https:\/\/cdn.openai.com\/research-covers\/language-unsupervised\/language_understanding_paper.pdf. Accessed 20 Sep 2024."},{"key":"2178_CR5","unstructured":"World Health Organization., Global and regional STI estimates. WHO. 2024. https:\/\/www.who.int\/data\/gho\/data\/themes\/topics\/global-and-regional-sti-estimates.\u00a0Accesed 20 Sept 2024."},{"key":"2178_CR6","unstructured":"Weiss BD (2007) Health literacy and patient safety: Help patients understand (manual for clinicians, second edition). American Medical Association Foundation and American Medical Association"},{"key":"2178_CR7","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1016\/j.eururo.2023.07.004","volume":"85","author":"D Musheyev","year":"2024","unstructured":"Musheyev D, Pan A, Loeb S, Kabarriti AE (2024) How Well Do Artificial Intelligence Chatbots Respond to the Top Search Queries About Urological Malignancies? Eur Urol 85:13\u201316. https:\/\/doi.org\/10.1016\/j.eururo.2023.07.004","journal-title":"Eur Urol"},{"key":"2178_CR8","doi-asserted-by":"publisher","unstructured":"Wang, J., and Yun, X., Accuracy and readability of kidney stone patient information materials generated by a large language model compared to official urologic organizations.\u00a0Urol.\u00a0189:e12, 2024. https:\/\/doi.org\/10.1016\/j.urology.2024.03.029","DOI":"10.1016\/j.urology.2024.03.029"},{"key":"2178_CR9","doi-asserted-by":"publisher","unstructured":"\u015eahin., M. F., Kele\u015f, A., \u00d6zcan, R., Do\u011fan, \u00c7., Topka\u00e7, E. C., Akg\u00fcl, M., and Yaz\u0131ci, C. M., Evaluation of information accuracy and clarity: ChatGPT responses to the most frequently asked questions about premature ejaculation. Sex Med. 12:qfae036, 2024. https:\/\/doi.org\/10.1093\/sexmed\/qfae036","DOI":"10.1093\/sexmed\/qfae036"},{"key":"2178_CR10","doi-asserted-by":"publisher","unstructured":"Talyshinskii, A., Naik, N., Hameed, B. M. Z., Julieb\u00f8-Jones, P., and Somani, B. K., Potential of AI-driven chatbots in urology: Revolutionizing patient care through artificial intelligence. Curr Urol Rep. 25:9\u201318, 2024. https:\/\/doi.org\/10.1007\/s11934-023-01184-3","DOI":"10.1007\/s11934-023-01184-3"},{"key":"2178_CR11","doi-asserted-by":"publisher","unstructured":"Whiles, B. B., Bird, V. G., Canales, B. K., DiBianco, J. M., and Terry, R. S., Caution! AI bot has entered the patient chat: ChatGPT has limitations in providing accurate urologic healthcare advice. Urol. 180:278\u2013284, 2023. https:\/\/doi.org\/10.1016\/j.urology.2023.07.010","DOI":"10.1016\/j.urology.2023.07.010"},{"key":"2178_CR12","doi-asserted-by":"publisher","first-page":"3149","DOI":"10.1007\/s00345-023-04563-0","volume":"41","author":"JJ Szczesniewski","year":"2023","unstructured":"Szczesniewski JJ, Tellez Fouz C, Ramos Alba A, Diaz Goizueta FJ, Garc\u00eda Tello A, Llanes Gonz\u00e1lez L (2023) ChatGPT and most frequent urological diseases: analysing the quality of information and potential risks for patients. World J Urol 41:3149\u20133153. https:\/\/doi.org\/10.1007\/s00345-023-04563-0","journal-title":"World J Urol"},{"key":"2178_CR13","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1007\/s10916-024-02056-0","volume":"48","author":"MF \u015eahin","year":"2024","unstructured":"\u015eahin MF, Ate\u015f H, Kele\u015f A, \u00d6zcan R, Do\u011fan \u00c7, Akg\u00fcl M, Yaz\u0131c\u0131 CM (2024) Responses of Five Different Artificial Intelligence Chatbots to the Top Searched Queries About Erectile Dysfunction: A Comparative Analysis. J Med Syst 48:38. https:\/\/doi.org\/10.1007\/s10916-024-02056-0","journal-title":"J Med Syst"},{"key":"2178_CR14","doi-asserted-by":"publisher","unstructured":"Kim, S. H., Tae, J. H., Chang, I. H., Kim, T.-H., Myung, S. C., Nguyen, T. T., Choi, J., Kim, J. H., Kim, J. W., Lee, Y. S., and Choi, S. Y., Changes in patient perceptions regarding ChatGPT-written explanations on lifestyle modifications for preventing urolithiasis recurrence. Digit Health. 9:20552076231203940, 2023. https:\/\/doi.org\/10.1177\/20552076231203940","DOI":"10.1177\/20552076231203940"},{"key":"2178_CR15","doi-asserted-by":"publisher","unstructured":"Mavragani, A., and Ochoa, G., Google trends in infodemiology and infoveillance: Methodology framework. JMIR Public Health Surveill. 5:e13439, 2019. https:\/\/doi.org\/10.2196\/13439","DOI":"10.2196\/13439"},{"key":"2178_CR16","doi-asserted-by":"publisher","unstructured":"Scharkow, M., and Vogelgesang, J., Measuring the public agenda using search engine queries. Int J Public Opin Res. 23:104\u2013113, 2011. https:\/\/doi.org\/10.1093\/ijpor\/edq048","DOI":"10.1093\/ijpor\/edq048"},{"key":"2178_CR17","doi-asserted-by":"publisher","unstructured":"Xu, C., Wang, Y., Yang, H., Hou, J., Sun, L., Zhang, X., Cao, X., Hou, Y., Wang, L., Cai, Q., and Wang, Y. Association between cancer incidence and mortality in web-based data in China: Infodemiology study. J Med Internet Res. 21:e10677, 2019. https:\/\/doi.org\/10.2196\/10677","DOI":"10.2196\/10677"},{"key":"2178_CR18","doi-asserted-by":"publisher","unstructured":"Watad, A., Watad, S., Mahroum, N., Sharif, K., Amital, H., Bragazzi, N. L., and Adawi, M. Forecasting the west nile virus in the United States: An extensive novel data streams-based time series analysis and structural equation modeling of related digital searching behavior. JMIR Public Health Surveill. 5:e9176, 2019. https:\/\/doi.org\/10.2196\/publichealth.9176","DOI":"10.2196\/publichealth.9176"},{"key":"2178_CR19","unstructured":"Smith, E. A., and Senter, R. J., Automated readability index. AMRL-TR Aerospace Medical Research Laboratories. 1\u201314, 1967. https:\/\/apps.dtic.mil\/sti\/pdfs\/AD0667273.pdf"},{"key":"2178_CR20","unstructured":"Kincaid, P. J., Derivation of new readability formulas. Chief of Naval Technical Training, Naval Air Station Memphis. 1975."},{"key":"2178_CR21","doi-asserted-by":"publisher","unstructured":"Coleman, M., and Liau, T. L., A computer readability formula designed for machine scoring. J Appl Psychol. 60:283\u2013284, 1975. https:\/\/doi.org\/10.1037\/h0076540","DOI":"10.1037\/h0076540"},{"key":"2178_CR22","doi-asserted-by":"publisher","unstructured":"Hedman, A. S., Using the SMOG formula to revise a health-related document. Am J Health Educ. 39:61\u201364, 2008. https:\/\/doi.org\/10.1080\/19325037.2008.10599016","DOI":"10.1080\/19325037.2008.10599016"},{"key":"2178_CR23","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","volume":"53","author":"D Charnock","year":"1999","unstructured":"Charnock D, Shepperd S, Needham G, Gann R (1999) DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health (1978) 53:105\u2013111. https:\/\/doi.org\/10.1136\/jech.53.2.105","journal-title":"J Epidemiol Community Health (1978)"},{"key":"2178_CR24","doi-asserted-by":"publisher","first-page":"e31","DOI":"10.1016\/j.wneu.2012.11.003","volume":"82","author":"AG Weil","year":"2014","unstructured":"Weil AG, Bojanowski MW, Jamart J, Gustin T, L\u00e9v\u00eaque M (2014) Evaluation of the Quality of Information on the Internet Available to Patients Undergoing Cervical Spine Surgery. World Neurosurg 82:e31\u2013e39. https:\/\/doi.org\/10.1016\/j.wneu.2012.11.003","journal-title":"World Neurosurg"},{"key":"2178_CR25","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","volume":"7","author":"B Moult","year":"2004","unstructured":"Moult B, Franck LS, Brady H (2004) Ensuring Quality Information for Patients: development and preliminary validation of a new instrument to improve the quality of written health care information. Health Expectations 7:165\u2013175. https:\/\/doi.org\/10.1111\/j.1369-7625.2004.00273.x","journal-title":"Health Expectations"},{"key":"2178_CR26","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1046\/j.1369-6513.2002.00189.x","volume":"5","author":"T Hain","year":"2002","unstructured":"Hain T (2002) Improving the quality of health information: the contribution of C\u2010H\u2010i\u2010Q. Health Expectations 5:270\u2013273. https:\/\/doi.org\/10.1046\/j.1369-6513.2002.00189.x","journal-title":"Health Expectations"},{"key":"2178_CR27","doi-asserted-by":"publisher","unstructured":"Silberg, W. M., Lundberg, G. D., and Musacchio, R. A., Assessing, controlling, and assuring the quality of medical information on the internet: Caveant Lector et Viewor - Let the reader and viewer beware. JAMA. 277:1244\u20131245, 1997. https:\/\/doi.org\/10.1001\/jama.1997.03540390074039","DOI":"10.1001\/jama.1997.03540390074039"},{"key":"2178_CR28","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen J (1960) A Coefficient of Agreement for Nominal Scales. Educ Psychol Meas 20:37\u201346. https:\/\/doi.org\/10.1177\/001316446002000104","journal-title":"Educ Psychol Meas"},{"key":"2178_CR29","doi-asserted-by":"publisher","unstructured":"McCool, M. E., Wahl, J., Schlecht, I., and Apfelbacher, C., Evaluating written patient information for eczema in German: Comparing the reliability of two instruments, DISCERN and EQIP. PLoS One. 10:e0139895, 2015. https:\/\/doi.org\/10.1371\/journal.pone.0139895","DOI":"10.1371\/journal.pone.0139895"},{"key":"2178_CR30","doi-asserted-by":"publisher","unstructured":"Gen\u00e7, M., YouTube as a source of patient information on positron emission tomography. J Health Sci Med. 6:597\u2013603, 2023. https:\/\/doi.org\/10.32322\/jhsm.1245143","DOI":"10.32322\/jhsm.1245143"},{"key":"2178_CR31","doi-asserted-by":"publisher","unstructured":"Branum, C., and Schiavenato, M., Can ChatGPT accurately answer a PICOT question? Assessing AI response to a clinical question. Nurse Educ. 48:231\u2013233, 2023. https:\/\/doi.org\/10.1097\/NNE.0000000000001436","DOI":"10.1097\/NNE.0000000000001436"},{"key":"2178_CR32","doi-asserted-by":"publisher","unstructured":"Wang, L. W., Miller, M. J., Schmitt, M. R., and Wen, F. K., Assessing readability formula differences with written health information materials: Application, results, and recommendations. Res Social Adm Pharm. 9:503\u2013516, 2013. https:\/\/doi.org\/10.1016\/j.sapharm.2012.05.009","DOI":"10.1016\/j.sapharm.2012.05.009"},{"key":"2178_CR33","first-page":"11880","volume":"31","author":"NA Shayegh","year":"2024","unstructured":"Shayegh NA, Byer D, Griffiths Y, Coleman PW, Deane LA, Tonkin J (2024) Assessing artificial intelligence responses to common patient questions regarding inflatable penile prostheses using a publicly available natural language processing tool (ChatGPT). Can J Urol 31:11880\u201311885","journal-title":"Can J Urol"},{"key":"2178_CR34","doi-asserted-by":"crossref","unstructured":"Mavragani, A., and Tsagarakis, K. P., Predicting referendum results in the Big Data Era. J Big Data. 6:3, 2019.","DOI":"10.1186\/s40537-018-0166-z"},{"key":"2178_CR35","doi-asserted-by":"publisher","unstructured":"Rovetta, A., Reliability of google trends: Analysis of the limits and potential of web infoveillance during COVID-19 pandemic and for future research. Front Res Metr Anal. 6:670226, 2021. https:\/\/doi.org\/10.3389\/frma.2021.670226","DOI":"10.3389\/frma.2021.670226"},{"key":"2178_CR36","doi-asserted-by":"publisher","unstructured":"Alibudbud, R., Google trends for health research: Its advantages, application, methodological considerations, and limitations in psychiatric and mental health infodemiology. Front Big Data. 6:1132764, 2023. https:\/\/doi.org\/10.3389\/fdata.2023.1132764","DOI":"10.3389\/fdata.2023.1132764"}],"container-title":["Journal of Medical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-025-02178-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10916-025-02178-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-025-02178-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,4]],"date-time":"2025-04-04T16:43:38Z","timestamp":1743785018000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10916-025-02178-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,3]]},"references-count":36,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["2178"],"URL":"https:\/\/doi.org\/10.1007\/s10916-025-02178-z","relation":{},"ISSN":["1573-689X"],"issn-type":[{"value":"1573-689X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,3]]},"assertion":[{"value":"30 October 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 March 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 April 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study did not include human subject research, and therefore, it did not require institutional ethics review board submission.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical statement"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing \u0130nterests"}}],"article-number":"43"}}