{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,9]],"date-time":"2026-11-09T21:14:33Z","timestamp":1794258873572,"version":"4.9.0"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2025,6,3]],"date-time":"2025-06-03T00:00:00Z","timestamp":1748908800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,6,3]],"date-time":"2025-06-03T00:00:00Z","timestamp":1748908800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Impot Res"],"published-print":{"date-parts":[[2025,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>This study aims to evaluate and compare the performance of artificial intelligence chatbots by assessing the reliability and quality of the information they provide regarding penis enhancement (PE). Search trends for keywords related to PE were determined using Google Trends (<jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/trends.google.com\" ext-link-type=\"uri\">https:\/\/trends.google.com<\/jats:ext-link>) and Semrush (<jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/www.semrush.com\" ext-link-type=\"uri\">https:\/\/www.semrush.com<\/jats:ext-link>). Data covering a ten-year period was analyzed, taking into account regional trends and changes in search volume. Based on these trends, 25 questions were selected and categorized into three groups: general information (GI), surgical treatment (ST) and myths\/misconceptions (MM). These questions were posed to three advanced chatbots: ChatGPT-4, Gemini Pro and Llama 3.1. Responses from each model were analyzed for readability using the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease Score (FRES), while the quality of the responses was evaluated using the Ensuring Quality Information for Patients (EQIP) tool and the Modified DISCERN Score. All chatbot responses exhibited difficulty in readability and understanding according to FKGL and FRES, with no statistically significant differences among them (FKGL: p\u2009=\u20090.167; FRES: p\u2009=\u20090.366). Llama achieved the highest median Modified DISCERN score (4 [IQR:1]), significantly outperforming ChatGPT (3 [IQR:0]) and Gemini (3 [IQR:2]) (p\u2009&lt;\u20090.001). Pairwise comparisons showed no significant difference between ChatGPT and Gemini (p\u2009=\u20090.070), but Llama was superior to both (p\u2009&lt;\u20090.001). In EQIP scores, Llama also scored highest (73.8\u2009\u00b1\u20092.2), significantly surpassing ChatGPT (68.7\u2009\u00b1\u20092.1) and Gemini (54.2\u2009\u00b1\u20091.3) (p\u2009&lt;\u20090.001). Across categories, Llama consistently achieved higher EQIP scores (GI:71.1\u2009\u00b1\u20091.6; ST: 73.6\u2009\u00b1\u20094.1; MM: 76.3\u2009\u00b1\u20092.1) and Modified DISCERN scores (GI:4 [IQR:0]; ST:4 [IQR:1]; MM:3 [IQR:1]) compared to ChatGPT (EQIP: GI:68.4\u2009\u00b1\u20091.1; ST: 65.7\u2009\u00b1\u20092.2; MM:71.1\u2009\u00b1\u20091.7; Modified DISCERN: GI:3 [IQR:1]; ST:3 [IQR:1]; MM:3 [IQR:0]) and Gemini (EQIP: GI:55.2\u2009\u00b1\u20091.4; ST:55.2\u2009\u00b1\u20091.6; MM:2.6\u2009\u00b1\u20092.5; Modified DISCERN: GI:1 [IQR:2]; ST:1 [IQR:2]; MM:3 [IQR:0]) (p\u2009&lt;\u20090.001). This study highlights Llama\u2019s superior reliability in providing PE-related health information, though all chatbots struggled with readability.<\/jats:p>","DOI":"10.1038\/s41443-025-01098-3","type":"journal-article","created":{"date-parts":[[2025,6,3]],"date-time":"2025-06-03T08:33:40Z","timestamp":1748939620000},"page":"558-563","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Evaluating AI chatbots in penis enhancement information: a comparative analysis of readability, reliability and quality"],"prefix":"10.1038","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7349-9952","authenticated-orcid":false,"given":"Mehmet Vehbi","family":"Kayra","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hakan","family":"Anil","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ilturk","family":"Ozdogan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Suhail Mohamed Amin","family":"Baradia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Serdar","family":"Toksoz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,6,3]]},"reference":[{"key":"1098_CR1","doi-asserted-by":"publisher","first-page":"995","DOI":"10.1097\/PRS.0000000000007249","volume":"146","author":"KS Vyas","year":"2020","unstructured":"Vyas KS, Abu-Ghname A, Banuelos J, Morrison SD, Manrique O. Aesthetic augmentation phalloplasty: a systematic review of techniques and outcomes. Plast Reconstr Surg. 2020;146:995\u20131006.","journal-title":"Plast Reconstr Surg"},{"key":"1098_CR2","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1038\/s41443-022-00580-6","volume":"34","author":"C Zaccaro","year":"2022","unstructured":"Zaccaro C, Subir\u00e0 D, L\u00f3pez-Diez I, Manfredi C, Ascensios-Vargas JP, Moncada-Iribarren I. History and future perspectives of male aesthetic genital surgery. Int J Impot Res. 2022;34:327\u201331.","journal-title":"Int J Impot Res"},{"key":"1098_CR3","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1111\/bju.15926","volume":"131","author":"M G\u00fcl","year":"2023","unstructured":"G\u00fcl M, Altintas E, \u00d6zkent MS, Fenner A, Serefoglu EC. Depictions of penises in historical paintings reflect changing perceptions of the ideal penis size. BJU Int. 2023;131:581\u20137.","journal-title":"BJU Int"},{"key":"1098_CR4","doi-asserted-by":"publisher","first-page":"460","DOI":"10.1016\/j.sxmr.2021.10.003","volume":"10","author":"A Soubra","year":"2022","unstructured":"Soubra A, Natale C, Brimley S, Hellstrom WJ. Revelations on men who seek penile augmentation surgery: a review. Sex Med Rev. 2022;10:460\u20137.","journal-title":"Sex Med Rev"},{"key":"1098_CR5","doi-asserted-by":"publisher","first-page":"978","DOI":"10.1111\/bju.13010","volume":"115","author":"D Veale","year":"2015","unstructured":"Veale D, Miles S, Bramley S, Muir G, Hodsoll J. Am I normal? A systematic review and construction of nomograms for flaccid and erect penis length and circumference in up to 15 521 men. BJU Int. 2015;115:978\u201386.","journal-title":"BJU Int"},{"key":"1098_CR6","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1037\/1524-9220.7.3.129","volume":"7","author":"J Lever","year":"2006","unstructured":"Lever J, Frederick DA, Peplau LA. Does size matter? Men\u2019s and women\u2019s views on penis size across the lifespan. Psychol Men Masc. 2006;7:129\u201343.","journal-title":"Psychol Men Masc"},{"key":"1098_CR7","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1080\/0092623X.2020.1787279","volume":"47","author":"BM King","year":"2020","unstructured":"King BM. Average-size erect penis: fiction, fact, and the need for counseling. J Sex Marital Ther. 2020;47:80\u20139.","journal-title":"J Sex Marital Ther"},{"key":"1098_CR8","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1038\/s41443-022-00646-5","volume":"36","author":"A \u00c7a\u011flayan","year":"2024","unstructured":"\u00c7a\u011flayan A, G\u00fcl M. #Penisenlargement on Instagram: a mixed-methods study. Int J Impot Res. 2024;36:218\u201322.","journal-title":"Int J Impot Res"},{"key":"1098_CR9","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1007\/s10439-022-03121-w","volume":"51","author":"MR King","year":"2023","unstructured":"King MR. The future of AI in medicine: a perspective from a Chatbot. Ann Biomed Eng. 2023;51:291\u20135.","journal-title":"Ann Biomed Eng"},{"key":"1098_CR10","first-page":"1","volume":"3","author":"C Bulla","year":"2020","unstructured":"Bulla C, Parushetti C, Teli A, Aski S, Koppad S. A review of AI based medical assistant chatbot. Res Appl Web Dev Des. 2020;3:1\u201314.","journal-title":"Res Appl Web Dev Des"},{"key":"1098_CR11","doi-asserted-by":"publisher","first-page":"e2336483","DOI":"10.1001\/jamanetworkopen.2023.36483","volume":"6","author":"RS Goodman","year":"2023","unstructured":"Goodman RS, Patrinely JR, Stone CA, Zimmerman E, Donald RR, Chang SS, et al. Accuracy and reliability of chatbot responses to physician questions. JAMA Netw Open. 2023;6:e2336483.","journal-title":"JAMA Netw Open"},{"key":"1098_CR12","doi-asserted-by":"publisher","DOI":"10.1093\/sexmed\/qfae036","volume":"12","author":"MF \u015eahin","year":"2024","unstructured":"\u015eahin MF, Kele\u015f A, \u00d6zcan R, Do\u011fan \u00c7, Topka\u00e7 EC, Akg\u00fcl M, Yaz\u0131c\u0131 CM. Evaluation of information accuracy and clarity: ChatGPT responses to the most frequently asked questions about premature ejaculation. Sex Med. 2024;12:qfae036.","journal-title":"Sex Med"},{"key":"1098_CR13","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-024-02056-0","volume":"48","author":"MF \u015eahin","year":"2024","unstructured":"\u015eahin MF, Ate\u015f H, Kele\u015f A, \u00d6zcan R, Do\u011fan \u00c7, Akg\u00fcl M, et al. Responses of five different artificial intelligence chatbots to the top searched queries about erectile dysfunction: a comparative analysis. J Med Syst. 2024;48:38.","journal-title":"J Med Syst"},{"key":"1098_CR14","doi-asserted-by":"publisher","first-page":"e141","DOI":"10.1016\/j.fertnstert.2024.07.520","volume":"122","author":"C Lee","year":"2024","unstructured":"Lee C, Chae HJ, Kim HM, Lee H, Choi S, Min HS. Enhancing infertility care with a ChatGPT-based chatbot: integrating clinical and community insights for improved patient support. Fertil Steril. 2024;122:e141\u2013e142.","journal-title":"Fertil Steril"},{"key":"1098_CR15","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1016\/j.urology.2023.07.010","volume":"180","author":"BB Whiles","year":"2023","unstructured":"Whiles BB, Bird VG, Canales BK, DiBianco JM, Terry RS. Caution! AI bot has entered the patient chat: ChatGPT has limitations in providing accurate urologic healthcare advice. Urology. 2023;180:278\u201384.","journal-title":"Urology"},{"key":"1098_CR16","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1016\/j.eururo.2023.07.004","volume":"85","author":"D Musheyev","year":"2024","unstructured":"Musheyev D, Pan A, Loeb S, Kabarriti AE. How well do artificial intelligence chatbots respond to the top search queries about urological malignancies? Eur Urol. 2024;85:13\u201316.","journal-title":"Eur Urol"},{"key":"1098_CR17","doi-asserted-by":"publisher","first-page":"688","DOI":"10.1097\/JU.0000000000003615","volume":"210","author":"R Davis","year":"2023","unstructured":"Davis R, Eppler M, Ayo-Ajibola O, Loh-Doyle JC, Nabhani J, Samplaski M, et al. Evaluating the effectiveness of artificial intelligence\u2013powered large language models application in disseminating appropriate and readable health information in urology. J Urol. 2023;210:688\u201394.","journal-title":"J Urol"},{"key":"1098_CR18","unstructured":"Wang H, Liu C, Xi N, Qiang Z, Zhao S, Qin B, et al. Huatuo: Tuning llama model with Chinese medical knowledge. arXiv:2304.06975. 2023. https:\/\/arxiv.org\/abs\/2304.06975#."},{"key":"1098_CR19","doi-asserted-by":"publisher","first-page":"93","DOI":"10.4018\/978-1-5225-7365-4.ch008","volume-title":"Advanced methodologies and technologies in modern education delivery","author":"JC Brewer","year":"2019","unstructured":"Brewer JC. Measuring text readability using reading level. Advanced methodologies and technologies in modern education delivery. IGI Global; 2019. p. 93\u2013103."},{"key":"1098_CR20","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","volume":"7","author":"B Moult","year":"2004","unstructured":"Moult B, Franck LS, Brady H. Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information. Health Expect. 2004;7:165\u201375.","journal-title":"Health Expect"},{"key":"1098_CR21","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","volume":"53","author":"D Charnock","year":"1999","unstructured":"Charnock D, Shepperd S, Needham G, Gann R. DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53:105\u201311.","journal-title":"J Epidemiol Community Health"},{"key":"1098_CR22","doi-asserted-by":"publisher","first-page":"899","DOI":"10.3899\/jrheum.111114","volume":"39","author":"AG Singh","year":"2012","unstructured":"Singh AG, Singh S, Singh PP. YouTube for information on rheumatoid arthritis\u2014a wakeup call? J Rheumatol. 2012;39:899\u2013903.","journal-title":"J Rheumatol"},{"key":"1098_CR23","doi-asserted-by":"publisher","DOI":"10.1016\/j.fjurol.2024.102636","volume":"34","author":"O Perrot","year":"2024","unstructured":"Perrot O, Schirmann A, Vidart A, Guillot-Tantay C, Izard V, Lebret T, et al. Chatbots vs andrologists: testing 25 clinical cases. Fr J Urol. 2024;34:102636.","journal-title":"Fr J Urol"},{"key":"1098_CR24","doi-asserted-by":"crossref","unstructured":"Warren CJ, Payne NG, Edmonds VS, Voleti SS, Choudry MM, Punjani N, et al. Quality of chatbot information related to benign prostatic hyperplasia. Prostate. 2025.","DOI":"10.1002\/pros.24814"},{"key":"1098_CR25","doi-asserted-by":"crossref","unstructured":"Naz R, Delia\u011fa H, Kaya M. Assessment of hypospadias patient education using large language models. J Pediatr Surg Nurs. 2024:23320249241282449.","DOI":"10.1177\/23320249241282449"},{"key":"1098_CR26","doi-asserted-by":"publisher","first-page":"879","DOI":"10.21037\/tau-23-635","volume":"13","author":"BD Simon","year":"2024","unstructured":"Simon BD, Gelikman DG, Turkbey B. Evaluating the efficacy of artificial intelligence chatbots in urological health: insights for urologists on patient interactions with large language models. Transl Androl Urol. 2024;13:879\u201390.","journal-title":"Transl Androl Urol"},{"key":"1098_CR27","doi-asserted-by":"publisher","first-page":"e47479","DOI":"10.2196\/47479","volume":"25","author":"HL Walker","year":"2023","unstructured":"Walker HL, Ghani S, Kuemmerli C, Nebiker CA, M\u00fcller BP, Raptis DA, et al. Reliability of medical information provided by ChatGPT: assessment against clinical guidelines and patient information quality instrument. J Med Internet Res. 2023;25:e47479.","journal-title":"J Med Internet Res"},{"key":"1098_CR28","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1177\/1062860617751639","volume":"33","author":"L Daraz","year":"2018","unstructured":"Daraz L, Morrow AS, Ponce OJ, Farah W, Katabi A, Majzoub A, et al. Readability of online health information: a meta-narrative systematic review. Am J Med Qual. 2018;33:487\u201392.","journal-title":"Am J Med Qual"},{"key":"1098_CR29","doi-asserted-by":"publisher","first-page":"7482","DOI":"10.3390\/jcm13247482","volume":"13","author":"J Schmidt","year":"2024","unstructured":"Schmidt J, Lichy I, Kurz T, Peters R, Hofbauer S, Plage H, et al. ChatGPT as a support tool for informed consent and preoperative patient education prior to penile prosthesis implantation. J Clin Med. 2024;13:7482.","journal-title":"J Clin Med"}],"container-title":["International Journal of Impotence Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41443-025-01098-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41443-025-01098-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41443-025-01098-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,6]],"date-time":"2025-09-06T17:21:21Z","timestamp":1757179281000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41443-025-01098-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,3]]},"references-count":29,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,7]]}},"alternative-id":["1098"],"URL":"https:\/\/doi.org\/10.1038\/s41443-025-01098-3","relation":{},"ISSN":["0955-9930","1476-5489"],"issn-type":[{"value":"0955-9930","type":"print"},{"value":"1476-5489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,3]]},"assertion":[{"value":"12 January 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 May 2025","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 May 2025","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 June 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"Not necessary as there is no clinical study.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}