{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T11:32:22Z","timestamp":1782905542033,"version":"3.54.5"},"reference-count":17,"publisher":"Georg Thieme Verlag KG","issue":"02","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2026,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>This study aimed to explore the performance of ChatGPT version 4.0 (GPT-4) and Gemini Advanced (Gemini) large language models (LLMs) in addressing common patient questions after gynecology surgery with regards to accuracy, relevance, helpfulness to the average patient, and readability.<\/jats:p>\n                  <jats:p>In this cross-sectional study, the two LLMs were prompted to generate answers to postoperative patient questions after gynecologic surgery. Postoperative patient questions were developed to simulate common patient questions after gynecologic surgery, based on expert opinion and compiled from anonymous posters on Reddit (r\/endometriosis). Questions were focused on six topics: endometriosis, vaginal bleeding, bowel\/bladder function, incision care, resumption of activities, and sexual function. Questions were asked in a systematic three-step submission process with the memory reset after each query. Responses were then blinded and independently assessed for accuracy and relevance on a 5-Point Likert scale by four board-certified gynecologic surgeons with fellowship training in gynecologic surgery. Readability of the answers was calculated with the Flesch Kincaid grade level calculator. Responses were also evaluated by three clinic nurses.<\/jats:p>\n                  <jats:p>A total of 41 questions were posed to GPT-4 and Gemini three times. The responses were independently evaluated by four surgeons and three nurses leading to a total of 1,968 evaluations for accuracy, relevance, helpfulness to the average patient, and readability. Surgeons and nurses graded Gemini responses as more accurate (4.23 vs. 4.03, p\u2009=\u20090.015) and helpful (4.37 vs. 4.21, p\u2009=\u20090.025) than GPT-4 responses. Responses from both models were similarly found to be relevant or very relevant (4.45 vs. 4.36, p\u2009=\u20090.2). Most responses by GPT-4 (85%) and Gemini (87%) were consistent across all questions. The average reading level for GPT-4 and Gemini responses were 11th and 10th grade, above the recommended 6th grade reading level for patient information.<\/jats:p>\n                  <jats:p>GPT-4 and Gemini provided overall accurate, relevant, and helpful responses to common postoperative patient questions for gynecologic surgery. Gemini outperformed GPT-4 in both accuracy and helpfulness and had objectively more readable responses.<\/jats:p>","DOI":"10.1055\/a-2818-1611","type":"journal-article","created":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T23:55:59Z","timestamp":1771977359000},"page":"172-176","source":"Crossref","is-referenced-by-count":1,"title":["Evaluation of ChatGPT and Gemini in Answering Patient Questions after Gynecologic Surgery"],"prefix":"10.1055","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4444-7237","authenticated-orcid":false,"given":"Petra C.","family":"Voigt","sequence":"additional","affiliation":[{"name":"Division of Minimally Invasive Gynecologic Surgery, Department of Obstetrics and Gynecology, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rhea S.","family":"Sharma","sequence":"additional","affiliation":[{"name":"Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Angela","family":"Chaudhari","sequence":"additional","affiliation":[{"name":"Division of Minimally Invasive Gynecologic Surgery, Department of Obstetrics and Gynecology, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Susan","family":"Tsai","sequence":"additional","affiliation":[{"name":"Division of Minimally Invasive Gynecologic Surgery, Department of Obstetrics and Gynecology, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Magdy P.","family":"Milad","sequence":"additional","affiliation":[{"name":"Division of Minimally Invasive Gynecologic Surgery, Department of Obstetrics and Gynecology, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Linda C.","family":"Yang","sequence":"additional","affiliation":[{"name":"Division of Minimally Invasive Gynecologic Surgery, Department of Obstetrics and Gynecology, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"194","published-online":{"date-parts":[[2026,3,30]]},"reference":[{"issue":"07","key":"ref1","doi-asserted-by":"crossref","first-page":"1073","DOI":"10.1377\/hlthaff.2018.05509","article-title":"Physicians' well-being linked to in-basket messages generated by algorithms in electronic health records","volume":"38","author":"M Tai-Seale","year":"2019","journal-title":"Health Aff (Millwood)"},{"issue":"06","key":"ref2","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"J W Ayers","year":"2023","journal-title":"JAMA Intern Med"},{"issue":"08","key":"ref3","doi-asserted-by":"crossref","first-page":"1165","DOI":"10.1097\/MAO.0000000000003125","article-title":"Quality improvement in otologic surgery postoperative instructions","volume":"42","author":"C E Sluder","year":"2021","journal-title":"Otol Neurotol"},{"issue":"05","key":"ref4","doi-asserted-by":"crossref","first-page":"1627","DOI":"10.1055\/a-2628-8408","article-title":"Comparing the performances of a 54-year-old computer-based consultation to ChatGPT-4o","volume":"16","author":"E B Verdi","year":"2025","journal-title":"Appl Clin Inform"},{"issue":"05","key":"ref5","doi-asserted-by":"crossref","first-page":"AIdbp2300192","DOI":"10.1056\/AIdbp2300192","article-title":"GPT versus resident physicians\u2014a benchmark based on official board scores","volume":"1","author":"U Katz","year":"2024","journal-title":"NEJM AI"},{"issue":"03","key":"ref6","doi-asserted-by":"crossref","first-page":"718","DOI":"10.1055\/a-2565-9155","article-title":"Automating responses to patient portal messages using generative AI","volume":"16","author":"A Kaur","year":"2025","journal-title":"Appl Clin Inform"},{"issue":"06","key":"ref7","doi-asserted-by":"crossref","first-page":"696","DOI":"10.1016\/j.ajog.2023.03.009","article-title":"The exciting potential for ChatGPT in obstetrics and gynecology","volume":"228","author":"A Gr\u00fcnebaum","year":"2023","journal-title":"Am J Obstet Gynecol"},{"issue":"01","key":"ref8","doi-asserted-by":"crossref","first-page":"e1","DOI":"10.1097\/AOG.0000000000005446","article-title":"The genie is out of the bottle: what ChatGPT can and cannot do for medical professionals","volume":"143","author":"P Morales-Ramirez","year":"2024","journal-title":"Obstet Gynecol"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1101\/2023.08.28.23294730","article-title":"Assessing racial and ethnic bias in text generation for healthcare-related tasks by ChatGPT(1)","author":"J J Hanna","year":"2023","journal-title":"medRxiv"},{"issue":"02","key":"ref10","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1016\/j.jacr.2023.09.011","article-title":"Enhancing patient communication with Chat-GPT in radiology: evaluating the efficacy and readability of answers to common imaging-related questions","volume":"21","author":"E B Gordon","year":"2024","journal-title":"J Am Coll Radiol"},{"issue":"05","key":"ref11","first-page":"e59960","article-title":"Assessing the responses of large language models (ChatGPT-4, Gemini, and Microsoft Copilot) to frequently asked questions in breast imaging: a study on readability and accuracy","volume":"16","author":"M Tepe","year":"2024","journal-title":"Cureus"},{"key":"ref12","doi-asserted-by":"crossref","first-page":"111901","DOI":"10.1016\/j.ijporl.2024.111901","article-title":"The utility and accuracy of ChatGPT in providing post-operative instructions following tonsillectomy: a pilot study","volume":"179","author":"S Dhar","year":"2024","journal-title":"Int J Pediatr Otorhinolaryngol"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1101\/2024.08.08.24311699","article-title":"Leveraging large language models in gynecologic oncology: a systematic review of current applications and challenges","author":"A Mudrik","journal-title":"medRxiv"},{"issue":"08","key":"ref14","doi-asserted-by":"crossref","first-page":"1154","DOI":"10.1111\/1471-0528.17746","article-title":"ChatGPT compared with Google Search and healthcare institution as sources of postoperative patient instructions after gynecological surgery","volume":"131","author":"R Meyer","year":"2024","journal-title":"BJOG"},{"issue":"06","key":"ref15","doi-asserted-by":"crossref","first-page":"1790","DOI":"10.1007\/s11695-023-06603-5","article-title":"Assessing the accuracy of responses by the language model ChatGPT to questions regarding bariatric surgery","volume":"33","author":"J S Samaan","year":"2023","journal-title":"Obes Surg"},{"issue":"01","key":"ref16","first-page":"44","article-title":"Large language models in biomedicine and healthcare","volume":"1","author":"J Zhou","year":"2025","journal-title":"Artif Intell"},{"key":"ref17","doi-asserted-by":"crossref","first-page":"20S","DOI":"10.1097\/AOG.0000000000005851.31","article-title":"Closing the gap: using chatbots to improve the readability of patient education materials in urogynecology","volume":"145","author":"B Belmonte","year":"2025","journal-title":"Obstet Gynecol"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/a-2818-1611.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,30]],"date-time":"2026-03-30T23:05:55Z","timestamp":1774911955000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/a-2818-1611"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,24]]},"references-count":17,"journal-issue":{"issue":"02","published-online":{"date-parts":[[2026,3,16]]},"published-print":{"date-parts":[[2026,3]]}},"URL":"https:\/\/doi.org\/10.1055\/a-2818-1611","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,2,24]]},"published":{"date-parts":[[2026,2,24]]}}}