{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T10:54:20Z","timestamp":1791975260173,"version":"4.4.0"},"reference-count":20,"publisher":"Wiley","issue":"4","license":[{"start":{"date-parts":[[2025,11,24]],"date-time":"2025-11-24T00:00:00Z","timestamp":1763942400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,11,24]],"date-time":"2025-11-24T00:00:00Z","timestamp":1763942400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["obgyn.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["BJOG"],"published-print":{"date-parts":[[2026,3]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Although large language models are increasingly used in clinical and research settings, the validity of the information they provide remains uncertain. This study aimed to evaluate the accuracy, consistency, and reliability of three large language models\u2014ChatGPT 4.0, DeepSeek R1, and Gemini 2.0\u2014in answering cervical cancer\u2010related questions based on the ESGO\/ESTRO\/ESP guidelines.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Design<\/jats:title>\n                    <jats:p>Prospective, comparative in silico benchmarking study.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Setting<\/jats:title>\n                    <jats:p>Fondazione Policlinico Universitario A. Gemelli, Rome, Italy. Population or Sample: Fifty questions derived from the ESGO\/ESTRO\/ESP (European Society of Gynaecologic Oncology\/European Society for Radiotherapy and Oncology\/European Society of Pathology) Guidelines for Cervical Cancer.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Each question was submitted simultaneously to ChatGPT 4.0, DeepSeek R1, and Gemini 2.0, and re\u2010entered twice to assess response repeatability. Answers were evaluated for accuracy using a Global Quality Score (GQS) from 1 (poor) to 5 (completely accurate). Consistency (intra\u2010model response stability) and reliability (alignment with guidelines) were assessed using binary classification. Main Outcome Measures: Median GQS, percentage of GQS 5 responses, consistency between repeated answers, and reliability.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      ChatGPT 4.0 achieved the highest performance, with 42% of responses rated GQS 5, followed by Gemini 2.0 (30%) and DeepSeek R1 (28%). DeepSeek R1 and Gemini 2.0 scored lower in median GQS (3.50) compared to ChatGPT 4.0 (4.00). Response consistency varied significantly, with ChatGPT 4.0 and DeepSeek R1 showing differences from Gemini 2.0 (\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.034 and\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.044, respectively). No significant difference was observed in reliability (\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.602).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>All models demonstrated suboptimal accuracy in aligning with clinical guidelines. ChatGPT 4.0 was the most accurate and consistent whereas DeepSeek R1 underperformed. Despite similar reliability across models, expert oversight remains essential to ensure safe clinical application and prevent misinformation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1111\/1471-0528.70095","type":"journal-article","created":{"date-parts":[[2025,11,25]],"date-time":"2025-11-25T05:25:36Z","timestamp":1764048336000},"page":"771-778","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Assessing the Accuracy of Large Language Models on European Guidelines for Cervical Cancer: An In Silico Benchmarking Study"],"prefix":"10.1111","volume":"133","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9791-9464","authenticated-orcid":false,"given":"Matteo","family":"Pavone","sequence":"first","affiliation":[{"name":"UOC Ginecologia Oncologica, Dipartimento di Scienze per la Salute Della Donna e del Bambino e di Sanit\u00e0 Pubblica, Fondazione Policlinico Universitario A. Gemelli, IRCCS  Rome Italy"},{"name":"Research Institute Against Digestive Cancer, IRCAD Strasbourg  Strasbourg France"},{"name":"Institute of Image\u2010Guided Surgery, IHU Strasbourg  Strasbourg France"},{"name":"ICube, Laboratory of Engineering, Computer Science and Imaging, Department of Robotics, Imaging, Teledetection and Healthcare Technologies, University of Strasbourg, CNRS, UMR 7357  Strasbourg France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chiara","family":"Innocenzi","sequence":"additional","affiliation":[{"name":"UOC Ginecologia Oncologica, Dipartimento di Scienze per la Salute Della Donna e del Bambino e di Sanit\u00e0 Pubblica, Fondazione Policlinico Universitario A. Gemelli, IRCCS  Rome Italy"},{"name":"Research Institute Against Digestive Cancer, IRCAD Strasbourg  Strasbourg France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicola","family":"Macellari","sequence":"additional","affiliation":[{"name":"UOC Ginecologia Oncologica, Dipartimento di Scienze per la Salute Della Donna e del Bambino e di Sanit\u00e0 Pubblica, Fondazione Policlinico Universitario A. Gemelli, IRCCS  Rome Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chiara","family":"Cantarini","sequence":"additional","affiliation":[{"name":"UOC Ginecologia Oncologica, Dipartimento di Scienze per la Salute Della Donna e del Bambino e di Sanit\u00e0 Pubblica, Fondazione Policlinico Universitario A. Gemelli, IRCCS  Rome Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Massimo","family":"Criscione","sequence":"additional","affiliation":[{"name":"UOC Ginecologia Oncologica, Dipartimento di Scienze per la Salute Della Donna e del Bambino e di Sanit\u00e0 Pubblica, Fondazione Policlinico Universitario A. Gemelli, IRCCS  Rome Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Rosati","sequence":"additional","affiliation":[{"name":"UOC Ginecologia Oncologica, Dipartimento di Scienze per la Salute Della Donna e del Bambino e di Sanit\u00e0 Pubblica, Fondazione Policlinico Universitario A. Gemelli, IRCCS  Rome Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lise","family":"Lecointre","sequence":"additional","affiliation":[{"name":"Institute of Image\u2010Guided Surgery, IHU Strasbourg  Strasbourg France"},{"name":"ICube, Laboratory of Engineering, Computer Science and Imaging, Department of Robotics, Imaging, Teledetection and Healthcare Technologies, University of Strasbourg, CNRS, UMR 7357  Strasbourg France"},{"name":"Department of Gynecologic Surgery University Hospitals of Strasbourg  Strasbourg France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antonella","family":"Carcagn\u00ec","sequence":"additional","affiliation":[{"name":"Facility of Epidemiology and Biostatistics \u2010 Gemelli Generator, Fondazione Policlinico Universitario A. Gemelli IRCCS  Rome Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Barbara","family":"Costantini","sequence":"additional","affiliation":[{"name":"UOC Ginecologia Oncologica, Dipartimento di Scienze per la Salute Della Donna e del Bambino e di Sanit\u00e0 Pubblica, Fondazione Policlinico Universitario A. Gemelli, IRCCS  Rome Italy"},{"name":"UniCamillus, International Medical University  Rome Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jacques","family":"Marescaux","sequence":"additional","affiliation":[{"name":"Research Institute Against Digestive Cancer, IRCAD Strasbourg  Strasbourg France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anna","family":"Fagotti","sequence":"additional","affiliation":[{"name":"UOC Ginecologia Oncologica, Dipartimento di Scienze per la Salute Della Donna e del Bambino e di Sanit\u00e0 Pubblica, Fondazione Policlinico Universitario A. Gemelli, IRCCS  Rome Italy"},{"name":"Universit\u00e0 Cattolica del Sacro Cuore  Rome Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Francesco","family":"Fanfani","sequence":"additional","affiliation":[{"name":"UOC Ginecologia Oncologica, Dipartimento di Scienze per la Salute Della Donna e del Bambino e di Sanit\u00e0 Pubblica, Fondazione Policlinico Universitario A. Gemelli, IRCCS  Rome Italy"},{"name":"Universit\u00e0 Cattolica del Sacro Cuore  Rome Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Cibula","sequence":"additional","affiliation":[{"name":"Department of Obstetrics and Gynecology First Faculty of Medicine, Charles University, General University Hospital in Prague  Prague Czechia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Denis","family":"Querleu","sequence":"additional","affiliation":[{"name":"UOC Ginecologia Oncologica, Dipartimento di Scienze per la Salute Della Donna e del Bambino e di Sanit\u00e0 Pubblica, Fondazione Policlinico Universitario A. Gemelli, IRCCS  Rome Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1727-904X","authenticated-orcid":false,"given":"Nicol\u00f2","family":"Bizzarri","sequence":"additional","affiliation":[{"name":"UOC Ginecologia Oncologica, Dipartimento di Scienze per la Salute Della Donna e del Bambino e di Sanit\u00e0 Pubblica, Fondazione Policlinico Universitario A. Gemelli, IRCCS  Rome Italy"},{"name":"Universit\u00e0 Cattolica del Sacro Cuore  Rome Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2025,11,24]]},"reference":[{"key":"e_1_2_12_2_1","doi-asserted-by":"publisher","DOI":"10.1097\/AS9.0000000000000021"},{"issue":"3","key":"e_1_2_12_3_1","first-page":"1009","article-title":"Artificial Intelligence and ChatGPT in Abdominopelvic Surgery: A Systematic Review of Applications and Impact","volume":"38","author":"Goglia M.","year":"2024","journal-title":"Vivo Athens Greece"},{"key":"e_1_2_12_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/ijgo.15861"},{"key":"e_1_2_12_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ygyno.2023.11.008"},{"key":"e_1_2_12_6_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpubh.2023.1166120"},{"key":"e_1_2_12_7_1","doi-asserted-by":"publisher","DOI":"10.1136\/ijgc-2023-005162"},{"key":"e_1_2_12_8_1","doi-asserted-by":"publisher","DOI":"10.1111\/1471-0528.17641"},{"key":"e_1_2_12_9_1","doi-asserted-by":"publisher","DOI":"10.52054\/FVVO.16.4.052"},{"key":"e_1_2_12_10_1","doi-asserted-by":"publisher","DOI":"10.1111\/1471-0528.17929"},{"key":"e_1_2_12_11_1","article-title":"ChatGPT, Google and Healthcare Institution Sources of Postoperative Patient Instructions","author":"Daungsupawong H.","year":"2024","journal-title":"BJOG: An International Journal of Obstetrics & Gynaecology [Internet]"},{"key":"e_1_2_12_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ygyno.2024.07.007"},{"key":"e_1_2_12_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.surfin.2024.104081"},{"key":"e_1_2_12_14_1","doi-asserted-by":"publisher","DOI":"10.1111\/1471-0528.15429"},{"key":"e_1_2_12_15_1","doi-asserted-by":"publisher","DOI":"10.1136\/ijgc-2023-004429"},{"key":"e_1_2_12_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.gore.2024.101477"},{"key":"e_1_2_12_17_1","doi-asserted-by":"publisher","DOI":"10.2196\/45312"},{"key":"e_1_2_12_18_1","doi-asserted-by":"publisher","DOI":"10.2196\/46599"},{"key":"e_1_2_12_19_1","doi-asserted-by":"publisher","DOI":"10.1093\/asj\/sjad128"},{"key":"e_1_2_12_20_1","doi-asserted-by":"publisher","DOI":"10.1177\/20552076241310630"},{"key":"e_1_2_12_21_1","doi-asserted-by":"publisher","DOI":"10.1111\/ijd.17729"}],"container-title":["BJOG: An International Journal of Obstetrics &amp; Gynaecology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/obgyn.onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/1471-0528.70095","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/obgyn.onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/1471-0528.70095","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/obgyn.onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/1471-0528.70095","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,27]],"date-time":"2026-09-27T23:47:09Z","timestamp":1790552829000},"score":1,"resource":{"primary":{"URL":"https:\/\/obgyn.onlinelibrary.wiley.com\/doi\/10.1111\/1471-0528.70095"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,24]]},"references-count":20,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,3]]}},"alternative-id":["10.1111\/1471-0528.70095"],"URL":"https:\/\/doi.org\/10.1111\/1471-0528.70095","archive":["Portico"],"relation":{},"ISSN":["1470-0328","1471-0528"],"issn-type":[{"value":"1470-0328","type":"print"},{"value":"1471-0528","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,24]]},"assertion":[{"value":"2025-05-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-11","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}