{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T10:40:17Z","timestamp":1792233617656,"version":"4.4.0"},"reference-count":24,"publisher":"Wiley","license":[{"start":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T00:00:00Z","timestamp":1792195200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"},{"start":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T00:00:00Z","timestamp":1792195200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2019YFC0840703"],"award-info":[{"award-number":["2019YFC0840703"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["obgyn.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Intl J Gynecology &amp; Obste"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Large language models are widely used in clinical care. Their reliability in specialized gynecological diseases such as premature ovarian insufficiency (POI) has not been fully verified. This study aimed to systematically evaluate and compare the accuracy and completeness of three mainstream large language models when answering standardized clinical questions about POI.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We developed 34 standardized questions covering five core domains of POI according to international clinical guidelines. Two senior obstetricians and gynecologists independently scored responses using a 5\u2010point Likert scale. Inter\u2010rater reliability was assessed using Cohen's kappa coefficient, intraclass correlation coefficient, and mean absolute error. Pairwise between\u2010model score differences were computed and visualized using item\u2010level heatmaps.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>All three models showed high and steady performance. Accuracy scores ranged from 4.59 to 4.84, and completeness scores ranged from 4.43 to 4.65. Inter\u2010rater agreement was \u201cgood\u201d to \u201c excellent\u201d, with exact agreement between 91.18% and 97.06% and Cohen's kappa between 0.675 and 0.915. GPT\u20105.5 performed better in the area of clinical manifestations and treatment. DeepSeek\u2010R1 showed greater stability, defined as maintaining consistently high scores with limited score fluctuation, in etiology\u2010 and counseling\u2010related questions. Item\u2010level heatmaps revealed that most individual\u2010item differences were within \u00b10.5 points, whereas pairwise differences in overall mean scores fell within \u00b10.5, and some individual\u2010item comparisons exceeded this range, with no obvious systematic bias.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>GPT\u20105.5, Claude Opus\u20104.7, and DeepSeek\u2010R1 demonstrated favorable accuracy and completeness in answering standardized POI questions; overall performance was similar despite deviations across several individual questions. Item\u2010level difference heatmaps further confirmed the consistency and reliability of the three models. These models can serve as reliable auxiliary tools for clinical information retrieval and patient education under professional supervision, meaning that all content generated by artificial intelligence must be checked by qualified clinicians before clinical use.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/ijgo.71422","type":"journal-article","created":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T09:40:59Z","timestamp":1792230059000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Responses of large language models to common questions on premature ovarian insufficiency: A comparative analysis"],"prefix":"10.1002","author":[{"given":"Jing","family":"Zhe","sequence":"first","affiliation":[{"name":"Department of Obstetrics and Gynecology, Center for Reproductive Medicine, Guiyang Maternal and Child Health Care Hospital Guiyang Children's Hospital  Guiyang China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-1148-8506","authenticated-orcid":false,"given":"Dan","family":"Zheng","sequence":"additional","affiliation":[{"name":"Department of Maternity Health Care, Guiyang Maternal and Child Health Care Hospital Guiyang Children's Hospital  Guiyang China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,10,17]]},"reference":[{"issue":"1","key":"e_1_2_11_2_1","doi-asserted-by":"crossref","first-page":"320","DOI":"10.1186\/s13287-022-03012-w","article-title":"Cell\u2010free fat extract improves ovarian function and fertility in mice with premature ovarian insufficiency","volume":"13","author":"Liu M","year":"2022","journal-title":"Stem Cell Res Ther"},{"issue":"6","key":"e_1_2_11_3_1","first-page":"116","article-title":"Examining the effect of Kuntai capsule combined with HRT in the treatment of premature ovarian insufficiency: a systematic review and meta\u2010analysis","volume":"29","author":"Lin F","year":"2025","journal-title":"Exp Ther Med"},{"key":"e_1_2_11_4_1","doi-asserted-by":"crossref","DOI":"10.3389\/fendo.2021.711902","article-title":"Human umbilical cord mesenchymal stem cell\u2010derived exosomes improve ovarian function and proliferation of premature ovarian insufficiency by regulating the hippo signaling pathway","volume":"12","author":"Li Z","year":"2021","journal-title":"Front Endocrinol"},{"key":"e_1_2_11_5_1","doi-asserted-by":"crossref","DOI":"10.3389\/fendo.2022.1025018","article-title":"The effects and mechanism of taxanes on chemotherapy\u2010associated ovarian damage: a review of current evidence","volume":"13","author":"Wu C","year":"2022","journal-title":"Front Endocrinol"},{"issue":"1","key":"e_1_2_11_6_1","doi-asserted-by":"crossref","first-page":"1266","DOI":"10.1186\/s40359-025-03536-9","article-title":"The relationship between stress, anxiety and depression with sexual function among women with premature ovarian insufficiency in the west of Iran","volume":"13","author":"Jenabi E","year":"2025","journal-title":"BMC Psychol"},{"issue":"4","key":"e_1_2_11_7_1","doi-asserted-by":"crossref","first-page":"310","DOI":"10.2174\/0118715303313887240624071238","article-title":"A glimpse of research trends and frontiers in the etiology of premature ovarian insufficiency via bibliometric analysis","volume":"25","author":"Li D","year":"2025","journal-title":"Endocr Metab Immune Disord Drug Targets"},{"issue":"1","key":"e_1_2_11_8_1","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1186\/s40942-025-00764-4","article-title":"Global perspectives of ophthalmologists on artificial intelligence adoption in clinical practice","volume":"11","author":"Ferro Desideri L","year":"2025","journal-title":"Int J Retina Vitreous"},{"key":"e_1_2_11_9_1","doi-asserted-by":"crossref","DOI":"10.2196\/75103","article-title":"Evaluating and improving syndrome differentiation thinking ability in large language models: method development study","volume":"13","author":"Chen C","year":"2025","journal-title":"JMIR Med Inform"},{"issue":"1","key":"e_1_2_11_10_1","doi-asserted-by":"crossref","DOI":"10.1186\/s12911-026-03371-x","article-title":"Evaluation of large language models on pediatric asthma: a comparative study of Claude3\u2010Opus, Gemini 2.0, ChatGPT\u20104o, and DeepSeek\u2010a cross\u2010sectional questionnaire study","volume":"26","author":"Hang YQ","year":"2026","journal-title":"BMC Med Inform Decis Mak"},{"issue":"2","key":"e_1_2_11_11_1","doi-asserted-by":"crossref","first-page":"850","DOI":"10.1002\/ijgo.70455","article-title":"Using artificial intelligence as a technological tool in gynecologic and obstetric health: a narrative literature review","volume":"172","author":"Dos Santos GG","year":"2026","journal-title":"Int J Gynaecol Obstet"},{"issue":"1","key":"e_1_2_11_12_1","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1097\/jnr.0b013e3182829b26","article-title":"Menarche and menstruation through the eyes of pubescent students in eastern Taiwan: implications in sociocultural influence and gender differences issues","volume":"21","author":"Chang YT","year":"2013","journal-title":"J Nurs Res"},{"issue":"2","key":"e_1_2_11_13_1","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.fertnstert.2024.11.007","article-title":"Evidence\u2010based guideline: premature ovarian insufficiency","volume":"123","author":"Panay N","year":"2025","journal-title":"Fertil Steril"},{"issue":"4","key":"e_1_2_11_14_1","doi-asserted-by":"crossref","DOI":"10.1093\/hropen\/hoae065","article-title":"Evidence\u2010based guideline: premature ovarian insufficiency","volume":"2024","author":"Panay N","year":"2024","journal-title":"Hum Reprod Open"},{"issue":"1","key":"e_1_2_11_15_1","doi-asserted-by":"crossref","DOI":"10.1080\/09513590.2024.2360085","article-title":"Psychological stressors involved in the pathogenesis of premature ovarian insufficiency and potential intervention measures","volume":"40","author":"Xu YP","year":"2024","journal-title":"Gynecol Endocrinol"},{"key":"e_1_2_11_16_1","doi-asserted-by":"crossref","DOI":"10.2196\/53888","article-title":"Exploring the use and implications of AI in sexual and reproductive health and rights: protocol for a scoping review","volume":"13","author":"Tamrat T","year":"2024","journal-title":"JMIR Res Protoc"},{"issue":"1","key":"e_1_2_11_17_1","doi-asserted-by":"crossref","first-page":"26301","DOI":"10.1038\/s41598-025-04704-y","article-title":"Application of machine learning algorithms and SHAP explanations to predict fertility preference among reproductive women in Somalia","volume":"15","author":"Sani J","year":"2025","journal-title":"Sci Rep"},{"issue":"1","key":"e_1_2_11_18_1","doi-asserted-by":"crossref","first-page":"37824","DOI":"10.1038\/s41598-025-21689-w","article-title":"Evaluating the performance of large language models versus human researchers on real world complex medical queries","volume":"15","author":"Idan D","year":"2025","journal-title":"Sci Rep"},{"issue":"1","key":"e_1_2_11_19_1","doi-asserted-by":"crossref","DOI":"10.1186\/s12903-025-07153-1","article-title":"Comparison of the information of generative artificial intelligence large language models and professional guidelines regarding nutritional advice for orthodontic patients","volume":"25","author":"Sokmen T","year":"2025","journal-title":"BMC Oral Health"},{"key":"e_1_2_11_20_1","doi-asserted-by":"crossref","DOI":"10.2196\/59505","article-title":"Multimodal large language models in health care: applications, challenges, and future outlook","volume":"26","author":"AlSaad R","year":"2024","journal-title":"J Med Internet Res"},{"issue":"1","key":"e_1_2_11_21_1","doi-asserted-by":"crossref","first-page":"1513","DOI":"10.1186\/s12913-025-13668-8","article-title":"Comparative feasibility of reasoning and non\u2010reasoning large language models for gynecologic cancer emergency care","volume":"25","author":"Kim J","year":"2025","journal-title":"BMC Health Serv Res"},{"issue":"3","key":"e_1_2_11_22_1","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pdig.0001029","article-title":"Cardiology knowledge assessment of retrieval\u2010augmented open versus proprietary large language models","volume":"5","author":"Tarabanis C","year":"2026","journal-title":"PLOS Digit Health"},{"issue":"11","key":"e_1_2_11_23_1","doi-asserted-by":"crossref","first-page":"1159","DOI":"10.3390\/bioengineering11111159","article-title":"Artificial intelligence diagnosing of oral lichen planus: a comparative study","volume":"11","author":"Yu S","year":"2024","journal-title":"Bioengineering"},{"key":"e_1_2_11_24_1","doi-asserted-by":"crossref","DOI":"10.1016\/j.ejogrb.2026.114960","article-title":"Assessment of AI\u2010generated responses to common pregnancy questions: a blinded expert evaluation","volume":"318","author":"Atay AO","year":"2026","journal-title":"Eur J Obstet Gynecol Reprod Biol"},{"key":"e_1_2_11_25_1","doi-asserted-by":"crossref","DOI":"10.1016\/j.ijmedinf.2025.106034","article-title":"Comparing ChatGPT and physicians' answers to endometriosis questions on reddit: a blind expert evaluation","volume":"203","author":"Beaulieu C","year":"2025","journal-title":"Int J Med Inform"}],"container-title":["International Journal of Gynecology &amp; Obstetrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/obgyn.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/ijgo.71422","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/obgyn.onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/ijgo.71422","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/obgyn.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/ijgo.71422","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T09:41:03Z","timestamp":1792230063000},"score":1,"resource":{"primary":{"URL":"https:\/\/obgyn.onlinelibrary.wiley.com\/doi\/10.1002\/ijgo.71422"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,17]]},"references-count":24,"alternative-id":["10.1002\/ijgo.71422"],"URL":"https:\/\/doi.org\/10.1002\/ijgo.71422","archive":["Portico"],"relation":{},"ISSN":["0020-7292","1879-3479"],"issn-type":[{"value":"0020-7292","type":"print"},{"value":"1879-3479","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,17]]},"assertion":[{"value":"2026-05-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-21","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-10-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"ijgo.71422"}}