{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T06:16:57Z","timestamp":1778221017537,"version":"3.51.4"},"reference-count":48,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T00:00:00Z","timestamp":1778198400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Large language models (LLMs) hold considerable potential in medical and health education; however, their reliability and interpretability in highly sensitive areas and in decision-making remain unclear. This study focuses on four publicly available LLMs and systematically evaluates their applicability in fertility preservation scenarios for breast cancer patients, thereby providing guidance for targeted use.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This study utilizes Google Trends to identify and filter information on topics related to fertility preservation for breast cancer patients, and analyses the dialogue outputs of models such as GPT-5.4 Thinking, Gemini 3.0, DeepSeek-V3.2, and Microsoft Copilot. To ensure consistency in responses and the fairness of LLM baseline performance, only one response is generated per query, and no responses are generated repeatedly; all dialogues are submitted to the four large language models using standardized prompts. The study found that 26 fertility-preserving response outputs in breast cancer patients exhibited varying patterns, revealing characteristics relevant to fertility-preserving treatments and decision-making for breast cancer patients. The study utilized reliability assessment tools, including DISCERN, EQIP (Evaluation of Information Quality to Patients), GQS (Global Quality Score) and JAMA (JAMA benchmark criteria), a comprehensive assessment based on six widely used readability metrics [Automated Readability Index (ARI), Coleman\u2013Liau Index (CLI), Flesch\u2013Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Simple Measure of Gobbledygook (SMOG) with Flesch Reading Ease Score (FRES)].<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      The findings indicate that there are statistically significant differences in the reliability of various artificial intelligence programmes when it comes to providing highly sensitive, decision-intensive, multidisciplinary medical consultations regarding fertility preservation for breast cancer patients. The average intraclass correlation coefficients for all LLMs ranged from 0.715 to 0.978 (with all\n                      <jats:italic>p<\/jats:italic>\n                      -values &amp;lt; 0.001). Microsoft Copilot demonstrates superior performance in terms of information reliability and structural quality, DISCERN [56.5 (49.25, 61)], [EQIP65.0 (51.25, 75.0)], [GQS3.0 (3.0, 4.0)], JAMA [1.0 (1.0, 2.0)], with a higher score than GPT-5.4 Thinking, Gemini 3.0 and DeepSeek-V3.2, the model is capable of providing more reliable information and better decision-making support. The responses generated by all LLMs are too complex for the general public and fail to meet the recommended reading comprehension standards for years 6 to 8; the writing standards of most outputs are equivalent to those of secondary school education, or the reading level required for legal documents.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>This study reveals differences in the information provided by various LLMs regarding fertility preservation decisions for breast cancer patients, and recommends selecting a model suited to the specific clinical context; the Microsoft Copilot model demonstrated the best performance. Although LLMs demonstrate a certain degree of reliability when handling complex health enquiries, none have met the readability benchmark recommended for a year 6 reading level. Future research should focus on improving the reliability and readability of health information generated by LLMs to enhance comprehension among a wider audience.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2026.1845389","type":"journal-article","created":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T05:56:23Z","timestamp":1778219783000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["A centralized decision-making support consultation response for fertility preservation in breast cancer patients: benchmark performance of generative large language models in terms of reliability and readability"],"prefix":"10.3389","volume":"14","author":[{"given":"Jincheng","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Intensive Care Unit, Zhejiang Hospital","place":["Hangzhou, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yang Yang","family":"Zhan","sequence":"additional","affiliation":[{"name":"Department of Intensive Care Unit, Zhejiang Hospital","place":["Hangzhou, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yikuan","family":"Shen","sequence":"additional","affiliation":[{"name":"Department of Intensive Care Unit, Zhejiang Hospital","place":["Hangzhou, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Feng","family":"Cao","sequence":"additional","affiliation":[{"name":"Department of Intensive Care Unit, Zhejiang Hospital","place":["Hangzhou, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiayan","family":"Ling","sequence":"additional","affiliation":[{"name":"Department of Intensive Care Unit, Zhejiang Hospital","place":["Hangzhou, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2026,5,8]]},"reference":[{"key":"B1","volume-title":"Breast Cancer","year":"2024"},{"key":"B2","doi-asserted-by":"publisher","first-page":"e9304","DOI":"10.1002\/pon.9304","article-title":"Reproductive concerns among young adult women with breast cancer: a systematic review and meta-analysis","volume":"33","author":"Hu","year":"2024","journal-title":"Psycho-oncol."},{"key":"B3","doi-asserted-by":"publisher","first-page":"1946","DOI":"10.3390\/cancers15071946","article-title":"Experiences of female childhood cancer patients and survivors regarding information and counselling on gonadotoxicity risk and fertility preservation at diagnosis: a systematic review","volume":"15","author":"Clasen","year":"2023","journal-title":"Cancers"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1660","DOI":"10.1016\/j.ejso.2014.02.002","article-title":"Are breast cancer patients being adequately referred to fertility services?","volume":"40","author":"Panter-Brick","year":"2014","journal-title":"Eur J SurgOncol."},{"key":"B5","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1016\/j.mayocp.2019.10.009","article-title":"Fertility preservation in women: indications and options for therapy","volume":"95","author":"Hussein","year":"2020","journal-title":"Mayo Clinic Proceedings"},{"key":"B6","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1016\/j.annonc.2023.11.016","article-title":"Early breast cancer: ESMO Clinical Practice Guideline for diagnosis, treatment and follow-up","volume":"35","author":"Loibl","year":"2024","journal-title":"Ann Oncol."},{"key":"B7","doi-asserted-by":"publisher","first-page":"1488","DOI":"10.1200\/JCO-24-02782","article-title":"Fertility preservation in people with cancer: ASCO guideline update","volume":"43","author":"Su","year":"","journal-title":"J Clin Oncol: Off J Am Society Clin Oncol."},{"key":"B8","doi-asserted-by":"publisher","first-page":"1477898","DOI":"10.3389\/fmed.2024.1477898","article-title":"Large language models in patient education: a scoping review of applications in medicine","volume":"11","author":"Aydin","year":"2024","journal-title":"Front Med."},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.2196\/55138","article-title":"Characterizing the adoption and experiences of users of artificial intelligence-generated health information in the United States: cross-sectional questionnaire study","author":"Ayo-Ajibola","year":"","journal-title":"J Med Int Res."},{"key":"B10","doi-asserted-by":"publisher","first-page":"210","DOI":"10.5694\/mja2.52598","article-title":"Use of ChatGPT to obtain health information in Australia, 2024: insights from a nationally representative survey","volume":"222","author":"Ayre","year":"2025","journal-title":"Med J Australia"},{"key":"B11","doi-asserted-by":"publisher","first-page":"104620","DOI":"10.1016\/j.jbi.2024.104620","article-title":"Evaluation of ChatGPT-generated medical responses: a systematic review and meta-analysis","volume":"151","author":"Wei","year":"2024","journal-title":"J Biomed Info"},{"key":"B12","doi-asserted-by":"publisher","first-page":"e000759","DOI":"10.1136\/bmjonc-2025-000759","article-title":"Large language models in oncology: a review","volume":"4","author":"Chen","year":"2025","journal-title":"BMJ Oncol."},{"key":"B13","doi-asserted-by":"publisher","first-page":"e244630","DOI":"10.1001\/jamanetworkopen.2024.4630","article-title":"Quality of large language model responses to radiation oncology patient care questions","volume":"7","author":"Yalamanchili","year":"2024","journal-title":"JAMA Network Open."},{"key":"B14","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","year":"2024","journal-title":"NPJ Dig Med"},{"key":"B15","doi-asserted-by":"publisher","first-page":"e109583","DOI":"10.1371\/journal.pone.0109583","article-title":"The use of google trends in health care research: a systematic review","volume":"9","author":"Nuti","year":"2014","journal-title":"PloS ONE."},{"key":"B16","doi-asserted-by":"publisher","first-page":"1123","DOI":"10.3348\/kjr.2025.1522","article-title":"Minimum reporting items for clear evaluation of accuracy reports of large language models in healthcare (MI-CLEAR-LLM): 2025 updates","volume":"26","author":"Park","year":"2025","journal-title":"Korean J Radiol."},{"key":"B17","doi-asserted-by":"publisher","first-page":"569","DOI":"10.1136\/bmj.324.7337.569","article-title":"Examination of instruments used to rate quality of health information on the internet: chronicle of a voyage with an unclear destination","volume":"324","author":"Gagliardi","year":"2002","journal-title":"BMJ (Clin Res Ed)."},{"key":"B18","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices","volume":"53","author":"Charnock","year":"1999","journal-title":"J Epidemiol Community Health."},{"key":"B19","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","article-title":"Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information","volume":"7","author":"Moult","year":"2004","journal-title":"Health Expectations: Internat J Public Participation Health Care Health Policy."},{"key":"B20","doi-asserted-by":"publisher","first-page":"2070","DOI":"10.1111\/j.1572-0241.2007.01325.x","article-title":"A systematic review of patient inflammatory bowel disease information resources on the World Wide Web","volume":"102","author":"Bernard","year":"2007","journal-title":"Am J Gastroenterol."},{"key":"B21","doi-asserted-by":"publisher","first-page":"1244","DOI":"10.1001\/jama.1997.03540390074039","article-title":"Assessing, controlling, and assuring the quality of medical information on the internet: Caveant lector et viewor\u2013Let the reader and viewer beware","volume":"277","author":"Silberg","year":"1997","journal-title":"JAMA"},{"key":"B22","doi-asserted-by":"publisher","first-page":"503","DOI":"10.1016\/j.sapharm.2012.05.009","article-title":"Assessing readability formula differences with written health information materials: application, results, and recommendations","volume":"9","author":"Wang","year":"2013","journal-title":"Res Social Admin Pharm: RSAP"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1166","DOI":"10.1038\/s41390-023-02608-z","article-title":"The readability of parent information leaflets in paediatric studies","volume":"94","author":"Nash","year":"2023","journal-title":"Pediatric Res."},{"key":"B24","doi-asserted-by":"publisher","first-page":"2374373521998847","DOI":"10.1177\/2374373521998847","article-title":"Readability of patient education materials from high-impact medical journals: a 20-year analysis","volume":"8","author":"Rooney","year":"2021","journal-title":"J Patient Exp."},{"key":"B25","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1016\/j.imed.2025.03.002","article-title":"Evaluating large language models and agents in healthcare: key challenges in clinical applications","volume":"5","author":"Chen","year":"2025","journal-title":"Intell Med."},{"key":"B26","doi-asserted-by":"publisher","first-page":"e112","DOI":"10.2196\/jmir.4018","article-title":"Low health literacy and evaluation of online health information: a systematic review of the literature","volume":"17","author":"Diviani","year":"2015","journal-title":"J Medical Internet Res."},{"key":"B27","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1111\/imj.16607","article-title":"Can artificial intelligence improve patient educational material readability? A systematic review and narrative synthesis","volume":"55","author":"Nasra","year":"2025","journal-title":"Internal Med J"},{"key":"B28","article-title":"RephQA: evaluating readability of large language models in public health question answering","author":"Qiu","year":"2025","journal-title":"arXiv."},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1056\/AIoa2300015","article-title":"People over trust AI-generated medical responses and view them to be as valid as doctors, despite low accuracy","author":"Shekar","year":"2024","journal-title":"arXiv."},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01793-x","article-title":"Patient perceptions of empathy in physician and artificial intelligence chatbot responses to patient questions about cancer","author":"Chen","year":"2025"},{"key":"B31","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1038\/s41591-024-03425-5","article-title":"The TRIPOD-LLM reporting guideline for studies using large language models","volume":"31","author":"Gallifant","year":"2025","journal-title":"Nature Med."},{"key":"B32","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1038\/s41586-025-09422-z","article-title":"DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning","volume":"645","author":"DeepSeek-AI","year":"2025","journal-title":"Nature."},{"key":"B33","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1038\/s41405-025-00349-1","article-title":"Assessing the power of AI: a comparative evaluation of large language models in generating patient education materials in dentistry","volume":"11","author":"Sivaramakrishnan","year":"2025","journal-title":"BDJ Open."},{"key":"B34","doi-asserted-by":"publisher","first-page":"3828","DOI":"10.3390\/nu17243828","article-title":"Efficacy of large language models in providing evidence-based patient education for celiac disease: a comparative analysis","volume":"17","author":"Bertin","year":"2025","journal-title":"Nutrients"},{"key":"B35","doi-asserted-by":"publisher","first-page":"108656","DOI":"10.1016\/j.pec.2025.108656","article-title":"Readability of written information for patients across 30 years: a systematic review of systematic reviews","volume":"135","author":"Okuhara","year":"2025","journal-title":"Patient Educ Counsel"},{"key":"B36","doi-asserted-by":"publisher","first-page":"103722","DOI":"10.1016\/j.breast.2024.103722","article-title":"A systematic review and meta-analysis of English language online patient education materials in breast cancer: is readability the only story?","volume":"75","author":"Gu","year":"2024","journal-title":"Breast (Edinburgh, Scotland)"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1097\/MD.0000000000042135","article-title":"Assessing the quality and readability of patient education materials on chemotherapy cardiotoxicity from artificial intelligence chatbots: an observational cross-sectional study","author":"Stephenson-Moe","year":"2025","journal-title":"Medicine."},{"key":"B38","doi-asserted-by":"publisher","first-page":"100594","DOI":"10.1016\/j.ajpc.2023.100594","article-title":"Readability and reliability of online patient education materials about statins","volume":"16","author":"Ngo","year":"2023","journal-title":"Am J Prev Cardiol"},{"key":"B39","doi-asserted-by":"publisher","first-page":"111013","DOI":"10.1016\/j.compbiomed.2025.111013","article-title":"A comprehensive evaluation of large language models for information extraction from unstructured electronic health records in residential aged care","volume":"197","author":"Vithanag","year":"2025","journal-title":"Comp Biol Med."},{"key":"B40","doi-asserted-by":"publisher","first-page":"2260","DOI":"10.1097\/CORR.0000000000002668","article-title":"Can artificial intelligence improve the readability of patient education materials?","volume":"481","author":"Kirchner","year":"2023","journal-title":"Clin Orthopaed Rel Res."},{"key":"B41","doi-asserted-by":"publisher","first-page":"582","DOI":"10.4048\/jbc.2023.26.e44","article-title":"Multidisciplinary shared decision making for fertility preservation in young women with breast cancer","volume":"26","author":"Baek","year":"2023","journal-title":"J Breast Cancer."},{"key":"B42","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1016\/j.fertnstert.2025.08.017","article-title":"Fertility preservation for cancer: referral guidelines, treatment options, and specific considerations","volume":"124","author":"Kalluru","year":"2025","journal-title":"Fertil Steril."},{"key":"B43","doi-asserted-by":"publisher","first-page":"1664","DOI":"10.1016\/j.annonc.2020.09.006","article-title":"Fertility preservation and post-treatment pregnancies in post-pubertal cancer patients: ESMO clinical practice guidelines\u2020","volume":"31","author":"Lambertini","year":"","journal-title":"Ann Oncol: Official J European Society Med Oncol."},{"key":"B44","doi-asserted-by":"publisher","first-page":"1619274","DOI":"10.3389\/fpubh.2025.1619274","article-title":"HEAL-Summ: a lightweight and ethical framework for accessible summarization of health information","volume":"13","author":"Fisher","year":"2025","journal-title":"Front Public Health."},{"key":"B45","doi-asserted-by":"publisher","first-page":"e2400149","DOI":"10.1200\/CCI-24-00149","article-title":"Patients facing large language models in oncology: a narrative review","volume":"8","author":"Raynaud","year":"2024","journal-title":"JCO Clin Cancer Info."},{"key":"B46","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1038\/s44401-024-00011-2","article-title":"Current and future state of evaluation of large language models for medical summarization tasks","volume":"2","author":"Croxford","year":"2025","journal-title":"NPJ Health Sys."},{"key":"B47","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1038\/s41746-025-01963-x","article-title":"The evaluation illusion of large language models in medicine","volume":"8","author":"Agrawal","year":"2025","journal-title":"NPJ Digital Med"},{"key":"B48","doi-asserted-by":"publisher","first-page":"1447","DOI":"10.1016\/j.annonc.2025.09.001","article-title":"ESMO guidance on the use of large language models in clinical practice (ELCAP)","volume":"36","author":"Wong","year":"2025","journal-title":"Ann Oncol: Official J Eur Society Med Oncol."}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1845389\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T05:56:24Z","timestamp":1778219784000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1845389\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,8]]},"references-count":48,"alternative-id":["10.3389\/fpubh.2026.1845389"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1845389","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,8]]},"article-number":"1845389"}}