{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T17:14:26Z","timestamp":1790010866882,"version":"4.0.1"},"reference-count":32,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Large language models (LLMs) have been proposed as decision-support tools in assisted reproductive technology (ART), but it remains unclear whether posttraining alignment strategies translate into clinically acceptable decision support. Outcome-based benchmarks may reward token-level correctness while overlooking the reasoning quality that clinicians rely on.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to evaluate whether 4 mainstream alignment paradigms for medical LLMs (supervised fine-tuning [SFT], direct preference optimization [DPO], group relative policy optimization [GRPO], and in-context learning [ICL]) produce comparable algorithmic and clinical alignment (SFT vs GRPO) when used for infertility diagnosis and treatment planning.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n This retrospective single-center study used 8201 deidentified electronic health records from West China Second University Hospital, collected between January 2020 and December 2022 (mean age 31.79, SD 4.63 y). All 4 strategies were built on a shared open-source biomedical backbone. Evaluation comprised the following 2 tiers: (1) automatic field-level metrics (accuracy, macro-\n F<\/jats:italic>\n 1<\/jats:sub>\n , and mean absolute error [MAE]) on 5 structured decision fields (infertility type, initial diagnosis, ART strategy, controlled ovarian stimulation [COS] regimen, and gonadotropin starting dose); and (2) blinded independent expert review by 2 reproductive medicine specialists on 100 paired cases across 4 clinical dimensions (reasoning capability, diagnostic accuracy, treatment feasibility, and hallucination). Automatic field-level evaluation included all 4 strategies, whereas blinded expert review was restricted to the SFT versus GRPO contrast.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n GRPO achieved the highest average automatic performance (eg, infertility type accuracy=92.57%, COS regimen accuracy=62.36%, ART strategy accuracy=76.49%, and gonadotropin dose MAE=44.94). However, in blinded expert review, the conservative SFT baseline showed directionally higher expert ratings than GRPO on reasoning capability and treatment feasibility; diagnostic-accuracy differences were not significant. In the 3-way best-response comparison including the original physician-charted plan, the SFT baseline was selected as the best response in 51.2% (102.3\/200) of cases compared with 26.2% (52.4\/200) for GRPO, and 22.6% (45.3\/200) for the charted plan. This result reflects preference within the standardized review format, not evidence that model-generated decisions are clinically superior to physician decision-making. Hallucination rates were 15% (GRPO) and 18.5% (SFT), indicating that higher automatic performance did not eliminate clinically unsupported content and that neither model is ready for clinical deployment. Subgroup analyses showed GRPO improved\n F<\/jats:italic>\n 1<\/jats:sub>\n in in vitro fertilization (IVF) and preimplantation genetic testing (PGT) but decreased\n F<\/jats:italic>\n 1<\/jats:sub>\n in intracytoplasmic sperm injection (ICSI) cases, where male-factor information was largely captured only in unstructured fields.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Outcome-based metrics alone are insufficient proxies for clinical utility in ART decision support. Algorithmic improvement and clinical alignment suggest a possible decoupling; a phenomenon we term the alignment paradox. However, because the clinical review was based on 2 reproductive medicine specialists with marginal interrater agreement, the clinical-alignment findings should be interpreted as exploratory. External multicenter validation is required before clinical deployment.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/97221","type":"journal-article","created":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T16:45:34Z","timestamp":1790009134000},"page":"e97221-e97221","source":"Crossref","is-referenced-by-count":0,"title":["The Alignment Paradox of Medical Large Language Models in Infertility Care: Decoupling Algorithmic Improvement From Clinical Decision-Making Quality"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-3489-9540","authenticated-orcid":false,"given":"Dou","family":"Liu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4148-0800","authenticated-orcid":false,"given":"Ying","family":"Long","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5811-5527","authenticated-orcid":false,"given":"Sophia","family":"Zuoqiu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-6433-5904","authenticated-orcid":false,"given":"Di","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8136-9816","authenticated-orcid":false,"given":"Kang","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-9327-2118","authenticated-orcid":false,"given":"Kaipeng","family":"Xie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-8808-1926","authenticated-orcid":false,"given":"Runze","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-1612-5672","authenticated-orcid":false,"given":"Yiting","family":"Lin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0464-3840","authenticated-orcid":false,"given":"Hanyi","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4596-5407","authenticated-orcid":false,"given":"Rong","family":"Yin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-7483-5376","authenticated-orcid":false,"given":"Tian","family":"Tang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,9,21]]},"reference":[{"key":"R1","unstructured":"Infertility prevalence estimates, 1990\u20132021. World Health Organization; 2023. URL: https:\/\/www.who.int\/publications\/i\/item\/978920068315 [Accessed 12-09-2026]"},{"issue":"1","key":"R2","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1111\/dmcn.15332","article-title":"Assisted reproductive technology: short\u2010 and long\u2010term outcomes","volume":"65","author":"Graham","journal-title":"Develop Med Child Neuro"},{"issue":"2","key":"R3","doi-asserted-by":"publisher","first-page":"70","DOI":"10.4103\/0974-1208.86080","article-title":"Ovarian hyperstimulation syndrome","volume":"4","author":"Kumar","journal-title":"J Hum Reprod Sci"},{"issue":"12","key":"R4","doi-asserted-by":"publisher","first-page":"2506","DOI":"10.1016\/j.ajem.2019.12.020","article-title":"Decision fatigue in the emergency department: how does emergency physician decision making change over an eight-hour shift?","volume":"38","author":"Zheng","journal-title":"Am J Emerg Med"},{"key":"R5","unstructured":"DeepSeek-AIGuo D Yang D Zhang H et al. DeepSeek-R1: incentivizing reasoning capability in llms via reinforcement learning. arXiv. Preprint posted online on Jan 2025. [doi: 10.48550\/arXiv.2501.12948]"},{"key":"R6","doi-asserted-by":"crossref","unstructured":"Rafailov R Sharma A Mitchell E Manning CD Ermon S Finn C . Direct preference optimization: your language model is secretly a reward model. Presented at: Advances in Neural Information Processing Systems 36; 53728-53741; New Orleans, Louisiana, USA. Dec 15, 2023.[doi: 10.52202\/075280-2338]","DOI":"10.52202\/075280-2338"},{"issue":"6","key":"R7","doi-asserted-by":"publisher","first-page":"2727","DOI":"10.1109\/TMI.2026.3661001","article-title":"Med-R1: reinforcement learning for generalizable medical reasoning in vision-language models","volume":"45","author":"Lai","journal-title":"IEEE Trans Med Imaging"},{"key":"R8","unstructured":"Shao Z Wang P Zhu Q et al. DeepSeekMath: pushing the limits of mathematical reasoning in open language models. arXiv. Preprint posted online on Feb 2024. [doi: 10.48550\/arXiv.2402.03300]"},{"key":"R9","unstructured":"Dao A Vu DB . AlphaMaze: enhancing large language models\u2019 spatial intelligence via GRPO. arXiv. Preprint posted online on Feb 2025. [doi: 10.48550\/arXiv.2502.14669]"},{"key":"R10","unstructured":"Chen J Cai Z Ji K et al. HuatuoGPT-o1, towards medical complex reasoning with LLMs. arXiv. Preprint posted online on Dec 2024. [doi: 10.48550\/arXiv.2412.18925]"},{"issue":"3","key":"R11","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1038\/s41591-024-03423-7","article-title":"Toward expert-level medical question answering with large language models","volume":"31","author":"Singhal","journal-title":"Nat Med"},{"key":"R12","unstructured":"Team L Xu W Chan HP et al. Lingshu: a generalist foundation model for unified multimodal medical understanding and reasoning. arXiv. Preprint posted online on Jun 2025. [doi: 10.48550\/arXiv.2506.07044]"},{"issue":"1","key":"R13","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01684-1","article-title":"Evaluating large language model workflows in clinical decision support for triage and referral and diagnosis","volume":"8","author":"Gaber","journal-title":"NPJ Digit Med"},{"issue":"2","key":"R14","doi-asserted-by":"publisher","first-page":"210","DOI":"10.1080\/24725579.2026.2651352","article-title":"Automated procedural analysis via video-language models for AI-assisted nursing skills assessment","volume":"16","author":"Chang","journal-title":"IISE Transactions on Healthcare Systems Engineering"},{"key":"R15","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-95-0211-0_36","article-title":"Evaluating the application of chatgpt in outpatient triage guidance: a comparative study","author":"Liu","journal-title":"Proc 22nd Congr Int Ergon Assoc"},{"key":"R16","unstructured":"Liu D Long Y Zuoqiu S Tang T Yin R . Evaluating the feasibility and accuracy of large language models for medical history-taking in obstetrics and gynecology. arXiv. Preprint posted online on Mar 2025. [doi: 10.48550\/arXiv.2504.00061]"},{"issue":"4","key":"R17","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1002\/hcs2.61","article-title":"Large language models in health care: development, applications, and challenges","volume":"2","author":"Yang","journal-title":"Health Care Sci"},{"issue":"9","key":"R18","doi-asserted-by":"publisher","first-page":"2613","DOI":"10.1038\/s41591-024-03097-1","article-title":"Evaluation and mitigation of the limitations of large language models in clinical decision-making","volume":"30","author":"Hager","journal-title":"Nat Med"},{"key":"R19","doi-asserted-by":"crossref","unstructured":"Liu J Wang W Ma Z et al. MedChain: bridging the gap between LLM agents and clinical practice with interactive sequence. Presented at: Advances in Neural Information Processing Systems 38; Dec 2-7, 2025; San Diego, California, USA and Mexico City, Mexico. [doi: 10.52202\/085713-2498]","DOI":"10.52202\/085713-2498"},{"issue":"1","key":"R20","doi-asserted-by":"publisher","DOI":"10.1186\/s13000-024-01464-7","article-title":"Challenges and barriers of using large language models (LLM) such as ChatGPT for diagnostic medicine with a focus on digital pathology - a recent scoping review","volume":"19","author":"Ullah","journal-title":"Diagn Pathol"},{"issue":"9","key":"R21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3561048","article-title":"Explainable AI (XAI): core ideas, techniques, and solutions","volume":"55","author":"Dwivedi","journal-title":"ACM Comput Surv"},{"key":"R22","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2022.03.009","article-title":"The explainability paradox: challenges for xAI in digital pathology","volume":"133","author":"Evans","journal-title":"Future Generation Computer Systems"},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2022.02.021","article-title":"Interpretable clinical time-series modeling with intelligent feature selection for early prediction of antimicrobial multidrug resistance","volume":"133","author":"Mart\u00ednez-Ag\u00fcero","journal-title":"Future Generation Computer Systems"},{"key":"R24","unstructured":"Wei J Bosma M Zhao VY et al. Finetuned language models are zero-shot learners. arXiv. Preprint posted online on Sep 2022. [doi: 10.48550\/arXiv.2109.01652]"},{"key":"R25","unstructured":"Wang Y Yue X Chen W . Critique fine-tuning: learning to critique is more effective than learning to imitate. arXiv. Preprint posted online on Jan 2025. [doi: 10.48550\/arXiv.2501.17703]"},{"key":"R26","doi-asserted-by":"crossref","unstructured":"Dai D Chen P Ekbote C Liang P . QoQ-Med: building multimodal clinical foundation models with domain-aware GRPO training. Presented at: Advances in Neural Information Processing Systems 38; Dec 2-7, 2025; San Diego, California, USA and Mexico City, Mexico. [doi: 10.52202\/085713-1256]","DOI":"10.52202\/085713-1256"},{"issue":"1","key":"R27","doi-asserted-by":"publisher","DOI":"10.2196\/85206","article-title":"Reliability of large language model generated clinical reasoning in assisted reproductive technology: blinded comparative evaluation study","volume":"28","author":"Liu","journal-title":"J Med Internet Res"},{"key":"R28","unstructured":"OpenBioLLM-8B. Hugging Face. URL: https:\/\/huggingface.co\/aaditya\/Llama3-OpenBioLLM-8B [Accessed 29-07-2025]"},{"key":"R29","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2024.107437","article-title":"Vision-BioLLM: large vision language model for visual dialogue in biomedical imagery","volume":"103","author":"AlShibli","journal-title":"Biomed Signal Process Control"},{"issue":"5","key":"R30","doi-asserted-by":"publisher","first-page":"2486","DOI":"10.3390\/app15052486","article-title":"Fine-tuning a personalized OpenBioLLM using offline reinforcement learning","volume":"15","author":"Shi","journal-title":"Applied Sciences"},{"key":"R31","unstructured":"Schulman J Wolski F Dhariwal P Radford A Klimov O . Proximal policy optimization algorithms. arXiv. Preprint posted online on Jul 2017. [doi: 10.48550\/arXiv.1707.06347]"},{"key":"R32","unstructured":"Schulman J Moritz P Levine S Jordan M Abbeel P . High-dimensional continuous control using generalized advantage estimation. arXiv. Preprint posted online on Jun 2018. [doi: 10.48550\/arXiv.1506.02438]"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T16:45:40Z","timestamp":1790009140000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e97221"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,21]]},"references-count":32,"URL":"https:\/\/doi.org\/10.2196\/97221","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,21]]},"article-number":"v28i1e97221"}}