{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T17:02:43Z","timestamp":1778518963111,"version":"3.51.4"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2025,12,1]],"date-time":"2025-12-01T00:00:00Z","timestamp":1764547200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"},{"start":{"date-parts":[[2025,12,1]],"date-time":"2025-12-01T00:00:00Z","timestamp":1764547200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"}],"funder":[{"name":"Severance Hospital Research fund for Clinical excellenc","award":["C-2024-0041"],"award-info":[{"award-number":["C-2024-0041"]}]},{"name":"Bio&Medical Technology Development Program of the National Research Foundation","award":["MSIT; RS-2024- 00509289"],"award-info":[{"award-number":["MSIT; RS-2024- 00509289"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Healthc Inform Res"],"published-print":{"date-parts":[[2026,6]]},"DOI":"10.1007\/s41666-025-00218-4","type":"journal-article","created":{"date-parts":[[2025,12,1]],"date-time":"2025-12-01T17:33:48Z","timestamp":1764610428000},"page":"498-522","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Susceptibility of Large Language Models to User-Driven Factors in Medical Queries"],"prefix":"10.1007","volume":"10","author":[{"given":"Kyung Ho","family":"Lim","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ujin","family":"Kang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiang","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jin Sung","family":"Kim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Young-Chul","family":"Jung","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9223-3172","authenticated-orcid":false,"given":"Sangjoon","family":"Park","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2501-038X","authenticated-orcid":false,"given":"Byung-Hoon","family":"Kim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,12,1]]},"reference":[{"issue":"1","key":"218_CR1","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1038\/s43856-023-00370-1","volume":"3","author":"J Clusmann","year":"2023","unstructured":"Clusmann J, Kolbinger FR, Muti HS, Carrero ZI, Eckardt J-N, Laleh NG et al (2023) The future landscape of large language models in medicine. Commun Med 3(1):141","journal-title":"Commun Med"},{"issue":"1","key":"218_CR2","doi-asserted-by":"publisher","first-page":"8236","DOI":"10.1038\/s41467-024-52415-1","volume":"15","author":"CY Williams","year":"2024","unstructured":"Williams CY, Miao BY, Kornblith AE, Butte AJ (2024) Evaluating the use of large language models to provide clinical recommendations in the Emergency Department. Nat Commun 15(1):8236","journal-title":"Nat Commun"},{"key":"218_CR3","doi-asserted-by":"publisher","first-page":"1477898","DOI":"10.3389\/fmed.2024.1477898","volume":"11","author":"S Aydin","year":"2024","unstructured":"Aydin S, Karabacak M, Vlachos V, Margetis K (2024) Large language models in patient education: a scoping review of applications in medicine. Front Med Lausanne 11:1477898","journal-title":"Front Med Lausanne"},{"key":"218_CR4","doi-asserted-by":"crossref","unstructured":"Khalifa M, Albadawy M, Iqbal U (2024) Advancing clinical decision support: the role of artificial intelligence across six domains. Comput. Methods Programs Biomed\u00a0Update 100142","DOI":"10.1016\/j.cmpbup.2024.100142"},{"issue":"1","key":"218_CR5","doi-asserted-by":"publisher","first-page":"1391","DOI":"10.1186\/s12909-024-06399-7","volume":"24","author":"E Br\u00fcgge","year":"2024","unstructured":"Br\u00fcgge E, Ricchizzi S, Arenbeck M, Keller MN, Schur L, Stummer W et al (2024) Large language models improve clinical decision making of medical students through patient simulation and structured feedback: a randomized controlled trial. BMC Med Educ 24(1):1391","journal-title":"BMC Med Educ"},{"issue":"2","key":"218_CR6","doi-asserted-by":"publisher","first-page":"121","DOI":"10.3349\/ymj.2024.0126","volume":"66","author":"C Han","year":"2024","unstructured":"Han C, Jung YJ, Park JE, Chung WY, Yoon D (2024) Artificial intelligence-based early prediction of acute respiratory failure in the emergency department using biosignal and clinical data. Yonsei Med J 66(2):121","journal-title":"Yonsei Med J"},{"key":"218_CR7","doi-asserted-by":"crossref","unstructured":"Park I, Park JH, Koo YH, Koo C-H, Koo B-W, Kim J-H et al (2024) Feasibility of a machine learning classifier for predicting post-induction hypotension in non-cardiac surgery. Yonsei Med J 66(3):160","DOI":"10.3349\/ymj.2024.0020"},{"key":"218_CR8","doi-asserted-by":"crossref","unstructured":"Chen H, Fang Z, Singla Y, Dredze M (2025) Benchmarking large language models on answering and explaining challenging medical questions. In: Proceedings of the 2025 conference of the Nations of the Americas chapter of the association for computational linguistics: human language technologies, vol 1: Long Papers.\u00a0Association for Computational Linguistics, pp 3563\u20133599","DOI":"10.18653\/v1\/2025.naacl-long.182"},{"key":"218_CR9","doi-asserted-by":"crossref","unstructured":"Liu J, Zhou P, Hua Y, Chong D, Tian Z, Liu A et al (2024) Benchmarking large language models on cmexam-a comprehensive chinese medical exam dataset. Adv Neural Inform Process Syst 36:52430\u201352452","DOI":"10.52202\/075280-2283"},{"key":"218_CR10","doi-asserted-by":"publisher","unstructured":"Van Veen D, Van Uden C, Blankemeier L, Delbrouck JB, Aali A, Bluethgen C et al (2023) Clinical text summarization: adapting large language models can outperform human experts. Research Square. https:\/\/doi.org\/10.21203\/rs.3.rs-3483777\/v1","DOI":"10.21203\/rs.3.rs-3483777\/v1"},{"key":"218_CR11","unstructured":"McDuff D, Schaekermann M, Tu T, Palepu A, Wang A, Garrison J et al (2023) Towards accurate differential diagnosis with large language models. Nature, pp 1\u20137"},{"key":"218_CR12","doi-asserted-by":"publisher","unstructured":"Nair V, Schumacher E, Kannan A (2023) Generating medically-accurate summaries of patient-provider dialogue: a multi-stage approach using large language models.\u00a0arXiv preprint, arXiv:2305.05982 v1. https:\/\/doi.org\/10.48550\/arXiv.2305.05982","DOI":"10.48550\/arXiv.2305.05982"},{"key":"218_CR13","doi-asserted-by":"crossref","unstructured":"Nazi ZA, Peng W (2024) Large language models in healthcare and medical domain: a review. In: Informatics. MDPI 11(3):57","DOI":"10.3390\/informatics11030057"},{"issue":"11","key":"218_CR14","doi-asserted-by":"publisher","first-page":"299","DOI":"10.1007\/s10462-024-10921-0","volume":"57","author":"D Wang","year":"2024","unstructured":"Wang D, Zhang S (2024) Large language models in medical and healthcare fields: applications, advances, and challenges. Artif Intell Rev 57(11):299","journal-title":"Artif Intell Rev"},{"issue":"1","key":"218_CR15","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1038\/s43856-024-00717-2","volume":"5","author":"F Busch","year":"2025","unstructured":"Busch F, Hoffmann L, Rueger C, van Dijk EH, Kader R, Ortiz-Prado E et al (2025) Current applications and challenges in large language models for patient care: a systematic review. Commun Med 5(1):26","journal-title":"Commun Med"},{"key":"218_CR16","doi-asserted-by":"publisher","first-page":"e58329","DOI":"10.2196\/58329","volume":"26","author":"J Seo","year":"2024","unstructured":"Seo J, Choi D, Kim T, Cha WC, Kim M, Yoo H et al (2024) Evaluation framework of large language models in medical documentation: development and usability study. J Med Internet Res 26:e58329","journal-title":"J Med Internet Res"},{"key":"218_CR17","doi-asserted-by":"publisher","unstructured":"Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW, et al (2022) Large language models encode clinical knowledge. arXiv preprint, arXiv:2212.13138 v1. https:\/\/doi.org\/10.48550\/arXiv.2212.13138","DOI":"10.48550\/arXiv.2212.13138"},{"key":"218_CR18","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2024.109713","author":"X Meng","year":"2024","unstructured":"Meng X, Yan X, Zhang K, Liu D, Cui X, Yang Y et al (2024) The application of large language models in medicine: a scoping review. iScience. https:\/\/doi.org\/10.1016\/j.isci.2024.109713","journal-title":"iScience"},{"key":"218_CR19","unstructured":"Liu L, Yang X, Lei J, Liu X, Shen Y, Zhang Z et al (2024) A survey on medical large language models: technology, application, trustworthiness, and future directions. arXiv preprint arXiv:2406.03712"},{"key":"218_CR20","doi-asserted-by":"crossref","unstructured":"Wu C, Lin W, Zhang X, Zhang Y, Wang Y, Xie W (2023) PMC-LLaMA: toward building open-source language models for medicine. J Am Med Inform Assoc 31(9):1833\u20131843","DOI":"10.1093\/jamia\/ocae045"},{"key":"218_CR21","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-024-03423-7","author":"K Singhal","year":"2025","unstructured":"Singhal K, Tu T, Gottweis J, Sayres R, Wulczyn E, Amin M et al (2025) Toward expert-level medical question answering with large language models. Nat Med. https:\/\/doi.org\/10.1038\/s41591-024-03423-7","journal-title":"Nat Med"},{"key":"218_CR22","doi-asserted-by":"publisher","unstructured":"Eyal B, Bachar A, Haroche O, Mahabi M, Elhadad M (2023) Semantic decomposition of question and SQL for text-to-SQL parsing. arXiv preprint, arXiv:2310.13575 v1. https:\/\/doi.org\/10.48550\/arXiv.2310.13575","DOI":"10.48550\/arXiv.2310.13575"},{"key":"218_CR23","doi-asserted-by":"publisher","unstructured":"Schmidgall S, Harris C, Essien I, Olshvang D, Rahman T, Kim JW et al (2024) Addressing cognitive bias in medical language models. arXiv preprint, arXiv:2402.08113 v1. https:\/\/doi.org\/10.48550\/arXiv.2402.08113","DOI":"10.48550\/arXiv.2402.08113"},{"key":"218_CR24","doi-asserted-by":"publisher","unstructured":"Poulain R, Fayyaz H, Beheshti R (2024) Bias patterns in the application of LLMs for clinical decision support: a comprehensive study. arXiv preprint, arXiv:2404.15149 v1. https:\/\/doi.org\/10.48550\/arXiv.2404.15149","DOI":"10.48550\/arXiv.2404.15149"},{"issue":"1","key":"218_CR25","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1038\/s41746-024-01283-6","volume":"7","author":"S Schmidgall","year":"2024","unstructured":"Schmidgall S, Harris C, Essien I, Olshvang D, Rahman T, Kim JW et al (2024) Evaluation and mitigation of cognitive biases in medical language models. NPJ Digit Med 7(1):295","journal-title":"NPJ Digit Med"},{"key":"218_CR26","doi-asserted-by":"publisher","unstructured":"Choi J, Hong Y, Kim B (2024) People will agree what I think: investigating LLM's false consensus effect. arXiv preprint, arXiv:2407.12007 v1. https:\/\/doi.org\/10.48550\/arXiv.2407.12007","DOI":"10.48550\/arXiv.2407.12007"},{"issue":"Suppl 7","key":"218_CR27","doi-asserted-by":"publisher","first-page":"S295","DOI":"10.4103\/2152-7806.139612","volume":"5","author":"NE Epstein","year":"2014","unstructured":"Epstein NE (2014) Multidisciplinary in-hospital teams improve patient outcomes: a review. Surg Neurol Int 5(Suppl 7):S295","journal-title":"Surg Neurol Int"},{"issue":"14","key":"218_CR28","doi-asserted-by":"publisher","DOI":"10.3390\/app11146421","volume":"11","author":"D Jin","year":"2021","unstructured":"Jin D, Pan E, Oufattole N, Weng W-H, Fang H, Szolovits P (2021) What disease does this patient have? A large-scale open domain question answering dataset from medical exams. Appl Sci 11(14):6421","journal-title":"Appl Sci"},{"issue":"1","key":"218_CR29","doi-asserted-by":"publisher","first-page":"14156","DOI":"10.1038\/s41598-024-64827-6","volume":"14","author":"J Maharjan","year":"2024","unstructured":"Maharjan J, Garikipati A, Singh NP, Cyrus L, Sharma M, Ciobanu M et al (2024) OpenMedLM: prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language models. Sci Rep 14(1):14156","journal-title":"Sci Rep"},{"key":"218_CR30","doi-asserted-by":"publisher","unstructured":"Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman FL et al (2023) GPT-4 technical report. arXiv preprint, arXiv:2303.08774 v1. https:\/\/doi.org\/10.48550\/arXiv.2303.08774","DOI":"10.48550\/arXiv.2303.08774"},{"key":"218_CR31","doi-asserted-by":"publisher","unstructured":"Huang Z, Wang Z, Xia S, Liu P (2024) OlympicArena medal ranks: who is the most intelligent AI so far? arXiv preprint, arXiv:2406.16772 v1. https:\/\/doi.org\/10.48550\/arXiv.2406.16772","DOI":"10.48550\/arXiv.2406.16772"},{"key":"218_CR32","unstructured":"OpenAI GPT-4o. https:\/\/openai.com\/index\/hello-gpt-4o\/?utm_source=chatgpt.com. Accessed 18 Feb 2025"},{"key":"218_CR33","unstructured":"Anthropic Claude 3.5 Sonnet. https:\/\/www.anthropic.com\/claude\/sonnet?utm_source=chatgpt.com. Accessed 18 Feb 2025"},{"key":"218_CR34","unstructured":"Anthropic Claude 3.5 Haiku. https:\/\/www.anthropic.com\/claude\/haiku?utm_source=chatgpt.com. Accessed 18 Feb 2025"},{"key":"218_CR35","unstructured":"Google Gemini API Models. https:\/\/ai.google.dev\/gemini-api\/docs\/models\/gemini?utm_source=chatgpt.com&hl=ko. Accessed 18 Feb 2025"},{"key":"218_CR36","unstructured":"Meta Llama. https:\/\/www.llama.com\/. Accessed 18 Feb 2025"},{"key":"218_CR37","unstructured":"HuggingFace Llama3-Med42\u20138B. https:\/\/huggingface.co\/m42-health\/Llama3-Med42-8B?utm_source=chatgpt.com. Accessed Feb 18 2025."},{"key":"218_CR38","unstructured":"HuggingFace DeepSeek-R1-Distill-Llama-8B. https:\/\/huggingface.co\/deepseek-ai\/DeepSeek-R1-Distill-Llama-8B?utm_source=chatgpt.com. Accessed 18 Feb 2025"},{"key":"218_CR39","unstructured":"Holmes J (1987) Hedging, fencing and other conversational gambits: an analysis of gender differences in New Zealand speech. Women and language in Australian and New Zealand society, pp 59\u201379"},{"issue":"1","key":"218_CR40","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1017\/S0047404500000051","volume":"2","author":"R Lakoff","year":"1973","unstructured":"Lakoff R (1973) Language and woman\u2019s place. Lang Soc 2(1):45\u201379","journal-title":"Lang Soc"},{"key":"218_CR41","unstructured":"Le T, Ascha M, Bhardwaj A, Boushra M, Griffin D (2023) First Aid for the USMLE Step 2 CK, 11th edn. McGraw Hill \/ Medical New York, NY, p 864"},{"key":"218_CR42","doi-asserted-by":"publisher","unstructured":"Tian K, Mitchell E, Yao H, Manning CD, Finn C (2023) Fine-tuning language models for factuality. arXiv preprint, arXiv:2311.08401 v1. https:\/\/doi.org\/10.48550\/arXiv.2311.08401","DOI":"10.48550\/arXiv.2311.08401"},{"key":"218_CR43","unstructured":"Wong E (2024) Comparative analysis of open source and proprietary large language models: performance and accessibility. Adv Comput Sci 7(1):1-7"},{"key":"218_CR44","doi-asserted-by":"publisher","unstructured":"Li Y, Harrigian K, Zirikly A, Dredze M (2024) Are clinical T5 models better for clinical text? arXiv preprint, arXiv:2412.05845 v1. https:\/\/doi.org\/10.48550\/arXiv.2412.05845","DOI":"10.48550\/arXiv.2412.05845"},{"key":"218_CR45","doi-asserted-by":"publisher","unstructured":"Jeong DP, Garg S, Lipton ZC, Oberst M (2024) Medical adaptation of large language and vision-language models: are we making progress? arXiv preprint, arXiv:2411.04118 v1. https:\/\/doi.org\/10.48550\/arXiv.2411.04118","DOI":"10.48550\/arXiv.2411.04118"},{"key":"218_CR46","doi-asserted-by":"publisher","unstructured":"Liu F, Li Z, Zhou H, Yin Q, Yang J, Tang X et al (2024) Large language models in the clinic: a comprehensive benchmark. arXiv preprint, arXiv:2405.00716 v2. https:\/\/doi.org\/10.48550\/arXiv.2405.00716","DOI":"10.48550\/arXiv.2405.00716"},{"issue":"4","key":"218_CR47","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","volume":"30","author":"D Van Veen","year":"2024","unstructured":"Van Veen D, Van Uden C, Blankemeier L, Delbrouck J-B, Aali A, Bluethgen C et al (2024) Adapted large language models can outperform medical experts in clinical text summarization. Nat Med 30(4):1134\u20131142","journal-title":"Nat Med"}],"container-title":["Journal of Healthcare Informatics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41666-025-00218-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41666-025-00218-4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41666-025-00218-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T06:36:46Z","timestamp":1778222206000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41666-025-00218-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,1]]},"references-count":47,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,6]]}},"alternative-id":["218"],"URL":"https:\/\/doi.org\/10.1007\/s41666-025-00218-4","relation":{},"ISSN":["2509-4971","2509-498X"],"issn-type":[{"value":"2509-4971","type":"print"},{"value":"2509-498X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,1]]},"assertion":[{"value":"2 April 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 October 2025","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 October 2025","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 December 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The clinician evaluation component of this study was approved by the Institutional Review Board of Severance Hospital, Yonsei University Health System (IRB No. 4\u20132025-0714). All clinician raters participated voluntarily after being informed of the study\u2019s purpose and procedure.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval and Consent to Participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}},{"value":"J.S.K. is a shareholder and employee of Oncosoft Inc., which could potentially benefit from the research findings presented in this paper. This conflict of interest has been disclosed and managed in accordance with institutional policies. B.-H.K. is a shareholder and outside board member of EverEx, which could potentially benefit from this study.\u00a0The other authors have no competing interests to declare.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}},{"value":"The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Disclaimer"}}]}}