{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T00:43:54Z","timestamp":1785545034415,"version":"3.56.0"},"reference-count":22,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec sec-type=\"background\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Patients frequently access laboratory results through patient portals, but many struggle to interpret these values and formulate relevant questions for their clinicians. Question prompt lists (QPLs) can enhance communication but are rarely tailored to individual clinical contexts.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"objective\">\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study evaluated the feasibility of using large language models (LLMs) to generate patient-friendly, clinically relevant questions grounded in electronic health record (EHR) laboratory data.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"methods\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We extracted deidentified clinical profiles, including laboratory results, diagnoses, and medications, from patients with chronic conditions (eg, diabetes and chronic kidney disease). Using 9 deidentified clinical profiles from the OneFlorida Data Trust, we generated 486 questions across all rounds: 126 with GPT-4o in round 1, 120 with GPT-4o in round 2, 180 with GPT-4o in round 3, and 60 with LLaMA 3.2 in round 3. Prompt refinements were informed by clinician ratings consisting of 2 binary questions (ie, clear phrasing and clinical validity) and 3 Likert-scale questions (ie, significance for the patient\u2019s health; clinical appropriateness, that is, the likelihood of being asked in primary care setting; and willingness to answer). Refinements were incorporated after each round. Patient participants then evaluated selected questions for understandability, perceived usefulness, and intention to use. Readability was assessed with standard indices.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"results\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Iterative clinician feedback improved question clarity and reduced clinically irrelevant suggestions. Across rounds, GPT-4o consistently produced more coherent and patient-friendly questions, while LLaMA 3.2 demonstrated competitive performance on Likert-scale metrics. It exhibited greater variability in clinical appropriateness as noted by clinicians. In round 3, the binary metric \u201cclear phrasing\u201d reached a ceiling effect for both models, while clinical validity ratings showed greater variability, particularly from one clinician. Likert-scale evaluations tended to favor LLaMA 3.2 across all 3 clinicians for clinical appropriateness (3.37\u20104.82 vs 3.02\u20104.67), significance for the patient\u2019s health (3.38\u20104.28 vs 2.97\u20103.83), and willingness to answer (3.17\u20104.70 vs 2.82\u20104.47), with multiple comparisons reaching statistical significance after Bonferroni correction. Patient evaluation (N=134) of GPT-4o\u2013generated questions showed that 25 of 30 questions had moderate to high understandability (average Likert score \u22653.5), and 19 of 30 questions had moderate to high usefulness (average Likert score \u22653.5).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"conclusions\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>This study supports the feasibility of using LLMs with structured EHR-derived laboratory data to generate contextualized QPLs, but model outputs varied in clinical appropriateness and readability. Clinician-in-the-loop review remains necessary before patient-facing use.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/87280","type":"journal-article","created":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T12:25:07Z","timestamp":1781267107000},"page":"e87280-e87280","source":"Crossref","is-referenced-by-count":0,"title":["Generating Question Prompt Lists From Electronic Health Record Data Using Large Language Models: Iterative Evaluation Study"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3608-0244","authenticated-orcid":false,"given":"Zhe","family":"He","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9890-4627","authenticated-orcid":false,"given":"Balu","family":"Bhasuran","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1144-2985","authenticated-orcid":false,"given":"Mia Liza A","family":"Lustria","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2194-8875","authenticated-orcid":false,"given":"Karim","family":"Hanna","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2287-9007","authenticated-orcid":false,"given":"Michael","family":"Killian","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-9088-5692","authenticated-orcid":false,"given":"Cindy","family":"Shavor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8760-511X","authenticated-orcid":false,"given":"Mandy","family":"Dailey","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3977-7581","authenticated-orcid":false,"given":"Sai Sidharth","family":"Manikandan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3649-9785","authenticated-orcid":false,"given":"Xiao","family":"Luo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,7,9]]},"reference":[{"issue":"9","key":"R1","doi-asserted-by":"publisher","first-page":"1218","DOI":"10.1093\/jamia\/ocy065","article-title":"The 21st Century Cures Act and electronic health records one year later: will patients see the benefits?","volume":"25","author":"Lye","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"R2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0154743","article-title":"Direct release of test results to patients increases patient engagement and utilization of care","volume":"11","author":"Pillemer","journal-title":"PLoS One"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1055\/s-0038-1677528","article-title":"How patients use a patient portal: an institutional case study of demographics and usage patterns","volume":"10","author":"Tsai","journal-title":"Appl Clin Inform"},{"issue":"3","key":"R4","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1515\/cclm-2018-0634","article-title":"Communicating laboratory results to patients and families","volume":"57","author":"Witteman","journal-title":"Clin Chem Lab Med"},{"issue":"6","key":"R5","doi-asserted-by":"publisher","first-page":"1335","DOI":"10.1016\/j.pec.2021.01.012","article-title":"Effect of question prompt lists for cancer patients on communication and mental health outcomes: a systematic review","volume":"104","author":"Keinki","journal-title":"Patient Educ Couns"},{"issue":"12","key":"R6","doi-asserted-by":"publisher","first-page":"1454","DOI":"10.1016\/j.pec.2015.05.015","article-title":"Question prompt lists in health consultations: a review","volume":"98","author":"Sansoni","journal-title":"Patient Educ Couns"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.2196\/56655","article-title":"Quality of answers of generative large language models versus peer users for interpreting laboratory test results for lay patients: evaluation study","volume":"26","author":"He","journal-title":"J Med Internet Res"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00958-w","article-title":"A study of generative large language model for medical research and healthcare","volume":"6","author":"Peng","journal-title":"NPJ Digit Med"},{"issue":"6","key":"R9","doi-asserted-by":"publisher","first-page":"1367","DOI":"10.1093\/jamia\/ocae052","article-title":"Leveraging large language models for generating responses to patient messages: a subjective analysis","volume":"31","author":"Liu","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01556-8","article-title":"Preliminary analysis of the impact of lab results on large language model generated differential diagnoses","volume":"8","author":"Bhasuran","journal-title":"NPJ Digit Med"},{"issue":"2","key":"R11","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/s00296-023-05464-6","article-title":"Diagnostic accuracy of a large language model in rheumatology: comparison of physician and ChatGPT-4","volume":"44","author":"Krusche","journal-title":"Rheumatol Int"},{"key":"R12","unstructured":"Pal A Umapathi LK Sankarasubbu M . MedMCQA: large-scale multi-subject multi-choice dataset for medical domain question answering. In: Proceedings of the Conference on Health, Inference, and Learning. Vol 174. Association for Computing Machinery; 2022:248-260. URL: https:\/\/proceedings.mlr.press\/v174\/pal22a\/pal22a.pdf [Accessed 21-05-2025]"},{"issue":"7","key":"R13","doi-asserted-by":"publisher","first-page":"1158","DOI":"10.1515\/cclm-2023-0355","article-title":"Potentials and pitfalls of chatgpt and natural-language artificial intelligence models for the understanding of laboratory medicine test results: an assessment by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group on Artificial Intelligence (WG-AI)","volume":"61","author":"Cadamuro","journal-title":"Clin Chem Lab Med"},{"issue":"7","key":"R14","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.22399","article-title":"Large language model-based responses to patients' in-basket messages","volume":"7","author":"Small","journal-title":"JAMA Netw Open"},{"issue":"10","key":"R15","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.38573","article-title":"Utility of artificial intelligence-generative draft replies to patient messages","volume":"7","author":"English","journal-title":"JAMA Netw Open"},{"issue":"8","key":"R16","doi-asserted-by":"publisher","first-page":"1665","DOI":"10.1093\/jamia\/ocae142","article-title":"Using large language model to guide patients to create efficient and comprehensive clinical care message","volume":"31","author":"Liu","journal-title":"J Am Med Inform Assoc"},{"key":"R17","doi-asserted-by":"crossref","unstructured":"Gatto J Seegmiller P Burdick TE Khayal IS DeLozier S Preum SM . Follow-up question generation for enhanced patient-provider conversations. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1. Association for Computational Linguistics; 25222-25240. URL: https:\/\/aclanthology.org\/2025.acl-long [Accessed 30-06-2026] [doi: 10.18653\/v1\/2025.acl-long.1226]","DOI":"10.18653\/v1\/2025.acl-long.1226"},{"key":"R18","unstructured":"Winston C Winston C Winston C Winston C . Medical question-generation for pre-consultation with LLM in-context learning. In: NIPS \u201924: 38th International Conference on Neural Information Processing Systems (Workshop GenAI for Health: Potential, Trust and Policy Compliance). Curran Associates Inc; URL: https:\/\/neurips.cc\/virtual\/2024\/106861 [Accessed 30-06-2026]"},{"key":"R19","unstructured":"OpenAI. GPT-4o system card. arXiv. Preprint posted online on Oct 25, 2024. [doi: 10.48550\/arXiv.2410.21276]"},{"key":"R20","unstructured":"Llama 32: revolutionizing edge AI and vision with open, customizable models. Meta AI. 2024. URL: https:\/\/ai.meta.com\/blog\/llama-3-2-connect-2024-vision-edge-mobile-devices\/ [Accessed 06-03-2025]"},{"key":"R21","unstructured":"Jan - Open-Source ChatGPT Replacement. URL: https:\/\/jan.ai\/ [Accessed 06-03-2025]"},{"issue":"3","key":"R22","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1016\/j.pec.2014.05.027","article-title":"Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information","volume":"96","author":"Shoemaker","journal-title":"Patient Educ Couns"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2026\/1\/e87280","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:25:17Z","timestamp":1785536717000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e87280"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,9]]},"references-count":22,"URL":"https:\/\/doi.org\/10.2196\/87280","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,9]]},"article-number":"v28i9e87280"}}