{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T20:57:52Z","timestamp":1791233872257,"version":"4.1.0"},"reference-count":52,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR AI"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Quality-of-life (QoL) questionnaires are an established instrument designed to assess overall well-being and QoL of patients. They are important in predicting the outcome of the disease and understanding the needs of individual patients. However, their repeated collection imposes a substantial burden on both patients and clinical professionals. Many patients seek emotional support and mutual exchange in online communities for peer support, where they frequently share detailed descriptions of symptoms and treatment experiences, addressing topics covered in QoL questionnaires. The emergence of large language models (LLMs) uncovers potential for automatic extraction of relevant QoL information from patient-generated text.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n The aim of this study is to evaluate and compare various open-source LLMs and optimization approaches for automated extraction of QoL information from forum posts.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n The dataset consisted of 840 English-language posts from patients with breast cancer recruited on Inspire online communities, manually annotated with sentence-level text spans indicating whether and where posts contained information relevant to 53 QoL questions from standardized questionnaires. Eleven open-source LLMs were evaluated in a zero-shot setup under 2 input conditions: post-only and post with additional context. For the GPT-OSS-20B model, additional experiments assessed the impact of chain-of-thought prompting, instruction optimization, few-shot prompting, simultaneous all-questions prompting, and parameter-efficient fine-tuning. For correctly classified yes and no instances, the overlap between model-generated evidence and human-annotated spans was evaluated.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Across 11 evaluated LLMs, Qwen3-14B achieved the highest macro\n F<\/jats:italic>\n 1<\/jats:sub>\n -score (0.59) in the zero-shot post-only setting. Providing additional context consistently reduced the performance of all models. Model size did not correlate with\n F<\/jats:italic>\n 1<\/jats:sub>\n -score, with several midsized models (14B-30B) outperforming 70B models. For GPT-OSS-20B, chain-of-thought prompting, instruction optimization, and simultaneous all-questions prompting decreased performance. Bootstrap few-shot prompting with random search achieved slightly better performance than the baseline. Parameter-efficient fine-tuning with low-rank adaptation (LoRA) achieved the highest overall performance (0.71). Across all experiments, a strong class imbalance was observed, with models performing substantially better on the majority class \u201cnot in the text\u201d than on the minority classes \u201cyes\u201d and \u201cno.\u201d Most classification errors occurred in semantically broad or ambiguous terms and the fallback question. For correctly predicted yes and no answers, model-generated evidence matched or partially matched human-annotated spans in 89% (42\/47) of cases.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Automated extraction of QoL information from patient-generated text using open-source LLMs remains a challenging task. While prompt optimization techniques failed to improve baseline zero-shot performance, parameter-efficient fine-tuning with LoRA significantly increased accuracy. However, current models still struggle to reliably detect explicit symptom expressions in heavily imbalanced data, too often predicting the majority class \u201cnot in the text.\u201d Before such tools can be successfully integrated into clinical practice, future research must prioritize strategies to capture these minority-class signals.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/92716","type":"journal-article","created":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T19:10:05Z","timestamp":1785438605000},"page":"e92716-e92716","source":"Crossref","is-referenced-by-count":0,"title":["Extracting Quality-of-Life Information of Patients Diagnosed With Breast Cancer From Health Care Online Forum Posts Using Open-Source Large Language Models: Algorithm Development and Evaluation Study"],"prefix":"10.2196","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-2344-7704","authenticated-orcid":false,"given":"Karolina Hanna","family":"Czok","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7728-2884","authenticated-orcid":false,"given":"David Maria","family":"Schmidt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-8854-5780","authenticated-orcid":false,"given":"Brian Po-Han","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1477-3170","authenticated-orcid":false,"given":"Deborah","family":"Kuk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-1751-4459","authenticated-orcid":false,"given":"Josh","family":"Feldman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4771-441X","authenticated-orcid":false,"given":"Philipp","family":"Cimiano","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,5]]},"reference":[{"key":"R1","unstructured":"The Constitution of the World Health Organisation. Global Health & Human Rights Database. WHO; 1948. URL: https:\/\/www.globalhealthrights.org\/instrument\/constitution-of-the-world-health-organization-who\/ [Accessed 20-09-2026]"},{"issue":"6","key":"R2","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1192\/bjp.bp.112.123463","article-title":"Evaluating the impact of a quality of life assessment with feedback to clinicians in patients with schizophrenia: randomised controlled trial","volume":"202","author":"Boyer","journal-title":"Br J Psychiatry"},{"issue":"5","key":"R3","doi-asserted-by":"publisher","first-page":"203156","DOI":"10.1016\/j.jeph.2025.203156","article-title":"Harnessing generative AI for quality of life assessment: foundations for a new research agenda","volume":"73","author":"Zhang","journal-title":"J Epidemiol Popul Health"},{"issue":"5","key":"R4","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1093\/jnci\/85.5.365","article-title":"The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology","volume":"85","author":"Aaronson","journal-title":"J Natl Cancer Inst"},{"issue":"8","key":"R5","doi-asserted-by":"publisher","first-page":"1181","DOI":"10.1016\/s0959-8049(98)00018-5","article-title":"Quality of life assessment in daily clinical oncology practice: a feasibility study","volume":"34","author":"Detmar","journal-title":"Eur J Cancer"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1023\/a:1016692804023","article-title":"Oncologists\u2019 use of quality of life information: results of a survey of Eastern Cooperative Oncology Group physicians","volume":"10","author":"Bezjak","journal-title":"Qual Life Res"},{"issue":"6","key":"R7","doi-asserted-by":"publisher","first-page":"513","DOI":"10.1023\/a:1008826408328","article-title":"Patients\u2019 experiences using a computerized program with a touch-sensitive video monitor for the assessment of health-related quality of life","volume":"7","author":"Buxton","journal-title":"Qual Life Res"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1038\/sj.bjc.6600001","article-title":"Computer-based quality of life questionnaires may contribute to doctor-patient interactions in oncology","volume":"86","author":"Velikova","journal-title":"Br J Cancer"},{"key":"R9","doi-asserted-by":"crossref","unstructured":"Natalia P Alejandro V Konstantina K C\u00e9lia B . Online health information search: what struggles and empowers the users? results of an online survey. In: Studies in Health Technology and Informatics. IOS Press; 2012. [doi: 10.3233\/978-1-61499-101-4-843]","DOI":"10.3233\/978-1-61499-101-4-843"},{"issue":"4","key":"R10","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1046\/j.1369-6513.2002.00191.x","article-title":"Information and support for women following the primary treatment of breast cancer","volume":"5","author":"Raupach","journal-title":"Health Expect"},{"issue":"1-2","key":"R11","doi-asserted-by":"publisher","first-page":"51","DOI":"10.3166\/lcn.12.1-2.51-71","article-title":"Information m\u00e9dicale et \u00e9motion dans les forums de sant\u00e9","volume":"12","author":"Batta\u00efa","journal-title":"LCN"},{"issue":"7-8","key":"R12","doi-asserted-by":"publisher","first-page":"466","DOI":"10.1016\/j.gofs.2024.02.003","article-title":"Les forums de discussion d\u00e9di\u00e9s au cancer du sein peuvent-ils \u00eatre utiles aux soignants? analyse des messages initiaux du forum de la Ligue nationale contre le cancer pendant une ann\u00e9e","volume":"52","author":"Burgu\u00e9","journal-title":"Gynecol Obstet Fertil Senol"},{"issue":"1","key":"R13","doi-asserted-by":"publisher","DOI":"10.2196\/76044","article-title":"Extracting quality of life information of patients diagnosed with breast cancer from health care online forum posts: data feasibility study","volume":"12","author":"Schmidt","journal-title":"JMIR Cancer"},{"issue":"3","key":"R14","doi-asserted-by":"publisher","first-page":"229","DOI":"10.3322\/caac.21834","article-title":"Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"74","author":"Bray","journal-title":"CA Cancer J Clin"},{"key":"R15","unstructured":"Survival rates for breast cancer. American Cancer Society. URL: https:\/\/www.cancer.org\/cancer\/types\/breast-cancer\/understanding-a-breast-cancer-diagnosis\/breast-cancer-survival-rates.html [Accessed 22-06-2026]"},{"issue":"2","key":"R16","doi-asserted-by":"publisher","first-page":"409","DOI":"10.1007\/s00520-016-3417-6","article-title":"Quality of life (QOL) and symptom burden (SB) in patients with breast cancer","volume":"25","author":"Hamer","journal-title":"Support Care Cancer"},{"issue":"1","key":"R17","doi-asserted-by":"publisher","DOI":"10.1186\/1477-7525-5-24","article-title":"Quality of life assessment in women with breast cancer: benefits, acceptability and utilization","volume":"5","author":"Perry","journal-title":"Health Qual Life Outcomes"},{"issue":"2","key":"R18","doi-asserted-by":"crossref","first-page":"110","DOI":"10.18053\/jctres.09.202302.006","volume":"9","author":"Nolazco","journal-title":"J Clin Transl Res"},{"key":"R19","doi-asserted-by":"publisher","DOI":"10.1016\/j.eclinm.2025.103153","article-title":"Prognostic value of baseline EORTC QLQ-C30 scores for overall survival across 46 clinical trials covering 17 cancer types: a validation study","volume":"82","author":"Lim","journal-title":"EClinicalMedicine"},{"issue":"3","key":"R20","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1159\/000531136","article-title":"The role of social media in breast cancer care and survivorship: a narrative review","volume":"18","author":"Aristokleous","journal-title":"Breast Care (Basel)"},{"issue":"6","key":"R21","doi-asserted-by":"publisher","first-page":"1492","DOI":"10.1016\/j.jpainsymman.2018.02.016","article-title":"Machine learning methods to extract documentation of breast cancer symptoms from electronic health records","volume":"55","author":"Forsyth","journal-title":"J Pain Symptom Manage"},{"issue":"1","key":"R22","doi-asserted-by":"publisher","DOI":"10.2196\/68776","article-title":"Leveraging large language models for accurate retrieval of patient information from medical reports: systematic evaluation study","volume":"4","author":"Garcia-Carmona","journal-title":"JMIR AI"},{"issue":"1","key":"R23","doi-asserted-by":"publisher","DOI":"10.2196\/64544","article-title":"Using large language models to automate data extraction from surgical pathology reports: retrospective cohort study","volume":"9","author":"Lee","journal-title":"JMIR Form Res"},{"issue":"1","key":"R24","doi-asserted-by":"publisher","DOI":"10.2196\/62909","article-title":"Summarizing online patient conversations using generative language models: experimental and comparative study","volume":"13","author":"Nair","journal-title":"JMIR Med Inform"},{"issue":"10","key":"R25","doi-asserted-by":"publisher","first-page":"2756","DOI":"10.1200\/JCO.1996.14.10.2756","article-title":"The European Organization for Research and Treatment of Cancer breast cancer-specific quality-of-life questionnaire module: first results from a three-country field study","volume":"14","author":"Sprangers","journal-title":"J Clin Oncol"},{"issue":"4","key":"R26","doi-asserted-by":"publisher","DOI":"10.3390\/bioengineering11040337","article-title":"The role of AI in hospitals and clinics: transforming healthcare in the 21st century","volume":"11","author":"Maleki Varnosfaderani","journal-title":"Bioengineering (Basel)"},{"issue":"1","key":"R27","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00873-0","article-title":"The imperative for regulatory oversight of large language models (or generative AI) in healthcare","volume":"6","author":"Mesk\u00f3","journal-title":"NPJ Digit Med"},{"issue":"8","key":"R28","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","journal-title":"Nat Med"},{"issue":"8","key":"R29","doi-asserted-by":"publisher","first-page":"2546","DOI":"10.1038\/s41591-025-03727-2","article-title":"Benchmark evaluation of DeepSeek large language models in clinical decision-making","volume":"31","author":"Sandmann","journal-title":"Nat Med"},{"key":"R30","unstructured":"Agarwal S Ahmad L Ai J et al. Gpt-oss-120b & gpt-oss-20b model card. arXiv. Preprint posted online on Aug 8, 2025. [doi: 10.48550\/arXiv.2508.10925]"},{"key":"R31","unstructured":"Yang A Li A Yang B et al. Qwen3 technical report. arXiv. Preprint posted online on May 14, 2025. [doi: 10.48550\/arXiv.2505.09388]"},{"key":"R32","unstructured":"Team G Kamath A Ferret J et al. Gemma 3 technical report. arXiv. Preprint posted online on Mar 25, 2025. [doi: 10.48550\/arXiv.2503.19786]"},{"key":"R33","unstructured":"DeepSeek-AIGuo D Yang D et al. DeepSeek-R1: incentivizing reasoning capability in llms via reinforcement learning. Preprint posted online on Jan 22, 2025. [doi: 10.48550\/arXiv.2501.12948]"},{"key":"R34","unstructured":"Grattafiori A Dubey A Jauhri A et al. The llama 3 herd of models. arXiv. Preprint posted online on Jul 31, 2024. [doi: 10.48550\/arXiv.2407.21783]"},{"key":"R35","unstructured":"Abdin M Aneja J Behl H et al. Phi-4 technical report. arXiv. Preprint posted online on Dec 12, 2024. [doi: 10.48550\/arXiv.2412.08905]"},{"key":"R36","unstructured":"Ollama. Github. 2025. URL: https:\/\/github.com\/ollama\/ollama [Accessed 10-12-2025]"},{"key":"R37","unstructured":"Khattab O Singhvi A Maheshwari P et al. DSPy: compiling declarative language model calls into self-improving pipelines. arXiv. Preprint posted online on Oct 5, 2023. [doi: 10.48550\/arXiv.2310.03714]"},{"key":"R38","doi-asserted-by":"crossref","unstructured":"Advanced Data Mining Techniques. Springer; 2008. [doi: 10.1007\/978-3-540-76917-0] ISBN: 9783540769163","DOI":"10.1007\/978-3-540-76917-0"},{"key":"R39","unstructured":"Wei J Wang X Schuurmans D et al. Chain-of-thought prompting elicits reasoning in large language models. arXiv. Preprint posted online on Jan 10, 2023. [doi: 10.48550\/arXiv.2201.11903]"},{"key":"R40","doi-asserted-by":"crossref","unstructured":"Opsahl-Ong K Ryan MJ Purtell J et al. Optimizing instructions and demonstrations for multi-stage language model programs. In: Chen YN Bansal M Chen YN , editors. Presented at: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing; Nov 12-16, 2024:9340-9366; Miami, FL. URL: https:\/\/aclanthology.org\/2024.emnlp-main [Accessed 08-09-2026] [doi: 10.18653\/v1\/2024.emnlp-main.525]","DOI":"10.18653\/v1\/2024.emnlp-main.525"},{"key":"R41","unstructured":"Prompt optimizing with GEPA. DSPy. URL: https:\/\/dspy.ai\/learn\/optimization\/optimizers\/ [Accessed 12-12-2025]"},{"issue":"1","key":"R42","doi-asserted-by":"publisher","DOI":"10.1016\/j.mcpdig.2024.11.005","article-title":"Fine-tuning large language models for specialized use cases","volume":"3","author":"Anisuzzaman","journal-title":"Mayo Clin Proc Digit Health"},{"issue":"1","key":"R43","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1109\/TAI.2025.3569516","article-title":"Inadequacies of large language model benchmarks in the era of generative artificial intelligence","volume":"7","author":"McIntosh","journal-title":"IEEE Trans Artif Intell"},{"issue":"4","key":"R44","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1002\/hcs2.61","article-title":"Large language models in health care: Development, applications, and challenges","volume":"2","author":"Yang","journal-title":"Health Care Sci"},{"key":"R45","unstructured":"Hu EJ Shen Y Wallis P et al. LoRA: low-rank adaptation of large language models. arXiv. Preprint posted online on Oct 16, 2021. [doi: 10.48550\/arXiv.2106.09685]"},{"key":"R46","doi-asserted-by":"crossref","unstructured":"Singhal R Ponkshe K Vepakomma P . FedEx-lora: exact aggregation for federated and efficient fine-tuning of large language models. In: Che W Nabende J Shutova E Pilehvar MT , editors. Presented at: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Jul 27 to Aug 1, 2025:1316-1336; Vienna, Austria. [doi: 10.18653\/v1\/2025.acl-long.67]","DOI":"10.18653\/v1\/2025.acl-long.67"},{"key":"R47","unstructured":"Goel A Gueta A Gilon O et al. LLMs Accelerate annotation for medical information extraction. Arxiv. Preprint posted online on Dec 4, 2023. [doi: 10.48550\/arXiv.2312.02296]"},{"key":"R48","doi-asserted-by":"crossref","unstructured":"Tan Z Li D Wang S et al. Large language models for data annotation and synthesis: a survey. In: Chen YN Bansal M Chen YN , editors. Presented at: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing; Nov 12-16, 2024:930-957; Miami, FL. [doi: 10.18653\/v1\/2024.emnlp-main.54]","DOI":"10.18653\/v1\/2024.emnlp-main.54"},{"key":"R49","unstructured":"Liu R Geng J Wu AJ et al. Mind your step (by step): chain-of-thought can reduce performance on tasks where thinking makes humans worse. arXiv. Preprint posted online on Jun 13, 2025. [doi: 10.48550\/arXiv.2410.21333]"},{"key":"R50","doi-asserted-by":"publisher","DOI":"10.1016\/j.chbr.2024.100407","article-title":"Efficiency of natural language processing as a tool for analysing quality of life in patients with chronic diseases. A systematic review","volume":"14","author":"L\u00e1zaro","journal-title":"Comput Hum Behav Rep"},{"issue":"1","key":"R51","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1016\/j.cmpb.2015.03.008","article-title":"Screening Internet forum participants for depression symptoms by assembling and enhancing multiple NLP methods","volume":"120","author":"Karmen","journal-title":"Comput Methods Programs Biomed"},{"key":"R52","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2021.103864","article-title":"Development of a generalizable natural language processing pipeline to extract physician-reported pain from clinical reports: generated using publicly-available datasets and tested on institutional clinical reports for cancer patients with bone metastases","volume":"120","author":"Naseri","journal-title":"J Biomed Inform"}],"container-title":["JMIR AI"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T20:24:30Z","timestamp":1791231870000},"score":1,"resource":{"primary":{"URL":"https:\/\/ai.jmir.org\/2026\/1\/e92716"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,5]]},"references-count":52,"URL":"https:\/\/doi.org\/10.2196\/92716","relation":{},"ISSN":["2817-1705"],"issn-type":[{"value":"2817-1705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,5]]},"article-number":"v5i6e92716"}}