{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,4]],"date-time":"2026-11-04T02:46:37Z","timestamp":1793760397241,"version":"4.6.1"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,2,20]],"date-time":"2025-02-20T00:00:00Z","timestamp":1740009600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,2,20]],"date-time":"2025-02-20T00:00:00Z","timestamp":1740009600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Eunice Kennedy Shriver National Institute of Child Health and Human Development","award":["K23HD110716"],"award-info":[{"award-number":["K23HD110716"]}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"crossref","award":["U01NS114042"],"award-info":[{"award-number":["U01NS114042"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Large language models (LLMs) can improve text analysis efficiency in healthcare. This study explores the application of LLMs to analyze patient perspectives within the exception from informed consent (EFIC) process, which waives consent in emergency research. Our objective is to assess whether LLMs can analyze patient perspectives in EFIC interviews with performance comparable to human reviewers.\u00a0We analyzed 102 EFIC community interviews from 9 sites, each with 46 questions, as part of the Pediatric Dose Optimization for Seizures in Emergency Medical Services study. We evaluated 5 LLMs, including GPT-4, to assess sentiment polarity on a 5-point scale and classify responses into predefined thematic classes. Three human reviewers conducted parallel analyses, with agreement measured by Cohen\u2019s Kappa and classification accuracy.\u00a0Polarity scores between LLM and human reviewers showed substantial agreement (Cohen\u2019s kappa: 0.69, 95% CI 0.61\u20130.76), with major discrepancies in only 4.7% of responses. LLM achieved high thematic classification accuracy (0.868, 95% CI 0.853\u20130.881), comparable to inter-rater agreement among human reviewers (0.867, 95% CI 0.836\u20130.901). LLMs enabled large-scale visual analysis, comparing response statistics across sites, questions, and classes. LLMs efficiently analyzed patient perspectives in EFIC interviews, showing substantial sentiment assessment and thematic classification performance. However, occasional underperformance suggests LLMs should complement, not replace, human judgment. Future work should evaluate LLM integration in EFIC to enhance efficiency, reduce subjectivity, and support accurate patient perspective analysis.<\/jats:p>","DOI":"10.1038\/s41598-025-89996-w","type":"journal-article","created":{"date-parts":[[2025,2,20]],"date-time":"2025-02-20T06:31:49Z","timestamp":1740033109000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Analyzing patient perspectives with large language models: a cross-sectional study of sentiment and thematic classification on exception from informed consent"],"prefix":"10.1038","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1344-575X","authenticated-orcid":false,"given":"Aaron E.","family":"Kornblith","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0318-2340","authenticated-orcid":false,"given":"Chandan","family":"Singh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7933-836X","authenticated-orcid":false,"given":"Johanna C.","family":"Innes","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4508-2551","authenticated-orcid":false,"given":"Todd P.","family":"Chang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4405-0769","authenticated-orcid":false,"given":"Kathleen M.","family":"Adelgais","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6405-3687","authenticated-orcid":false,"given":"Maija","family":"Holsti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joy","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bradford","family":"McClain","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4952-8212","authenticated-orcid":false,"given":"Daniel K.","family":"Nishijima","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steffanie","family":"Rodgers","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0886-8747","authenticated-orcid":false,"given":"Manish I.","family":"Shah","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Harold K.","family":"Simon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1350-5764","authenticated-orcid":false,"given":"John M.","family":"VanBuren","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5272-8552","authenticated-orcid":false,"given":"Caleb E.","family":"Ward","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7152-8887","authenticated-orcid":false,"given":"Catherine R.","family":"Counts","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,2,20]]},"reference":[{"key":"89996_CR1","unstructured":"Zhang, W. et al. Sentiment Analysis in the Era of Large Language Models: A Reality Check. Published online May 24, 2023. Accessed December 7, (2023). http:\/\/arxiv.org\/abs\/2305.15005"},{"key":"89996_CR2","doi-asserted-by":"publisher","unstructured":"Zhong, Q. et al. Can ChatGPT Understand too? A comparative study on ChatGPT and Fine-tuned BERT. Published Online March. 2 https:\/\/doi.org\/10.48550\/arXiv.2302.10198 (2023).","DOI":"10.48550\/arXiv.2302.10198"},{"key":"89996_CR3","doi-asserted-by":"publisher","unstructured":"Wang, Z. et al. Is ChatGPT a good sentiment Analyzer? A preliminary study. Published Online April. 9 https:\/\/doi.org\/10.48550\/arXiv.2304.04339 (2023).","DOI":"10.48550\/arXiv.2304.04339"},{"issue":"1","key":"89996_CR4","doi-asserted-by":"publisher","first-page":"210","DOI":"10.1038\/s41746-023-00958-w","volume":"6","author":"C Peng","year":"2023","unstructured":"Peng, C. et al. A study of generative large language model for medical research and healthcare. Npj Digit. Med. 6 (1), 210. https:\/\/doi.org\/10.1038\/s41746-023-00958-w (2023).","journal-title":"Npj Digit. Med."},{"issue":"9","key":"89996_CR5","doi-asserted-by":"publisher","first-page":"866","DOI":"10.1001\/jama.2023.14217","volume":"330","author":"NH Shah","year":"2023","unstructured":"Shah, N. H., Entwistle, D. & Pfeffer, M. A. Creation and adoption of large Language models in Medicine. JAMA 330 (9), 866\u2013869. https:\/\/doi.org\/10.1001\/jama.2023.14217 (2023).","journal-title":"JAMA"},{"issue":"9","key":"89996_CR6","doi-asserted-by":"publisher","first-page":"2739","DOI":"10.46743\/2160-3715\/2023.6536","volume":"28","author":"P Christou","year":"2023","unstructured":"Christou, P. The Use of Artificial Intelligence (AI) in qualitative research for Theory Development. Qual. Rep. 28 (9), 2739\u20132755. https:\/\/doi.org\/10.46743\/2160-3715\/2023.6536 (2023).","journal-title":"Qual. Rep."},{"key":"89996_CR7","unstructured":"Exception from Informed Consent Requirements for Emergency Research. FDA; Accessed February 10, 2024. (2020). https:\/\/www.fda.gov\/regulatory-information\/search-fda-guidance-documents\/exception-informed-consent-requirements-emergency-research"},{"issue":"1","key":"89996_CR8","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1016\/j.artmed.2015.03.006","volume":"64","author":"K Denecke","year":"2015","unstructured":"Denecke, K. & Deng, Y. Sentiment analysis in medical settings: new opportunities and challenges. Artif. Intell. Med. 64 (1), 17\u201327. https:\/\/doi.org\/10.1016\/j.artmed.2015.03.006 (2015).","journal-title":"Artif. Intell. Med."},{"issue":"1","key":"89996_CR9","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1080\/23294515.2020.1818878","volume":"12","author":"M Chisolm-Straker","year":"2021","unstructured":"Chisolm-Straker, M. et al. Exception from informed consent: how IRB reviewers Assess Community Consultation and Public Disclosure. AJOB Empir. Bioeth. 12 (1), 24\u201332. https:\/\/doi.org\/10.1080\/23294515.2020.1818878 (2021).","journal-title":"AJOB Empir. Bioeth."},{"issue":"7","key":"89996_CR10","doi-asserted-by":"publisher","first-page":"e197569","DOI":"10.1001\/jamanetworkopen.2019.7569","volume":"2","author":"MJ Lanspa","year":"2019","unstructured":"Lanspa, M. J., Fan, E. & Morris, A. H. How should we apply the Wisdom of the crowd to clinical trials with exception from informed consent? JAMA Netw. Open. 2 (7), e197569. https:\/\/doi.org\/10.1001\/jamanetworkopen.2019.7569 (2019).","journal-title":"JAMA Netw. Open."},{"issue":"6","key":"89996_CR11","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1002\/eahr.500109","volume":"43","author":"AR Johnson","year":"2021","unstructured":"Johnson, A. R. et al. An Approach to reviewing local context for exception from informed consent trials using a single IRB. Ethics Hum. Res. 43 (6), 42\u201348. https:\/\/doi.org\/10.1002\/eahr.500109 (2021).","journal-title":"Ethics Hum. Res."},{"key":"89996_CR12","unstructured":"Shah, M. Pediatric Dose Optimization for Seizures in EMS (PediDOSE). clinicaltrials.gov; Accessed December 31, 2023. (2023). https:\/\/clinicaltrials.gov\/study\/NCT05121324"},{"key":"89996_CR13","doi-asserted-by":"publisher","unstructured":"Ward, C. E. et al. Public support for and concerns regarding pediatric dose optimization for seizures in emergency medical services: an exception from informed consent (EFIC) trial. Acad. Emerg. Med. Off J. Soc. Acad. Emerg. Med. Published Online March. 7 https:\/\/doi.org\/10.1111\/acem.14884 (2024).","DOI":"10.1111\/acem.14884"},{"issue":"8","key":"89996_CR14","doi-asserted-by":"publisher","first-page":"573","DOI":"10.7326\/0003-4819-147-8-200710160-00010","volume":"147","author":"E von Elm","year":"2007","unstructured":"von Elm, E. et al. The strengthening the reporting of Observational studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann. Intern. Med. 147 (8), 573\u2013577. https:\/\/doi.org\/10.7326\/0003-4819-147-8-200710160-00010 (2007).","journal-title":"Ann. Intern. Med."},{"issue":"10","key":"89996_CR15","doi-asserted-by":"publisher","first-page":"e2335924","DOI":"10.1001\/jamanetworkopen.2023.35924","volume":"6","author":"RH Perlis","year":"2023","unstructured":"Perlis, R. H. & Fihn, S. D. Evaluating the application of large Language models in Clinical Research contexts. JAMA Netw. Open. 6 (10), e2335924. https:\/\/doi.org\/10.1001\/jamanetworkopen.2023.35924 (2023).","journal-title":"JAMA Netw. Open."},{"key":"89996_CR16","doi-asserted-by":"publisher","unstructured":"Flanagin, A. et al. Reporting Use of AI in Research and Scholarly Publication\u2014JAMA Network Guidance. JAMA Published Online March. 7 https:\/\/doi.org\/10.1001\/jama.2024.3471 (2024).","DOI":"10.1001\/jama.2024.3471"},{"key":"89996_CR17","doi-asserted-by":"publisher","unstructured":"OpenAI, Achiam, J. et al. GPT-4 Technical Report. Published online December 18, (2023). https:\/\/doi.org\/10.48550\/arXiv.2303.08774","DOI":"10.48550\/arXiv.2303.08774"},{"key":"89996_CR18","doi-asserted-by":"publisher","unstructured":"Ouyang, L. et al. Training language models to follow instructions with human feedback. Published Online March. 4 https:\/\/doi.org\/10.48550\/arXiv.2203.02155 (2022).","DOI":"10.48550\/arXiv.2203.02155"},{"issue":"2","key":"89996_CR19","doi-asserted-by":"publisher","first-page":"e0192360","DOI":"10.1371\/journal.pone.0192360","volume":"13","author":"S Gehrmann","year":"2018","unstructured":"Gehrmann, S. et al. Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives. PloS One. 13 (2), e0192360. https:\/\/doi.org\/10.1371\/journal.pone.0192360 (2018).","journal-title":"PloS One"},{"key":"89996_CR20","doi-asserted-by":"publisher","unstructured":"Tu, T. et al. Towards conversational diagnostic AI. Published Online January. 10 https:\/\/doi.org\/10.48550\/arXiv.2401.05654 (2024).","DOI":"10.48550\/arXiv.2401.05654"},{"issue":"7972","key":"89996_CR21","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","volume":"620","author":"K Singhal","year":"2023","unstructured":"Singhal, K. et al. Large language models encode clinical knowledge. Nature 620 (7972), 172\u2013180. https:\/\/doi.org\/10.1038\/s41586-023-06291-2 (2023).","journal-title":"Nature"},{"key":"89996_CR22","doi-asserted-by":"publisher","unstructured":"Morris, J. X. et al. Tree Prompting: efficient Task Adaptation without Fine-tuning. Published Online Oct. 21 https:\/\/doi.org\/10.48550\/arXiv.2310.14034 (2023).","DOI":"10.48550\/arXiv.2310.14034"},{"key":"89996_CR23","doi-asserted-by":"publisher","unstructured":"Nori, H. et al. Can Generalist Foundation models outcompete special-purpose tuning? Case Study in Medicine. Published Online November. 27 https:\/\/doi.org\/10.48550\/arXiv.2311.16452 (2023).","DOI":"10.48550\/arXiv.2311.16452"},{"key":"89996_CR24","doi-asserted-by":"publisher","unstructured":"Singh, C. et al. Rethinking interpretability in the era of large Language models. Published Online January. 30 https:\/\/doi.org\/10.48550\/arXiv.2402.01761 (2024).","DOI":"10.48550\/arXiv.2402.01761"},{"key":"89996_CR25","doi-asserted-by":"publisher","unstructured":"Gero, Z. et al. Self-Verification improves few-shot clinical information extraction. Published Online May. 30 https:\/\/doi.org\/10.48550\/arXiv.2306.00024 (2023).","DOI":"10.48550\/arXiv.2306.00024"},{"key":"89996_CR26","doi-asserted-by":"publisher","unstructured":"Pan, L. et al. Automatically correcting large Language models: surveying the landscape of diverse self-correction strategies. Published Online August. 29 https:\/\/doi.org\/10.48550\/arXiv.2308.03188 (2023).","DOI":"10.48550\/arXiv.2308.03188"},{"issue":"7","key":"89996_CR27","doi-asserted-by":"publisher","first-page":"e197591","DOI":"10.1001\/jamanetworkopen.2019.7591","volume":"2","author":"WB Feldman","year":"2019","unstructured":"Feldman, W. B. et al. Public Approval of Exception from Informed Consent in emergency clinical trials: a systematic review of Community Consultation surveys. JAMA Netw. Open. 2 (7), e197591. https:\/\/doi.org\/10.1001\/jamanetworkopen.2019.7591 (2019).","journal-title":"JAMA Netw. Open."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-89996-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-89996-w","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-89996-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,20]],"date-time":"2025-02-20T06:31:58Z","timestamp":1740033118000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-89996-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,20]]},"references-count":27,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["89996"],"URL":"https:\/\/doi.org\/10.1038\/s41598-025-89996-w","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,20]]},"assertion":[{"value":"27 November 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 February 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 February 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Dr. Aaron Kornblith is a consultant and co-founder for the University of California San Francisco spinout Capture Dx, Inc. and is supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health under award number\u202fK23HD110716 (AK). This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS, or the U.S. Government. All the remaining authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"This study was approved under a Single IRB by the University of Utah Institutional Review Board, under the PECARN SIRB PediDOSE Protocol 052, IRB Approval number 001397941.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}}],"article-number":"6179"}}