{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T21:43:54Z","timestamp":1792100634450,"version":"4.4.0"},"reference-count":30,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Regulatory frameworks such as the Belmont Report, the Common Rule, and the Declaration of Helsinki require informed consent to ensure participants understand a study\u2019s purpose and can make voluntary decisions about their involvement. Regulations, including the General Data Protection Regulation (Regulation [EU] 2016\/679), further emphasize that consent must be freely given and revocable without disadvantage. Although informed consent forms (ICFs) are intended to be clear and accessible, they have become increasingly lengthy and complex. Large language models (LLMs) offer the potential to navigate and interpret this complexity and have shown promise in biomedical information extraction tasks. However, their susceptibility to hallucinations limits their reliability in high-stakes settings. Retrieval-augmented generation (RAG) can mitigate such errors.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study evaluates the integration of GPT-4.1 with RAG for reviewing data reuse language in ICFs and its ability to interpret complex ICF language.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n First, we processed 438 ICFs from different trials, including multiple countries, languages, and versions of ICFs. Using expertly curated prompts, we extracted information about data reuse using GPT-4.1. By comparing the LLM-generated data reuse outputs with human expert ground truth, we evaluated accuracy and the time required to extract information for each ICF. To further validate the workflow, we evaluated an independent set of 488 ICFs spanning additional trials, languages, and regions. During the validation phase, the workflow was evaluated using a baseline assessment (89 ICFs) followed by 3 iterative optimization rounds assessed on 14, 281, and 104 ICFs, respectively. For this cohort, we assessed the correctness of LLM outputs along with the quality of supporting evidence provided by the model.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Across the initial set of 438 ICFs, the system achieved 81.6% accuracy. During validation, the baseline accuracy using the preoptimized prompts was 84.0% (89 documents). Accuracy improved sequentially across the 3 iterative optimization rounds, increasing from 82.3% in the first round (14 documents) to 90.7% in the second round (281 documents), and reaching 92.1% in the third and final round on an independent holdout cohort of 104 documents. Ultimately, using our RAG-based approach, the system accurately extracted data reuse information across multiple languages.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Our RAG-based workflow demonstrated the ability to effectively support the ICF review process for data reuse and interpret complex consent language, contributing to improved consistency in ICF evaluations for data reuse.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/95417","type":"journal-article","created":{"date-parts":[[2026,8,14]],"date-time":"2026-08-14T17:45:05Z","timestamp":1786729505000},"page":"e95417-e95417","source":"Crossref","is-referenced-by-count":0,"title":["Large Language Model\u2013Assisted Review of Clinical Trial Informed Consent for Data Reuse: Retrieval-Augmented Generation Approach"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6468-2643","authenticated-orcid":false,"given":"Lameck Mbangula","family":"Amugongo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-7890-6673","authenticated-orcid":false,"given":"Lena","family":"Schaller","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-1038-3639","authenticated-orcid":false,"given":"Helene","family":"Wendt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6000-938X","authenticated-orcid":false,"given":"R Maarten","family":"van Dijk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2924-1241","authenticated-orcid":false,"given":"Enrica","family":"Zanuttigh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-3202-228X","authenticated-orcid":false,"given":"Claudia","family":"Neumann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1412-3742","authenticated-orcid":false,"given":"Andreas","family":"Freisinger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-5652-4909","authenticated-orcid":false,"given":"Jaroslaw","family":"Deska","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,15]]},"reference":[{"key":"R1","unstructured":"45 CFR Part 46\u2014Protection of Human Subjects. Electronic Code of Federal Regulations (eCFR). 2018. URL: https:\/\/www.ecfr.gov\/on\/2018-07-19\/title-45\/subtitle-A\/subchapter-A\/part-46 [Accessed 23-09-2026]"},{"issue":"7","key":"R2","doi-asserted-by":"publisher","DOI":"10.1056\/AIoa2400181","article-title":"Retrieval-augmented generation-enabled GPT-4 for clinical trial screening","volume":"1","author":"Unlu","journal-title":"NEJM AI"},{"issue":"12","key":"R3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3571730","article-title":"Survey of hallucination in natural language generation","volume":"55","author":"Ji","journal-title":"ACM Comput Surv"},{"key":"R4","unstructured":"Lewis P Perez E Piktus A et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Larochelle H Ranzato MA Hadsell R Balcan MF Lin HT , editors. NIPS \u201920: Proceedings of the 34th International Conference on Neural Information Processing Systems. Curran Associates Inc; 2020:9459-9474. URL: https:\/\/dl.acm.org\/doi\/abs\/10.5555\/3495724.3496517 [Accessed 23-09-2026]"},{"issue":"6","key":"R5","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000877","article-title":"Retrieval augmented generation for large language models in healthcare: a systematic review","volume":"4","author":"Amugongo","journal-title":"PLoS Digit Health"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.2196\/68139","article-title":"Transforming informed consent generation using large language models: mixed methods study","volume":"13","author":"Shi","journal-title":"JMIR Med Inform"},{"key":"R7","unstructured":"Wang Z Gao J Danek B et al. InformGen: an AI copilot for accurate and compliant clinical research consent document generation. arXiv. Preprint posted online on Apr 1, 2025. [doi: 10.48550\/arxiv.2504.00934]"},{"key":"R8","unstructured":"Liu H Borgman A Gagnon J Boisvert D Valian H . Revolutionizing informed consent form analysis with generative AI: enhancing efficiency in drug development and data re-use. Presented at: Proceedings of PHUSE US Connect 2024; Feb 25-28, 2024; Bethesda, Maryland, USA. URL: https:\/\/phuse.s3.eu-central-1.amazonaws.com\/Archive\/2024\/Connect\/US\/Bethesda\/PRE_IC07.pdf [Accessed 23-09-2026]"},{"issue":"10","key":"R9","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.36997","article-title":"Large language model-based chatbot vs surgeon-generated informed consent documentation for common procedures","volume":"6","author":"Decker","journal-title":"JAMA Netw Open"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/j.jcms.2024.10.002","article-title":"Evaluating AI-generated informed consent documents in oral surgery: a comparative study of ChatGPT-4, Bard Gemini advanced, and human-written consents","volume":"53","author":"Vaira","journal-title":"J Craniomaxillofac Surg"},{"key":"R11","unstructured":"OpenAIAchiam J Adler S Agarwal S et al. GPT-4 technical report. arXiv. Preprint posted online on Mar 15, 2023. [doi: 10.48550\/arxiv.2303.08774]"},{"key":"R12","unstructured":"Gemini TeamAnil R Borgeaud S et al. Gemini: a family of highly capable multimodal models. arXiv. Preprint posted online on Dec 19, 2023. [doi: 10.48550\/arxiv.2312.11805]"},{"issue":"2","key":"R13","doi-asserted-by":"publisher","first-page":"1145","DOI":"10.3390\/make6020053","article-title":"Evaluation of AI chatbots for the creation of patient-informed consent sheets","volume":"6","author":"Raimann","journal-title":"Mach Learn Knowl Extr"},{"issue":"2","key":"R14","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1136\/jme-2024-110652","article-title":"Large language models for surgical informed consent: an ethical perspective on simulated empathy","volume":"52","author":"Rudra","journal-title":"J Med Ethics"},{"issue":"2","key":"R15","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1111\/bioe.70030","article-title":"Empowering patient autonomy: the role of large language models (LLMs) in scaffolding informed consent in medical practice","volume":"40","author":"Allen","journal-title":"Bioethics"},{"key":"R16","unstructured":"Brown TB Mann B Ryder N et al. Language models are few-shot learners. In: Larochelle H Ranzato MA Hadsell R Balcan MF Lin HT , editors. NIPS \u201920: Proceedings of the 34th International Conference on Neural Information Processing Systems. Curran Associates Inc; 2020:1877-1901. URL: https:\/\/dl.acm.org\/doi\/10.5555\/3495724.3495883 [Accessed 23-09-2026]"},{"key":"R17","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-5446","article-title":"GLUE: a multi-task benchmark and analysis platform for natural language understanding","author":"Wang","journal-title":"Proc EMNLP Workshop BlackboxNLP"},{"key":"R18","unstructured":"Adiwardana D Luong MT So DR et al. Towards a human-like open-domain chatbot. arXiv. Preprint posted online on Jan 27, 2020. [doi: 10.48550\/arxiv.2001.09977]"},{"key":"R19","unstructured":"Comanici G Bieber E Schaekermann M et al. Gemini 2.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv. Preprint posted online on Jul 7, 2025. [doi: 10.48550\/arxiv.2507.06261]"},{"key":"R20","unstructured":"Introducing GPT\u20114.1 in the API. OpenAI. 2025. URL: https:\/\/openai.com\/index\/gpt-4-1\/ [Accessed 06-02-2026]"},{"key":"R21","unstructured":"Text-embedding-3-small. OpenAI Developer. URL: https:\/\/developers.openai.com\/api\/docs\/models\/text-embedding-3-small [Accessed 23-09-2026]"},{"key":"#cr-split#-R22.1","unstructured":"Regulation (EU) 2016\/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95\/46\/EC (General Data Protection Regulation). European Union"},{"key":"#cr-split#-R22.2","unstructured":"May 4, 2016. URL: https:\/\/eur-lex.europa.eu\/eli\/reg\/2016\/679\/oj\/eng [Accessed 23-09-2026]"},{"key":"R23","doi-asserted-by":"crossref","unstructured":"Geroimenko V . Key techniques for writing effective prompts. In: The Essential Guide to Prompt Engineering Key Principles, Techniques, Challenges, and Security Risks. Springer; 2025:37-83. [doi: 10.1007\/978-3-031-86206-9_3]","DOI":"10.1007\/978-3-031-86206-9_3"},{"key":"R24","unstructured":"Colvin S Jolibois E Ramezani H et al. Pydantic. Zenodo. 2025. URL: https:\/\/zenodo.org\/records\/15097903 [Accessed 23-09-2026]"},{"key":"R25","unstructured":"Streamlit. URL: https:\/\/streamlit.io\/ [Accessed 19-02-2026]"},{"key":"R26","unstructured":"Posit Connect Documentation Version 2026.09.0. Connect Documentation. URL: https:\/\/docs.posit.co\/connect\/ [Accessed 23-09-2026]"},{"issue":"9","key":"R27","doi-asserted-by":"publisher","first-page":"412","DOI":"10.3390\/fi17090412","article-title":"GPT-4.1 sets the standard in automated experiment design using novel Python libraries","volume":"17","author":"Fachada","journal-title":"Future Internet"},{"issue":"8047","key":"R28","doi-asserted-by":"publisher","first-page":"778","DOI":"10.1038\/d41586-025-00068-5","article-title":"AI hallucinations can\u2019t be stopped\u2014but these techniques can limit their damage","volume":"637","author":"Jones","journal-title":"Nature"},{"key":"R29","doi-asserted-by":"publisher","DOI":"10.52202\/068431-1800","article-title":"Chain-of-thought prompting elicits reasoning in large language models","author":"Wei","journal-title":"Proc 36th Int Conf Neural Inf Process Syst"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T20:44:55Z","timestamp":1792097095000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e95417"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,15]]},"references-count":30,"URL":"https:\/\/doi.org\/10.2196\/95417","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,15]]},"article-number":"v14i7e95417"}}