{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T16:42:50Z","timestamp":1791996170646,"version":"4.4.0"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T00:00:00Z","timestamp":1768953600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T00:00:00Z","timestamp":1768953600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["AAPS J"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    To evaluate the strengths and limitations of retrieval-augmented generative (RAG) artificial intelligence (AI) for natural language querying of biologics immunogenicity data. The package inserts for drugs approved with biologics license applications (BLA) were retrieved from DailyMed (\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/dailymed.nlm.nih.gov\/dailymed\/\" ext-link-type=\"uri\">https:\/\/dailymed.nlm.nih.gov\/dailymed\/<\/jats:ext-link>\n                    ). The RAG system integrated natural language processing, retrieval, and large language model (LLM) components. ChatGPT, Gemini, DeepSeek, and Llama were queried with five clinical pharmacology-focused questions on factors influencing anti-drug antibody (ADA) incidence and tolerability, including effects of target protein, administration route, and citrate excipients. Outputs were assessed for relevance, faithfulness, and domain-specific accuracy. The dataset included 663 biologics, of which 206 (31.1%) were monoclonal antibodies. The RAG system retrieved relevant contexts for all queries, but several contexts contained inaccuracies related to the presence of non-antibody protein drugs. All four LLMs generated coherent summaries and identified determinants of ADA incidence, such as drug type, assay methods, and concomitant therapy. All models found that injection-site pain occurred with some protein therapeutics containing citrate excipients, and that evidence for a direct causal role of citrate was mixed. Comparative evaluation showed that LLM outputs were generally relevant and faithful to the source text, with variation in the level of detail and comprehensiveness across models. Domain-specific evaluations indicated that responses accurately identified trends in immunogenicity and highlighted the knowledge gaps. While RAG-based systems can retrieve and synthesize immunogenicity assessments from multiple source documents, significant limitations were noted in this use case. The effectiveness of the retriever can limit RAG performance and warrant refinement.\n                  <\/jats:p>\n                  <jats:p>\n                    <jats:bold>Graphical Abstract<\/jats:bold>\n                  <\/jats:p>","DOI":"10.1208\/s12248-025-01199-3","type":"journal-article","created":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T21:57:55Z","timestamp":1769032675000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Retrieval Augmented Generation (RAG) for Natural Language Querying of Immunogenicity Data for Protein Drugs"],"prefix":"10.1208","volume":"28","author":[{"given":"Nikhil","family":"Advani","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amruta Gajanan","family":"Bhat","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sathy","family":"Balu-Iyer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Murali","family":"Ramanathan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,1,21]]},"reference":[{"issue":"5","key":"1199_CR1","doi-asserted-by":"publisher","first-page":"1637","DOI":"10.1016\/j.xphs.2018.12.014","volume":"108","author":"R Dingman","year":"2019","unstructured":"Dingman R, Balu-Iyer SV. Immunogenicity of protein pharmaceuticals. J Pharm Sci. 2019;108(5):1637\u201354.","journal-title":"J Pharm Sci"},{"issue":"2","key":"1199_CR2","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1208\/s12248-015-9857-y","volume":"18","author":"YM Wang","year":"2016","unstructured":"Wang YM, Wang J, Hon YY, Zhou L, Fang L, Ahn HY. Evaluating and reporting the immunogenicity impacts for biological products--a clinical pharmacology perspective. AAPS J. 2016;18(2):395\u2013403.","journal-title":"AAPS J"},{"key":"1199_CR3","doi-asserted-by":"publisher","DOI":"10.3389\/fimmu.2020.01301","volume":"11","author":"V Jawa","year":"2020","unstructured":"Jawa V, et al. T-cell dependent immunogenicity of protein therapeutics pre-clinical assessment and mitigation-updated consensus and review 2020. Front Immunol. 2020;11:1301.","journal-title":"Front Immunol"},{"key":"1199_CR4","unstructured":"OpenAI. ChatGPT (June 26 version) Large language model. 2023. <https:\/\/chat.openai.com\/chat>."},{"key":"1199_CR5","unstructured":"Google AI. Bard Large language model. 2023 <https:\/\/bard.google.com>."},{"key":"1199_CR6","unstructured":"Meta. Llama 2: open source, free for research and commercial use. 2024. <https:\/\/llama.meta.com\/llama2\/>."},{"key":"1199_CR7","unstructured":"Anthropic. Meet Claude. 2024 <https:\/\/www.anthropic.com\/claude>."},{"key":"1199_CR8","unstructured":"DeepSeek-AI et al. DeepSeek-V2: a strong, economical, and efficient mixture-of-experts language model. 2024. <https:\/\/ui.adsabs.harvard.edu\/abs\/2024arXiv240504434D>."},{"key":"1199_CR9","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1002\/cpt.3279","volume":"116","author":"W Anderson","year":"2024","unstructured":"Anderson W, et al. Unlocking the capabilities of large language models for accelerating drug development. Clin Pharmacol Ther. 2024;116:38\u201341.","journal-title":"Clin Pharmacol Ther"},{"key":"1199_CR10","doi-asserted-by":"publisher","first-page":"3320","DOI":"10.1111\/bcp.16215","volume":"90","author":"E Shin","year":"2024","unstructured":"Shin E, Hartman M, Ramanathan M. Performance of the ChatGPT large language model for decision support in community pharmacy. Br J Clin Pharmacol. 2024;90:3320\u201333.","journal-title":"Br J Clin Pharmacol"},{"key":"1199_CR11","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1007\/s10928-023-09892-6","volume":"51","author":"E Shin","year":"2024","unstructured":"Shin E, Ramanathan M. Evaluation of prompt engineering strategies for pharmacokinetic data analysis with the ChatGPT large language model. J Pharmacokinet Pharmacodyn. 2024;51:101\u20138.","journal-title":"J Pharmacokinet Pharmacodyn"},{"key":"1199_CR12","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1007\/s10928-024-09921-y","volume":"51","author":"E Shin","year":"2024","unstructured":"Shin E, Yu Y, Bies RR, Ramanathan M. Evaluation of ChatGPT and Gemini large language models for pharmacometrics with NONMEM. J Pharmacokinet Pharmacodyn. 2024;51:187\u201397.","journal-title":"J Pharmacokinet Pharmacodyn"},{"key":"1199_CR13","unstructured":"Lewis P, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. 2020. <https:\/\/ui.adsabs.harvard.edu\/abs\/2020arXiv200511401L>."},{"key":"1199_CR14","doi-asserted-by":"crossref","unstructured":"Xiong G, Jin Q, Lu Z, Zhang A. Benchmarking retrieval-augmented generation for medicine. 2024. <https:\/\/ui.adsabs.harvard.edu\/abs\/2024arXiv240213178X>.","DOI":"10.18653\/v1\/2024.findings-acl.372"},{"key":"1199_CR15","doi-asserted-by":"publisher","first-page":"655","DOI":"10.1007\/s40264-025-01520-1","volume":"48","author":"L Wu","year":"2025","unstructured":"Wu L, Fang H, Qu Y, Xu J, Tong W. Leveraging FDA labeling documents and large language model to enhance annotation, profiling, and classification of drug adverse events with AskFDALabel. Drug Saf. 2025;48:655\u201365.","journal-title":"Drug Saf"},{"key":"1199_CR16","unstructured":"LangChain. LangChain. 2023. <https:\/\/www.langchain.com>. Accessed May 7, 2024 2024."},{"key":"1199_CR17","unstructured":"Anonymous. FDALabel. 2024. <https:\/\/nctr-crs.fda.gov\/fdalabel\/ui\/search>."},{"key":"1199_CR18","unstructured":"Wolf T, et al. HuggingFace's Transformers: State-of-the-art Natural Language Processing. 2019 <https:\/\/ui.adsabs.harvard.edu\/abs\/2019arXiv191003771W>"},{"key":"1199_CR19","unstructured":"Song K, Tan X, Qin T, Lu J, Liu T-Y. MPNet: masked and permuted pre-training for language understanding. 2020. <https:\/\/ui.adsabs.harvard.edu\/abs\/2020arXiv200409297S>."},{"key":"1199_CR20","unstructured":"Adams D, Suri G, Chali Y. Combining state-of-the-art models with maximal marginal relevance for few-shot and zero-shot multi-document summarization. 2022. <https:\/\/ui.adsabs.harvard.edu\/abs\/2022arXiv221110808A>."},{"key":"1199_CR21","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1561\/1500000019","volume":"3","author":"S Robertson","year":"2009","unstructured":"Robertson S, Zaragoza H. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends\u00ae in Information Retrieval. 2009;3:333\u201389. https:\/\/doi.org\/10.1561\/1500000019.","journal-title":"Foundations and Trends\u00ae in Information Retrieval"},{"key":"1199_CR22","unstructured":"DeepEval Documentation. Answer relevancy. 2025. <https:\/\/deepeval.com\/docs\/metrics-answer-relevancy>. Accessed November 22 2025."},{"key":"1199_CR23","unstructured":"Yu F. When AIs Judge AIs: the rise of agent-as-a-judge evaluation for LLMs. 2025. <https:\/\/ui.adsabs.harvard.edu\/abs\/2025arXiv250802994Y>."},{"key":"1199_CR24","unstructured":"Papadimitriou I, Gialampoukidis I, Vrochidis S, Ioannis & Kompatsiaris. RAG playground: a framework for systematic evaluation of retrieval strategies and prompt engineering in RAG systems. 2024 <https:\/\/ui.adsabs.harvard.edu\/abs\/2024arXiv241212322P>."},{"key":"1199_CR25","doi-asserted-by":"publisher","DOI":"10.3389\/fimmu.2022.975803","volume":"13","author":"HCJ Ertl","year":"2022","unstructured":"Ertl HCJ. Immunogenicity and toxicity of AAV gene therapy. Front Immunol. 2022;13:975803.","journal-title":"Front Immunol"},{"key":"1199_CR26","doi-asserted-by":"publisher","first-page":"1517","DOI":"10.1056\/NEJMoa1614062","volume":"376","author":"PM Ridker","year":"2017","unstructured":"Ridker PM, et al. Lipid-reduction variability and antidrug-antibody formation with bococizumab. N Engl J Med. 2017;376:1517\u201326.","journal-title":"N Engl J Med"},{"key":"1199_CR27","doi-asserted-by":"publisher","first-page":"1244","DOI":"10.1111\/jth.12634","volume":"12","author":"SR Lentz","year":"2014","unstructured":"Lentz SR, et al. Recombinant factor VIIa analog in the management of hemophilia with inhibitors: results from a multicenter, randomized, controlled trial of vatreptacog alfa. J Thromb Haemost. 2014;12:1244\u201353.","journal-title":"J Thromb Haemost"},{"key":"1199_CR28","doi-asserted-by":"publisher","first-page":"1989","DOI":"10.1111\/jth.13141","volume":"13","author":"JN Mahlangu","year":"2015","unstructured":"Mahlangu JN, et al. Changes in the amino acid sequence of the recombinant human factor VIIa analog, vatreptacog alfa, are associated with clinical immunogenicity. J Thromb Haemost. 2015;13:1989\u201398.","journal-title":"J Thromb Haemost"},{"key":"1199_CR29","doi-asserted-by":"publisher","first-page":"3241","DOI":"10.1182\/blood.V98.12.3241","volume":"98","author":"J Li","year":"2001","unstructured":"Li J, et al. Thrombocytopenia caused by the development of antibodies to thrombopoietin. Blood. 2001;98:3241\u20138.","journal-title":"Blood"},{"issue":"5","key":"1199_CR30","doi-asserted-by":"publisher","DOI":"10.1208\/s12248-023-00846-x","volume":"25","author":"F McCush","year":"2023","unstructured":"McCush F, Wang E, Yunis C, Schwartz P, Baltrukonis D. Anti-drug antibody magnitude and clinical relevance using signal to noise (S\/N): bococizumab case study. AAPS J. 2023;25(5):85.","journal-title":"AAPS J"},{"key":"1199_CR31","doi-asserted-by":"publisher","first-page":"1719","DOI":"10.1007\/s11095-013-1062-z","volume":"30","author":"V Brinks","year":"2013","unstructured":"Brinks V, et al. Preclinical models used for immunogenicity prediction of therapeutic proteins. Pharm Res. 2013;30:1719\u201328.","journal-title":"Pharm Res"},{"issue":"6","key":"1199_CR32","doi-asserted-by":"publisher","first-page":"1587","DOI":"10.1208\/s12248-017-0143-z","volume":"19","author":"J Gokemeijer","year":"2017","unstructured":"Gokemeijer J, Jawa V, Mitra-Kaushik S. How close are we to profiling immunogenicity risk using in silico algorithms and in vitro methods?: an industry perspective. AAPS J. 2017;19(6):1587\u201392.","journal-title":"AAPS J"},{"key":"1199_CR33","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1093\/toxsci\/kfae122","volume":"203","author":"SA Carratt","year":"2025","unstructured":"Carratt SA, et al. An industry perspective on the FDA Modernization Act 2.0\/3.0: potential next steps for sponsors to reduce animal use in drug development. Toxicol Sci. 2025;203:28\u201334.","journal-title":"Toxicol Sci"},{"issue":"6","key":"1199_CR34","doi-asserted-by":"publisher","DOI":"10.1007\/s10928-025-10005-8","volume":"52","author":"AG Bhat","year":"2025","unstructured":"Bhat AG, Shin E, Roy A, Ramanathan M. Scoping review of the role of pharmacometrics in model-informed drug development. J Pharmacokinet Pharmacodyn. 2025;52(6):56.","journal-title":"J Pharmacokinet Pharmacodyn"},{"key":"1199_CR35","unstructured":"United States Food & Drug Administration (FDA). FDA Launches Agency-Wide AI Tool to Optimize Performance for the American People. (ed. U.S. Department of Health and Human Services) (U.S. Department of Health and Human Services, Rockville, MD, 2025) <https:\/\/www.fda.gov\/news-events\/press-announcements\/fda-launches-agency-wide-ai-tool-optimize-performance-american-people>."}],"container-title":["The AAPS Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1208\/s12248-025-01199-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1208\/s12248-025-01199-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1208\/s12248-025-01199-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T17:35:09Z","timestamp":1774460109000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1208\/s12248-025-01199-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,21]]},"references-count":35,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["1199"],"URL":"https:\/\/doi.org\/10.1208\/s12248-025-01199-3","relation":{},"ISSN":["1550-7416"],"issn-type":[{"value":"1550-7416","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,21]]},"assertion":[{"value":"20 October 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 December 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 January 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Artificial intelligence-enabled tools, such as Grammarly, Microsoft Word, and ChatGPT, were used to improve readability and language. Bibliographic tools, including Google and EndNote, helped curate references, add citations, and generate bibliographies. The generative AI features of graphics tools, like Adobe Illustrator, were utilized to create prototypes for some figures.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Artificial Intelligence Statement"}},{"value":"Nikhil Advani has nothing to disclose.Amruta Gajanan Bhat has nothing to disclose.Sathy Balu-Iyer\u2019s patents have been licensed by three biotech companies Immunomodulatory Therapies, Truvai Biosciences, and Immunotolerx.Murali Ramanathan received research funding from the\u00a0Gates Foundation, National Multiple Sclerosis Society, the Department of Defense, and the National Institute of Neurological Diseases and Stroke. He receives a royalty from a self-published textbook.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Disclosure"}}],"article-number":"51"}}