{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,30]],"date-time":"2026-05-30T01:02:28Z","timestamp":1780102948557,"version":"3.54.0"},"reference-count":56,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T00:00:00Z","timestamp":1760313600000},"content-version":"vor","delay-in-days":43,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["92370106"],"award-info":[{"award-number":["92370106"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62072376"],"award-info":[{"award-number":["62072376"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,8,31]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Understanding the roles of genes, phenotypes, and diseases is crucial for advancing biomedical research. However, efficient and accessible retrieval of biomedical knowledge remains a challenge due to the complexity of the relevant data. We introduce BioRAGent, an intelligent biomedical assistant that combines Tool-augmented retrieval-augmented generation (RAG) with a multiagent system. Leveraging the ability of large language models, BioRAGent facilitates natural language queries about genes, phenotypes, diseases, and their interrelationships. BioRAGent employs three specialized agents: Guide (query optimization), Retriever (data retrieval), and Reviewer (answer validation) to access authoritative biomedical databases and to generate accurate responses. We evaluate the performance of BioRAGent on a benchmark of eleven single-hop and three multi-hop tasks, demonstrating superior results compared with state-of-the-art models. User evaluations highlight the practicality and robust user experience of BioRAGent, particularly in handling complex multi-hop queries. Moreover, ablation experiments validate the contribution of each agent in improving retrieval accuracy.<\/jats:p>","DOI":"10.1093\/bib\/bbaf539","type":"journal-article","created":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:36:44Z","timestamp":1760362604000},"source":"Crossref","is-referenced-by-count":5,"title":["BioRAGent: natural language biomedical querying with retrieval-augmented multiagent systems"],"prefix":"10.1093","volume":"26","author":[{"given":"Manlian","family":"Bi","sequence":"first","affiliation":[{"name":"AI for Science Interdisciplinary Research Center, School of Computer Science, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]},{"name":"Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhijie","family":"Bao","sequence":"additional","affiliation":[{"name":"AI for Science Interdisciplinary Research Center, School of Computer Science, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]},{"name":"School of Data Science, Fudan University , No. 220 Handan Road, Shanghai 200433 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dongna","family":"Xie","sequence":"additional","affiliation":[{"name":"AI for Science Interdisciplinary Research Center, School of Computer Science, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]},{"name":"Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaohan","family":"Xie","sequence":"additional","affiliation":[{"name":"AI for Science Interdisciplinary Research Center, School of Computer Science, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]},{"name":"Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Changxiao","family":"Yang","sequence":"additional","affiliation":[{"name":"AI for Science Interdisciplinary Research Center, School of Computer Science, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]},{"name":"Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tao","family":"Wang","sequence":"additional","affiliation":[{"name":"AI for Science Interdisciplinary Research Center, School of Computer Science, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]},{"name":"Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongtian","family":"Wang","sequence":"additional","affiliation":[{"name":"AI for Science Interdisciplinary Research Center, School of Computer Science, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]},{"name":"Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiajie","family":"Peng","sequence":"additional","affiliation":[{"name":"AI for Science Interdisciplinary Research Center, School of Computer Science, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]},{"name":"Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology, Northwestern Polytechnical University , No. 1 Dongxiang Road, Xi\u2019an 710129 ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,10,13]]},"reference":[{"key":"2025101309272600600_ref1","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1038\/s41586-020-2817-4","article-title":"Strategic vision for improving human health at the forefront of genomics","volume":"586","author":"Green","year":"2020","journal-title":"Nature"},{"key":"2025101309272600600_ref2","doi-asserted-by":"publisher","DOI":"10.1016\/j.drudis.2024.104280","article-title":"Biomedical data analytics for better patient outcomes","volume":"30","author":"Ghofrani","year":"2024","journal-title":"Drug Discov Today"},{"key":"2025101309272600600_ref3","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1038\/s41586-019-1879-7","article-title":"A brief history of human disease genetics","volume":"577","author":"Claussnitzer","year":"2020","journal-title":"Nature"},{"key":"2025101309272600600_ref4","doi-asserted-by":"publisher","first-page":"D1","DOI":"10.1093\/nar\/gkp1077","article-title":"The 2010 nucleic acids research database issue and online database collection: a community of data resources","volume":"38","author":"Cochrane","year":"2010","journal-title":"Nucleic Acids Res"},{"key":"2025101309272600600_ref5","doi-asserted-by":"publisher","first-page":"D1","DOI":"10.1093\/nar\/gkae1220","article-title":"The 2025 nucleic acids research database issue and the online molecular biology database collection","volume":"53","author":"Rigden","year":"2025","journal-title":"Nucleic Acids Res"},{"key":"2025101309272600600_ref6","doi-asserted-by":"publisher","DOI":"10.1093\/database\/bax061","article-title":"A publicly available benchmark for biomedical dataset retrieval: the reference standard for the 2016 bioCADDIE dataset retrieval challenge","volume":"2017","author":"Cohen","year":"2017","journal-title":"Database (Oxford)"},{"key":"2025101309272600600_ref7","doi-asserted-by":"publisher","first-page":"BII.S31559","DOI":"10.4137\/BII.S31559","article-title":"Big data application in biomedical research and health care: a literature review","volume":"8","author":"Luo","year":"2016","journal-title":"Biomed Inform Insights"},{"key":"2025101309272600600_ref8","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1038\/s41746-019-0158-1","article-title":"Why digital medicine depends on interoperability","volume":"2","author":"Lehne","year":"2019","journal-title":"NPJ Digit Med"},{"key":"2025101309272600600_ref9","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1093\/bioinformatics\/9.1.49","article-title":"SRS\u2013an indexing and retrieval tool for flat file data libraries","volume":"9","author":"Etzold","year":"1993","journal-title":"Comput Appl Biosci"},{"key":"2025101309272600600_ref10","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1016\/S0076-6879(96)66012-1","article-title":"Entrez: molecular biology database and retrieval system","volume":"266","author":"Schuler","year":"1996","journal-title":"Methods Enzymol"},{"key":"2025101309272600600_ref11","article-title":"Bioagents: an industrial perspective","volume-title":"Network Tools and Applications in Biology","author":"Toldo"},{"key":"2025101309272600600_ref12","article-title":"A survey of large language models","author":"Zhao","year":"2023"},{"key":"2025101309272600600_ref13","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbad493","article-title":"Opportunities and challenges for ChatGPT and large language models in biomedicine and health","volume":"25","author":"Tian","year":"2023","journal-title":"Brief Bioinform"},{"key":"2025101309272600600_ref14","article-title":"DISC-MedLLM: bridging general large language models and real-world medical consultation","author":"Bao","year":"2023"},{"key":"2025101309272600600_ref15","doi-asserted-by":"publisher","first-page":"1233","DOI":"10.1056\/NEJMsr2214184","article-title":"Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine","volume":"388","author":"Lee","year":"2023","journal-title":"N Engl J Med"},{"key":"2025101309272600600_ref16","article-title":"Retrieval-augmented generation for large language models: a survey","author":"Gao","year":"2023"},{"key":"2025101309272600600_ref17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3571730","article-title":"Survey of hallucination in natural language generation","volume":"55","author":"Ji","year":"2023","journal-title":"ACM Comput Surv"},{"key":"2025101309272600600_ref18","article-title":"BioRAG: a RAG-LLM framework for biological question reasoning","author":"Wang","year":"2024"},{"key":"2025101309272600600_ref19","doi-asserted-by":"publisher","first-page":"2878","DOI":"10.1038\/s41591-024-03148-7","article-title":"Outpatient reception via collaboration between nurses and a large language model: a randomized controlled trial","volume":"30","author":"Wan","year":"2024","journal-title":"Nat Med"},{"key":"2025101309272600600_ref20","first-page":"126544","article-title":"Gorilla: large language model connected with massive APIs","volume":"37","author":"Patil","year":"2024","journal-title":"Adv Neural Inform Process Syst"},{"key":"2025101309272600600_ref21","doi-asserted-by":"publisher","first-page":"68539","DOI":"10.48550\/arXiv.2302.04761","article-title":"Toolformer: language models can teach themselves to use tools","volume":"36","author":"Schick","year":"2023","journal-title":"Adv Neural Inform Process Syst"},{"key":"2025101309272600600_ref22","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.16789","article-title":"ToolLLM: facilitating large language models to master 16000+ real-world APIs","author":"Qin","year":"2023"},{"key":"2025101309272600600_ref23","first-page":"38154","article-title":"HuggingGPT: solving AI tasks with ChatGPT and its friends in hugging face","volume":"36","author":"Shen","year":"2023","journal-title":"Adv Neural Inform Process Syst"},{"key":"2025101309272600600_ref24","article-title":"Large language model based multi-agents: a survey of progress and challenges","author":"Guo","year":"2024"},{"key":"2025101309272600600_ref25","article-title":"CellAgent: an LLM-driven multi-agent framework for automated single-cell data analysis","author":"Xiao","year":"2024"},{"key":"2025101309272600600_ref26","article-title":"Chateval: towards better LLM-based evaluators through multi-agent debate","author":"Chan","year":"2023"},{"key":"2025101309272600600_ref27","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-031-88720-8_1","article-title":"BioRAGent: a retrieval-augmented generation system for showcasing generative query expansion and domain-specific search for Scientific Q&A","volume-title":"European Conference on Information Retrieval, Lucca, Italy","author":"Ateia"},{"key":"2025101309272600600_ref28","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","volume":"33","author":"Lewis","year":"2020","journal-title":"Adv Neural Inform Process Syst"},{"key":"2025101309272600600_ref29","doi-asserted-by":"crossref","article-title":"Unleashing the emergent cognitive synergy in large language models: a task-solving agent through multi-persona self-collaboration","author":"Wang","DOI":"10.18653\/v1\/2024.naacl-long.15"},{"key":"2025101309272600600_ref30","article-title":"An LLM compiler for parallel function calling","volume-title":"Forty-first International Conference on Machine Learning","author":"Kim"},{"key":"2025101309272600600_ref31","doi-asserted-by":"publisher","first-page":"D1333","DOI":"10.1093\/nar\/gkad1005","article-title":"The human phenotype ontology in 2024: phenotypes around the world","volume":"52","author":"Gargano","year":"2024","journal-title":"Nucleic Acids Res"},{"key":"2025101309272600600_ref32","doi-asserted-by":"publisher","first-page":"D20","DOI":"10.1093\/nar\/gkae979","article-title":"Database resources of the National Center for biotechnology information in 2025","volume":"53","author":"Sayers","year":"2025","journal-title":"Nucleic Acids Res"},{"key":"2025101309272600600_ref33","doi-asserted-by":"publisher","first-page":"D891","DOI":"10.1093\/nar\/gkad1049","article-title":"Ensembl 2024","volume":"52","author":"Harrison","year":"2024","journal-title":"Nucleic Acids Res"},{"key":"2025101309272600600_ref34","doi-asserted-by":"publisher","first-page":"D609","DOI":"10.1093\/nar\/gkae1010","article-title":"UniProt: the universal protein knowledgebase in 2025","volume":"53","author":"UniProt Consortium","year":"2025","journal-title":"Nucleic Acids Res"},{"key":"2025101309272600600_ref35","doi-asserted-by":"publisher","first-page":"W541","DOI":"10.1093\/nar\/gkr469","article-title":"BioPortal: enhanced functionality via new web services from the National Center for biomedical ontology to access and use ontologies in software applications","volume":"39","author":"Whetzel","year":"2011","journal-title":"Nucleic Acids Res"},{"key":"2025101309272600600_ref36","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1038\/s41431-019-0508-0","article-title":"Estimating cumulative point prevalence of rare diseases: analysis of the Orphanet database","volume":"28","author":"Nguengang Wakap","year":"2020","journal-title":"Eur J Hum Genet"},{"key":"2025101309272600600_ref37","doi-asserted-by":"publisher","first-page":"1050","DOI":"10.59287\/icaens.1127","article-title":"Creating large language model applications utilizing LangChain: a primer on developing LLM apps fast","volume":"1","author":"Topsakal","year":"2023"},{"key":"2025101309272600600_ref38","first-page":"599","article-title":"MedAgents: large language models as collaborators for zero-shot medical reasoning","author":"Tang","year":"2024"},{"key":"2025101309272600600_ref39","article-title":"GeneTuring tests GPT models in genomics","volume":"2023","author":"Hou","year":"2023","journal-title":"bioRxiv"},{"key":"2025101309272600600_ref40","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btae075","article-title":"GeneGPT: augmenting large language models with domain tools for improved access to biomedical information","volume":"40","author":"Jin","year":"2024","journal-title":"Bioinformatics"},{"key":"2025101309272600600_ref41","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3490238","article-title":"Biomedical question answering: a survey of approaches and challenges","volume":"55","author":"Jin","year":"2022","journal-title":"ACM Comput Surveys (CSUR)"},{"key":"2025101309272600600_ref42","author":"Open AI"},{"key":"2025101309272600600_ref43","author":"Open AI"},{"key":"2025101309272600600_ref44","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1007\/979-8-8688-0447-2_2","article-title":"An introduction to Microsoft Copilot","volume-title":"Copilot for Microsoft 365: Harness the Power of Generative AI in the Microsoft Apps you Use every Day","author":"Stratton","year":"2024"},{"key":"2025101309272600600_ref45","author":"Meta AI"},{"key":"2025101309272600600_ref46","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbac409","article-title":"BioGPT: generative pre-trained transformer for biomedical text generation and mining","volume":"23","author":"Luo","year":"2022","journal-title":"Brief Bioinform"},{"key":"2025101309272600600_ref47","author":"Stanford"},{"key":"2025101309272600600_ref48","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI blog"},{"key":"2025101309272600600_ref49","doi-asserted-by":"publisher","DOI":"10.1101\/2024.01.31.578275","article-title":"Phenomics assistant: an interface for LLM-based biomedical knowledge graph exploration. bioRxiv","author":"O\u2019Neil","year":"2024"},{"key":"2025101309272600600_ref50","doi-asserted-by":"publisher","first-page":"D712","DOI":"10.1093\/nar\/gkw1128","article-title":"The monarch initiative: an integrative data and analytic platform connecting phenotypes to genotypes across species","volume":"45","author":"Mungall","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2025101309272600600_ref51","article-title":"Openbiollms: advancing open-source large language models for healthcare and life sciences","author":"Pal","year":"2024","journal-title":"Hugging Face Repository"},{"key":"2025101309272600600_ref52","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1038\/s41597-023-02068-4","article-title":"BioASQ-QA: a manually curated corpus for biomedical question answering","volume":"10","author":"Krithara","year":"2023","journal-title":"Sci Data"},{"key":"2025101309272600600_ref53","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Adv Neural Inform Process Syst"},{"key":"2025101309272600600_ref54","article-title":"Bioagent: a mobile agent system for bioscientists","volume-title":"Proceedings of the Network Tools and Applications in Biology Workshop Agents in Bioinformatics","author":"Merelli","year":"2002"},{"key":"2025101309272600600_ref55","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1093\/bib\/bbl014","article-title":"Agents in bioinformatics, computational and systems biology","volume":"8","author":"Merelli","year":"2007","journal-title":"Brief Bioinform"},{"key":"2025101309272600600_ref56","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/5\/bbaf539\/64659570\/bbaf539.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/26\/5\/bbaf539\/64659570\/bbaf539.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:36:58Z","timestamp":1760362618000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbaf539\/8284869"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,31]]},"references-count":56,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,8,31]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbaf539","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,9]]},"published":{"date-parts":[[2025,8,31]]},"article-number":"bbaf539"}}