{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,22]],"date-time":"2026-05-22T23:06:14Z","timestamp":1779491174294,"version":"3.53.1"},"reference-count":34,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2026,3,30]],"date-time":"2026-03-30T00:00:00Z","timestamp":1774828800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Intelligent agent-driven research co-pilots, leveraging advances in generative AI, are transforming how scientists access biomedical knowledge. This paper presents Med.ai ASK, an agentic question-answering system designed to address biomedical inquiries through dynamic retrieval augmentation and tool-driven reasoning. We aim to develop a system capable of parsing the nuance in biomedical scientists\u2019 research questions to provide reliable, grounded responses that are more accurate than other generative AI solutions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>We adopt the ReAct framework\u2019s tool-calling architecture and leverage atomic reasoning from Self-Discover to build Med.ai ASK. It selectively queries multiple biomedical knowledge bases and employs map-reduce tools for vector database retrieval, alongside external API and NER tool integration. We ingested 44 million biomedical documents from diverse sources. The agent is evaluated on a range of biomedical question-answering datasets.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Human evaluation on an internal dataset shows strong performance and stability. Ratings from a large language model are aligned with human assessments, supporting its use in further experiments. Automatic evaluations indicate superior performance in long-form answers regarding accuracy, faithfulness, factuality, and reduced hallucinations. For short-form and multiple-choice answers, performance is competitive with state-of-the-art systems. The agent\u2019s detailed answers are more interpretable than other systems attributed to its agentic design. The agent effectively selects tools based on question type and is deployed in a production-level chat platform with over 1600 users and 25 000 answered questions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Med.ai ASK dynamically orchestrates biomedical information retrieval tools to deliver robust interpretative, accurate, and factual answers, which is crucial in the biomedical domain.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocag038","type":"journal-article","created":{"date-parts":[[2026,3,30]],"date-time":"2026-03-30T18:18:17Z","timestamp":1774894697000},"page":"1134-1145","source":"Crossref","is-referenced-by-count":0,"title":["Med.ai ASK: an agentic system for biomedical question answering"],"prefix":"10.1093","volume":"33","author":[{"given":"Nhung T H","family":"Nguyen","sequence":"first","affiliation":[{"name":"Data Science and Digital Health, Innovative Medicine, Johnson & Johnson , Titusville, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dmytro S","family":"Lituiev","sequence":"additional","affiliation":[{"name":"Data Science and Digital Health, Innovative Medicine, Johnson & Johnson , Titusville, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhimin","family":"Liu","sequence":"additional","affiliation":[{"name":"Data Science and Digital Health, Innovative Medicine, Johnson & Johnson , Titusville, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aditya","family":"Kashyap","sequence":"additional","affiliation":[{"name":"Data Science and Digital Health, Innovative Medicine, Johnson & Johnson , Titusville, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Garrett","family":"Jenkinson","sequence":"additional","affiliation":[{"name":"Data Science and Digital Health, Innovative Medicine, Johnson & Johnson , Titusville, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kevin","family":"Kuhl","sequence":"additional","affiliation":[{"name":"Data Science and Digital Health, Innovative Medicine, Johnson & Johnson , Titusville, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"Corrado","sequence":"additional","affiliation":[{"name":"Architecture and Engineering, JJT; Johnson & Johnson , Raritan, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Naisargi Manishkumar","family":"Patel","sequence":"additional","affiliation":[{"name":"Data Science and Digital Health, Innovative Medicine, Johnson & Johnson , Titusville, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kirti","family":"Snigdha","sequence":"additional","affiliation":[{"name":"Data Science and Digital Health, Innovative Medicine, Johnson & Johnson , Titusville, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sirwe","family":"Saeedi","sequence":"additional","affiliation":[{"name":"Data Science and Digital Health, Innovative Medicine, Johnson & Johnson , Titusville, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Smith","sequence":"additional","affiliation":[{"name":"Discovery, Product Development & Supply, Innovative Medicine; Johnson & Johnson , San Diego, CA,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicholas","family":"Baro","sequence":"additional","affiliation":[{"name":"Data Science and Digital Health, Innovative Medicine, Johnson & Johnson , Titusville, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Timothy","family":"Schultz","sequence":"additional","affiliation":[{"name":"Data Science and Digital Health, Innovative Medicine, Johnson & Johnson , Titusville, NJ,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,3,30]]},"reference":[{"key":"2026052218571288000_ocag038-B1","author":"Lewis","year":"2020"},{"key":"2026052218571288000_ocag038-B2","author":"Song","year":"2024"},{"key":"2026052218571288000_ocag038-B3","author":"Guti\u00e9rrez","year":"2024"},{"key":"2026052218571288000_ocag038-B4","first-page":"2023:194","volume-title":"Experimental IR Meets Multilinguality, Multimodality, and Interaction","author":"Nentidis","year":"2023"},{"key":"2026052218571288000_ocag038-B5","author":"Ateia","year":"2024"},{"key":"2026052218571288000_ocag038-B6","author":"Merker","year":"2024"},{"key":"2026052218571288000_ocag038-B7","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-3-031-71908-0_1","volume-title":"Experimental IR Meets Multilinguality, Multimodality, and Interaction","author":"Nentidis","year":"2024"},{"key":"2026052218571288000_ocag038-B8","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1093\/jamia\/ocaf008","article-title":"Improving large language model applications in biomedicine with retrieval-augmented generation: a systematic review, meta-analysis, and clinical development guidelines","volume":"32","author":"Liu","year":"2025","journal-title":"J Am Med Inform Assoc"},{"key":"2026052218571288000_ocag038-B9","author":"Wiesinger","year":"2024"},{"key":"2026052218571288000_ocag038-B10","author":"Gridach","year":"2025"},{"key":"2026052218571288000_ocag038-B11","author":"Yao","year":"2023"},{"key":"2026052218571288000_ocag038-B12","author":"Wei","year":"2022"},{"key":"2026052218571288000_ocag038-B13","author":"Yao","year":"2023"},{"key":"2026052218571288000_ocag038-B14","author":"Gottweis","year":"2025"},{"key":"2026052218571288000_ocag038-B15","author":"Lu","year":"2024"},{"key":"2026052218571288000_ocag038-B16","author":"Baek","year":"2025"},{"key":"2026052218571288000_ocag038-B17","author":"Zhou","year":"2024"},{"key":"2026052218571288000_ocag038-B18","first-page":"1","article-title":"A survey on the memory mechanism of large language model based agents","volume":"43","author":"Zhang","year":"2025","journal-title":"ACM Trans Inf Syst"},{"key":"2026052218571288000_ocag038-B19","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1038\/s41597-023-02068-4","article-title":"BioASQ-QA: a manually curated corpus for biomedical question answering","volume":"10","author":"Krithara","year":"2023","journal-title":"Sci Data"},{"key":"2026052218571288000_ocag038-B20","author":"Laurent","year":"2024"},{"key":"2026052218571288000_ocag038-B21","doi-asserted-by":"crossref","first-page":"6421","DOI":"10.3390\/app11146421","article-title":"What disease does this patient have? A large-scale open domain question answering dataset from medical exams","volume":"11","author":"Jin","year":"2021","journal-title":"Appl. Sci"},{"key":"2026052218571288000_ocag038-B22","author":"Hou","year":"2025"},{"key":"2026052218571288000_ocag038-B23","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1038\/s41591-024-03423-7","article-title":"Toward expert-level medical question answering with large language models","volume":"31","author":"Singhal","year":"2025","journal-title":"Nat Med"},{"key":"2026052218571288000_ocag038-B24","author":"Saab","year":"2024"},{"key":"2026052218571288000_ocag038-B25","author":"Zhang","year":"2024"},{"key":"2026052218571288000_ocag038-B26","author":"Yoon","year":"2022"},{"key":"2026052218571288000_ocag038-B27","doi-asserted-by":"crossref","first-page":"D20","DOI":"10.1093\/nar\/gkae979","article-title":"Database resources of the National Center for Biotechnology Information in 2025","volume":"53","author":"Sayers","year":"2025","journal-title":"Nucleic Acids Res"},{"key":"2026052218571288000_ocag038-B28","author":"Zheng","year":"2023"},{"key":"2026052218571288000_ocag038-B29","author":"Jeong","year":"2024"},{"key":"2026052218571288000_ocag038-B30","author":"Es","year":"2024"},{"key":"2026052218571288000_ocag038-B31","author":"Grattafiori","year":"2024"},{"key":"2026052218571288000_ocag038-B32","author":"Kalai","year":"2025"},{"key":"2026052218571288000_ocag038-B33","volume-title":"MemGPT: Towards LLMs as Operating Systems","author":"Packer"},{"key":"2026052218571288000_ocag038-B34","author":"Kang","year":"2025"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/advance-article-pdf\/doi\/10.1093\/jamia\/ocag038\/67659234\/ocag038.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/33\/6\/1134\/67659234\/ocag038.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/33\/6\/1134\/67659234\/ocag038.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,22]],"date-time":"2026-05-22T22:57:18Z","timestamp":1779490638000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/33\/6\/1134\/8559661"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,30]]},"references-count":34,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2026,3,30]]},"published-print":{"date-parts":[[2026,6,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocag038","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,6]]},"published":{"date-parts":[[2026,3,30]]}}}