{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T14:56:37Z","timestamp":1791471397607,"version":"4.3.4"},"reference-count":34,"publisher":"Oxford University Press (OUP)","license":[{"start":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T00:00:00Z","timestamp":1791417600000},"content-version":"vor","delay-in-days":280,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["NRF-2023R1A2C3004176"],"award-info":[{"award-number":["NRF-2023R1A2C3004176"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008903","name":"Ministry of Health and Welfare","doi-asserted-by":"publisher","award":["HR20C002103"],"award-info":[{"award-number":["HR20C002103"]}],"id":[{"id":"10.13039\/100008903","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004332","name":"Seoul National University Hospital","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004332","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100014188","name":"Ministry of Science and ICT, South Korea","doi-asserted-by":"publisher","award":["RS-2023-00262002"],"award-info":[{"award-number":["RS-2023-00262002"]}],"id":[{"id":"10.13039\/501100014188","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100014188","name":"MSIT","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100014188","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009950","name":"MOE","doi-asserted-by":"publisher","award":["RS-2025-16652968"],"award-info":[{"award-number":["RS-2025-16652968"]}],"id":[{"id":"10.13039\/100009950","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003710","name":"KHIDI","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003710","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003665","name":"NIPA","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003665","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,1,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Robust and trustworthy biomedical question answering (QA) remains a critical challenge for large language models (LLMs), especially in complex domains such as rare diseases, where information is fragmented across multiple sources. MedHopQA 2025 benchmark introduces 10\u2009000 multi-step reasoning questions curated from Wikipedia, requiring systems to extract, connect, and synthesize biomedical knowledge across interlinked documents. In this work, we present a retrieval-augmented generation (RAG) and decision-making framework that integrates diverse retrieval strategies, including Query2Doc-based, Rationale-based, and Web-augmented retrieval, and employs a dedicated Decision Maker model to select or directly generate the most accurate and well-reasoned answers. Our system not only leverages evidence from both Wikipedia and the web but also explicitly evaluates and compares candidate answers to ensure answer reliability and reasoning transparency. Experiments on the test set demonstrate that our ensemble approach achieves state-of-the-art performance, yielding an Exact Match score of 0.84, and highlighting the importance of hybrid retrieval and robust decision-making in advancing biomedical multi-step QA. Furthermore, comprehensive analyses across retrieval stages reveal that mitigating model self-bias and ensuring a balanced integration of diverse sources are crucial for maximizing overall system performance, which ultimately reaches a score of 0.87.<\/jats:p>","DOI":"10.1093\/database\/baag061","type":"journal-article","created":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T11:18:50Z","timestamp":1789989530000},"source":"Crossref","is-referenced-by-count":0,"title":["Ensemble multi-retrieval methodologies with reasoning model decision in multi-step biomedical QA"],"prefix":"10.1093","volume":"2026","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-9690-8337","authenticated-orcid":false,"given":"Jongmyung","family":"Jung","sequence":"first","affiliation":[{"name":"Korea University Department of Computer Science and Engineering, , 145 Anam-ro, Seongbuk-gu, Seoul, 02841 ,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-8931-4596","authenticated-orcid":false,"given":"Hyeongsoon","family":"Hwang","sequence":"additional","affiliation":[{"name":"Korea University Department of Computer Science and Engineering, , 145 Anam-ro, Seongbuk-gu, Seoul, 02841 ,","place":["Republic of Korea"]},{"name":"AIGEN Sciences , 58 Wangsimni-ro, Seongdong-gu, Seoul, 04778 ,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-8582-7817","authenticated-orcid":false,"given":"Yein","family":"Park","sequence":"additional","affiliation":[{"name":"Korea University Department of Computer Science and Engineering, , 145 Anam-ro, Seongbuk-gu, Seoul, 02841 ,","place":["Republic of Korea"]},{"name":"AIGEN Sciences , 58 Wangsimni-ro, Seongdong-gu, Seoul, 04778 ,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9091-1550","authenticated-orcid":false,"given":"Minju","family":"Song","sequence":"additional","affiliation":[{"name":"Korea University Department of Computer Science and Engineering, , 145 Anam-ro, Seongbuk-gu, Seoul, 02841 ,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-7288-3347","authenticated-orcid":false,"given":"Jaehoon","family":"Yun","sequence":"additional","affiliation":[{"name":"AIGEN Sciences , 58 Wangsimni-ro, Seongdong-gu, Seoul, 04778 ,","place":["Republic of Korea"]},{"name":"College of Medicine, Hanyang University , 222 Wangsimni-ro, Seongdong-gu, Seoul, 04763 ,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-7851-3834","authenticated-orcid":false,"given":"Hyeon","family":"Hwang","sequence":"additional","affiliation":[{"name":"Korea University Department of Computer Science and Engineering, , 145 Anam-ro, Seongbuk-gu, Seoul, 02841 ,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-4741-3331","authenticated-orcid":false,"given":"Sanghoon","family":"Lee","sequence":"additional","affiliation":[{"name":"Korea University Department of Computer Science and Engineering, , 145 Anam-ro, Seongbuk-gu, Seoul, 02841 ,","place":["Republic of Korea"]},{"name":"AIGEN Sciences , 58 Wangsimni-ro, Seongdong-gu, Seoul, 04778 ,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-7962-5833","authenticated-orcid":false,"given":"Jiwoong","family":"Sohn","sequence":"additional","affiliation":[{"name":"ETH Zurich Department of Biosystems Science and Engineering, , Klingelbergstrasse 48, Basel, 4056 ,","place":["Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6798-9106","authenticated-orcid":false,"given":"Jaewoo","family":"Kang","sequence":"additional","affiliation":[{"name":"Korea University Department of Computer Science and Engineering, , 145 Anam-ro, Seongbuk-gu, Seoul, 02841 ,","place":["Republic of Korea"]},{"name":"AIGEN Sciences , 58 Wangsimni-ro, Seongdong-gu, Seoul, 04778 ,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,10,8]]},"reference":[{"key":"2026100810222217700_bib1","doi-asserted-by":"crossref","unstructured":"Jin Q, Dhingra B, Liu Z \u00a0et al. \u00a0PubMedQA: A Dataset for Biomedical Research Question Answering. Hong Kong, China: Association for Computational Linguistics, 2019.","DOI":"10.18653\/v1\/D19-1259"},{"key":"2026100810222217700_bib2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-032-04354-2","article-title":"Overview of BioASQ 2025: the Thirteenth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering","volume-title":"Experimental IR Meets Multilinguality, Multimodality, and Interaction","author":"Nentidis"},{"key":"2026100810222217700_bib3","doi-asserted-by":"crossref","unstructured":"Jin D, Pan E, Oufattole N \u00a0et al. \u00a0What disease does this patient have? A large-scale open domain question answering dataset from medical exams. Appl Sci. 2021;11:6421. 10.3390\/app11146421","DOI":"10.3390\/app11146421"},{"key":"2026100810222217700_bib4","article-title":"Assessing LLM Reasoning Steps via Principal Knowledge Grounding. Findings of the Association for Computational Linguistics","author":"Hwang","year":"2025"},{"key":"2026100810222217700_bib5","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2025.emnlp-main.837","article-title":"Med-PRM: medical reasoning models with stepwise, guideline-verified process rewards","volume-title":"Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing","author":"Yun","year":"2025"},{"key":"2026100810222217700_bib6","doi-asserted-by":"publisher","DOI":"10.1145\/3637528.3671576","article-title":"RareBench: can LLMs serve as rare diseases specialists?","volume-title":"Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","author":"Chen","year":"2024"},{"key":"2026100810222217700_bib7","doi-asserted-by":"publisher","first-page":"775","DOI":"10.1038\/s41586-025-10097-9","article-title":"An agentic system for rare disease diagnosis with traceable reasoning","volume":"651","author":"Zhao","year":"2026","journal-title":"Nature"},{"key":"2026100810222217700_bib8","article-title":"Overview of the MedHopQA track at BioCreative IX: track description, participation and evaluation of systems for multi-hop medical question answering","volume-title":"Proceedings of the BioCreative IX Challenge and Workshop (BC9): Large Language Models for Clinical and Biomedical NLP at the International Joint Conference on Artificial Intelligence (IJCAI)","author":"Islamaj","year":"2025"},{"key":"2026100810222217700_bib9","article-title":"The corpus of the MedHopQA track at BioCreative IX","volume-title":"Proceedings of the BioCreative IX Challenge and Workshop (BC9): Large Language Models for Clinical and Biomedical NLP at the International Joint Conference on Artificial Intelligence (IJCAI)","author":"Islamaj","year":"2025"},{"key":"2026100810222217700_bib10","volume-title":"Query2doc: Query Expansion with Large Language Models","author":"Wang"},{"key":"2026100810222217700_bib11","author":"Gemini"},{"key":"2026100810222217700_bib12","doi-asserted-by":"crossref","unstructured":"Yang Z, Qi P, Zhang S \u00a0et al. \u00a0HotpotQA: a dataset for diverse, explainable multi-hop question answering. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018. 10.18653\/v1\/D18-1259","DOI":"10.18653\/v1\/D18-1259"},{"key":"2026100810222217700_bib13","doi-asserted-by":"crossref","unstructured":"Ho X, Nguyen A-KD, Sugawara S \u00a0et al. \u00a0Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. Proceedings of the 28th International Conference on Computational Linguistics, 2020. 10.18653\/v1\/2020.coling-main.580","DOI":"10.18653\/v1\/2020.coling-main.580"},{"key":"2026100810222217700_bib14","doi-asserted-by":"publisher","first-page":"539","DOI":"10.1162\/tacl_a_00475","article-title":"\u266b MuSiQue: multihop Questions via Single-hop Question Composition","volume":"10","author":"Trivedi","year":"2022","journal-title":"Trans Assoc Comput Linguist"},{"key":"2026100810222217700_bib15","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","volume":"33","author":"Lewis","year":"2020","journal-title":"Adv Neural Inf Process Syst"},{"key":"2026100810222217700_bib16","doi-asserted-by":"publisher","volume-title":"Benchmarking Retrieval-Augmented Generation for Medicine","author":"Xiong","DOI":"10.18653\/v1\/2024.findings-acl.372"},{"key":"2026100810222217700_bib17","doi-asserted-by":"publisher","DOI":"10.1142\/9789819807024_0015","article-title":"Improving retrieval-augmented generation in medicine with iterative follow-up questions","volume-title":"Biocomputing 2025: Proceedings of the Pacific Symposium","author":"Xiong","year":"2024"},{"key":"2026100810222217700_bib18","doi-asserted-by":"publisher","volume-title":"Rationale-Guided Retrieval Augmented Generation for Medical Question Answering","author":"Sohn","DOI":"10.18653\/v1\/2025.naacl-long.635"},{"key":"2026100810222217700_bib19","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2023.acl-long.792","article-title":"LLM-Blender: ensembling large language models with pairwise ranking and generative fusion","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics","author":"Jiang","year":"2023"},{"key":"2026100810222217700_bib20","article-title":"Getting MoRE out of Mixture of Language Model Reasoning Experts","volume":"2023","author":"Si","journal-title":"Findings of the Association for Computational Linguistics."},{"key":"2026100810222217700_bib21","author":"Cohere"},{"key":"2026100810222217700_bib22","author":"Cohere"},{"key":"2026100810222217700_bib23","doi-asserted-by":"publisher","DOI":"10.1109\/ICBAIE63306.2024.11117089","article-title":"Optimizing Retrieval-Augmented Generation with Elasticsearch for Enhanced Question-Answering Systems","volume-title":"2024 5th International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE)","author":"Chen","year":"2024"},{"key":"2026100810222217700_bib24","author":"Olsson"},{"key":"2026100810222217700_bib25","volume-title":"Introducing GPT-4o and More Tools to ChatGPT Free Users 2024","author":"OpenAI"},{"key":"2026100810222217700_bib26","author":"OpenAI"},{"key":"2026100810222217700_bib27","author":"Chase"},{"key":"2026100810222217700_bib28","doi-asserted-by":"publisher","first-page":"btad651","DOI":"10.1093\/bioinformatics\/btad651","article-title":"MedCPT: contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval","volume":"39","author":"Jin","year":"2023","journal-title":"Bioinformatics"},{"key":"2026100810222217700_bib29","doi-asserted-by":"publisher","article-title":"Gemini 2.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities","author":"Comanici","DOI":"10.48550\/arXiv.2507.06261"},{"key":"2026100810222217700_bib30","author":"Pichai"},{"key":"2026100810222217700_bib31","author":"OpenAI"},{"key":"2026100810222217700_bib32","doi-asserted-by":"publisher","article-title":"Qwen3 embedding: advancing text embedding and reranking through foundation models","author":"Zhang","DOI":"10.48550\/arXiv.2506.05176"},{"key":"2026100810222217700_bib33","author":"Wikimedia Foundation"},{"key":"2026100810222217700_bib34","doi-asserted-by":"publisher","first-page":"68772","DOI":"10.52202\/079017-2197","article-title":"LLM Evaluators Recognize and Favor Their Own Generations","volume":"37","author":"Panickssery","year":"2024","journal-title":"Adv Neural Inf Proc Syst"}],"container-title":["Database"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/database\/article-pdf\/doi\/10.1093\/database\/baag061\/71718592\/baag061.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/database\/article-pdf\/doi\/10.1093\/database\/baag061\/71718592\/baag061.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T14:22:31Z","timestamp":1791469351000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/database\/article\/doi\/10.1093\/database\/baag061\/8885590"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":34,"URL":"https:\/\/doi.org\/10.1093\/database\/baag061","relation":{},"ISSN":["1758-0463"],"issn-type":[{"value":"1758-0463","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026]]},"published":{"date-parts":[[2026]]},"article-number":"baag061"}}