{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,9,26]],"date-time":"2026-09-26T13:59:57Z","timestamp":1790431197364,"version":"4.1.0"},"posted":{"date-parts":[[2026,9,23]]},"group-title":"Neurology","reference-count":34,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T00:00:00Z","timestamp":1790121600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"National Institutes of Health","award":["K23NS124656"],"award-info":[{"award-number":["K23NS124656"]}]},{"name":"National Institutes of Health","award":["R21NS142800"],"award-info":[{"award-number":["R21NS142800"]}]},{"name":"National Institutes of Health","award":["RF1AG064312"],"award-info":[{"award-number":["RF1AG064312"]}]},{"name":"National Institutes of Health","award":["RF1NS120947"],"award-info":[{"award-number":["RF1NS120947"]}]},{"name":"National Institutes of Health","award":["R01AG073410"],"award-info":[{"award-number":["R01AG073410"]}]},{"name":"National Institutes of Health","award":["R01HL161253"],"award-info":[{"award-number":["R01HL161253"]}]},{"name":"National Institutes of Health","award":["R01NS126282"],"award-info":[{"award-number":["R01NS126282"]}]},{"name":"National Institutes of Health","award":["R01AG073598"],"award-info":[{"award-number":["R01AG073598"]}]},{"name":"National Institutes of Health","award":["R01NS131347"],"award-info":[{"award-number":["R01NS131347"]}]},{"name":"National Institutes of Health","award":["R01NS130119"],"award-info":[{"award-number":["R01NS130119"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,9,23]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                <jats:sec>\n                  <jats:title>Background<\/jats:title>\n                  <jats:p>\n                    Clinicians increasingly use AI systems to search\n                    <jats:bold>the<\/jats:bold>\n                    medical literature, but current benchmarks do not jointly\n                    <jats:bold>test<\/jats:bold>\n                    whether a response identifies the originating paper and its supporting passage. LitBench evaluates four dimensions: stated versus interpretation-requiring facts, the number and composition\n                    <jats:bold>of<\/jats:bold>\n                    competing papers, one-versus two-paper evidence, and refusal when evidence is absent.\n                  <\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>LitBench comprises 1,980 open-access papers (1,000 epilepsy papers holding the answers, 980 decoys from unrelated fields) and 9,472 human-reviewed facts, giving 2,188 single-paper questions and 170 requiring a fact from each of two papers. Difficulty rose by burying the answer among up to 2,000 distractors, then again over live PubMed Central. The same questions were then asked with the answering paper removed, so that refusal was the correct response. Four systems were tested: Gemma-4B, Gemma-12B, Sonnet-5, and DeepSeek-V4-Flash (refusal conditions only). Three model judges scored each answer by majority; two-paper questions counted only when both facts were found.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Across fixed-corpus single-paper conditions, accuracy ranged from 55.9% to 71.9% for Gemma-4B, 61.2% to 75.5% for Gemma-12B, and 90.0% to 91.3% for Sonnet-5. Similar epilepsy papers were harder than mixed candidate sets for both Gemma configurations. Two-paper accuracy was 5.7%, 17.1%, and 37.8%, respectively. With evidence absent, Gemma-12B refusal fell from 88.3% to 39.9% as candidate sets grew; Gemma-4B almost never refused. Sonnet-5 refused on 94.9% of sampled single-paper and all sampled two-paper cells. DeepSeek-V4-Flash also refused frequently, but on 80.3% of matched answer-present controls.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>No system was reliable across every axis tested: accuracy fell as competing papers were added, interpretation-requiring facts were harder than stated ones, two-paper synthesis was rarely achieved, and only some systems recognized absent evidence. What counts as \u201csuccessful\u201d literature search carries considerable nuance; LitBench can evaluate proposed tools at a fine-grained level.<\/jats:p>\n                <\/jats:sec>","DOI":"10.64898\/2026.09.17.26363288","type":"posted-content","created":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T17:55:16Z","timestamp":1790186116000},"source":"Crossref","is-referenced-by-count":0,"title":["LitBench: A Benchmark for Retrieval-Grounded Multi-Paper Evidence Synthesis in Epilepsy"],"prefix":"10.64898","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-7542-8608","authenticated-orcid":false,"given":"Alon","family":"Gorenshtein","sequence":"first","affiliation":[{"name":"Department of Neurology, Beth Israel Deaconess Medical Center, Boston, MA, USA"},{"name":"Harvard Medical School, Boston, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tianyu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Neurology, Beth Israel Deaconess Medical Center, Boston, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4803-312X","authenticated-orcid":false,"given":"M. Brandon","family":"Westover","sequence":"additional","affiliation":[{"name":"Department of Neurology, Beth Israel Deaconess Medical Center, Boston, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8370-2758","authenticated-orcid":false,"given":"Daniel M.","family":"Goldenholz","sequence":"additional","affiliation":[{"name":"Department of Neurology, Beth Israel Deaconess Medical Center, Boston, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","reference":[{"key":"2026092606050580000_2026.09.17.26363288v1.1","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-025-03626-6"},{"key":"2026092606050580000_2026.09.17.26363288v1.2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02448-8"},{"key":"2026092606050580000_2026.09.17.26363288v1.3","doi-asserted-by":"publisher","DOI":"10.1056\/AIoa2300068"},{"key":"2026092606050580000_2026.09.17.26363288v1.4","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2024.0555"},{"key":"2026092606050580000_2026.09.17.26363288v1.5","article-title":"LAB-Bench: measuring capabilities of language models for biology research","year":"2024","journal-title":"arXiv preprint"},{"key":"2026092606050580000_2026.09.17.26363288v1.6","article-title":"Language agents achieve superhuman synthesis of scientific knowledge","year":"2024","journal-title":"arXiv preprint"},{"key":"2026092606050580000_2026.09.17.26363288v1.7","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-acl.372"},{"key":"2026092606050580000_2026.09.17.26363288v1.8","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2025.findings-acl.668"},{"key":"2026092606050580000_2026.09.17.26363288v1.9","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2024.21700"},{"key":"2026092606050580000_2026.09.17.26363288v1.10","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.557"},{"key":"2026092606050580000_2026.09.17.26363288v1.11","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-024-03425-5"},{"key":"2026092606050580000_2026.09.17.26363288v1.12","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/p18-2124"},{"key":"2026092606050580000_2026.09.17.26363288v1.13","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i16.29728"},{"key":"2026092606050580000_2026.09.17.26363288v1.14","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.503"},{"key":"2026092606050580000_2026.09.17.26363288v1.15","article-title":"Replacing judges with juries: evaluating LLM generations with a panel of diverse models","year":"2024","journal-title":"arXiv preprint"},{"key":"2026092606050580000_2026.09.17.26363288v1.16","doi-asserted-by":"publisher","DOI":"10.52202\/075280-2020"},{"key":"2026092606050580000_2026.09.17.26363288v1.17","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-026-04431-5"},{"key":"2026092606050580000_2026.09.17.26363288v1.18","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-025-04151-2"},{"key":"2026092606050580000_2026.09.17.26363288v1.19","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2025.findings-acl.967"},{"key":"2026092606050580000_2026.09.17.26363288v1.20","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.741"},{"key":"2026092606050580000_2026.09.17.26363288v1.21","doi-asserted-by":"publisher","DOI":"10.6028\/nist.sp.1329.overview.biogen-coordinators"},{"key":"2026092606050580000_2026.09.17.26363288v1.22","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.findings-emnlp.347"},{"key":"2026092606050580000_2026.09.17.26363288v1.23","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1259"},{"key":"2026092606050580000_2026.09.17.26363288v1.24","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-015-0564-6"},{"key":"2026092606050580000_2026.09.17.26363288v1.25","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01840-7"},{"key":"2026092606050580000_2026.09.17.26363288v1.26","article-title":"EpiGraph: building generalists for evidence-intensive epilepsy reasoning in the wild","journal-title":"arXiv"},{"key":"2026092606050580000_2026.09.17.26363288v1.27","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-025-10072-4"},{"key":"2026092606050580000_2026.09.17.26363288v1.28","article-title":"AstaBench: rigorous benchmarking of AI agents with a scientific research suite","year":"2026","journal-title":"International Conference on Learning Representations (ICLR 2026)"},{"key":"2026092606050580000_2026.09.17.26363288v1.29","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.naacl-long.74"},{"key":"2026092606050580000_2026.09.17.26363288v1.30","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2025.acl-long.263"},{"key":"2026092606050580000_2026.09.17.26363288v1.31","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2026.acl-long.1023"},{"key":"2026092606050580000_2026.09.17.26363288v1.32","article-title":"ResearchQA: evaluating scholarly question answering with verbatim citation accuracy","journal-title":"arXiv"},{"key":"2026092606050580000_2026.09.17.26363288v1.33","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.b2680"},{"key":"2026092606050580000_2026.09.17.26363288v1.34","doi-asserted-by":"publisher","DOI":"10.1186\/s41073-025-00173-z"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.09.17.26363288","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,26]],"date-time":"2026-09-26T13:05:16Z","timestamp":1790427916000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.64898\/2026.09.17.26363288"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,23]]},"references-count":34,"URL":"https:\/\/doi.org\/10.64898\/2026.09.17.26363288","relation":{},"subject":[],"published":{"date-parts":[[2026,9,23]]},"subtype":"preprint"}}