{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T22:25:43Z","timestamp":1792794343835,"version":"4.5.0"},"reference-count":31,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T00:00:00Z","timestamp":1778025600000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Individualised Care"},{"name":"Innovative Health Initiative Joint Undertaking","id":[{"id":"https:\/\/ror.org\/04afqts81","id-type":"ROR","asserted-by":"crossref"}]},{"name":"BreakthroughT1D","award":["101112022"],"award-info":[{"award-number":["101112022"]}]},{"name":"European Union\u2019s Horizon Europe research and innovation program"},{"name":"COCIR","id":[{"id":"https:\/\/ror.org\/0315a7e71","id-type":"ROR","asserted-by":"crossref"}]},{"DOI":"10.13039\/100013322","name":"EFPIA","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100013322","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Vaccines Europe"},{"name":"EuropaBio","id":[{"id":"https:\/\/ror.org\/029pbd337","id-type":"ROR","asserted-by":"crossref"}]},{"name":"MedTech Europe"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,5,18]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Aims<\/jats:title>\n                    <jats:p>Artificial intelligence (AI) tools utilizing large language models (LLMs) can accelerate scientific literature reviews by automating title, abstract, and full-text-based screenings of relevant patient populations and biomarkers. We developed an AI-based tool to automate and improve full-text screening performance using LLMs to accurately identify relevant publications that meet complex criteria.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods and results<\/jats:title>\n                    <jats:p>We conducted a literature review utilizing the Population, Intervention\u2014biomarkers, Comparison, Outcome framework to define our inclusion and exclusion criteria, focusing on biomarkers in heart failure with reduced ejection fraction (HFrEF). An AI-based full-text screening tool was created to process 5405 selected publications, combining multi-level and task-oriented retrieval-augmented generation (RAG) and agent-based methods, establishing ground truth standards to evaluate performance metrics both for the tool and human reviewers. Intra-LLM reliability was assessed by rerunning screenings on a batch of publications. Among the public and private domain models, LLaMA 3.3 70B was selected for its superior accuracy (82%), precision (71%), and recall (100%) in screening 49 manuscripts by LLMs. During the training phase, based on several hundred manuscripts, performance metrics significantly improved. Validation results showed a sensitivity of 91.4%, specificity of 53.2%, a false positive rate of 46.8%, and a false negative rate of 8.6%. The LLM outperformed human reviewers in F1 score and interrater reliability, achieving 100% consistency across multiple runs, with each run consisting of multiple LLMs on 1000 documents.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Our study demonstrated that AI tool can reduce labour-intensive efforts while maintaining accuracy in literature reviews, with greater inter-rater agreement compared to human reviewers.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/ehjdh\/ztag070","type":"journal-article","created":{"date-parts":[[2026,5,5]],"date-time":"2026-05-05T11:33:00Z","timestamp":1777980780000},"source":"Crossref","is-referenced-by-count":1,"title":["Automated full-text screening and accelerated reviews using large language models with context-aware agents: an exploratory analysis in biomarker research"],"prefix":"10.1093","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4982-0814","authenticated-orcid":false,"given":"Yongxin","family":"Ye","sequence":"first","affiliation":[{"name":"NOVEL Translational Medicine, Novo Nordisk A\/S , Vandt\u00e5rnsvej 108, S\u00f8borg 2860 ,","place":["Denmark"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martina","family":"Colombo","sequence":"additional","affiliation":[{"name":"Department of Acute Brain and Cardiovascular Injury, Istituto di Ricerche Farmacologiche Mario Negri IRCCS , Milan ,","place":["Italy"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1744-6899","authenticated-orcid":false,"given":"Jennifer","family":"Meessen","sequence":"additional","affiliation":[{"name":"Department of Acute Brain and Cardiovascular Injury, Istituto di Ricerche Farmacologiche Mario Negri IRCCS , Milan ,","place":["Italy"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6404-1018","authenticated-orcid":false,"given":"Negin","family":"Hajizadeh","sequence":"additional","affiliation":[{"name":"Bayer Espana AG , Madrid ,","place":["Spain"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lauren","family":"De Vore","sequence":"additional","affiliation":[{"name":"Evotec International, GmbH , Hamburg ,","place":["Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1658-3733","authenticated-orcid":false,"given":"Hajar Ghaem","family":"Sigarchian","sequence":"additional","affiliation":[{"name":"Mobilab & Care, Centre of Expertise Care and Well-Being, Thomas More University of Applied Sciences , Geel ,","place":["Belgium"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-0263-1470","authenticated-orcid":false,"given":"Christof","family":"Peters","sequence":"additional","affiliation":[{"name":"Department of Cardiology, Cardiovascular Research Institute Maastricht (CARIM), Maastricht University Medical Center , Maastricht ,","place":["the Netherlands"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Supraja","family":"Krovvidi","sequence":"additional","affiliation":[{"name":"CapsicoHealth , Dublin Ireland and Palo Alto, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shloke","family":"Meresh","sequence":"additional","affiliation":[{"name":"CapsicoHealth , Dublin Ireland and Palo Alto, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laurent","family":"Hasson","sequence":"additional","affiliation":[{"name":"CapsicoHealth , Dublin Ireland and Palo Alto, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"SaiRamesh","family":"Jakka","sequence":"additional","affiliation":[{"name":"CapsicoHealth , Dublin Ireland and Palo Alto, CA ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5297-8412","authenticated-orcid":false,"given":"Benoit","family":"Tyl","sequence":"additional","affiliation":[{"name":"Bayer HealthCare SAS , La Garenne Colombes ,","place":["France"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"name":"iCARE4CVD consortium","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Arantxa","family":"Barandiaran","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bjarke","family":"Folin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Deborah","family":"Novelli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dirk","family":"Mueller-Wieland","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ece","family":"Buldurluoglu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Katarina","family":"Pardali","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maria Chiara","family":"Bassi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michela","family":"Cinquini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carolina","family":"Lavazza","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Noemi","family":"Pavo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul","family":"Balfanz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roberto","family":"Latini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sara","family":"Klas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Silvia Maria","family":"Balosso","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,5,5]]},"reference":[{"key":"2026052116174281700_ztag070-B1","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1038\/s41746-023-00988-4","article-title":"Generative AI and large language models in health care: pathways to implementation","volume":"7","author":"Raza","year":"2024","journal-title":"NPJ Digit Med"},{"key":"2026052116174281700_ztag070-B2","doi-asserted-by":"crossref","first-page":"104480","DOI":"10.1016\/j.jbi.2023.104480","article-title":"Mining electronic health records using artificial intelligence: bibliometric and content analyses for current research status and product conversion","volume":"146","author":"Liang","year":"2023","journal-title":"J Biomed Inform"},{"key":"2026052116174281700_ztag070-B3","doi-asserted-by":"crossref","first-page":"111746","DOI":"10.1016\/j.jclinepi.2025.111746","article-title":"Large language models for conducting systematic reviews: on the rise, but not yet ready for use-a scoping review","volume":"181","author":"Lieberum","year":"2025","journal-title":"J Clin Epidemiol"},{"key":"2026052116174281700_ztag070-B4","doi-asserted-by":"crossref","first-page":"e52399","DOI":"10.2196\/52399","article-title":"Potential of large language models in health care: delphi study","volume":"26","author":"Denecke","year":"2024","journal-title":"J Med Internet Res"},{"key":"2026052116174281700_ztag070-B5","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1186\/s13643-024-02682-2","article-title":"Leveraging artificial intelligence to enhance systematic reviews in health research: advanced tools and challenges","volume":"13","author":"Ge","year":"2024","journal-title":"Syst Rev"},{"key":"2026052116174281700_ztag070-B6","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1186\/s12874-025-02583-5","article-title":"Streamlining systematic reviews with large language models using prompt engineering and retrieval augmented generation","volume":"25","author":"Trad","year":"2025","journal-title":"BMC Med Res Methodol"},{"key":"2026052116174281700_ztag070-B7","volume-title":"BEST (Biomarkers, EndpointS, and Other Tools) Resource","author":"Group F-NBW","year":"2016"},{"key":"2026052116174281700_ztag070-B8","doi-asserted-by":"crossref","first-page":"548","DOI":"10.2105\/AJPH.2021.306658","article-title":"Value and challenges of using observational studies in systematic reviews of public health interventions","volume":"112","author":"Hilton Boon","year":"2022","journal-title":"Am J Public Health"},{"key":"2026052116174281700_ztag070-B9","doi-asserted-by":"crossref","first-page":"1567","DOI":"10.3758\/BF03193491","article-title":"The effects of prior knowledge and text structure on comprehension processes during Reading of scientific texts","volume":"35","author":"Kendeou","year":"2007","journal-title":"Mem Cognit"},{"key":"2026052116174281700_ztag070-B10","doi-asserted-by":"crossref","first-page":"e076912","DOI":"10.1136\/bmjopen-2023-076912","article-title":"Inter-reviewer reliability of human literature reviewing and implications for the introduction of machine-assisted systematic reviews: a mixed-methods review","volume":"14","author":"Hanegraaf","year":"2024","journal-title":"BMJ Open"},{"key":"2026052116174281700_ztag070-B11","doi-asserted-by":"crossref","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","article-title":"Adapted large language models can outperform medical experts in clinical text summarization","volume":"30","author":"Van Veen","year":"2024","journal-title":"Nat Med"},{"key":"2026052116174281700_ztag070-B12","doi-asserted-by":"crossref","first-page":"e2411962122","DOI":"10.1073\/pnas.2411962122","article-title":"Transforming literature screening: the emerging role of large language models in systematic reviews","volume":"122","author":"Delgado-Chaves","year":"2025","journal-title":"Proc Natl Acad Sci U S A"},{"key":"2026052116174281700_ztag070-B13","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1186\/s13643-024-02575-4","article-title":"Title and abstract screening for literature reviews using large language models: an exploratory study in the biomedical domain","volume":"13","author":"Dennst\u00e4dt","year":"2024","journal-title":"Syst Rev"},{"key":"2026052116174281700_ztag070-B14","doi-asserted-by":"crossref","first-page":"e2425953","DOI":"10.1001\/jamanetworkopen.2024.25953","article-title":"Accuracy, consistency, and hallucination of large language models when analyzing unstructured clinical notes in electronic medical records","volume":"7","author":"Shah","year":"2024","journal-title":"JAMA Netw Open"},{"key":"2026052116174281700_ztag070-B15","doi-asserted-by":"crossref","first-page":"2139","DOI":"10.1093\/eurheartj\/ehaf128","article-title":"iCARE4CVD: an innovative European consortium aiming to improve personalized cardiovascular care","volume":"46","author":"Brunner-La Rocca","year":"2025","journal-title":"Eur Heart J"},{"key":"2026052116174281700_ztag070-B16","doi-asserted-by":"crossref","DOI":"10.1002\/9781119536604","volume-title":"Cochrane Handbook for Systematic Reviews of Interventions","author":"Higgins","year":"2019","edition":"2nd ed"},{"key":"2026052116174281700_ztag070-B17","author":"Lewis"},{"key":"2026052116174281700_ztag070-B18","doi-asserted-by":"publisher","author":"Gao","DOI":"10.48550\/arXiv.2312.10997"},{"key":"2026052116174281700_ztag070-B19","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1016\/j.ins.2015.02.024","article-title":"Learning similarity with cosine similarity ensemble","volume":"307","author":"Xia","year":"2015","journal-title":"Inf Sci"},{"key":"2026052116174281700_ztag070-B20","first-page":"1","article-title":"Large language models empowered agent-based modeling and simulation: a survey and perspectives","volume":"11.1","author":"Gao","year":"2024","journal-title":"Humanit Soc Sci Commun"},{"key":"2026052116174281700_ztag070-B21","author":"Krovvidi"},{"key":"2026052116174281700_ztag070-B22","author":"Sairamesh"},{"key":"2026052116174281700_ztag070-B23","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: the kappa statistic","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochem Med (Zagreb)"},{"key":"2026052116174281700_ztag070-B24","doi-asserted-by":"crossref","first-page":"616","DOI":"10.1093\/jamia\/ocaf030","article-title":"Enhancing systematic literature reviews with generative artificial intelligence: development, applications, and performance evaluation","volume":"32","author":"Li","year":"2025","journal-title":"J Am Med Inform Assoc"},{"key":"2026052116174281700_ztag070-B25","author":"Dai"},{"key":"2026052116174281700_ztag070-B26","doi-asserted-by":"crossref","first-page":"962","DOI":"10.1136\/bjo-2024-326254","article-title":"Can large language models fully automate or partially assist paper selection in systematic reviews?","volume":"109","author":"Chen","year":"2025","journal-title":"Br J Ophthalmol"},{"key":"2026052116174281700_ztag070-B27","doi-asserted-by":"crossref","first-page":"1071","DOI":"10.1093\/jamia\/ocaf063","article-title":"The emergence of large language models as tools in literature reviews: a large language model-assisted systematic review","volume":"32","author":"Scherbakov","year":"2025","journal-title":"J Am Med Inform Assoc"},{"key":"2026052116174281700_ztag070-B28","doi-asserted-by":"crossref","first-page":"616","DOI":"10.1002\/jrsm.1715","article-title":"Can large language models replace humans in systematic reviews? Evaluating GPT-4\u2019s efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages","volume":"15","author":"Khraisha","year":"2024","journal-title":"Res Synth Methods"},{"key":"2026052116174281700_ztag070-B29","doi-asserted-by":"crossref","first-page":"e48996","DOI":"10.2196\/48996","article-title":"Automated paper screening for clinical reviews using large language models: data analysis study","volume":"26","author":"Guo","year":"2024","journal-title":"J Med Internet Res"},{"key":"2026052116174281700_ztag070-B30","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.1038\/s41591-026-04229-5","article-title":"LLM-assisted systematic review of large language models in clinical medicine","volume":"32","author":"Chen","year":"2026","journal-title":"Nat Med"},{"key":"2026052116174281700_ztag070-B31","doi-asserted-by":"crossref","first-page":"ztaf127","DOI":"10.1093\/ehjdh\/ztaf127","article-title":"Large language models for structured cardiovascular data extraction: a foundation for scalable research and clinical applications","volume":"7","author":"van der Loo","year":"2025","journal-title":"Eur Heart J Digit Health"}],"container-title":["European Heart Journal - Digital Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/ehjdh\/advance-article-pdf\/doi\/10.1093\/ehjdh\/ztag070\/68223865\/ztag070.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ehjdh\/article-pdf\/7\/5\/ztag070\/68223865\/ztag070.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ehjdh\/article-pdf\/7\/5\/ztag070\/68223865\/ztag070.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T20:17:50Z","timestamp":1779394670000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/ehjdh\/article\/doi\/10.1093\/ehjdh\/ztag070\/8670091"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,5]]},"references-count":31,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,5,18]]}},"URL":"https:\/\/doi.org\/10.1093\/ehjdh\/ztag070","relation":{},"ISSN":["2634-3916"],"issn-type":[{"value":"2634-3916","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,6]]},"published":{"date-parts":[[2026,5,5]]},"article-number":"ztag070"}}