{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T09:53:15Z","timestamp":1785405195257,"version":"3.56.0"},"posted":{"date-parts":[[2026,6,15]]},"group-title":"Health Informatics","reference-count":50,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T00:00:00Z","timestamp":1781481600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"award":["R00 LM013367"],"award-info":[{"award-number":["R00 LM013367"]}],"id":[{"id":"https:\/\/ror.org\/0060t0j89","id-type":"ROR","asserted-by":"publisher"}]},{"award":["R01MH129764"],"award-info":[{"award-number":["R01MH129764"]}],"id":[{"id":"https:\/\/ror.org\/04xeg9z08","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,6,24]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                <jats:sec>\n                  <jats:title>Background<\/jats:title>\n                  <jats:p>Systematic reviews of observational studies are central to causal inference in chronic disease epidemiology but are increasingly limited by the scale of the literature and heterogeneity in confounder control. There is a need for transparent, open methods that reduce screening burden and make reported exposures, outcomes, and covariates comparable across studies.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>To develop and evaluate modular LLM-based pipelines, LitScreen and VarEx, that automate study screening and variable extraction for observational systematic reviews across multiple use cases, including hypertension as a primary exposure with Alzheimer\u2019s disease and related dementias (ADRD) as outcomes, and posttraumatic stress disorder (PTSD) as the exposure with self-harm, self-injury, and suicidality as outcomes.<\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Methods and Materials<\/jats:title>\n                  <jats:p>\n                    We built an end-to-end workflow in which reproducible MEDLINE via Ovid queries yield RIS corpora that are processed by LitScreen, a three-phase screening pipeline combining abstract-level evidence extraction, criterion-wise inclusion adjudication with high-recall gates, and full-text retrieval-augmented verification. Screened-in articles enter VarEx, a retrieval-augmented extraction pipeline that identifies role-specific passages and performs evidence-grounded extraction and semantic classification of exposures, outcomes, and covariates into predefined categories aligned with Metaconfoundr. Performance was evaluated on six labeled SYNERGY datasets and expert-annotated hypertension-to-ADRD and education-to-dementia corpora using precision, recall, F\n                    <jats:sub>1<\/jats:sub>\n                    , a strict score requiring correct variable identity and category, and time-per-reference estimates.\n                  <\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>\n                    In the primary hypertension-to-ADRD reference set, VarEx achieved covariate-level precision of 0.80, recall of 0.79, and F\n                    <jats:sub>1<\/jats:sub>\n                    of 0.76, with classification accuracy of 0.97 and similar performance for education-to-dementia and SYNERGY validation datasets. LitScreen preserved high recall while excluding most ineligible records and reduced total screening and extraction time by roughly 80\u201390 percent relative to manual review baselines by routing only uncertain or borderline citations to full-text verification.\n                  <\/jats:p>\n                <\/jats:sec>\n                <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>A retrieval-augmented LLM framework can automate major components of screening and variable extraction for observational systematic reviews, generating reusable structured covariate inventories that integrate with causal confounder assessment tools and substantially improve the efficiency and reproducibility of evidence synthesis, while remaining an assistant to, rather than a replacement for, human reviewers.<\/jats:p>\n                <\/jats:sec>","DOI":"10.64898\/2026.06.13.26355589","type":"posted-content","created":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T00:50:25Z","timestamp":1781571025000},"source":"Crossref","is-referenced-by-count":0,"title":["A Pilot Project Leveraging Large Language Models for Automated Screening and Variable Extraction in Observational Studies"],"prefix":"10.64898","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-7576-4833","authenticated-orcid":false,"given":"Manjil M.","family":"Pradhan","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of New Mexico, Albuquerque, NM, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-3045-1089","authenticated-orcid":false,"given":"Rajesh","family":"Upadhayaya","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of New Mexico, Albuquerque, NM, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0468-2523","authenticated-orcid":false,"given":"Sarah C.","family":"Wenyon","sequence":"additional","affiliation":[{"name":"Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexandria","family":"Viszolay","sequence":"additional","affiliation":[{"name":"School of Medicine, University of New Mexico Health Sciences Center, Albuquerque, NM, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5322-9368","authenticated-orcid":false,"given":"Melissa","family":"Rethlefsen","sequence":"additional","affiliation":[{"name":"Health Sciences Library and Informatics Center, University of New Mexico Health Sciences Center, Albuquerque, NM, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8041-0370","authenticated-orcid":false,"given":"Vincent","family":"Metzger","sequence":"additional","affiliation":[{"name":"Division of Translational Informatics, University of New Mexico Health Sciences Center, Albuquerque, NM, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7985-0347","authenticated-orcid":false,"given":"Gerardo","family":"Villarreal","sequence":"additional","affiliation":[{"name":"Department of Psychiatry and Behavioral Sciences, University of New Mexico School of Medicine, Albuquerque, NM, USA"},{"name":"Veterans Health Administration, Albuquerque, NM, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Santiago","family":"Alvarez Lesmes","sequence":"additional","affiliation":[{"name":"Northwell Health, Lake Success, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Andrade","sequence":"additional","affiliation":[{"name":"Department of Psychiatry and Behavioral Sciences, University of New Mexico School of Medicine, Albuquerque, NM, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-9681-5157","authenticated-orcid":false,"given":"Jason","family":"Timm","sequence":"additional","affiliation":[{"name":"Division of Translational Informatics, University of New Mexico Health Sciences Center, Albuquerque, NM, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1696-1781","authenticated-orcid":false,"given":"Scott A.","family":"Malec","sequence":"additional","affiliation":[{"name":"Division of Translational Informatics, University of New Mexico Health Sciences Center, Albuquerque, NM, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","reference":[{"issue":"1","key":"2026062602551821000_2026.06.13.26355589v2.1","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1038\/s41370-020-0228-0","article-title":"Challenges and recommendations on the conduct of systematic reviews of observational epidemiologic studies in environmental and occupational health","volume":"31","year":"2021","journal-title":"Journal of Exposure Science & Environmental Epidemiology"},{"key":"2026062602551821000_2026.06.13.26355589v2.2","doi-asserted-by":"crossref","unstructured":"Iain J. Marshall and Byron C. Wallace . Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Systematic Reviews, 8(1):163, 2019.","DOI":"10.1186\/s13643-019-1074-9"},{"key":"2026062602551821000_2026.06.13.26355589v2.3","doi-asserted-by":"crossref","unstructured":"Annette M. O\u2019Connor , Guy Tsafnat , James Thomas , Paul Glasziou , Stephen B. Gilbert , and Brian Hutton . A question of trust: can we build an evidence base to gain trust in systematic review automation technologies? Systematic Reviews, 8(1):143, 2019.","DOI":"10.1186\/s13643-019-1062-0"},{"key":"2026062602551821000_2026.06.13.26355589v2.4","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2016.09.001"},{"key":"2026062602551821000_2026.06.13.26355589v2.5","doi-asserted-by":"crossref","unstructured":"I. J. Marshall and B. C. Wallace . Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Systematic Reviews, 8(1):163, 2019.","DOI":"10.1186\/s13643-019-1074-9"},{"key":"2026062602551821000_2026.06.13.26355589v2.6","doi-asserted-by":"publisher","DOI":"10.1007\/s10654-019-00494-6"},{"key":"2026062602551821000_2026.06.13.26355589v2.7","doi-asserted-by":"publisher","DOI":"10.1097\/EDE.0b013e3181a819a1"},{"key":"2026062602551821000_2026.06.13.26355589v2.8","unstructured":"Miguel A. Hern\u00e1n and James M. Robins . Causal Inference: What If. Chapman & Hall\/CRC, Boca Raton, 2020. Available at https:\/\/www.hsph.harvard.edu\/miguel-hernan\/causal-inference-book\/."},{"issue":"5","key":"2026062602551821000_2026.06.13.26355589v2.9","first-page":"307","article-title":"Evaluation of confounding in epidemiologic studies assessing the association between maternal medication use and congenital anomalies","volume":"112","year":"2020","journal-title":"Birth Defects Research"},{"key":"2026062602551821000_2026.06.13.26355589v2.10","first-page":"165","article-title":"Body mass index in midlife and dementia: Systematic review and meta-regression analysis of 589,649 men and women followed in longitudinal studies","volume":"8","year":"2017","journal-title":"Alzheimer\u2019s & Dementia (Amsterdam"},{"key":"2026062602551821000_2026.06.13.26355589v2.11","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1002\/jrsm.1544","article-title":"The confounder matrix: A tool to assess confounding bias in systematic reviews of observational etiology studies","volume":"13","year":"2022","journal-title":"Research Synthesis Methods"},{"key":"2026062602551821000_2026.06.13.26355589v2.12","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2004.10.012"},{"key":"2026062602551821000_2026.06.13.26355589v2.13","unstructured":"Miguel A. Hern\u00e1n and James M. Robins. Causal Inference: What If. Chapman & Hall\/CRC, Boca Raton, FL, 2020."},{"key":"2026062602551821000_2026.06.13.26355589v2.14","doi-asserted-by":"crossref","unstructured":"Allison Gates , Cydney Johnson , and Lisa Hartling . Technology-assisted title and abstract screening for systematic reviews: a retrospective evaluation of the abstrackr machine learning tool. Systematic Reviews, 7(1):45, 2018.","DOI":"10.1186\/s13643-018-0707-8"},{"key":"2026062602551821000_2026.06.13.26355589v2.15","doi-asserted-by":"crossref","unstructured":"Alison O\u2019Mara-Eves , James Thomas , John McNaught , Makoto Miwa , and Sophia Anani-adou . Using text mining for study identification in systematic reviews: a systematic review of current approaches. Systematic Reviews, 4(1):5, 2015.","DOI":"10.1186\/2046-4053-4-5"},{"key":"2026062602551821000_2026.06.13.26355589v2.16","unstructured":"Aaron Grattafiori and AI @ Meta) others (Llama Team. The llama 3 herd of models, 2024."},{"key":"2026062602551821000_2026.06.13.26355589v2.17","unstructured":"Scott A. Malec et al. Literature-based discovery of confounding in observational clinical data. AMIA Annual Symposium Proceedings, 2016:1920\u20131929, 2016."},{"key":"2026062602551821000_2026.06.13.26355589v2.18","doi-asserted-by":"crossref","unstructured":"Scott A. Malec , Peng Wei , Elmer V. Bernstam , Richard D. Boyce , and Trevor Cohen . Using computable knowledge mined from the literature to elucidate confounders for EHR-based pharmacovigilance. Journal of Biomedical Informatics, 117:103719, 2021.","DOI":"10.1016\/j.jbi.2021.103719"},{"key":"2026062602551821000_2026.06.13.26355589v2.19","doi-asserted-by":"crossref","unstructured":"Scott A. Malec et al. Causal feature selection using a knowledge graph combining structured knowledge from the biomedical literature and ontologies: A use case studying depression as a risk factor for Alzheimer\u2019s disease. Journal of Biomedical Informatics, 142:104368, 2023.","DOI":"10.1016\/j.jbi.2023.104368"},{"key":"2026062602551821000_2026.06.13.26355589v2.20","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.6785307"},{"issue":"1","key":"2026062602551821000_2026.06.13.26355589v2.21","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1038\/s41597-024-03171-w","article-title":"An open source knowledge graph ecosystem for the life sciences","volume":"11","year":"2024","journal-title":"Scientific Data"},{"key":"2026062602551821000_2026.06.13.26355589v2.22","doi-asserted-by":"publisher","DOI":"10.7326\/M18-2159"},{"key":"2026062602551821000_2026.06.13.26355589v2.23","doi-asserted-by":"publisher","DOI":"10.1093\/ije\/dyz261"},{"key":"2026062602551821000_2026.06.13.26355589v2.24","doi-asserted-by":"publisher","DOI":"10.2196\/65984"},{"key":"2026062602551821000_2026.06.13.26355589v2.25","unstructured":"others. Developing foundations for biomedical knowledgebases from literature using large language models: A systematic assessment. PLOS Computational Biology, 2025. PMCID: PMC12329539."},{"issue":"12","key":"2026062602551821000_2026.06.13.26355589v2.26","first-page":"1","article-title":"Survey of hallucination in natural language generation","volume":"55","year":"2023","journal-title":"ACM Computing Surveys"},{"key":"2026062602551821000_2026.06.13.26355589v2.27","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","volume":"33","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026062602551821000_2026.06.13.26355589v2.28","unstructured":"Jonathan De Bruin , Yongchao Ma , Gerbrich Ferdinands , Jelle Teijema , and Rens Van de Schoot . SYNERGY - Open machine learning dataset on study selection in systematic reviews, 2023."},{"issue":"11","key":"2026062602551821000_2026.06.13.26355589v2.29","doi-asserted-by":"crossref","first-page":"1493","DOI":"10.1016\/j.jalz.2018.01.007","article-title":"Coronary heart disease, heart failure, and the risk of dementia: A systematic review and meta-analysis","volume":"14","year":"2018","journal-title":"Alzheimer\u2019s & Dementia"},{"issue":"11","key":"2026062602551821000_2026.06.13.26355589v2.30","doi-asserted-by":"crossref","first-page":"1482","DOI":"10.1016\/j.jalz.2018.04.007","article-title":"Cerebral small vessel disease and the risk of dementia: A systematic review and meta-analysis of population-based evidence","volume":"14","year":"2018","journal-title":"Alzheimer\u2019s & Dementia"},{"key":"2026062602551821000_2026.06.13.26355589v2.31","doi-asserted-by":"crossref","unstructured":"Marlies E. Brouwer , Alishia D. Williams , Mitzy Kennis , Zhongfang Fu , Nicola S. Klein , Pim Cuijpers , and Claudi L. H. Bockting . Psychological theories of depressive relapse and recurrence: A systematic review and meta-analysis of prospective studies. Clinical Psychology Review, 74:101773, 2019.","DOI":"10.1016\/j.cpr.2019.101773"},{"issue":"2","key":"2026062602551821000_2026.06.13.26355589v2.32","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1080\/00273171.2017.1412293","article-title":"Bayesian ptsd-trajectory analysis with informed priors based on a systematic literature search and expert elicitation","volume":"53","year":"2018","journal-title":"Multivariate Behavioral Research"},{"key":"2026062602551821000_2026.06.13.26355589v2.33","doi-asserted-by":"crossref","unstructured":"Eline S. van der Valk , Ozair Abawi , Mostafa Mohseni , Amir Abdelmoumen , Vincent L. Wester , Bibian van der Voorn , Anand M. Iyer , Erica L. T. van den Akker , Sanne E. Hoeks , Sjoerd A. A. van den Berg , Yolanda B. de Rijke , Tobias Stalder , and Elisabeth F. C. van Rossum . Cross-sectional relation of long-term glucocorticoids in hair with anthropometric measurements and their possible determinants: A systematic review and meta-analysis. Obesity Reviews, 23(3), 2022.","DOI":"10.1111\/obr.13376"},{"key":"2026062602551821000_2026.06.13.26355589v2.34","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.39335.541782.AD"},{"key":"2026062602551821000_2026.06.13.26355589v2.35","unstructured":"NeuML. PubMedBERT embeddings (pubmedbert-base-embeddings). https:\/\/huggingface.co\/NeuML\/pubmedbert-base-embeddings, 2023. Hugging Face model."},{"issue":"1","key":"2026062602551821000_2026.06.13.26355589v2.36","first-page":"1","article-title":"Domain-specific language model pretraining for biomedical natural language processing","volume":"3","year":"2021","journal-title":"ACM Transactions on Computing for Healthcare"},{"key":"2026062602551821000_2026.06.13.26355589v2.37","doi-asserted-by":"crossref","unstructured":"Shitao Xiao , Zheng Liu , Peitian Zhang , Niklas Muennighoff , Defu Lian , and Jian-Yun Nie . C-pack: Packed resources for general chinese embeddings, 2024.","DOI":"10.1145\/3626772.3657878"},{"key":"2026062602551821000_2026.06.13.26355589v2.38","article-title":"Decomposed prompting: A modular approach for solving complex tasks","year":"2023","journal-title":"In International Conference on Learning Representations (ICLR"},{"key":"2026062602551821000_2026.06.13.26355589v2.39","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026062602551821000_2026.06.13.26355589v2.40","article-title":"Gpt-4o system card","year":"2024","journal-title":"arXiv preprint arXiv"},{"key":"2026062602551821000_2026.06.13.26355589v2.41","doi-asserted-by":"crossref","unstructured":"Michihiro Yasunaga , Jure Leskovec , and Percy Liang . Linkbert: Pretraining language models with document links. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022.","DOI":"10.18653\/v1\/2022.acl-long.551"},{"key":"2026062602551821000_2026.06.13.26355589v2.42","article-title":"DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning","journal-title":"arXiv preprint arXiv"},{"key":"2026062602551821000_2026.06.13.26355589v2.43","doi-asserted-by":"publisher","DOI":"10.1002\/nav.3800020109"},{"key":"2026062602551821000_2026.06.13.26355589v2.44","unstructured":"Matthew J Page , Joanne E McKenzie , Patrick M Bossuyt , Isabelle Boutron , Tammy C Hoffmann , Cynthia D Mulrow , Larissa Shamseer , Jennifer M Tetzlaff , Elie A Akl , Sue E Brennan , Roger Chou , Julie Glanville , Jeremy M Grimshaw , Asbj\u00f8rn Hr\u00f3bjartsson , Manoj M Lalu , Tianjing Li , Elizabeth W Loder , Evan Mayo-Wilson , Steve McDonald , Luke A McGuinness , Lesley A Stewart , James Thomas , Andrea C Tricco , Vivian A Welch , Penny Whiting , and David Moher . The prisma 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, 2021."},{"key":"2026062602551821000_2026.06.13.26355589v2.45","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2014.06.011"},{"key":"2026062602551821000_2026.06.13.26355589v2.46","unstructured":"Zhen Wang , Asi Noor , Tarig Elraiyah, and Mohammad Hassan Murad. Dual monitors to increase efficiency of conducting systematic reviews. In Abstracts of the 21st Cochrane Colloquium, Quebec City, Canada, 2013. Conference abstract."},{"key":"2026062602551821000_2026.06.13.26355589v2.47","article-title":"Hallucination is inevitable: An innate limitation of large language models","year":"2024","journal-title":"arXiv preprint arXiv"},{"key":"2026062602551821000_2026.06.13.26355589v2.48","doi-asserted-by":"crossref","unstructured":"Aman Madaan , Niket Tandon , Prakhar Gupta , Skyler Hallinan , Luyu Gao , Sarah Wiegr-effe , Uri Alon , Nouha Dziri , Shrimai Prabhumoye , Yiming Yang , Shashank Gupta , Bod-hisattwa Prasad Majumder , Katherine Hermann , Sean Welleck , Amir Yazdanbakhsh , and Peter Clark . Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems (NeurIPS ), 2023.","DOI":"10.52202\/075280-2019"},{"key":"2026062602551821000_2026.06.13.26355589v2.49","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","year":"2020","journal-title":"Advances in Neural Information Processing Systems (NeurIPS)"},{"key":"2026062602551821000_2026.06.13.26355589v2.50","doi-asserted-by":"crossref","unstructured":"Swaroop Mishra , Daniel Khashabi , Chitta Baral , Yejin Choi , and Hannaneh Hajishirzi . Reframing instructional prompts to GPTk\u2019s language. In Findings of the Association for Computational Linguistics: ACL 2022, pages 589\u2013612, 2022.","DOI":"10.18653\/v1\/2022.findings-acl.50"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.06.13.26355589","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T09:55:58Z","timestamp":1782467758000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.64898\/2026.06.13.26355589"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,15]]},"references-count":50,"URL":"https:\/\/doi.org\/10.64898\/2026.06.13.26355589","relation":{},"subject":[],"published":{"date-parts":[[2026,6,15]]},"subtype":"preprint"}}