{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,18]],"date-time":"2026-10-18T13:03:19Z","timestamp":1792328599077,"version":"4.4.0"},"reference-count":49,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T00:00:00Z","timestamp":1783555200000},"content-version":"vor","delay-in-days":2,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Korea Health Technology R&D Project through the Korea Health Industry Development Institute"},{"name":"Ministry of Health & Welfare, Republic of Korea","award":["RS-2025-25459621"],"award-info":[{"award-number":["RS-2025-25459621"]}]},{"name":"NAVER Digital Bio Innovation Research Fund"},{"DOI":"10.13039\/100022745","name":"NAVER Corporation","doi-asserted-by":"publisher","award":["3720230020"],"award-info":[{"award-number":["3720230020"]}],"id":[{"id":"10.13039\/100022745","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,7,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Implementing target trial emulation (TTE) studies as standardized, reproducible analytic workflows is technically demanding. We developed Text-guided Health-study Estimation and Specification Engine Using Strategus (THESEUS), which uses large language models (LLMs) to translate free-text study descriptions into structured analytic specifications and Strategus R scripts within the Observational Health Data Sciences and Informatics (OHDSI) ecosystem.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>THESEUS executes 2 steps: an LLM maps study descriptions to a JavaScript Object Notation (JSON) schema, and validated specifications are converted into Strategus R scripts through rule-based logic. For standardization evaluation, we compared specifications generated by 8 LLMs using 15 OHDSI-based TTE studies and 15 non-OHDSI studies under primary-analysis and full-analyses settings.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Under the primary-analysis setting, overall standardization accuracy ranged from 0.93 to 0.97 across models in OHDSI studies and from 0.82 to 0.95 in non-OHDSI studies. Gemini-3.1-Pro achieved the highest overall accuracy in OHDSI studies, while Gemini-3.1-Pro and Gpt-5.5 jointly achieved the highest overall accuracy in non-OHDSI studies. Under the full-analyses setting, field-level sensitivity ranged from 0.83 to 0.97 in OHDSI studies, with 0.07\u20130.80 false positives (FPs) per study, and from 0.77 to 0.89 in non-OHDSI studies, with 0.53\u20131.20 FPs per study. Gpt-5.5 performed best at the field level. THESEUS was implemented as a web application and coding-agent tools.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>Pairing a standardized data model with a structured analysis framework enables reliable LLM-assisted interpretation of study descriptions and deterministic workflow construction in observational research.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>THESEUS supports translation of natural language study descriptions into executable, shareable code in standardized observational research settings.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamiaopen\/ooag131","type":"journal-article","created":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T17:22:42Z","timestamp":1783617762000},"source":"Crossref","is-referenced-by-count":3,"title":["From study design to executable code: automating target trial emulation with large language models"],"prefix":"10.1093","volume":"9","author":[{"given":"Hanjae","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Biomedical Systems Informatics, Yonsei University College of Medicine , Seoul,","place":["Republic of Korea"]},{"name":"Yonsei Institute for Digital Health, Yonsei University , Seoul,","place":["Republic of Korea"]},{"name":"Institute for AI and Social Innovation, Yonsei University , Seoul,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Minseong","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Biomedical Systems Informatics, Yonsei University College of Medicine , Seoul,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seonji","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Biomedical Systems Informatics, Yonsei University College of Medicine , Seoul,","place":["Republic of Korea"]},{"name":"Yonsei Institute for Digital Health, Yonsei University , Seoul,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seng Chan","family":"You","sequence":"additional","affiliation":[{"name":"Department of Biomedical Systems Informatics, Yonsei University College of Medicine , Seoul,","place":["Republic of Korea"]},{"name":"Yonsei Institute for Digital Health, Yonsei University , Seoul,","place":["Republic of Korea"]},{"name":"Institute for AI and Social Innovation, Yonsei University , Seoul,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,7,9]]},"reference":[{"key":"2026080604270434000_ooag131-B1","doi-asserted-by":"crossref","first-page":"2446","DOI":"10.1001\/jama.2022.21383","article-title":"Target trial emulation: a framework for causal inference from observational data","volume":"328","author":"Hern\u00e1n","year":"2022","journal-title":"JAMA."},{"key":"2026080604270434000_ooag131-B2","doi-asserted-by":"crossref","first-page":"e2200140","DOI":"10.1200\/CCI.22.00140","article-title":"Target trial emulation: a design tool for cancer clinical trials","volume":"7","author":"Kwee","year":"2023","journal-title":"JCO Clin Cancer Inform."},{"key":"2026080604270434000_ooag131-B3","doi-asserted-by":"crossref","first-page":"104897","DOI":"10.1016\/j.jbi.2025.104897","article-title":"Review of tools to support target trial emulation","volume":"171","author":"van Hal","year":"2025","journal-title":"J Biomed Inform."},{"key":"2026080604270434000_ooag131-B4","doi-asserted-by":"crossref","first-page":"e240091","DOI":"10.57264\/cer-2024-0091","article-title":"Why is target trial emulation not being used in health technology assessment real-world data submissions?","volume":"13","author":"Castanon","year":"2024","journal-title":"J Comp Eff Res."},{"key":"2026080604270434000_ooag131-B5","author":"Observational Health Data Sciences and Informatics","year":"2019"},{"key":"2026080604270434000_ooag131-B6","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1055\/s-0041-1726481","article-title":"Drawing reproducible conclusions from observational clinical data with OHDSI","volume":"30","author":"Hripcsak","year":"2021","journal-title":"Yearb Med Inform."},{"key":"2026080604270434000_ooag131-B7","first-page":"966","article-title":"Health-analytics data to evidence suite (HADES): open-source software for observational research","volume":"310","author":"Schuemie","year":"2024","journal-title":"Stud Health Technol Inform."},{"key":"2026080604270434000_ooag131-B8","author":"Sena","year":"2026"},{"key":"2026080604270434000_ooag131-B9","doi-asserted-by":"crossref","first-page":"e67469","DOI":"10.2196\/67469","article-title":"Large language models in randomized controlled trials design: observational study","volume":"27","author":"Jin","year":"2025","journal-title":"J Med Internet Res."},{"key":"2026080604270434000_ooag131-B10","doi-asserted-by":"crossref","first-page":"e71252","DOI":"10.2196\/71252","article-title":"Large language models for automating clinical trial criteria conversion to observational medical outcomes partnership common data model queries: validation and evaluation study","volume":"13","author":"Lee","year":"2025","journal-title":"JMIR Med Inform."},{"key":"2026080604270434000_ooag131-B11","doi-asserted-by":"crossref","first-page":"104649","DOI":"10.1016\/j.jbi.2024.104649","article-title":"Criteria2Query 3.0: leveraging generative large language models for clinical trial eligibility query generation","volume":"154","author":"Park","year":"2024","journal-title":"J Biomed Inform."},{"key":"2026080604270434000_ooag131-B12","doi-asserted-by":"crossref","first-page":"1994","DOI":"10.1093\/jamia\/ocae072","article-title":"Large language models facilitate the generation of electronic health record phenotyping algorithms","volume":"31","author":"Yan","year":"2024","journal-title":"J Am Med Inform Assoc."},{"key":"2026080604270434000_ooag131-B13","doi-asserted-by":"crossref","first-page":"e23983","DOI":"10.2196\/23983","article-title":"A framework (SOCRATex) for hierarchical annotation of unstructured electronic health records and integration into a standardized medical database: development and usability study","volume":"9","author":"Park","year":"2021","journal-title":"JMIR Med Inform."},{"key":"2026080604270434000_ooag131-B14","doi-asserted-by":"crossref","first-page":"1123","DOI":"10.3348\/kjr.2025.1522","article-title":"Minimum reporting items for clear evaluation of accuracy reports of large language models in healthcare (MI-CLEAR-LLM): 2025 updates","volume":"26","author":"Park","year":"2025","journal-title":"Korean J Radiol."},{"key":"2026080604270434000_ooag131-B15","author":"Schuemie","year":"2026"},{"key":"2026080604270434000_ooag131-B16","author":"Ushey","year":"2026"},{"key":"2026080604270434000_ooag131-B17","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1001\/jamaophthalmol.2024.6555","article-title":"Semaglutide and nonarteritic anterior ischemic optic neuropathy","volume":"143","author":"Cai","year":"2025","journal-title":"JAMA Ophthalmol."},{"key":"2026080604270434000_ooag131-B18","doi-asserted-by":"crossref","first-page":"733","DOI":"10.1016\/j.oret.2024.03.014","article-title":"Similar risk of kidney failure among patients with blinding diseases who receive ranibizumab, aflibercept, and bevacizumab: an observational health data sciences and informatics network study","volume":"8","author":"Cai","year":"2024","journal-title":"Ophthalmol Retina."},{"key":"2026080604270434000_ooag131-B19","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1016\/j.jagp.2020.09.017","article-title":"Stroke risk among elderly users of haloperidol and typical antipsychotics versus atypical antipsychotics: a real-world study from a US health insurance claims database","volume":"29","author":"Fife","year":"2021","journal-title":"Am J Geriatr Psychiatry."},{"key":"2026080604270434000_ooag131-B20","doi-asserted-by":"crossref","first-page":"904","DOI":"10.1016\/j.jacc.2024.05.069","article-title":"Comparative effectiveness of second-line antihyperglycemic agents for cardiovascular outcomes","volume":"84","author":"Khera","year":"2024","journal-title":"JACC"},{"key":"2026080604270434000_ooag131-B21","doi-asserted-by":"crossref","first-page":"1515","DOI":"10.7326\/M22-0511","article-title":"Comparative effectiveness and safety between apixaban, dabigatran, edoxaban, and rivaroxaban among patients with atrial fibrillation","volume":"175","author":"Lau","year":"2022","journal-title":"Ann Intern Med."},{"key":"2026080604270434000_ooag131-B22","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1186\/s12888-024-05578-6","article-title":"Comparative estimation of the effects of antihypertensive medications on schizophrenia occurrence: a multinational observational cohort study","volume":"24","author":"Lee","year":"2024","journal-title":"BMC Psychiatry."},{"key":"2026080604270434000_ooag131-B23","doi-asserted-by":"crossref","first-page":"127044","DOI":"10.1016\/j.vaccine.2025.127044","article-title":"A rapid cycle analytics framework for vaccine safety surveillance within a real-world data network: experience with enhanced surveillance of the Janssen COVID-19 vaccine","volume":"55","author":"Newbern","year":"2025","journal-title":"Vaccine."},{"key":"2026080604270434000_ooag131-B24","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1186\/s40360-022-00549-7","article-title":"Comparative risk of incidence and clinical outcomes of COVID-19 among proton pump inhibitor and histamine-2 receptor antagonist short-term users: a nationwide retrospective cohort study","volume":"23","author":"Park","year":"2022","journal-title":"BMC Pharmacol Toxicol."},{"key":"2026080604270434000_ooag131-B25","doi-asserted-by":"crossref","first-page":"692","DOI":"10.14309\/ajg.0000000000001153","article-title":"Comparative effectiveness of famotidine in hospitalized COVID-19 patients","volume":"116","author":"Shoaibi","year":"2021","journal-title":"Am J Gastroenterol."},{"key":"2026080604270434000_ooag131-B26","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1097\/AOG.0000000000003656","article-title":"Relative risk of cervical neoplasms among copper and levonorgestrel-releasing intrauterine system users","volume":"135","author":"Spotnitz","year":"2020","journal-title":"Obstet Gynecol."},{"key":"2026080604270434000_ooag131-B27","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1007\/s40264-022-01198-9","article-title":"Hip fracture risk after treatment with tramadol or codeine: an observational study","volume":"45","author":"Voss","year":"2022","journal-title":"Drug Saf."},{"key":"2026080604270434000_ooag131-B28","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1007\/s40264-021-01060-4","article-title":"Comparative risk assessment of severe uterine bleeding following exposure to direct oral anticoagulants: a network study across four observational databases in the USA","volume":"44","author":"Weaver","year":"2021","journal-title":"Drug Saf."},{"key":"2026080604270434000_ooag131-B29","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1186\/s12874-024-02428-7","article-title":"The necessity of validity diagnostics when drawing causal inferences from observational data: lessons from a multi-database evaluation of the risk of non-infectious uveitis among patients exposed to Remicade\u00ae","volume":"24","author":"Weaver","year":"2024","journal-title":"BMC Med Res Methodol."},{"key":"2026080604270434000_ooag131-B30","doi-asserted-by":"crossref","first-page":"1640","DOI":"10.1001\/jama.2020.16167","article-title":"Association of ticagrelor vs clopidogrel with net adverse clinical events in patients with acute coronary syndrome undergoing percutaneous coronary intervention","volume":"324","author":"You","year":"2020","journal-title":"JAMA."},{"key":"2026080604270434000_ooag131-B31","doi-asserted-by":"crossref","first-page":"e2333495","DOI":"10.1001\/jamanetworkopen.2023.33495","article-title":"Ranitidine use and incident cancer in a multinational cohort","volume":"6","author":"You","year":"2023","journal-title":"JAMA Netw Open."},{"key":"2026080604270434000_ooag131-B32","doi-asserted-by":"crossref","first-page":"992","DOI":"10.3390\/cancers13050992","article-title":"Emulation of a target trial to evaluate the causal effect of palliative care consultation on the survival time of patients with Hepatocellular Carcinoma","volume":"13","author":"Buranupakorn","year":"2021","journal-title":"Cancers"},{"key":"2026080604270434000_ooag131-B33","doi-asserted-by":"crossref","first-page":"101678","DOI":"10.1016\/j.eclinm.2022.101678","article-title":"Comparative effectiveness of valproic acid in different serum concentrations for maintenance treatment of bipolar disorder: a retrospective cohort study using target trial emulation framework","volume":"54","author":"Chen","year":"2022","journal-title":"EClinicalMedicine."},{"key":"2026080604270434000_ooag131-B34","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1186\/s12916-025-04423-1","article-title":"Target trial emulation of DPP-4 inhibitors in patients with T2DM for pulmonary tuberculosis: a nationwide observational data","volume":"23","author":"Chen","year":"2025","journal-title":"BMC Med."},{"key":"2026080604270434000_ooag131-B35","doi-asserted-by":"crossref","first-page":"656","DOI":"10.7326\/M21-4009","article-title":"Risks for anaphylaxis with intravenous iron formulations: a retrospective cohort study","volume":"175","author":"Dave","year":"2022","journal-title":"Ann Intern Med."},{"key":"2026080604270434000_ooag131-B36","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1002\/cpt.2304","article-title":"Calibrating real-world evidence studies against randomized trials: treatment effectiveness of infliximab in Crohn\u2019s disease","volume":"111","author":"Kirchgesner","year":"2022","journal-title":"Clin Pharmacol Ther."},{"key":"2026080604270434000_ooag131-B37","doi-asserted-by":"crossref","first-page":"167","DOI":"10.7326\/ANNALS-24-03237","article-title":"Cardiovascular safety and fracture prevention effectiveness of denosumab versus oral bisphosphonates in patients receiving dialysis: a target trial emulation","volume":"178","author":"Masuda","year":"2025","journal-title":"Ann Intern Med."},{"key":"2026080604270434000_ooag131-B38","doi-asserted-by":"crossref","first-page":"e080035","DOI":"10.1136\/bmj-2024-080035","article-title":"Comparative effectiveness of sodium-glucose cotransporter-2 inhibitors for recurrent nephrolithiasis among patients with pre-existing nephrolithiasis or gout: target trial emulation studies","volume":"387","author":"McCormick","year":"2024","journal-title":"BMJ"},{"key":"2026080604270434000_ooag131-B39","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.jclinepi.2021.07.001","article-title":"An emulated target trial analysis based on Medicare data suggested non-inferiority of Dabigatran versus Rivaroxaban","volume":"139","author":"Mei","year":"2021","journal-title":"J Clin Epidemiol."},{"key":"2026080604270434000_ooag131-B40","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.3390\/e22121349","article-title":"Evaluation of survival outcomes of endovascular versus open aortic repair for abdominal aortic aneurysms with a big data approach","volume":"22","author":"Mei","year":"2020","journal-title":"Entropy"},{"key":"2026080604270434000_ooag131-B41","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1186\/s13054-022-04016-x","article-title":"Effect of erythromycin on mortality and the host response in critically ill patients with sepsis: a target trial emulation","volume":"26","author":"Reijnders","year":"2022","journal-title":"Crit Care"},{"key":"2026080604270434000_ooag131-B42","doi-asserted-by":"crossref","first-page":"927","DOI":"10.7326\/M21-4012","article-title":"Cardiovascular outcomes in patients initiating first-line treatment of type 2 diabetes with sodium\u2013glucose cotransporter-2 inhibitors versus metformin: a cohort study","volume":"175","author":"Shin","year":"2022","journal-title":"Ann Intern Med."},{"key":"2026080604270434000_ooag131-B43","doi-asserted-by":"crossref","first-page":"1767","DOI":"10.1681\/ASN.2022020135","article-title":"Association of rosuvastatin use with risk of hematuria and proteinuria","volume":"33","author":"Shin","year":"2022","journal-title":"J Am Soc Nephrol."},{"key":"2026080604270434000_ooag131-B44","doi-asserted-by":"crossref","first-page":"2072","DOI":"10.1161\/HYPERTENSIONAHA.125.25549","article-title":"Comparative effectiveness of angiotensin-converting enzyme inhibitors versus angiotensin receptor blockers: multidatabase target trial emulation studies","volume":"82","author":"Xie","year":"2025","journal-title":"Hypertension."},{"key":"2026080604270434000_ooag131-B45","doi-asserted-by":"crossref","first-page":"l1580","DOI":"10.1136\/bmj.l1580","article-title":"Estimates of all cause mortality and cause specific mortality associated with proton pump inhibitors among US veterans: cohort study","volume":"365","author":"Xie","year":"2019","journal-title":"BMJ"},{"key":"2026080604270434000_ooag131-B46","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1016\/S1470-2045(23)00170-5","article-title":"Radical cystectomy versus trimodality therapy for muscle-invasive bladder cancer: a multi-institutional propensity score matched and weighted analysis","volume":"24","author":"Zlotta","year":"2023","journal-title":"Lancet Oncol."},{"key":"2026080604270434000_ooag131-B47","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1093\/jamia\/ocad009","article-title":"Reproducible variability: assessing investigator discordance across 9 research teams attempting to reproduce the same observational study","volume":"30","author":"Ostropolets","year":"2023","journal-title":"J Am Med Inform Assoc."},{"key":"2026080604270434000_ooag131-B48","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1016\/j.jclinepi.2022.08.012","article-title":"Understanding variation in the results of real-world evidence studies that seem to address the same question","volume":"151","author":"Wang","year":"2022","journal-title":"J Clin Epidemiol."},{"key":"2026080604270434000_ooag131-B49","doi-asserted-by":"crossref","first-page":"1036","DOI":"10.1093\/jamia\/ocae005","article-title":"Improving reporting standards for phenotyping algorithm in biomedical research: 5 fundamental dimensions","volume":"31","author":"Wei","year":"2024","journal-title":"J Am Med Inform Assoc."}],"container-title":["JAMIA Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/9\/4\/ooag131\/68739869\/ooag131.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/9\/4\/ooag131\/68739869\/ooag131.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T08:27:14Z","timestamp":1786004834000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/doi\/10.1093\/jamiaopen\/ooag131\/8729191"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,7]]},"references-count":49,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,7,7]]}},"URL":"https:\/\/doi.org\/10.1093\/jamiaopen\/ooag131","relation":{},"ISSN":["2574-2531"],"issn-type":[{"value":"2574-2531","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,8]]},"published":{"date-parts":[[2026,7,7]]},"article-number":"ooag131"}}