{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,1,16]],"date-time":"2026-01-16T12:16:45Z","timestamp":1768565805075,"version":"3.49.0"},"posted":{"date-parts":[[2023,11,2]]},"group-title":"Genetic and Genomic Medicine","reference-count":37,"publisher":"openRxiv","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2023,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                <jats:p>The value of genetic information for improving the performance of clinical risk prediction models has yielded variable conclusions. Many methodological decisions have the potential to contribute to differential results across studies. Here, we performed multiple modeling experiments integrating clinical and demographic data from electronic health records (EHR) and genetic data to understand which decision points may affect performance. Clinical data in the form of structured diagnostic codes, medications, procedural codes, and demographics were extracted from two large independent health systems and polygenic risk scores (PRS) were generated across all patients with genetic data in the corresponding biobanks. Crohn\u2019s disease was used as the model phenotype based on its substantial genetic component, established EHR-based definition, and sufficient prevalence for model training and testing. We investigated the impact of PRS integration method, as well as choices regarding training sample, model complexity, and performance metrics. Overall, our results show that including PRS resulted in higher performance by some metrics but the gain in performance was only robust when combined with demographic data alone. Improvements were inconsistent or negligible after including additional clinical information. The impact of genetic information on performance also varied by PRS integration method, with a small improvement in some cases from combining PRS with the output of a clinical model (late-fusion) compared to its inclusion an additional feature (early-fusion). The effects of other modeling decisions varied between institutions though performance increased with more compute-intensive models such as random forest. This work highlights the importance of considering methodological decision points in interpreting the impact on prediction performance when including PRS information in clinical models.<\/jats:p>","DOI":"10.1101\/2023.11.01.23297927","type":"posted-content","created":{"date-parts":[[2023,11,2]],"date-time":"2023-11-02T04:20:17Z","timestamp":1698898817000},"source":"Crossref","is-referenced-by-count":0,"title":["Evaluating the impact of modeling choices on the performance of integrated genetic and clinical models"],"prefix":"10.64898","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-6006-5903","authenticated-orcid":false,"given":"Theodore J.","family":"Morley","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Drew","family":"Willimitis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9613-346X","authenticated-orcid":false,"given":"Michael","family":"Ripperger","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4694-7016","authenticated-orcid":false,"given":"Hyunjoon","family":"Lee","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9077-7766","authenticated-orcid":false,"given":"Lide","family":"Han","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Zhou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6603-0988","authenticated-orcid":false,"given":"Jooeun","family":"Kang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5143-2282","authenticated-orcid":false,"given":"Lea K.","family":"Davis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0381-6334","authenticated-orcid":false,"given":"Jordan W.","family":"Smoller","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3914-2431","authenticated-orcid":false,"given":"Karmel W.","family":"Choi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9379-2056","authenticated-orcid":false,"given":"Colin G.","family":"Walsh","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2365-386X","authenticated-orcid":false,"given":"Douglas M.","family":"Ruderfer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"54368","reference":[{"key":"2023110407500675000_2023.11.01.23297927v1.1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2019.21782"},{"key":"2023110407500675000_2023.11.01.23297927v1.2","doi-asserted-by":"publisher","DOI":"10.1038\/s41588-018&ndash;0183-z"},{"key":"2023110407500675000_2023.11.01.23297927v1.3","doi-asserted-by":"crossref","first-page":"e003304","DOI":"10.1161\/CIRCGEN.120.003304","article-title":"Integrated Polygenic Tool Substantially Enhances Coronary Artery Disease Prediction","volume":"14","year":"2021","journal-title":"Circ Genom Precis Med"},{"key":"2023110407500675000_2023.11.01.23297927v1.4","doi-asserted-by":"crossref","first-page":"1082","DOI":"10.1001\/jamainternmed.2022.3171","article-title":"Incremental Value of Polygenic Risk Scores in Primary Prevention of Coronary Heart Disease: A Review","volume":"182","year":"2022","journal-title":"JAMA Intern Med"},{"key":"2023110407500675000_2023.11.01.23297927v1.5","doi-asserted-by":"crossref","unstructured":"Hahn, S.-J. , Kim, S. , Choi, Y. S. , Lee, J. & Kang, J . Prediction of type 2 diabetes using genome-wide polygenic risk score and metabolic profiles: A machine learning analysis of population-based 10-year prospective cohort study. eBioMedicine 86, (2022).","DOI":"10.1016\/j.ebiom.2022.104383"},{"key":"2023110407500675000_2023.11.01.23297927v1.6","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-43144-3"},{"key":"2023110407500675000_2023.11.01.23297927v1.7","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1001\/jamacardio.2023.0857","article-title":"Cardiovascular Disease Risk Assessment Using Traditional Risk Factors and Polygenic Risk Scores in the Million Veteran Program","volume":"8","year":"2023","journal-title":"JAMA Cardiology"},{"key":"2023110407500675000_2023.11.01.23297927v1.8","doi-asserted-by":"crossref","unstructured":"Kang, E. A. et al. Development of a Clinical and Genetic Prediction Model for Early Intestinal Resection in Patients with Crohn\u2019s Disease: Results from the IMPACT Study. Journal of Clinical Medicine 10, 633 (2021).","DOI":"10.3390\/jcm10040633"},{"key":"2023110407500675000_2023.11.01.23297927v1.9","doi-asserted-by":"publisher","DOI":"10.1101\/2020.06.01.20119297"},{"key":"2023110407500675000_2023.11.01.23297927v1.10","doi-asserted-by":"publisher","DOI":"10.1016\/j.eururo.2011.01.017"},{"key":"2023110407500675000_2023.11.01.23297927v1.11","doi-asserted-by":"crossref","unstructured":"Elliott, J. et al. Predictive Accuracy of a Polygenic Risk Score\u2013Enhanced Prediction Model vs a Clinical Risk Score for Coronary Artery Disease. JAMA 323, 636 (2020).","DOI":"10.1001\/jama.2019.22241"},{"key":"2023110407500675000_2023.11.01.23297927v1.12","doi-asserted-by":"publisher","DOI":"10.1158\/1940-6207.CAPR-17-0141"},{"key":"2023110407500675000_2023.11.01.23297927v1.13","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-020-0800-0"},{"key":"2023110407500675000_2023.11.01.23297927v1.14","first-page":"3437","article-title":"Affect recognition from face and body: early fusion vs. late fusion. in 2005 IEEE International Conference on Systems","volume":"4","year":"2005","journal-title":"Man and Cybernetics"},{"key":"2023110407500675000_2023.11.01.23297927v1.15","doi-asserted-by":"crossref","unstructured":"Lee, T. C. , Shah, N. U. , Haack, A. & Baxter, S. L . Clinical Implementation of Predictive Models Embedded within Electronic Health Record Systems: A Systematic Review. Informatics (MDPI ) 7, 25 (2020).","DOI":"10.3390\/informatics7030025"},{"key":"2023110407500675000_2023.11.01.23297927v1.16","doi-asserted-by":"publisher","DOI":"10.1038\/clpt.2014.137"},{"key":"2023110407500675000_2023.11.01.23297927v1.17","doi-asserted-by":"publisher","DOI":"10.1200\/JCO.2013.53.6607"},{"key":"2023110407500675000_2023.11.01.23297927v1.18","unstructured":"Linder, J. E. et al. Returning integrated genomic risk and clinical recommendations: The eMERGE study. Genet Med 25, 100006 (2023)."},{"key":"2023110407500675000_2023.11.01.23297927v1.19","doi-asserted-by":"crossref","unstructured":"Dahlhamer, J. M. Prevalence of Inflammatory Bowel Disease Among Adults Aged \u226518 Years \u2014 United States, 2015. MMWR Morb Mortal Wkly Rep 65, (2016).","DOI":"10.15585\/mmwr.mm6542a3"},{"key":"2023110407500675000_2023.11.01.23297927v1.20","unstructured":"Crohn\u2019s Disease - Demonstration Project | PheKB. https:\/\/phekb.org\/phenotype\/crohns-disease-demonstration-project."},{"key":"2023110407500675000_2023.11.01.23297927v1.21","unstructured":"Definitions, epidemiology, and risk factors for inflammatory bowel disease - UpToDate. https:\/\/www.uptodate.com\/contents\/definitions-epidemiology-and-risk-factors-for-inflammatory-bowel-disease."},{"key":"2023110407500675000_2023.11.01.23297927v1.22","doi-asserted-by":"crossref","first-page":"e10243","DOI":"10.1002\/lrh2.10243","article-title":"Development and evaluation of an EHR-based computable phenotype for identification of pediatric Crohn\u2019s disease patients in a National Pediatric Learning Health System","volume":"4","year":"2020","journal-title":"Learning Health Systems"},{"key":"2023110407500675000_2023.11.01.23297927v1.23","unstructured":"PsycheMERGE. https:\/\/psychemerge.com\/."},{"key":"2023110407500675000_2023.11.01.23297927v1.24","first-page":"1044","article-title":"Calculating the benefits of a Research Patient Data Repository","volume":"2006","year":"2006","journal-title":"AMIA \u2026 Annual Symposium proceedings. AMIA Symposium"},{"key":"2023110407500675000_2023.11.01.23297927v1.25","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajhg.2010.03.003"},{"key":"2023110407500675000_2023.11.01.23297927v1.26","unstructured":"Healthcare Cost and Utilization Project. HCUP Clinical Classifications Software (CCS) for ICD-9-CM. (2006)."},{"key":"2023110407500675000_2023.11.01.23297927v1.27","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000116"},{"key":"2023110407500675000_2023.11.01.23297927v1.28","unstructured":"Jr, F. E. H. & Dupont, C. Hmisc: Harrell Miscellaneous. (2023)."},{"key":"2023110407500675000_2023.11.01.23297927v1.29","doi-asserted-by":"publisher","DOI":"10.1038\/ng.3760."},{"key":"2023110407500675000_2023.11.01.23297927v1.30","doi-asserted-by":"crossref","first-page":"1776","DOI":"10.1038\/s41467-019-09718-5","article-title":"Polygenic prediction via Bayesian regression and continuous shrinkage priors","volume":"10","year":"2019","journal-title":"Nature Communications"},{"key":"2023110407500675000_2023.11.01.23297927v1.31","doi-asserted-by":"crossref","unstructured":"Chang, C. C. et al. Second-generation PLINK: rising to the challenge of larger and richer datasets. GigaScience 4, s13742-015-0047\u20138 (2015).","DOI":"10.1186\/s13742-015-0047-8"},{"key":"2023110407500675000_2023.11.01.23297927v1.32","unstructured":"Purcell, S. & Chang, C. PLINK 1.9."},{"key":"2023110407500675000_2023.11.01.23297927v1.33","doi-asserted-by":"crossref","unstructured":"Varma, S. & Simon, R . Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics 7, 91 (2006).","DOI":"10.1186\/1471-2105-7-91"},{"key":"2023110407500675000_2023.11.01.23297927v1.34","doi-asserted-by":"publisher","DOI":"10.1524\/auto.2011.0951"},{"key":"2023110407500675000_2023.11.01.23297927v1.35","doi-asserted-by":"publisher","DOI":"10.1002\/sim.2929"},{"key":"2023110407500675000_2023.11.01.23297927v1.36","doi-asserted-by":"publisher","DOI":"10.1002\/sim.1844"},{"key":"2023110407500675000_2023.11.01.23297927v1.37","doi-asserted-by":"crossref","unstructured":"Cavazos, T. B. & Witte, J. S . Inclusion of variants discovered from diverse populations improves polygenic risk score transferability. HGG Adv 2, 100017 (2020).","DOI":"10.1016\/j.xhgg.2020.100017"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1101\/2023.11.01.23297927","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T19:44:18Z","timestamp":1768506258000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.1101\/2023.11.01.23297927"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,2]]},"references-count":37,"URL":"https:\/\/doi.org\/10.1101\/2023.11.01.23297927","relation":{"is-preprint-of":[{"id-type":"doi","id":"10.1016\/j.gim.2024.101353","asserted-by":"subject"}]},"subject":[],"published":{"date-parts":[[2023,11,2]]},"subtype":"preprint"}}