{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T23:46:32Z","timestamp":1776815192430,"version":"3.51.2"},"reference-count":27,"publisher":"Oxford University Press (OUP)","issue":"20","license":[{"start":{"date-parts":[[2019,3,26]],"date-time":"2019-03-26T00:00:00Z","timestamp":1553558400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"NIH Intramural Research program"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,10,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Polygenic risk score (PRS) methods based on genome-wide association studies (GWAS) have a potential for predicting the risk of developing complex diseases and are expected to become more accurate with larger training datasets and innovative statistical methods. The area under the ROC curve (AUC) is often used to evaluate the performance of PRSs, which requires individual genotypic and phenotypic data in an independent GWAS validation dataset. We are motivated to develop methods for approximating AUC of PRSs based on the summary level data of the validation dataset, which will greatly facilitate the development of PRS models for complex diseases.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We develop statistical methods and an R package SummaryAUC for approximating the AUC and its variance of a PRS when only the summary level data of the validation dataset are available. SummaryAUC can be applied to PRSs with SNPs either genotyped or imputed in the validation dataset. We examined the performance of SummaryAUC using a large-scale GWAS of schizophrenia. SummaryAUC provides accurate approximations to AUCs and their variances. The bias of AUC is typically &amp;lt;0.5% in most analyses. SummaryAUC cannot be applied to PRSs that use all SNPs in the genome because it is computationally prohibitive.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>https:\/\/github.com\/lsncibb\/SummaryAUC.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Supplementary information<\/jats:title>\n                    <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btz176","type":"journal-article","created":{"date-parts":[[2019,3,22]],"date-time":"2019-03-22T08:12:28Z","timestamp":1553242348000},"page":"4038-4044","source":"Crossref","is-referenced-by-count":20,"title":["SummaryAUC: a tool for evaluating the performance of polygenic risk prediction models in validation datasets with only summary level statistics"],"prefix":"10.1093","volume":"35","author":[{"given":"Lei","family":"Song","sequence":"first","affiliation":[{"name":"Biostatistics Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute , Bethesda, MD, USA"},{"name":"Cancer Genomics Research Laboratory, Division of Cancer Epidemiology and Genetics , National Cancer Institute, Bethesda, MD, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aiyi","family":"Liu","sequence":"additional","affiliation":[{"name":"Biostatistics and Bioinformatics Branch, Division of Intramural Population Health Research , National Institute of Child Health and Human Development, Bethesda, MD, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianxin","family":"Shi","sequence":"additional","affiliation":[{"name":"Biostatistics Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute , Bethesda, MD, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"name":"Molecular Genetics of Schizophrenia Consortium","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"P V","family":"Gejman","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"A R","family":"Sanders","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"J","family":"Duan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"C R","family":"Cloninger","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"D M","family":"Svrakic","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"N G","family":"Buccola","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"D F","family":"Levinson","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"B J","family":"Mowry","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"R","family":"Freedman","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"A","family":"Olincy","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"F","family":"Amin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"D W","family":"Black","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"J M","family":"Silverman","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"W F","family":"Byerley","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2019,3,26]]},"reference":[{"key":"2023013108275153400_btz176-B1","doi-asserted-by":"crossref","first-page":"1103","DOI":"10.1038\/ng.3094","article-title":"A meta-analysis of 87,040 individuals identifies 23 new susceptibility loci for prostate cancer","volume":"46","author":"Al Olama","year":"2014","journal-title":"Nat. Genet"},{"key":"2023013108275153400_btz176-B2","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1038\/ng.3211","article-title":"LD score regression distinguishes confounding from polygenicity in genome-wide association studies","volume":"47","author":"Bulik-Sullivan","year":"2015","journal-title":"Nat. Genet"},{"key":"2023013108275153400_btz176-B3","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1038\/ng.2579","article-title":"Projecting the performance of risk prediction based on polygenic analyses of genome-wide association studies","volume":"45","author":"Chatterjee","year":"2013","journal-title":"Nat. Genet"},{"key":"2023013108275153400_btz176-B4","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1038\/nrg.2016.27","article-title":"Developing and evaluating polygenic risk prediction models for stratified disease prevention","volume":"17","author":"Chatterjee","year":"2016","journal-title":"Nat. Rev. Genet"},{"key":"2023013108275153400_btz176-B6","doi-asserted-by":"crossref","first-page":"e1003348","DOI":"10.1371\/journal.pgen.1003348","article-title":"Power and predictive accuracy of polygenic risk scores","volume":"9","author":"Dudbridge","year":"2013","journal-title":"PLoS Genet"},{"key":"2023013108275153400_btz176-B7","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1016\/j.ajhg.2014.09.007","article-title":"Effective genetic-risk prediction using mixed models","volume":"95","author":"Golan","year":"2014","journal-title":"Am. J. Hum. Genet"},{"key":"2023013108275153400_btz176-B8","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1148\/radiology.143.1.7063747","article-title":"The meaning and use of the area under a receiver operating characteristic (Roc) curve","volume":"143","author":"Hanley","year":"1982","journal-title":"Radiology"},{"key":"2023013108275153400_btz176-B9","doi-asserted-by":"crossref","first-page":"e1000167","DOI":"10.1371\/journal.pgen.1000167","article-title":"Resolving individuals contributing trace amounts of DNA to highly complex mixtures using high-density SNP genotyping microarrays","volume":"4","author":"Homer","year":"2008","journal-title":"PLoS Genet"},{"key":"2023013108275153400_btz176-B10","doi-asserted-by":"crossref","first-page":"e1000529","DOI":"10.1371\/journal.pgen.1000529","article-title":"A flexible and accurate genotype imputation method for the next generation of genome-wide association studies","volume":"5","author":"Howie","year":"2009","journal-title":"PLoS Genet"},{"key":"2023013108275153400_btz176-B11","doi-asserted-by":"crossref","first-page":"e1006836","DOI":"10.1371\/journal.pgen.1006836","article-title":"Joint modeling of genetically correlated diseases and functional annotations increases accuracy of polygenic risk prediction","volume":"13","author":"Hu","year":"2017","journal-title":"PLoS Genet"},{"key":"2023013108275153400_btz176-B12","doi-asserted-by":"crossref","first-page":"1253","DOI":"10.1038\/ng.455","article-title":"A new statistic and its power to infer membership in a genome-wide association study using genotype frequencies","volume":"41","author":"Jacobs","year":"2009","journal-title":"Nat. Genet"},{"key":"2023013108275153400_btz176-B13","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.ypmed.2018.02.013","article-title":"A novel metric that quantifies risk stratification for evaluating diagnostic tests: the example of evaluating cervical-cancer screening tests across populations","volume":"110","author":"Katki","year":"2018","journal-title":"Prev. Med"},{"key":"2023013108275153400_btz176-B14","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1002\/gepi.20509","article-title":"Risk prediction using genome-wide association studies","volume":"34","author":"Kooperberg","year":"2010","journal-title":"Genet. Epidemiol"},{"key":"2023013108275153400_btz176-B15","doi-asserted-by":"crossref","DOI":"10.1201\/9781439800225","volume-title":"ROC Curves for Continuous Data","author":"Krzanowski","year":"2009"},{"key":"2023013108275153400_btz176-B16","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1016\/j.ajhg.2014.12.006","article-title":"Joint analysis of psychiatric disorders increases accuracy of risk prediction for schizophrenia, bipolar disorder, and major depressive disorder","volume":"96","author":"Maier","year":"2015","journal-title":"Am. J. Hum. Genet"},{"key":"2023013108275153400_btz176-B17","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1038\/nature24284","article-title":"Association analysis identifies 65 new breast cancer risk loci","volume":"551","author":"Michailidou","year":"2017","journal-title":"Nature"},{"key":"2023013108275153400_btz176-B18","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780198509844.001.0001","volume-title":"The Statistical Evaluation of Medical Tests for Classification and Prediction","author":"Pepe","year":"2003"},{"key":"2023013108275153400_btz176-B19","doi-asserted-by":"crossref","first-page":"748","DOI":"10.1038\/nature08185","article-title":"Common polygenic variation contributes to risk of schizophrenia and bipolar disorder","volume":"460","author":"Purcell","year":"2009","journal-title":"Nature"},{"key":"2023013108275153400_btz176-B20","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1038\/nature13595","article-title":"Biological insights from 108 schizophrenia-associated genetic loci","volume":"511","year":"2014","journal-title":"Nature"},{"key":"2023013108275153400_btz176-B21","doi-asserted-by":"crossref","first-page":"2888","DOI":"10.2337\/db16-1253","article-title":"An expanded genome-wide association study of type 2 diabetes in Europeans","volume":"66","author":"Scott","year":"2017","journal-title":"Diabetes"},{"key":"2023013108275153400_btz176-B22","doi-asserted-by":"crossref","first-page":"753","DOI":"10.1038\/nature08192","article-title":"Common variants on chromosome 6p22.1 are associated with schizophrenia","volume":"460","author":"Shi","year":"2009","journal-title":"Nature"},{"key":"2023013108275153400_btz176-B23","doi-asserted-by":"crossref","first-page":"e1006493","DOI":"10.1371\/journal.pgen.1006493","article-title":"Winner\u2019s curse correction and variable thresholding improve performance of polygenic risk modeling based on genome-wide association study summary-level data","volume":"12","author":"Shi","year":"2016","journal-title":"PLoS Genet"},{"key":"2023013108275153400_btz176-B24","doi-asserted-by":"crossref","first-page":"1550","DOI":"10.1101\/gr.169375.113","article-title":"MultiBLUP: improved SNP-based prediction for complex traits","volume":"24","author":"Speed","year":"2014","journal-title":"Genome Res"},{"key":"2023013108275153400_btz176-B25","doi-asserted-by":"crossref","first-page":"576","DOI":"10.1016\/j.ajhg.2015.09.001","article-title":"Modeling linkage disequilibrium increases accuracy of polygenic risk scores","volume":"97","author":"Vilhjalmsson","year":"2015","journal-title":"Am. J. Hum. Genet"},{"key":"2023013108275153400_btz176-B26","doi-asserted-by":"crossref","first-page":"e1000678","DOI":"10.1371\/journal.pgen.1000678","article-title":"From disease association to risk assessment: an optimistic view from genome-wide association studies on type 1 diabetes","volume":"5","author":"Wei","year":"2009","journal-title":"PLoS Genet"},{"key":"2023013108275153400_btz176-B27","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1038\/ng.608","article-title":"Common SNPs explain a large proportion of the heritability for human height","volume":"42","author":"Yang","year":"2010","journal-title":"Nat. Genet"},{"key":"2023013108275153400_btz176-B28","volume-title":"Statistical Evaluation of Diagnostic Performance: Topics in ROC Analysis","author":"Zou","year":"2011"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/20\/4038\/48976279\/bioinformatics_35_20_4038.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/20\/4038\/48976279\/bioinformatics_35_20_4038.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,15]],"date-time":"2024-07-15T21:44:35Z","timestamp":1721079875000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/35\/20\/4038\/5419857"}},"subtitle":[],"editor":[{"given":"Russell","family":"Schwartz","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2019,3,26]]},"references-count":27,"journal-issue":{"issue":"20","published-print":{"date-parts":[[2019,10,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btz176","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/359463","asserted-by":"object"}]},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,10,15]]},"published":{"date-parts":[[2019,3,26]]}}}