{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T15:40:08Z","timestamp":1768405208416,"version":"3.49.0"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,5,18]],"date-time":"2022-05-18T00:00:00Z","timestamp":1652832000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,5,18]],"date-time":"2022-05-18T00:00:00Z","timestamp":1652832000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>Identifying associations among biological variables is a major challenge in modern quantitative biological research, particularly given the systemic and statistical noise endemic to biological systems. Drug sensitivity data has proven to be a particularly challenging field for identifying associations to inform patient treatment.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>To address this, we introduce two semi-parametric variations on the commonly used concordance index: the robust concordance index and the kernelized concordance index (rCI, kCI), which incorporate measurements about the noise distribution from the data. We demonstrate that common statistical tests applied to the concordance index and its variations fail to control for false positives, and introduce efficient implementations to compute p-values using adaptive permutation testing. We then evaluate the statistical power of these coefficients under simulation and compare with Pearson and Spearman correlation coefficients. Finally, we evaluate the various statistics in matching drugs across pharmacogenomic datasets.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>We observe that the rCI and kCI are better powered than the concordance index in simulation and show some improvement on real data. Surprisingly, we observe that the Pearson correlation was the most robust to measurement noise among the different metrics.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s12859-022-04693-z","type":"journal-article","created":{"date-parts":[[2022,5,18]],"date-time":"2022-05-18T13:03:04Z","timestamp":1652878984000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Evaluation of statistical approaches for association testing in noisy drug screening data"],"prefix":"10.1186","volume":"23","author":[{"given":"Petr","family":"Smirnov","sequence":"first","affiliation":[]},{"given":"Ian","family":"Smith","sequence":"additional","affiliation":[]},{"given":"Zhaleh","family":"Safikhani","sequence":"additional","affiliation":[]},{"given":"Wail","family":"Ba-alawi","sequence":"additional","affiliation":[]},{"given":"Farnoosh","family":"Khodakarami","sequence":"additional","affiliation":[]},{"given":"Eva","family":"Lin","sequence":"additional","affiliation":[]},{"given":"Yihong","family":"Yu","sequence":"additional","affiliation":[]},{"given":"Scott","family":"Martin","sequence":"additional","affiliation":[]},{"given":"Janosch","family":"Ortmann","sequence":"additional","affiliation":[]},{"given":"Tero","family":"Aittokallio","sequence":"additional","affiliation":[]},{"given":"Marc","family":"Hafner","sequence":"additional","affiliation":[]},{"given":"Benjamin","family":"Haibe-Kains","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,5,18]]},"reference":[{"key":"4693_CR1","doi-asserted-by":"publisher","first-page":"1896","DOI":"10.1002\/jcp.24662","volume":"229","author":"CS Greene","year":"2014","unstructured":"Greene CS, Tan J, Ung M, Moore JH, Cheng C. Big data bioinformatics. J Cell Physiol. 2014;229:1896\u2013900.","journal-title":"J Cell Physiol"},{"key":"4693_CR2","doi-asserted-by":"publisher","first-page":"20170387","DOI":"10.1098\/rsif.2017.0387","volume":"15","author":"T Ching","year":"2018","unstructured":"Ching T, et al. Opportunities and obstacles for deep learning in biology and medicine. J R Soc Interface. 2018;15:20170387.","journal-title":"J R Soc Interface"},{"key":"4693_CR3","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1002\/jcp.21218","volume":"213","author":"JH Moore","year":"2007","unstructured":"Moore JH. Bioinformatics. J Cell Physiol. 2007;213:365\u20139.","journal-title":"J Cell Physiol"},{"key":"4693_CR4","doi-asserted-by":"publisher","first-page":"026601","DOI":"10.1088\/0034-4885\/77\/2\/026601","volume":"77","author":"LS Tsimring","year":"2014","unstructured":"Tsimring LS. Noise in biology reports on progress in physics. Phys Soc (Great Britain). 2014;77:026601.","journal-title":"Phys Soc (Great Britain)"},{"key":"4693_CR5","doi-asserted-by":"publisher","first-page":"1202","DOI":"10.1038\/nbt.2877","volume":"32","author":"JC Costello","year":"2014","unstructured":"Costello JC, et al. A community effort to assess and improve drug sensitivity prediction algorithms. Nat Biotechnol. 2014;32:1202\u201312.","journal-title":"Nat Biotechnol"},{"key":"4693_CR6","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1038\/nature17987","volume":"533","author":"PM Haverty","year":"2016","unstructured":"Haverty PM, et al. Reproducible pharmacogenomic profiling of cancer cell line panels. Nature. 2016;533:333\u20137.","journal-title":"Nature"},{"key":"4693_CR7","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1214\/aoms\/1177732543","volume":"7","author":"H Hotelling","year":"1936","unstructured":"Hotelling H, Pabst MR. Rank correlation and tests of significance involving no assumption of normality. Ann Math Stat. 1936;7:29\u201343.","journal-title":"Ann Math Stat"},{"key":"4693_CR8","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1080\/00031305.1957.10501091","volume":"11","author":"S Siegel","year":"1957","unstructured":"Siegel S. Nonparametric statistics. Am Stat. 1957;11:13\u20139.","journal-title":"Am Stat"},{"key":"4693_CR9","unstructured":"Bradley JV. Distribution-free statistical tests. http:\/\/citeseerx.ist.psu.edu\/viewdoc\/summary? https:\/\/doi.org\/10.1.1.977.3717 1968."},{"key":"4693_CR10","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1177\/014662168701100407","volume":"11","author":"RL Fowler","year":"1987","unstructured":"Fowler RL. Power and robustness in product-moment correlation. Appl Psychol Meas. 1987;11:419\u201328.","journal-title":"Appl Psychol Meas"},{"key":"4693_CR11","doi-asserted-by":"publisher","first-page":"513","DOI":"10.1016\/0306-4573(88)90021-0","volume":"24","author":"G Salton","year":"1988","unstructured":"Salton G, Buckley C. Term-weighting approaches in automatic text retrieval. Inf Process Manage. 1988;24:513\u201323.","journal-title":"Inf Process Manage"},{"key":"4693_CR12","first-page":"1","volume":"13","author":"L Song","year":"2012","unstructured":"Song L, Langfelder P, Horvath S. Comparison of co-expression measures: mutual information, correlation, and model based indices. BMC Bioinformatics. 2012;13:1\u201321.","journal-title":"BMC Bioinformatics"},{"key":"4693_CR13","first-page":"77","volume":"19","author":"R Henkel","year":"2018","unstructured":"Henkel R, et al. Notions of similarity for systems biology models. Brief Bioinform. 2018;19:77\u201388.","journal-title":"Brief Bioinform"},{"key":"4693_CR14","doi-asserted-by":"publisher","unstructured":"Metcalf L, Casey W. in Cybersecurity and Applied Mathematics (eds Metcalf, L. & Casey, W.) 3-22 (Syngress, Boston, Jan. 2016). ISBN: 978-0-12-804452-0. https:\/\/doi.org\/10.1016\/B978-0-12-804452-0.00002-6.","DOI":"10.1016\/B978-0-12-804452-0.00002-6"},{"key":"4693_CR15","volume-title":"Pearson\u2019s Versus Spearman\u2019s and Kendall\u2019s Correlation Coeffcients for Continuous Data University of Pittsburgh ETD","author":"NS Chok","year":"2010","unstructured":"Chok NS. Pearson\u2019s Versus Spearman\u2019s and Kendall\u2019s Correlation Coeffcients for Continuous Data University of Pittsburgh ETD. Sept.: University of Pittsburgh; 2010."},{"key":"4693_CR16","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1037\/a0028087","volume":"17","author":"AJ Bishara","year":"2012","unstructured":"Bishara AJ, Hittner JB. Testing the significance of a correlation with nonnormal data: comparison of pearson, spearman, transformation, and resampling approaches. Psychol Methods. 2012;17:399\u2013417.","journal-title":"Psychol Methods"},{"key":"4693_CR17","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1016\/j.anbehav.2014.05.003","volume":"93","author":"M-T Puth","year":"2014","unstructured":"Puth M-T, Neuh\u00e4user M, Ruxton GD. Effective use of Pearson\u2019s product-moment correlation coeffcient. Anim Behav. 2014;93:183\u20139.","journal-title":"Anim Behav"},{"key":"4693_CR18","doi-asserted-by":"publisher","first-page":"438","DOI":"10.1038\/s41598-019-57247-4","volume":"10","author":"E Saccenti","year":"2020","unstructured":"Saccenti E, Hendriks MHWB, Smilde AK. Corruption of the Pearson correlation coeffcient by measurement error and its estimation, bias, and correction under different error models. Sci Rep. 2020;10:438.","journal-title":"Sci Rep"},{"key":"4693_CR19","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1177\/0013164414557639","volume":"75","author":"AJ Bishara","year":"2015","unstructured":"Bishara AJ, Hittner JB. Reducing bias and error in the correlation coeffcient due to nonnormality. Educ Psychol Measur. 2015;75:785\u2013804.","journal-title":"Educ Psychol Measur"},{"key":"4693_CR20","doi-asserted-by":"publisher","unstructured":"Cormack GV, Lynam TR. Power and bias of subset pooling strategies. In proceedings of the 30th annual international ACM SIGIR Conference on research and development in information retrieval (association for computing machinery, New York, NY, USA, July 2007), 837\u2013838. https:\/\/doi.org\/10.1145\/1277741.1277934.","DOI":"10.1145\/1277741.1277934"},{"key":"4693_CR21","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1007\/s12154-009-0029-3","volume":"3","author":"H Prinz","year":"2010","unstructured":"Prinz H. Hill coeffcients, dose-response curves and allosteric mechanisms. J Chem Biol. 2010;3:37\u201344.","journal-title":"J Chem Biol"},{"key":"4693_CR22","doi-asserted-by":"publisher","first-page":"387","DOI":"10.2203\/dose-response.09-030.Beam","volume":"9","author":"AL Beam","year":"2011","unstructured":"Beam AL, Motsinger-Reif AA. Optimization of nonlinear dose- and concentration-response models utilizing evolutionary computation. Dose-Response. 2011;9:387\u2013409.","journal-title":"Dose-Response"},{"key":"4693_CR23","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1038\/nature12831","volume":"504","author":"B Haibe-Kains","year":"2013","unstructured":"Haibe-Kains B, et al. Inconsistency in large pharmacogenomic studies. Nature. 2013;504:389\u201393.","journal-title":"Nature"},{"key":"4693_CR24","doi-asserted-by":"publisher","unstructured":"Safikhani Z, Selby H, Sayad A, Hatzis C, Haibe-Kains B. In High Throughput Screening Methods, Dec. 2016;181-213. https:\/\/doi.org\/10.1039\/9781782626770-00181.","DOI":"10.1039\/9781782626770-00181"},{"key":"4693_CR25","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1080\/01621459.1966.10480879","volume":"61","author":"WR Knight","year":"1966","unstructured":"Knight WR. A computer method for calculating Kendall\u2019s Tau with ungrouped data. J Am Stat Assoc. 1966;61:436\u20139.","journal-title":"J Am Stat Assoc"},{"key":"4693_CR26","unstructured":"Noether GE. Elements of nonparametric statistics. (John Wiley & Sons, Jan. 1967)."},{"key":"4693_CR27","doi-asserted-by":"publisher","first-page":"2109","DOI":"10.1002\/sim.1802","volume":"23","author":"MJ Pencina","year":"2004","unstructured":"Pencina MJ, D\u2019Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med. 2004;23:2109\u201323.","journal-title":"Stat Med"},{"key":"4693_CR28","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1093\/biomet\/38.1-2.131","volume":"38","author":"ST David","year":"1951","unstructured":"David ST, Kendall MG, Stuart A. Some questions of distribution in the theory of rank correlation. Biometrika. 1951;38:131\u201340.","journal-title":"Biometrika"},{"key":"4693_CR29","doi-asserted-by":"crossref","unstructured":"Hayes AF, Permutation test is not distribution-free: testing $$\\text{H\/Em}_{0}: \\rho = 0$$. Psychol Methods 1, 184. (19960101).","DOI":"10.1037\/1082-989X.1.2.184"},{"key":"4693_CR30","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1002\/gepi.22268","volume":"44","author":"J Hecker","year":"2020","unstructured":"Hecker J, et al. A flexible and nearly optimal sequential testing approach to randomized testing: QUICK-STOP. Genet Epidemiol. 2020;44:139\u201347.","journal-title":"Genet Epidemiol"},{"key":"4693_CR31","doi-asserted-by":"crossref","unstructured":"Guo W, Peddada S. Adaptive choice of the number of bootstrap samples in large scale multiple testing. Statistical applications in genetics and molecular biology 7, Article13. 2008.","DOI":"10.2202\/1544-6115.1360"},{"key":"4693_CR32","doi-asserted-by":"publisher","first-page":"740","DOI":"10.1016\/j.cell.2016.06.017","volume":"166","author":"F Iorio","year":"2016","unstructured":"Iorio F, et al. A landscape of pharmacogenomic interactions in cancer. Cell. 2016;166:740\u201354.","journal-title":"Cell"},{"key":"4693_CR33","doi-asserted-by":"publisher","first-page":"955","DOI":"10.1093\/nar\/gks1111","volume":"41","author":"W Yang","year":"2013","unstructured":"Yang W, et al. Genomics of drug sensitivity in cancer (GDSC): a resource for therapeutic biomarker discovery in cancer cells. Nucleic Acids Res. 2013;41:955\u201361.","journal-title":"Nucleic Acids Res"},{"key":"4693_CR34","doi-asserted-by":"publisher","first-page":"570","DOI":"10.1038\/nature11005","volume":"483","author":"MJ Garnett","year":"2012","unstructured":"Garnett MJ, et al. Systematic identification of genomic markers of drug sensitivity in cancer cells. Nature. 2012;483:570\u20135.","journal-title":"Nature"},{"key":"4693_CR35","doi-asserted-by":"publisher","first-page":"1210","DOI":"10.1158\/2159-8290.CD-15-0235","volume":"5","author":"B Seashore-Ludlow","year":"2015","unstructured":"Seashore-Ludlow B, et al. Harnessing connectivity in a large-scale small-molecule sensitivity dataset. Cancer Discov. 2015;5:1210\u201323.","journal-title":"Cancer Discov"},{"key":"4693_CR36","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1038\/nchembio.1986","volume":"12","author":"MG Rees","year":"2016","unstructured":"Rees MG, et al. Correlating chemical sensitivity and basal gene expression reveals mechanism of action. Nat Chem Biol. 2016;12:109\u201316.","journal-title":"Nat Chem Biol"},{"key":"4693_CR37","doi-asserted-by":"publisher","first-page":"1151","DOI":"10.1016\/j.cell.2013.08.003","volume":"154","author":"A Basu","year":"2013","unstructured":"Basu A, et al. An interactive resource to identify cancer genetic and lineage dependencies targeted by small molecules. Cell. 2013;154:1151\u201361.","journal-title":"Cell"},{"key":"4693_CR38","doi-asserted-by":"publisher","first-page":"E5","DOI":"10.1038\/nature20171","volume":"540","author":"JP Mpindi","year":"2016","unstructured":"Mpindi JP, et al. Consistency in drug response profiling. Nature. 2016;540:E5\u20136.","journal-title":"Nature"},{"key":"4693_CR39","doi-asserted-by":"publisher","unstructured":"Hafner M, et al. Quantification of sensitivity and resistance of breast cancer cell lines to anti- cancer drugs using gr metrics. Scientific Data. https:\/\/doi.org\/10.1038\/sdata.2017.166 2017.","DOI":"10.1038\/sdata.2017.166"},{"key":"4693_CR40","doi-asserted-by":"publisher","first-page":"1244","DOI":"10.1093\/bioinformatics\/btv723","volume":"32","author":"P Smirnov","year":"2016","unstructured":"Smirnov P, et al. PharmacoGx: an R package for analysis of large pharmacogenomic datasets. Bioinformatics. 2016;32:1244\u20136.","journal-title":"Bioinformatics"},{"key":"4693_CR41","doi-asserted-by":"crossref","unstructured":"Mammoliti A, et al. Orchestrating and sharing large multimodal data for transparent and reproducible research. bioRxiv, 2020.09.18.303842. 2021.","DOI":"10.1101\/2020.09.18.303842"},{"key":"4693_CR42","doi-asserted-by":"crossref","unstructured":"Safikhani Z, et al. Revisiting inconsistency in large pharmacogenomic studies. F1000Research 5, 2333. 2017.","DOI":"10.12688\/f1000research.9611.2"},{"key":"4693_CR43","unstructured":"Margolius BH. Permutations with inversions. J Integer Seq 4. https:\/\/cs.uwaterloo.ca\/journals\/JIS\/VOL4\/MARGOLIUS\/inversions.pdf 2001."},{"key":"4693_CR44","unstructured":"R Core Team. R: A Language and environment for statistical computing manual. R Foundation for Statistical Computing (Vienna, Austria, 2020)."},{"key":"4693_CR45","unstructured":"MacMahon PA. Combinatory analysis, Volumes I and II isbn: 978-0-8218-2832-8 (American Mathematical Soc., 2001)."},{"key":"4693_CR46","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1016\/j.jcta.2015.03.012","volume":"134","author":"JB Remmel","year":"2015","unstructured":"Remmel JB, Wilson AT. An extension of MacMahon\u2019s equidistribution theorem to ordered set partitions. J Combin Theory Ser A. 2015;134:242\u201377.","journal-title":"J Combin Theory Ser A"},{"key":"4693_CR47","doi-asserted-by":"crossref","unstructured":"Olkin I, Trikalinos TA. Constructions for a bivariate beta distribution. arXiv:1406.5881 [math, stat]. June 2014.","DOI":"10.1016\/j.spl.2014.09.013"},{"key":"4693_CR48","doi-asserted-by":"publisher","first-page":"479","DOI":"10.1111\/1467-9868.00346","volume":"64","author":"JD Storey","year":"2002","unstructured":"Storey JD. A direct approach to false discovery rates. J R Stat Soc B. 2002;64:479\u201398.","journal-title":"J. R. Stat. Soc. B"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-022-04693-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-022-04693-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-022-04693-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,25]],"date-time":"2024-09-25T10:15:48Z","timestamp":1727259348000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-022-04693-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,18]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["4693"],"URL":"https:\/\/doi.org\/10.1186\/s12859-022-04693-z","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,18]]},"assertion":[{"value":"18 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 April 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 May 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"BHK is a shareholder and paid consultant for Code Ocean Inc. The authors have no other competing interests to declare.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"188"}}