{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T11:03:00Z","timestamp":1770462180681,"version":"3.49.0"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,10,2]],"date-time":"2021-10-02T00:00:00Z","timestamp":1633132800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,10,2]],"date-time":"2021-10-02T00:00:00Z","timestamp":1633132800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cheminform"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Confidence predictors can deliver predictions with the associated confidence required for decision making and can play an important role in drug discovery and toxicity predictions. In this work we investigate a recently introduced version of conformal prediction, synergy conformal prediction, focusing on the predictive performance when applied to bioactivity data. We compare the performance to other variants of conformal predictors for multiple partitioned datasets and demonstrate the utility of synergy conformal predictors for federated learning where data cannot be pooled in one location. Our results show that synergy conformal predictors based on training data randomly sampled with replacement can compete with other conformal setups, while using completely separate training sets often results in worse performance. However, in a federated setup where no method has access to all the data, synergy conformal prediction is shown to give promising results. Based on our study, we conclude that synergy conformal predictors are a valuable addition to the conformal prediction toolbox.<\/jats:p>","DOI":"10.1186\/s13321-021-00555-7","type":"journal-article","created":{"date-parts":[[2021,10,4]],"date-time":"2021-10-04T13:42:51Z","timestamp":1633354971000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Synergy conformal prediction applied to large-scale bioactivity datasets and in federated learning"],"prefix":"10.1186","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3107-331X","authenticated-orcid":false,"given":"Ulf","family":"Norinder","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8083-2864","authenticated-orcid":false,"given":"Ola","family":"Spjuth","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5556-8133","authenticated-orcid":false,"given":"Fredrik","family":"Svensson","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,10,2]]},"reference":[{"key":"555_CR1","first-page":"1","volume-title":"Algorithmic learning in a random world","author":"V Vovk","year":"2005","unstructured":"Vovk V, Gammerman A, Shafer G (2005) Algorithmic learning in a random world. Springer, New York, pp 1\u2013324"},{"key":"555_CR2","doi-asserted-by":"crossref","unstructured":"Cort\u00e9s-Ciriano I, Bender A (2021) Concepts and applications of conformal prediction in computational drug discovery. In: Artificial intelligence in drug discovery, the royal society of chemistry, pp 63\u2013101","DOI":"10.1039\/9781788016841-00063"},{"key":"555_CR3","doi-asserted-by":"publisher","DOI":"10.1016\/j.drudis.2020.11.027","author":"LH Mervin","year":"2020","unstructured":"Mervin LH, Johansson S, Semenova E et al (2020) Uncertainty quantification in drug design. Drug Discov Today. https:\/\/doi.org\/10.1016\/j.drudis.2020.11.027","journal-title":"Drug Discov Today"},{"key":"555_CR4","doi-asserted-by":"publisher","first-page":"9097","DOI":"10.1021\/acs.jmedchem.7b00487","volume":"60","author":"F Lombardo","year":"2017","unstructured":"Lombardo F, Desai PV, Arimoto R et al (2017) In Silico absorption, distribution, metabolism, excretion, and pharmacokinetics (ADME\u2013PK): utility and best practices. an industry perspective from the International Consortium for innovation through quality in pharmaceutical development. J Med Chem 60:9097\u20139113. https:\/\/doi.org\/10.1021\/acs.jmedchem.7b00487","journal-title":"J Med Chem"},{"key":"555_CR5","doi-asserted-by":"publisher","first-page":"348","DOI":"10.1007\/978-3-642-41142-7_36","volume-title":"BT\u2013artificial intelligence applications and innovations","author":"V Vovk","year":"2013","unstructured":"Vovk V (2013) Transductive conformal predictors . In: Papadopoulos H, Andreou AS, Iliadis L, Maglogiannis I (eds) BT\u2013artificial intelligence applications and innovations. Springer, Berlin, pp 348\u2013360"},{"key":"555_CR6","volume-title":"Tools in artificial intelligence","author":"H Papadopoulos","year":"2008","unstructured":"Papadopoulos H (2008) Inductive conformal prediction: theory and application to Neural networks. In: Fritzsche P (ed) Tools in artificial intelligence. InTech, London"},{"key":"555_CR7","doi-asserted-by":"publisher","first-page":"1596","DOI":"10.1021\/ci5001168","volume":"54","author":"U Norinder","year":"2014","unstructured":"Norinder U, Carlsson L, Boyer S, Eklund M (2014) Introducing conformal prediction in predictive modeling. a transparent and flexible alternative to applicability domain determination. J Chem Inf Model 54:1596\u20131603. https:\/\/doi.org\/10.1021\/ci5001168","journal-title":"J Chem Inf Model"},{"key":"555_CR8","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.xphs.2020.09.055","volume":"110","author":"J Alvarsson","year":"2021","unstructured":"Alvarsson J, Arvidsson McShane S, Norinder U, Spjuth O (2021) Predicting with confidence: using conformal prediction in drug discovery. J Pharm Sci 110:42\u201349. https:\/\/doi.org\/10.1016\/j.xphs.2020.09.055","journal-title":"J Pharm Sci"},{"key":"555_CR9","doi-asserted-by":"publisher","first-page":"1355","DOI":"10.3233\/IDA-150786","volume":"19","author":"T L\u00f6fstr\u00f6m","year":"2015","unstructured":"L\u00f6fstr\u00f6m T, Bostr\u00f6m H, Linusson H, Johansson U (2015) Bias reduction through conditional conformal prediction. Intell Data Anal 19:1355\u20131375","journal-title":"Intell Data Anal"},{"key":"555_CR10","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1016\/j.jmgm.2017.01.008","volume":"72","author":"U Norinder","year":"2017","unstructured":"Norinder U, Boyer S (2017) Binary classification of imbalanced datasets using conformal prediction. J Mol Graph Model 72:256\u2013265. https:\/\/doi.org\/10.1016\/j.jmgm.2017.01.008","journal-title":"J Mol Graph Model"},{"key":"555_CR11","volume-title":"Artificial intelligence applications and innovations: AIAI 2014 workshops: CoPA, MHDW, IIVC, and MT4BD, Rhodes, Greece, September 19\u201321, 2014. Proceedings","author":"L Carlsson","year":"2014","unstructured":"Carlsson L, Eklund M, Norinder U (2014) Aggregated conformal prediction. In: Iliadis L, Maglogiannis I, Papadopoulos H et al (eds) Artificial intelligence applications and innovations: AIAI 2014 workshops: CoPA, MHDW, IIVC, and MT4BD, Rhodes, Greece, September 19\u201321, 2014. Proceedings. Springer International Publishing, Berlin"},{"key":"555_CR12","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1007\/s10472-013-9368-4","volume":"74","author":"V Vovk","year":"2015","unstructured":"Vovk V (2015) Cross-conformal predictors. Ann Math Artif Intell 74:9\u201328. https:\/\/doi.org\/10.1007\/s10472-013-9368-4","journal-title":"Ann Math Artif Intell"},{"key":"555_CR13","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1039\/C6TX00252H","volume":"6","author":"F Svensson","year":"2017","unstructured":"Svensson F, Norinder U, Bender A (2017) Modelling compound cytotoxicity using conformal prediction and PubChem HTS data. Toxicol Res (Camb) 6:73\u201380. https:\/\/doi.org\/10.1039\/C6TX00252H","journal-title":"Toxicol Res (Camb)"},{"key":"555_CR14","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1186\/s13321-018-0325-4","volume":"11","author":"N Bosc","year":"2019","unstructured":"Bosc N, Atkinson F, Felix E et al (2019) Large scale comparison of QSAR and conformal prediction methods and their applications in drug discovery. J Cheminform 11:4. https:\/\/doi.org\/10.1186\/s13321-018-0325-4","journal-title":"J Cheminform"},{"key":"555_CR15","unstructured":"Gauraha N, Spjuth O Synergy Conformal Prediction. Department of Pharmaceutical Biosciences, Faculty of Pharmacy, Disciplinary Domain of Medicine and Pharmacy, Uppsala University"},{"key":"555_CR16","doi-asserted-by":"crossref","unstructured":"Gauraha N, Spjuth O (2021) Synergy conformal prediction for regression. In: Proceedings of the 10th International conference on pattern recognition applications and methods - Volume 1, ICPRAM, SciTePress, pp 212\u2013221","DOI":"10.5220\/0010229402120221"},{"key":"555_CR17","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1186\/s13321-021-00511-5","volume":"13","author":"A Morger","year":"2021","unstructured":"Morger A, Svensson F, Arvidsson McShane S et al (2021) Assessing the calibration in toxicological in vitro models with conformal prediction. J Cheminform 13:35. https:\/\/doi.org\/10.1186\/s13321-021-00511-5","journal-title":"J Cheminform"},{"key":"555_CR18","doi-asserted-by":"publisher","first-page":"12598","DOI":"10.1038\/s41598-020-69250-1","volume":"10","author":"MJ Sheller","year":"2020","unstructured":"Sheller MJ, Edwards B, Reina GA et al (2020) Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci Rep 10:12598. https:\/\/doi.org\/10.1038\/s41598-020-69250-1","journal-title":"Sci Rep"},{"key":"555_CR19","doi-asserted-by":"crossref","unstructured":"Kairouz P, McMahan HB, Avent B, et al (2021) Advances and Open Problems in Federated Learning. ArXiv. https:\/\/arxiv.org\/abs\/1912.04977","DOI":"10.1561\/9781680837896"},{"key":"555_CR20","volume-title":"Proceedings of the eighth symposium on conformal and probabilistic prediction and applications","author":"O Spjuth","year":"2019","unstructured":"Spjuth O, Br\u00e4nnstr\u00f6m RC, Carlsson L, Gauraha N (2019) Combining prediction intervals on multi-source non-disclosed regression datasets. In: Gammerman A, Vovk V, Luo Z, Smirnov E (eds) Proceedings of the eighth symposium on conformal and probabilistic prediction and applications. Proc Mach Learn Res, Bulgaria"},{"key":"555_CR21","doi-asserted-by":"publisher","DOI":"10.1186\/s13321-018-0260-4","author":"F Svensson","year":"2018","unstructured":"Svensson F, Afzal AM, Norinder U, Bender A (2018) Maximizing gain in high-throughput screening using conformal prediction. J Cheminform. https:\/\/doi.org\/10.1186\/s13321-018-0260-4","journal-title":"J Cheminform"},{"key":"555_CR22","doi-asserted-by":"publisher","first-page":"1598","DOI":"10.1021\/acs.jcim.9b00027","volume":"59","author":"U Norinder","year":"2019","unstructured":"Norinder U, Svensson F (2019) Multitask modeling with confidence using matrix factorization and conformal prediction. J Chem Inf Model 59:1598\u20131604. https:\/\/doi.org\/10.1021\/acs.jcim.9b00027","journal-title":"J Chem Inf Model"},{"key":"555_CR23","doi-asserted-by":"publisher","first-page":"D400","DOI":"10.1093\/nar\/gkr1132","volume":"40","author":"Y Wang","year":"2012","unstructured":"Wang Y, Xiao J, Suzek TO et al (2012) PubChem\u2019s bioassay database. Nucleic Acids Res 40:D400\u2013D412. https:\/\/doi.org\/10.1093\/nar\/gkr1132","journal-title":"Nucleic Acids Res"},{"key":"555_CR24","unstructured":"IMI eTOX project standardizer, version 017. https:\/\/pypi.python.org\/pypi\/standardiser"},{"key":"555_CR25","unstructured":"MolVS standardizer, version 009. https:\/\/pypi.python.org\/pypi\/MolVS"},{"key":"555_CR26","doi-asserted-by":"publisher","first-page":"439","DOI":"10.1021\/acs.jcim.6b00532","volume":"57","author":"F Svensson","year":"2017","unstructured":"Svensson F, Norinder U, Bender A (2017) Improving screening efficiency through iterative screening using docking and conformal prediction. J Chem Inf Model 57:439\u2013444. https:\/\/doi.org\/10.1021\/acs.jcim.6b00532","journal-title":"J Chem Inf Model"},{"key":"555_CR27","unstructured":"RDKit: Open-source cheminformatics, version 2018.09.1.0. http:\/\/www.rdkit.org"},{"key":"555_CR28","doi-asserted-by":"publisher","first-page":"742","DOI":"10.1021\/ci100050t","volume":"50","author":"D Rogers","year":"2010","unstructured":"Rogers D, Hahn M (2010) Extended-connectivity fingerprints. J Chem Inf Model 50:742\u2013754. https:\/\/doi.org\/10.1021\/ci100050t","journal-title":"J Chem Inf Model"},{"key":"555_CR29","doi-asserted-by":"publisher","first-page":"2825","DOI":"10.1007\/s13398-014-0173-7.2","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825\u20132830. https:\/\/doi.org\/10.1007\/s13398-014-0173-7.2","journal-title":"J Mach Learn Res"},{"key":"555_CR30","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1186\/s13321-018-0304-9","volume":"10","author":"A Kensert","year":"2018","unstructured":"Kensert A, Alvarsson J, Norinder U, Spjuth O (2018) Evaluating parameters for ligand-based modeling with random forest on sparse data sets. J Cheminform 10:49. https:\/\/doi.org\/10.1186\/s13321-018-0304-9","journal-title":"J Cheminform"},{"key":"555_CR31","doi-asserted-by":"publisher","first-page":"1132","DOI":"10.1021\/acs.jcim.8b00054","volume":"58","author":"F Svensson","year":"2018","unstructured":"Svensson F, Aniceto N, Norinder U et al (2018) Conformal regression for quantitative structure-activity relationship modelling\u2013quantifying prediction uncertainty. J Chem Inf Model 58:1132\u20131140. https:\/\/doi.org\/10.1021\/acs.jcim.8b00054","journal-title":"J Chem Inf Model"}],"container-title":["Journal of Cheminformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-021-00555-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13321-021-00555-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-021-00555-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,11]],"date-time":"2023-01-11T01:01:01Z","timestamp":1673398861000},"score":1,"resource":{"primary":{"URL":"https:\/\/jcheminf.biomedcentral.com\/articles\/10.1186\/s13321-021-00555-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,2]]},"references-count":31,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["555"],"URL":"https:\/\/doi.org\/10.1186\/s13321-021-00555-7","relation":{},"ISSN":["1758-2946"],"issn-type":[{"value":"1758-2946","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,2]]},"assertion":[{"value":"6 May 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 September 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 October 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"OS is co-founder of Scaleout Systems AB, a Swedish company involved in federated learning.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"77"}}