{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T15:49:43Z","timestamp":1778168983857,"version":"3.51.4"},"reference-count":49,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T00:00:00Z","timestamp":1619654400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T00:00:00Z","timestamp":1619654400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"HaVo-Stiftung"},{"name":"FUBright Mobility Allowances"},{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["031A262C"],"award-info":[{"award-number":["031A262C"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002283","name":"Alzheimer\u2019s Research UK","doi-asserted-by":"crossref","award":["560832"],"award-info":[{"award-number":["560832"]}],"id":[{"id":"10.13039\/501100002283","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001862","name":"Svenska Forskningsr\u00e5det Formas","doi-asserted-by":"publisher","award":["2018-00924"],"award-info":[{"award-number":["2018-00924"]}],"id":[{"id":"10.13039\/501100001862","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004359","name":"Vetenskapsr\u00e5det","doi-asserted-by":"publisher","award":["2020-03731"],"award-info":[{"award-number":["2020-03731"]}],"id":[{"id":"10.13039\/501100004359","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004359","name":"Vetenskapsr\u00e5det","doi-asserted-by":"publisher","award":["2020-01865"],"award-info":[{"award-number":["2020-01865"]}],"id":[{"id":"10.13039\/501100004359","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001729","name":"Stiftelsen f\u00f6r Strategisk Forskning","doi-asserted-by":"publisher","award":["grant BD150008"],"award-info":[{"award-number":["grant BD150008"]}],"id":[{"id":"10.13039\/501100001729","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Charit\u00e9"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cheminform"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Machine learning methods are widely used in drug discovery and toxicity prediction. While showing overall good performance in cross-validation studies, their predictive power (often) drops in cases where the query samples have drifted from the training data\u2019s descriptor space. Thus, the assumption for applying machine learning algorithms, that training and test data stem from the same distribution, might not always be fulfilled. In this work, conformal prediction is used to assess the calibration of the models. Deviations from the expected error may indicate that training and test data originate from different distributions. Exemplified on the Tox21 datasets, composed of chronologically released Tox21Train, Tox21Test and Tox21Score subsets, we observed that while internally valid models could be trained using cross-validation on Tox21Train, predictions on the external Tox21Score data resulted in higher error rates than expected. To improve the prediction on the external sets, a strategy exchanging the calibration set with more recent data, such as Tox21Test, has successfully been introduced. We conclude that conformal prediction can be used to diagnose data drifts and other issues related to model calibration. The proposed improvement strategy\u2014exchanging the calibration data only\u2014is convenient as it does not require retraining of the underlying model.<\/jats:p>","DOI":"10.1186\/s13321-021-00511-5","type":"journal-article","created":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T09:03:19Z","timestamp":1619686999000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Assessing the calibration in toxicological in vitro models with conformal prediction"],"prefix":"10.1186","volume":"13","author":[{"given":"Andrea","family":"Morger","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fredrik","family":"Svensson","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Staffan","family":"Arvidsson McShane","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Niharika","family":"Gauraha","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ulf","family":"Norinder","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ola","family":"Spjuth","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3760-580X","authenticated-orcid":false,"given":"Andrea","family":"Volkamer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,4,29]]},"reference":[{"key":"511_CR1","doi-asserted-by":"publisher","first-page":"30","DOI":"10.3389\/fchem.2018.00030","volume":"6","author":"H Yang","year":"2018","unstructured":"Yang H, Sun L, Li W, Liu G, Tang Y (2018) in silico prediction of chemical toxicity for drug design using machine learning methods and structural alerts. Front Chem 6:30. https:\/\/doi.org\/10.3389\/fchem.2018.00030","journal-title":"Front Chem"},{"issue":"3","key":"511_CR2","doi-asserted-by":"publisher","first-page":"945","DOI":"10.1021\/acs.jcim.9b00136","volume":"59","author":"G Klambauer","year":"2019","unstructured":"Klambauer G, Hochreiter S, Rarey M (2019) Machine learning in drug discovery. J Chem Inf Model 59(3):945\u2013946. https:\/\/doi.org\/10.1021\/acs.jcim.9b00136","journal-title":"J Chem Inf Model"},{"issue":"1","key":"511_CR3","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1021\/acs.chemrestox.9b00227","volume":"33","author":"AH Vo","year":"2020","unstructured":"Vo AH, Van Vleet TR, Gupta RR, Liguori MJ, Rao MS (2020) An overview of machine learning and big data for drug toxicity evaluation. Chem Res Toxicol 33(1):20\u201337. https:\/\/doi.org\/10.1021\/acs.chemrestox.9b00227","journal-title":"Chem Res Toxicol"},{"key":"511_CR4","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1007\/978-3-642-35374-1_36","volume-title":"Regulatory toxicology","author":"T Steger-Hartmann","year":"2020","unstructured":"Steger-Hartmann T, Boyer S (2020) Computer-based prediction models in regulatory toxicology. Regulatory toxicology. Springer, Berlin, pp 123\u2013131. https:\/\/doi.org\/10.1007\/978-3-642-35374-1_36"},{"issue":"8","key":"511_CR5","doi-asserted-by":"publisher","first-page":"1225","DOI":"10.1021\/acs.chemrestox.6b00135","volume":"29","author":"AM Richard","year":"2016","unstructured":"Richard AM, Judson RS, Houck KA, Grulke CM, Volarath P, Thillainadarajah I, Yang C, Rathman J, Martin MT, Wambaugh JF, Knudsen TB, Kancherla J, Mansouri K, Patlewicz G, Williams AJ, Little SB, Crofton KM, Thomas RS (2016) ToxCast chemical landscape: paving the road to 21st century toxicology. Chem Res Toxicol 29(8):1225\u20131251. https:\/\/doi.org\/10.1021\/acs.chemrestox.6b00135","journal-title":"Chem Res Toxicol"},{"key":"511_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/srep05664","volume":"4","author":"R Huang","year":"2014","unstructured":"Huang R, Sakamuru S, Martin MT, Reif DM, Judson RS, Houck KA, Casey W, Hsieh JH, Shockley KR, Ceger P, Fostel J, Witt KL, Tong W, Rotroff DM, Zhao T, Shinn P, Simeonov A, Dix DJ, Austin CP, Kavlock RJ, Tice RR, Xia M (2014) Profiling of the Tox21 10K compound library for agonists and antagonists of the estrogen receptor alpha signaling pathway. Sci Rep 4:1\u20139. https:\/\/doi.org\/10.1038\/srep05664","journal-title":"Sci Rep"},{"key":"511_CR7","doi-asserted-by":"publisher","DOI":"10.1021\/acs.chemrestox.0c00264","author":"AM Richard","year":"2020","unstructured":"Richard AM, Huang R, Waidyanatha S, Shinn P, Collins B, Thillainadarajah I, Grulke CM, Williams AJ, Lougee RR, Judson RS, Houck KA, Shobair M, Yang C, Rathman JF, Yasgar A, Fitzpatrick SC, Simeonov A, Thomas RS, Crofton KM, Paules RS, Bucher JR, Austin CP, Kavlock RJ, Tice RR (2020) The Tox21 10K compound library: collaborative chemistry advancing toxicology. Chem Res Toxicol. https:\/\/doi.org\/10.1021\/acs.chemrestox.0c00264","journal-title":"Chem Res Toxicol"},{"issue":"11","key":"511_CR8","doi-asserted-by":"publisher","first-page":"21,136","DOI":"10.3390\/ijms151121136","volume":"15","author":"M Cases","year":"2014","unstructured":"Cases M, Briggs K, Steger-Hartmann T, Pognan F, Marc P, Klein\u00f6der T, Schwab CH, Pastor M, Wichard J, Sanz F (2014) The eTOX data-sharing project to advance in Silico drug-induced toxicity prediction. Int J Mol Sci 15(11):21,136-21,154. https:\/\/doi.org\/10.3390\/ijms151121136","journal-title":"Int J Mol Sci"},{"key":"511_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fphar.2018.01147","volume":"9","author":"M Pastor","year":"2018","unstructured":"Pastor M, Quintana J, Sanz F (2018) Development of an infrastructure for the prediction of biological endpoints in industrial environments. Lessons learned at the eTOX project. Front Pharmacol 9:1\u20138. https:\/\/doi.org\/10.3389\/fphar.2018.01147","journal-title":"Front Pharmacol"},{"issue":"17","key":"511_CR10","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1023\/A:1025386326946","volume":"2003","author":"A Golbraikh","year":"2003","unstructured":"Golbraikh A, Shen M, Xiao Z, Xiao YD, Lee KH, Tropsha A (2003) Rational selection of training and test sets for the development of validated QSAR models. J Comput Aided Mol Des 2003(17):241\u201353. https:\/\/doi.org\/10.1023\/A:1025386326946","journal-title":"J Comput Aided Mol Des"},{"issue":"10","key":"511_CR11","doi-asserted-by":"publisher","first-page":"2570","DOI":"10.1021\/ci300338w","volume":"52","author":"TM Martin","year":"2012","unstructured":"Martin TM, Harten P, Young DM, Muratov EN, Golbraikh A, Zhu H, Tropsha A (2012) Does rational selection of training and test sets improve the outcome of QSAR modeling? J Chem Inf Model 52(10):2570\u20132578. https:\/\/doi.org\/10.1021\/ci300338w","journal-title":"J Chem Inf Model"},{"issue":"3","key":"511_CR12","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1002\/qsar.200510161","volume":"25","author":"JT Leonard","year":"2006","unstructured":"Leonard JT, Roy K (2006) On selection of training and test sets for the development of predictive QSAR models. QSAR Combinat Sci 25(3):235\u2013251. https:\/\/doi.org\/10.1002\/qsar.200510161","journal-title":"QSAR Combinat Sci"},{"issue":"12","key":"511_CR13","doi-asserted-by":"publisher","first-page":"1011","DOI":"10.1080\/1062936X.2017.1397056","volume":"28","author":"MF Andrada","year":"2017","unstructured":"Andrada MF, Vega-Hissi EG, Estrada MR, Garro Martinez JC (2017) Impact assessment of the rational selection of training and test sets on the predictive ability of QSAR models. SAR QSAR Environ Res 28(12):1011\u20131023. https:\/\/doi.org\/10.1080\/1062936X.2017.1397056","journal-title":"SAR QSAR Environ Res"},{"issue":"3","key":"511_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0213848","volume":"14","author":"K Klimenko","year":"2019","unstructured":"Klimenko K, Rosenberg SA, Dybdahl M, Wedebye EB, Nikolov NG (2019) QSAR modelling of a large imbalanced aryl hydrocarbon activation dataset by rational and random sampling and screening of 80,086 REACH pre-registered and\/or registered substances. PLoS ONE 14(3):1\u201321. https:\/\/doi.org\/10.1371\/journal.pone.0213848","journal-title":"PLoS ONE"},{"issue":"5","key":"511_CR15","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1002\/minf.201501019","volume":"35","author":"M Mathea","year":"2016","unstructured":"Mathea M, Klingspohn W, Baumann K (2016) Chemoinformatic classification methods and their applicability domain. Mol Inf 35(5):160\u2013180. https:\/\/doi.org\/10.1002\/minf.201501019","journal-title":"Mol Inf"},{"issue":"11","key":"511_CR16","doi-asserted-by":"publisher","first-page":"865","DOI":"10.1080\/1062936X.2016.1250229","volume":"27","author":"T Hanser","year":"2016","unstructured":"Hanser T, Barber C, Marchaland JF, Werner S (2016) Applicability domain: towards a more formal definition. SAR QSAR Environ Res 27(11):865\u2013881. https:\/\/doi.org\/10.1080\/1062936X.2016.1250229","journal-title":"SAR QSAR Environ Res"},{"issue":"3","key":"511_CR17","doi-asserted-by":"publisher","first-page":"504","DOI":"10.1016\/j.datak.2008.08.001","volume":"67","author":"Z Bosni\u0107","year":"2008","unstructured":"Bosni\u0107 Z, Kononenko I (2008) Comparison of approaches for estimating reliability of individual regression predictions. Data Knowl Eng 67(3):504\u2013516. https:\/\/doi.org\/10.1016\/j.datak.2008.08.001","journal-title":"Data Knowl Eng"},{"issue":"1","key":"511_CR18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13321-016-0182-y","volume":"8","author":"N Aniceto","year":"2016","unstructured":"Aniceto N, Freitas AA, Bender A, Ghafourian T (2016) A novel applicability domain technique for mapping predictive reliability across the chemical space of a QSAR: Reliability-density neighbourhood. J Cheminf 8(1):1\u201320. https:\/\/doi.org\/10.1186\/s13321-016-0182-y","journal-title":"J Cheminf"},{"issue":"7","key":"511_CR19","doi-asserted-by":"publisher","first-page":"1762","DOI":"10.1021\/ci9000579","volume":"49","author":"H Dragos","year":"2009","unstructured":"Dragos H, Gilles M, Alexandre V (2009) Predicting the predictability: a unified approach to the applicability domain problem of QSAR models. J Chem Inf Model 49(7):1762\u20131776. https:\/\/doi.org\/10.1021\/ci9000579","journal-title":"J Chem Inf Model"},{"issue":"6","key":"511_CR20","doi-asserted-by":"publisher","first-page":"1098","DOI":"10.1021\/acs.jcim.5b00110","volume":"55","author":"RP Sheridan","year":"2015","unstructured":"Sheridan RP (2015) The relative importance of domain applicability metrics for estimating prediction errors in QSAR varies with training set diversity. J Chem Inf Model 55(6):1098\u20131107. https:\/\/doi.org\/10.1021\/acs.jcim.5b00110","journal-title":"J Chem Inf Model"},{"key":"511_CR21","doi-asserted-by":"publisher","first-page":"526","DOI":"10.1016\/j.fct.2017.04.008","volume":"112","author":"VM Alves","year":"2018","unstructured":"Alves VM, Muratov EN, Zakharov A, Muratov NN, Andrade CH, Tropsha A (2018) Chemical toxicity prediction for major classes of industrial chemicals: Is it possible to develop universal models covering cosmetics, drugs, and pesticides? Food Chem Toxicol 112:526\u2013534. https:\/\/doi.org\/10.1016\/j.fct.2017.04.008","journal-title":"Food Chem Toxicol"},{"issue":"1","key":"511_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13321-020-00444-5","volume":"12","author":"I Cort\u00e9s-Ciriano","year":"2020","unstructured":"Cort\u00e9s-Ciriano I, \u0160kuta C, Bender A, Svozil D (2020) QSAR-derived affinity fingerprints (part 2): modeling performance for potency prediction. J Cheminf 12(1):1\u201317. https:\/\/doi.org\/10.1186\/s13321-020-00444-5","journal-title":"J Cheminf"},{"issue":"2\u20133","key":"511_CR23","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1007\/s10994-013-5355-6","volume":"92","author":"V Vovk","year":"2013","unstructured":"Vovk V (2013) Conditional validity of inductive conformal predictors. Mach Learn 92(2\u20133):349\u2013376. https:\/\/doi.org\/10.1007\/s10994-013-5355-6","journal-title":"Mach Learn"},{"key":"511_CR24","volume-title":"Algorithmic learning in a random world","author":"V Vovk","year":"2005","unstructured":"Vovk V, Gammerman A, Shafer G (2005) Algorithmic learning in a random world. Springer Science & Business Media, Berlin"},{"key":"511_CR25","unstructured":"Fedorova V, Gammerman A, Nouretdinov I, Vovk V (2012) Plug-in martingales for testing exchangeability on-line. In: Proceedings of the 29th international conference on machine learning, ICML 2012 2. pp 1639\u20131646"},{"issue":"1","key":"511_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13321-020-00422-x","volume":"12","author":"A Morger","year":"2020","unstructured":"Morger A, Mathea M, Achenbach JH, Wolf A, Buesen R, Schleifer KJ, Landsiedel R, Volkamer A (2020) KnowTox: pipeline and case study for confident prediction of potential toxic effects of compounds in early phases of development. J Cheminf 12(1):1\u201317. https:\/\/doi.org\/10.1186\/s13321-020-00422-x","journal-title":"J Cheminf"},{"issue":"3","key":"511_CR27","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1038\/ncomms","volume":"5","author":"Ruili Huang","year":"2017","unstructured":"Huang Ruili, Xia Menghang (2017) Editorial: Tox21 challenge to build predictive models of nuclear receptor and stress response pathways as mediated by exposure to environmental toxicants and drugs. Front Environ Sci 5(3):5. https:\/\/doi.org\/10.1038\/ncomms","journal-title":"Front Environ Sci"},{"key":"511_CR28","unstructured":"NIH\u2014National Center for Advancing Translational Sciences (2014) Tox21 data challenge. https:\/\/tripod.nih.gov\/tox21\/challenge\/data.jsp"},{"key":"511_CR29","unstructured":"Atkinson FCGEE (2014) Standardiser. https:\/\/github.com\/flatkinson\/standardiser"},{"key":"511_CR30","doi-asserted-by":"publisher","DOI":"10.1186\/s13321-015-0068-4","author":"SR Heller","year":"2015","unstructured":"Heller SR, McNaught A, Pletnev I, Stein S, Tchekhovskoi D (2015) InChI, the IUPAC international chemical identifier. J Cheminf. https:\/\/doi.org\/10.1186\/s13321-015-0068-4","journal-title":"J Cheminf"},{"key":"511_CR31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fenvs.2015.00085","volume":"3","author":"R Huang","year":"2016","unstructured":"Huang R, Xia M, Nguyen DT, Zhao T, Sakamuru S, Zhao J, Shahane SA, Rossoshek A, Simeonov A (2016) Tox21 challenge to build predictive models of nuclear receptor and stress response pathways as mediated by exposure to environmental chemicals and drugs. Front Environ Sci 3:1\u20139. https:\/\/doi.org\/10.3389\/fenvs.2015.00085","journal-title":"Front Environ Sci"},{"issue":"3","key":"511_CR32","doi-asserted-by":"publisher","first-page":"707","DOI":"10.1021\/ci020345w","volume":"43","author":"JL Faulon","year":"2003","unstructured":"Faulon JL, Visco DP, Pophale RS (2003) The signature molecular descriptor. 1. Using extended valence sequences in QSAR and QSPR studies. J Chem Inf Comput Sci 43(3):707\u2013720. https:\/\/doi.org\/10.1021\/ci020345w","journal-title":"J Chem Inf Comput Sci"},{"issue":"2","key":"511_CR33","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1021\/ci0341823","volume":"44","author":"JL Faulon","year":"2004","unstructured":"Faulon JL, Collins MJ, Carr RD (2004) The signature molecular descriptor. 4. Canonizing molecules using extended valence sequences. J Chem Inf Comput Sci 44(2):427\u2013436. https:\/\/doi.org\/10.1021\/ci0341823","journal-title":"J Chem Inf Comput Sci"},{"key":"511_CR34","unstructured":"Aros Bio (2020) CPSign. https:\/\/arosbio.com\/cpsign\/"},{"issue":"1","key":"511_CR35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13321-018-0304-9","volume":"10","author":"A Kensert","year":"2018","unstructured":"Kensert A, Alvarsson J, Norinder U, Spjuth O (2018) Evaluating parameters for ligand-based modeling with random forest on sparse data sets. J Cheminf 10(1):1\u201310. https:\/\/doi.org\/10.1186\/s13321-018-0304-9","journal-title":"J Cheminf"},{"issue":"10","key":"511_CR36","doi-asserted-by":"publisher","first-page":"2647","DOI":"10.1021\/ci500361u","volume":"54","author":"J Alvarsson","year":"2014","unstructured":"Alvarsson J, Eklund M, Engkvist O, Spjuth O, Carlsson L, Wikberg JE, Noeske T (2014a) Ligand-based target prediction with signature fingerprints. J Chem Inf Model 54(10):2647\u20132653. https:\/\/doi.org\/10.1021\/ci500361u","journal-title":"J Chem Inf Model"},{"issue":"11","key":"511_CR37","doi-asserted-by":"publisher","first-page":"3211","DOI":"10.1021\/ci500344v","volume":"54","author":"J Alvarsson","year":"2014","unstructured":"Alvarsson J, Eklund M, Andersson C, Carlsson L, Spjuth O, Wikberg JE (2014b) Benchmarking study of parameter variation when using signature fingerprints together with support vector machines. J Chem Inf Model 54(11):3211\u20133217. https:\/\/doi.org\/10.1021\/ci500344v","journal-title":"J Chem Inf Model"},{"issue":"7","key":"511_CR38","doi-asserted-by":"publisher","first-page":"1591","DOI":"10.1021\/acs.jcim.7b00159","volume":"57","author":"J Sun","year":"2017","unstructured":"Sun J, Carlsson L, Ahlberg E, Norinder U, Engkvist O, Chen H (2017) Applying Mondrian cross-conformal prediction to estimate prediction confidence on large imbalanced bioactivity data sets. J Chem Inf Model 57(7):1591\u20131598. https:\/\/doi.org\/10.1021\/acs.jcim.7b00159","journal-title":"J Chem Inf Model"},{"issue":"3","key":"511_CR39","doi-asserted-by":"publisher","first-page":"489","DOI":"10.1007\/s10994-018-5754-9","volume":"108","author":"P Toccaceli","year":"2019","unstructured":"Toccaceli P, Gammerman A (2019) Combination of inductive Mondrian conformal predictors. Mach Learn 108(3):489\u2013510. https:\/\/doi.org\/10.1007\/s10994-018-5754-9","journal-title":"Mach Learn"},{"issue":"1","key":"511_CR40","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.xphs.2020.09.055","volume":"110","author":"J Alvarsson","year":"2021","unstructured":"Alvarsson J, Arvidsson McShane S, Norinder U, Spjuth O (2021) Predicting with confidence: using conformal prediction in drug discovery. J Pharm Sci 110(1):42\u201349. https:\/\/doi.org\/10.1016\/j.xphs.2020.09.055","journal-title":"J Pharm Sci"},{"issue":"6","key":"511_CR41","doi-asserted-by":"publisher","first-page":"1596","DOI":"10.1021\/ci5001168","volume":"54","author":"U Norinder","year":"2014","unstructured":"Norinder U, Carlsson L, Boyer S, Eklund M (2014) Introducing conformal prediction in predictive modeling. A transparent and flexible alternative to applicability domain determination. J Chem Inf Model 54(6):1596\u20131603. https:\/\/doi.org\/10.1021\/ci5001168","journal-title":"J Chem Inf Model"},{"key":"511_CR42","unstructured":"Linusson H, Norinder U, Bostr\u00f6m H, Johansson U, L\u00f6fstr\u00f6m T (2017) On the calibration of aggregated conformal predictors. In: Proceedings of the sixth workshop on conformal and probabilistic prediction and applications 60. pp 154\u2013173"},{"key":"511_CR43","doi-asserted-by":"crossref","unstructured":"Carlsson L, Eklund M, Norinder U, Carlsson L, Eklund M, Norinder U, Conformal A, Lazaros P, Maglogiannis I, Papadopoulos H, Sioutas S, Ifip CM (2014) Aggregated conformal prediction. In: IFIP advances in information and communication technology. pp 231\u2013240","DOI":"10.1007\/978-3-662-44722-2_25"},{"key":"511_CR44","unstructured":"Gauraha N, Spjuth O (2018) Synergy conformal prediction. DiVA preprint 360504. http:\/\/urn.kb.se\/resolve?urn=urn:nbn:se:uu:diva-360504"},{"key":"511_CR45","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825\u20132830","journal-title":"J Mach Learn Res"},{"key":"511_CR46","unstructured":"Linusson H (2015) Nonconformist. http:\/\/donlnz.github.io\/nonconformist\/"},{"key":"511_CR47","unstructured":"Gauraha N (2018) Synergy conformal prediction code. https:\/\/github.com\/niha21\/scp_code"},{"issue":"15\u201316","key":"511_CR48","doi-asserted-by":"publisher","first-page":"716","DOI":"10.1016\/j.drudis.2013.05.015.","volume":"18","author":"MS Attene-Ramos","year":"2013","unstructured":"Attene-Ramos MS, Miller N, Huang R, Michael S, Itkin M, Kavlock RJ, Austin CP, Shinn P, Simeonov A, Tice RR, Xia M (2013) The Tox21 robotic platform for the assessment of environmental chemicals\u2014from vision to reality. Drug Discov Today 18(15\u201316):716\u2013723. https:\/\/doi.org\/10.1016\/j.drudis.2013.05.015.","journal-title":"Drug Discov Today"},{"key":"511_CR49","doi-asserted-by":"publisher","DOI":"10.1208\/s12248-019-0354-6","author":"M Azadeh","year":"2019","unstructured":"Azadeh M, Sondag P, Wang Y, Raines M, Sailstad J (2019) Quality controls in ligand binding assays: recommendations and best practices for preparation, qualification, maintenance of lot to lot consistency, and prevention of assay drift. AAPS J. https:\/\/doi.org\/10.1208\/s12248-019-0354-6","journal-title":"AAPS J"}],"container-title":["Journal of Cheminformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-021-00511-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13321-021-00511-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-021-00511-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T09:13:59Z","timestamp":1619687639000},"score":1,"resource":{"primary":{"URL":"https:\/\/jcheminf.biomedcentral.com\/articles\/10.1186\/s13321-021-00511-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,29]]},"references-count":49,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["511"],"URL":"https:\/\/doi.org\/10.1186\/s13321-021-00511-5","relation":{},"ISSN":["1758-2946"],"issn-type":[{"value":"1758-2946","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,29]]},"assertion":[{"value":"7 February 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 April 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 April 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"OS declares ownership of Aros Bio AB, a company developing the CPSign software. SAMS declares contributions to the CPSign codebase.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"35"}}