{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T01:51:06Z","timestamp":1780710666893,"version":"3.54.1"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,5,26]],"date-time":"2021-05-26T00:00:00Z","timestamp":1621987200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,5,26]],"date-time":"2021-05-26T00:00:00Z","timestamp":1621987200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","award":["CZ.02.1.01\/0.0\/0.0\/16_019\/0000868"],"award-info":[{"award-number":["CZ.02.1.01\/0.0\/0.0\/16_019\/0000868"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001823","name":"Ministerstvo \u0160kolstv\u00ed, Ml\u00e1de\u017ee a T\u011blov\u00fdchovy","doi-asserted-by":"publisher","award":["LM2018131"],"award-info":[{"award-number":["LM2018131"]}],"id":[{"id":"10.13039\/501100001823","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014809","name":"Technology Agency of the Czech Republic","doi-asserted-by":"publisher","award":["TN01000013"],"award-info":[{"award-number":["TN01000013"]}],"id":[{"id":"10.13039\/100014809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cheminform"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Interpretation of QSAR models is useful to understand the complex nature of biological or physicochemical processes, guide structural optimization or perform knowledge-based validation of QSAR models. Highly predictive models are usually complex and their interpretation is non-trivial. This is particularly true for modern neural networks. Various approaches to interpretation of these models exist. However, it is difficult to evaluate and compare performance and applicability of these ever-emerging methods. Herein, we developed several benchmark data sets with end-points determined by pre-defined patterns. These data sets are purposed for evaluation of the ability of interpretation approaches to retrieve these patterns. They represent tasks with different complexity levels: from simple atom-based additive properties to pharmacophore hypothesis. We proposed several quantitative metrics of interpretation performance. Applicability of benchmarks and metrics was demonstrated on a set of conventional models and end-to-end graph convolutional neural networks, interpreted by the previously suggested universal ML-agnostic approach for structural interpretation. We anticipate these benchmarks to be useful in evaluation of new interpretation approaches and investigation of decision making of complex \u201cblack box\u201d models. <\/jats:p>","DOI":"10.1186\/s13321-021-00519-x","type":"journal-article","created":{"date-parts":[[2021,5,26]],"date-time":"2021-05-26T20:03:20Z","timestamp":1622059400000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":69,"title":["Benchmarks for interpretation of QSAR models"],"prefix":"10.1186","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5373-9923","authenticated-orcid":false,"given":"Mariia","family":"Matveieva","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5088-8149","authenticated-orcid":false,"given":"Pavel","family":"Polishchuk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,5,26]]},"reference":[{"key":"519_CR1","doi-asserted-by":"publisher","first-page":"2618","DOI":"10.1021\/acs.jcim.7b00274","volume":"57","author":"P Polishchuk","year":"2017","unstructured":"Polishchuk P (2017) Interpretation of quantitative structure-activity relationship models: past, present, and future. J Chem Inf Model 57:2618\u20132639. https:\/\/doi.org\/10.1021\/acs.jcim.7b00274","journal-title":"J Chem Inf Model"},{"key":"519_CR2","doi-asserted-by":"publisher","first-page":"10520","DOI":"10.1021\/acs.chemrev.8b00728","volume":"119","author":"X Yang","year":"2019","unstructured":"Yang X, Wang Y, Byrne R, Schneider G, Yang S (2019) Concepts of artificial intelligence for computer-assisted drug discovery. Chem Rev 119:10520\u201310594. https:\/\/doi.org\/10.1021\/acs.chemrev.8b00728","journal-title":"Chem Rev"},{"key":"519_CR3","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-019-08987-4","author":"S Lapuschkin","year":"2019","unstructured":"Lapuschkin S, Waldchen S, Binder A, Montavon G, Samek W, Muller K (2019) Unmasking Clever Hans predictors and assessing what machines really learn. Nat Commun. https:\/\/doi.org\/10.1038\/s41467-019-08987-4","journal-title":"Nat Commun"},{"key":"519_CR4","doi-asserted-by":"publisher","first-page":"573","DOI":"10.1038\/s42256-020-00236-4","volume":"2","author":"J Jimenez-Luna","year":"2020","unstructured":"Jimenez-Luna J, Grisoni F, Schneider G (2020) Drug discovery with explainable artificial intelligence. Nature Mach Intell 2:573\u2013584. https:\/\/doi.org\/10.1038\/s42256-020-00236-4","journal-title":"Nature Mach Intell"},{"key":"519_CR5","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0130140","author":"S Bach","year":"2015","unstructured":"Bach S, Binder A, Montavon G, Klauschen F, Muller K, Samek W (2015) On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE. https:\/\/doi.org\/10.1371\/journal.pone.0130140","journal-title":"PLoS ONE"},{"key":"519_CR6","unstructured":"Shrikumar A, Greenside P, Kundaje A (2019) Learning important features through propagating activation differences. arXiv: 1704.02685"},{"key":"519_CR7","doi-asserted-by":"publisher","unstructured":"Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. 2016 IEEE conference on computer vision and pattern recognition (Cvpr) 2921\u20132929. https:\/\/doi.org\/10.1109\/CVPR.2016.319","DOI":"10.1109\/CVPR.2016.319"},{"key":"519_CR8","doi-asserted-by":"publisher","unstructured":"Selvaraju R, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: visual explanations from deep networks via gradient-based localization. 2017 IEEE international conference on computer vision (Iccv) 618\u2013626. https:\/\/doi.org\/10.1109\/ICCV.2017.74","DOI":"10.1109\/ICCV.2017.74"},{"key":"519_CR9","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. arXivpreprint arXiv: 1706.03762"},{"key":"519_CR10","doi-asserted-by":"publisher","first-page":"11624","DOI":"10.1073\/pnas.1820657116","volume":"116","author":"K McCloskey","year":"2019","unstructured":"McCloskey K, Taly A, Monti F, Brenner M, Colwell L (2019) Using attribution to decode binding mechanism in neural network models for chemistry. Proc Natl Acad Sci USA 116:11624\u201311629. https:\/\/doi.org\/10.1073\/pnas.1820657116","journal-title":"Proc Natl Acad Sci USA"},{"key":"519_CR11","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1007\/s10115-013-0679-x","volume":"41","author":"E Strumbelj","year":"2014","unstructured":"Strumbelj E, Kononenko I (2014) Explaining prediction models and individual predictions with feature contributions. Knowl Inf Syst 41:647\u2013665. https:\/\/doi.org\/10.1007\/s10115-013-0679-x","journal-title":"Knowl Inf Syst"},{"key":"519_CR12","doi-asserted-by":"publisher","first-page":"8761","DOI":"10.1021\/acs.jmedchem.9b01101","volume":"63","author":"R Rodriguez-Perez","year":"2020","unstructured":"Rodriguez-Perez R, Bajorath J (2020) Interpretation of compound activity predictions from complex machine learning models using local approximations and shapley values. J Med Chem 63:8761\u20138777. https:\/\/doi.org\/10.1021\/acs.jmedchem.9b01101","journal-title":"J Med Chem"},{"key":"519_CR13","doi-asserted-by":"publisher","first-page":"731","DOI":"10.1007\/s10822-020-00310-4","volume":"34","author":"H Webel","year":"2020","unstructured":"Webel H, Kimber T, Radetzki S, Neuenschwander M, Nazare M, Volkamer A (2020) Revealing cytotoxic substructures in molecules using deep learning. J Comput Aided Mol Des 34:731\u2013746. https:\/\/doi.org\/10.1007\/s10822-020-00310-4","journal-title":"J Comput Aided Mol Des"},{"key":"519_CR14","doi-asserted-by":"publisher","first-page":"843","DOI":"10.1002\/minf.201300029","volume":"32","author":"PG Polishchuk","year":"2013","unstructured":"Polishchuk PG, Kuz\u2019min VE, Artemenko AG, Muratov EN (2013) Universal approach for structural interpretation of QSAR\/QSPR models. Mol Inf 32:843\u2013853. https:\/\/doi.org\/10.1002\/minf.201300029","journal-title":"Mol Inf"},{"key":"519_CR15","doi-asserted-by":"publisher","DOI":"10.1186\/s13321-020-0414-z","author":"B Tang","year":"2020","unstructured":"Tang B, Kramer S, Fang M, Qiu Y, Wu Z, Xu D (2020) A self-attention based message passing neural network for predicting molecular lipophilicity and aqueous solubility. J Cheminform. https:\/\/doi.org\/10.1186\/s13321-020-0414-z","journal-title":"J Cheminform"},{"key":"519_CR16","doi-asserted-by":"publisher","first-page":"1324","DOI":"10.1021\/acs.jcim.8b00825","volume":"59","author":"R Sheridan","year":"2019","unstructured":"Sheridan R (2019) Interpretation of QSAR models by coloring atoms according to changes in predicted activity: how robust is it? J Chem Inf Model 59:1324\u20131337. https:\/\/doi.org\/10.1021\/acs.jcim.8b00825","journal-title":"J Chem Inf Model"},{"key":"519_CR17","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1186\/1758-2946-5-43","volume":"5","author":"S Riniker","year":"2013","unstructured":"Riniker S, Landrum GA (2013) Similarity maps\u2014a visualization strategy for molecular fingerprints and machine-learning methods. J Cheminform 5:43. https:\/\/doi.org\/10.1186\/1758-2946-5-43","journal-title":"J Cheminform"},{"key":"519_CR18","first-page":"5898","volume-title":"Advances in neural information processing systems","author":"B Sanchez-Lengeling","year":"2020","unstructured":"Sanchez-Lengeling B, Wei J, Lee B, Reif E, Wang P, Qian W, McCloskey K, Colwell L, Wiltschko A (2020) Evaluating attribution for graph neural networks. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H (eds) Advances in neural information processing systems, vol 33. MIT Press, Cambridge, pp 5898\u20135910"},{"key":"519_CR19","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1021\/ci00063a006","volume":"29","author":"VN Viswanadhan","year":"1989","unstructured":"Viswanadhan VN, Ghose AK, Revankar GR, Robins RK (1989) Atomic physicochemical parameters for three dimensional structure directed quantitative structure-activity relationships. 4. Additional parameters for hydrophobic and dispersive interactions and their application for an automated superposition of certain naturally occurring nucleoside antibiotics. J Chem Inform Comput Sci 29:163\u2013172. https:\/\/doi.org\/10.1021\/ci00063a006","journal-title":"J Chem Inform Comput Sci"},{"key":"519_CR20","doi-asserted-by":"publisher","first-page":"2140","DOI":"10.1021\/ci700257y","volume":"47","author":"T Cheng","year":"2007","unstructured":"Cheng T, Zhao Y, Li X, Lin F, Xu Y, Zhang X, Li Y, Wang R, Lai L (2007) Computation of octanol\u2212water partition coefficients by guiding an additive model with knowledge. J Chem Inf Model 47:2140\u20132148. https:\/\/doi.org\/10.1021\/ci700257y","journal-title":"J Chem Inf Model"},{"key":"519_CR21","unstructured":"Ramsundar B (2018) Molecular machine learning with DeepChem. Abstracts of Papers of the American Chemical Society, 255"},{"key":"519_CR22","doi-asserted-by":"publisher","unstructured":"Kutlushina A, Khakimova A, Madzhidov T, Polishchuk P (2019) Kutlushina, A., et al. Ligand-Based Pharmacophore Modeling Using Novel 3D Pharmacophore Signatures (vol 23, pg 3094, 2018). Molecules https:\/\/doi.org\/10.3390\/molecules24061052","DOI":"10.3390\/molecules24061052"},{"key":"519_CR23","doi-asserted-by":"publisher","DOI":"10.1186\/s13321-020-00431-w","author":"P Polishchuk","year":"2020","unstructured":"Polishchuk P (2020) CReM: chemically reasonable mutations framework for structure generation. J Cheminform. https:\/\/doi.org\/10.1186\/s13321-020-00431-w","journal-title":"J Cheminform"},{"key":"519_CR24","doi-asserted-by":"publisher","first-page":"6074","DOI":"10.1021\/acs.jcim.0c00792","volume":"60","author":"P Polishchuk","year":"2020","unstructured":"Polishchuk P (2020) Control of synthetic feasibility of compounds generated with CReM. J Chem Inf Model 60:6074\u20136080","journal-title":"J Chem Inf Model"},{"key":"519_CR25","unstructured":"RDKit: Open-source cheminformatics Software 2019.09.3 (2019). http:\/\/www.rdkit.org"},{"key":"519_CR26","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O (2011) Scikit-learn: machine learning in python. JMLR 12:2825\u20132830","journal-title":"JMLR"},{"key":"519_CR27","doi-asserted-by":"publisher","first-page":"1455","DOI":"10.1021\/acs.jcim.6b00371","volume":"56","author":"P Polishchuk","year":"2016","unstructured":"Polishchuk P, Tinkov O, Khristova T, Ognichenko L, Kosinskaya A, Varnek A, Kuz\u2019min V, (2016) Structural and physico-chemical interpretation (spci) of qsar models and its comparison with matched molecular pair analysis. J Chem Inf Model 56:1455\u20131469. https:\/\/doi.org\/10.1021\/acs.jcim.6b00371","journal-title":"J Chem Inf Model"},{"key":"519_CR28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1021\/ci400572x","volume":"54","author":"A Golbraikh","year":"2014","unstructured":"Golbraikh A, Muratov E, Fourches D, Tropsha A (2014) Data set modelability by QSAR. J Chem Inf Model 54:1\u20134. https:\/\/doi.org\/10.1021\/ci400572x","journal-title":"J Chem Inf Model"},{"key":"519_CR29","unstructured":"DeepChem 2.5.0. https:\/\/github.com\/deepchem\/deepchem"},{"key":"519_CR30","doi-asserted-by":"publisher","first-page":"717","DOI":"10.1007\/s10822-019-00274-0","volume":"34","author":"M Robinson","year":"2020","unstructured":"Robinson M, Glen R, Lee A (2020) Validating the validation: reanalyzing a large-scale comparison of deep learning and machine learning models for bioactivity prediction. J Comput Aided Mol Des 34:717\u2013730. https:\/\/doi.org\/10.1007\/s10822-019-00274-0","journal-title":"J Comput Aided Mol Des"},{"key":"519_CR31","doi-asserted-by":"publisher","first-page":"2281","DOI":"10.1002\/cmdc.201800309","volume":"13","author":"A Finkelmann","year":"2018","unstructured":"Finkelmann A, Goldmann D, Schneider G, Goller A (2018) MetScore: site of metabolism prediction beyond cytochrome P450 enzymes. ChemMedChem 13:2281\u20132289. https:\/\/doi.org\/10.1002\/cmdc.201800309","journal-title":"ChemMedChem"}],"container-title":["Journal of Cheminformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-021-00519-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13321-021-00519-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-021-00519-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,26]],"date-time":"2021-05-26T20:19:30Z","timestamp":1622060370000},"score":1,"resource":{"primary":{"URL":"https:\/\/jcheminf.biomedcentral.com\/articles\/10.1186\/s13321-021-00519-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,26]]},"references-count":31,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["519"],"URL":"https:\/\/doi.org\/10.1186\/s13321-021-00519-x","relation":{},"ISSN":["1758-2946"],"issn-type":[{"value":"1758-2946","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,26]]},"assertion":[{"value":"23 February 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 May 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 May 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"41"}}