{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T19:03:22Z","timestamp":1784055802561,"version":"3.55.0"},"reference-count":56,"publisher":"Oxford University Press (OUP)","issue":"23","license":[{"start":{"date-parts":[[2018,6,20]],"date-time":"2018-06-20T00:00:00Z","timestamp":1529452800000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"NSERC","doi-asserted-by":"publisher","award":["RGPIN 355789"],"award-info":[{"award-number":["RGPIN 355789"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"NSERC","doi-asserted-by":"publisher","award":["RGPIN 1016552"],"award-info":[{"award-number":["RGPIN 1016552"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001804","name":"Canada Research Chair","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001804","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100007631","name":"Canadian Institute for Advanced Research","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007631","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007631","name":"CIFAR","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007631","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Calcul Qu\u00e9bec and Compute Canada"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Motivation<\/jats:title><jats:p>The computational prediction of a protein structure from its sequence generally relies on a method to assess the quality of protein models. Most assessment methods rank candidate models using heavily engineered structural features, defined as complex functions of the atomic coordinates. However, very few methods have attempted to learn these features directly from the data.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We show that deep convolutional networks can be used to predict the ranking of model structures solely on the basis of their raw three-dimensional atomic densities, without any feature tuning. We develop a deep neural network that performs on par with state-of-the-art algorithms from the literature. The network is trained on decoys from the CASP7 to CASP10 datasets and its performance is tested on the CASP11 dataset. Additional testing on decoys from the CASP12, CAMEO and 3DRobot datasets confirms that the network performs consistently well across a variety of protein structures. While the network learns to assess structural decoys globally and does not rely on any predefined features, it can be analyzed to show that it implicitly identifies regions that deviate from the native structure.<\/jats:p><\/jats:sec><jats:sec><jats:title>Availability and implementation<\/jats:title><jats:p>The code and the datasets are available at https:\/\/github.com\/lamoureux-lab\/3DCNN_MQA.<\/jats:p><\/jats:sec><jats:sec><jats:title>Supplementary information<\/jats:title><jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p><\/jats:sec>","DOI":"10.1093\/bioinformatics\/bty494","type":"journal-article","created":{"date-parts":[[2018,6,15]],"date-time":"2018-06-15T19:38:25Z","timestamp":1529091505000},"page":"4046-4053","source":"Crossref","is-referenced-by-count":94,"title":["Deep convolutional networks for quality assessment of protein folds"],"prefix":"10.1093","volume":"34","author":[{"given":"Georgy","family":"Derevyanko","sequence":"first","affiliation":[{"name":"Department of Chemistry and Biochemistry and Centre for Research in Molecular Modeling (CERMM), Concordia University, Montr\u00e9al, Qu\u00e9bec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1903-7220","authenticated-orcid":false,"given":"Sergei","family":"Grudinin","sequence":"additional","affiliation":[{"name":"Inria, Universit\u00e9 Grenoble Alpes, CNRS, Grenoble INP, LJK, Grenoble, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yoshua","family":"Bengio","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Operations Research, Universit\u00e9 de Montr\u00e9al, Montr\u00e9al, Qu\u00e9bec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7656-0176","authenticated-orcid":false,"given":"Guillaume","family":"Lamoureux","sequence":"additional","affiliation":[{"name":"Department of Chemistry and Biochemistry and Centre for Research in Molecular Modeling (CERMM), Concordia University, Montr\u00e9al, Qu\u00e9bec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2018,6,19]]},"reference":[{"key":"2023012712284806900_bty494-B1","doi-asserted-by":"crossref","first-page":"831","DOI":"10.1038\/nbt.3300","article-title":"Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning","volume":"33","author":"Alipanahi","year":"2015","journal-title":"Nat. Biotechnol"},{"key":"2023012712284806900_bty494-B2","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1016\/S0022-2836(05)80360-2","article-title":"Basic local alignment search tool","volume":"215","author":"Altschul","year":"1990","journal-title":"J. Mol. Biol"},{"key":"2023012712284806900_bty494-B3","author":"Bau","year":"2017"},{"key":"2023012712284806900_bty494-B4","author":"Bengio","year":"1990"},{"key":"2023012712284806900_bty494-B5","doi-asserted-by":"crossref","first-page":"495.","DOI":"10.1186\/s12859-016-1405-y","article-title":"DeepQA: improving the estimation of single protein model quality with deep belief networks","volume":"17","author":"Cao","year":"2016","journal-title":"BMC Bioinformatics"},{"key":"2023012712284806900_bty494-B6","doi-asserted-by":"crossref","first-page":"e1003926.","DOI":"10.1371\/journal.pcbi.1003926","article-title":"ECOD: an evolutionary classification of protein domains","volume":"10","author":"Cheng","year":"2014","journal-title":"PLoS Comput. Biol"},{"key":"2023012712284806900_bty494-B7","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1093\/bioinformatics\/btv601","article-title":"3DRobot: automated generation of diverse and well-packed protein structure decoys","volume":"32","author":"Deng","year":"2016","journal-title":"Bioinformatics"},{"key":"2023012712284806900_bty494-B8","doi-asserted-by":"crossref","first-page":"1042","DOI":"10.1126\/science.1219021","article-title":"The protein-folding problem, 50 years on","volume":"338","author":"Dill","year":"2012","journal-title":"Science"},{"key":"2023012712284806900_bty494-B9","author":"Elofsson","year":"2017"},{"key":"2023012712284806900_bty494-B10","doi-asserted-by":"crossref","first-page":"W30","DOI":"10.1093\/nar\/gkv397","article-title":"HMMER web server: 2015 update","volume":"43","author":"Finn","year":"2015","journal-title":"Nucleic Acids Res"},{"key":"2023012712284806900_bty494-B11","doi-asserted-by":"crossref","first-page":"D279","DOI":"10.1093\/nar\/gkv1344","article-title":"The Pfam protein families database: towards a more sustainable future","volume":"44","author":"Finn","year":"2016","journal-title":"Nucleic Acids Res"},{"key":"2023012712284806900_bty494-B12","author":"Gong","year":"2013"},{"key":"2023012712284806900_bty494-B13","doi-asserted-by":"crossref","first-page":"bat031.","DOI":"10.1093\/database\/bat031","article-title":"The Protein Model Portal\u2014a comprehensive resource for protein structure and model information","volume":"2013","author":"Haas","year":"2013","journal-title":"Database"},{"key":"2023012712284806900_bty494-B14","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the dimensionality of data with neural networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"2023012712284806900_bty494-B15","doi-asserted-by":"crossref","first-page":"1527","DOI":"10.1162\/neco.2006.18.7.1527","article-title":"A fast learning algorithm for deep belief nets","volume":"18","author":"Hinton","year":"2006","journal-title":"Neural Comput"},{"key":"2023012712284806900_bty494-B16","doi-asserted-by":"crossref","first-page":"1144","DOI":"10.1126\/science.7761829","article-title":"Classical electrostatics in biology and chemistry","volume":"268","author":"Honig","year":"1995","journal-title":"Science"},{"key":"2023012712284806900_bty494-B17","doi-asserted-by":"crossref","first-page":"1866","DOI":"10.1002\/jcc.20504","article-title":"An iterative knowledge-based scoring function to predict protein\u2013ligand interactions: I. Derivation of interaction potentials","volume":"27","author":"Huang","year":"2006","journal-title":"J. Comput. Chem"},{"key":"2023012712284806900_bty494-B18","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1002\/prot.21949","article-title":"An iterative knowledge-based scoring function for protein\u2013protein recognition","volume":"72","author":"Huang","year":"2008","journal-title":"Proteins"},{"key":"2023012712284806900_bty494-B19","author":"Ioffe","year":"2015"},{"key":"2023012712284806900_bty494-B20","doi-asserted-by":"crossref","first-page":"3036","DOI":"10.1093\/bioinformatics\/btx350","article-title":"DeepSite: protein-binding site predictor using 3D-convolutional neural networks","volume":"33","author":"Jim\u00e9nez","year":"2017","journal-title":"Bioinformatics"},{"key":"2023012712284806900_bty494-B21","doi-asserted-by":"crossref","first-page":"31571.","DOI":"10.1038\/srep31571","article-title":"Sorting protein decoys by machine-learning-to-rank","volume":"6","author":"Jing","year":"2016","journal-title":"Sci. Rep"},{"key":"2023012712284806900_bty494-B22","first-page":"133","author":"Joachims","year":"2002"},{"key":"2023012712284806900_bty494-B23","author":"Kingma","year":"2014"},{"key":"2023012712284806900_bty494-B24","doi-asserted-by":"crossref","first-page":"1893","DOI":"10.1021\/ci300604z","article-title":"Lessons learned in empirical scoring with Smina from the CSAR 2011 benchmarking exercise","volume":"53","author":"Koes","year":"2013","journal-title":"J. Chem. Inf. Model"},{"key":"2023012712284806900_bty494-B25","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1002\/prot.22488","article-title":"Improved prediction of protein side-chain conformations with SCWRL4","volume":"77","author":"Krivov","year":"2009","journal-title":"Proteins"},{"key":"2023012712284806900_bty494-B26","first-page":"1097","author":"Krizhevsky","year":"2012"},{"key":"2023012712284806900_bty494-B27","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1002\/prot.24919","article-title":"Methods of model accuracy estimation can help selecting the best models from decoy sets: assessment of model accuracy estimations in CASP11","volume":"84","author":"Kryshtafovych","year":"2016","journal-title":"Proteins"},{"key":"2023012712284806900_bty494-B28","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1016\/B978-0-12-381270-4.00019-6","article-title":"Rosetta3: an object-oriented software suite for the simulation and design of macromolecules","volume":"487","author":"Leaver-Fay","year":"2011","journal-title":"Methods Enzymol"},{"key":"2023012712284806900_bty494-B29","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1162\/neco.1989.1.4.541","article-title":"Backpropagation applied to handwritten zip code recognition","volume":"1","author":"LeCun","year":"1989","journal-title":"Neural Comput"},{"key":"2023012712284806900_bty494-B30","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"2023012712284806900_bty494-B31","doi-asserted-by":"crossref","first-page":"2722","DOI":"10.1093\/bioinformatics\/btt473","article-title":"lDDT: a local superposition-free score for comparing protein structures and models using distance difference tests","volume":"29","author":"Mariani","year":"2013","journal-title":"Bioinformatics"},{"key":"2023012712284806900_bty494-B32","doi-asserted-by":"crossref","first-page":"ii","DOI":"10.1002\/prot.340230303","article-title":"A large-scale experiment to assess protein structure prediction methods","volume":"23","author":"Moult","year":"1995","journal-title":"Proteins"},{"key":"2023012712284806900_bty494-B33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/prot.24452","article-title":"Critical assessment of methods of protein structure prediction (CASP)\u2014round x","volume":"82","author":"Moult","year":"2014","journal-title":"Proteins"},{"key":"2023012712284806900_bty494-B34","author":"Nguyen","year":"2014"},{"key":"2023012712284806900_bty494-B35","doi-asserted-by":"crossref","first-page":"1131","DOI":"10.1002\/prot.25278","article-title":"VoroMQA: assessment of protein structure quality using interatomic contact areas","volume":"85","author":"Olechnovi\u010d","year":"2017","journal-title":"Proteins"},{"key":"2023012712284806900_bty494-B36","doi-asserted-by":"crossref","first-page":"942","DOI":"10.1021\/acs.jcim.6b00740","article-title":"Protein\u2013ligand scoring with convolutional neural networks","volume":"57","author":"Ragoza","year":"2017","journal-title":"J. Chem. Inf. Model"},{"key":"2023012712284806900_bty494-B37","doi-asserted-by":"crossref","first-page":"224.","DOI":"10.1186\/1471-2105-13-224","article-title":"Improved model quality assessment using ProQ2","volume":"13","author":"Ray","year":"2012","journal-title":"BMC Bioinformatics"},{"key":"2023012712284806900_bty494-B38","author":"Sch\u00fctt","year":"2017"},{"key":"2023012712284806900_bty494-B39","doi-asserted-by":"crossref","first-page":"13890.","DOI":"10.1038\/ncomms13890","article-title":"Quantum-chemical insights from deep tensor neural networks","volume":"8","author":"Sch\u00fctt","year":"2017","journal-title":"Nat. Commun"},{"key":"2023012712284806900_bty494-B40","author":"Selvaraju","year":"2016"},{"key":"2023012712284806900_bty494-B41","doi-asserted-by":"crossref","first-page":"2507","DOI":"10.1110\/ps.062416606","article-title":"Statistical potential for assessment and prediction of protein structures","volume":"15","author":"Shen","year":"2006","journal-title":"Protein Sci"},{"key":"2023012712284806900_bty494-B42","first-page":"124","volume-title":"Graphics Gems III","author":"Shoemake","year":"1992"},{"key":"2023012712284806900_bty494-B43","doi-asserted-by":"crossref","first-page":"3192","DOI":"10.1039\/C6SC05720A","article-title":"ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost","volume":"8","author":"Smith","year":"2017","journal-title":"Chem. Sci"},{"key":"2023012712284806900_bty494-B44","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1021\/jp102587q","article-title":"Calculation of local water densities in biological systems: a comparison of molecular dynamics simulations and the 3D-RISM-KH molecular theory of solvation","volume":"115","author":"Stumpe","year":"2011","journal-title":"J. Phys. Chem. B"},{"key":"2023012712284806900_bty494-B45","doi-asserted-by":"crossref","first-page":"302.","DOI":"10.1186\/s12859-017-1702-0","article-title":"3D deep convolutional neural networks for amino acid environment similarity analysis","volume":"18","author":"Torng","year":"2017","journal-title":"BMC Bioinformatics"},{"key":"2023012712284806900_bty494-B46","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1002\/jcc.21334","article-title":"AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading","volume":"31","author":"Trott","year":"2010","journal-title":"J. Comput. Chem"},{"key":"2023012712284806900_bty494-B47","doi-asserted-by":"crossref","first-page":"33509.","DOI":"10.1038\/srep33509","article-title":"ProQ3: improved model quality assessments using Rosetta energy terms","volume":"6","author":"Uziela","year":"2016","journal-title":"Sci. Rep"},{"key":"2023012712284806900_bty494-B48","doi-asserted-by":"crossref","first-page":"1578","DOI":"10.1093\/bioinformatics\/btw819","article-title":"ProQ3D: improved model quality assessments using deep learning","volume":"33","author":"Uziela","year":"2017","journal-title":"Bioinformatics"},{"key":"2023012712284806900_bty494-B49","author":"Wallach","year":"2015"},{"key":"2023012712284806900_bty494-B50","author":"Worrall","year":"2016"},{"key":"2023012712284806900_bty494-B51","doi-asserted-by":"crossref","first-page":"1254806.","DOI":"10.1126\/science.1254806","article-title":"The human splicing code reveals new insights into the genetic determinants of disease","volume":"347","author":"Xiong","year":"2015","journal-title":"Science"},{"key":"2023012712284806900_bty494-B52","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1002\/prot.10052","article-title":"Processing and evaluation of predictions in CASP4","volume":"45","author":"Zemla","year":"2001","journal-title":"Proteins"},{"key":"2023012712284806900_bty494-B53","doi-asserted-by":"crossref","first-page":"e15386.","DOI":"10.1371\/journal.pone.0015386","article-title":"A novel side-chain orientation dependent potential derived from random-walk reference state for protein fold selection and structure prediction","volume":"5","author":"Zhang","year":"2010","journal-title":"PloS One"},{"key":"2023012712284806900_bty494-B54","doi-asserted-by":"crossref","first-page":"702","DOI":"10.1002\/prot.20264","article-title":"Scoring function for automated assessment of protein structure template quality","volume":"57","author":"Zhang","year":"2004","journal-title":"Proteins Struct. Funct. Bioinf"},{"key":"2023012712284806900_bty494-B55","doi-asserted-by":"crossref","first-page":"2714","DOI":"10.1110\/ps.0217002","article-title":"Distance-scaled, finite ideal-gas reference state improves structure-derived potentials of mean force for structure selection and stability prediction","volume":"11","author":"Zhou","year":"2009","journal-title":"Protein Sci"},{"key":"2023012712284806900_bty494-B56","doi-asserted-by":"crossref","first-page":"931","DOI":"10.1038\/nmeth.3547","article-title":"Predicting effects of noncoding variants with deep learning-based sequence model","volume":"12","author":"Zhou","year":"2015","journal-title":"Nat. Methods"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/34\/23\/4046\/48919689\/bioinformatics_34_23_4046.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/34\/23\/4046\/48919689\/bioinformatics_34_23_4046.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,7]],"date-time":"2024-07-07T14:27:36Z","timestamp":1720362456000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/34\/23\/4046\/5040325"}},"subtitle":[],"editor":[{"given":"Alfonso","family":"Valencia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2018,6,19]]},"references-count":56,"journal-issue":{"issue":"23","published-print":{"date-parts":[[2018,12,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/bty494","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2018,12,1]]},"published":{"date-parts":[[2018,6,19]]}}}