{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T19:03:17Z","timestamp":1784055797273,"version":"3.55.0"},"reference-count":32,"publisher":"Oxford University Press (OUP)","issue":"18","license":[{"start":{"date-parts":[[2019,2,19]],"date-time":"2019-02-19T00:00:00Z","timestamp":1550534400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100001665","name":"L\u2019Agence Nationale de la Recherche","doi-asserted-by":"crossref","award":["ANR-15-CE11\u20130029\u201303"],"award-info":[{"award-number":["ANR-15-CE11\u20130029\u201303"]}],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,9,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Protein model quality assessment (QA) is a crucial and yet open problem in structural bioinformatics. The current best methods for single-model QA typically combine results from different approaches, each based on different input features constructed by experts in the field. Then, the prediction model is trained using a machine-learning algorithm. Recently, with the development of convolutional neural networks (CNN), the training paradigm has changed. In computer vision, the expert-developed features have been significantly overpassed by automatically trained convolutional filters. This motivated us to apply a three-dimensional (3D) CNN to the problem of protein model QA.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We developed Ornate (Oriented Routed Neural network with Automatic Typing)\u2014a novel method for single-model QA. Ornate is a residue-wise scoring function that takes as input 3D density maps. It predicts the local (residue-wise) and the global model quality through a deep 3D CNN. Specifically, Ornate aligns the input density map, corresponding to each residue and its neighborhood, with the backbone topology of this residue. This circumvents the problem of ambiguous orientations of the initial models. Also, Ornate includes automatic identification of atom types and dynamic routing of the data in the network. Established benchmarks (CASP 11 and CASP 12) demonstrate the state-of-the-art performance of our approach among single-model QA methods.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>The method is available at https:\/\/team.inria.fr\/nano-d\/software\/Ornate\/. It consists of a C++ executable that transforms molecular structures into volumetric density maps, and a Python code based on the TensorFlow framework for applying the Ornate model to these maps.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Supplementary information<\/jats:title>\n                    <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btz122","type":"journal-article","created":{"date-parts":[[2019,2,17]],"date-time":"2019-02-17T23:25:53Z","timestamp":1550445953000},"page":"3313-3319","source":"Crossref","is-referenced-by-count":86,"title":["Protein model quality assessment using 3D oriented convolutional neural networks"],"prefix":"10.1093","volume":"35","author":[{"given":"Guillaume","family":"Pag\u00e8s","sequence":"first","affiliation":[{"name":"Universit\u00e9 Grenoble Alpes, Inria, CNRS , Grenoble INP, LJK, Grenoble, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benoit","family":"Charmettant","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Grenoble Alpes, Inria, CNRS , Grenoble INP, LJK, Grenoble, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sergei","family":"Grudinin","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Grenoble Alpes, Inria, CNRS , Grenoble INP, LJK, Grenoble, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2019,2,19]]},"reference":[{"key":"2023013108051881200_btz122-B1","first-page":"265","article-title":"Tensorflow: a system for large-scale machine learning","volume-title":"Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation","author":"Abadi","year":"2016"},{"key":"2023013108051881200_btz122-B2","doi-asserted-by":"crossref","first-page":"23990","DOI":"10.1038\/srep23990","article-title":"Protein single-model quality assessment by feature-based probability density functions","volume":"6","author":"Cao","year":"2016","journal-title":"Sci. Rep."},{"key":"2023013108051881200_btz122-B3","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1186\/s12859-016-1405-y","article-title":"DeepQA: improving the estimation of single protein model quality with deep belief networks","volume":"17","author":"Cao","year":"2016","journal-title":"BMC Bioinform."},{"key":"2023013108051881200_btz122-B4","article-title":"Fast and accurate deep network learning by exponential linear units (elus)","author":"Clevert","year":"2016","journal-title":"International Conf. on Learning Representations"},{"key":"2023013108051881200_btz122-B5","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1002\/prot.21669","article-title":"Assessment of predictions in the model quality assessment category","volume":"69","author":"Cozzetto","year":"2007","journal-title":"ProteinsStruct. Funct. Bioinform."},{"key":"2023013108051881200_btz122-B6","doi-asserted-by":"crossref","first-page":"4046","DOI":"10.1093\/bioinformatics\/bty494","article-title":"Deep convolutional networks for quality assessment of protein folds","volume":"34","author":"Derevyanko","year":"2018","journal-title":"Bioinformatics"},{"key":"2023013108051881200_btz122-B7","doi-asserted-by":"crossref","first-page":"1015","DOI":"10.1093\/bioinformatics\/btg124","article-title":"3D-Jury: a simple approach to improve protein structure predictions","volume":"19","author":"Ginalski","year":"2003","journal-title":"Bioinformatics"},{"key":"2023013108051881200_btz122-B8","article-title":"Decision forests, convolutional networks and the models in-between","author":"Ioannou","year":"2016"},{"key":"2023013108051881200_btz122-B9","first-page":"448","article-title":"Batch normalization: accelerating deep network training by reducing internal covariate shift","author":"Ioffe","year":"2015","journal-title":"International Conference on Machine Learning"},{"key":"2023013108051881200_btz122-B10","doi-asserted-by":"crossref","first-page":"3036","DOI":"10.1093\/bioinformatics\/btx350","article-title":"DeepSite: protein-binding site predictor using 3D-convolutional neural networks","volume":"33","author":"Jim\u00e9nez","year":"2017","journal-title":"Bioinformatics"},{"key":"2023013108051881200_btz122-B11","doi-asserted-by":"crossref","DOI":"10.1093\/bioinformatics\/bty1037","article-title":"Smooth orientation-dependent scoring function for coarse-grained protein quality assessment","author":"Karasikov","year":"2019","journal-title":"Bioinformatics"},{"key":"2023013108051881200_btz122-B12","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1016\/B978-0-12-381270-4.00019-6","article-title":"ROSETTA3: an object-oriented software suite for the simulation and design of macromolecules","volume":"487","author":"Leaver-Fay","year":"2011","journal-title":"Methods Enzymol."},{"key":"2023013108051881200_btz122-B13","doi-asserted-by":"crossref","first-page":"609","DOI":"10.1145\/1553374.1553453","article-title":"Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations","volume-title":"Proceedings of the 26th Annual International Conference on Machine Learning","author":"Lee","year":"2009"},{"key":"2023013108051881200_btz122-B14","doi-asserted-by":"crossref","first-page":"2354","DOI":"10.1110\/ps.08501","article-title":"Pcons: a neural-network\u2013based consensus predictor that improves fold recognition","volume":"10","author":"Lundstr\u00f6m","year":"2001","journal-title":"Protein Sci."},{"key":"2023013108051881200_btz122-B15","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1038\/356083a0","article-title":"Assessment of protein models with three-dimensional profiles","volume":"356","author":"L\u00fcthy","year":"1992","journal-title":"Nature"},{"key":"2023013108051881200_btz122-B16","doi-asserted-by":"crossref","first-page":"2722","DOI":"10.1093\/bioinformatics\/btt473","article-title":"lDDT: a local superposition-free score for comparing protein structures and models using distance difference tests","volume":"29","author":"Mariani","year":"2013","journal-title":"Bioinformatics"},{"key":"2023013108051881200_btz122-B17","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1002\/prot.25415","article-title":"Critical assessment of methods of protein structure prediction (CASP)\u2014Round XII","volume":"86","author":"Moult","year":"2018","journal-title":"Proteins Struct. Funct. Bioinform."},{"key":"2023013108051881200_btz122-B18","doi-asserted-by":"crossref","first-page":"1131","DOI":"10.1002\/prot.25278","article-title":"VoroMQA: assessment of protein structure quality using interatomic contact areas","volume":"85","author":"Olechnovi\u010d","year":"2017","journal-title":"Proteins Struct. Funct. Bioinform."},{"key":"2023013108051881200_btz122-B19","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1002\/prot.24172","article-title":"CAD-score: a new contact area difference-based function for evaluation of protein structural models","volume":"81","author":"Olechnovi\u010d","year":"2013","journal-title":"Proteins Struct. Funct. Bioinform."},{"key":"2023013108051881200_btz122-B20","article-title":"Comparative analysis of methods for evaluation of protein models against native structures","author":"Olechnovi\u010d","year":"2018","journal-title":"Bioinformatics"},{"key":"2023013108051881200_btz122-B21","doi-asserted-by":"crossref","first-page":"942","DOI":"10.1021\/acs.jcim.6b00740","article-title":"Protein\u2013ligand scoring with convolutional neural networks","volume":"57","author":"Ragoza","year":"2017","journal-title":"J. Chem. Inf. Model."},{"key":"2023013108051881200_btz122-B22","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1186\/1471-2105-13-224","article-title":"Improved model quality assessment using ProQ2","volume":"13","author":"Ray","year":"2012","journal-title":"BMC Bioinform."},{"key":"2023013108051881200_btz122-B23","doi-asserted-by":"crossref","first-page":"3274","DOI":"10.1093\/bioinformatics\/bts618","article-title":"Fast protein structure alignment using Gaussian overlap scoring of backbone peptide fragment similarity","volume":"28","author":"Ritchie","year":"2012","journal-title":"Bioinformatics"},{"key":"2023013108051881200_btz122-B24","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1002\/prot.340170404","article-title":"Recognition of errors in three-dimensional structures of proteins","volume":"17","author":"Sippl","year":"1993","journal-title":"Proteins Struct. Funct. Bioinform."},{"key":"2023013108051881200_btz122-B25","doi-asserted-by":"crossref","first-page":"1282","DOI":"10.1093\/bioinformatics\/btm098","article-title":"Uniref: comprehensive and non-redundant UniProt reference clusters","volume":"23","author":"Suzek","year":"2007","journal-title":"Bioinformatics"},{"key":"2023013108051881200_btz122-B26","article-title":"Generalizable protein interface prediction with end-to-end learning","author":"Townshend","year":"2018"},{"key":"2023013108051881200_btz122-B27","first-page":"1578","article-title":"ProQ3D: improved model quality assessments using deep learning","volume":"33","author":"Uziela","year":"2017","journal-title":"Bioinformatics"},{"key":"2023013108051881200_btz122-B28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1198\/10618600152418584","article-title":"The art of data augmentation","volume":"10","author":"Van Dyk","year":"2001","journal-title":"J. Comput. Graph. Statist."},{"key":"2023013108051881200_btz122-B29","article-title":"AtomNet: a deep convolutional neural network for bioactivity prediction in structure-based drug discovery","author":"Wallach","year":"2015"},{"key":"2023013108051881200_btz122-B30","first-page":"5028","article-title":"Harmonic networks: deep translation and rotation equivariance","volume-title":"Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)","author":"Worrall","year":"2017"},{"key":"2023013108051881200_btz122-B31","doi-asserted-by":"crossref","first-page":"e15386","DOI":"10.1371\/journal.pone.0015386","article-title":"A novel side-chain orientation dependent potential derived from random-walk reference state for protein fold selection and structure prediction","volume":"5","author":"Zhang","year":"2010","journal-title":"PLoS One"},{"key":"2023013108051881200_btz122-B32","doi-asserted-by":"crossref","first-page":"702","DOI":"10.1002\/prot.20264","article-title":"Scoring function for automated assessment of protein structure template quality","volume":"57","author":"Zhang","year":"2004","journal-title":"Proteins Struct. Funct. Bioinform."}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/18\/3313\/48975313\/bioinformatics_35_18_3313.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/18\/3313\/48975313\/bioinformatics_35_18_3313.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T08:38:51Z","timestamp":1675154331000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/35\/18\/3313\/5341430"}},"subtitle":[],"editor":[{"given":"Alfonso","family":"Valencia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2019,2,19]]},"references-count":32,"journal-issue":{"issue":"18","published-print":{"date-parts":[[2019,9,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btz122","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/432146","asserted-by":"object"}]},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,9,15]]},"published":{"date-parts":[[2019,2,19]]}}}