{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T19:03:18Z","timestamp":1784055798671,"version":"3.55.0"},"reference-count":42,"publisher":"Oxford University Press (OUP)","issue":"22-23","license":[{"start":{"date-parts":[[2020,12,16]],"date-time":"2020-12-16T00:00:00Z","timestamp":1608076800000},"content-version":"vor","delay-in-days":15,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01GM089753"],"award-info":[{"award-number":["R01GM089753"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["DBI1564955"],"award-info":[{"award-number":["DBI1564955"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Accurately estimating protein model quality in the absence of experimental structure is not only important for model evaluation and selection but also useful for model refinement. Progress has been steadily made by introducing new features and algorithms (especially deep neural networks), but the accuracy of quality assessment (QA) is still not very satisfactory, especially local QA on hard protein targets.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We propose a new single-model-based QA method ResNetQA for both local and global quality assessment. Our method predicts model quality by integrating sequential and pairwise features using a deep neural network composed of both 1D and 2D convolutional residual neural networks (ResNet). The 2D ResNet module extracts useful information from pairwise features such as model-derived distance maps, co-evolution information, and predicted distance potential from sequences. The 1D ResNet is used to predict local (global) model quality from sequential features and pooled pairwise information generated by 2D ResNet. Tested on the CASP12 and CASP13 datasets, our experimental results show that our method greatly outperforms existing state-of-the-art methods. Our ablation studies indicate that the 2D ResNet module and pairwise features play an important role in improving model quality assessment.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>https:\/\/github.com\/AndersJing\/ResNetQA.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Supplementary information<\/jats:title>\n                    <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btaa1037","type":"journal-article","created":{"date-parts":[[2020,12,6]],"date-time":"2020-12-06T07:07:14Z","timestamp":1607238434000},"page":"5361-5367","source":"Crossref","is-referenced-by-count":11,"title":["Improved protein model quality assessment by integrating sequential and pairwise features using deep learning"],"prefix":"10.1093","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7193-9101","authenticated-orcid":false,"given":"Xiaoyang","family":"Jing","sequence":"first","affiliation":[{"name":"Toyota Technological Institute at Chicago , Chicago, IL 60637, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinbo","family":"Xu","sequence":"additional","affiliation":[{"name":"Toyota Technological Institute at Chicago , Chicago, IL 60637, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,12,16]]},"reference":[{"key":"2023062803481013100_btaa1037-B1","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1002\/prot.25403","article-title":"Definition and classification of evaluation units for tertiary structure prediction in CASP12 facilitated through semi-automated metrics","volume":"86","author":"Abriata","year":"2018","journal-title":"Proteins Struct. Funct. Bioinf"},{"key":"2023062803481013100_btaa1037-B2","doi-asserted-by":"crossref","first-page":"2301","DOI":"10.3390\/ijms20092301","article-title":"Methods for the refinement of protein structure 3D models","volume":"20","author":"Adiyaman","year":"2019","journal-title":"Int. J. Mol. Sci"},{"key":"2023062803481013100_btaa1037-B3","doi-asserted-by":"crossref","first-page":"3389","DOI":"10.1093\/nar\/25.17.3389","article-title":"Gapped BLAST and PSI-BLAST: a new generation of protein database search programs","volume":"25","author":"Altschul","year":"1997","journal-title":"Nucleic Acids Res"},{"key":"2023062803481013100_btaa1037-B4","doi-asserted-by":"crossref","first-page":"D158","DOI":"10.1093\/nar\/gkw1099","article-title":"UniProt: the universal protein knowledgebase","volume":"45","author":"Bateman","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2023062803481013100_btaa1037-B5","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1093\/bioinformatics\/btw694","article-title":"QAcon: single model quality assessment using protein structural and contact information with machine learning techniques","volume":"33","author":"Cao","year":"2017","journal-title":"Bioinformatics"},{"key":"2023062803481013100_btaa1037-B6","doi-asserted-by":"crossref","first-page":"1361","DOI":"10.1002\/prot.25767","article-title":"Estimation of model accuracy in CASP13","volume":"87","author":"Cheng","year":"2019","journal-title":"Proteins Struct. Funct. Bioinf"},{"key":"2023062803481013100_btaa1037-B7","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1002\/prot.21669","article-title":"Assessment of predictions in the model quality assessment category","volume":"69","author":"Cozzetto","year":"2007","journal-title":"Proteins Struct. Funct. Bioinf"},{"key":"2023062803481013100_btaa1037-B8","doi-asserted-by":"crossref","first-page":"D289","DOI":"10.1093\/nar\/gkw1098","article-title":"CATH: an expanded resource to predict protein function through structure and sequence","volume":"45","author":"Dawson","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2023062803481013100_btaa1037-B9","doi-asserted-by":"crossref","first-page":"4046","DOI":"10.1093\/bioinformatics\/bty494","article-title":"Deep convolutional networks for quality assessment of protein folds","volume":"34","author":"Derevyanko","year":"2018","journal-title":"Bioinformatics"},{"key":"2023062803481013100_btaa1037-B10","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1038\/s41467-019-11994-0","article-title":"Deep learning extends de novo protein modelling coverage of genomes using iteratively predicted structural constraints","volume":"10","author":"Greener","year":"2019","journal-title":"Nat. Commun"},{"key":"2023062803481013100_btaa1037-B11","doi-asserted-by":"crossref","first-page":"bat031","DOI":"10.1093\/database\/bat031","article-title":"The Protein Model Portal\u2014a comprehensive resource for protein structure and model information","volume":"2013","author":"Haas","year":"2013","journal-title":"Database (Oxford)"},{"key":"2023062803481013100_btaa1037-B12","first-page":"770","author":"He","year":"2016"},{"key":"2023062803481013100_btaa1037-B13","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1002\/prot.25759","article-title":"Driven to near-experimental accuracy by refinement via molecular dynamics simulations","volume":"87","author":"Heo","year":"2019","journal-title":"Proteins Struct. Funct. Bioinf"},{"key":"2023062803481013100_btaa1037-B14","doi-asserted-by":"crossref","DOI":"10.1101\/2020.07.17.209643","article-title":"Improved protein structure refinement guided by deep learning based accuracy estimation","author":"Hiranuma","year":"2020"},{"key":"2023062803481013100_btaa1037-B15","doi-asserted-by":"crossref","DOI":"10.1101\/590620","article-title":"Deep convolutional neural networks for predicting the quality of single protein structural models","author":"Hou","year":"2019"},{"key":"2023062803481013100_btaa1037-B16","article-title":"Deep transfer learning in the assessment of the quality of protein models","author":"Hurtado","year":"2018"},{"key":"2023062803481013100_btaa1037-B17","doi-asserted-by":"crossref","DOI":"10.1101\/2020.04.27.063586","article-title":"VoroCNN: deep convolutional neural network built on 3D Voronoi tessellation of protein structures","author":"Igashov","year":"2020"},{"key":"2023062803481013100_btaa1037-B18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/srep31571","article-title":"Sorting protein decoys by machine-learning-to-rank","volume":"6","author":"Jing","year":"2016","journal-title":"Sci. Rep"},{"key":"2023062803481013100_btaa1037-B19","doi-asserted-by":"crossref","first-page":"2577","DOI":"10.1002\/bip.360221211","article-title":"Dictionary of protein secondary structure: pattern recognition of hydrogen-bonded and geometrical features","volume":"22","author":"Kabsch","year":"1983","journal-title":"Biopolymers"},{"key":"2023062803481013100_btaa1037-B20","doi-asserted-by":"crossref","first-page":"2801","DOI":"10.1093\/bioinformatics\/bty1037","article-title":"Smooth orientation-dependent scoring function for coarse-grained protein quality assessment","volume":"35","author":"Karasikov","year":"2019","journal-title":"Bioinformatics"},{"key":"2023062803481013100_btaa1037-B21","doi-asserted-by":"crossref","first-page":"1021","DOI":"10.1002\/prot.25775","article-title":"CASP13 target classification into tertiary structure prediction categories","volume":"87","author":"Kinch","year":"2019","journal-title":"Proteins Struct. Funct. Bioinf"},{"key":"2023062803481013100_btaa1037-B22","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1002\/prot.25371","article-title":"Assessment of model accuracy estimations in CASP12","volume":"86","author":"Kryshtafovych","year":"2018","journal-title":"Proteins Struct. Funct. Bioinf"},{"key":"2023062803481013100_btaa1037-B23","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1002\/prot.25823","article-title":"Critical assessment of methods of protein structure prediction (CASP)\u2014Round XIII","volume":"87","author":"Kryshtafovych","year":"2019","journal-title":"Proteins Struct. Funct. Bioinf"},{"key":"2023062803481013100_btaa1037-B24","doi-asserted-by":"crossref","first-page":"W416","DOI":"10.1093\/nar\/gkx332","article-title":"ModFOLD6: an accurate web server for the global and local quality estimation of 3D protein models","volume":"45","author":"Maghrabi","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2023062803481013100_btaa1037-B25","doi-asserted-by":"crossref","first-page":"1131","DOI":"10.1002\/prot.25278","article-title":"VoroMQA: assessment of protein structure quality using interatomic contact areas","volume":"85","author":"Olechnovi\u010d","year":"2017","journal-title":"Proteins Struct. Funct. Bioinf"},{"key":"2023062803481013100_btaa1037-B26","doi-asserted-by":"crossref","first-page":"3313","DOI":"10.1093\/bioinformatics\/btz122","article-title":"Protein model quality assessment using 3D oriented convolutional neural networks","volume":"35","author":"Pag\u00e8s","year":"2019","journal-title":"Bioinformatics"},{"key":"2023062803481013100_btaa1037-B27","doi-asserted-by":"crossref","first-page":"1276","DOI":"10.1002\/prot.25784","article-title":"High-accuracy refinement using Rosetta in CASP13","volume":"87","author":"Park","year":"2019","journal-title":"Proteins Struct. Funct. Bioinf\/"},{"key":"2023062803481013100_btaa1037-B28","first-page":"8026","volume-title":"Advances in Neural Information Processing Systems 32","author":"Paszke","year":"2019"},{"key":"2023062803481013100_btaa1037-B29","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1038\/nmeth.1818","article-title":"HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment","volume":"9","author":"Remmert","year":"2012","journal-title":"Nat. Methods"},{"key":"2023062803481013100_btaa1037-B30","doi-asserted-by":"crossref","DOI":"10.1101\/2020.04.06.028266","article-title":"ProteinGCN: protein model quality assessment using graph convolutional networks","author":"Sanyal","year":"2020"},{"key":"2023062803481013100_btaa1037-B31","doi-asserted-by":"crossref","first-page":"3128","DOI":"10.1093\/bioinformatics\/btu500","article-title":"CCMpred\u2014fast and precise prediction of protein residue\u2013residue contacts from correlated mutations","volume":"30","author":"Seemayer","year":"2014","journal-title":"Bioinformatics"},{"key":"2023062803481013100_btaa1037-B32","doi-asserted-by":"crossref","first-page":"706","DOI":"10.1038\/s41586-019-1923-7","article-title":"Improved protein structure prediction using potentials from deep learning","volume":"577","author":"Senior","year":"2020","journal-title":"Nature"},{"key":"2023062803481013100_btaa1037-B33","doi-asserted-by":"crossref","first-page":"i285","DOI":"10.1093\/bioinformatics\/btaa455","article-title":"QDeep: distance-based protein model quality estimation by residue-level ensemble error classifications using stacked deep residual neural networks","volume":"36","author":"Shuvo","year":"2020","journal-title":"Bioinformatics"},{"key":"2023062803481013100_btaa1037-B34","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1186\/s12859-019-3019-7","article-title":"HH-suite3 for fast remote homology detection and deep protein annotation","volume":"20","author":"Steinegger","year":"2019","journal-title":"BMC Bioinformatics"},{"key":"2023062803481013100_btaa1037-B35","article-title":"Instance Normalization: the missing ingredient for fast stylization","author":"Ulyanov","year":"2017"},{"key":"2023062803481013100_btaa1037-B36","doi-asserted-by":"crossref","first-page":"1578","DOI":"10.1093\/bioinformatics\/btw819","article-title":"ProQ3D: improved model quality assessments using deep learning","volume":"33","author":"Uziela","year":"2017","journal-title":"Bioinformatics"},{"key":"2023062803481013100_btaa1037-B37","doi-asserted-by":"crossref","first-page":"e1005324","DOI":"10.1371\/journal.pcbi.1005324","article-title":"Accurate de novo prediction of protein contact map by ultra-deep learning model","volume":"13","author":"Wang","year":"2017","journal-title":"PLoS Comput. Biol"},{"key":"2023062803481013100_btaa1037-B38","doi-asserted-by":"crossref","first-page":"W430","DOI":"10.1093\/nar\/gkw306","article-title":"RaptorX-Property: a web server for protein structure property prediction","volume":"44","author":"Wang","year":"2016","journal-title":"Nucleic Acids Res"},{"key":"2023062803481013100_btaa1037-B39","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1002\/prot.25804","article-title":"Assessment of protein model structure accuracy estimation in CASP13: challenges in the era of deep learning","volume":"87","author":"Won","year":"2019","journal-title":"Proteins Struct. Funct. Bioinf"},{"key":"2023062803481013100_btaa1037-B40","doi-asserted-by":"crossref","first-page":"16856","DOI":"10.1073\/pnas.1821309116","article-title":"Distance-based protein folding powered by deep learning","volume":"116","author":"Xu","year":"2019","journal-title":"Proc Natl Acad Sci U S A"},{"key":"2023062803481013100_btaa1037-B41","doi-asserted-by":"crossref","first-page":"3370","DOI":"10.1093\/nar\/gkg571","article-title":"LGA: a method for finding 3D similarities in protein structures","volume":"31","author":"Zemla","year":"2003","journal-title":"Nucleic Acids Res"},{"key":"2023062803481013100_btaa1037-B42","doi-asserted-by":"crossref","first-page":"i263","DOI":"10.1093\/bioinformatics\/bty278","article-title":"Protein threading using residue co-variation and deep learning","volume":"34","author":"Zhu","year":"2018","journal-title":"Bioinformatics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btaa1037\/35176640\/btaa1037.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/22-23\/5361\/50716317\/btaa1037.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/36\/22-23\/5361\/50716317\/btaa1037.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,27]],"date-time":"2023-06-27T23:49:13Z","timestamp":1687909753000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/36\/22-23\/5361\/6039121"}},"subtitle":[],"editor":[{"given":"Arne","family":"Elofsson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2020,12,1]]},"references-count":42,"journal-issue":{"issue":"22-23","published-print":{"date-parts":[[2021,4,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btaa1037","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2020.09.30.321661","asserted-by":"object"}]},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,12,1]]},"published":{"date-parts":[[2020,12,1]]}}}