{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T18:54:13Z","timestamp":1788288853786,"version":"build-2803163510"},"reference-count":24,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2018,8,29]],"date-time":"2018-08-29T00:00:00Z","timestamp":1535500800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100004504","name":"Research Council of Lithuania","doi-asserted-by":"publisher","award":["S-MIP-17-60"],"award-info":[{"award-number":["S-MIP-17-60"]}],"id":[{"id":"10.13039\/501100004504","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000057","name":"US National Institute of General Medical Sciences","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000057","name":"NIGMS","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["GM100482"],"award-info":[{"award-number":["GM100482"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,3,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Measuring discrepancies between protein models and native structures is at the heart of development of protein structure prediction methods and comparison of their performance. A number of different evaluation methods have been developed; however, their comprehensive and unbiased comparison has not been performed.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We carried out a comparative analysis of several popular model assessment methods (RMSD, TM-score, GDT, QCS, CAD-score, LDDT, SphereGrinder and RPF) to reveal their relative strengths and weaknesses. The analysis, performed on a large and diverse model set derived in the course of three latest community-wide CASP experiments (CASP10\u201312), had two major directions. First, we looked at general differences between the scores by analyzing distribution, correspondence and correlation of their values as well as differences in selecting best models. Second, we examined the score differences taking into account various structural properties of models (stereochemistry, hydrogen bonds, packing of domains and chain fragments, missing residues, protein length and secondary structure). Our results provide a solid basis for an informed selection of the most appropriate score or combination of scores depending on the task at hand.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Supplementary information<\/jats:title>\n                    <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/bty760","type":"journal-article","created":{"date-parts":[[2018,8,29]],"date-time":"2018-08-29T15:37:49Z","timestamp":1535557069000},"page":"937-944","source":"Crossref","is-referenced-by-count":54,"title":["Comparative analysis of methods for evaluation of protein models against native structures"],"prefix":"10.1093","volume":"35","author":[{"given":"Kliment","family":"Olechnovi\u010d","sequence":"first","affiliation":[{"name":"Institute of Biotechnology Life Sciences Center Vilnius University, Saul\u0117tekio 7, Vilnius, Lithuania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bohdan","family":"Monastyrskyy","sequence":"additional","affiliation":[{"name":"Genome Center UC Davis, 451 Health Sciences Drive, Davis, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andriy","family":"Kryshtafovych","sequence":"additional","affiliation":[{"name":"Genome Center UC Davis, 451 Health Sciences Drive, Davis, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"\u010ceslovas","family":"Venclovas","sequence":"additional","affiliation":[{"name":"Institute of Biotechnology Life Sciences Center Vilnius University, Saul\u0117tekio 7, Vilnius, Lithuania"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2018,8,29]]},"reference":[{"key":"2023013107264708200_bty760-B1","doi-asserted-by":"crossref","first-page":"899","DOI":"10.1038\/nmeth.3587","article-title":"Association, correlation and causation","volume":"12","author":"Altman","year":"2015","journal-title":"Nat. Methods"},{"key":"2023013107264708200_bty760-B2","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1107\/S0907444909042073","article-title":"MolProbity: all-atom structure validation for macromolecular crystallography","volume":"66","author":"Chen","year":"2010","journal-title":"Acta Crystallogr. D Biol. Crystallogr"},{"key":"2023013107264708200_bty760-B3","doi-asserted-by":"crossref","first-page":"3371","DOI":"10.1093\/bioinformatics\/btr572","article-title":"An automatic method for CASP9 free modeling structure prediction assessment","volume":"27","author":"Cong","year":"2011","journal-title":"Bioinformatics"},{"key":"2023013107264708200_bty760-B4","first-page":"507","article-title":"Frequency distribution of the values of the correlation coefficient in samples of an indefinitely large population","volume":"10","author":"Fisher","year":"1915","journal-title":"Biometrika"},{"key":"2023013107264708200_bty760-B5","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1002\/prot.25431","article-title":"Continuous Automated Model EvaluatiOn (CAMEO) complementing the critical assessment of structure prediction in CASP12","volume":"86","author":"Haas","year":"2018","journal-title":"Proteins"},{"key":"2023013107264708200_bty760-B6","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1002\/prot.24488","article-title":"Assessment of template-based protein structure predictions in CASP10","volume":"82","author":"Huang","year":"2014","journal-title":"Proteins"},{"key":"2023013107264708200_bty760-B7","doi-asserted-by":"crossref","first-page":"W542","DOI":"10.1093\/nar\/gks373","article-title":"RPF: a quality assessment tool for protein NMR structures","volume":"40","author":"Huang","year":"2012","journal-title":"Nucleic Acids Res"},{"key":"2023013107264708200_bty760-B8","doi-asserted-by":"crossref","first-page":"922","DOI":"10.1107\/S0567739476001873","article-title":"A solution for the best rotation to relate two sets of vectors","volume":"32","author":"Kabsch","year":"1976","journal-title":"Acta Crystallogr. A"},{"key":"2023013107264708200_bty760-B9","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1002\/prot.23190","article-title":"CASP9 target classification","volume":"79","author":"Kinch","year":"2011","journal-title":"Proteins"},{"key":"2023013107264708200_bty760-B10","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1002\/prot.24399","article-title":"CASP prediction center infrastructure and evaluation measures in CASP10 and CASP ROLL","volume":"82","author":"Kryshtafovych","year":"2014","journal-title":"Proteins"},{"key":"2023013107264708200_bty760-B11","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1007\/978-1-61779-588-6_10","article-title":"Methods of protein structure comparison","volume":"857","author":"Kufareva","year":"2012","journal-title":"Methods Mol. Biol"},{"key":"2023013107264708200_bty760-B12","doi-asserted-by":"crossref","first-page":"5913","DOI":"10.1073\/pnas.95.11.5913","article-title":"A unified statistical framework for sequence comparison and structure comparison","volume":"95","author":"Levitt","year":"1998","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023013107264708200_bty760-B13","first-page":"665","volume-title":"2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)","author":"Lukasiak","year":"2015"},{"key":"2023013107264708200_bty760-B14","doi-asserted-by":"crossref","first-page":"1233","DOI":"10.1080\/03610927808827707","article-title":"Some properties of clasical multi-dimensional scaling","volume":"7","author":"Mardia","year":"1978","journal-title":"Commun. Stat"},{"key":"2023013107264708200_bty760-B15","doi-asserted-by":"crossref","first-page":"2722","DOI":"10.1093\/bioinformatics\/btt473","article-title":"lDDT: a local superposition-free score for comparing protein structures and models using distance difference tests","volume":"29","author":"Mariani","year":"2013","journal-title":"Bioinformatics"},{"key":"2023013107264708200_bty760-B16","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1006\/jmbi.1994.1334","article-title":"Satisfying hydrogen bonding potential in proteins","volume":"238","author":"McDonald","year":"1994","journal-title":"J. Mol. Biol"},{"key":"2023013107264708200_bty760-B17","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1002\/prot.25415","article-title":"Critical assessment of methods of protein structure prediction (CASP)-Round XII","volume":"86","author":"Moult","year":"2018","journal-title":"Proteins"},{"key":"2023013107264708200_bty760-B18","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1002\/prot.24172","article-title":"CAD-score: a new contact area difference-based function for evaluation of protein structural models","volume":"81","author":"Olechnovi\u010d","year":"2013","journal-title":"Proteins"},{"key":"2023013107264708200_bty760-B19","doi-asserted-by":"crossref","first-page":"W259","DOI":"10.1093\/nar\/gku294","article-title":"The CAD-score web server: contact area-based comparison of structures and interfaces of proteins, nucleic acids and their complexes","volume":"42","author":"Olechnovi\u010d","year":"2014","journal-title":"Nucleic Acids Res"},{"key":"2023013107264708200_bty760-B20","doi-asserted-by":"crossref","first-page":"1259","DOI":"10.1093\/bioinformatics\/btp148","article-title":"Structure similarity measure with penalty for close non-equivalent residues","volume":"25","author":"Sadreyev","year":"2009","journal-title":"Bioinformatics"},{"key":"2023013107264708200_bty760-B21","doi-asserted-by":"crossref","first-page":"654","DOI":"10.1002\/prot.25492","article-title":"Improved protein model quality assessments by changing the target function","volume":"86","author":"Uziela","year":"2018","journal-title":"Proteins"},{"key":"2023013107264708200_bty760-B22","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1002\/(SICI)1097-0134(1999)37:3+<22::AID-PROT5>3.0.CO;2-W","article-title":"Processing and analysis of CASP3 protein structure predictions","author":"Zemla","year":"1999","journal-title":"Proteins"},{"key":"2023013107264708200_bty760-B23","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1002\/prot.10052","article-title":"Processing and evaluation of predictions in CASP4","author":"Zemla","year":"2001","journal-title":"Proteins"},{"key":"2023013107264708200_bty760-B24","doi-asserted-by":"crossref","first-page":"702","DOI":"10.1002\/prot.20264","article-title":"Scoring function for automated assessment of protein structure template quality","volume":"57","author":"Zhang","year":"2004","journal-title":"Proteins"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/6\/937\/48967778\/bioinformatics_35_6_937.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/6\/937\/48967778\/bioinformatics_35_6_937.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T05:24:55Z","timestamp":1675142695000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/35\/6\/937\/5086387"}},"subtitle":[],"editor":[{"given":"Alfonso","family":"Valencia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2018,8,29]]},"references-count":24,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2019,3,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/bty760","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,3,15]]},"published":{"date-parts":[[2018,8,29]]}}}