{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T17:40:22Z","timestamp":1784050822110,"version":"3.55.0"},"reference-count":28,"publisher":"Oxford University Press (OUP)","issue":"21","license":[{"start":{"date-parts":[[2018,5,2]],"date-time":"2018-05-02T00:00:00Z","timestamp":1525219200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"HHMI International Early Career Scientist Program","award":["55007424"],"award-info":[{"award-number":["55007424"]}]},{"DOI":"10.13039\/501100003329","name":"MINECO","doi-asserted-by":"publisher","award":["BFU2015-68723-P"],"award-info":[{"award-number":["BFU2015-68723-P"]}],"id":[{"id":"10.13039\/501100003329","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Spanish Ministry of Economy and Competitiveness Centro de Excelencia Severo Ochoa","award":["SEV-2012-0208"],"award-info":[{"award-number":["SEV-2012-0208"]}]},{"name":"Secretaria d'Universitats i Recerca del Departament d'Economia i Coneixement de la Generalitat\u2019s AGAUR","award":["SGR 0974"],"award-info":[{"award-number":["SGR 0974"]}]},{"DOI":"10.13039\/100010663","name":"European Research Council","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]},{"name":"European Union's Seventh Framework Programme","award":["FP7\/2007-2013"],"award-info":[{"award-number":["FP7\/2007-2013"]}]},{"DOI":"10.13039\/100010663","name":"ERC","doi-asserted-by":"publisher","award":["335980_EinME"],"award-info":[{"award-number":["335980_EinME"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Russian Scientific Foundation","award":["RSF #14-24-00157"],"award-info":[{"award-number":["RSF #14-24-00157"]}]},{"name":"School of Molecular and Theoretical Biology"},{"DOI":"10.13039\/501100003002","name":"Dynasty Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>Computational prediction of the effect of mutations on protein stability is used by researchers in many fields. The utility of the prediction methods is affected by their accuracy and bias. Bias, a systematic shift of the predicted change of stability, has been noted as an issue for several methods, but has not been investigated systematically. Presence of the bias may lead to misleading results especially when exploring the effects of combination of different mutations.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Here we use a protocol to measure the bias as a function of the number of introduced mutations. It is based on a self-consistency test of the reciprocity the effect of a mutation. An advantage of the used approach is that it relies solely on crystal structures without experimentally measured stability values. We applied the protocol to four popular algorithms predicting change of protein stability upon mutation, FoldX, Eris, Rosetta and I-Mutant, and found an inherent bias. For one program, FoldX, we manage to substantially reduce the bias using additional relaxation by Modeller. Authors using algorithms for predicting effects of mutations should be aware of the bias described here.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>All calculations were implemented by in-house PERL scripts.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Note<\/jats:title>\n                  <jats:p>The article 10.1093\/bioinformatics\/bty348, published alongside this paper, also addresses the problem of biases in protein stability change predictions.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/bty340","type":"journal-article","created":{"date-parts":[[2018,4,30]],"date-time":"2018-04-30T12:01:54Z","timestamp":1525089714000},"page":"3653-3658","source":"Crossref","is-referenced-by-count":90,"title":["Self-consistency test reveals systematic bias in programs for prediction change of stability upon mutation"],"prefix":"10.1093","volume":"34","author":[{"given":"Dinara R","family":"Usmanova","sequence":"first","affiliation":[{"name":"Department of Systems Biology, Columbia University Medical Center, New York, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Natalya S","family":"Bogatyreva","sequence":"additional","affiliation":[{"name":"Bioinformatics and Genomics Programme, Centre for Genomic Regulation (CRG), Barcelona, Spain"},{"name":"Universitat Pompeu Fabra (UPF), Barcelona, Spain"},{"name":"Laboratory of Protein Physics, Institute of Protein Research of the Russian Academy of Sciences, Pushchino, Moscow Region, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joan","family":"Ari\u00f1o Bernad","sequence":"additional","affiliation":[{"name":"Centre de Formaci\u00f3 Interdisciplin\u00e0ria Superior, Universitat Polit\u00e8cnica de Catalunya, Barcelona, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aleksandra A","family":"Eremina","sequence":"additional","affiliation":[{"name":"School of Biological Sciences, College of Science and Engineering, University of Edinburgh, Edinburgh, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anastasiya A","family":"Gorshkova","sequence":"additional","affiliation":[{"name":"Biological Faculty, Lomonosov Moscow State University, Moscow, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"German M","family":"Kanevskiy","sequence":"additional","affiliation":[{"name":"Higher Chemical College of the Russian Academy of Sciences, Moscow, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lyubov R","family":"Lonishin","sequence":"additional","affiliation":[{"name":"Faculty of Technical Physics, Institute of Physics, Nanotechnology and Telecommunications, Peter the Great Saint-Petersburg Polytechnic University, Saint-Petersburg, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexander V","family":"Meister","sequence":"additional","affiliation":[{"name":"Department of Medicine, Novosibirsk State University, Novosibirsk, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alisa G","family":"Yakupova","sequence":"additional","affiliation":[{"name":"Biological Faculty, Lomonosov Moscow State University, Moscow, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fyodor A","family":"Kondrashov","sequence":"additional","affiliation":[{"name":"Institute of Science and Technology, Klosterneuburg, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dmitry N","family":"Ivankov","sequence":"additional","affiliation":[{"name":"Laboratory of Protein Physics, Institute of Protein Research of the Russian Academy of Sciences, Pushchino, Moscow Region, Russia"},{"name":"Institute of Science and Technology, Klosterneuburg, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2018,5,2]]},"reference":[{"key":"2023012712371192600_bty340-B1","doi-asserted-by":"crossref","first-page":"3389","DOI":"10.1093\/nar\/25.17.3389","article-title":"Gapped BLAST and PSI-BLAST: a new generation of protein database search programs","volume":"25","author":"Altschul","year":"1997","journal-title":"Nucleic Acids Res"},{"key":"2023012712371192600_bty340-B2","doi-asserted-by":"crossref","first-page":"1309","DOI":"10.1073\/pnas.47.9.1309","article-title":"The kinetics of formation of native ribonuclease during oxidation of the reduced polypeptide chain","volume":"47","author":"Anfinsen","year":"1961","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023012712371192600_bty340-B3","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1038\/nmeth0109-3","article-title":"Predicting free energy changes using structural ensembles","volume":"6","author":"Benedix","year":"2009","journal-title":"Nat. Methods"},{"key":"2023012712371192600_bty340-B4","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1093\/nar\/28.1.235","article-title":"The Protein Data Bank","volume":"28","author":"Berman","year":"2000","journal-title":"Nucleic Acids Res"},{"key":"2023012712371192600_bty340-B5","doi-asserted-by":"crossref","first-page":"S6.","DOI":"10.1186\/1471-2105-9-S2-S6","article-title":"A three-state prediction of single point mutations on protein stability changes","volume":"9","author":"Capriotti","year":"2008","journal-title":"BMC Bioinformatics"},{"key":"2023012712371192600_bty340-B6","doi-asserted-by":"crossref","first-page":"W306","DOI":"10.1093\/nar\/gki375","article-title":"I-Mutant2.0: predicting stability changes upon mutation from the protein sequence or structure","volume":"33","author":"Capriotti","year":"2005","journal-title":"Nucleic Acids Res"},{"key":"2023012712371192600_bty340-B7","doi-asserted-by":"crossref","first-page":"ii54","DOI":"10.1093\/bioinformatics\/bti1109","article-title":"Predicting protein stability changes from sequences using support vector machines","volume":"21","author":"Capriotti","year":"2005","journal-title":"Bioinformatics"},{"key":"2023012712371192600_bty340-B8","doi-asserted-by":"crossref","first-page":"3028","DOI":"10.1021\/ci300398z","article-title":"Accurate stabilities of laccase mutants predicted with a modified FoldX protocol","volume":"52","author":"Christensen","year":"2012","journal-title":"J. Chem. Inf. Model"},{"key":"2023012712371192600_bty340-B9","doi-asserted-by":"crossref","first-page":"3210","DOI":"10.1021\/ct4002152","article-title":"Stability mechanisms of laccase isoforms using a modified FoldX protocol applicable to widely different proteins","volume":"9","author":"Christensen","year":"2013","journal-title":"J. Chem. Theory Comput"},{"key":"2023012712371192600_bty340-B10","doi-asserted-by":"crossref","first-page":"2816","DOI":"10.1093\/bioinformatics\/btv291","article-title":"INPS: predicting the impact of non-synonymous variations on protein stability from sequence","volume":"31","author":"Fariselli","year":"2015","journal-title":"Bioinformatics"},{"key":"2023012712371192600_bty340-B11","doi-asserted-by":"crossref","first-page":"W395","DOI":"10.1093\/nar\/gkv343","article-title":"ENCoM server: exploring protein conformational space and the effect of mutations on protein function and stability","volume":"43","author":"Frappier","year":"2015","journal-title":"Nucleic Acids Res"},{"key":"2023012712371192600_bty340-B12","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1093\/protein\/13.12.849","article-title":"PoPMuSiC, an algorithm for predicting protein mutant stability changes: application to prion proteins","volume":"13","author":"Gilis","year":"2000","journal-title":"Protein Eng"},{"key":"2023012712371192600_bty340-B13","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1016\/j.molcel.2016.06.012","article-title":"Automated structure- and sequence-based design of proteins for high bacterial expression and stability","volume":"63","author":"Goldenzweig","year":"2016","journal-title":"Mol. Cell"},{"key":"2023012712371192600_bty340-B14","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1016\/S0022-2836(02)00442-4","article-title":"Predicting changes in the stability of proteins and protein complexes: a study of more than 1000 mutations","volume":"320","author":"Guerois","year":"2002","journal-title":"J. Mol. Biol"},{"key":"2023012712371192600_bty340-B15","doi-asserted-by":"crossref","first-page":"727.","DOI":"10.1002\/msb.20145092","article-title":"Structure-energy-based predictions and network modelling of RASopathy and cancer missense mutations","volume":"10","author":"Kiel","year":"2014","journal-title":"Mol. Syst. Biol"},{"key":"2023012712371192600_bty340-B16","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1093\/protein\/gzp030","article-title":"Assessing computational methods for predicting protein stability upon mutation: good on average but not in the details","volume":"22","author":"Potapov","year":"2009","journal-title":"Protein Eng. Des. Sel"},{"key":"2023012712371192600_bty340-B17","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/S0076-6879(04)83004-0","article-title":"Protein structure prediction using Rosetta","volume":"383","author":"Rohl","year":"2004","journal-title":"Methods Enzymol"},{"key":"2023012712371192600_bty340-B18","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1038\/nature17995","article-title":"Local fitness landscape of the green fluorescent protein","volume":"533","author":"Sarkisyan","year":"2016","journal-title":"Nature"},{"key":"2023012712371192600_bty340-B19","doi-asserted-by":"crossref","first-page":"W382","DOI":"10.1093\/nar\/gki387","article-title":"The FoldX web server: an online force field","volume":"33","author":"Schymkowitz","year":"2005","journal-title":"Nucleic Acids Res"},{"key":"2023012712371192600_bty340-B20","doi-asserted-by":"crossref","first-page":"2309","DOI":"10.1016\/j.bpj.2010.01.051","article-title":"Protein thermostability calculations using alchemical free energy simulations","volume":"98","author":"Seeliger","year":"2010","journal-title":"Biophys. J"},{"key":"2023012712371192600_bty340-B21","doi-asserted-by":"crossref","first-page":"E3226","DOI":"10.1073\/pnas.1412933112","article-title":"Contingency and entrenchment in protein evolution under purifying selection","volume":"112","author":"Shah","year":"2015","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023012712371192600_bty340-B22","doi-asserted-by":"crossref","first-page":"558","DOI":"10.1016\/S0959-440X(00)00131-7","article-title":"High-throughput protein crystallization","volume":"10","author":"Stevens","year":"2000","journal-title":"Curr. Opin. Struct. Biol"},{"key":"2023012712371192600_bty340-B23","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/S0065-3233(08)60401-5","article-title":"Protein denaturation","volume":"23","author":"Tanford","year":"1968","journal-title":"Adv. Protein Chem"},{"key":"2023012712371192600_bty340-B24","doi-asserted-by":"crossref","first-page":"1318","DOI":"10.1016\/j.jmb.2007.03.069","article-title":"The stability effects of protein mutations appear to be universally distributed","volume":"369","author":"Tokuriki","year":"2007","journal-title":"J. Mol. Biol"},{"key":"2023012712371192600_bty340-B25","doi-asserted-by":"crossref","first-page":"e46084.","DOI":"10.1371\/journal.pone.0046084","article-title":"Assessing predictors of changes in protein stability upon mutation using self-consistency","volume":"7","author":"Thiltgen","year":"2012","journal-title":"PLoS ONE"},{"key":"2023012712371192600_bty340-B26","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/0263-7855(90)80070-V","article-title":"WHAT IF: a molecular modeling and drug design program","volume":"8","author":"Vriend","year":"1990","journal-title":"J. Mol. Graph"},{"key":"2023012712371192600_bty340-B27","doi-asserted-by":"crossref","first-page":"5.6.1","DOI":"10.1002\/0471250953.bi0506s47","article-title":"Comparative protein structure modeling using Modeller","volume":"47","author":"Webb","year":"2014","journal-title":"Curr. Protoc. Bioinf"},{"key":"2023012712371192600_bty340-B28","doi-asserted-by":"crossref","first-page":"466","DOI":"10.1038\/nmeth0607-466","article-title":"Eris: an automated estimator of protein stability","volume":"4","author":"Yin","year":"2007","journal-title":"Nat. Methods"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/34\/21\/3653\/48921217\/bioinformatics_34_21_3653.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/34\/21\/3653\/48921217\/bioinformatics_34_21_3653.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,27]],"date-time":"2023-01-27T13:27:37Z","timestamp":1674826057000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/34\/21\/3653\/4990823"}},"subtitle":[],"editor":[{"given":"Alfonso","family":"Valencia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2018,5,2]]},"references-count":28,"journal-issue":{"issue":"21","published-print":{"date-parts":[[2018,11,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/bty340","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2018,11,1]]},"published":{"date-parts":[[2018,5,2]]}}}