{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:51:29Z","timestamp":1760151089187,"version":"build-2065373602"},"reference-count":19,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,2,11]],"date-time":"2022-02-11T00:00:00Z","timestamp":1644537600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computation"],"abstract":"<jats:p>Confirming the result of a calculation by a calculation with a different method is often seen as a validity check. However, when the methods considered are all subject to the same (systematic) errors, this practice fails. Using a statistical approach, we define measures for reliability and similarity, and we explore the extent to which the similarity of results can help improve our judgment of the validity of data. This method is illustrated on synthetic data and applied to two benchmark datasets extracted from the literature: band gaps of solids estimated by various density functional approximations, and effective atomization energies estimated by ab initio and machine-learning methods. Depending on the levels of bias and correlation of the datasets, we found that similarity may provide a null-to-marginal improvement in reliability and was mostly effective in eliminating large errors.<\/jats:p>","DOI":"10.3390\/computation10020027","type":"journal-article","created":{"date-parts":[[2022,2,13]],"date-time":"2022-02-13T20:32:30Z","timestamp":1644784350000},"page":"27","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Should We Gain Confidence from the Similarity of Results between Methods?"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8586-6222","authenticated-orcid":false,"given":"Pascal","family":"Pernot","sequence":"first","affiliation":[{"name":"Institut de Chimie Physique, UMR8000, CNRS, Universit\u00e9 Paris-Saclay, 91405 Orsay, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8401-8037","authenticated-orcid":false,"given":"Andreas","family":"Savin","sequence":"additional","affiliation":[{"name":"Laboratoire de Chimie Th\u00e9orique, CNRS and UPMC Universit\u00e9 Paris 06, Sorbonne Universit\u00e9s, 75252 Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"164108","DOI":"10.1063\/5.0006202","article-title":"Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. I. Theory","volume":"152","author":"Pernot","year":"2020","journal-title":"J. Chem. Phys."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"164109","DOI":"10.1063\/5.0006204","article-title":"Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. II. Applications","volume":"152","author":"Pernot","year":"2020","journal-title":"J. Chem. Phys."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"241707","DOI":"10.1063\/1.5016248","article-title":"Probabilistic performance estimators for computational chemistry methods: The empirical cumulative distribution function of absolute errors","volume":"148","author":"Pernot","year":"2018","journal-title":"J. Chem. Phys."},{"key":"ref_4","unstructured":"Hausdorff, F. (1978). Set Theory, Chelsea."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1007\/s00214-021-02725-0","article-title":"Using the Gini coefficient to characterize the shape of computational chemistry error distributions","volume":"140","author":"Pernot","year":"2021","journal-title":"Theor. Chem. Acc."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1017\/S0305004100016108","article-title":"Note on Exchange Phenomena in the Thomas Atom","volume":"26","author":"Dirac","year":"1930","journal-title":"Math. Proc. Camb. Philos. Soc."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1200","DOI":"10.1139\/p80-159","article-title":"Accurate spin-dependent electron liquid correlation energies for local spin density calculations: A critical analysis","volume":"58","author":"Vosko","year":"1981","journal-title":"Can. J. Phys."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3865","DOI":"10.1103\/PhysRevLett.77.3865","article-title":"Generalized Gradient Approximation Made Simple","volume":"77","author":"Perdew","year":"1996","journal-title":"Phys. Rev. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"136406","DOI":"10.1103\/PhysRevLett.100.136406","article-title":"Restoring the Density-Gradient Expansion for Exchange in Solids and Surfaces","volume":"100","author":"Perdew","year":"2008","journal-title":"Phys. Rev. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1038\/s41524-018-0065-z","article-title":"Efficient first-principles prediction of solid stability: Towards chemical accuracy","volume":"4","author":"Zhang","year":"2018","journal-title":"Npj Comput. Mater."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"9982","DOI":"10.1063\/1.472933","article-title":"Rationale for mixing exact exchange with density functional approximations","volume":"105","author":"Perdew","year":"1996","journal-title":"J. Chem. Phys."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"6158","DOI":"10.1063\/1.478522","article-title":"Toward reliable density functional methods without adjustable parameters: The PBE0 model","volume":"110","author":"Adamo","year":"1999","journal-title":"J. Chem. Phys."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"8207","DOI":"10.1063\/1.1564060","article-title":"Hybrid functionals based on a screened Coulomb potential","volume":"118","author":"Heyd","year":"2003","journal-title":"J. Chem. Phys."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"224106","DOI":"10.1063\/1.2404663","article-title":"Influence of the exchange screening parameter on the performance of screened hybrid functionals","volume":"125","author":"Krukau","year":"2006","journal-title":"J. Chem. Phys."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"5069","DOI":"10.1021\/acs.jctc.9b00322","article-title":"Large-scale benchmark of exchange-correlation functionals for the determination of electronic band gaps of solids","volume":"15","author":"Borlido","year":"2019","journal-title":"J. Chem. Theor. Comput."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"095003","DOI":"10.1088\/1367-2630\/15\/9\/095003","article-title":"Machine learning of molecular electronic properties in chemical compound space","volume":"15","author":"Montavon","year":"2013","journal-title":"New J. Phys."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1546","DOI":"10.1021\/acs.jctc.8b00832","article-title":"Boosting Quantum Machine Learning Models with a Multilevel Combination Technique: Pople Diagrams Revisited","volume":"15","author":"Zaspel","year":"2019","journal-title":"J. Chem. Theory Comput."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"035011","DOI":"10.1088\/2632-2153\/aba184","article-title":"Impact of non-normal error distributions on the benchmarking and ranking of Quantum Machine Learning models","volume":"1","author":"Pernot","year":"2020","journal-title":"Mach. Learn. Sci. Technol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1042","DOI":"10.1002\/zaac.202000109","article-title":"Acknowledging User Requirements for Accuracy in Computational Chemistry Benchmarks","volume":"646","author":"Savin","year":"2020","journal-title":"Z. Anorg. Allg. Chem."}],"container-title":["Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-3197\/10\/2\/27\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:17:30Z","timestamp":1760134650000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-3197\/10\/2\/27"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,11]]},"references-count":19,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["computation10020027"],"URL":"https:\/\/doi.org\/10.3390\/computation10020027","relation":{},"ISSN":["2079-3197"],"issn-type":[{"type":"electronic","value":"2079-3197"}],"subject":[],"published":{"date-parts":[[2022,2,11]]}}}