{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,27]],"date-time":"2026-06-27T07:12:59Z","timestamp":1782544379786,"version":"3.54.5"},"reference-count":12,"publisher":"SAGE Publications","issue":"5","license":[{"start":{"date-parts":[[2020,3,20]],"date-time":"2020-03-20T00:00:00Z","timestamp":1584662400000},"content-version":"vor","delay-in-days":366,"URL":"http:\/\/www.sagepub.com\/licence-information-for-chorus"}],"funder":[{"DOI":"10.13039\/100006206","name":"Biological and Environmental Research","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006206","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2019,9]]},"abstract":"<jats:p>We present a methodology for solution reproducibility for the Energy Exascale Earth System Model during its ongoing software infrastructure development to prepare for exascale computers. The nonlinear chaotic nature of climate system simulations precludes traditional model verification approaches since machine precision differences\u2014resulting from code refactoring, changes in software environment, and so on\u2014grow exponentially to a different weather state. Here, we leverage the nature of climate as a statistical description of the atmosphere in order to establish model reproducibility. We evaluate the degree to which two-sample equality of distribution tests can confidently detect the change in climate from minor tuning parameter changes on model output variables in order to establish the level of difference that indicates a new climate. To apply this (baselined test), we target a section of the model\u2019s development cycle wherein no intentional science changes have been applied to its source code. We compare an ensemble of short simulations that were conducted using a verified model configuration against a new ensemble with the same configuration but with the latest software infrastructure (Common Infrastructure for Modeling the Earth, CIME5.0), compiler versions, and software libraries. We also compare these against ensemble simulations conducted using the original version of the software infrastructure (CIME4.0) of the earlier model configuration, but with the latest compilers and software libraries, to test the impact of new compilers and libraries in isolation from additional software infrastructure. The two-sample equality of distribution tests indicates that these ensembles indeed represent the same climate.<\/jats:p>","DOI":"10.1177\/1094342019837341","type":"journal-article","created":{"date-parts":[[2019,3,21]],"date-time":"2019-03-21T00:53:11Z","timestamp":1553129591000},"page":"784-790","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":7,"title":["Ongoing solution reproducibility of earth system models as they progress toward exascale computing"],"prefix":"10.1177","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5767-8590","authenticated-orcid":false,"given":"Salil","family":"Mahajan","sequence":"first","affiliation":[{"name":"Computational Earth Sciences, Oak Ridge National Laboratory, Oak Ridge, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8174-6450","authenticated-orcid":false,"given":"Katherine J","family":"Evans","sequence":"additional","affiliation":[{"name":"Computational Earth Sciences, Oak Ridge National Laboratory, Oak Ridge, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9348-693X","authenticated-orcid":false,"given":"Joseph H","family":"Kennedy","sequence":"additional","affiliation":[{"name":"Computational Earth Sciences, Oak Ridge National Laboratory, Oak Ridge, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min","family":"Xu","sequence":"additional","affiliation":[{"name":"Computational Earth Sciences, Oak Ridge National Laboratory, Oak Ridge, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mathew R","family":"Norman","sequence":"additional","affiliation":[{"name":"Computational Earth Sciences, Oak Ridge National Laboratory, Oak Ridge, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marcia L","family":"Branstetter","sequence":"additional","affiliation":[{"name":"Computational Earth Sciences, Oak Ridge National Laboratory, Oak Ridge, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2019,3,20]]},"reference":[{"key":"bibr1-1094342019837341","doi-asserted-by":"crossref","unstructured":"Baker AH, Hammerling DM, Levy MN, et al. (2015) A new ensemble-based consistency test for the community earth system model (pycect v1.0). Geoscientific Model Development 8(9): 2829\u20132840.","DOI":"10.5194\/gmd-8-2829-2015"},{"key":"bibr2-1094342019837341","doi-asserted-by":"publisher","DOI":"10.1002\/2014MS000329"},{"key":"bibr3-1094342019837341","doi-asserted-by":"publisher","DOI":"10.1175\/JCLI-D-15-0255.1"},{"key":"bibr4-1094342019837341","doi-asserted-by":"publisher","DOI":"10.1029\/2018JD028892"},{"key":"bibr5-1094342019837341","doi-asserted-by":"publisher","DOI":"10.1002\/2017MS000916"},{"key":"bibr6-1094342019837341","doi-asserted-by":"publisher","DOI":"10.1175\/1520-0469(1963)020<0130:DNF>2.0.CO;2"},{"key":"bibr7-1094342019837341","doi-asserted-by":"crossref","unstructured":"Mahajan S, Gaddis AL, Evans KJ, et al. (2017) Exploring an ensemble-based approach to atmospheric climate modeling and testing at scale. Procedia Computer Science 108: 735\u2013744. DOI: 10.1016\/j.procs.2017.05.259. Available at: http:\/\/www.sciencedirect.com\/science\/article\/pii\/S1877050917308906. International Conference on Computational Science, ICCS 2017, 12-14 June 2017, Zurich, Switzerland.","DOI":"10.1016\/j.procs.2017.05.259"},{"key":"bibr8-1094342019837341","unstructured":"Neale RB, Richter JH, Conley AJ, et al. (2010) Description of the NCAR community atmosphere model (cam4), tech. rep. ncar\/tn+str. Technical report, National Center for Atmospheric Research, Boulder, CO."},{"key":"bibr9-1094342019837341","doi-asserted-by":"publisher","DOI":"10.1137\/S1064827594275534"},{"key":"bibr10-1094342019837341","author":"Terai CR","year":"2017","journal-title":"Climate Dynamics"},{"key":"bibr11-1094342019837341","doi-asserted-by":"crossref","unstructured":"Yang B, Qian Y, Lin G, et al. (2013) Uncertainty quantification and parameter tuning in the cam5 zhang-mcfarlane convection scheme and impact of improved convection on the global circulation and climate. Journal of Geophysical Research: Atmospheres 118(2): 395\u2013415. DOI: 10.1029\/2012JD018213. Available at: https:\/\/agupubs.onlinelibrary.wiley.com\/doi\/abs\/10.1029\/2012JD018213.","DOI":"10.1029\/2012JD018213"},{"key":"bibr12-1094342019837341","doi-asserted-by":"publisher","DOI":"10.1080\/07055900.1995.9649539"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342019837341","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/1094342019837341","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342019837341","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342019837341","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T08:15:47Z","timestamp":1777450547000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1094342019837341"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,3,20]]},"references-count":12,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2019,9]]}},"alternative-id":["10.1177\/1094342019837341"],"URL":"https:\/\/doi.org\/10.1177\/1094342019837341","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"value":"1094-3420","type":"print"},{"value":"1741-2846","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,3,20]]}}}