{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T03:07:12Z","timestamp":1771470432010,"version":"3.50.1"},"reference-count":37,"publisher":"China Science Publishing & Media Ltd.","issue":"2","license":[{"start":{"date-parts":[[2022,3,7]],"date-time":"2022-03-07T00:00:00Z","timestamp":1646611200000},"content-version":"vor","delay-in-days":65,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Machine learning (ML) applications in weather and climate are gaining momentum as big data and the immense increase in High-performance computing (HPC) power are paving the way. Ensuring FAIR data and reproducible ML practices are significant challenges for Earth system researchers. Even though the FAIR principle is well known to many scientists, research communities are slow to adopt them. Canonical Workflow Framework for Research (CWFR) provides a platform to ensure the FAIRness and reproducibility of these practices without overwhelming researchers. This conceptual paper envisions a holistic CWFR approach towards ML applications in weather and climate, focusing on HPC and big data. Specifically, we discuss Fair Digital Object (FDO) and Research Object (RO) in the DeepRain project to achieve granular reproducibility. DeepRain is a project that aims to improve precipitation forecast in Germany by using ML. Our concept envisages the raster datacube to provide data harmonization and fast and scalable data access. We suggest the Juypter notebook as a single reproducible experiment. In addition, we envision JuypterHub as a scalable and distributed central platform that connects all these elements and the HPC resources to the researchers via an easy-to-use graphical interface.<\/jats:p>","DOI":"10.1162\/dint_a_00131","type":"journal-article","created":{"date-parts":[[2022,3,7]],"date-time":"2022-03-07T18:06:52Z","timestamp":1646676412000},"page":"271-285","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":9,"title":["HPC-oriented Canonical Workflows for Machine Learning Applications in\n                    Climate and Weather Prediction"],"prefix":"10.3724","volume":"4","author":[{"given":"Amirpasha","family":"Mozaffari","sequence":"first","affiliation":[{"name":"Forschungszentrum J\u00fclich GmbH, 52425 J\u00fclich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Langguth","sequence":"additional","affiliation":[{"name":"Forschungszentrum J\u00fclich GmbH, 52425 J\u00fclich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bing","family":"Gong","sequence":"additional","affiliation":[{"name":"Forschungszentrum J\u00fclich GmbH, 52425 J\u00fclich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jessica","family":"Ahring","sequence":"additional","affiliation":[{"name":"Forschungszentrum J\u00fclich GmbH, 52425 J\u00fclich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adrian Rojas","family":"Campos","sequence":"additional","affiliation":[{"name":"Osnabr\u00fcck University, 49074 Osnabr\u00fcck, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pascal","family":"Nieters","sequence":"additional","affiliation":[{"name":"Osnabr\u00fcck University, 49074 Osnabr\u00fcck, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Otoniel Jos\u00e9 Campos","family":"Escobar","sequence":"additional","affiliation":[{"name":"Jacobs University Bremen, 28759 Bremen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Martin","family":"Wittenbrink","sequence":"additional","affiliation":[{"name":"Deutscher Wetterdienst, 63067 Offenbach am Main, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Baumann","sequence":"additional","affiliation":[{"name":"Jacobs University Bremen, 28759 Bremen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Martin G.","family":"Schultz","sequence":"additional","affiliation":[{"name":"Forschungszentrum J\u00fclich GmbH, 52425 J\u00fclich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"2026","published-online":{"date-parts":[[2022,4,1]]},"reference":[{"key":"2022042714422121500_ref1","doi-asserted-by":"crossref","DOI":"10.1038\/sdata.2016.18","article-title":"Comment: The FAIR guiding principles for scientific data\n                        management and stewardship","volume":"3","author":"Wilkinson","year":"2016","journal-title":"Scientific Data"},{"issue":"2","key":"2022042714422121500_ref2","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/s00799-005-0128-x","article-title":"A framework for distributed digital object\n                        services","volume":"6","author":"Kahn","year":"2006","journal-title":"International Journal on Digital\n                        Libraries"},{"issue":"2","key":"2022042714422121500_ref3","doi-asserted-by":"crossref","DOI":"10.3390\/publications8020021","article-title":"FAIR digital objects for science: From data pieces to\n                        actionable knowledge units","volume":"8","author":"De\n                                Smedt","year":"2020","journal-title":"Publications"},{"issue":"1","key":"2022042714422121500_ref4","doi-asserted-by":"crossref","first-page":"37","DOI":"10.3233\/DS-190026","article-title":"Towards FAIR principles for research\n                        software","volume":"3","author":"Lamprecht","year":"2019","journal-title":"Data Science"},{"issue":"10","key":"2022042714422121500_ref5","doi-asserted-by":"crossref","first-page":"e1003285","DOI":"10.1371\/journal.pcbi.1003285","article-title":"Ten simple rules for reproducible computational\n                        research","volume":"9","author":"Sandve","year":"2013","journal-title":"PLoS Computational Biology"},{"issue":"10","key":"2022042714422121500_ref6","doi-asserted-by":"crossref","first-page":"388","DOI":"10.1002\/2015EA000136","article-title":"Toward the geoscience paper of the future: Best practices for\n                        documenting and sharing research from data to software to\n                        provenance","volume":"3","author":"Gil","year":"2016","journal-title":"Earth and Space Science"},{"issue":"3","key":"2022042714422121500_ref7","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1093\/biostatistics\/kxq028","article-title":"An invitation to reproducible computational\n                        research","volume":"11","author":"Donoho","year":"2010","journal-title":"Biostatistics"},{"key":"2022042714422121500_ref8","volume-title":"Improving reproducibility in machine learning research (A report\n                        from the NEURIPS 2019 Reproducibility Program)","author":"Pineau","year":"2020"},{"key":"2022042714422121500_ref9","volume-title":"Canonical Workflow Framework for Research (CWFR) \u2013 position\n                        paper \u2013 version 2 December 2020","author":"Hardisty"},{"key":"2022042714422121500_ref10","first-page":"669","volume-title":"Enabling interactive supercomputing at JSC Lessons\n                    Learned","author":"Gobbert","year":"2018"},{"issue":"7567","key":"2022042714422121500_ref11","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1038\/nature14956","article-title":"The quiet revolution of numerical weather\n                        prediction","volume":"525","author":"Bauer","year":"2015","journal-title":"Nature"},{"issue":"7743","key":"2022042714422121500_ref12","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1038\/s41586-019-0912-1","article-title":"Prabhat, deep learning and process understanding for\n                        data-driven earth system science","volume":"566","author":"Reichstein","year":"2019","journal-title":"Nature"},{"key":"2022042714422121500_ref13","volume-title":"Is there a reproducibility crisis in science? Nature\n                    (2016)","author":"Stoddart"},{"key":"2022042714422121500_ref14","volume-title":"A practical taxonomy of reproducibility for machine learning\n                        research","author":"Tatman"},{"issue":"4","key":"2022042714422121500_ref15","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1001\/jama.2019.20866","article-title":"Challenges to the reproducibility of machine learning models\n                        in health care","volume":"323","author":"Beam","year":"2020","journal-title":"JAMA"},{"key":"2022042714422121500_ref16","volume-title":"TensorFlow","author":"Developers"},{"key":"2022042714422121500_ref17","first-page":"1","volume-title":"et al.: PyTorch: An imperative style, high-performance deep learning\n                        library","author":"Paszke","year":"2019"},{"key":"2022042714422121500_ref18","volume-title":"DeepRain"},{"key":"2022042714422121500_ref19","volume-title":"A description of the nonhydrostatic regional COSMO model","author":"Doms"},{"issue":"2-3","key":"2022042714422121500_ref20","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.atmosres.2010.12.008","article-title":"Uncertainties in COSMO-DE precipitation forecasts introduced\n                        by model perturbations and variation of lateral boundaries","volume":"100","author":"Gebhardt","year":"2011","journal-title":"Atmospheric Research"},{"issue":"D7","key":"2022042714422121500_ref21","doi-asserted-by":"crossref","DOI":"10.1029\/2011JD016581","article-title":"Accounting for initial condition uncertainties in\n                        COSMO-DE-EPS","volume":"117","author":"Peralta","year":"2012","journal-title":"Journal of Geophysical Research:\n                        Atmospheres"},{"key":"2022042714422121500_ref22","volume-title":"Erstellung einer radargest\u00fctzten\n                        Niederschlagsklimatologie","author":"Winterrath"},{"key":"2022042714422121500_ref23","first-page":"132","volume-title":"An overview of the new radar-based precipitation climatology of the\n                        Deutscher Wetterdienst\u2014data, methods, products","author":"Winterrath","year":"2019"},{"key":"2022042714422121500_ref24","volume-title":"Common patterns in revolutionary Infrastructures and data","author":"Wittenburg"},{"issue":"3","key":"2022042714422121500_ref25","doi-asserted-by":"crossref","DOI":"10.1145\/3186266","article-title":"Reproducibility in scientific computing","volume":"51","author":"Ivie","year":"2019","journal-title":"ACM Computing Surveys"},{"issue":"1-2","key":"2022042714422121500_ref26","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1162\/dint_a_00034","article-title":"FAIR data and services in biodiversity science and\n                        geoscience","volume":"2","author":"Lannom","year":"2020","journal-title":"Data Intelligence"},{"key":"2022042714422121500_ref27","volume-title":"Research objects: Towards exchange and reuse of digital\n                        knowledge","author":"Bechhofer"},{"issue":"1-2","key":"2022042714422121500_ref28","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1162\/dint_a_00026","article-title":"Making data and workflows findable for\n                        machines","volume":"2","author":"Weigel","year":"2020","journal-title":"Data Intelligence"},{"issue":"2","key":"2022042714422121500_ref29","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1109\/MCSE.2021.3052101","article-title":"Using Jupyter for reproducible scientific\n                        workflows","volume":"23","author":"Beg","year":"2021","journal-title":"Computing in Science and\n                        Engineering"},{"key":"2022042714422121500_ref30","first-page":"47","volume-title":"Towards interactive, reproducible analytics at scale on HPC\n                        system","author":"Cholia","year":"2020"},{"key":"2022042714422121500_ref31","volume-title":"Pro Git: Everything you need to know about Git, Apress","author":"Chacon"},{"issue":"1","key":"2022042714422121500_ref32","doi-asserted-by":"crossref","DOI":"10.1186\/1751-0473-8-7","article-title":"Git can facilitate greater reproducibility and increased\n                        transparency in science","volume":"8","author":"Ram","year":"2013","journal-title":"Source Code for Biology and\n                        Medicine"},{"key":"2022042714422121500_ref33","volume-title":"Packaging research artefacts with RO-Crate","author":"Soiland-Reyes","year":"2021"},{"key":"2022042714422121500_ref34","first-page":"575","article-title":"The multidimensional database system","volume":"27","author":"Baumann","year":"1998","journal-title":"RasDaMan"},{"key":"2022042714422121500_ref35","doi-asserted-by":"crossref","DOI":"10.1186\/s40537-020-00399-2","article-title":"Array databases: Concepts, standards,\n                        implementations","volume":"8","author":"Baumann","year":"2021","journal-title":"Journal of Big Data"},{"key":"2022042714422121500_ref36","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1007\/s10707-009-0087-2","article-title":"The OGC Web coverage processing service (WCPS)\n                        standard","volume":"14","author":"Baumann","year":"2010","journal-title":"GeoInformatica"},{"key":"2022042714422121500_ref37","first-page":"1","article-title":"JUWELS: Modular Tier-0\/1 supercomputer at J\u00fclich\n                        Supercomputing Centre","volume":"5","author":"Support","year":"2019","journal-title":"Journal of Large-scale\n                        Research Facilities"}],"container-title":["Data Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/dint\/article-pdf\/4\/2\/271\/2012379\/dint_a_00131.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/dint\/article-pdf\/4\/2\/271\/2012379\/dint_a_00131.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,14]],"date-time":"2025-03-14T07:41:19Z","timestamp":1741938079000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.sciengine.com\/doi\/10.1162\/dint_a_00131"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022]]},"references-count":37,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,4,1]]}},"URL":"https:\/\/doi.org\/10.1162\/dint_a_00131","relation":{},"ISSN":["2641-435X"],"issn-type":[{"value":"2641-435X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022]]},"published":{"date-parts":[[2022]]}}}