{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T15:34:15Z","timestamp":1786980855197,"version":"build-2736575974"},"reference-count":56,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T00:00:00Z","timestamp":1667779200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T00:00:00Z","timestamp":1667779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Cluster Comput"],"published-print":{"date-parts":[[2023,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Deep learning has been postulated as a solution for numerous problems in different branches of science. Given the resource-intensive nature of these models, they often need to be executed on specialized hardware such graphical processing units (GPUs) in a distributed manner. In the academic field, researchers get access to this kind of resources through High Performance Computing (HPC) clusters. This kind of infrastructures make the training of these models difficult due to their multi-user nature and limited user permission. In addition, different HPC clusters may possess different peculiarities that can entangle the research cycle (e.g., libraries dependencies). In this paper we develop a workflow and methodology for the distributed training of deep learning models in HPC clusters which provides researchers with a series of novel advantages. It relies on udocker as containerization tool and on Horovod as library for the distribution of the models across multiple GPUs. udocker does not need any special permission, allowing researchers to run the entire workflow without relying on any administrator. Horovod ensures the efficient distribution of the training independently of the deep learning framework used. Additionally, due to containerization and specific features of the workflow, it provides researchers with a cluster-agnostic way of running their models. The experiments carried out show that the workflow offers good scalability in the distributed training of the models and that it easily adapts to different clusters.<\/jats:p>","DOI":"10.1007\/s10586-022-03798-7","type":"journal-article","created":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T17:06:44Z","timestamp":1667840804000},"page":"2815-2834","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["A container-based workflow for distributed training of deep learning algorithms in HPC clusters"],"prefix":"10.1007","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0764-3872","authenticated-orcid":false,"given":"Jose","family":"Gonz\u00e1lez-Abad","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0013-4602","authenticated-orcid":false,"given":"\u00c1lvaro","family":"L\u00f3pez Garc\u00eda","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8770-3619","authenticated-orcid":false,"given":"Valentin Y.","family":"Kozlov","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,11,7]]},"reference":[{"key":"3798_CR1","volume-title":"Deep learning","author":"I Goodfellow","year":"2006","unstructured":"Goodfellow, I., Bengio, Y., Courville, A.: Deep learning. MIT Press, Cambridge (2006)"},{"issue":"1","key":"3798_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/ncomms5308","volume":"5","author":"P Baldi","year":"2014","unstructured":"Baldi, P., Sadowski, P., Whiteson, D.: Searching for exotic particles in high-energy physics with deep learning. Nat. Commun. 5(1), 1\u20139 (2014). https:\/\/doi.org\/10.1038\/ncomms5308","journal-title":"Nat. Commun."},{"issue":"7","key":"3798_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/JHEP07(2016)069","volume":"2016","author":"L de Oliveira","year":"2016","unstructured":"de Oliveira, L., Kagan, M., Mackey, L., Nachman, B., Schwartzman, A.: Jet-images\u2013deep learning edition. J. High Energy Phys. 2016(7), 1\u201332 (2016). https:\/\/doi.org\/10.1007\/JHEP07(2016)069","journal-title":"J. High Energy Phys."},{"issue":"1","key":"3798_CR4","doi-asserted-by":"publisher","first-page":"894","DOI":"10.1093\/mnras\/stx3186","volume":"475","author":"D Tuccillo","year":"2018","unstructured":"Tuccillo, D., Huertas-Company, M., Decenci\u00e8re, E., Velasco-Forero, S., Dom\u00ednguez S\u00e1nchez, H., Dimauro, P.: Deep learning for galaxy surface brightness profile fitting. Mon. Not. R. Astron. Soc. 475(1), 894\u2013909 (2018). https:\/\/doi.org\/10.1093\/mnras\/stx3186","journal-title":"Mon. Not. R. Astron. Soc."},{"issue":"2","key":"3798_CR5","doi-asserted-by":"publisher","first-page":"114","DOI":"10.3847\/1538-4357\/aabfed","volume":"858","author":"J Primack","year":"2018","unstructured":"Primack, J., Dekel, A., Koo, D., Lapiner, S., Ceverino, D., Simons, R., Snyder, G., Bernardi, M., Chen, Z., Dom\u00ednguez-S\u00e1nchez, H., et al.: Deep learning identifies high-z galaxies in a central blue nugget phase in a characteristic mass range. Astrophys. J. 858(2), 114 (2018)","journal-title":"Astrophys. J."},{"issue":"7873","key":"3798_CR6","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","volume":"596","author":"J Jumper","year":"2021","unstructured":"Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., \u017d\u00eddek, A., Potapenko, A., et al.: Highly accurate protein structure prediction with alphafold. Nature 596(7873), 583\u2013589 (2021). https:\/\/doi.org\/10.1038\/s41586-021-03819-2","journal-title":"Nature"},{"issue":"22","key":"3798_CR7","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1029\/2018GL080704","volume":"45","author":"S Scher","year":"2018","unstructured":"Scher, S.: Toward data-driven weather and climate forecasting: Approximating a simple general circulation model with deep learning. Geophys. Res. Lett. 45(22), 12\u2013616 (2018). https:\/\/doi.org\/10.1029\/2018GL080704","journal-title":"Geophys. Res. Lett."},{"issue":"39","key":"3798_CR8","doi-asserted-by":"publisher","first-page":"9684","DOI":"10.1073\/pnas.1810286115","volume":"115","author":"S Rasp","year":"2018","unstructured":"Rasp, S., Pritchard, M.S., Gentine, P.: Deep learning to represent subgrid processes in climate models. Proc. Natl. Acad. Sci. 115(39), 9684\u20139689 (2018). https:\/\/doi.org\/10.1073\/pnas.1810286115","journal-title":"Proc. Natl. Acad. Sci."},{"key":"3798_CR9","doi-asserted-by":"publisher","first-page":"18681","DOI":"10.1109\/ACCESS.2020.2964386","volume":"8","author":"\u00c1 L\u00f3pez Garc\u00eda","year":"2020","unstructured":"L\u00f3pez Garc\u00eda, \u00c1., Marco de Lucas, J., Antonacci, M., Zu Castell, W., David, M., Hardt, M., Lloret Iglesias, L., Molt\u00f3, G., Plociennik, M., Tran, V., Alic, A.S., Caballer, M., Campos Plasencia, I., Costantini, A., Dlugolinsky, S., Duma, D.C., Donvito, G., Gomes, J., Heredia Cacha, I., Ito, K., Kozlov, V.Y., Nguyen, G., Orviz Fern\u00e1ndez, P., \u0160ustr, Z., Wolniewicz, P.: A cloud-based framework for machine learning workloads and applications. IEEE Access 8, 18681\u201318692 (2020). https:\/\/doi.org\/10.1109\/ACCESS.2020.2964386","journal-title":"IEEE Access"},{"key":"3798_CR10","first-page":"2","volume":"239","author":"D Merkel","year":"2014","unstructured":"Merkel, D.: Docker: lightweight linux containers for consistent development and deployment. Linux J. 239, 2 (2014)","journal-title":"Linux J."},{"issue":"5","key":"3798_CR11","doi-asserted-by":"publisher","first-page":"0177459","DOI":"10.1371\/journal.pone.0177459","volume":"12","author":"GM Kurtzer","year":"2017","unstructured":"Kurtzer, G.M., Sochat, V., Bauer, M.W.: Singularity: scientific containers for mobility of compute. PLoS ONE 12(5), 0177459 (2017). https:\/\/doi.org\/10.1371\/journal.pone.0177459","journal-title":"PLoS ONE"},{"key":"3798_CR12","doi-asserted-by":"publisher","unstructured":"Priedhorsky, R., Randles, T.: Charliecloud: unprivileged containers for user-defined software stacks in hpc. In: Proceedings of the international conference for high performance computing, networking, storage and analysis, pp. 1\u201310 (2017). https:\/\/doi.org\/10.1145\/3126908.3126925","DOI":"10.1145\/3126908.3126925"},{"key":"3798_CR13","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/898\/8\/082021","volume":"898","author":"L Gerhardt","year":"2017","unstructured":"Gerhardt, L., Bhimji, W., Canon, S., Fasel, M., Jacobsen, D., Mustafa, M., Porter, J., Tsulaia, V.: Shifter: containers for hpc. J. Phys.: Conf. Ser. 898, 082021 (2017). https:\/\/doi.org\/10.1088\/1742-6596\/898\/8\/082021","journal-title":"J. Phys.: Conf. Ser."},{"key":"3798_CR14","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1016\/j.cpc.2018.05.021","volume":"232","author":"J Gomes","year":"2018","unstructured":"Gomes, J., Bagnaschi, E., Campos, I., David, M., Alves, L., Martins, J., Pina, J., Lopez-Garcia, A., Orviz, P.: Enabling rootless linux containers in multi-user environments: the udocker tool. Comput. Phys. Commun. 232, 84\u201397 (2018). https:\/\/doi.org\/10.1016\/j.cpc.2018.05.021","journal-title":"Comput. Phys. Commun."},{"key":"3798_CR15","doi-asserted-by":"publisher","unstructured":"Younge, A.J., Pedretti, K., Grant, R.E., Brightwell, R.: A tale of two systems: using containers to deploy hpc applications on supercomputers and clouds. In: 2017 IEEE international conference on cloud computing technology and science (CloudCom), pp. 74\u201381. , IEEE (2017). https:\/\/doi.org\/10.1109\/CloudCom.2017.40.","DOI":"10.1109\/CloudCom.2017.40."},{"key":"3798_CR16","doi-asserted-by":"publisher","unstructured":"Brayford, D., Vallecorsa, S., Atanasov, A., Baruffa, F., Riviera, W.: Deploying AI frameworks on secure HPC systems with containers. In: 2019 IEEE high performance extreme computing conference (HPEC), pp. 1\u20136 (2019). https:\/\/doi.org\/10.1109\/HPEC.2019.8916576","DOI":"10.1109\/HPEC.2019.8916576"},{"key":"3798_CR17","doi-asserted-by":"publisher","unstructured":"Brayford, D., Vallecorsa, S.: Deploying scientific al networks at petaflop scale on secure large scale HPC production systems with containers. In: Proceedings of the platform for advanced scientific computing conference. PASC \u201920, pp. 1\u20138. Association for Computing Machinery, New York, NY, USA (2020). https:\/\/doi.org\/10.1145\/3394277.3401850","DOI":"10.1145\/3394277.3401850"},{"key":"3798_CR18","doi-asserted-by":"publisher","unstructured":"Sergeev, A., Del\u00a0Balso, M.: Horovod: fast and easy distributed deep learning in TensorFlow. arXiv:1802.05799 [cs, stat] (2018). https:\/\/doi.org\/10.48550\/arXiv.1802.05799","DOI":"10.48550\/arXiv.1802.05799"},{"key":"3798_CR19","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., Polosukhin, I.: Attention is all you need. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in neural information processing systems, vol. 30, pp. 5998\u20136008 (2017). https:\/\/papers.nips.cc\/paper\/2017\/hash\/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html"},{"key":"3798_CR20","doi-asserted-by":"publisher","unstructured":"Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. In: 2020 IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp. 8107\u20138116 (2020). https:\/\/doi.org\/10.1109\/CVPR42600.2020.00813","DOI":"10.1109\/CVPR42600.2020.00813"},{"issue":"1","key":"3798_CR21","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1007\/s10462-018-09679-z","volume":"52","author":"G Nguyen","year":"2019","unstructured":"Nguyen, G., Dlugolinsky, S., Bob\u00e1k, M., Tran, V., L\u00f3pez Garc\u00eda, \u00c1., Heredia, I., Mal\u00edk, P., Hluch\u00fd, L.: Machine learning and deep learning frameworks and libraries for large-scale data mining: a survey. Artif. Intell. Rev. 52(1), 77\u2013124 (2019). https:\/\/doi.org\/10.1007\/s10462-018-09679-z","journal-title":"Artif. Intell. Rev."},{"key":"3798_CR22","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., Kudlur, M., Levenberg, J., Monga, R., Moore, S., Murray, D.G., Steiner, B., Tucker, P., Vasudevan, V., Warden, P., Wicke, M., Yu, Y., Zheng, X.: Tensorflow: a system for large-scale machine learning. In: 12th USENIX symposium on operating systems design and implementation (OSDI 16), pp. 265\u2013283. USENIX Association, Savannah, GA (2016)"},{"key":"3798_CR23","unstructured":"Chollet, F., et al.: Keras (2015). https:\/\/keras.io"},{"key":"3798_CR24","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: PyTorch: An imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., Alch\u00e9-Buc, F.d., Fox, E., Garnett, R. (eds.) Advances in neural information processing systems, vol. 32, pp. 8024\u20138035 (2019). https:\/\/papers.nips.cc\/paper\/2019\/hash\/bdbca288fee7f92f2bfa9f7012727740-Abstract.html"},{"issue":"2","key":"3798_CR25","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1016\/j.jpdc.2008.09.002","volume":"69","author":"P Patarasuk","year":"2009","unstructured":"Patarasuk, P., Yuan, X.: Bandwidth optimal all-reduce algorithms for clusters of workstations. J. Parallel Distrib. Comput. 69(2), 117\u2013124 (2009). https:\/\/doi.org\/10.1016\/j.jpdc.2008.09.002","journal-title":"J. Parallel Distrib. Comput."},{"key":"3798_CR26","doi-asserted-by":"publisher","unstructured":"Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) Job scheduling strategies for parallel processing. Lecture notes in computer science, pp. 44\u201360. Springer, Berlin (2003). https:\/\/doi.org\/10.1007\/10968987_3","DOI":"10.1007\/10968987_3"},{"issue":"7","key":"3798_CR27","doi-asserted-by":"publisher","first-page":"2067","DOI":"10.5194\/gmd-8-2067-2015","volume":"8","author":"F Oesterle","year":"2015","unstructured":"Oesterle, F., Ostermann, S., Prodan, R., Mayr, G.J.: Experiences with distributed computing for meteorological applications: grid computing and cloud computing. Geosci. Model Dev. 8(7), 2067\u20132078 (2015). https:\/\/doi.org\/10.5194\/gmd-8-2067-2015","journal-title":"Geosci. Model Dev."},{"key":"3798_CR28","doi-asserted-by":"publisher","unstructured":"Felter, W., Ferreira, A., Rajamony, R., Rubio, J.: An updated performance comparison of virtual machines and Linux containers. In: 2015 IEEE international symposium on performance analysis of systems and software (ISPASS), pp. 171\u2013172 (2015). https:\/\/doi.org\/10.1109\/ISPASS.2015.7095802","DOI":"10.1109\/ISPASS.2015.7095802"},{"key":"3798_CR29","doi-asserted-by":"publisher","unstructured":"Torrez, A., Randles, T., Priedhorsky, R.: HPC Container runtimes have minimal or no performance impact. In: 2019 IEEE\/ACM international workshop on containers and new orchestration paradigms for isolated environments in HPC (CANOPIE-HPC), pp. 37\u201342 (2019). https:\/\/doi.org\/10.1109\/CANOPIE-HPC49598.2019.00010","DOI":"10.1109\/CANOPIE-HPC49598.2019.00010"},{"key":"3798_CR30","doi-asserted-by":"publisher","unstructured":"Chung, M.T., Quang-Hung, N., Nguyen, M.-T., Thoai, N.: Using Docker in high performance computing applications. In: 2016 IEEE sixth international conference on communications and electronics (ICCE), pp. 52\u201357 (2016). https:\/\/doi.org\/10.1109\/CCE.2016.7562612","DOI":"10.1109\/CCE.2016.7562612"},{"issue":"16","key":"3798_CR31","doi-asserted-by":"publisher","first-page":"5018","DOI":"10.1002\/cpe.5018","volume":"31","author":"J Sparks","year":"2019","unstructured":"Sparks, J.: Enabling Docker for HPC. Concurr. Comput.: Pract. Exp. 31(16), 5018 (2019). https:\/\/doi.org\/10.1002\/cpe.5018","journal-title":"Concurr. Comput.: Pract. Exp."},{"key":"3798_CR32","doi-asserted-by":"publisher","unstructured":"Azab, A.: Enabling Docker containers for high-performance and many-task computing. In: 2017 IEEE international conference on cloud engineering (IC2E), pp. 279\u2013285 (2017). https:\/\/doi.org\/10.1109\/IC2E.2017.52","DOI":"10.1109\/IC2E.2017.52"},{"key":"3798_CR33","unstructured":"Jacobsen, D.M., Canon, R.S.: Contain this, unleashing docker for HPC. In: Proceedings of the cray user group, p. 8 (2015)"},{"key":"3798_CR34","doi-asserted-by":"publisher","unstructured":"Grupp, A., Kozlov, V., Campos, I., David, M., Gomes, J., L\u00f3pez\u00a0Garc\u00eda, \u00c1.: Benchmarking deep learning infrastructures by means of tensorflow and containers. In: Weiland, M., Juckeland, G., Alam, S., Jagode, H. (eds.) High performance computing. Lecture notes in computer science, pp. 478\u2013489. Springer, Cham (2019). https:\/\/doi.org\/10.1007\/978-3-030-34356-9_36","DOI":"10.1007\/978-3-030-34356-9_36"},{"key":"3798_CR35","doi-asserted-by":"publisher","unstructured":"Gantikow, H., Walter, S., Reich, C.: Rootless containers with podman for HPC. In: Springer (ed.) International conference on high performance computing, pp. 343\u2013354 (2020). https:\/\/doi.org\/10.1007\/978-3-030-59851-8_23","DOI":"10.1007\/978-3-030-59851-8_23"},{"key":"3798_CR36","doi-asserted-by":"publisher","unstructured":"Gamblin, T., LeGendre, M., Collette, M.R., Lee, G.L., Moody, A., De\u00a0Supinski, B.R., Futral, S.: The spack package manager: bringing order to hpc software chaos. In: Proceedings of the international conference for high performance computing, networking, storage and analysis, pp. 1\u201312 (2015). https:\/\/doi.org\/10.1145\/2807591.2807623","DOI":"10.1145\/2807591.2807623"},{"key":"3798_CR37","doi-asserted-by":"publisher","unstructured":"H\u00f6b, M., Kranzlm\u00fcller, D.: Enabling easey deployment of containerized applications for future hpc systems. In: Springer (ed.) International conference on computational science, pp. 206\u2013219 (2020). https:\/\/doi.org\/10.1007\/978-3-030-50371-0_15","DOI":"10.1007\/978-3-030-50371-0_15"},{"key":"3798_CR38","doi-asserted-by":"publisher","unstructured":"Canon, R.S., Younge, A.: A case for portability and reproducibility of hpc containers. In: IEEE (ed.) 2019 IEEE\/ACM international workshop on containers and new orchestration paradigms for isolated environments in HPC (CANOPIE-HPC), pp. 49\u201354 (2019). https:\/\/doi.org\/10.1109\/CANOPIE-HPC49598.2019.00012","DOI":"10.1109\/CANOPIE-HPC49598.2019.00012"},{"key":"3798_CR39","doi-asserted-by":"publisher","unstructured":"Gabriel, E., Fagg, G.E., Bosilca, G., Angskun, T., Dongarra, J.J., Squyres, J.M., Sahay, V., Kambadur, P., Barrett, B., Lumsdaine, A., Castain, R.H., Daniel, D.J., Graham, R.L., Woodall, T.S.: Open MPI: Goals, concept, and design of a next generation MPI implementation. In: Proceedings, 11th European PVM\/MPI users\u2019 group meeting, Budapest, Hungary, pp. 97\u2013104 (2004). https:\/\/doi.org\/10.1007\/978-3-540-30218-6_19","DOI":"10.1007\/978-3-540-30218-6_19"},{"key":"3798_CR40","doi-asserted-by":"crossref","unstructured":"Sharma, T., Fragkoulis, M., Spinellis, D.: Does your configuration code smell? In: 2016 IEEE\/ACM 13th working conference on mining software repositories (MSR), pp. 189\u2013200 (2016)","DOI":"10.1145\/2901739.2901761"},{"key":"3798_CR41","doi-asserted-by":"publisher","first-page":"34127","DOI":"10.1109\/ACCESS.2020.2973750","volume":"8","author":"Y Wu","year":"2020","unstructured":"Wu, Y., Zhang, Y., Wang, T., Wang, H.: Characterizing the occurrence of dockerfile smells in open-source software: an empirical study. IEEE Access 8, 34127\u201334139 (2020). https:\/\/doi.org\/10.1109\/ACCESS.2020.2973750","journal-title":"IEEE Access"},{"key":"3798_CR42","doi-asserted-by":"publisher","first-page":"63650","DOI":"10.1109\/ACCESS.2019.2905424","volume":"7","author":"Z Lu","year":"2019","unstructured":"Lu, Z., Xu, J., Wu, Y., Wang, T., Huang, T.: An empirical case study on the temporary file smell in dockerfiles. IEEE Access 7, 63650\u201363659 (2019). https:\/\/doi.org\/10.1109\/ACCESS.2019.2905424","journal-title":"IEEE Access"},{"issue":"1","key":"3798_CR43","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1007\/s10723-020-09509-z","volume":"18","author":"P Orviz Fern\u00e1ndez","year":"2020","unstructured":"Orviz Fern\u00e1ndez, P., David, M., Duma, D.C., Ronchieri, E., Gomes, J., Salomoni, D.: Software quality assurance in INDIGO-datacloud project: a converging evolution of software engineering practices to support European research e-infrastructures. J. Grid Comput. 18(1), 81\u201398 (2020). https:\/\/doi.org\/10.1007\/s10723-020-09509-z","journal-title":"J. Grid Comput."},{"key":"3798_CR44","doi-asserted-by":"publisher","unstructured":"He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp. 770\u2013778 (2016). https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"3798_CR45","doi-asserted-by":"publisher","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp. 1\u20139 (2015). https:\/\/doi.org\/10.1109\/CVPR.2015.7298594","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"3798_CR46","doi-asserted-by":"publisher","unstructured":"Huang, G., Liu, Z., Van Der\u00a0Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp. 2261\u20132269 (2017). https:\/\/doi.org\/10.1109\/CVPR.2017.243","DOI":"10.1109\/CVPR.2017.243"},{"key":"3798_CR47","doi-asserted-by":"publisher","DOI":"10.1017\/9781107588783","volume-title":"Statistical downscaling and bias correction for climate research","author":"D Maraun","year":"2018","unstructured":"Maraun, D., Widmann, M.: Statistical downscaling and bias correction for climate research. Cambridge University Press, Cambridge (2018)"},{"key":"3798_CR48","doi-asserted-by":"publisher","unstructured":"Vandal, T., Kodra, E., Ganguly, S., Michaelis, A., Nemani, R., Ganguly, A.R.: DeepSD: generating high resolution climate change projections through single image super-resolution. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. KDD \u201917, pp. 1663\u20131672. Association for Computing Machinery, New York, NY, USA (2017). https:\/\/doi.org\/10.1145\/3097983.3098004","DOI":"10.1145\/3097983.3098004"},{"issue":"4","key":"3798_CR49","doi-asserted-by":"publisher","first-page":"2109","DOI":"10.5194\/gmd-13-2109-2020","volume":"13","author":"J Ba\u00f1o-Medina","year":"2020","unstructured":"Ba\u00f1o-Medina, J., Manzanas, R., Guti\u00e9rrez, J.M.: Configuration and intercomparison of deep learning neural models for statistical downscaling. Geosci. Model Dev. 13(4), 2109\u20132124 (2020). https:\/\/doi.org\/10.5194\/gmd-13-2109-2020","journal-title":"Geosci. Model Dev."},{"issue":"2","key":"3798_CR50","doi-asserted-by":"publisher","first-page":"1128","DOI":"10.1002\/joc.6769","volume":"41","author":"L Sun","year":"2021","unstructured":"Sun, L., Lan, Y.: Statistical downscaling of daily temperature and precipitation over China using deep learning neural models: Localization and comparison with other methods. Int. J. Climatol. 41(2), 1128\u20131147 (2021). https:\/\/doi.org\/10.1002\/joc.6769","journal-title":"Int. J. Climatol."},{"issue":"656","key":"3798_CR51","doi-asserted-by":"publisher","first-page":"553","DOI":"10.1002\/qj.828","volume":"137","author":"DP Dee","year":"2011","unstructured":"Dee, D.P., Uppala, S.M., Simmons, A.J., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U., Balmaseda, M.A., Balsamo, G., Bauer, P., Bechtold, P., Beljaars, A.C.M., Berg, L.V.D., Bidlot, J., Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, A.J., Haimberger, L., Healy, S.B., Hersbach, H., H\u00f3lm, E.V., Isaksen, L., K\u00e5llberg, P., K\u00f6hler, M., Matricardi, M., McNally, A.P., Monge-Sanz, B.M., Morcrette, J.-J., Park, B.-K., Peubey, C., Rosnay, P.D., Tavolato, C., Th\u00e9paut, J.-N., Vitart, F.: The ERA-interim reanalysis: configuration and performance of the data assimilation system. Q. J. R. Meteorol. Soc. 137(656), 553\u2013597 (2011). https:\/\/doi.org\/10.1002\/qj.828","journal-title":"Q. J. R. Meteorol. Soc."},{"key":"3798_CR52","doi-asserted-by":"publisher","DOI":"10.5880\/PIK.2019.004","author":"S Lange","year":"2019","unstructured":"Lange, S.: EartH2Observe, WFDEI and ERA-interim data merged and bias-corrected for ISIMIP (EWEMBI). GFZ Data Serv. (2019). https:\/\/doi.org\/10.5880\/PIK.2019.004","journal-title":"GFZ Data Serv."},{"key":"3798_CR53","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.envsoft.2018.09.009","volume":"111","author":"M Iturbide","year":"2019","unstructured":"Iturbide, M., Bedia, J., Herrera, S., Ba\u00f1o-Medina, J., Fern\u00e1ndez, J., Fr\u00edas, M.D., Manzanas, R., San-Mart\u00edn, D., Cimadevilla, E., Cofi\u00f1o, A.S., Guti\u00e9rrez, J.M.: The R-based climate4R open framework for reproducible climate data access and post-processing. Environ. Model Softw. 111, 42\u201354 (2019). https:\/\/doi.org\/10.1016\/j.envsoft.2018.09.009","journal-title":"Environ. Model Softw."},{"key":"3798_CR54","unstructured":"Schmuck, F., Haskin, R.: Gpfs: A shared-disk file system for large computing clusters. In: Conference on file and storage technologies (FAST 02) (2002)"},{"key":"3798_CR55","doi-asserted-by":"publisher","unstructured":"Braam, P.: The lustre storage architecture (2019). arXiv preprint arXiv:1903.01955. https:\/\/doi.org\/10.48550\/arXiv.1903.01955","DOI":"10.48550\/arXiv.1903.01955"},{"key":"3798_CR56","doi-asserted-by":"publisher","unstructured":"Keskar, N.S., Mudigere, D., Nocedal, J., Smelyanskiy, M., Tang, P.T.P.: On large-batch training for deep learning: generalization gap and sharp minima (2016). arXiv preprint arXiv:1609.04836. https:\/\/doi.org\/10.48550\/arXiv.1609.04836","DOI":"10.48550\/arXiv.1609.04836"}],"container-title":["Cluster Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10586-022-03798-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10586-022-03798-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10586-022-03798-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,26]],"date-time":"2023-08-26T16:24:46Z","timestamp":1693067086000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10586-022-03798-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,7]]},"references-count":56,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,10]]}},"alternative-id":["3798"],"URL":"https:\/\/doi.org\/10.1007\/s10586-022-03798-7","relation":{},"ISSN":["1386-7857","1573-7543"],"issn-type":[{"value":"1386-7857","type":"print"},{"value":"1573-7543","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,7]]},"assertion":[{"value":"15 March 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 October 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 October 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 November 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 February 2023","order":5,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Update","order":6,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The original online version of this article was revised: Missing Open Access funding information has been added in the Funding Note","order":7,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}