{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T18:47:11Z","timestamp":1771613231070,"version":"3.50.1"},"publisher-location":"Cham","reference-count":42,"publisher":"Springer International Publishing","isbn-type":[{"value":"9783030783068","type":"print"},{"value":"9783030783075","type":"electronic"}],"license":[{"start":{"date-parts":[[2021,7,1]],"date-time":"2021-07-01T00:00:00Z","timestamp":1625097600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,7,1]],"date-time":"2021-07-01T00:00:00Z","timestamp":1625097600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Traditional usage models of Supercomputing centres have been extended by High-Throughput Computing (HTC), High-Performance Data Analytics (HPDA) and Cloud Computing. The complexity of current compute platforms calls for solutions to simplify usage and conveniently orchestrate computing tasks. These enable also non-expert users to efficiently execute Big Data workflows. In this context, the LEXIS project (\u2018Large-scale EXecution for Industry and Society\u2019, H2020 GA 825532, <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/lexis-project.eu\">https:\/\/lexis-project.eu<\/jats:ext-link>) sets up an orchestration platform for compute- and data-intensive workflows. Its main objective is to implement a front-end and interfaces\/APIs for distributed data management and workflow orchestration. The platform uses an open-source Identity and Access Management solution and a custom billing system. The data management API allows data ingestion and staging between various infrastructures. The orchestration API allows execution of workflows specified in extended TOSCA. LEXIS uses innovative technologies like YORC and Alien4Cloud for orchestration or iRODS\/EUDAT-B2SAFE for data management, accelerated by Burst Buffers. Three pilot use cases from Aeronautics Engineering, Earthquake\/Tsunami Analysis, and Weather and Climate Prediction are used to test the services. On the road towards longer-term sustainability, we are expanding this user base and aiming at the immersion of more Supercomputing centres within the platform.<\/jats:p>","DOI":"10.1007\/978-3-030-78307-5_8","type":"book-chapter","created":{"date-parts":[[2022,4,28]],"date-time":"2022-04-28T07:03:15Z","timestamp":1651129395000},"page":"159-180","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Leveraging High-Performance Computing and Cloud Computing with Unified Big-Data Workflows: The LEXIS Project"],"prefix":"10.1007","author":[{"given":"Stephan","family":"Hachinger","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Martin","family":"Golasowski","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jan","family":"Martinovi\u010d","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohamad","family":"Hayek","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rub\u00e9n Jes\u00fas","family":"Garc\u00eda-Hern\u00e1ndez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kate\u0159ina","family":"Slaninov\u00e1","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marc","family":"Levrier","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alberto","family":"Scionti","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fr\u00e9d\u00e9ric","family":"Donnat","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Giacomo","family":"Vitali","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Donato","family":"Magarielli","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thierry","family":"Goubier","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Antonio","family":"Parodi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrea","family":"Parodi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Piyush","family":"Harsh","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aaron","family":"Dees","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Olivier","family":"Terzo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,7,1]]},"reference":[{"key":"8_CR1","unstructured":"Zillner, S., Curry, E., Metzger, A., Auer, S., Seidl, R. (Eds.) (2017). European big data value strategic research and innovation agenda. https:\/\/bdva.eu\/sites\/default\/files\/BDVA_SRIA_v4_Ed1.1.pdf. Cited 6 Nov 2020"},{"key":"8_CR2","unstructured":"Zillner, S., et al. (Eds.) (2020). Strategic research, innovation and deployment agenda\u2014AI, data and robotics partnership. Third Release. https:\/\/ai-data-robotics-partnership.eu\/wp-content\/uploads\/2020\/09\/AI-Data-Robotics-Partnership-SRIDA-V3.0.pdf. Cited 1 Feb 2021"},{"issue":"1","key":"8_CR3","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1016\/j.cpc.2007.02.021","volume":"177","author":"J Shiers","year":"2007","unstructured":"Shiers, J. (2007). The worldwide LHC computing grid (worldwide LCG). Computer Physics Communications, 177(1), 219\u2013223.","journal-title":"Computer Physics Communications"},{"key":"8_CR4","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4302-4864-4","volume-title":"Pro apache hadoop","author":"J Venner","year":"2014","unstructured":"Venner, J., Wadkar, S., & Siddalingaiah, M. (2014). Pro apache hadoop, 2nd ed. Expert\u2019s voice in big data. Apress, Berkeley, CA. doi: 10.1007\/978-1-4302-4864-4","edition":"2"},{"issue":"1","key":"8_CR5","doi-asserted-by":"publisher","first-page":"279","DOI":"10.2218\/ijdc.v8i1.260","volume":"8","author":"D Lecarpentier","year":"2013","unstructured":"Lecarpentier, D., Wittenburg, P., Elbers, W., Michelini, A., Kanso, R., Coveney, P. V., & Baxter, R. (2013). EUDAT: A new cross-disciplinary data infrastructure for science. International Journal of Digital Curation, 8(1), 279\u2013287.","journal-title":"International Journal of Digital Curation"},{"key":"8_CR6","unstructured":"EUDAT Collaborative Data Infrastructure (2020). EUDAT\u2014research data services, expertise & technology solutions. https:\/\/www.eudat.eu. Cited 6 Nov 2020"},{"key":"8_CR7","unstructured":"The Big Data Value Association (2020). BDVA. https:\/\/www.bdva.eu\/. Cited 6 Nov 2020"},{"key":"8_CR8","series-title":"In Proceedings of the conference on file and storage technologies, FAST \u201902 (pp. 231\u2013244)","volume-title":"GPFS: a shared-disk file system for large computing clusters","author":"F B Schmuck","year":"2002","unstructured":"Schmuck, F. B., & Haskin, R. L. (2002). GPFS: a shared-disk file system for large computing clusters. In Proceedings of the conference on file and storage technologies, FAST \u201902 (pp. 231\u2013244). USENIX Association, US."},{"key":"8_CR9","series-title":"In Proceedings of the 7th symposium on operating systems design and implementation OSDI \u201906 (pp. 307\u2013320)","volume-title":"Ceph: A scalable, high-performance distributed file system","author":"S A Weil","year":"2006","unstructured":"Weil, S. A., Brandt, S. A., Miller, E. L., Long, D. D. E., & Maltzahn, C. (2006). Ceph: A scalable, high-performance distributed file system. In Proceedings of the 7th symposium on operating systems design and implementation OSDI \u201906 (pp. 307\u2013320). USENIX Association, US."},{"key":"8_CR10","unstructured":"NVM Express, Inc. (2016). NVM express over fabrics 1.0. https:\/\/nvmexpress.org\/wp-content\/uploads\/NVMe_over_Fabrics_1_0_Gold_20160605.pdf. Cited 6 Nov 2020."},{"key":"8_CR11","doi-asserted-by":"crossref","unstructured":"Colonnelli, I., Cantalupo, B., Merelli, I., & Aldinucci, M. (2020). Streamflow: Cross-breeding cloud with HPC. IEEE Transactions on Emerging Topics in Computing. In print. doi: 10.1109\/TETC.2020.3019202.","DOI":"10.1109\/TETC.2020.3019202"},{"key":"8_CR12","unstructured":"Bull\/Atos (2020). Ystia Suite. https:\/\/ystia.github.io. Cited 6 Nov 2020."},{"key":"8_CR13","unstructured":"FastConnect, Bull\/Atos (2020). Alien 4 Cloud. http:\/\/alien4cloud.github.io. Cited 6 Nov 2020."},{"key":"8_CR14","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1007\/978-3-662-44879-3_13","volume-title":"Service-oriented and cloud computing","author":"A Brogi","year":"2014","unstructured":"Brogi, A., Soldani, J., & Wang, P. (2014). TOSCA in a Nutshell: Promises and perspectives. In Villari, M., Zimmermann, W., Lau, K. K. (Eds.), Service-oriented and cloud computing (pp. 171\u2013186). Springer, Heidelberg. doi: 10.1007\/978-3-662-44879-3_13"},{"key":"8_CR15","unstructured":"Svaton, V. (2020). Home \u22c5 Wiki \u22c5 ADAS\/HEAppE\/Middleware \u22c5 GitLab. http:\/\/heappe.eu. Cited 6 Nov 2020."},{"key":"8_CR16","doi-asserted-by":"publisher","first-page":"280","DOI":"10.1007\/978-3-030-22354-0_26","volume-title":"CISIS 2019, advances in intelligent systems and computing, vol. 993","author":"V Svaton","year":"2019","unstructured":"Svaton, V., Martinovic, J., Krenek, J., Esch, T., & Tomancak, P. (2019). HPC-as-a-service via HEAppE platform. In Barolli, L., Hussain, F. K., Ikeda, M. (Eds.), CISIS 2019, advances in intelligent systems and computing, vol. 993 (pp. 280\u2013293). Springer, Cham. doi: 10.1007\/978-3-030-22354-0_26"},{"key":"8_CR17","doi-asserted-by":"crossref","unstructured":"Imamagic, E., & Ferrari, T. (2014). EGI grid middleware and distributed computing infrastructures integration. In Proceedings of the international symposium on grids and clouds (ISGC) 2013\u2014PoS, vol. 179 (010). doi: 10.22323\/1.179.0010.","DOI":"10.22323\/1.179.0010"},{"key":"8_CR18","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1007\/978-3-030-50454-0_25","volume-title":"CISIS 2020, advances in intelligent systems and computing, vol. 1194","author":"A Parodi","year":"2020","unstructured":"Parodi, A., et al. (2020). LEXIS weather and climate large-scale pilot. In Barolli, L., Poniszewska-Maranda, A., Enokido, T. (Eds.), CISIS 2020, advances in intelligent systems and computing, vol. 1194 (pp. 267\u2013277). Springer, Cham. doi: 10.1007\/978-3-030-50454-0_25."},{"issue":"8","key":"8_CR19","doi-asserted-by":"publisher","first-page":"1717","DOI":"10.1175\/BAMS-D-15-00308.1","volume":"98","author":"J G Powers","year":"2017","unstructured":"Powers, J. G., et al. (2017). The weather research and forecasting model: Overview, system efforts, and future directions. Bulletin of the American Meteorological Society, 98(8), 1717\u20131737.","journal-title":"Bulletin of the American Meteorological Society"},{"issue":"10","key":"8_CR20","doi-asserted-by":"publisher","first-page":"2149","DOI":"10.1175\/BAMS-D-16-0279.1","volume":"98","author":"A Parodi","year":"2017","unstructured":"Parodi, A., et al. (2017). DRIHM (2US): An e-science environment for hydrometeorological research on high-impact weather events. Bulletin of the American Meteorological Society, 98(10), 2149\u20132166.","journal-title":"Bulletin of the American Meteorological Society"},{"key":"8_CR21","unstructured":"LEXIS project (2019). Deliverable 4.1: Analysis of mechanisms for securing federated infrastructure. https:\/\/cordis.europa.eu\/project\/id\/825532\/results. Cited 7 Jan 2021."},{"key":"8_CR22","unstructured":"JBoss (Red Hat Inc.), Keycloak Community (2020). Keycloak. https:\/\/www.keycloak.org\/. Cited 6 Nov 2020."},{"key":"8_CR23","unstructured":"Sakimura, N., Bradley, J., Jones, M. B., de Medeiros, B., & Mortimore, C. (2014). OpenID connect core 1.0 incorporating errata set 1. https:\/\/openid.net\/specs\/openid-connect-core-1_0.html. Cited 6 Nov 2020."},{"key":"8_CR24","unstructured":"Cantor, S., Kemp, J., Philpott, R., & Maler, E. (2005). Assertions and protocols for the OASIS security assertion markup language (SAML) V2.0. http:\/\/docs.oasis-open.org\/security\/saml\/v2.0\/saml-core-2.0-os.pdf. Cited 6 Nov 2020."},{"key":"8_CR25","doi-asserted-by":"crossref","unstructured":"Jones, M. B., Bradley, J., & Sakimura, N. (2015). RFC 7519 \u2013 JSON web token (JWT). https:\/\/tools.ietf.org\/html\/rfc7519. Cited 6 Nov 2020.","DOI":"10.17487\/RFC7519"},{"key":"8_CR26","unstructured":"Garc\u00eda-Hern\u00e1ndez, R. J., & Golasowski, M. (2020). Supporting Keycloak in iRODS systems with OpenID authentication. Presented at CS3\u2014workshop on cloud storage synchronization and sharing services. https:\/\/indico.cern.ch\/event\/854707\/contributions\/3681126. Cited 6 Nov 2020."},{"issue":"11","key":"8_CR27","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1145\/2934664","volume":"59","author":"M Zaharia","year":"2016","unstructured":"Zaharia, M., et al. (2016). Apache spark: A unified engine for big data processing. Communications of the ACM, 59(11), 56\u201365","journal-title":"Communications of the ACM"},{"key":"8_CR28","doi-asserted-by":"crossref","unstructured":"Xu, H., Russell, T., Coposky, J., Rajasekar, A., Moore, R., de Torcy, A., Wan, M., Shroeder, W., & Chen, S. Y. (2017). iRODS primer 2: Integrated rule-oriented data system. In Morgan and Claypool Publishers, Williston, VT. doi: 10.2200\/S00760ED1V01Y201702ICR057.","DOI":"10.2200\/S00760ED1V01Y201702ICR057"},{"key":"8_CR29","unstructured":"Depardon, B., Le Mahec, G., & S\u00e9guin, C. (2013). Analysis of six distributed file systems. Report hal-00789086. https:\/\/hal.inria.fr\/hal-00789086. Cited 7 Jan 2021."},{"key":"8_CR30","doi-asserted-by":"crossref","unstructured":"Fuhrmann, P., Antonacci, M., Donvito, G., Keeble, O., & Millar, P. (2018). Smart policy driven data management and data federations. In Proceedings of the international symposium on grids and clouds (ISGC) 2018 in conjunction with frontiers in computational drug discovery\u2014PoS, vol. 327 (001). doi: 10.22323\/1.327.0001.","DOI":"10.22323\/1.327.0001"},{"key":"8_CR31","unstructured":"Kawai, Y., & Hasan, A. (2010). High-availability iRODS system (HAIRS). In Proceedings of the iRODS user group meeting 2010: Policy-based data management, sharing and preservation. Chapel Hill, NC."},{"key":"8_CR32","unstructured":"Depuydt, J. (2015). Setup a redundant PostgreSQL database with repmgr and pgpool \u2014 Jensd\u2019s I\/O buffer. http:\/\/jensd.be\/591\/linux\/setup-a-redundant-postgresql-database-with-repmgr-and-pgpool. Cited 6 Nov 2020."},{"key":"8_CR33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/sdata.2016.18","volume":"3","author":"M D Wilkinson","year":"2016","unstructured":"Wilkinson, M. D., et al. (2016). The FAIR guiding principles for scientific data management and stewardship. Scientific Data, 3, 1\u20139.","journal-title":"Scientific Data"},{"key":"8_CR34","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1007\/3-540-45268-0_9","volume-title":"Research and advanced technology for digital libraries","author":"A Apps","year":"2000","unstructured":"Apps, A., & MacIntyre, R. (2000). Dublin core metadata for electronic journals. In Borbinha, J., Baker, T. (Eds.), Research and advanced technology for digital libraries (pp. 93\u2013102). Springer, Berlin. doi: 10.1007\/3-540-45268-0_9."},{"key":"8_CR35","doi-asserted-by":"crossref","unstructured":"Starr, J., & Gastl, A. (2011). isCitedBy: A metadata scheme for DataCite. D-Lib Magazine, 17(1\u20132). doi: 10.1045\/january2011-starr.","DOI":"10.1045\/january2011-starr"},{"key":"8_CR36","doi-asserted-by":"publisher","first-page":"W587","DOI":"10.1093\/nar\/gkw289","volume":"44","author":"A M Roberts","year":"2016","unstructured":"Roberts, A. M., Wong, A. K., Fisk, I., & Troyanskaya, O. (2016). GIANT API: An application programming interface for functional genomics. Nucleic Acids Research, 44, W587\u2013W592.","journal-title":"Nucleic Acids Research"},{"key":"8_CR37","doi-asserted-by":"crossref","unstructured":"Allcock, W., Bresnahan, J., Kettimuthu, R., & Link, M. (2005). The globus striped GridFTP framework and server. In SC \u201905: Proceedings of the 2005 ACM\/IEEE conference on supercomputing (pp. 54\u201354). doi: 10.1109\/SC.2005.72.","DOI":"10.1109\/SC.2005.72"},{"key":"8_CR38","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1109\/MIC.2011.64","volume":"15","author":"I Foster","year":"2011","unstructured":"Foster, I. (2011). Globus online: Accelerating and democratizing science through cloud-based services. IEEE Internet Computing, 15, 70\u201373.","journal-title":"IEEE Internet Computing"},{"key":"8_CR39","unstructured":"Thelin, J. (2011). Accessing Remote Files Easily and Securely. Linux Journal. https:\/\/www.linuxjournal.com\/content\/accessing-remote-files-easy-and-secure. Cited 6 Nov 2020."},{"key":"8_CR40","unstructured":"Facebook Inc., React contributors (2020). React\u2014A JavaScript library for building user interfaces. http:\/\/reactjs.org. Cited 6 Nov 2020."},{"key":"8_CR41","unstructured":"Google, Go Contributors (2020). The go programming language specification\u2014the go programming language. https:\/\/golang.org\/ref\/spec. Cited 6 Nov 2020."},{"key":"8_CR42","unstructured":"Kleidl, M., Transloadit, tus Collaboration (2020). tus\u2014resumable file uploads. https:\/\/tus.io\/. Cited 6 Nov 2020."}],"container-title":["Technologies and Applications for Big Data Value"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-030-78307-5_8","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,28]],"date-time":"2022-04-28T07:06:47Z","timestamp":1651129607000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-030-78307-5_8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,1]]},"ISBN":["9783030783068","9783030783075"],"references-count":42,"URL":"https:\/\/doi.org\/10.1007\/978-3-030-78307-5_8","relation":{},"subject":[],"published":{"date-parts":[[2021,7,1]]},"assertion":[{"value":"1 July 2021","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}}]}}