{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,1]],"date-time":"2025-11-01T21:52:31Z","timestamp":1762033951172,"version":"build-2065373602"},"reference-count":28,"publisher":"SAGE Publications","issue":"4","license":[{"start":{"date-parts":[[2021,3,3]],"date-time":"2021-03-03T00:00:00Z","timestamp":1614729600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100002809","name":"European Union through the Horizon 2020 research and innovation programme","doi-asserted-by":"publisher","award":["721865"],"award-info":[{"award-number":["721865"]}],"id":[{"id":"10.13039\/501100002809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:p> Task-based programming models offer a flexible way to express the unstructured parallelism patterns of nowadays complex applications. This expressive capability is required to achieve maximum possible performance for applications that are executed in distributed execution platforms. In current task-based workflows, tasks are launched for execution when their data dependencies are satisfied. However, even though the data dependencies of a certain task might have been already produced, the execution of this task will be delayed until its predecessor tasks completely finish their execution. As a consequence of this approach of releasing dependencies, the amount of parallelism inherent in applications is limited and performance improvement opportunities are wasted. To mitigate this limitation, we propose an eager approach for releasing data dependencies. Following this approach, the execution of tasks will not be delayed until their predecessor tasks completely finish their execution, instead, tasks will be launched for execution as soon as their data requirements are available. Hence, more parallelism is exposed and applications can achieve higher levels of performance by overlapping the execution of tasks. Towards achieving this goal, in this paper we propose applying two changes to task-based workflow systems. First, modifying the dependency relationships of tasks to be specified not only in terms of predecessor and successor tasks but also in terms of the data that caused these dependencies. Second, triggering the release of dependencies as soon as a predecessor task generates the output data instead of having to wait until the end of the predecessor execution to release all of its dependencies. We realize this proposal using PyCOMPSs: a task-based programming model for parallelizing Python applications. Our experiments show that using an eager approach for releasing dependencies achieves more than 50% performance improvement in the total execution time as compared to the default approach of releasing dependencies. <\/jats:p>","DOI":"10.1177\/1094342021997558","type":"journal-article","created":{"date-parts":[[2021,3,3]],"date-time":"2021-03-03T10:43:52Z","timestamp":1614768232000},"page":"325-343","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":5,"title":["Accelerated execution via eager-release of dependencies in task-based workflows"],"prefix":"10.1177","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8591-1502","authenticated-orcid":false,"given":"Hatem","family":"Elshazly","sequence":"first","affiliation":[{"name":"Barcelona Supercomputing Center, Barcelona, Spain"}]},{"given":"Francesc","family":"Lordan","sequence":"additional","affiliation":[{"name":"Barcelona Supercomputing Center, Barcelona, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4725-5097","authenticated-orcid":false,"given":"Jorge","family":"Ejarque","sequence":"additional","affiliation":[{"name":"Barcelona Supercomputing Center, Barcelona, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2941-5499","authenticated-orcid":false,"given":"Rosa M.","family":"Badia","sequence":"additional","affiliation":[{"name":"Barcelona Supercomputing Center, Barcelona, Spain"}]}],"member":"179","published-online":{"date-parts":[[2021,3,3]]},"reference":[{"key":"bibr1-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTR.2009.5289188"},{"key":"bibr2-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1016\/j.softx.2015.10.004"},{"key":"bibr3-1094342021997558","unstructured":"Broad Institute (2020) Available at: https:\/\/www.broadinstitute.org\/ (accessed 26 October, 2020)."},{"key":"bibr4-1094342021997558","unstructured":"Common Crawl (2007) Common Crawl Website. Available at: https:\/\/commoncrawl.org\/. (accessed 10 January 2020)."},{"key":"bibr5-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1109\/99.660313"},{"key":"bibr6-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1145\/1327452.1327492"},{"volume-title":"28th Euromicro International Conference on Parallel, Distributed and Network-based Processing","year":"2020","author":"Elshazly H","key":"bibr7-1094342021997558"},{"key":"bibr8-1094342021997558","unstructured":"Extrae Package (2007) Extrae Website. Available at: https:\/\/tools.bsc.es\/paraver. (accessed 7 February 2020)."},{"key":"bibr9-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1002\/nme.76"},{"key":"bibr10-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btq524"},{"key":"bibr11-1094342021997558","volume-title":"Using MPI: Portable Parallel Programming with the Message-Passing Interface","volume":"1","author":"Gropp W","year":"1999"},{"key":"bibr12-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1186\/s13321-016-0179-6"},{"key":"bibr13-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1109\/12.833109"},{"key":"bibr14-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btp698"},{"key":"bibr15-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2012.6232369"},{"key":"bibr16-1094342021997558","unstructured":"Luigi (2015) Luigi Source Code on GitHub. Available at: https:\/\/github.com\/spotify\/luigi (accessed 6 February, 2020)."},{"key":"bibr17-1094342021997558","doi-asserted-by":"publisher","DOI":"10.25080\/Majora-92bf1922-00a"},{"key":"bibr18-1094342021997558","unstructured":"MareNostrum 4 BSC-CNS, 2017 (2017) MareNostrum 4 Supercomputer. Available at: https:\/\/www.bsc.es\/marenostrum\/marenostrum (accessed 26 March 2020)."},{"key":"bibr19-1094342021997558","volume-title":"A Guide to NumPy","volume":"1","author":"Oliphant T","year":"2006"},{"key":"bibr20-1094342021997558","unstructured":"Paraver Tool (2007) Paraver Website. Available at: https:\/\/tools.bsc.es\/extrae (accessed 7 February 2020)."},{"key":"bibr21-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2017.69"},{"key":"bibr22-1094342021997558","doi-asserted-by":"publisher","DOI":"10.25080\/Majora-7b98e3ed-013"},{"key":"bibr23-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1109\/TMAG.2013.2251616"},{"key":"bibr24-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1177\/1094342015594678"},{"key":"bibr25-1094342021997558","unstructured":"The European Bioinformatics Institute (EMBL-EBI) Available at: http:\/\/www.msca-expertise.eu\/. (accessed 26 October 2020)."},{"key":"bibr26-1094342021997558","unstructured":"The EXPERTISE Project (EXPERTISE) Available at: http:\/\/www.msca-expertise.eu\/. (accessed 26 October 2020)."},{"key":"bibr27-1094342021997558","unstructured":"Web Archive Format (WARC) Available at: http:\/\/bibnum.bnf.fr\/WARC\/. (accessed 10 February 2020)."},{"key":"bibr28-1094342021997558","doi-asserted-by":"publisher","DOI":"10.1145\/2934664"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342021997558","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/1094342021997558","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342021997558","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,3]],"date-time":"2025-03-03T14:03:20Z","timestamp":1741010600000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1094342021997558"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,3]]},"references-count":28,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["10.1177\/1094342021997558"],"URL":"https:\/\/doi.org\/10.1177\/1094342021997558","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"type":"print","value":"1094-3420"},{"type":"electronic","value":"1741-2846"}],"subject":[],"published":{"date-parts":[[2021,3,3]]}}}