{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,16]],"date-time":"2026-01-16T09:17:31Z","timestamp":1768555051126,"version":"3.49.0"},"reference-count":45,"publisher":"Wiley","issue":"24","license":[{"start":{"date-parts":[[2019,4,22]],"date-time":"2019-04-22T00:00:00Z","timestamp":1555891200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61502097"],"award-info":[{"award-number":["61502097"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61572129"],"award-info":[{"award-number":["61572129"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61602112"],"award-info":[{"award-number":["61602112"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61702096"],"award-info":[{"award-number":["61702096"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61320106007"],"award-info":[{"award-number":["61320106007"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61632008"],"award-info":[{"award-number":["61632008"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["BK20160695"],"award-info":[{"award-number":["BK20160695"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["BK20170689"],"award-info":[{"award-number":["BK20170689"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002338","name":"Ministry of Education of the People's Republic of China","doi-asserted-by":"publisher","award":["93K9"],"award-info":[{"award-number":["93K9"]}],"id":[{"id":"10.13039\/501100002338","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011246","name":"State Key Laboratory of Novel Software Technology","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100011246","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Concurrency and Computation"],"published-print":{"date-parts":[[2019,12,25]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>Most large\u2010scale scientific workflows take place in multiple collaborative datacenters for access to community\u2010wide resources, while adhering to each datacenter's non\u2010uniform resource limits. However, moving both initial input datasets with predetermined locations and intermediate datasets needing placement decisions across geo\u2010distributed datacenters hinders efficient execution of large\u2010scale data\u2010intensive scientific workflows. Thus, scientific workflow's data and task co\u2010scheduling deal with situations such as pre\u2010placed initial input datasets, placement of intermediate datasets and each datacenter's non\u2010uniform computation and storage constraint, while minimizing the cross\u2010datacenter data transfer. Since this scheduling problem is known to be NP\u2010hard, here, we propose a novel approach, based on the multilevel graph coarsening and uncoarsening framework, together with a specialized hybrid genetic algorithm having distinctive graph partition driven features of repair and local improvement, for scheduling data\u2010intensive scientific workflows in geo\u2010distributed datacenters and optimizing the cross\u2010datacenter data transfer volume. Extensive simulations, based on four real\u2010world workflow traces, show that our algorithm significantly reduces the overall geo\u2010distributed data transfer and demonstrate its effectiveness.<\/jats:p>","DOI":"10.1002\/cpe.5245","type":"journal-article","created":{"date-parts":[[2019,4,22]],"date-time":"2019-04-22T08:16:51Z","timestamp":1555921011000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Graph partition\u2013based data and task co\u2010scheduling of scientific workflow in geo\u2010distributed datacenters"],"prefix":"10.1002","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9067-7896","authenticated-orcid":false,"given":"Jinghui","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering Southeast University  Nanjing China"}]},{"given":"Jian","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering Southeast University  Nanjing China"}]},{"given":"Jun","family":"Zhan","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering Southeast University  Nanjing China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9570-1456","authenticated-orcid":false,"given":"Jiahui","family":"Jin","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering Southeast University  Nanjing China"}]},{"given":"Aibo","family":"Song","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering Southeast University  Nanjing China"}]}],"member":"311","published-online":{"date-parts":[[2019,4,22]]},"reference":[{"key":"e_1_2_8_2_1","unstructured":"AMS02.http:\/\/www.ams02.org\/"},{"key":"e_1_2_8_3_1","unstructured":"ViswanathanR AnanthanarayananG AkellaA.Clarinet: WAN\u2010aware optimization for analytics queries. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16);2016;Savannah GA."},{"key":"e_1_2_8_4_1","doi-asserted-by":"crossref","unstructured":"HuZ LiB LuoJ.Flutter: scheduling tasks closer to data across geo\u2010distributed datacenters. Paper presented at: The 35th Annual IEEE International Conference on Computer Communications;2016;San Francisco CA.","DOI":"10.1109\/INFOCOM.2016.7524469"},{"key":"e_1_2_8_5_1","doi-asserted-by":"crossref","unstructured":"HungC GolubchikL YuM.Scheduling jobs across geo\u2010distributed datacenters. In: Proceedings of the 6th ACM Symposium on Cloud Computing (SoCC '15);2015;Kohala Coast HI.","DOI":"10.1145\/2806777.2806780"},{"key":"e_1_2_8_6_1","doi-asserted-by":"crossref","unstructured":"LiY ZhaoL CuiC YuC.Fast big data analysis in geo\u2010distributed cloud. Paper presented at: 2016 IEEE International Conference on Cluster Computing (CLUSTER);2016;Taipei Taiwan.","DOI":"10.1109\/CLUSTER.2016.28"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2013.121"},{"key":"e_1_2_8_8_1","unstructured":"HsiehA HarlapA VijaykumarN et al.Gaia: geo\u2010distributed machine learning approaching (LAN) speeds. In: 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17);2017;Boston MA."},{"key":"e_1_2_8_9_1","unstructured":"AgarwalS DunaganJ JainN SaroiuS WolmanA BhoganH.Volley: automated data placement for geo\u2010distributed cloud services. In: Proceedings of the 7th USENIX Conference on Networked Systems Design and Implementation NSDI'10;2010;Berkeley CA."},{"key":"e_1_2_8_10_1","doi-asserted-by":"crossref","unstructured":"YuB PanJ.Location\u2010aware associated data placement for geo\u2010distributed data\u2010intensive applications. Paper presented at: 2015 IEEE Conference on Computer Communications (INFOCOM);2015;Kowloon Hong Kong.","DOI":"10.1109\/INFOCOM.2015.7218428"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.07.007"},{"key":"e_1_2_8_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2014.01.003"},{"key":"e_1_2_8_13_1","doi-asserted-by":"crossref","unstructured":"XinY BaldineI MandalA HeermannC ChaseJ YumerefendiA.Embedding virtual topologies in networked clouds. In: Proceedings of the 6th International Conference on Future Internet Technologies;2011;Seoul South Korea.","DOI":"10.1145\/2002396.2002403"},{"key":"e_1_2_8_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2014.2359365"},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2829988.2787505"},{"key":"e_1_2_8_16_1","doi-asserted-by":"crossref","unstructured":"YinL SunJ ZhaoL CuiC XiaoJ YuC.Joint scheduling of data and computation in geo\u2010distributed cloud systems. In: 2015 15th IEEE\/ACM International Symposium on Cluster Cloud and Grid Computing (CCGrid);2015;Shenzhen China.","DOI":"10.1109\/CCGrid.2015.83"},{"key":"e_1_2_8_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TST.2016.7590316"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2012.10.029"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.3233\/ICA-150489"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.3601"},{"key":"e_1_2_8_21_1","doi-asserted-by":"crossref","unstructured":"LinX WuCQ.On scientific workflow scheduling in clouds under budget constraint. Paper presented at: 2013 42nd International Conference on Parallel Processing;2013;Lyon France.","DOI":"10.1109\/ICPP.2013.18"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2016.2599529"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2013.20"},{"key":"e_1_2_8_24_1","doi-asserted-by":"crossref","unstructured":"TanakaM TatebeO.Workflow scheduling to minimize data movement using multi\u2010constraint graph partitioning. In: Proceedings of the 2012 12th IEEE\/ACM International Symposium on Cluster Cloud and Grid Computing;2012;Washington DC.","DOI":"10.1109\/CCGrid.2012.134"},{"key":"e_1_2_8_25_1","doi-asserted-by":"crossref","unstructured":"SrinivasanS JuveG daSilvaRF VahiK DeelmanE.A cleanup algorithm for implementing storage constraints in scientific workflow executions. In: Proceedings of the 2014 9th Workshop on Workflows in Support of Large\u2010Scale Science;2014;New Orleans LA.","DOI":"10.1109\/WORKS.2014.8"},{"key":"e_1_2_8_26_1","unstructured":"daSilvaR FilgueiraR DeelmanE Pairo\u2010CastineiraE OvertonIM AtkinsonMP.Using simple PID controllers to prevent and mitigate faults in scientific workflows. In: Proceedings of the 11th Workshop on Workflows in Support of Large\u2010Scale Science co\u2010located with The International Conference for High Performance Computing Networking Storage and Analysis (SC);2016;Salt Lake City UT."},{"key":"e_1_2_8_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2010.02.004"},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCC.2015.2511745"},{"key":"e_1_2_8_29_1","doi-asserted-by":"crossref","unstructured":"DengK SongJ RenK YuanD ChenJ.Graph\u2010cut based coscheduling strategy towards efficient execution of scientific workflows in collaborative cloud environments. In: Proceedings of the 2011 IEEE\/ACM 12th International Conference on Grid Computing;2011;Lyon France.","DOI":"10.1109\/Grid.2011.14"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.3084"},{"key":"e_1_2_8_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2010.09.003"},{"key":"e_1_2_8_32_1","doi-asserted-by":"crossref","unstructured":"Cataly\u00fcrek\u00dc KayaK UcarB.Integrated data placement and task assignment for scientific workflows in clouds. In: Proceedings of the 4th International Workshop on Data\u2010intensive Distributed Computing;2011;San Jose CA.","DOI":"10.1145\/1996014.1996022"},{"key":"e_1_2_8_33_1","doi-asserted-by":"crossref","unstructured":"GolabL HadjieleftheriouM KarloffH SahaB.Distributed data placement to minimize communication costs via graph partitioning. In: Proceedings of the 26th International Conference on Scientific and Statistical Database Management (SSDBM);2014;Aalborg Denmark.","DOI":"10.1145\/2618243.2618258"},{"key":"e_1_2_8_34_1","doi-asserted-by":"crossref","unstructured":"ChenJ ZhangJ SongA.Efficient data and task co\u2010scheduling for scientific workflow in geo\u2010distributed datacenters. In: 5th International Conference on Advanced Cloud and Big Data (CBD);2017;Shanghai China.","DOI":"10.1109\/CBD.2017.19"},{"key":"e_1_2_8_35_1","doi-asserted-by":"crossref","unstructured":"KrauthgamerR NaorJ SchwartzR TalwarK.Non\u2010uniform graph partitioning. In: Proceedings of the 25th Annual ACM\u2010SIAM Symposium on Discrete Algorithms (SODA '14);2014;Portland OR.","DOI":"10.1137\/1.9781611973402.91"},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2018.03.028"},{"key":"e_1_2_8_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2017.2678507"},{"key":"e_1_2_8_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNS.2011.2146276"},{"key":"e_1_2_8_39_1","doi-asserted-by":"crossref","unstructured":"DahlhausE JohnsonDS PapadimitriouCH SeymourPD YannakakisM.The complexity of multiway cuts (extended abstract). In: Proceedings of the 24th Annual ACM Symposium on Theory of Computing (STOC '92);1992;Victoria Canada.","DOI":"10.1145\/129712.129736"},{"key":"e_1_2_8_40_1","unstructured":"G Karypis V Kumar 1998"},{"key":"e_1_2_8_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/71.780863"},{"key":"e_1_2_8_42_1","first-page":"1479","volume-title":"Encyclopedia of Parallel Computing","author":"Cataly\u00fcrek \u00dc","year":"2011"},{"key":"e_1_2_8_43_1","unstructured":"KarypisG KumarV.Metis\u00a0\u2013\u00a0unstructured graph partitioning and sparse matrix ordering system version 2.0;1995. Technical report."},{"key":"e_1_2_8_44_1","unstructured":"KarypisG KumarV.Multilevel k\u2010way hypergraph partitioning. In: Proceedings of the 36th Annual ACM\/IEEE Design Automation Conference (DAC '99);1999;New Orleans LA."},{"key":"e_1_2_8_45_1","doi-asserted-by":"crossref","unstructured":"BharathiS ChervenakA DeelmanE MehtaG SuMH VahiK.Characterization of scientific workflows. In: 2008 3rd Workshop on Workflows in Support of Large\u2010Scale Science.2008;Austin TX.","DOI":"10.1109\/WORKS.2008.4723958"},{"key":"e_1_2_8_46_1","unstructured":"Pegasus workflow generator.https:\/\/confluence.pegasus.isi.edu"}],"container-title":["Concurrency and Computation: Practice and Experience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fcpe.5245","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cpe.5245","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/cpe.5245","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/cpe.5245","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,8]],"date-time":"2023-09-08T23:46:06Z","timestamp":1694216766000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/cpe.5245"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,22]]},"references-count":45,"journal-issue":{"issue":"24","published-print":{"date-parts":[[2019,12,25]]}},"alternative-id":["10.1002\/cpe.5245"],"URL":"https:\/\/doi.org\/10.1002\/cpe.5245","archive":["Portico"],"relation":{},"ISSN":["1532-0626","1532-0634"],"issn-type":[{"value":"1532-0626","type":"print"},{"value":"1532-0634","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,4,22]]},"assertion":[{"value":"2018-12-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-03-06","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-04-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e5245"}}