{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,2]],"date-time":"2025-03-02T06:06:05Z","timestamp":1740895565912,"version":"3.38.0"},"reference-count":46,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2010,5,12]],"date-time":"2010-05-12T00:00:00Z","timestamp":1273622400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2011,5]]},"abstract":"<jats:p> We present a collection of techniques for exploiting latent I\/O asynchrony which can substantially improve performance in data-intensive parallel applications. Latent asynchrony refers to an application\u2019s tolerance for decoupling ancillary operations from its core computation, and is a property of HPC codes not fully explored by current HPC I\/O systems. Decoupling operations such as buffering and staging, reorganization, and format conversion in space and in time from core codes can shorten I\/O phases, preserving valuable MPP compute cycles. We describe in this paper DataTaps, IOgraphs, and Metabots, three tools which allow HPC developers to implement decoupled I\/O operations. Using these tools, asynchrony can be exploited by data generators which overlap computation with communication, and by data consumers that perform data conversion and reorganization out-of-band and on-demand. In the context of a data-intensive fusion simulation, we show that exploiting latent asynchrony through decoupling of operations can provide significant performance benefits. <\/jats:p>","DOI":"10.1177\/1094342010369117","type":"journal-article","created":{"date-parts":[[2010,5,13]],"date-time":"2010-05-13T00:35:23Z","timestamp":1273710923000},"page":"161-179","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":6,"title":["Exploiting Latent I\/O Asynchrony in Petascale Science Applications"],"prefix":"10.1177","volume":"25","author":[{"given":"Patrick","family":"Widener","sequence":"first","affiliation":[{"name":"Center for Comprehensive Informatics, Emory University, Atlanta, GA, USA,"}]},{"given":"Matthew","family":"Wolf","sequence":"additional","affiliation":[{"name":"College of Computing, Georgia Institute of Technology, Atlanta, GA, USA"}]},{"given":"Hasan","family":"Abbasi","sequence":"additional","affiliation":[{"name":"College of Computing, Georgia Institute of Technology, Atlanta, GA, USA"}]},{"given":"Scott","family":"McManus","sequence":"additional","affiliation":[{"name":"College of Computing, Georgia Institute of Technology, Atlanta, GA, USA"}]},{"given":"Mary","family":"Payne","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of New Mexico, Albuquerque, NM, USA"}]},{"given":"Matthew","family":"Barrick","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of New Mexico, Albuquerque, NM, USA"}]},{"given":"Jack","family":"Pulikottil","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of New Mexico, Albuquerque, NM, USA"}]},{"given":"Patrick","family":"Bridges","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of New Mexico, Albuquerque, NM, USA"}]},{"given":"Karsten","family":"Schwan","sequence":"additional","affiliation":[{"name":"College of Computing, Georgia Institute of Technology, Atlanta, GA, USA"}]}],"member":"179","published-online":{"date-parts":[[2010,5,12]]},"reference":[{"volume-title":"CLUSTER\u201904: Proceedings of the 2004 IEEE International Conference on Cluster Computing","author":"Abbasi, H.","key":"atypb1"},{"volume-title":"Proceedings of ACM High Performance Distributed Computing 2009","author":"Abbasi, H.","key":"atypb2"},{"volume-title":"IEEE Symposium on Mass Storage Systems","author":"Beynon, M.","key":"atypb3"},{"volume-title":"The Portals 3.0 message passing interface. Technical report SAND99-2959","year":"1999","author":"Brightwell, R.","key":"atypb4"},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2000.10046"},{"volume-title":"Proceedings of the 15th IEEE International Symposium on High Performance Distributed Computing (HPDC 2006)","author":"Cai, Z.","key":"atypb6"},{"volume-title":"MxN Parallel Data Redistribution @ ORNL","year":"2004","author":"CCA Forum.","key":"atypb7"},{"key":"atypb8","doi-asserted-by":"publisher","DOI":"10.1016\/S0749-6419(02)00109-2"},{"volume-title":"Lustre: a scalable, high-performance file system","year":"2002","author":"Cluster File Systems Inc.","key":"atypb9"},{"volume-title":"IEEE Visualization","author":"Crawford, D.","key":"atypb10"},{"key":"atypb11","doi-asserted-by":"publisher","DOI":"10.1023\/A:1024000426962"},{"volume-title":"Portable binary input\/output","author":"Eisenhauer, G.","key":"atypb12"},{"volume-title":"The EVPath Library","year":"2010","author":"Eisenhauer, G.","key":"atypb13"},{"key":"atypb14","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2002.1158262"},{"key":"atypb15","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2006.16"},{"volume-title":"Proceedings of the Heterogeneous Computing Workshop (HCW2000)","author":"Eisenhauer, G.","key":"atypb16"},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1109\/HPDC.2000.868641"},{"volume-title":"FDTree","key":"atypb18"},{"key":"atypb19","doi-asserted-by":"publisher","DOI":"10.1145\/384265.291029"},{"volume-title":"HEC-IWG File Systems and I\/O R&D Workshop","author":"Interagency Working Group on High End Computing.","key":"atypb20"},{"volume-title":"The ITER Project","author":"The ITER Organization.","key":"atypb21"},{"key":"atypb22","doi-asserted-by":"publisher","DOI":"10.1002\/nme.1147"},{"volume-title":"Proceedings of the 25th IEEE International Conference on Distributed Computing Systems (ICDCS-2005)","author":"Kumar, V.","key":"atypb23"},{"issue":"1","key":"atypb24","volume":"2","author":"Latham, R.","year":"2004","journal-title":"LinuxWorld"},{"volume-title":"Xchange: high performance data morphing in distributed applications. Report GIT-CC-05-14","year":"2005","author":"Lofstead, J.","key":"atypb25"},{"volume-title":"Proceedings of the 2008 IEEE Workshop of Challenges of Large Applications in Distributed Environments (CLADE)","author":"Lofstead, J.","key":"atypb26"},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.994"},{"key":"atypb28","doi-asserted-by":"publisher","DOI":"10.1177\/109434209801200303"},{"volume-title":"Proceedings of the 2006 IEEE Conference on Cluster Computing","author":"Oldfield, R.A.","key":"atypb29"},{"journal-title":"Proceedings of IEEE\/ACM Supercomputing","year":"2005","author":"Oliker, L.","key":"atypb30"},{"volume-title":"Exploratory Visualization Environment for REsearch in Science and Technology (EVEREST)","author":"ORNL Visualization Task Group.","key":"atypb31"},{"volume-title":"Object-based storage architecture: defining a new generation of storage systems built on distributed, intelligent storage devices","year":"2003","author":"Panasas Inc.","key":"atypb32"},{"key":"atypb33","doi-asserted-by":"publisher","DOI":"10.1145\/373626.373680"},{"issue":"4","key":"atypb34","first-page":"42","volume":"33","author":"Rajasekar, A.","year":"2003","journal-title":"Computer Society of India Journal, Special Issue on SAN"},{"key":"atypb35","doi-asserted-by":"publisher","DOI":"10.1109\/2.928624"},{"key":"atypb36","doi-asserted-by":"publisher","DOI":"10.1145\/1048935.1050172"},{"volume-title":"Auto-flow: autonomic information flows for critical information systems","year":"2006","author":"Schwan, K.","key":"atypb37"},{"key":"atypb38","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.938"},{"key":"atypb39","doi-asserted-by":"publisher","DOI":"10.1109\/FMPC.1999.750599"},{"volume-title":"HPDC\u201904: Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing (HPDC\u201904)","author":"Weinberg, J.","key":"atypb40"},{"volume-title":"Proceedings of the IEEE International Symposium on High Performance Distributed Computing (HPDC)","author":"Weng, L.","key":"atypb41"},{"volume-title":"Proceedings of IEEE\/ACM Supercomputing 2002","author":"Wolf, M.","key":"atypb42"},{"volume-title":"Proceedings of the IEEE Cluster Computing Conference 2005 (Cluster\u201905)","author":"Wolf, M.","key":"atypb43"},{"key":"atypb44","doi-asserted-by":"publisher","DOI":"10.1145\/1084805.1084814"},{"key":"atypb45","doi-asserted-by":"publisher","DOI":"10.1007\/11945918_26"},{"volume-title":"Proceedings of the 8th IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2008)","author":"Zang, T.","key":"atypb46"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342010369117","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342010369117","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,2]],"date-time":"2025-03-02T01:23:40Z","timestamp":1740878620000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1094342010369117"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010,5,12]]},"references-count":46,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2011,5]]}},"alternative-id":["10.1177\/1094342010369117"],"URL":"https:\/\/doi.org\/10.1177\/1094342010369117","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"type":"print","value":"1094-3420"},{"type":"electronic","value":"1741-2846"}],"subject":[],"published":{"date-parts":[[2010,5,12]]}}}