{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,20]],"date-time":"2025-11-20T12:53:16Z","timestamp":1763643196698,"version":"3.38.0"},"reference-count":72,"publisher":"SAGE Publications","issue":"6","license":[{"start":{"date-parts":[[2022,6,12]],"date-time":"2022-06-12T00:00:00Z","timestamp":1654992000000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/www.sagepub.com\/licence-information-for-chorus"}],"funder":[{"DOI":"10.13039\/100000015","name":"US Department of Energy","doi-asserted-by":"publisher","award":["17-SC-20-SC"],"award-info":[{"award-number":["17-SC-20-SC"]}],"id":[{"id":"10.13039\/100000015","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2021,11]]},"abstract":"<jats:p> A growing disparity between supercomputer computation speeds and I\/O rates means that it is rapidly becoming infeasible to analyze supercomputer application output only after that output has been written to a file system. Instead, data-generating applications must run concurrently with data reduction and\/or analysis operations, with which they exchange information via high-speed methods such as interprocess communications. The resulting parallel computing motif, online data analysis and reduction (ODAR), has important implications for both application and HPC systems design. Here we introduce the ODAR motif and its co-design concerns, describe a co-design process for identifying and addressing those concerns, present tools that assist in the co-design process, and present case studies to illustrate the use of the process and tools in practical settings. <\/jats:p>","DOI":"10.1177\/10943420211023549","type":"journal-article","created":{"date-parts":[[2021,6,12]],"date-time":"2021-06-12T11:45:26Z","timestamp":1623498326000},"page":"617-635","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":8,"title":["Online data analysis and reduction: An important Co-design motif for extreme-scale computers"],"prefix":"10.1177","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2129-5269","authenticated-orcid":false,"given":"Ian","family":"Foster","sequence":"first","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"},{"name":"University of Chicago, Chicago, IL, USA"}]},{"given":"Mark","family":"Ainsworth","sequence":"additional","affiliation":[{"name":"Brown University, Providence, RI, USA"}]},{"given":"Julie","family":"Bessac","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]},{"given":"Franck","family":"Cappello","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]},{"given":"Jong","family":"Choi","sequence":"additional","affiliation":[{"name":"Oak Ridge National Laboratory, Oak Ridge, TN, USA"}]},{"given":"Sheng","family":"Di","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]},{"given":"Zichao","family":"Di","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]},{"given":"Ali M","family":"Gok","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]},{"given":"Hanqi","family":"Guo","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]},{"given":"Kevin A","family":"Huck","sequence":"additional","affiliation":[{"name":"University of Oregon, Eugene, OR, USA"}]},{"given":"Christopher","family":"Kelly","sequence":"additional","affiliation":[{"name":"Brookhaven National Laboratory, Upton, NY, USA"}]},{"given":"Scott","family":"Klasky","sequence":"additional","affiliation":[{"name":"Oak Ridge National Laboratory, Oak Ridge, TN, USA"}]},{"given":"Kerstin","family":"Kleese van Dam","sequence":"additional","affiliation":[{"name":"Brookhaven National Laboratory, Upton, NY, USA"}]},{"given":"Xin","family":"Liang","sequence":"additional","affiliation":[{"name":"Oak Ridge National Laboratory, Oak Ridge, TN, USA"}]},{"given":"Kshitij","family":"Mehta","sequence":"additional","affiliation":[{"name":"Oak Ridge National Laboratory, Oak Ridge, TN, USA"}]},{"given":"Manish","family":"Parashar","sequence":"additional","affiliation":[{"name":"Rutgers University, New Brunswick, NJ, USA"}]},{"given":"Tom","family":"Peterka","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]},{"given":"Line","family":"Pouchard","sequence":"additional","affiliation":[{"name":"Brookhaven National Laboratory, Upton, NY, USA"}]},{"given":"Tong","family":"Shu","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"},{"name":"Southern Illinois University, Carbondale, IL, USA"}]},{"given":"Ozan","family":"Tugluk","sequence":"additional","affiliation":[{"name":"Brown University, Providence, RI, USA"}]},{"given":"Hubertus","family":"van Dam","sequence":"additional","affiliation":[{"name":"Brookhaven National Laboratory, Upton, NY, USA"}]},{"given":"Lipeng","family":"Wan","sequence":"additional","affiliation":[{"name":"Oak Ridge National Laboratory, Oak Ridge, TN, USA"}]},{"given":"Matthew","family":"Wolf","sequence":"additional","affiliation":[{"name":"Oak Ridge National Laboratory, Oak Ridge, TN, USA"}]},{"given":"Justin M","family":"Wozniak","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]},{"given":"Wei","family":"Xu","sequence":"additional","affiliation":[{"name":"Brookhaven National Laboratory, Upton, NY, USA"}]},{"given":"Igor","family":"Yakushin","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]},{"given":"Shinjae","family":"Yoo","sequence":"additional","affiliation":[{"name":"Brookhaven National Laboratory, Upton, NY, USA"}]},{"given":"Todd","family":"Munson","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]}],"member":"179","published-online":{"date-parts":[[2021,6,12]]},"reference":[{"doi-asserted-by":"publisher","key":"bibr1-10943420211023549","DOI":"10.1109\/HPDC.1995.518701"},{"doi-asserted-by":"publisher","key":"bibr2-10943420211023549","DOI":"10.1137\/19M1267878"},{"doi-asserted-by":"publisher","key":"bibr3-10943420211023549","DOI":"10.1137\/18M1166651"},{"doi-asserted-by":"publisher","key":"bibr4-10943420211023549","DOI":"10.1098\/rsta.2019.0056"},{"year":"2020","author":"Alexander FJ","journal-title":"International Journal of High-Performance Computing Applications","key":"bibr5-10943420211023549"},{"volume-title":"The Landscape of parallel Computing Research: A View from Berkeley","year":"2006","author":"Asanovic K","key":"bibr6-10943420211023549"},{"doi-asserted-by":"publisher","key":"bibr7-10943420211023549","DOI":"10.1109\/SC.2016.78"},{"doi-asserted-by":"publisher","key":"bibr8-10943420211023549","DOI":"10.1088\/0741-3335\/44\/5\/304"},{"doi-asserted-by":"publisher","key":"bibr9-10943420211023549","DOI":"10.1109\/JPROC.2018.2841200"},{"key":"bibr10-10943420211023549","first-page":"141","volume-title":"Transition of HPC towards Exascale Computing","author":"Barrett RF","year":"2013"},{"doi-asserted-by":"publisher","key":"bibr11-10943420211023549","DOI":"10.1111\/cgf.12930"},{"doi-asserted-by":"publisher","key":"bibr12-10943420211023549","DOI":"10.1145\/369028.369136"},{"unstructured":"Brace A, Lee H, Ma H, et al. (2021) Achieving 100X faster simulations of complex biological phenomena by coupling ML to HPC ensembles. arXiv preprint arXiv:2104.04797.","key":"bibr13-10943420211023549"},{"doi-asserted-by":"publisher","key":"bibr14-10943420211023549","DOI":"10.1145\/63334.63337"},{"doi-asserted-by":"publisher","key":"bibr15-10943420211023549","DOI":"10.1177\/1094342020935991"},{"doi-asserted-by":"publisher","key":"bibr16-10943420211023549","DOI":"10.1109\/eScience.2018.00133"},{"doi-asserted-by":"publisher","key":"bibr17-10943420211023549","DOI":"10.1109\/DRBSD-549595.2019.00011"},{"doi-asserted-by":"publisher","key":"bibr18-10943420211023549","DOI":"10.1109\/CLUSTER.2017.67"},{"doi-asserted-by":"publisher","key":"bibr19-10943420211023549","DOI":"10.1145\/50087.50089"},{"doi-asserted-by":"publisher","key":"bibr20-10943420211023549","DOI":"10.1109\/5.558708"},{"doi-asserted-by":"publisher","key":"bibr21-10943420211023549","DOI":"10.1109\/IPDPS.2016.11"},{"doi-asserted-by":"publisher","key":"bibr22-10943420211023549","DOI":"10.1137\/18M1168832"},{"doi-asserted-by":"publisher","key":"bibr23-10943420211023549","DOI":"10.1007\/s10586-011-0162-y"},{"doi-asserted-by":"publisher","key":"bibr24-10943420211023549","DOI":"10.1063\/1.5044707"},{"doi-asserted-by":"publisher","key":"bibr25-10943420211023549","DOI":"10.1109\/CLUSTER.2016.63"},{"doi-asserted-by":"publisher","key":"bibr26-10943420211023549","DOI":"10.1016\/j.ipl.2005.11.003"},{"volume-title":"Designing and Building Parallel Programs","year":"1994","author":"Foster I","key":"bibr27-10943420211023549"},{"doi-asserted-by":"publisher","key":"bibr28-10943420211023549","DOI":"10.1109\/HiPC.2017.00042"},{"doi-asserted-by":"publisher","key":"bibr29-10943420211023549","DOI":"10.1109\/2.809249"},{"doi-asserted-by":"publisher","key":"bibr30-10943420211023549","DOI":"10.1109\/MC.2006.397"},{"doi-asserted-by":"publisher","key":"bibr31-10943420211023549","DOI":"10.1016\/j.softx.2020.100561"},{"doi-asserted-by":"publisher","key":"bibr32-10943420211023549","DOI":"10.1109\/ICDE.2000.839382"},{"doi-asserted-by":"publisher","key":"bibr33-10943420211023549","DOI":"10.1109\/TVCG.2021.3073399"},{"unstructured":"Guo H, Xu J, Liang X, et al. (2020) FTK: the feature tracking kit. Available at: http:\/\/www.github.com\/CODARcode\/ftk (accessed 1 September 2020).","key":"bibr34-10943420211023549"},{"unstructured":"Ha S, Jeong W, Matyasfalvi G, et al. (2020) Chimbuko: a workflow-level scalable performance trace analysis tool. arXiv preprint arXiv:2008.13742","key":"bibr35-10943420211023549"},{"issue":"1","key":"bibr36-10943420211023549","first-page":"23","volume":"26","author":"He W","year":"2019","journal-title":"IEEE Transactions on Visualization & Computer Graphics"},{"doi-asserted-by":"publisher","key":"bibr37-10943420211023549","DOI":"10.1145\/2063348.2063356"},{"doi-asserted-by":"publisher","key":"bibr38-10943420211023549","DOI":"10.1109\/TVCG.2007.1017"},{"doi-asserted-by":"publisher","key":"bibr39-10943420211023549","DOI":"10.1109\/2.809252"},{"key":"bibr40-10943420211023549","first-page":"1","author":"Klasky S","year":"2011","journal-title":"Proceedings of SciDAC"},{"doi-asserted-by":"publisher","key":"bibr41-10943420211023549","DOI":"10.1145\/3144769.3144778"},{"doi-asserted-by":"publisher","key":"bibr42-10943420211023549","DOI":"10.1109\/DLS49591.2019.00007"},{"doi-asserted-by":"publisher","key":"bibr43-10943420211023549","DOI":"10.1109\/BigData.2018.8622520"},{"doi-asserted-by":"publisher","key":"bibr44-10943420211023549","DOI":"10.1088\/1742-6596\/78\/1\/012043"},{"doi-asserted-by":"publisher","key":"bibr45-10943420211023549","DOI":"10.1109\/SC.2018.00027"},{"doi-asserted-by":"publisher","key":"bibr46-10943420211023549","DOI":"10.1109\/SC.2016.59"},{"doi-asserted-by":"publisher","key":"bibr47-10943420211023549","DOI":"10.1145\/2807591.2807656"},{"doi-asserted-by":"publisher","key":"bibr48-10943420211023549","DOI":"10.1109\/WORKS49585.2019.00007"},{"volume-title":"A Philosophy of Software Design","year":"2018","author":"Ousterhout J","key":"bibr49-10943420211023549"},{"doi-asserted-by":"publisher","key":"bibr50-10943420211023549","DOI":"10.1109\/WSC.2016.7822090"},{"doi-asserted-by":"publisher","key":"bibr51-10943420211023549","DOI":"10.1007\/978-3-642-48354-7_20"},{"doi-asserted-by":"publisher","key":"bibr52-10943420211023549","DOI":"10.1109\/NYSDS.2018.8538951"},{"doi-asserted-by":"publisher","key":"bibr53-10943420211023549","DOI":"10.1109\/DCC.2006.35"},{"doi-asserted-by":"publisher","key":"bibr54-10943420211023549","DOI":"10.1016\/j.bdr.2016.10.002"},{"doi-asserted-by":"publisher","key":"bibr55-10943420211023549","DOI":"10.1177\/1094342006064482"},{"doi-asserted-by":"crossref","unstructured":"Shu T, Guo Y, Wozniak J, et al. (2020) In-situ workflow auto-tuning via combining performance models of component applications. arXiv preprint arXiv:2008.06991","key":"bibr56-10943420211023549","DOI":"10.1145\/3458817.3476197"},{"volume-title":"Reaction-Diffusion Tutorial","year":"2020","author":"Sims K","key":"bibr57-10943420211023549"},{"doi-asserted-by":"publisher","key":"bibr58-10943420211023549","DOI":"10.1145\/155332.155334"},{"doi-asserted-by":"publisher","key":"bibr59-10943420211023549","DOI":"10.1177\/10943420211019119"},{"doi-asserted-by":"publisher","key":"bibr60-10943420211023549","DOI":"10.1109\/IPDPS.2017.115"},{"doi-asserted-by":"publisher","key":"bibr61-10943420211023549","DOI":"10.1177\/1094342017737147"},{"unstructured":"Underwood R, Di S, Cappello F (2020) CODARcode\/libpressio. Available at: http:\/\/www.github.com\/CODARcode\/libpressio.","key":"bibr62-10943420211023549"},{"doi-asserted-by":"publisher","key":"bibr63-10943420211023549","DOI":"10.1016\/j.cpc.2010.04.018"},{"key":"bibr64-10943420211023549","first-page":"95","volume-title":"13th international conference on cluster, cloud and grid computing","author":"Wozniak JM","year":"2013"},{"doi-asserted-by":"publisher","key":"bibr65-10943420211023549","DOI":"10.1109\/MLHPC.2018.8638629"},{"doi-asserted-by":"publisher","key":"bibr66-10943420211023549","DOI":"10.1016\/j.future.2019.05.020"},{"doi-asserted-by":"publisher","key":"bibr67-10943420211023549","DOI":"10.1186\/s12859-018-2508-4"},{"doi-asserted-by":"publisher","key":"bibr68-10943420211023549","DOI":"10.1109\/TVCG.2018.2865026"},{"doi-asserted-by":"publisher","key":"bibr69-10943420211023549","DOI":"10.1145\/3409390.3409400"},{"doi-asserted-by":"publisher","key":"bibr70-10943420211023549","DOI":"10.1109\/ICDE.2016.7498277"},{"doi-asserted-by":"publisher","key":"bibr71-10943420211023549","DOI":"10.1109\/NYSDS.2018.8538942"},{"doi-asserted-by":"publisher","key":"bibr72-10943420211023549","DOI":"10.1145\/3369583.3392688"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420211023549","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/10943420211023549","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420211023549","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420211023549","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,2]],"date-time":"2025-03-02T16:21:25Z","timestamp":1740932485000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/10943420211023549"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,12]]},"references-count":72,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,11]]}},"alternative-id":["10.1177\/10943420211023549"],"URL":"https:\/\/doi.org\/10.1177\/10943420211023549","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"type":"print","value":"1094-3420"},{"type":"electronic","value":"1741-2846"}],"subject":[],"published":{"date-parts":[[2021,6,12]]}}}