{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T09:35:00Z","timestamp":1763458500406,"version":"3.45.0"},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2017,2,12]],"date-time":"2017-02-12T00:00:00Z","timestamp":1486857600000},"content-version":"vor","delay-in-days":366,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000015","name":"U.S. Department of Energy","doi-asserted-by":"publisher","award":["DE-AC04-94AL85000"],"award-info":[{"award-number":["DE-AC04-94AL85000"]}],"id":[{"id":"10.13039\/100000015","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ACM Trans. Model. Perform. Eval. Comput. Syst."],"published-print":{"date-parts":[[2016,3,31]]},"abstract":"<jats:p>\n                    Simulation is a widely adopted method to analyze and predict the performance of large-scale parallel applications. Validating the hardware model is highly important for complex simulations with a large number of parameters. Common practice involves calculating the percent error between the projected and the real execution time of a benchmark program. However, in a high-dimensional parameter space, this coarse-grained approach often suffers from parameter insensitivity, which may not be known\n                    <jats:italic toggle=\"yes\">a priori<\/jats:italic>\n                    . Moreover, the traditional approach cannot be applied to the validation of software models, such as application skeletons used in online simulations. In this work, we present a methodology and a toolset for validating both hardware and software models by quantitatively comparing fine-grained statistical characteristics obtained from execution traces. Although statistical information has been used in tasks like performance optimization, this is the first attempt to apply it to simulation validation. Our experimental results show that the proposed evaluation approach offers significant improvement in fidelity when compared to evaluation using total execution time, and the proposed metrics serve as reliable criteria that progress toward automating the simulation tuning process.\n                  <\/jats:p>","DOI":"10.1145\/2809778","type":"journal-article","created":{"date-parts":[[2016,5,21]],"date-time":"2016-05-21T18:27:38Z","timestamp":1463855258000},"page":"1-22","source":"Crossref","is-referenced-by-count":1,"title":["Validating the Simulation of Large-Scale Parallel Applications Using Statistical Characteristics"],"prefix":"10.1145","volume":"1","author":[{"given":"Deli","family":"Zhang","sequence":"first","affiliation":[{"name":"University of Central Florida, Orlando, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jeremiah","family":"Wilke","sequence":"additional","affiliation":[{"name":"Sandia National Laboratories, Livermore, CA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gilbert","family":"Hendry","sequence":"additional","affiliation":[{"name":"Sandia National Laboratories, Livermore, CA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Damian","family":"Dechev","sequence":"additional","affiliation":[{"name":"University of Central Florida, Sandia National Laboratories, Orlando, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2016,2,12]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","unstructured":"C. Albing N. Troullier S. Whalen R. Olson J. Glenski H. Pritchard and H. Mills. 2011. Scalable node allocation for improved performance in regular and anisotropic 3D torus supercomputers. In Recent Advances in the Message Passing Interface. Springer 61--70.","DOI":"10.5555\/2042476.2042485"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2011.69"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1177\/0037549709340840"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPPW.2011.33"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.v22:6"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.4108\/ICST.SIMUTOOLS2009.5753"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1851476.1851564"},{"key":"e_1_2_1_8_1","doi-asserted-by":"crossref","unstructured":"K. A. Huck A. D. Malony S. Shende and D. W. Jacobsen. 2014. Integrated measurement for cross-platform OpenMP performance analysis. In Using and Improving OpenMP for Devices Tasks and More. Springer 146--160.","DOI":"10.1007\/978-3-319-11454-5_11"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.4018\/jdst.2010040104"},{"key":"e_1_2_1_10_1","unstructured":"C. Janssen D. Quinlan and J. Shalf. 2011. Architectural simulation for exascale hardware\/software co-design."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2009.61"},{"key":"e_1_2_1_12_1","volume-title":"VAMPIR: Visualization and Analysis of MPI Resources. Citeseer.","author":"Nagel W. E.","year":"1996","unstructured":"W. E. Nagel, A. Arnold, M. Weber, H. C. Hoppe, and K. Solchenbach. 1996. VAMPIR: Visualization and Analysis of MPI Resources. Citeseer."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11227-009-0279-4"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2488551.2488564"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPADS.2009.116"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPP.2008.71"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","unstructured":"R. Reussner P. Sanders L. Prechelt and M. M\u00fcller. 1998. SKaMPI: A detailed accurate MPI benchmark. Recent Advances in Parallel Virtual Machine and Message Passing Interface (1998) 52--59.","DOI":"10.5555\/648135.748628"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1964218.1964225"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2011.300"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1177\/1094342006064482"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/370049.370603"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2486092.2486094"},{"volume-title":"Proceedings of the 2012 19th International Conference on High Performance Computing (HiPC). IEEE, 1--9.","author":"Sudheer C. D.","key":"e_1_2_1_23_1","unstructured":"C. D. Sudheer and A. Srinivasan. 2012. Optimization of the hop-byte metric for effective topology aware mapping. In Proceedings of the 2012 19th International Conference on High Performance Computing (HiPC). IEEE, 1--9."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.5555\/1413370.1413391"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-03869-3_16"},{"key":"e_1_2_1_26_1","volume-title":"IEEE International Parallel and Distributed Processing Symposium (IPDPS","author":"Underwood K. D.","year":"2007","unstructured":"K. D. Underwood, M. Levenhagen, and A. Rodrigues. 2007. Simulating red storm: Challenges and successes in building a system simulation. In IEEE International Parallel and Distributed Processing Symposium (IPDPS 2007). IEEE, 1--10."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.4108\/ICST.SIMUTOOLS2009.5592"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/568014.379590"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISPA.2012.40"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","unstructured":"J. J. Wilke K. Sargsyan J. P. Kenny B. Debusschere H. N. Najm and G. Hendry. 2013. Validation and uncertainty assessment of extreme-scale HPC simulation through Bayesian inference. In Euro-Par 2013 Parallel Processing. Springer 41--52. 10.1007\/978-3-642-40047-6_7","DOI":"10.1007\/978-3-642-40047-6_7"},{"volume-title":"Proceedings of the 10th LCI International Conference on High-Performance Clustered Computing. 10--12","author":"Wright N. J.","key":"e_1_2_1_31_1","unstructured":"N. J. Wright, W. Pfeiffer, and A. Snavely. 2009. Characterizing parallel scaling of scientific applications using IPM. In Proceedings of the 10th LCI International Conference on High-Performance Clustered Computing. 10--12."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1941553.1941569"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837853.1693493"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2014.105"},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the 18th International Parallel and Distributed Processing Symposium","author":"Zheng G.","year":"2004","unstructured":"G. Zheng, G. Kakulapati, and L. V. Kal\u00e9. 2004. Bigsim: A parallel simulator for performance prediction of extremely large parallel machines. In Proceedings of the 18th International Parallel and Distributed Processing Symposium, 2004. IEEE, 78."}],"container-title":["ACM Transactions on Modeling and Performance Evaluation of Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2809778","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2809778","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2809778","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T09:25:57Z","timestamp":1763457957000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2809778"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,2,12]]},"references-count":35,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2016,3,31]]}},"alternative-id":["10.1145\/2809778"],"URL":"https:\/\/doi.org\/10.1145\/2809778","relation":{},"ISSN":["2376-3639","2376-3647"],"issn-type":[{"type":"print","value":"2376-3639"},{"type":"electronic","value":"2376-3647"}],"subject":[],"published":{"date-parts":[[2016,2,12]]}}}