{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,9]],"date-time":"2025-09-09T21:08:39Z","timestamp":1757452119005,"version":"3.41.0"},"reference-count":30,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2021,3,8]],"date-time":"2021-03-08T00:00:00Z","timestamp":1615161600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100004063","name":"Knut and Alice Wallenberg Foundation","doi-asserted-by":"crossref","award":["2015.0153"],"award-info":[{"award-number":["2015.0153"]}],"id":[{"id":"10.13039\/501100004063","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["715283"],"award-info":[{"award-number":["715283"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2021,6,30]]},"abstract":"<jats:p>The SPEC CPU Benchmarks are used extensively for evaluating and comparing improvements to computer systems. This ubiquity makes characterization critical for researchers to understand the bottlenecks the benchmarks do and do not expose and where new designs should and should not be expected to show impact. However, in characterization there is a tradeoff between accuracy and reusability: The more precisely we characterize a benchmark\u2019s performance on a given system, the less usable it is across different micro-architectures and varying memory configurations. For SPEC, most existing characterizations include system-specific effects (e.g., via performance counters) and\/or only look at aggregate behavior (e.g., averages over the full application execution). While such approaches simplify characterization, they make it difficult to separate the applications\u2019 intrinsic behavior from the system-specific effects and\/or lose the diverse phase-based behaviors.<\/jats:p>\n          <jats:p>In this work we focus on characterizing the applications\u2019 intrinsic memory behaviour by isolating them from micro-architectural configuration specifics. We do this by providing a simplified generic system model that evaluates the applications\u2019 memory behavior across multiple cache sizes, with and without prefetching, and over time. The resulting characterization can be reused across a range of systems to understand application behavior and allow us to see how frequently different behaviors occur. We use this approach to compare the SPEC 2006 and 2017 suites, providing insight into their memory system behaviour beyond previous system-specific and\/or aggregate results. We demonstrate the ability to use this characterization in different contexts by showing a portion of the SPEC 2017 benchmark suite that could benefit from giga-scale caches, despite aggregate results indicating otherwise.<\/jats:p>","DOI":"10.1145\/3446200","type":"journal-article","created":{"date-parts":[[2021,3,8]],"date-time":"2021-03-08T17:06:27Z","timestamp":1615223187000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["A Reusable Characterization of the Memory System Behavior of SPEC2017 and SPEC2006"],"prefix":"10.1145","volume":"18","author":[{"given":"Muhammad","family":"Hassan","sequence":"first","affiliation":[{"name":"Uppsala University, Uppsala, Sweden"}]},{"given":"Chang Hyun","family":"Park","sequence":"additional","affiliation":[{"name":"Uppsala University, Uppsala, Sweden"}]},{"given":"David","family":"Black-Schaffer","sequence":"additional","affiliation":[{"name":"Uppsala University, Uppsala, Sweden"}]}],"member":"320","published-online":{"date-parts":[[2021,3,8]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2020. ChampSim. Retrieved from https:\/\/github.com\/ChampSim\/ChampSim.  2020. ChampSim. Retrieved from https:\/\/github.com\/ChampSim\/ChampSim."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2004.1291352"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/781131.781159"},{"key":"e_1_2_1_4_1","volume-title":"Hill","author":"Edler Jan","year":"2003","unstructured":"Jan Edler and Mark D . Hill . 2003 . Dinero. Retrieved from http:\/\/pages.cs.wisc.edu\/ markhill\/DineroIV\/. Jan Edler and Mark D. Hill. 2003. Dinero. Retrieved from http:\/\/pages.cs.wisc.edu\/ markhill\/DineroIV\/."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2010.5452069"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1241601.1241619"},{"key":"e_1_2_1_7_1","volume-title":"Chang Hyun Park, and David Black-Schaffer","author":"Hassan Muhammad","year":"2020","unstructured":"Muhammad Hassan , Chang Hyun Park, and David Black-Schaffer . 2020 . Architecturally-independent and time-based characterization of SPEC CPU 2017. Retrieved from http:\/\/urn.kb.se\/resolve?urn&equals;urn:nbn:se:uu:diva-417174. Muhammad Hassan, Chang Hyun Park, and David Black-Schaffer. 2020. Architecturally-independent and time-based characterization of SPEC CPU 2017. Retrieved from http:\/\/urn.kb.se\/resolve?urn&equals;urn:nbn:se:uu:diva-417174."},{"key":"e_1_2_1_8_1","volume-title":"Chang Hyun Park, and David Black-Schaffer","author":"Hassan Muhammad","year":"2020","unstructured":"Muhammad Hassan , Chang Hyun Park, and David Black-Schaffer . 2020 . Raw-data : A reusable characterization of the memory system behavior of SPEC 2017 and SPEC 2006. Retrieved from http:\/\/urn.kb.se\/resolve?urn&equals;urn:nbn:se:uu:diva-428738. Muhammad Hassan, Chang Hyun Park, and David Black-Schaffer. 2020. Raw-data: A reusable characterization of the memory system behavior of SPEC 2017 and SPEC 2006. Retrieved from http:\/\/urn.kb.se\/resolve?urn&equals;urn:nbn:se:uu:diva-428738."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1241601.1241618"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSAMOS.2008.4664856"},{"key":"e_1_2_1_12_1","unstructured":"Osnat Levi. 2018. Pin\u2014A dynamic binary instrumentation tool. Retrieved from https:\/\/software.intel.com\/en-us\/articles\/pin-a-dynamic-binary-instrumentation-tool.  Osnat Levi. 2018. Pin\u2014A dynamic binary instrumentation tool. Retrieved from https:\/\/software.intel.com\/en-us\/articles\/pin-a-dynamic-binary-instrumentation-tool."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2018.00028"},{"key":"e_1_2_1_14_1","volume-title":"Memory hierarchy characterization of SPEC CPU2006 and SPEC CPU2017 on the Intel Xeon Skylake-SP. PLoS One 14, 8 (8","author":"Navarro-Torres Agust\u00edn","year":"2019","unstructured":"Agust\u00edn Navarro-Torres , Jes\u00fas Alastruey-Bened\u00e9 , Pablo Ib\u00e1\u00f1ez-Mar\u00edn , and V\u00edctor Vi\u00f1als-Y\u00fafera . 2019. Memory hierarchy characterization of SPEC CPU2006 and SPEC CPU2017 on the Intel Xeon Skylake-SP. PLoS One 14, 8 (8 2019 ), 1--24. DOI:https:\/\/doi.org\/10.1371\/journal.pone.0220135 Agust\u00edn Navarro-Torres, Jes\u00fas Alastruey-Bened\u00e9, Pablo Ib\u00e1\u00f1ez-Mar\u00edn, and V\u00edctor Vi\u00f1als-Y\u00fafera. 2019. Memory hierarchy characterization of SPEC CPU2006 and SPEC CPU2017 on the Intel Xeon Skylake-SP. PLoS One 14, 8 (8 2019), 1--24. DOI:https:\/\/doi.org\/10.1371\/journal.pone.0220135"},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the IEEE International Symposium on High Performance Computer Architecture (HPCA\u201918)","author":"Panda Reena","year":"2018","unstructured":"Reena Panda , Shuang Song , Joseph Dean , and Lizy K. John . 2018. Wait of a Decade: Did SPEC CPU 2017 broaden the performance horizon? In Proceedings of the IEEE International Symposium on High Performance Computer Architecture (HPCA\u201918) . 271--282. DOI:https:\/\/doi.org\/10.1109\/HPCA. 2018 .00032 Reena Panda, Shuang Song, Joseph Dean, and Lizy K. John. 2018. Wait of a Decade: Did SPEC CPU 2017 broaden the performance horizon? In Proceedings of the IEEE International Symposium on High Performance Computer Architecture (HPCA\u201918). 271--282. DOI:https:\/\/doi.org\/10.1109\/HPCA.2018.00032"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1250662.1250713"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1241601.1241616"},{"key":"e_1_2_1_18_1","first-page":"36","article-title":"Performance characterization of spec cpu2006 benchmarks on intel core 2 duo processor","volume":"2","author":"Prakash Tribuvan Kumar","year":"2008","unstructured":"Tribuvan Kumar Prakash and Lu Peng . 2008 . Performance characterization of spec cpu2006 benchmarks on intel core 2 duo processor . ISAST Trans. Comput. Softw. Eng. 2 , 1 (2008), 36 -- 41 . Tribuvan Kumar Prakash and Lu Peng. 2008. Performance characterization of spec cpu2006 benchmarks on intel core 2 duo processor. ISAST Trans. Comput. Softw. Eng. 2, 1 (2008), 36--41.","journal-title":"ISAST Trans. Comput. Softw. Eng."},{"volume-title":"SPEC CPU2017: Performance, event, and energy characterization on the core i7-8700K. In Proceedings of the 2019 ACM\/SPEC International Conference on Performance Engineering, (ICPE\u201919)","author":"Ranjan Hebbar S.","key":"e_1_2_1_19_1","unstructured":"Ranjan Hebbar S. R. and Aleksandar Milenkovic. 2019 . SPEC CPU2017: Performance, event, and energy characterization on the core i7-8700K. In Proceedings of the 2019 ACM\/SPEC International Conference on Performance Engineering, (ICPE\u201919) .111--118. DOI:https:\/\/doi.org\/10.1145\/3297663.3310314 Ranjan Hebbar S. R. and Aleksandar Milenkovic. 2019. SPEC CPU2017: Performance, event, and energy characterization on the core i7-8700K. In Proceedings of the 2019 ACM\/SPEC International Conference on Performance Engineering, (ICPE\u201919).111--118. DOI:https:\/\/doi.org\/10.1145\/3297663.3310314"},{"volume-title":"SPEC CPU2017: Performance, event, and energy characterization on the core I7-8700K. In Proceedings of the 2019 ACM\/SPEC International Conference on Performance Engineering (ICPE\u201919)","author":"Ranjan Hebbar S.","key":"e_1_2_1_20_1","unstructured":"Ranjan Hebbar S. R. and Aleksandar Milenkovi\u0107. 2019 . SPEC CPU2017: Performance, event, and energy characterization on the core I7-8700K. In Proceedings of the 2019 ACM\/SPEC International Conference on Performance Engineering (ICPE\u201919) . ACM, New York, NY, 111--118. DOI:https:\/\/doi.org\/10.1145\/3297663.3310314 Ranjan Hebbar S. R. and Aleksandar Milenkovi\u0107. 2019. SPEC CPU2017: Performance, event, and energy characterization on the core I7-8700K. In Proceedings of the 2019 ACM\/SPEC International Conference on Performance Engineering (ICPE\u201919). ACM, New York, NY, 111--118. DOI:https:\/\/doi.org\/10.1145\/3297663.3310314"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2015.29"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2259016.2259040"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2011.6114207"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1024393.1024414"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3297663.3310311"},{"key":"e_1_2_1_26_1","volume-title":"John","author":"Song Shuang","year":"2018","unstructured":"Shuang Song , Qinzhe Wu , Steven Flolid , Joseph Dean , Reena Panda , Junyong Deng , and Lizy K . John . May 2018 . Experiments with Spec cpu 2017: Similarity, Balance, Phase Behavior and Simpoints. Technical Report. TR-180515-01, LCA Group, Department of Electrical and Computer Engineering, The University of Texas at Austin. Shuang Song, Qinzhe Wu, Steven Flolid, Joseph Dean, Reena Panda, Junyong Deng, and Lizy K. John. May 2018. Experiments with Spec cpu 2017: Similarity, Balance, Phase Behavior and Simpoints. Technical Report. TR-180515-01, LCA Group, Department of Electrical and Computer Engineering, The University of Texas at Austin."},{"key":"e_1_2_1_27_1","volume-title":"SPEC CPU2006","author":"Standard Performance Evaluation Corporation","year":"2018","unstructured":"Standard Performance Evaluation Corporation . 2018 . SPEC CPU2006 . Retrieved from https:\/\/www.spec.org\/cpu 2006. Standard Performance Evaluation Corporation. 2018. SPEC CPU2006. Retrieved from https:\/\/www.spec.org\/cpu2006."},{"key":"e_1_2_1_28_1","volume-title":"SPEC CPU2017","author":"Standard Performance Evaluation Corporation","year":"2020","unstructured":"Standard Performance Evaluation Corporation . 2020 . SPEC CPU2017 . Retrieved from https:\/\/www.spec.org\/cpu 2017. Standard Performance Evaluation Corporation. 2020. SPEC CPU2017. Retrieved from https:\/\/www.spec.org\/cpu2017."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2005.59"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2018.8573479"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/PACT.2003.1238004"}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3446200","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3446200","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:47:04Z","timestamp":1750193224000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3446200"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,8]]},"references-count":30,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,6,30]]}},"alternative-id":["10.1145\/3446200"],"URL":"https:\/\/doi.org\/10.1145\/3446200","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"type":"print","value":"1544-3566"},{"type":"electronic","value":"1544-3973"}],"subject":[],"published":{"date-parts":[[2021,3,8]]},"assertion":[{"value":"2020-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-12-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-03-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}