{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T15:46:49Z","timestamp":1772725609781,"version":"3.50.1"},"reference-count":68,"publisher":"Association for Computing Machinery (ACM)","issue":"5s","license":[{"start":{"date-parts":[[2017,9,27]],"date-time":"2017-09-27T00:00:00Z","timestamp":1506470400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Google and AMD"},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CCF-1527463"],"award-info":[{"award-number":["CCF-1527463"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2017,10,31]]},"abstract":"<jats:p>This paper presents BenchPrime, an automated benchmark analysis toolset that is systematic and extensible to analyze the similarity and diversity of benchmark suites. BenchPrime takes multiple benchmark suites and their evaluation metrics as inputs and generates a hybrid benchmark suite comprising only essential applications. Unlike prior work, BenchPrime uses linear discriminant analysis rather than principal component analysis, as well as selects the best clustering algorithm and the optimized number of clusters in an automated and metric-tailored way, thereby achieving high accuracy. In addition, BenchPrime ranks the benchmark suites in terms of their application set diversity and estimates how unique each benchmark suite is compared to other suites.<\/jats:p>\n          <jats:p>As a case study, this work for the first time compares the DenBench with the MediaBench and MiBench using four different metrics to provide a multi-dimensional understanding of the benchmark suites. For each metric, BenchPrime measures to what degree DenBench applications are irreplaceable with those in MediaBench and MiBench. This provides means for identifying an essential subset from the three benchmark suites without compromising the application balance of the full set. The experimental results show that the necessity of including DenBench applications varies across the target metrics and that significant redundancy exists among the three benchmark suites.<\/jats:p>","DOI":"10.1145\/3126499","type":"journal-article","created":{"date-parts":[[2017,9,27]],"date-time":"2017-09-27T12:33:53Z","timestamp":1506515633000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":14,"title":["BenchPrime"],"prefix":"10.1145","volume":"16","author":[{"given":"Qingrui","family":"Liu","sequence":"first","affiliation":[{"name":"Virginia Tech, Kraft Drive,Blacksburg, VA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaolong","family":"Wu","sequence":"additional","affiliation":[{"name":"Virginia Tech, Kraft Drive,Blacksburg, VA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Larry","family":"Kittinger","sequence":"additional","affiliation":[{"name":"Virginia Tech, Kraft Drive,Blacksburg, VA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Markus","family":"Levy","sequence":"additional","affiliation":[{"name":"EEMBC"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Changhee","family":"Jung","sequence":"additional","affiliation":[{"name":"Virginia Tech, Kraft Drive,Blacksburg, VA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,9,27]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Vignesh Adhinarayanan and Wu-chun Feng. 2015. An Automated Framework for Characterizing and Subsetting GPGPU Workloads. (2015).  Vignesh Adhinarayanan and Wu-chun Feng. 2015. An Automated Framework for Characterizing and Subsetting GPGPU Workloads. (2015).","DOI":"10.1109\/ISPASS.2016.7482105"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2016.7581275"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1201\/b17320"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.908974"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2007.443"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2024716.2024718"},{"key":"e_1_2_1_7_1","unstructured":"C.M. Bishop and etal 2006. Pattern Recognition and Machine Learning. Springer New York.   C.M. Bishop and et al. 2006. Pattern Recognition and Machine Learning. Springer New York."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1629395.1629413"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v025.i04"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/339647.339657"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/1778978.1778996"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of The 54th Annual Design Automation Conference (DAC).","author":"Didehban Moslem","year":"2017"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897937.2898054"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCAD.2017.8203792"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/645989.674311"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/1128020.1128563"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2006.302732"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1152154.1152174"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2008.4636100"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2015.7095786"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2013.14"},{"key":"e_1_2_1_22_1","volume-title":"Characterizing and Subsetting Big Data Workloads. CoRR abs\/1409.0792","author":"Jia Zhen","year":"2014"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TC.2006.85"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1669112.1669122"},{"key":"e_1_2_1_26_1","volume-title":"Proc. of the Workshop on Parallel Execution of Sequential Programs on Multicore Architectures. 2--11","author":"Jung Changhee","year":"2009"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568311"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1065944.1065976"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.5555\/1898953.1899071"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1993498.1993509"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1289927.1289961"},{"key":"e_1_2_1_32_1","first-page":"526","article-title":"Adaptive Execution Method for Multithreaded Processor-based Parallel System. (April 28 2009)","volume":"7","author":"Jung Chang Hee","year":"2009","journal-title":"US Patent"},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Soontae Kim N. Vijaykrishnan Mahmut Kandemir Anand Sivasubrmaniam and M. J. Irwin. 2003. Partitioned Instruction Cache Architecture for Energy Efficiency. 2 2 (March 2003) 163--165.  Soontae Kim N. Vijaykrishnan Mahmut Kandemir Anand Sivasubrmaniam and M. J. Irwin. 2003. Partitioned Instruction Cache Architecture for Energy Efficiency. 2 2 (March 2003) 163--165.","DOI":"10.1145\/643470.643473"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2744769.2744846"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.5555\/977395.977673"},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of the 30th Annual ACM\/IEEE International Symposium on Microarchitecture (MICRO 30)","author":"Lee Chunho"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2008.224"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2009.10.008"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1176760.1176810"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568307"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1669112.1669172"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1150019.1136494"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/NVMSA.2016.7547183"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2670529.2754959"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.5555\/3014904.3014931"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2930667"},{"key":"e_1_2_1_47_1","volume-title":"Proceedings of the 49th International Symposium on Microarchitecture (MICRO).","author":"Liu Qingrui","year":"2016"},{"key":"e_1_2_1_48_1","volume-title":"Bugbench: Benchmarks for Evaluating Bug Detection Tools. In In Workshop on the Evaluation of Software Defect Detection Tools.","author":"Lu Shan","year":"2005"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2005.37"},{"key":"e_1_2_1_50_1","unstructured":"S. S. Mukherjee C. Weaver J. Emer S. Reinhardt and T. Austin. 2003. A Systematic Methodology to Compute the Architectural Vulnerability Factors for a High Performance Microprocessor. 29--42.   S. S. Mukherjee C. Weaver J. Emer S. Reinhardt and T. Austin. 2003. A Systematic Methodology to Compute the Architectural Vulnerability Factors for a High Performance Microprocessor. 29--42."},{"key":"e_1_2_1_51_1","volume-title":"Psychometric theory","author":"Nunnally J. C."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2968456.2968469"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2005.1430555"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273440.1250713"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/1250662.1250713"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273440.1250713"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/MM.2009.74"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/1065910.1065929"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/605397.605403"},{"key":"e_1_2_1_60_1","doi-asserted-by":"crossref","unstructured":"Aviral Shrivastava Illya Issenin and Nikil Dutt. 2005. Compilation Techniques for Energy Reduction in Horizontally Partitioned Cache Architectures.  Aviral Shrivastava Illya Issenin and Nikil Dutt. 2005. Compilation Techniques for Energy Reduction in Horizontally Partitioned Cache Architectures.","DOI":"10.1145\/1086297.1086310"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2593069.2593195"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/1816038.1816023"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2012.6189228"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/92.335012"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2038642.2038650"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2006.302733"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/2384616.2384651"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3037697.3037708"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/2872362.2872384"}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3126499","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3126499","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3126499","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T19:05:01Z","timestamp":1750273501000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3126499"}},"subtitle":["Effective Building of a Hybrid Benchmark Suite"],"short-title":[],"issued":{"date-parts":[[2017,9,27]]},"references-count":68,"journal-issue":{"issue":"5s","published-print":{"date-parts":[[2017,10,31]]}},"alternative-id":["10.1145\/3126499"],"URL":"https:\/\/doi.org\/10.1145\/3126499","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"value":"1539-9087","type":"print"},{"value":"1558-3465","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,9,27]]},"assertion":[{"value":"2017-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-09-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}