{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T09:27:19Z","timestamp":1768901239421,"version":"3.49.0"},"reference-count":59,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2025,3,20]],"date-time":"2025-03-20T00:00:00Z","timestamp":1742428800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"crossref","award":["62272176 and 62302180"],"award-info":[{"award-number":["62272176 and 62302180"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2022YFC3802101, 2023YFB3308301"],"award-info":[{"award-number":["2022YFC3802101, 2023YFB3308301"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Shenzhen Science and Technology Program","award":["JCYJ20220818101607015"],"award-info":[{"award-number":["JCYJ20220818101607015"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2025,3,31]]},"abstract":"<jats:p>We find existing benchmark suites for smartphone CPU micro-architecture design such as Geekbench 5.0 fail to authentically represent the micro-architecture-level performance behavior of widely used real Android applications with interactive operations such as screen sliding. It is therefore crucial to systematically construct a benchmark suite as a supplementary to Geekbench to represent the user interaction behavior of Android applications for CPU micro-architecture design. The key is to identify a small number of representative programs from a large number of real applications. To this end, a set of features used to represent a program need to be constructed, and these features should be fair for different micro-architectures and can be collected efficiently. However, this is extremely difficult for Android applications. For example, the feature collection tools for Android applications are unavailable for benchmark selection.<\/jats:p>\n          <jats:p>\n            In this article, we propose a novel benchmark suite construction approach dubbed BEMAP to efficiently build a supplementary benchmark suite from real-world Android applications to represent their user interaction behavior.\n            <jats:xref ref-type=\"fn\">\n              <jats:sup>1<\/jats:sup>\n            <\/jats:xref>\n            BEMAP innovates four techniques. The first technique, called two-stage RFC (representative feature construction), constructs program features from performance counters (events) to represent a program for selecting benchmarks from a large number of real Android applications in two stages. The first stage identifies a set of important performance events in terms of IPC (instructions per cycle) by employing a machine learning algorithm named SGBRT (Stochastic Gradient Boosted Regression Tree). The second stage constructs representative features based on the important performance events by using ICA (independent component analysis). The second technique, named SPC-MMA (source performance counters from multiple micro-architectures), collects the performance events from multiple mobile CPUs with different micro-architectures and mixes them as the source of RFC. The goal of these two innovations is to make the program features fair to different mobile CPU micro-architectures. The third technique, called ES (Elbow-Silhouette) approach, artfully leverages the synergy between the elbow method and the silhouette method to determine an optimal\n            <jats:italic>K<\/jats:italic>\n            when we use K-Means to group Android applications. The fourth technique is that we design a new tool named AutoProfiler to automatically profile the micro-architecture events (e.g., IPC, L1 Icache misses) of Android applications with interactive operations.\n          <\/jats:p>\n          <jats:p>\n            Using the proposed BEMAP methodology,\n            <jats:xref ref-type=\"fn\">\n              <jats:sup>2<\/jats:sup>\n            <\/jats:xref>\n            we constructed SPBench, a novel benchmark suite supplementary to traditional mobile benchmark suites like Geekbench, for mobile CPU micro-architecture design. It consists of 15 benchmarks selected from 100 real Android applications with 3 common user interaction operations, which is the fifth innovation of this article. The experimental results on four significantly different micro-architectures show that SPBench can represent the micro-architecture performance behaviors of the 100 real-world applications with 3 common user interactive operations on each micro-architecture with significantly higher accuracy than benchmark suites produced by the state-of-the-art approaches.\n          <\/jats:p>","DOI":"10.1145\/3701999","type":"journal-article","created":{"date-parts":[[2024,10,29]],"date-time":"2024-10-29T10:11:59Z","timestamp":1730196719000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Constructing a Supplementary Benchmark Suite to Represent Android Applications with User Interactions by using Performance Counters"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8698-2356","authenticated-orcid":false,"given":"Chenghao","family":"Ouyang","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China and Center of Heterogeneous Intelligent Computer Architecture and Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1900-7774","authenticated-orcid":false,"given":"Jinhan","family":"Xin","sequence":"additional","affiliation":[{"name":"Shuhai Lab, Huawei Cloud Computing Technologies Co Ltd, Guiyang, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-0343-8285","authenticated-orcid":false,"given":"Siqi","family":"Zeng","sequence":"additional","affiliation":[{"name":"Center of Heterogeneous Intelligent Computer Architecture and Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6984-1914","authenticated-orcid":false,"given":"Guohui","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5265-7624","authenticated-orcid":false,"given":"Jianjun","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8067-9612","authenticated-orcid":false,"given":"Zhibin","family":"Yu","sequence":"additional","affiliation":[{"name":"Center of Heterogeneous Intelligent Computer Architecture and Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China and Shuhai Lab, Huawei Cloud Computing Technologies Co Ltd, Guiyang, China"}]}],"member":"320","published-online":{"date-parts":[[2025,3,20]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/2541228.2555294"},{"key":"e_1_3_3_3_2","unstructured":"Derek Bruening. 2004. Efficient Transparent and Comprehensive Runtime Code Manipulation. Ph.D. thesis Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science."},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.3390\/s22020513"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/IWSSIP.2013.6623485"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/1065010.1065034"},{"key":"e_1_3_3_7_2","first-page":"368","volume-title":"International Conference on Interactive Mobile Communication Technologies and Learning (IMCL\u201914)","author":"Christou Charalambos S.","year":"2014","unstructured":"Charalambos S. Christou. 2014. Investigating smartphone and smartphone-apps utilization: A survey among Cypriot university students. In International Conference on Interactive Mobile Communication Technologies and Learning (IMCL\u201914). 368\u2013372."},{"key":"e_1_3_3_8_2","unstructured":"Standard Performance Evaluation Corporation. 2017. SPEC CPU 2017. Retrieved from https:\/\/www.spec.org\/cpu2017\/"},{"key":"e_1_3_3_9_2","unstructured":"De Melo and Arnaldo Carvalho. 2010. The new Linux \u2018perf\u2019 tools. Slides from Linux Kongress 18 (2010) 1\u201342. http:\/\/oldvger.kernel.org\/acme\/perf\/lk2010-perf-acme.pdf"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2016.09.001"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2656.2008.01390.x"},{"key":"e_1_3_3_12_2","unstructured":"Futuremark. 2018. 3DMark for Android. Retrieved from https:\/\/www.futuremark.com\/benchmarks\/3dmark-android"},{"key":"e_1_3_3_13_2","unstructured":"Geekbench. 2019. Workload in Geekbench. Retrieved from https:\/\/www.geekbench.com\/doc\/geekbench5-cpu-workloads.pdf"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/1389586.1389680"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3032970.3032979"},{"key":"e_1_3_3_16_2","first-page":"3","volume-title":"4th Annual IEEE International Workshop on Workload Characterization (WWC\u201901)","author":"Guthaus M. R.","year":"2001","unstructured":"M. R. Guthaus, J. S. Ringenberg, D. Ernst, T. M. Austin, T. Mudge, and R. B. Brown. 2001. MiBench: A free, commercially representative embedded benchmark suite. In 4th Annual IEEE International Workshop on Workload Characterization (WWC\u201901). 3\u201314."},{"key":"e_1_3_3_17_2","first-page":"81","volume-title":"IEEE International Symposium on Workload Characterization (IISWC\u201911)","author":"Gutierrez Anthony","year":"2011","unstructured":"Anthony Gutierrez, Ronald G. Dreslinski, Thomas F. Wenisch, Trevor Mudge, Ali Saidi, Chris Emmons, and Nigel Paver. 2011. Full-system analysis and characterization of interactive smartphone applications. In IEEE International Symposium on Workload Characterization (IISWC\u201911). 81\u201390."},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2016.7446054"},{"issue":"1","key":"e_1_3_3_19_2","first-page":"100","article-title":"Algorithm AS 136: A k-means clustering algorithm","volume":"28","author":"Hartigan John A.","year":"1979","unstructured":"John A. Hartigan and Manchek A. Wong. 1979. Algorithm AS 136: A k-means clustering algorithm. J. R. Stat. Societ. Series C (Appl. Stat.) 28, 1 (1979), 100\u2013108.","journal-title":"J. R. Stat. Societ. Series C (Appl. Stat.)"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/2.869367"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3349341.3349474"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2014.6844460"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/72.761722"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(00)00026-5"},{"key":"e_1_3_3_25_2","unstructured":"Google Inc.2013. UIAutomator Testing Framework. Retrieved from https:\/\/developer.android.com\/training\/testing\/ui-automator"},{"key":"e_1_3_3_26_2","first-page":"13","volume-title":"International Symposium on Performance Evaluation of Computer Telecommunication Systems","author":"Jamal M. Hasan","year":"2009","unstructured":"M. Hasan Jamal, Ghulam Mustafa, Abdul Waheed, and Waqar Mahmood. 2009. An extensible infrastructure for benchmarking multi-core processors based systems. In International Symposium on Performance Evaluation of Computer Telecommunication Systems, Vol. 41. 13\u201320."},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/1400112.1400115"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2006.302733"},{"key":"e_1_3_3_29_2","article-title":"The AM-Bench: An Android Multimedia Benchmark Suite","author":"Kim C.","year":"2012","unstructured":"C. Kim, L. Euna, and K. Hyesoon. 2012. The AM-Bench: An Android Multimedia Benchmark Suite. Technical report. Georgia Institute of Technology, Tech. Rep. GIT-CERCS-12-04.","journal-title":"Technical report. Georgia Institute of Technology, Tech. Rep. GIT-CERCS-12-04"},{"key":"e_1_3_3_30_2","first-page":"56","volume-title":"IEEE International Symposium on Workload Characterization (IISWC\u201905)","author":"Lieven Eeckhout","year":"2005","unstructured":"Eeckhout Lieven, Sundareswara Rashmi, J. Yi Joshua, J. Lilja David, and Schrater Paul. 2005. Accurate statistical approaches for generating representative workload compositions. In IEEE International Symposium on Workload Characterization (IISWC\u201905). 56\u201366."},{"key":"e_1_3_3_31_2","unstructured":"Kishonti Ltd.2018. GFXBench for Android. Retrieved from https:\/\/gfxbench.com\/"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/MICRO.2018.00056"},{"key":"e_1_3_3_33_2","first-page":"355","volume-title":"IEEE 22nd International Conference on High Performance Computing (HiPC\u201915)","author":"Mudge Trevor","year":"2015","unstructured":"Trevor Mudge. 2015. The architecture of smart phones. In IEEE 22nd International Conference on High Performance Computing (HiPC\u201915). 355\u2013355."},{"key":"e_1_3_3_34_2","unstructured":"Android NDK. 2016. SimplePerf. Retrieved from https:\/\/android.googlesource.com\/platform\/system\/extras\/+\/master\/simpleperf\/doc\/README.md"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2018.12.010"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCE.2018.2873182"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/PACT.2017.44"},{"key":"e_1_3_3_38_2","first-page":"271","volume-title":"IEEE International Symposium on High Performance Computer Architecture (HPCA\u201918)","author":"Panda Reena","year":"2018","unstructured":"Reena Panda, Shuang Song, Joseph Dean, and Lizy K. John. 2018. Wait of a decade: Did SPEC CPU 2017 broaden the performance horizon? In IEEE International Symposium on High Performance Computer Architecture (HPCA\u201918). IEEE, 271\u2013282."},{"key":"e_1_3_3_39_2","first-page":"193","volume-title":"12th International Conference on Information Communication Technology and System (ICTS\u201919)","author":"Park Je Won","year":"2019","unstructured":"Je Won Park, Chang Min Eun, Hyun Hak Cho, and Ok Hyun Jeong. 2019. A branch predictor design to improve prediction rate by reducing index aliasing in application processors. In 12th International Conference on Information Communication Technology and System (ICTS\u201919). 193\u2013196."},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/1250662.1250713"},{"key":"e_1_3_3_41_2","unstructured":"Ben Popper. 2017. Google Announces over 2 Billion Monthly Active Devices on Android. Retrieved from https:\/\/www.theverge.com\/2017\/5\/17\/15654454\/android-reaches-2-billion-monthly-active-users"},{"issue":"4","key":"e_1_3_3_42_2","first-page":"38","article-title":"A RISC-V simulator and benchmark suite for designing and evaluating vector architectures","volume":"17","author":"Ram\u00edrez Crist\u00f3bal","year":"2020","unstructured":"Crist\u00f3bal Ram\u00edrez, C\u00e9sar Alejandro Hern\u00e1ndez, Oscar Palomar, Osman Unsal, Marco Antonio Ram\u00edrez, and Adri\u00e1n Cristal. 2020. A RISC-V simulator and benchmark suite for designing and evaluating vector architectures. ACM Trans. Archit. Code Optim. 17, 4, Article 38 (Nov.2020), 30 pages.","journal-title":"ACM Trans. Archit. Code Optim."},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/0377-0427(87)90125-7"},{"key":"e_1_3_3_44_2","unstructured":"scikit. 2017. Elbow Method. Retrieved from https:\/\/www.scikit-yb.org\/en\/latest\/api\/cluster\/elbow.html"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.2991\/csece-18.2018.90"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2013.6758177"},{"key":"e_1_3_3_47_2","article-title":"A tutorial on independent component analysis","author":"Shlens Jonathon","year":"2014","unstructured":"Jonathon Shlens. 2014. A tutorial on independent component analysis. arXiv preprint arXiv:1404.2986 (2014).","journal-title":"arXiv preprint arXiv:1404.2986"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00118"},{"key":"e_1_3_3_49_2","unstructured":"DynamoRIO Team. 2011. DynamoRIO: Dynamic Instrumentation Tool Platform. Retrieved from https:\/\/dynamorio.org\/"},{"key":"e_1_3_3_50_2","unstructured":"JUnit Team. 1997. JUnit Testing Framework. Retrieved from https:\/\/junit.org\/junit5\/"},{"key":"e_1_3_3_51_2","unstructured":"Jessica Thornsby. 2019. How to Use Gestures and Gesture-based Navigation in Your Android App. Retrieved from https:\/\/www.androidauthority.com\/android-gesture-tutorial-1032515\/#::text=Android"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3014165"},{"key":"e_1_3_3_53_2","first-page":"5","volume-title":"3rd International Conference on Eco-friendly Computing and Communication Systems","author":"Vikas B.","year":"2014","unstructured":"B. Vikas and Basavaraj Talawar. 2014. On the cache behavior of SPLASH-2 benchmarks on ARM and ALPHA processors in Gem5 full system simulator. In 3rd International Conference on Eco-friendly Computing and Communication Systems. 5\u20138."},{"key":"e_1_3_3_54_2","first-page":"795","volume-title":"IEEE 3rd Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC\u201919)","author":"Wang Dongxu","year":"2019","unstructured":"Dongxu Wang, Jing Hu, Xiwei Zhang, and Jingning Wang. 2019. K value selection method for test data similarity division based on K-Means algorithm. In IEEE 3rd Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC\u201919). 795\u2013798."},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISVLSI.2008.25"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3297663.3310304"},{"key":"e_1_3_3_57_2","unstructured":"Wiki. 2009. Perf: Linux Profiling with Performance Counters. Retrieved from https:\/\/en.wikipedia.org\/wiki\/Perf_(Linux)"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.3390\/app11031225"},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2017.2758781"},{"key":"e_1_3_3_60_2","first-page":"411","volume-title":"IEEE International Symposium on High Performance Computer Architecture (HPCA\u201920)","author":"Zhu Liang","year":"2020","unstructured":"Liang Zhu, Chao Chen, Zihao Su, Weiguang Chen, Tao Li, and Zhibin Yu. 2020. BBS: Micro-architecture benchmarking blockchain systems through machine learning and fuzzy set. In IEEE International Symposium on High Performance Computer Architecture (HPCA\u201920). 411\u2013423."}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3701999","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3701999","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:57:16Z","timestamp":1750298236000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3701999"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,20]]},"references-count":59,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,3,31]]}},"alternative-id":["10.1145\/3701999"],"URL":"https:\/\/doi.org\/10.1145\/3701999","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,20]]},"assertion":[{"value":"2024-03-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-03","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}