{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T02:39:00Z","timestamp":1774579140997,"version":"3.50.1"},"reference-count":90,"publisher":"Association for Computing Machinery (ACM)","issue":"ISSTA","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,22]]},"abstract":"<jats:p>Modern software systems are often highly configurable to tailor varied requirements from diverse stakeholders. Understanding the mapping between configurations and the desired performance attributes plays a fundamental role in advancing the controllability and tuning of the underlying system, yet has long been a dark hole of knowledge due to its black-box nature. While there have been previous efforts in performance analysis for these systems, they analyze the configurations as isolated data points without considering their inherent spatial relationships. This renders them incapable of interrogating many important aspects of the configuration space like local optima. In this work, we advocate a novel perspective to rethink performance analysis\u2014modeling the configuration space as a structured \u201clandscape\u201d. To support this proposition, we utilized GraphFLA, an open-source, graph data mining empowered fitness landscape analysis (FLA) framework. By applying this framework to 86M benchmarked configurations from 32 running workloads of 3 real-world systems, we arrived at 6 main findings, which together constitute a holistic picture of the landscape topography that could have implications on both configuration tuning and performance modeling.<\/jats:p>","DOI":"10.1145\/3728954","type":"journal-article","created":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T10:52:56Z","timestamp":1750589576000},"page":"1748-1771","source":"Crossref","is-referenced-by-count":2,"title":["Rethinking Performance Analysis for Configurable Software Systems: A Case Study from a Fitness Landscape Perspective"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2829-8673","authenticated-orcid":false,"given":"Mingyu","family":"Huang","sequence":"first","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2937-5063","authenticated-orcid":false,"given":"Peili","family":"Mao","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7200-4244","authenticated-orcid":false,"given":"Ke","family":"Li","sequence":"additional","affiliation":[{"name":"University of Exeter, Exeter, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,6,22]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41559-016-0045"},{"key":"e_1_2_1_2_1","volume-title":"Feature-Oriented Software Product Lines - Concepts and Implementation","author":"Apel Sven","unstructured":"Sven Apel, Don S. Batory, Christian K\u00e4stner, and Gunter Saake. 2013. Feature-Oriented Software Product Lines - Concepts and Implementation. Springer."},{"key":"e_1_2_1_3_1","volume-title":"SC\u201913: Proc. of the International Conference for High Performance Computing. ACM, 68:1\u201368:12","author":"Behzad Babak","year":"2013","unstructured":"Babak Behzad, Huong Vu Thanh Luu, Joseph Huchette, Surendra Byna, Prabhat, Ruth A. Aydt, Quincey Koziol, and Marc Snir. 2013. Taming parallel I\/O complexity with auto-tuning. In SC\u201913: Proc. of the International Conference for High Performance Computing. ACM, 68:1\u201368:12."},{"key":"e_1_2_1_4_1","first-page":"2546","article-title":"Algorithms for Hyper-Parameter Optimization. In NIPS\u201911","volume":"24","author":"Bergstra James","year":"2011","unstructured":"James Bergstra, R\u00e9mi Bardenet, Yoshua Bengio, and Bal\u00e1zs K\u00e9gl. 2011. Algorithms for Hyper-Parameter Optimization. In NIPS\u201911: Proc. of the Advances in Neural Information Processing Systems 24. 2546\u20132554.","journal-title":"Proc. of the Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_5_1","volume-title":"Hoos","author":"Biedenkapp Andre","year":"2017","unstructured":"Andre Biedenkapp, Marius Lindauer, Katharina Eggensperger, Frank Hutter, Chris Fawcett, and Holger H. Hoos. 2017. Efficient Parameter Importance Analysis via Ablation with Surrogates. In AAAI\u201917: Proc. of the 31th AAAI Conference on Artificial Intelligence. AAAI Press, 773\u2013779."},{"key":"e_1_2_1_6_1","volume-title":"Workload Characterization: A Survey Revisited. ACM Comput. Surv., 48, 3","author":"Calzarossa Maria Carla","year":"2016","unstructured":"Maria Carla Calzarossa, Luisa Massari, and Daniele Tessera. 2016. Workload Characterization: A Survey Revisited. ACM Comput. Surv., 48, 3 (2016), 48:1\u201348:43."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.953"},{"key":"e_1_2_1_8_1","volume-title":"ESEC\/FSE\u201921: Proc. of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 453\u2013465","author":"Chen Tao","year":"2021","unstructured":"Tao Chen and Miqing Li. 2021. Multi-objectivizing software configuration tuning. In ESEC\/FSE\u201921: Proc. of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 453\u2013465."},{"key":"e_1_2_1_9_1","article-title":"Do Performance Aspirations Matter for Guiding Software Configuration Tuning? An Empirical Investigation under Dual Performance Objectives","volume":"32","author":"Chen Tao","year":"2023","unstructured":"Tao Chen and Miqing Li. 2023. Do Performance Aspirations Matter for Guiding Software Configuration Tuning? An Empirical Investigation under Dual Performance Objectives. ACM Trans. Softw. Eng. Methodol., 32, 3 (2023), 68:1\u201368:41.","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"e_1_2_1_10_1","article-title":"HINNPerf: Hierarchical Interaction Neural Network for Performance Prediction of Configurable Systems","volume":"32","author":"Cheng Jiezhu","year":"2023","unstructured":"Jiezhu Cheng, Cuiyun Gao, and Zibin Zheng. 2023. HINNPerf: Hierarchical Interaction Neural Network for Performance Prediction of Configurable Systems. ACM Trans. Softw. Eng. Methodol., 32, 2 (2023), 46:1\u201346:30.","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"e_1_2_1_11_1","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1038\/nrg3744","article-title":"Empirical fitness landscapes and the predictability of evolution","volume":"15","author":"de Visser J. Arjan G. M.","year":"2014","unstructured":"J. Arjan G. M. de Visser and Joachim Krug. 2014. Empirical fitness landscapes and the predictability of evolution. Nat. Rev. Gen., 15, 7 (2014), 480\u2013490.","journal-title":"Nat. Rev. Gen."},{"key":"e_1_2_1_12_1","volume-title":"ICPADS\u201915: Proc. of the 21st IEEE International Conference on Parallel and Distributed Systems","author":"Ding Xiaoan","unstructured":"Xiaoan Ding, Yi Liu, and Depei Qian. 2015. JellyFish: Online Performance Tuning with Adaptive Configuration and Elastic Container in Hadoop Yarn. In ICPADS\u201915: Proc. of the 21st IEEE International Conference on Parallel and Distributed Systems. IEEE Computer Society, 831\u2013836."},{"key":"e_1_2_1_13_1","volume-title":"Bayesian learning for hardware and software configuration co-optimization","author":"Ding Yi","year":"2020","unstructured":"Yi Ding, Ahsan Pervaiz, Sanjay Krishnan, and Henry Hoffmann. 2020. Bayesian learning for hardware and software configuration co-optimization. University of Chicago, Tech. Rep, 13 (2020)."},{"key":"e_1_2_1_14_1","volume-title":"Mastering Uncertainty in Performance Estimations of Configurable Software Systems. In ASE\u201920: Proc. of the 35th IEEE\/ACM International Conference on Automated Software Engineering. IEEE, 684\u2013696","author":"Dorn Johannes","year":"2020","unstructured":"Johannes Dorn, Sven Apel, and Norbert Siegmund. 2020. Mastering Uncertainty in Performance Estimations of Configurable Software Systems. In ASE\u201920: Proc. of the 35th IEEE\/ACM International Conference on Automated Software Engineering. IEEE, 684\u2013696."},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","first-page":"238701","DOI":"10.1103\/PhysRevLett.88.238701","article-title":"Network topology of a potential energy landscape: A static scale-free network. Phy","volume":"88","author":"Doye Jonathan PK","year":"2002","unstructured":"Jonathan PK Doye. 2002. Network topology of a potential energy landscape: A static scale-free network. Phy. Rev. Lett., 88, 23 (2002), 238701.","journal-title":"Rev. Lett."},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","first-page":"6405","DOI":"10.1038\/s41467-024-50712-3","article-title":"Neural network extrapolation to distant regions of the protein fitness landscape","volume":"15","author":"Freschlin Chase R.","year":"2024","unstructured":"Chase R. Freschlin, Sarah A. Fahlberg, Pete Heinzelman, and Philip A. Romero. 2024. Neural network extrapolation to distant regions of the protein fitness landscape. Nature Communications, 15, 1 (2024), 6405.","journal-title":"Nature Communications"},{"key":"e_1_2_1_17_1","volume-title":"An Empirical Study on Encoding Schemes. In MSR\u201922: Proc. of the 19th IEEE\/ACM International Conference on Mining Software Repositories. ACM, 482\u2013494","author":"Gong Jingzhi","year":"2022","unstructured":"Jingzhi Gong and Tao Chen. 2022. Does Configuration Encoding Matter in Learning Software Performance? An Empirical Study on Encoding Schemes. In MSR\u201922: Proc. of the 19th IEEE\/ACM International Conference on Mining Software Repositories. ACM, 482\u2013494."},{"key":"e_1_2_1_18_1","volume-title":"Deep Configuration Performance Learning: A Systematic Survey and Taxonomy. CoRR, abs\/2403.03322","author":"Gong Jingzhi","year":"2024","unstructured":"Jingzhi Gong and Tao Chen. 2024. Deep Configuration Performance Learning: A Systematic Survey and Taxonomy. CoRR, abs\/2403.03322 (2024)."},{"key":"e_1_2_1_19_1","volume-title":"Proc. ACM Softw. Eng., 1, FSE","author":"Gong Jingzhi","year":"2024","unstructured":"Jingzhi Gong and Tao Chen. 2024. Predicting Configuration Performance in Multiple Environments with Sequential Meta-Learning. Proc. ACM Softw. Eng., 1, FSE (2024), 359\u2013382."},{"key":"e_1_2_1_20_1","unstructured":"L\u00e9o Grinsztajn Edouard Oyallon and Ga\u00ebl Varoquaux. 2022. Why do tree-based models still outperform deep learning on typical tabular data? In NeurIPS\u201922: Advances in Neural Information Processing Systems 35."},{"key":"e_1_2_1_21_1","volume-title":"ASE\u201913: Proc. of the 28th IEEE\/ACM International Conference on Automated Software Engineering. IEEE, 301\u2013311","author":"Guo Jianmei","year":"2013","unstructured":"Jianmei Guo, Krzysztof Czarnecki, Sven Apel, Norbert Siegmund, and Andrzej Wasowski. 2013. Variability-aware performance prediction: A statistical learning approach. In ASE\u201913: Proc. of the 28th IEEE\/ACM International Conference on Automated Software Engineering. IEEE, 301\u2013311."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9573-6"},{"key":"e_1_2_1_23_1","unstructured":"Jichi Guo Qing Yi and Apan Qasem. 2010. Evaluating the role of optimization-specific search heuristics in effective autotuning. Citeseer."},{"key":"e_1_2_1_24_1","volume-title":"ICSE\u201919: Proc. of the 41st International Conference on Software Engineering. IEEE \/ ACM, 1095\u20131106","author":"Ha Huong","year":"2019","unstructured":"Huong Ha and Hongyu Zhang. 2019. DeepPerf: performance prediction for configurable software with deep sparse neural network. In ICSE\u201919: Proc. of the 41st International Conference on Software Engineering. IEEE \/ ACM, 1095\u20131106."},{"key":"e_1_2_1_25_1","volume-title":"ICSME\u201919: Proc. of the IEEE International Conference on Software Maintenance and Evolution","author":"Ha Huong","unstructured":"Huong Ha and Hongyu Zhang. 2019. Performance-Influence Model for Highly Configurable Software with Fourier Learning and Lasso Regression. In ICSME\u201919: Proc. of the IEEE International Conference on Software Maintenance and Evolution. IEEE, 470\u2013480."},{"key":"e_1_2_1_26_1","first-page":"1","article-title":"An Empirical Study on Performance Bugs for Highly Configurable Software Systems. In ESEM\u201916: Proc. of the 10th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"23","author":"Han Xue","year":"2016","unstructured":"Xue Han and Tingting Yu. 2016. An Empirical Study on Performance Bugs for Highly Configurable Software Systems. In ESEM\u201916: Proc. of the 10th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement. ACM, 23:1\u201323:10.","journal-title":"ACM"},{"key":"e_1_2_1_27_1","doi-asserted-by":"crossref","unstructured":"Xue Han Tingting Yu and Michael Pradel. 2021. ConfProf: White-Box Performance Profiling of Configuration Options. In ICPE\u201921: Proc. of the ACM\/SPEC International Conference on Performance Engineering Johann Bourcier Zhen Ming (Jack) Jiang Cor-Paul Bezemer Vittorio Cortellessa Daniele Di Pompeo and Ana Lucia Varbanescu (Eds.). ACM 1\u20138.","DOI":"10.1145\/3427921.3450255"},{"key":"e_1_2_1_28_1","volume-title":"Search-based software engineering: Trends, techniques and applications. ACM Comput. Surv., 45, 1","author":"Harman Mark","year":"2012","unstructured":"Mark Harman, S. Afshin Mansouri, and Yuanyuan Zhang. 2012. Search-based software engineering: Trends, techniques and applications. ACM Comput. Surv., 45, 1 (2012), 11:1\u201311:61."},{"key":"e_1_2_1_29_1","volume-title":"ICDAR\u201995: Proc. of the 3rd International Conference on Document Analysis and Recognition. IEEE Computer Society, 278\u2013282","author":"Ho Tin Kam","year":"1995","unstructured":"Tin Kam Ho. 1995. Random decision forests. In ICDAR\u201995: Proc. of the 3rd International Conference on Document Analysis and Recognition. IEEE Computer Society, 278\u2013282."},{"key":"e_1_2_1_30_1","volume-title":"IJCAI\u201923: Proc. of the Thirty-Second International Joint Conference on Artificial Intelligence. ijcai.org, 5595\u20135603","author":"Huang Mingyu","year":"2023","unstructured":"Mingyu Huang and Ke Li. 2023. Exploring Structural Similarity in Fitness Landscapes via Graph Data Mining: A Case Study on Number Partitioning Problems. In IJCAI\u201923: Proc. of the Thirty-Second International Joint Conference on Artificial Intelligence. ijcai.org, 5595\u20135603."},{"key":"e_1_2_1_31_1","unstructured":"Mingyu Huang and K. Li. 2023. Towards the Inferrence of Structural Similarity of Combinatorial Landscapes. CoRR arXiv:2312.02720."},{"key":"e_1_2_1_32_1","volume-title":"On the Hyperparameter Loss Landscapes of Machine Learning Models: An Exploratory Study. In KDD\u201925: Proc of the 31st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM.","author":"Huang Mingyu","unstructured":"Mingyu Huang and K. Li. 2025. On the Hyperparameter Loss Landscapes of Machine Learning Models: An Exploratory Study. In KDD\u201925: Proc of the 31st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2013.10.003"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155625"},{"key":"e_1_2_1_35_1","doi-asserted-by":"crossref","unstructured":"Pooyan Jamshidi Miguel Velez Christian K\u00e4stner and Norbert Siegmund. 2018. Learning to sample: exploiting similarities across environments to learn performance models for configurable systems. In ESEC\/FSE\u201918. ACM 71\u201382.","DOI":"10.1145\/3236024.3236074"},{"key":"e_1_2_1_36_1","volume-title":"SEAMS\u201917","author":"Jamshidi Pooyan","unstructured":"Pooyan Jamshidi, Miguel Velez, Christian K\u00e4stner, Norbert Siegmund, and Prasad Kawthekar. 2017. Transfer Learning for Improving Model Predictions in Highly Configurable Software. In SEAMS\u201917. IEEE Computer Society, 31\u201341."},{"key":"e_1_2_1_37_1","volume-title":"Proceedings of the 41st International Conference on Software Engineering, ICSE 2019, Montreal, QC","author":"Kaltenecker Christian","year":"2019","unstructured":"Christian Kaltenecker, Alexander Grebhahn, Norbert Siegmund, Jianmei Guo, and Sven Apel. 2019. Distance-based sampling of software configuration spaces. In Proceedings of the 41st International Conference on Software Engineering, ICSE 2019, Montreal, QC, Canada, May 25-31, 2019. IEEE \/ ACM, 1084\u20131094."},{"key":"e_1_2_1_38_1","volume-title":"The Origins of Order: Self-Organization and Selection in Evolution","author":"Kauffman Stuart A.","unstructured":"Stuart A. Kauffman. 1993. The Origins of Order: Self-Organization and Selection in Evolution. Oxford University Press."},{"key":"e_1_2_1_39_1","article-title":"Information Reuse and Stochastic Search: Managing Uncertainty in Self-^ * Systems","volume":"15","author":"Kinneer Cody","year":"2021","unstructured":"Cody Kinneer, David Garlan, and Claire Le Goues. 2021. Information Reuse and Stochastic Search: Managing Uncertainty in Self-^ * Systems. ACM Trans. Auton. Adapt. Syst., 15, 1 (2021), 3:1\u20133:36.","journal-title":"ACM Trans. Auton. Adapt. Syst."},{"key":"e_1_2_1_40_1","volume-title":"ICLR\u201924: Proc. of the 12th International Conference on Learning Representations. OpenReview.net.","author":"Kirjner Andrew","unstructured":"Andrew Kirjner, Jason Yim, Raman Samusevich, Shahar Bracha, Tommi S. Jaakkola, Regina Barzilay, and Ila R. Fiete. 2024. Improving protein optimization with smoothed fitness landscapes. In ICLR\u201924: Proc. of the 12th International Conference on Learning Representations. OpenReview.net."},{"key":"e_1_2_1_41_1","volume-title":"Meta-Surrogate Benchmarking for Hyperparameter Optimization. In NeurIPS\u201919: Proc. of Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems","author":"Klein Aaron","year":"2019","unstructured":"Aaron Klein, Zhenwen Dai, Frank Hutter, Neil D. Lawrence, and Javier Gonz\u00e1lez. 2019. Meta-Surrogate Benchmarking for Hyperparameter Optimization. In NeurIPS\u201919: Proc. of Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019. 6267\u20136277."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-018-0662-9"},{"key":"e_1_2_1_43_1","volume-title":"HPDC\u201914: Proc. of the 23rd International Symposium on High-Performance Parallel and Distributed Computing. ACM, 165\u2013176","author":"Li Min","unstructured":"Min Li, Liangzhao Zeng, Shicong Meng, Jian Tan, Li Zhang, Ali Raza Butt, and Nicholas C. Fuller. 2014. MRONLINE: MapReduce online performance tuning. In HPDC\u201914: Proc. of the 23rd International Symposium on High-Performance Parallel and Distributed Computing. ACM, 165\u2013176."},{"key":"e_1_2_1_44_1","volume-title":"Learning Transferable Features with Deep Adaptation Networks. In ICML\u201915: Proc. of the 32nd International Conference on Machine Learning (JMLR Workshop and Conference Proceedings","volume":"105","author":"Long Mingsheng","unstructured":"Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan. 2015. Learning Transferable Features with Deep Adaptation Networks. In ICML\u201915: Proc. of the 32nd International Conference on Machine Learning (JMLR Workshop and Conference Proceedings, Vol. 37). JMLR.org, 97\u2013105."},{"key":"e_1_2_1_45_1","article-title":"Learning from Very Little Data: On the Value of Landscape Analysis for Predicting Software Project Health","volume":"33","author":"Lustosa Andre","year":"2024","unstructured":"Andre Lustosa and Tim Menzies. 2024. Learning from Very Little Data: On the Value of Landscape Analysis for Predicting Software Project Health. ACM Trans. Softw. Eng. Methodol., 33, 3 (2024), 58:1\u201358:22.","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.3390\/a14020040"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2013.04.015"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2021.3116768"},{"key":"e_1_2_1_49_1","unstructured":"Duncan C. McElfresh Sujay Khandagale Jonathan Valverde Vishak Prasad C. Ganesh Ramakrishnan Micah Goldblum and Colin White. 2023. When Do Neural Nets Outperform Boosted Trees on Tabular Data? In NeurIPS\u201923: Advances in Neural Information Processing Systems 36."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00861"},{"key":"e_1_2_1_51_1","volume-title":"AutoRL Hyperparameter Landscapes. CoRR, abs\/2304.02396","author":"Mohan Aditya","year":"2023","unstructured":"Aditya Mohan, Carolin Benjamins, Konrad Wienecke, Alexander Dockhorn, and Marius Lindauer. 2023. AutoRL Hyperparameter Landscapes. CoRR, abs\/2304.02396 (2023)."},{"key":"e_1_2_1_52_1","volume-title":"ICSE\u201923: Proc. of the IEEE\/ACM 45th International Conference on Software Engineering","author":"M\u00fchlbauer Stefan","year":"2085","unstructured":"Stefan M\u00fchlbauer, Florian Sattler, Christian Kaltenecker, Johannes Dorn, Sven Apel, and Norbert Siegmund. 2023. Analysing the Impact of Workloads on Modeling the Performance of Configurable Software Systems. In ICSE\u201923: Proc. of the IEEE\/ACM 45th International Conference on Software Engineering. IEEE Computer Society, 2085\u20132097."},{"key":"e_1_2_1_53_1","volume-title":"ASPLOS\u201909: Proc. of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems. ACM, 265\u2013276","author":"Mytkowicz Todd","unstructured":"Todd Mytkowicz, Amer Diwan, Matthias Hauswirth, and Peter F. Sweeney. 2009. Producing wrong data without doing anything obviously wrong!. In ASPLOS\u201909: Proc. of the 14th International Conference on Architectural Support for Programming Languages and Operating Systems. ACM, 265\u2013276."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106238"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-017-0225-2"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2870895"},{"key":"e_1_2_1_57_1","volume-title":"ESEC\/FSE\u201917: Proc of the 11th Joint Meeting on Foundations of Software Engineering. ACM, 61\u201371","author":"Oh Jeho","year":"2017","unstructured":"Jeho Oh, Don S. Batory, Margaret Myers, and Norbert Siegmund. 2017. Finding near-optimal configurations in product lines by random sampling. In ESEC\/FSE\u201917: Proc of the 11th Joint Meeting on Foundations of Software Engineering. ACM, 61\u201371."},{"key":"e_1_2_1_58_1","first-page":"E2301","article-title":"Inferring fitness landscapes by regression produces biased estimates of epistasis","volume":"111","author":"Otwinowski Jakub","year":"2014","unstructured":"Jakub Otwinowski and Joshua B. Plotkin. 2014. Inferring fitness landscapes by regression produces biased estimates of epistasis. Proceedings of the National Academy of Sciences, 111, 22 (2014), E2301\u2013E2309.","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"e_1_2_1_59_1","volume-title":"Asymmetric Transitivity Preserving Graph Embedding. In KDD\u201916: Proc of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1105\u20131114","author":"Ou Mingdong","year":"2016","unstructured":"Mingdong Ou, Peng Cui, Jian Pei, Ziwei Zhang, and Wenwu Zhu. 2016. Asymmetric Transitivity Preserving Graph Embedding. In KDD\u201916: Proc of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1105\u20131114."},{"key":"e_1_2_1_60_1","volume-title":"Jose Antonio Escudero, and Andreas Wagner","author":"Papkou Andrei","year":"2023","unstructured":"Andrei Papkou, Lucia Garcia-Pastor, Jose Antonio Escudero, and Andreas Wagner. 2023. A rugged yet easily navigable fitness landscape. Science, 382, 6673 (2023), eadh3860."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2699485"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2011.2163638"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3558774"},{"key":"e_1_2_1_64_1","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1023\/A:1026543900054","article-title":"The Earth Mover\u2019s Distance as a Metric for Image","volume":"40","author":"Rubner Yossi","year":"2000","unstructured":"Yossi Rubner, Carlo Tomasi, and Leonidas J. Guibas. 2000. The Earth Mover\u2019s Distance as a Metric for Image Retrieval. Int. J. Comput. Vis., 40, 2 (2000), 99\u2013121.","journal-title":"Retrieval. Int. J. Comput. Vis."},{"key":"e_1_2_1_65_1","volume-title":"EMNLP\u201917: Proc. of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 372\u2013382","author":"Ruder Sebastian","year":"2017","unstructured":"Sebastian Ruder and Barbara Plank. 2017. Learning to select data for transfer learning with Bayesian Optimization. In EMNLP\u201917: Proc. of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 372\u2013382."},{"key":"e_1_2_1_66_1","volume-title":"ASE\u201915","author":"Sarkar Atri","unstructured":"Atri Sarkar, Jianmei Guo, Norbert Siegmund, Sven Apel, and Krzysztof Czarnecki. 2015. Cost-Efficient Sampling for Performance Prediction of Configurable Systems (T). In ASE\u201915. IEEE Computer Society, 342\u2013352."},{"key":"e_1_2_1_67_1","volume-title":"ESEC\/FSE\u201920: Proc. of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 1039\u20131051","author":"Shahbazian Arman","year":"2020","unstructured":"Arman Shahbazian, Suhrid Karthik, Yuriy Brun, and Nenad Medvidovic. 2020. eQual: informing early design decisions. In ESEC\/FSE\u201920: Proc. of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 1039\u20131051."},{"key":"e_1_2_1_68_1","volume-title":"ESEC\/FSE\u201915: Proc. of the 10th Joint Meeting on Foundations of Software Engineering. ACM, 284\u2013294","author":"Siegmund Norbert","year":"2015","unstructured":"Norbert Siegmund, Alexander Grebhahn, Sven Apel, and Christian K\u00e4stner. 2015. Performance-influence models for highly configurable systems. In ESEC\/FSE\u201915: Proc. of the 10th Joint Meeting on Foundations of Software Engineering. ACM, 284\u2013294."},{"key":"e_1_2_1_69_1","volume-title":"ICSE\u201912: Proc. of the 34th International Conference on Software Engineering. IEEE Computer Society, 167\u2013177","author":"Siegmund Norbert","year":"2012","unstructured":"Norbert Siegmund, Sergiy S. Kolesnikov, Christian K\u00e4stner, Sven Apel, Don S. Batory, Marko Rosenm\u00fcller, and Gunter Saake. 2012. Predicting performance via automated feature-interaction detection. In ICSE\u201912: Proc. of the 34th International Conference on Software Engineering. IEEE Computer Society, 167\u2013177."},{"key":"e_1_2_1_70_1","volume-title":"NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search. CoRR, abs\/2008.09777","author":"Siems Julien","year":"2020","unstructured":"Julien Siems, Lucas Zimmer, Arber Zela, Jovita Lukasik, Margret Keuper, and Frank Hutter. 2020. NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search. CoRR, abs\/2008.09777 (2020)."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2013.2281502"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.4121211"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-022-04506-6"},{"key":"e_1_2_1_74_1","volume-title":"Transferring Performance Prediction Models Across Different Hardware Platforms. In ICPE\u201917: Proc. of the 8th ACM\/SPEC on International Conference on Performance Engineering. ACM, 39\u201350","author":"Valov Pavel","year":"2017","unstructured":"Pavel Valov, Jean-Christophe Petkovich, Jianmei Guo, Sebastian Fischmeister, and Krzysztof Czarnecki. 2017. Transferring Performance Prediction Models Across Different Hardware Platforms. In ICPE\u201917: Proc. of the 8th ACM\/SPEC on International Conference on Performance Engineering. ACM, 39\u201350."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-020-00273-8"},{"key":"e_1_2_1_76_1","doi-asserted-by":"crossref","unstructured":"Miguel Velez Pooyan Jamshidi Norbert Siegmund Sven Apel and Christian K\u00e4stner. 2021. White-Box Analysis over Machine Learning: Modeling Performance of Configurable Systems. In ICSE\u201921. accepted for publication","DOI":"10.1109\/ICSE43902.2021.00100"},{"key":"e_1_2_1_77_1","first-page":"188101","article-title":"Energy Landscape Theory","volume":"90","author":"Wang Jin","year":"2003","unstructured":"Jin Wang and Gennady M. Verkhivker. 2003. Energy Landscape Theory, Funnels, Specificity, and Optimal Criterion of Biomolecular Binding. Phys. Rev. Lett., 90 (2003), May, 188101.","journal-title":"Funnels, Specificity, and Optimal Criterion of Biomolecular Binding. Phys. Rev. Lett."},{"key":"e_1_2_1_78_1","volume-title":"NeurIPS\u201920: Proc. of Advances in Neural Information Processing Systems.","author":"Wang Linnan","year":"2020","unstructured":"Linnan Wang, Rodrigo Fonseca, and Yuandong Tian. 2020. Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search. In NeurIPS\u201920: Proc. of Advances in Neural Information Processing Systems."},{"key":"e_1_2_1_79_1","volume-title":"White-Box Performance-Influence Models: A Profiling and Learning Approach. In ICSE\u201923: Proc. of the 43rd IEEE\/ACM International Conference on Software Engineering. IEEE, 1059\u20131071","author":"Weber Max","year":"2021","unstructured":"Max Weber, Sven Apel, and Norbert Siegmund. 2021. White-Box Performance-Influence Models: A Profiling and Learning Approach. In ICSE\u201923: Proc. of the 43rd IEEE\/ACM International Conference on Software Engineering. IEEE, 1059\u20131071."},{"key":"e_1_2_1_80_1","volume-title":"ICML\u201918: Proc. of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"5081","author":"Wei Ying","year":"2018","unstructured":"Ying Wei, Yu Zhang, Junzhou Huang, and Qiang Yang. 2018. Transfer Learning via Learning to Transfer. In ICML\u201918: Proc. of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80). PMLR, 5072\u20135081."},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00202749"},{"key":"e_1_2_1_82_1","volume-title":"Global optimization algorithms-theory and application. Self-Published Thomas Weise, 361","author":"Weise Thomas","year":"2009","unstructured":"Thomas Weise. 2009. Global optimization algorithms-theory and application. Self-Published Thomas Weise, 361 (2009)."},{"key":"e_1_2_1_83_1","volume-title":"ECML\/PKDD\u201915: Proc. of the 2015 Machine Learning and Knowledge Discovery in Databases - European Conference on Machine Learning. 9285","author":"Wistuba Martin","unstructured":"Martin Wistuba, Nicolas Schilling, and Lars Schmidt-Thieme. 2015. Hyperparameter Search Space Pruning - A New Component for Sequential Model-Based Hyperparameter Optimization. In ECML\/PKDD\u201915: Proc. of the 2015 Machine Learning and Knowledge Discovery in Databases - European Conference on Machine Learning. 9285, Springer, 104\u2013119."},{"key":"e_1_2_1_84_1","volume-title":"Proc. of the 11th International Congress of Genetics. 1, 356\u2013366","author":"Wright Sewall","year":"1932","unstructured":"Sewall Wright. 1932. The roles of mutations, inbreeding, crossbreeding and selection in evolution. In Proc. of the 11th International Congress of Genetics. 1, 356\u2013366."},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1145\/988672.988711"},{"key":"e_1_2_1_86_1","volume-title":"ESEC\/FSE\u201915: Proc. of the 10th Joint Meeting on Foundations of Software Engineering. ACM, 307\u2013319","author":"Xu Tianyin","year":"2015","unstructured":"Tianyin Xu, Long Jin, Xuepeng Fan, Yuanyuan Zhou, Shankar Pasupathy, and Rukma Talwadker. 2015. Hey, you have given me too many knobs!: understanding and dealing with over-designed configuration in system software. In ESEC\/FSE\u201915: Proc. of the 10th Joint Meeting on Foundations of Software Engineering. ACM, 307\u2013319."},{"key":"e_1_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.1145\/781027.781052"},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2013.53"},{"key":"e_1_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2015.15"},{"key":"e_1_2_1_90_1","volume-title":"Proc. ACM Softw. Eng., 1, FSE","author":"Zhou Shasha","year":"2024","unstructured":"Shasha Zhou, Mingyu Huang, Yanan Sun, and Ke Li. 2024. Evolutionary Multi-objective Optimization for Contextual Adversarial Example Generation. Proc. ACM Softw. Eng., 1, FSE (2024), 2285\u20132308."}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3728954","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,16]],"date-time":"2025-07-16T16:47:46Z","timestamp":1752684466000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3728954"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,22]]},"references-count":90,"journal-issue":{"issue":"ISSTA","published-print":{"date-parts":[[2025,6,22]]}},"alternative-id":["10.1145\/3728954"],"URL":"https:\/\/doi.org\/10.1145\/3728954","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,22]]}}}