{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,4,4]],"date-time":"2024-04-04T00:42:59Z","timestamp":1712191379425},"reference-count":131,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"name":"NSF","award":["CCF1212683 and ACI-1550493"]},{"DOI":"10.13039\/501100005367","name":"Universidad Nacional de Educacion a Distancia","doi-asserted-by":"crossref","award":["2021V\/PUNED\/008 and 2022V\/PUNED\/007"]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,1,31]]},"abstract":"\n A\n Software Product Line<\/jats:bold>\n (\n SPL<\/jats:monospace>\n ) is a family of similar programs. Each program is defined by a unique set of features, called a\n \n configuration<\/jats:italic>\n <\/jats:bold>\n , that satisfies all feature constraints. \u201cWhat configuration achieves the best performance for a given workload?\u201d is the\n \n SPL<\/jats:bold>\n <\/jats:monospace>\n Optimization<\/jats:bold>\n (\n SPLO<\/jats:monospace>\n ) challenge.\n SPLO<\/jats:monospace>\n is daunting: just 80 unconstrained features yield 10\n 24<\/jats:sup>\n unique configurations, which equals the estimated number of stars in the universe. We explain (a) how uniform random sampling and random search algorithms solve\n SPLO<\/jats:monospace>\n more efficiently and accurately than current machine-learned performance models and (b) how to compute statistical guarantees on the quality of a returned configuration; i.e., it is within\n x%<\/jats:italic>\n of optimal with\n y%<\/jats:italic>\n confidence.\n <\/jats:p>","DOI":"10.1145\/3611663","type":"journal-article","created":{"date-parts":[[2023,8,7]],"date-time":"2023-08-07T11:34:16Z","timestamp":1691408056000},"page":"1-36","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Finding Near-optimal Configurations in Colossal Spaces with Statistical Guarantees"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-5599-268X","authenticated-orcid":false,"given":"Jeho","family":"Oh","sequence":"first","affiliation":[{"name":"The University of Texas at Austin, United States"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-8739-3841","authenticated-orcid":false,"given":"Don","family":"Batory","sequence":"additional","affiliation":[{"name":"The University of Texas at Austin, United States"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-7131-0482","authenticated-orcid":false,"given":"Rub\u00e9n","family":"Heradio","sequence":"additional","affiliation":[{"name":"Universidad Nacional de Educaci\u00f3n a Distancia, Spain"}]}],"member":"320","published-online":{"date-parts":[[2023,11,23]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"crossref","unstructured":"2021. Learning software configuration spaces: A systematic literature review. JSS 182 (2021).","DOI":"10.1016\/j.jss.2021.111044"},{"key":"e_1_3_3_3_2","unstructured":"7z. 2021. 7z Website. https:\/\/www.7-zip.org\/download.html"},{"key":"e_1_3_3_4_2","volume-title":"Learning Very Large Configuration Spaces: What Matters for Linux Kernel Sizes","author":"Acher M.","year":"2019","unstructured":"M. Acher et al. 2019. Learning Very Large Configuration Spaces: What Matters for Linux Kernel Sizes. Technical Report hal-02314830. Inria Rennes."},{"key":"e_1_3_3_5_2","volume-title":"SPLC","author":"Acher M.","year":"2022","unstructured":"M. Acher et al. 2022. Feature subset selection for learning huge configuration spaces: The case of Linux kernel size. In SPLC."},{"key":"e_1_3_3_6_2","volume-title":"SAT","author":"Achlioptas D.","year":"2018","unstructured":"D. Achlioptas, Z. S. Hammoudeh, and P. Theodoropoulos. 2018. Fast sampling of perfectly uniform satisfying assignments. In SAT."},{"key":"e_1_3_3_7_2","volume-title":"VLDB","author":"Agrawal S.","year":"2000","unstructured":"S. Agrawal, S. Chaudhuri, and V. Narasayya. 2000. Automated selection of materialized views and indexes in SQL databases. In VLDB."},{"key":"e_1_3_3_8_2","volume-title":"SIGMOD","author":"Aken D. Van","year":"2017","unstructured":"D. Van Aken, A. Pavlo, G. J. Gordon, and B. Zhang. 2017. Automatic database management system tuning through large-scale machine learning. In SIGMOD."},{"key":"e_1_3_3_9_2","unstructured":"Apache Web Server. 2002. Apache HTTP Server Project. https:\/\/httpd.apache.org\/"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-37521-7"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898719062"},{"key":"e_1_3_3_12_2","unstructured":"axTLS. 2018. AxTLS Website. http:\/\/axtls.sourceforge.net\/"},{"key":"e_1_3_3_13_2","volume-title":"SAT","author":"Aziz R. A.","year":"2015","unstructured":"R. A. Aziz, G. Chu, C. J. Muise, and P. J. Stuckey. 2015. # \\(\\exists\\) SAT: Projected model counting. In SAT."},{"key":"e_1_3_3_14_2","volume-title":"SPLC","author":"Batory D.","year":"2005","unstructured":"D. Batory. 2005. Feature models, grammars, and propositional formulas. In SPLC."},{"key":"e_1_3_3_15_2","volume-title":"Automated Software Design: Volume 1","author":"Batory D.","year":"2021","unstructured":"D. Batory. 2021. Automated Software Design: Volume 1. Lulu Press."},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/319758.319760"},{"key":"e_1_3_3_17_2","volume-title":"Logic, Computation and Rigorous Methods","author":"Batory D.","year":"2021","unstructured":"D. Batory, J. Oh, R. Heradio, and D. Benavides. 2021. Product optimization in stepwise design. In Logic, Computation and Rigorous Methods, A. Raschke, E. Riccobene, and K. D. Schewe (Eds.). Springer."},{"key":"e_1_3_3_18_2","unstructured":"BDDSampler. 2022. BDDSampler Website. https:\/\/github.com\/davidfa71\/BDDSampler"},{"key":"e_1_3_3_19_2","article-title":"Random search for hyper-parameter optimization","author":"Bergstra J.","year":"2012","unstructured":"J. Bergstra and Y. Bengio. 2012. Random search for hyper-parameter optimization. JMLR (2012).","journal-title":"JMLR"},{"key":"e_1_3_3_20_2","unstructured":"BerkeleyDB Website. 2021. BerkeleyDB Website. https:\/\/www.oracle.com\/database\/technologies\/related\/berkeleydb.html"},{"key":"e_1_3_3_21_2","volume-title":"Handbook of Satisfiability: Volume 185: Frontiers in Artificial Intelligence and Applications","author":"Biere A.","year":"2009","unstructured":"A. Biere, M. Heule, H. Maaren, and T. Walsh. 2009. Handbook of Satisfiability: Volume 185: Frontiers in Artificial Intelligence and Applications. IOS Press."},{"key":"e_1_3_3_22_2","volume-title":"Elementary Statistics: A Step by Step Approach","author":"Bluman A.","year":"2009","unstructured":"A. Bluman. 2009. Elementary Statistics: A Step by Step Approach. McGraw-Hill Higher Education."},{"key":"e_1_3_3_23_2","volume-title":"Classification and Regression Trees","author":"Breiman L.","year":"1984","unstructured":"L. Breiman, J. Friedman, R. Olshen, and C. Stone. 1984. Classification and Regression Trees. Wadsworth and Brooks\/Cole Advanced Books and Software."},{"key":"e_1_3_3_24_2","volume-title":"Practical Statistics","author":"Bruce P.","year":"2020","unstructured":"P. Bruce, A. Bruce, and P. Gedeck. 2020. Practical Statistics. O\u2019Reilly."},{"key":"e_1_3_3_25_2","volume-title":"TACAS","author":"Bryant R.","year":"2007","unstructured":"R. Bryant et al. 2007. Deciding bit-vector arithmetic with abstraction. In TACAS."},{"key":"e_1_3_3_26_2","volume-title":"TACAS","author":"Chakraborty S.","year":"2015","unstructured":"S. Chakraborty, D. J. Fremont, K. S. Meel, S. A. Seshia, and M. Y. Vardi. 2015. On parallel scalable uniform SAT witness generation. In TACAS."},{"key":"e_1_3_3_27_2","volume-title":"CAV","author":"Chakraborty S.","year":"2013","unstructured":"S. Chakraborty, K. S. Meel, and M. Y. Vardi. 2013. A scalable and nearly uniform generator of SAT witnesses. In CAV."},{"key":"e_1_3_3_28_2","volume-title":"CP","author":"Chakraborty S.","year":"2013","unstructured":"S. Chakraborty, K. Meel, and M. Vardi. 2013. A scalable approximate model counter. In CP."},{"key":"e_1_3_3_29_2","volume-title":"DAC","author":"Chakraborty S.","year":"2014","unstructured":"S. Chakraborty, K. S. Meel, and M. Y. Vardi. 2014. Balancing scalability and uniformity in SAT witness generator. In DAC."},{"key":"e_1_3_3_30_2","volume-title":"PODS","author":"Chaudhuri S.","year":"1998","unstructured":"S. Chaudhuri. 1998. An overview of query optimization in relational systems. In PODS."},{"key":"e_1_3_3_31_2","volume-title":"ASIAN","author":"Chen T. Y.","year":"2004","unstructured":"T. Y. Chen, H. Leung, and I. K. Mak. 2004. Adaptive random testing. In ASIAN."},{"key":"e_1_3_3_32_2","unstructured":"CUDD Website. 2022. CUDD Website. https:\/\/github.com\/vscosta\/cudd"},{"key":"e_1_3_3_33_2","article-title":"A fast and elitist multiobjective genetic algorithm: NSGA-II","author":"Deb K.","year":"2002","unstructured":"K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE TEVC (April 2002).","journal-title":"IEEE TEVC"},{"key":"e_1_3_3_34_2","volume-title":"ASE","author":"Dorn J.","year":"2020","unstructured":"J. Dorn, S. Apel, and N. Siegmund. 2020. Mastering uncertainty in performance estimations of configurable software systems. In ASE."},{"key":"e_1_3_3_35_2","volume-title":"ICSE","author":"Dutra R.","year":"2018","unstructured":"R. Dutra, K. Laeufer, J. Bachrach, and K. Sen. 2018. Efficient sampling of SAT solutions for testing. In ICSE."},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.3390\/math8081253"},{"key":"e_1_3_3_37_2","volume-title":"SPLC","author":"Fernandez-Amoros D.","year":"2019","unstructured":"D. Fernandez-Amoros, R. Heradio, C. Mayr-Dorn, and A. Egyed. 2019. A Kconfig translation to logic with one-way validation system. In SPLC."},{"key":"e_1_3_3_38_2","volume-title":"ASE","author":"Fernandez-Amoros D.","year":"2022","unstructured":"D. Fernandez-Amoros, R. Heradio, C. Mayr-Dorn, and A. Egyed. 2022. Scalable sampling of highly-configurable systems: Generating random instances of the Linux kernel. In ASE."},{"key":"e_1_3_3_39_2","volume-title":"Discovering Statistics Using R","author":"Field A.","year":"2012","unstructured":"A. Field, J. Miles, and Zo\u00eb Field. 2012. Discovering Statistics Using R. SAGE Publications Ltd."},{"key":"e_1_3_3_40_2","volume-title":"Uniform Sampling from Kconfig Feature Models","author":"Gazillo P.","year":"2019","unstructured":"P. Gazillo, J. Oh, and M. Myers. 2019. Uniform Sampling from Kconfig Feature Models. Technical Report TR-19-02. University of Texas at Austin, Department of Computer Science."},{"key":"e_1_3_3_41_2","volume-title":"CP","author":"Gogate V.","year":"2006","unstructured":"V. Gogate and R. Dechter. 2006. A new algorithm for sampling CSP solutions uniformly at random. In CP."},{"key":"e_1_3_3_42_2","first-page":"1","article-title":"SMTIBEA: A hybrid multi-objective optimization algorithm for configuring large constrained software product lines","author":"Guo J.","year":"2017","unstructured":"J. Guo et al. 2017. SMTIBEA: A hybrid multi-objective optimization algorithm for configuring large constrained software product lines. SoSyM (2017), 1\u201320.","journal-title":"SoSyM"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9573-6"},{"key":"e_1_3_3_44_2","volume-title":"ASE","author":"Guo J.","year":"2013","unstructured":"J. Guo, K. Czarnecki, S. Apel, N. Siegmund, and A. Wasowski. 2013. Variability-aware performance prediction: A statistical learning approach. In ASE."},{"key":"e_1_3_3_45_2","volume-title":"ICSE","author":"Guo J.","year":"2018","unstructured":"J. Guo and K. Shi. 2018. To preserve or not to preserve invalid solutions in search-bassed software engineering: A case study in software produce lines. In ICSE."},{"key":"e_1_3_3_46_2","unstructured":"H.264 Video Compression. 2022. H.264 Video Compression. https:\/\/www.haivision.com\/resources\/streaming-video-definitions\/h-264\/"},{"key":"e_1_3_3_47_2","volume-title":"ICSE","author":"Ha H.","year":"2019","unstructured":"H. Ha and H. Zhang. 2019. Deepperf: Performance prediction for configurable software with deep sparse neural network. In ICSE."},{"key":"e_1_3_3_48_2","volume-title":"ICSME","author":"Ha H.","year":"2019","unstructured":"H. Ha and H. Zhang. 2019. Performance-influence model for highly configurable software with Fourier learning and lasso regression. In ICSME."},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-018-9635-4"},{"key":"e_1_3_3_50_2","volume-title":"ICSE","author":"Henard C.","year":"2015","unstructured":"C. Henard, M. Papadakis, M. Harman, and Y. Traon. 2015. Combining multi-objective search and constraint solving for configuring large software product lines. In ICSE."},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-021-10102-5"},{"key":"e_1_3_3_52_2","volume-title":"SPLC","author":"Heradio R.","year":"2020","unstructured":"R. Heradio, D. Fernandez-Amoros, J. A. Galindo, and D. Benavides. 2020. Uniform and scalable SAT-sampling for configurable systems. In SPLC."},{"key":"e_1_3_3_53_2","volume-title":"HVC","author":"Heule M. J.","year":"2011","unstructured":"M. J. Heule, O. Kullmann, S. Wieringa, and A. Biere. 2011. Cube and conquer: Guiding CDCL SAT solvers by lookaheads. In HVC."},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177730150"},{"key":"e_1_3_3_55_2","volume-title":"Automated Algorithm Configuration and Parameter Tuning","author":"Hoos H. H.","year":"2012","unstructured":"H. H. Hoos. 2012. Automated Algorithm Configuration and Parameter Tuning."},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2017.10.018"},{"key":"e_1_3_3_57_2","volume-title":"LION","author":"Hutter F.","year":"2011","unstructured":"F. Hutter, H. H. Hoos, and K. Leyton-Brown. 2011. Sequential model-based optimization for general algorithm configuration. In LION."},{"key":"e_1_3_3_58_2","volume-title":"EuroSys","author":"Iqbal M. S.","year":"2022","unstructured":"M. S. Iqbal, R. Krishna, M. A. Javidian, B. Ray, and P. Jamshidi. 2022. Unicorn: Reasoning about configurable system performance through the lens of causality. In EuroSys."},{"key":"e_1_3_3_59_2","volume-title":"SAT","author":"Jackson P.","year":"2004","unstructured":"P. Jackson and D. Sheridan. 2004. Clause form conversions for Boolean circuits. In SAT."},{"key":"e_1_3_3_60_2","volume-title":"ASE","author":"Jamshidi P.","year":"2017","unstructured":"P. Jamshidi et al. 2017. Transfer learning for performance modeling of configurable systems: An exploratory analysis. In ASE."},{"key":"e_1_3_3_61_2","volume-title":"MASCOTS","author":"Jamshidi P.","year":"2016","unstructured":"P. Jamshidi and G. Casale. 2016. An uncertainty-aware approach to optimal configuration of stream processing systems. In MASCOTS."},{"key":"e_1_3_3_62_2","volume-title":"TACAS","author":"J\u00e4rvisalo M.","year":"2010","unstructured":"M. J\u00e4rvisalo, A. Biere, and M. J. H. Heule. 2010. Blocked clause elimination. In TACAS."},{"key":"e_1_3_3_63_2","volume-title":"ICSE","author":"Kaltenecker C.","year":"2019","unstructured":"C. Kaltenecker, A. Grebhahn, N. Siegmund, J. Guo, and S. Apel. 2019. Distance-based sampling of software configuration spaces. In ICSE."},{"issue":"5","key":"e_1_3_3_64_2","article-title":"The map method for synthesis of combinational logic circuits","volume":"72","author":"Karnaugh M.","year":"1953","unstructured":"M. Karnaugh. 1953. The map method for synthesis of combinational logic circuits. IEEE Commun. Electron. 72, 5 (1953).","journal-title":"IEEE Commun. Electron."},{"key":"e_1_3_3_65_2","unstructured":"Kconfig Language. 2018. Kconfig Language Specification. https:\/\/www.kernel.org\/doc\/Documentation\/kbuild\/kconfig-language.txt"},{"key":"e_1_3_3_66_2","unstructured":"Kconfig Tool. 2018. Kconfig Tool Specification. https:\/\/www.kernel.org\/doc\/Documentation\/kbuild\/kconfig.txt"},{"key":"e_1_3_3_67_2","volume-title":"The Art of Computer Programming, Volume 4, Fascicle 1: Bitwise Tricks & Techniques, Binary Decision Diagrams","author":"Knuth D. E.","year":"2009","unstructured":"D. E. Knuth. 2009. The Art of Computer Programming, Volume 4, Fascicle 1: Bitwise Tricks & Techniques, Binary Decision Diagrams. Addison-Wesley Professional."},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-018-0662-9"},{"key":"e_1_3_3_69_2","article-title":"Gale: Geometric active learning for search-based software engineering","author":"Krall J.","year":"2015","unstructured":"J. Krall, T. Menzies, and M. Davies. 2015. Gale: Geometric active learning for search-based software engineering. IEEE TSE (Oct. 2015).","journal-title":"IEEE TSE"},{"key":"e_1_3_3_70_2","volume-title":"SPLC","author":"Krieter S.","year":"2019","unstructured":"S. Krieter. 2019. Enabling efficient automated configuration generation and management. In SPLC."},{"key":"e_1_3_3_71_2","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1952.10483441"},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0166-218X(99)00037-2"},{"issue":"1","key":"e_1_3_3_73_2","first-page":"1","article-title":"Set the configuration for the heart of the os: On the practicality of operating system kernel debloating","volume":"4","author":"Kuo H.","year":"2020","unstructured":"H. Kuo, J. Chen, S. Mohan, and T. Xu. 2020. Set the configuration for the heart of the os: On the practicality of operating system kernel debloating. POMACS 4, 1 (2020), 1\u201327.","journal-title":"POMACS"},{"key":"e_1_3_3_74_2","volume-title":"Machine Learning with R","author":"Lantz B.","year":"2019","unstructured":"B. Lantz. 2019. Machine Learning with R. Packt Publishing."},{"key":"e_1_3_3_75_2","volume-title":"Robust Tests for Equality of Variances","author":"Levene H.","year":"1960","unstructured":"H. Levene. 1960. Robust Tests for Equality of Variances. Stanford University Press."},{"key":"e_1_3_3_76_2","unstructured":"LLVM. 2020. The LLVM Compiler Infrastructure. https:\/\/llvm.org\/"},{"key":"e_1_3_3_77_2","volume-title":"CAV","author":"M. Soos S. Gocht,","year":"2020","unstructured":"S. Gocht, M. Soos and K. S. Meel. 2020. Tinted, detached, and lazy CNF-XOR solving and its applications to counting and sampling. In CAV."},{"key":"e_1_3_3_78_2","unstructured":"Mann Whitney. 2020. Mann-Whitney U-Test (Wilcoxon Rank Sum Test). https:\/\/sphweb.bumc.bu.edu\/otlt\/mph-modules\/bs\/bs704_nonparametric\/BS704_Nonparametric4.html"},{"key":"e_1_3_3_79_2","volume-title":"ASE","author":"Marker B.","year":"2014","unstructured":"B. Marker, D. Batory, and R. Geijn. 2014. Understanding performance stairs: Elucidating heuristics. In ASE."},{"key":"e_1_3_3_80_2","article-title":"Transfer learning across variants and versions: The case of Linux kernel size","author":"Martin H.","year":"2021","unstructured":"H. Martin et al. 2021. Transfer learning across variants and versions: The case of Linux kernel size. IEEE TSE (Sept. 2021).","journal-title":"IEEE TSE"},{"key":"e_1_3_3_81_2","volume-title":"SPLC","author":"Martin H.","year":"2021","unstructured":"H. Martin, M. Acher, J. A. Pereira, and J. J\u00e9z\u00e9quel. 2021. A comparison of performance specialization learning for configurable systems. In SPLC."},{"key":"e_1_3_3_82_2","unstructured":"MathIsFun.com. 2019. Standard Deviation Formulas. https:\/\/www.mathsisfun.com\/data\/standard-deviation-formulas.html"},{"key":"e_1_3_3_83_2","volume-title":"ICSE","author":"Medeiros F.","year":"2016","unstructured":"F. Medeiros, C. K\u00e4stner, M. Ribeiro, R. Gheyi, and S. Apel. 2016. A comparison of 10 sampling algorithms for configurable systems. In ICSE."},{"key":"e_1_3_3_84_2","volume-title":"SPLC","author":"Munoz D.","year":"2019","unstructured":"D. Munoz, J. Oh, M. Pinto, L. Fuentes, and D. Batory. 2019. Uniform random sampling product configurations of feature models that have numerical features. In SPLC."},{"key":"e_1_3_3_85_2","volume-title":"ICSR","author":"Munoz D.","year":"2022","unstructured":"D. Munoz, J. Oh, M. Pinto, L. Fuentes, and D. Batory. 2022. Nemo: A tool to transform feature models with numerical features and arithmetic constraints. In ICSR."},{"key":"e_1_3_3_86_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00607-018-0632-7"},{"key":"e_1_3_3_87_2","doi-asserted-by":"publisher","DOI":"10.5555\/539877"},{"key":"e_1_3_3_88_2","volume-title":"FSE","author":"Nair V.","year":"2017","unstructured":"V. Nair, T. Menzies, N. Siegmund, and S. Apel. 2017. Using bad learners to find good configurations. In FSE."},{"key":"e_1_3_3_89_2","article-title":"Finding faster configurations using FLASH","author":"Nair V.","year":"2020","unstructured":"V. Nair, Z. Yu, T. Menzies, N. Siegmund, and S. Apel. 2020. Finding faster configurations using FLASH. IEEE TSE (July 2020).","journal-title":"IEEE TSE"},{"key":"e_1_3_3_90_2","volume-title":"ASE","author":"N\u00f6hrer A.","year":"2013","unstructured":"A. N\u00f6hrer and A. Egyed. 2013. C2O configurator: A tool for guided decision-making. In ASE."},{"key":"e_1_3_3_91_2","unstructured":"J. Oh. 2022. Finding Near-optimal Configurations in Colossal Product Spaces of Highly Configurable Systems. Ph.D. Dissertation. University of Texas at Austin Dept. of Computer Science."},{"key":"e_1_3_3_92_2","volume-title":"FSE","author":"Oh J.","year":"2017","unstructured":"J. Oh, D. Batory, M. Myers, and N. Siegmund. 2017. Finding near-optimal configurations in product lines by random sampling. In FSE."},{"key":"e_1_3_3_93_2","volume-title":"Scalable Uniform Sampling for Real-world Software Product Lines","author":"Oh J.","year":"2020","unstructured":"J. Oh, P. Gazzillo, D. Batory, M. Heule, and M. Myers. 2020. Scalable Uniform Sampling for Real-world Software Product Lines. Technical Report TR-20-01. Dept. of Computer Science, University of Texas at Austin."},{"key":"e_1_3_3_94_2","volume-title":"ICSE","author":"Olaechea R.","year":"2014","unstructured":"R. Olaechea, D. Rayside, J. Guo, and K. Czarnecki. 2014. Comparison of exact and approximate multi-objective optimization for software product lines. In ICSE."},{"key":"e_1_3_3_95_2","volume-title":"ICPE","author":"Pereira J.","year":"2020","unstructured":"J. Pereira, M. Acher, H. Martin, and J. M. J\u00e9z\u00e9quel. 2020. Sampling effect on performance prediction of configurable systems: A case study. In ICPE."},{"key":"e_1_3_3_96_2","volume-title":"Programming Machine Learning: From Coding to Deep Learning","author":"Perrotta P.","year":"2020","unstructured":"P. Perrotta. 2020. Programming Machine Learning: From Coding to Deep Learning. The Pragmatic Bookshelf."},{"key":"e_1_3_3_97_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0747-7171(86)80028-1"},{"key":"e_1_3_3_98_2","volume-title":"ICST","author":"Plazar Q.","year":"2019","unstructured":"Q. Plazar, M. Acher, G. Perrouin, X. Devroey, and M. Cordy. 2019. Uniform sampling of SAT solutions for configurable systems: Are we there yet? In ICST."},{"key":"e_1_3_3_99_2","unstructured":"T. Puiu. 2021. What Does 5-sigma Mean in Science?https:\/\/www.zmescience.com\/science\/what-5-sigma-means-0423423\/"},{"issue":"2","key":"e_1_3_3_100_2","first-page":"115","article-title":"An extension of Shapiro and Wilk\u2019s w test for normality to large samples","volume":"31","author":"Royston J. P.","year":"1982","unstructured":"J. P. Royston. 1982. An extension of Shapiro and Wilk\u2019s w test for normality to large samples. J. R. Stat. Soc. 31, 2 (1982), 115\u2013124.","journal-title":"J. R. Stat. Soc."},{"key":"e_1_3_3_101_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2018.12.002"},{"key":"e_1_3_3_102_2","volume-title":"ASE","author":"Sarkar A.","year":"2015","unstructured":"A. Sarkar, J. Guo, N. Siegmund, S. Apel, and K. Czarnecki. 2015. Cost-efficient sampling for performance prediction of configurable systems. In ASE."},{"key":"e_1_3_3_103_2","volume-title":"LPAR","author":"Sharma S.","year":"2018","unstructured":"S. Sharma, R. Gupta, S. Roy, and K. S. Meel. 2018. Knowledge compilation meets uniform sampling. In LPAR."},{"key":"e_1_3_3_104_2","volume-title":"ICSME","author":"Shi K.","year":"2017","unstructured":"K. Shi. 2017. Combining evolutionary algorithms with constraint solving for configuration optimization. In ICSME."},{"key":"e_1_3_3_105_2","volume-title":"Nonparametric Statistics for the Behavioral Sciences","author":"Siegel S.","year":"1988","unstructured":"S. Siegel and J. Castellan. 1988. Nonparametric Statistics for the Behavioral Sciences. McGraw Hill."},{"key":"e_1_3_3_106_2","unstructured":"N. Siegmund et al. 2012. Dataset for Siegmund2012. http:\/\/fosd.de\/SPLConqueror"},{"key":"e_1_3_3_107_2","volume-title":"ICSE","author":"Siegmund N.","year":"2012","unstructured":"N. Siegmund et al. 2012. Predicting performance via automated feature-interaction detection. In ICSE."},{"key":"e_1_3_3_108_2","volume-title":"FSE","author":"Siegmund N.","year":"2015","unstructured":"N. Siegmund, A. Grebhahn, S. Apel, and C. K\u00e4stner. 2015. Performance-influence models for highly configurable systems. In FSE."},{"key":"e_1_3_3_109_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-011-9152-9"},{"key":"e_1_3_3_110_2","volume-title":"FSE","author":"Siegmund N.","year":"2017","unstructured":"N. Siegmund, S. Sobernig, and S. Apel. 2017. Attributed variability models: Outside the comfort zone. In FSE."},{"key":"e_1_3_3_111_2","doi-asserted-by":"publisher","DOI":"10.1214\/009053607000000505"},{"key":"e_1_3_3_112_2","volume-title":"SAT","author":"Thurley M.","year":"2006","unstructured":"M. Thurley. 2006. SharpSAT\u2013counting models with advanced component caching and implicit BCP. In SAT."},{"key":"e_1_3_3_113_2","unstructured":"Toybox. 2018. Toybox Website. http:\/\/landley.net\/toybox\/"},{"key":"e_1_3_3_114_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-81955-1_28"},{"key":"e_1_3_3_115_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-16313-5"},{"key":"e_1_3_3_116_2","volume-title":"ICSE","author":"Velez M.","year":"2021","unstructured":"M. Velez, P. Jamshidi, N. Siegmund, S. Apel, and C. K\u00e4stner. 2021. White-box analysis over machine learning: Modeling performance of configurable systems. In ICSE."},{"key":"e_1_3_3_117_2","unstructured":"VP9 Website. 2021. VP9 Website. https:\/\/www.webmproject.org\/vp9\/"},{"key":"e_1_3_3_118_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2009.02.011"},{"key":"e_1_3_3_119_2","volume-title":"SPLC","author":"White J.","year":"2008","unstructured":"J. White, B. Doughtery, and D. Schmidt. 2008. Filtered Cartesian flattening: An approximation technique for optimally selecting features while adhering to resource constraints. In SPLC."},{"key":"e_1_3_3_120_2","doi-asserted-by":"crossref","unstructured":"Wikipedia. 2016. 68-95-99.7 Rule. https:\/\/en.wikipedia.org\/wiki\/68%E2%80%9395%E2%80%9399.7_rule","DOI":"10.1136\/jclinpath-2014-202767corr1"},{"key":"e_1_3_3_121_2","unstructured":"Wikipedia. 2018. Conjunctive Normal Form. https:\/\/en.wikipedia.org\/wiki\/Conjunctive_normal_form"},{"key":"e_1_3_3_122_2","unstructured":"Wikipedia. 2021. Box Plot. https:\/\/en.wikipedia.org\/wiki\/Box_plot"},{"key":"e_1_3_3_123_2","unstructured":"Wikipedia. 2021. Integer Programming. https:\/\/en.wikipedia.org\/wiki\/Integer_programming"},{"key":"e_1_3_3_124_2","unstructured":"Wikipedia. 2021. Random Search. https:\/\/en.wikipedia.org\/wiki\/Random_search"},{"key":"e_1_3_3_125_2","unstructured":"Wikipedia. 2022. Chain Rule. https:\/\/en.wikipedia.org\/wiki\/Chain_rule"},{"key":"e_1_3_3_126_2","unstructured":"Wikipedia. 2022. Equisatisfiability. https:\/\/en.wikipedia.org\/wiki\/Equisatisfiability"},{"key":"e_1_3_3_127_2","unstructured":"Wikipedia. 2022. Order Statistics. https:\/\/en.wikipedia.org\/wiki\/Order_statistic"},{"key":"e_1_3_3_128_2","volume-title":"ICSE","author":"Xue Y.","year":"2018","unstructured":"Y. Xue and Y. Li. 2018. Multi-objective integer programming approaches for solving optimal feature selection problem. In ICSE."},{"key":"e_1_3_3_129_2","article-title":"A recursive random search algorithm for large-scale network parameter configuration","author":"Ye T.","year":"2003","unstructured":"T. Ye and S. Kalyanaraman. 2003. A recursive random search algorithm for large-scale network parameter configuration. PER (June 2003).","journal-title":"PER"},{"key":"e_1_3_3_130_2","volume-title":"ASE","author":"Zhang Y.","year":"2015","unstructured":"Y. Zhang, J. Guo, E. Blais, and K. Czarnecki. 2015. Performance prediction of configurable software systems by Fourier learning. In ASE."},{"key":"e_1_3_3_131_2","volume-title":"SoCC","author":"Zhu Y.","year":"2017","unstructured":"Y. Zhu et al. 2017. BestConfig: Tapping the performance potential of systems via automatic configuration tuning. In SoCC."},{"key":"e_1_3_3_132_2","volume-title":"PMLR","author":"Zuluaga M.","year":"2013","unstructured":"M. Zuluaga, G. Sergent, A. Krause, and M. P\u00fcschel. 2013. Active learning for multi-objective optimization. In PMLR."}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3611663","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,3]],"date-time":"2024-04-03T04:56:27Z","timestamp":1712120187000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3611663"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,23]]},"references-count":131,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1,31]]}},"alternative-id":["10.1145\/3611663"],"URL":"http:\/\/dx.doi.org\/10.1145\/3611663","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,23]]},"assertion":[{"value":"2022-06-19","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-12","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}