{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:53:22Z","timestamp":1760144002045,"version":"build-2065373602"},"reference-count":65,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2024,3,14]],"date-time":"2024-03-14T00:00:00Z","timestamp":1710374400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["62002249","A2112"],"award-info":[{"award-number":["62002249","A2112"]}]},{"name":"Open Project Program of the State Key Lab of CADCG, Zhejiang University","award":["62002249","A2112"],"award-info":[{"award-number":["62002249","A2112"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The No Free Lunch Theorem tells us that no algorithm can beat other algorithms on all types of problems. The algorithm selection structure is proposed to select the most suitable algorithm from a set of algorithms for an unknown optimization problem. This paper introduces an innovative algorithm selection approach called the CNN-HT, which is a two-stage algorithm selection framework. In the first stage, a Convolutional Neural Network (CNN) is employed to classify problems. In the second stage, the Hypothesis Testing (HT) technique is used to suggest the best-performing algorithm based on the statistical analysis of the performance metric of algorithms that address various problem categories. The two-stage approach can adapt to different algorithm combinations without the need to retrain the entire model, and modifications can be made in the second stage only, which is an improvement of one-stage approaches. To provide a more general structure for the classification model, we adopt Exploratory Landscape Analysis (ELA) features of the problem as input and utilize feature selection techniques to reduce the redundant ones. In problem classification, the average accuracy of classifying problems using CNN is 96%, which demonstrates the advantages of CNN compared to Random Forest and Support Vector Machines. After feature selection, the accuracy increases to 98.8%, further improving the classification performance while reducing the computational cost. This demonstrates the effectiveness of the first stage of the CNN-HT method, which provides a basis for algorithm selection. In the experiments, CNN-HT shows the advantages of the second stage algorithm as well as good performance with better average rankings in different algorithm combinations compared to the individual algorithms and another algorithm combination approach.<\/jats:p>","DOI":"10.3390\/e26030262","type":"journal-article","created":{"date-parts":[[2024,3,15]],"date-time":"2024-03-15T09:32:30Z","timestamp":1710495150000},"page":"262","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["CNN-HT: A Two-Stage Algorithm Selection Framework"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-6529-4402","authenticated-orcid":false,"given":"Siyi","family":"Xu","sequence":"first","affiliation":[{"name":"School of Computer Science, Sichuan Normal University, Chengdu 610068, China"}]},{"given":"Wenwen","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science, Sichuan Normal University, Chengdu 610068, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-9460-4807","authenticated-orcid":false,"given":"Chengpei","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Computer Science, Sichuan Normal University, Chengdu 610068, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5323-858X","authenticated-orcid":false,"given":"Junli","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science, Sichuan Normal University, Chengdu 610068, China"},{"name":"Visual Computing and Virtual Reality Key Laboratory of Sichuan, Sichuan Normal University, Chengdu 610068, China"},{"name":"Key Laboratory of Land Resources Evaluation and Monitoring in Southwest (Sichuan Normal University), Ministry of Education, Chengdu 610068, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1334","DOI":"10.1021\/acs.accounts.0c00713","article-title":"Black-box optimization for automated discovery","volume":"54","author":"Terayama","year":"2021","journal-title":"Accounts Chem. Res."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1016\/j.cirp.2008.09.007","article-title":"Recent advances in engineering design optimisation: Challenges and future trends","volume":"57","author":"Roy","year":"2008","journal-title":"CIRP Ann."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"715","DOI":"10.1080\/10789669.2013.803915","article-title":"A novel artificial bee colony algorithm for HVAC optimization problems","volume":"19","author":"Zhang","year":"2013","journal-title":"HVAC&R Res."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"929","DOI":"10.1109\/TEVC.2017.2694221","article-title":"DG2: A faster and more accurate differential grouping for large-scale black-box optimization","volume":"21","author":"Omidvar","year":"2017","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Hansen, N., Auger, A., Ros, R., Finck, S., and Po\u0161\u00edk, P. (2010, January 7\u201311). Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009. Proceedings of the 12th Annual Conference Companion on Genetic and Evolutionary Computation, Portland, OR, USA.","DOI":"10.1145\/1830761.1830790"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"27434","DOI":"10.1109\/ACCESS.2021.3058128","article-title":"A modified sine cosine algorithm for solving optimization problems","volume":"9","author":"Wang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"782","DOI":"10.1016\/j.cma.2018.12.026","article-title":"Filter-based adaptive Kriging method for black-box optimization problems with expensive objective and constraints","volume":"347","author":"Shi","year":"2019","journal-title":"Comput. Methods Appl. Mech. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1016\/j.swevo.2018.04.005","article-title":"On constructing alternative benchmark suite for evolutionary algorithms","volume":"44","author":"Lou","year":"2019","journal-title":"Swarm Evol. Comput."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"100828","DOI":"10.1016\/j.swevo.2020.100828","article-title":"Non-revisiting stochastic search revisited: Results, perspectives, and future directions","volume":"61","author":"Lou","year":"2021","journal-title":"Swarm Evol. Comput."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1109\/4235.585893","article-title":"No free lunch theorems for optimization","volume":"1","author":"Wolpert","year":"1997","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"782","DOI":"10.1109\/TEVC.2010.2040183","article-title":"Population-based algorithm portfolios for numerical optimization","volume":"14","author":"Peng","year":"2010","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1162\/evco_a_00215","article-title":"Leveraging TSP solver complementarity through machine learning","volume":"26","author":"Kerschke","year":"2018","journal-title":"Evol. Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1162\/evco_a_00242","article-title":"Automated algorithm selection: Survey and perspectives","volume":"27","author":"Kerschke","year":"2019","journal-title":"Evol. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep Learning in Neural Networks: An Overview","volume":"61","author":"Schmidhuber","year":"2015","journal-title":"Neural Netw."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4052","DOI":"10.1109\/TCYB.2020.3013251","article-title":"Predicting network controllability robustness: A convolutional neural network approach","volume":"52","author":"Lou","year":"2022","journal-title":"IEEE Trans. Cybern."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/S0065-2458(08)60520-3","article-title":"The algorithm selection problem","volume":"Volume 15","author":"Rice","year":"1976","journal-title":"Advances in Computers"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bischl, B., Mersmann, O., Trautmann, H., and Preu\u00df, M. (2012, January 7\u201311). Algorithm selection based on exploratory landscape analysis and cost-sensitive learning. Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation, Philadelphia, PA, USA.","DOI":"10.1145\/2330163.2330209"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Li, Z., Tian, X., Liu, X., Liu, Y., and Shi, X. (2022). A Two-Stage Industrial Defect Detection Framework Based on Improved-YOLOv5 and Optimized-Inception-ResnetV2 Models. Appl. Sci., 12.","DOI":"10.3390\/app12020834"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Khan, M.A., Karim, M.R., and Kim, Y. (2018). A Two-Stage Big Data Analytics Framework with Real World Applications Using Spark Machine Learning and Long Short-Term Memory Network. Symmetry, 10.","DOI":"10.3390\/sym10100485"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liu, A., Xiao, Y., Ji, X., Wang, K., Tsai, S.B., Lu, H., Cheng, J., Lai, X., and Wang, J. (2018). A novel two-stage integrated model for supplier selection of green fresh product. Sustainability, 10.","DOI":"10.3390\/su10072371"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, X., Yang, W., Tang, X., and Liu, J. (2018). A fast learning method for accurate and robust lane detection using two-stage feature extraction with YOLO v3. Sensors, 18.","DOI":"10.3390\/s18124308"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"106138","DOI":"10.1016\/j.asoc.2020.106138","article-title":"Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis","volume":"90","author":"Eftimov","year":"2020","journal-title":"Appl. Soft Comput."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Renau, Q., Doerr, C., Dreo, J., and Doerr, B. (2020, January 5\u20139). Exploratory landscape analysis is strongly sensitive to the sampling strategy. Proceedings of the Parallel Problem Solving from Nature\u2013PPSN XVI: 16th International Conference, PPSN 2020, Leiden, The Netherlands. Proceedings, Part II 16.","DOI":"10.1007\/978-3-030-58115-2_10"},{"key":"ref_24","unstructured":"Mersmann, O., Preuss, M., and Trautmann, H. (2010). Parallel Problem Solving from Nature, Springer."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kerschke, P., and Bossek, J.T.H. (2018, January 15\u201319). Parameterization of state-of-the-art performance indicators: A robustness study based on inexact TSP solvers. Proceedings of the 20th Annual Conference on Genetic and Evolutionary Computation (GECCO) Companion, Kyoto, Japan.","DOI":"10.1145\/3205651.3208233"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1109\/TAI.2020.3022339","article-title":"A recommender system for metaheuristic algorithms for continuous optimization based on deep recurrent neural networks","volume":"1","author":"Tian","year":"2020","journal-title":"IEEE Trans. Artif. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Mersmann, O., Bischl, B., Trautmann, H., Preuss, M., Weihs, C., and Rudolph, G. (2011, January 12\u201316). Exploratory landscape analysis. Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, Dublin, Ireland.","DOI":"10.1145\/2001576.2001690"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Malan, K.M., Oberholzer, J.F., and Engelbrecht, A.P. (2015, January 25\u201328). Characterising constrained continuous optimisation problems. Proceedings of the 2015 IEEE Congress on Evolutionary Computation (CEC), Sendai, Japan.","DOI":"10.1109\/CEC.2015.7257045"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3787","DOI":"10.1007\/s00500-016-2091-4","article-title":"Bag of local landscape features for fitness landscape analysis","volume":"20","author":"Shirakawa","year":"2016","journal-title":"Soft Comput."},{"key":"ref_30","first-page":"1","article-title":"flacco: Feature-based landscape analysis of continuous and constrained optimization problems","volume":"1","author":"Kerschke","year":"2017","journal-title":"R-Package Version"},{"key":"ref_31","first-page":"1","article-title":"Feature selection: A data perspective","volume":"50","author":"Li","year":"2017","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1016\/j.inffus.2022.09.026","article-title":"Fusion of statistical importance for feature selection in Deep Neural Network-based Intrusion Detection System","volume":"90","author":"Thakkar","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_33","unstructured":"Bol\u00f3n-Canedo, V., Alonso-Betanzos, A., Mor\u00e1n-Fern\u00e1ndez, L., and Cancela, B. (2022). Advances in Selected Artificial Intelligence Areas: World Outstanding Women in Artificial Intelligence, Springer."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1197","DOI":"10.1109\/TFUZZ.2021.3053844","article-title":"Feature selection with missing labels using multilabel fuzzy neighborhood rough sets and maximum relevance minimum redundancy","volume":"30","author":"Sun","year":"2021","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_35","first-page":"3","article-title":"A review of feature selection and its methods","volume":"19","author":"Venkatesh","year":"2019","journal-title":"Cybern. Inf. Technol."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Abbas, F., Zhang, F., Abbas, F., Ismail, M., Iqbal, J., Hussain, D., Khan, G., Alrefaei, A.F., and Albeshr, M.F. (2023). Landslide Susceptibility Mapping: Analysis of Different Feature Selection Techniques with Artificial Neural Network Tuned by Bayesian and Metaheuristic Algorithms. Remote Sens., 15.","DOI":"10.20944\/preprints202307.1467.v1"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"108256","DOI":"10.1016\/j.knosys.2022.108256","article-title":"Multi-label feature selection based on label correlations and feature redundancy","volume":"241","author":"Fan","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_38","first-page":"1205","article-title":"Efficient feature selection via analysis of relevance and redundancy","volume":"5","author":"Yu","year":"2004","journal-title":"J. Mach. Learn. Res."},{"key":"ref_39","first-page":"1060","article-title":"Stability of feature selection algorithm: A review","volume":"34","author":"Khaire","year":"2022","journal-title":"J. King Saud Univ.-Comput. Inf. Sci."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"3577","DOI":"10.1016\/j.ins.2008.05.024","article-title":"Neighborhood rough set based heterogeneous feature subset selection","volume":"178","author":"Hu","year":"2008","journal-title":"Inf. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"5739","DOI":"10.1109\/TNNLS.2021.3071367","article-title":"Knowledge-based prediction of network controllability robustness","volume":"33","author":"Lou","year":"2022","journal-title":"IEEE Trans. Neural Networks Learn. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"3209","DOI":"10.1109\/TNSE.2021.3107186","article-title":"A Convolutional Neural Network Approach to Predicting Network Connectedness Robustness","volume":"8","author":"Lou","year":"2021","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"ref_43","unstructured":"Lou, Y., Wu, C., Li, J., Wang, L., and Chen, G. (2023). IEEE Transactions on Neural Networks and Learning Systems, IEEE."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1016\/j.patcog.2017.10.013","article-title":"Recent advances in convolutional neural networks","volume":"77","author":"Gu","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_45","unstructured":"Li, Z., Liu, F., Yang, W., Peng, S., and Zhou, J. (2021). IEEE Transactions on Neural Networks and Learning Systems, IEEE."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Albawi, S., Mohammed, T.A., and Al-Zawi, S. (2017, January 21\u201323). Understanding of a convolutional neural network. Proceedings of the 2017 International Conference on Engineering and Technology (ICET), Antalya, Turkey.","DOI":"10.1109\/ICEngTechnol.2017.8308186"},{"key":"ref_47","unstructured":"Dhaenens, C., Jourdan, L., and Marmion, M.E. (2015, January 12\u201315). Improving the State of the Art in Inexact TSP Solving Using Per-Instance Algorithm Selection. Proceedings of the Learning and Intelligent Optimization, Lille, France."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Loreggia, A., Malitsky, Y., Samulowitz, H., and Saraswat, V. (2016, January 12\u201317). Deep learning for algorithm portfolios. Proceedings of the Aaai Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10170"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1016\/j.swevo.2018.07.001","article-title":"A sequential algorithm portfolio approach for black box optimization","volume":"44","author":"He","year":"2019","journal-title":"Swarm Evol. Comput."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Wilcox, R.R. (2011). Introduction to Robust Estimation and Hypothesis Testing, Academic Press.","DOI":"10.1016\/B978-0-12-386983-8.00001-9"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.cor.2013.11.015","article-title":"Towards objective measures of algorithm performance across instance space","volume":"45","author":"Baatar","year":"2014","journal-title":"Comput. Oper. Res."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1080\/01621459.1952.10483441","article-title":"Use of ranks in one-criterion variance analysis","volume":"47","author":"Kruskal","year":"1952","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_53","unstructured":"Lou, Y., Yuen, S.Y., and Chen, G. (2019). From Parallel to Emergent Computing, CRC Press."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Lou, Y., Yuen, S.Y., and Chen, G. (2018, January 15\u201319). Evolving benchmark functions using kruskal-wallis test. Proceedings of the Genetic and Evolutionary Computation Conference Companion, Kyoto, Japan.","DOI":"10.1145\/3205651.3208257"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1080\/00401706.2000.10485979","article-title":"A comparison of three methods for selecting values of input variables in the analysis of output from a computer code","volume":"42","author":"McKay","year":"2000","journal-title":"Technometrics"},{"key":"ref_56","first-page":"18","article-title":"Classification and regression by randomForest","volume":"2","author":"Liaw","year":"2002","journal-title":"R News"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v011.i09","article-title":"kernlab-an S4 package for kernel methods in R","volume":"11","author":"Karatzoglou","year":"2004","journal-title":"J. Stat. Softw."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/TEVC.2010.2087271","article-title":"Differential evolution with composite trial vector generation strategies and control parameters","volume":"15","author":"Wang","year":"2011","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1162\/106365601750190398","article-title":"Completely derandomized self-adaptation in evolution strategies","volume":"9","author":"Hansen","year":"2001","journal-title":"Evol. Comput."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.swevo.2018.02.013","article-title":"A novel nature-inspired algorithm for optimization: Squirrel search algorithm","volume":"44","author":"Jain","year":"2019","journal-title":"Swarm Evol. Comput."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Tanabe, R., and Fukunaga, A.S. (2014, January 6\u201311). Improving the search performance of SHADE using linear population size reduction. Proceedings of the 2014 IEEE Congress on Evolutionary Computation (CEC), Beijing, China.","DOI":"10.1109\/CEC.2014.6900380"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"49445","DOI":"10.1109\/ACCESS.2022.3151641","article-title":"Zebra Optimization Algorithm: A New Bio-Inspired Optimization Algorithm for Solving Optimization Algorithm","volume":"10","author":"Dehghani","year":"2022","journal-title":"IEEE Access"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1007\/s10898-007-9149-x","article-title":"A powerful and efficient algorithm for numerical function optimization: Artificial bee colony (ABC) algorithm","volume":"39","author":"Karaboga","year":"2007","journal-title":"J. Glob. Optim."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1109\/TEVC.2008.927706","article-title":"Differential evolution algorithm with strategy adaptation for global numerical optimization","volume":"13","author":"Qin","year":"2008","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"1088","DOI":"10.1016\/j.ejor.2006.06.043","article-title":"Global and local real-coded genetic algorithms based on parent-centric crossover operators","volume":"185","author":"Lozano","year":"2008","journal-title":"Eur. J. Oper. Res."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/3\/262\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:13:39Z","timestamp":1760105619000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/3\/262"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,14]]},"references-count":65,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["e26030262"],"URL":"https:\/\/doi.org\/10.3390\/e26030262","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2024,3,14]]}}}