{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T07:10:50Z","timestamp":1777705850561,"version":"3.51.4"},"reference-count":37,"publisher":"SAGE Publications","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IFS"],"published-print":{"date-parts":[[2022,4,28]]},"abstract":"<jats:p>In recent years, solving combinatorial optimization problems involves more complications, high dimensions, and multi-objective considerations. Combining the advantages of other evolutionary algorithms to enhance the performance of a unique evolutionary algorithm and form a new hybrid heuristic algorithm has become a way to strengthen the performance of the algorithm effectively. However, the intelligent hybrid heuristic algorithm destroys the integrity, universality, and robustness of the original algorithm to a certain extent and increases its time complexity. This paper implements a new idea \u201cML to choose heuristics\u201d (a heuristic algorithm combined with machine learning technology) which uses the Q-learning method to learn different strategies in genetic algorithm. Moreover, a selection-based hyper-heuristic algorithm is obtained that can guide the algorithm to make decisions at different time nodes to select appropriate strategies. The algorithm is the hybrid strategy using Q-learning on StudGA (HSQ-StudGA). The experimental results show that among the 14 standard test functions, the evolutionary algorithm guided by Q-learning can effectively improve the quality of arithmetic solution. Under the premise of not changing the evolutionary structure of the algorithm, the hyper-heuristic algorithm represents a new method to solve combinatorial optimization problems.<\/jats:p>","DOI":"10.3233\/jifs-211250","type":"journal-article","created":{"date-parts":[[2022,4,26]],"date-time":"2022-04-26T13:33:21Z","timestamp":1650980001000},"page":"5041-5053","source":"Crossref","is-referenced-by-count":3,"title":["A novel intelligent hyper-heuristic algorithm for solving optimization problems"],"prefix":"10.1177","volume":"42","author":[{"given":"Zhao","family":"Tong","sequence":"first","affiliation":[{"name":"College of Information Science and Engineering, Hunan Normal University, Changsha, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongjian","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Hunan Normal University, Changsha, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bilan","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Hunan Normal University, Changsha, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinhui","family":"Cai","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Hunan Normal University, Changsha, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuo","family":"Cai","sequence":"additional","affiliation":[{"name":"The School of Computer and Communication Engineering, Changsha University of Science and Technology, Changsha, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"key":"10.3233\/JIFS-211250_ref1","doi-asserted-by":"crossref","first-page":"5553","DOI":"10.1007\/s00521-019-04118-8","article-title":"Ql-heft: a novel machine learning scheduling scheme base on cloud computing environment","volume":"32","author":"Tong","year":"2020","journal-title":"Neural Computing and Applications"},{"issue":"99","key":"10.3233\/JIFS-211250_ref2","first-page":"1","article-title":"Containment control of semi-markovian multiagent systems with switching topologies","volume":"PP","author":"Liang","year":"2021","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics: Systems"},{"key":"10.3233\/JIFS-211250_ref3","doi-asserted-by":"crossref","unstructured":"Holland J.H. , et al, Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence, MIT press, 1992.","DOI":"10.7551\/mitpress\/1090.001.0001"},{"issue":"3","key":"10.3233\/JIFS-211250_ref4","doi-asserted-by":"crossref","first-page":"692","DOI":"10.1109\/TPDS.2020.3030920","article-title":"Hierarchical multi-agent optimization for resource allocation in cloud computing","volume":"32","author":"Gao","year":"2021","journal-title":"IEEE Transactions on Parallel and Distributed Systems"},{"issue":"6","key":"10.3233\/JIFS-211250_ref5","doi-asserted-by":"crossref","first-page":"1362","DOI":"10.1109\/TSMCB.2009.2015956","article-title":"Adaptive particle swarm optimization","volume":"39","author":"Zhan","year":"2009","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)"},{"key":"10.3233\/JIFS-211250_ref6","doi-asserted-by":"crossref","unstructured":"Zhang W. , Wang C. , Lin W. and Lin J. , Continuous-domain ant colony optimization algorithm based on reinforcement learning, International Journal of Wavelets, Multiresolution and Information Processing, 2020.","DOI":"10.1142\/S0219691320500848"},{"key":"10.3233\/JIFS-211250_ref7","doi-asserted-by":"crossref","unstructured":"Dorigo M. and St\u00fctzle T. , The ant colony optimization metaheuristic: Algorithms, applications, and advances, In Handbook of metaheuristics, pages 250\u2013285, Springer, 2003.","DOI":"10.1007\/0-306-48056-5_9"},{"issue":"3","key":"10.3233\/JIFS-211250_ref8","first-page":"112","article-title":"Multi-objective data collecting strategies for wireless sensor network based on the time variable multi-salesman problem and genetic algorithm","volume":"38","author":"Feng","year":"2017","journal-title":"Journal on Communication"},{"issue":"5","key":"10.3233\/JIFS-211250_ref9","doi-asserted-by":"crossref","first-page":"885","DOI":"10.1007\/s11432-010-0080-2","article-title":"Basin filling algorithm for the circular packing problem with equilibrium behavioral constraints","volume":"53","author":"Liu","year":"2010","journal-title":"Science China Information Sciences"},{"issue":"2","key":"10.3233\/JIFS-211250_ref10","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1007\/s10619-017-7215-z","article-title":"An adaptive multi-objective evolutionary algorithm for constrained workflow scheduling in clouds","volume":"36","author":"Zhang","year":"2018","journal-title":"Distributed and Parallel Databases"},{"key":"10.3233\/JIFS-211250_ref11","doi-asserted-by":"crossref","unstructured":"Khatib W. and Fleming P.J. , The stud ga: a mini revolution? In International Conference on Parallel Problem Solving from Nature, pages 683\u2013691, Springer, 1998.","DOI":"10.1007\/BFb0056910"},{"issue":"20","key":"10.3233\/JIFS-211250_ref12","first-page":"239","article-title":"Opposition-based stud genetic algorithm","volume":"35","author":"Dong","year":"2009","journal-title":"Computer Engineering"},{"issue":"14","key":"10.3233\/JIFS-211250_ref13","first-page":"1","article-title":"Review of research progress of hyper-heuristic algorithms","volume":"2017","author":"Xie","year":"2017","journal-title":"Computer Engineering and Applications"},{"key":"10.3233\/JIFS-211250_ref14","unstructured":"Tian Y. , Lu C. , Zhang X. , Tan K.C. and Jin Y. , Solving large-scale multi-objective optimization problems with sparse optimal solutions via unsupervised neural networks, IEEE Transactions on Cybernetics PP(99) (2020)."},{"issue":"3-4","key":"10.3233\/JIFS-211250_ref15","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Machine Learning"},{"key":"10.3233\/JIFS-211250_ref16","volume-title":"Markov decision processes: discrete stochastic dynamic programming","author":"Puterman","year":"2014"},{"issue":"2","key":"10.3233\/JIFS-211250_ref17","first-page":"193","article-title":"Advances in co-evolutionary algorithms","volume":"30","author":"Wang","year":"2015","journal-title":"Control and Decision"},{"key":"10.3233\/JIFS-211250_ref18","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1016\/j.ins.2017.08.042","article-title":"Task aware hybrid dvfs for multi-core real-time systems using machine learning","volume":"433","author":"ul Islam","year":"2018","journal-title":"Information Sciences"},{"key":"10.3233\/JIFS-211250_ref19","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/j.ins.2018.01.005","article-title":"Automatic design of hyper-heuristic based on reinforcement learning","volume":"436","author":"Choong","year":"2018","journal-title":"Information Sciences"},{"key":"10.3233\/JIFS-211250_ref20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ins.2017.10.041","article-title":"A q-learning-based memetic algorithm for multi-objective dynamic software project scheduling","volume":"428","author":"Shen","year":"2018","journal-title":"Information Sciences"},{"key":"10.3233\/JIFS-211250_ref21","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1016\/j.autcon.2012.11.002","article-title":"A new tabu search-based hyper-heuristic algorithm for solving construction leveling problems with limited resource availabilities","volume":"31","author":"Koulinas","year":"2013","journal-title":"Automation in Construction"},{"key":"10.3233\/JIFS-211250_ref22","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.engappai.2016.02.004","article-title":"A novel multistart hyper-heuristic algorithm on the grid for the quadratic assignment problem","volume":"52","author":"Dokeroglu","year":"2016","journal-title":"Engineering Applications of Artificial Intelligence"},{"key":"10.3233\/JIFS-211250_ref23","doi-asserted-by":"crossref","unstructured":"Qian Z. , Zhao Y. , Wang S. , Leng L. and Wang W. , A hyper heuristic algorithm for low carbon location routing problem, In International Symposium on Neural Networks, pages 173\u2013182. Springer, 2018.","DOI":"10.1007\/978-3-319-92537-0_21"},{"issue":"11","key":"10.3233\/JIFS-211250_ref24","doi-asserted-by":"crossref","first-page":"10307","DOI":"10.1109\/TVT.2018.2868942","article-title":"Parallel hyper-heuristic algorithm for multi-objective route planning in a smart city","volume":"67","author":"Yao","year":"2018","journal-title":"IEEE Transactions on Vehicular Technology"},{"key":"10.3233\/JIFS-211250_ref25","volume-title":"Genetic algorithms","author":"Goldberg","year":"2006"},{"key":"10.3233\/JIFS-211250_ref26","unstructured":"Syswerda G. , Uniform crossover in genetic algorithms, In Proceedings of the 3rd international conference on genetic algorithms, pages 2\u20139, 1989."},{"key":"10.3233\/JIFS-211250_ref27","unstructured":"Tanese R. , Distributed genetic algorithms for function optimization, 1989."},{"issue":"6","key":"10.3233\/JIFS-211250_ref28","doi-asserted-by":"crossref","first-page":"702","DOI":"10.1109\/TEVC.2008.919004","article-title":"Biogeography-based optimization","volume":"12","author":"Simon","year":"2008","journal-title":"IEEE Transactions on Evolutionary Computation"},{"issue":"4","key":"10.3233\/JIFS-211250_ref29","doi-asserted-by":"crossref","first-page":"474","DOI":"10.1177\/0272989X09353194","article-title":"Markov decision processes: A tool for sequential decision making under uncertainty","volume":"30","author":"Alagoz","year":"2010","journal-title":"Medical Decision Making"},{"issue":"4","key":"10.3233\/JIFS-211250_ref30","doi-asserted-by":"crossref","first-page":"2483","DOI":"10.1109\/JIOT.2020.3033285","article-title":"Offloading time optimization via markov decision process in mobile-edge computing","volume":"8","author":"Yang","year":"2021","journal-title":"IEEE Internet of Things Journal"},{"issue":"3","key":"10.3233\/JIFS-211250_ref31","doi-asserted-by":"crossref","first-page":"1092","DOI":"10.1109\/TMC.2019.2959772","article-title":"Learning-based mobile edge computing resource management to support public blockchain networks","volume":"20","author":"Asheralieva","year":"2021","journal-title":"IEEE Transactions on Mobile Computing"},{"issue":"3","key":"10.3233\/JIFS-211250_ref32","doi-asserted-by":"crossref","first-page":"2341","DOI":"10.1007\/s00500-019-04064-6","article-title":"Gas chimney and hydrocarbon detection using combined bbo and artificial neural network with hybrid seismic attributes","volume":"24","author":"Ramya","year":"2020","journal-title":"Soft Computing"},{"issue":"99","key":"10.3233\/JIFS-211250_ref33","first-page":"1","article-title":"Maximizing radiated high-power electromagnetic threat to transmission line system under the constraints of bounded bandwidth and amplitude","volume":"PP","author":"Liang","year":"2020","journal-title":"IEEE Transactions on Electromagnetic Compatibility"},{"issue":"2","key":"10.3233\/JIFS-211250_ref34","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/4235.771163","article-title":"Evolutionary programming made faster","volume":"3","author":"Yao","year":"1999","journal-title":"IEEE Transactions on Evolutionary Computation"},{"issue":"7","key":"10.3233\/JIFS-211250_ref35","doi-asserted-by":"crossref","first-page":"2015","DOI":"10.1007\/s00521-015-1971-3","article-title":"Species co-evolutionary algorithm: a novel evolutionary algorithm based on the ecology and environments for optimization","volume":"31","author":"Li","year":"2019","journal-title":"Neural Computing and Applications"},{"issue":"3","key":"10.3233\/JIFS-211250_ref36","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1080\/921622054","article-title":"Minimizing multimodal functions by simplex coding genetic algorithm","volume":"18","author":"Hedar","year":"2003","journal-title":"Optimization Methods and Software"},{"issue":"18","key":"10.3233\/JIFS-211250_ref37","doi-asserted-by":"crossref","first-page":"3444","DOI":"10.1016\/j.ins.2010.05.035","article-title":"An analysis of the equilibrium of migration models for biogeography-based optimization","volume":"180","author":"Ma","year":"2010","journal-title":"Information Sciences"}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/JIFS-211250","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:45:24Z","timestamp":1777455924000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/JIFS-211250"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,28]]},"references-count":37,"journal-issue":{"issue":"6"},"URL":"https:\/\/doi.org\/10.3233\/jifs-211250","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,28]]}}}