{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T21:50:37Z","timestamp":1774129837469,"version":"3.50.1"},"reference-count":57,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2020,3,11]],"date-time":"2020-03-11T00:00:00Z","timestamp":1583884800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"Wuhan Technology and Business University","award":["D2019010"],"award-info":[{"award-number":["D2019010"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,1,19]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>As a novel swarm intelligence optimization algorithm, cuckoo search (CS) has been successfully applied to solve diverse problems in the real world. Despite its efficiency and wide use, CS has some disadvantages, such as premature convergence, easy to fall into local optimum and poor balance between exploitation and exploration. In order to improve the optimization performance of the CS algorithm, a new CS extension with multi-swarms and Q-Learning namely MP-QL-CS is proposed. The step size strategy of the CS algorithm is that an individual fitness value is examined based on a one-step evolution effect of an individual instead of evaluating the step size from the multi-step evolution effect. In the MP-QL-CS algorithm, a step size control strategy is considered as action, which is used to examine the individual multi-stepping evolution effect and learn the individual optimal step size by calculating the Q function value. In this way, the MP-QL-CS algorithm can increase the adaptability of individual evolution, and a good balance between diversity and intensification can be achieved. Comparing the MP-QL-CS algorithm with various CS algorithms, variants of differential evolution (DE) and improved particle swarm optimization (PSO) algorithms, the results demonstrate that the MP-QL-CS algorithm is a competitive swarm algorithm.<\/jats:p>","DOI":"10.1093\/comjnl\/bxz149","type":"journal-article","created":{"date-parts":[[2019,10,31]],"date-time":"2019-10-31T20:42:51Z","timestamp":1572554571000},"page":"108-131","source":"Crossref","is-referenced-by-count":14,"title":["Multi-Swarm Cuckoo Search Algorithm with <i>Q-<\/i>Learning Model"],"prefix":"10.1093","volume":"64","author":[{"given":"Juan","family":"Li","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence, Wuhan Technology and Business University, Wuhan 430065, China"},{"name":"School of Artificial Intelligence, Wuchang University of Technology, Wuhan 430223, China"},{"name":"Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, No. 2699 Qianjin Street, Gaoxin District, Changchun, Jilin, China, 130012"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dan-dan","family":"Xiao","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Wuhan Technology and Business University, Wuhan 430065, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ting","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Wuhan Technology and Business University, Wuhan 430065, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chun","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Wuhan Technology and Business University, Wuhan 430065, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuan-xiang","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer, Wuhan University, No. 299 Bayi Street, Wuchang District, Wuhan, Hubei, China, 430072"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gai-ge","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Ocean University of China, No. 238 Songling Street, Laoshan District, Qingdao, Shandong, China, 266100"},{"name":"Institute of Algorithm and Big Data Analysis, Northeast Normal University, No. 5268 Renmin Street, Nanguan District, Changchun, Jilin, China, 130117"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2020,3,11]]},"reference":[{"key":"2021011807350541500_ref1","first-page":"718","volume-title":"IEEE Congr Evol Comput","author":"Chakraborty","year":"2008"},{"key":"2021011807350541500_ref2","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1016\/j.asoc.2017.12.002","article-title":"A novel parallel hurricane optimization algorithm for secure emission\/economic load dispatch solution","volume":"63","author":"Rizk-Allah","year":"2018","journal-title":"Appl. Soft. Comput."},{"key":"2021011807350541500_ref3","first-page":"358","article-title":"A survey on clustering algorithms for wireless sensor networks","volume":"30","author":"Boyinbode","year":"2010","journal-title":"Int. Conf. Netw. Info."},{"key":"2021011807350541500_ref4","first-page":"452","article-title":"A multi-channel phase-coherent X-band frequency synthesizer for array radar applications","author":"Chen","year":"2017","journal-title":"IEEE Asia Pacific Micro. Conf."},{"key":"2021011807350541500_ref5","doi-asserted-by":"crossref","first-page":"3538","DOI":"10.1016\/j.eswa.2013.10.059","article-title":"Cuckoo search algoriynm and wind driven optimization based study of satellite image segmentation for multilevel thresholding using kapur\u2019s entropy","volume":"41","author":"Bhandan","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"2021011807350541500_ref6","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1007\/BF02823145","article-title":"An introduction to genetic algorithms","volume":"24","author":"Deb","year":"1999","journal-title":"Sadhan"},{"key":"2021011807350541500_ref7","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1109\/TEVC.2007.894200","article-title":"Opposition-based differential evolution","volume":"12","author":"Rahnamayan","year":"2008","journal-title":"IEEE Trans. Evol. Comput."},{"key":"2021011807350541500_ref8","doi-asserted-by":"crossref","first-page":"2906","DOI":"10.1016\/j.asoc.2012.04.013","article-title":"A new hybrid artificial bee colony algorithm for robust optimal design and manufacturing","volume":"13","author":"Yildiz","year":"2013","journal-title":"Appl. Soft. Comput."},{"key":"2021011807350541500_ref9","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.ins.2014.04.050","article-title":"Bidirectional teaching and peer-learning particle swarm optimization","volume":"280","author":"Lim","year":"2014","journal-title":"Inf. Sci."},{"key":"2021011807350541500_ref10","doi-asserted-by":"crossref","first-page":"706","DOI":"10.1016\/j.eswa.2011.07.062","article-title":"Two-stage updating pheromone for invariant ant colony optimization algorithm","volume":"39","author":"Zhang","year":"2012","journal-title":"Expert Syst. Appl."},{"key":"2021011807350541500_ref11","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.ins.2014.02.123","article-title":"Chaotic krill herd algorithm","volume":"274","author":"Wang","year":"2014","journal-title":"Inf. Sci."},{"key":"2021011807350541500_ref12","doi-asserted-by":"crossref","first-page":"2454","DOI":"10.1016\/j.apm.2013.10.052","article-title":"An effective krill herd algorithm with migration operator in biogeography-based optimization","volume":"38","author":"Wang","year":"2014","journal-title":"Appl. Math Model"},{"key":"2021011807350541500_ref13","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1016\/j.neucom.2013.08.031","article-title":"Stud krill herd algorithm","volume":"128","author":"Wang","year":"2014","journal-title":"Neurocomputing"},{"key":"2021011807350541500_ref14","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1007\/s00521-012-1304-8","article-title":"Incorporating mutation scheme into krill herd algorithm for global numerical optimization","volume":"24","author":"Wang","year":"2014","journal-title":"Neural Comput. Appl."},{"key":"2021011807350541500_ref15","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/s00500-014-1502-7","article-title":"Hybridizing harmony search algorithm with cuckoo search for global numerical optimization","volume":"20","author":"Wang","year":"2016","journal-title":"Soft. Comput."},{"key":"2021011807350541500_ref16","doi-asserted-by":"crossref","first-page":"2318","DOI":"10.1166\/jctn.2013.3207","article-title":"Hybridizing harmony search with biogeography based optimization for global numerical optimization","volume":"10","author":"Wang","year":"2013","journal-title":"J. Comput. Theor. Nanos."},{"key":"2021011807350541500_ref17","doi-asserted-by":"crossref","first-page":"10708","DOI":"10.1109\/ACCESS.2018.2809445","article-title":"Binary moth search algorithm for discounted {0-1} knapsack problem","volume":"6","author":"Feng","year":"2018","journal-title":"IEEE Access"},{"key":"2021011807350541500_ref18","first-page":"1","article-title":"Monarch butterfly optimization","author":"Wang","year":"2015","journal-title":"Neural Comput. Appl."},{"key":"2021011807350541500_ref19","first-page":"731","article-title":"A new monarch butterfly optimization with an improved crossover operator","volume":"18","author":"Wang","year":"2018","journal-title":"Oper. Res."},{"key":"2021011807350541500_ref20","first-page":"1","article-title":"The discovery of population interaction with a power law distribution in brain storm optimization","volume":"5939","author":"Wang","year":"2017","journal-title":"Memet. Comput."},{"key":"2021011807350541500_ref21","doi-asserted-by":"crossref","first-page":"1233","DOI":"10.1007\/s00521-013-1354-6","article-title":"Enhancing the performance of cuckoo search algorithm using orthogonal learning method","volume":"24","author":"Li","year":"2014","journal-title":"Neural Comput. Appl."},{"key":"2021011807350541500_ref22","doi-asserted-by":"crossref","first-page":"1659","DOI":"10.1007\/s00521-013-1402-2","article-title":"Discrete cuckoo search algorithm for the travelling salesman problem","volume":"24","author":"Ouaarab","year":"2014","journal-title":"Neural Comput. Appl."},{"key":"2021011807350541500_ref23","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1016\/j.jpdc.2016.10.011","article-title":"A novel oriented cuckoo search algorithm to improve DV-Hop performance for cyber-physical systems","volume":"103","author":"Cui","year":"2017","journal-title":"J. Parallel Distr. Comput."},{"key":"2021011807350541500_ref24","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1016\/j.ijepes.2014.10.004","article-title":"Modified cuckoo search algorithm for short-term hydrothermal scheduling","volume":"65","author":"Nguyen","year":"2015","journal-title":"Electr. Power Energy Syst."},{"key":"2021011807350541500_ref25","first-page":"210","article-title":"Cuckoo search via L\u00e9vy flights","volume":"71","author":"Yang","year":"2010","journal-title":"World Cong. Nat. Biol. Inspir. Comput."},{"key":"2021011807350541500_ref26","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1016\/j.cie.2012.07.011","article-title":"Improved cuckoo search for reliability optimization problems","volume":"64","author":"Vallan","year":"2013","journal-title":"Comput. Ind. Eng."},{"key":"2021011807350541500_ref27","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.ins.2014.11.042","article-title":"Modified cuckoo search algorithm with self adaptive parameter method","volume":"298","author":"Li","year":"2015","journal-title":"Inf. Sci."},{"key":"2021011807350541500_ref28","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1504\/IJBIC.2016.079569","article-title":"A new hybrid method based on krill herd and cuckoo search for global optimization tasks","volume":"8","author":"Wang","year":"2016","journal-title":"Int. J. Bio.-Inspir. Comput."},{"key":"2021011807350541500_ref29","first-page":"1172","article-title":"Hybrid optimization algorithm of PSO and cuckoo search","author":"Wang","year":"2011","journal-title":"Int. Conf. Artif. Intell."},{"key":"2021011807350541500_ref30","first-page":"16:28","article-title":"A cuckoo optimization algorithm using elite opposition-based learning and chaotic disturbance","volume":"10","author":"Li","year":"2016","journal-title":"J. Softw. Eng."},{"key":"2021011807350541500_ref31","first-page":"330","article-title":"Engineering optimization by cuckoo search","volume":"1","author":"Yang","year":"2010","journal-title":"J. Mathl. Model Numer. Optim."},{"key":"2021011807350541500_ref32","first-page":"31","article-title":"Improved cuckoo search for global optimization","volume":"1","author":"Valian","year":"2011","journal-title":"Int. J. Commun. Info. Technol."},{"key":"2021011807350541500_ref33","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1002\/tal.1033","article-title":"Design optimization of truss structures using cuckoo search algorithm","volume":"22","author":"Gandomi","year":"2012","journal-title":"Struct. Des. Tall Spec. Build."},{"key":"2021011807350541500_ref34","first-page":"564","volume-title":"Proc. of 3rd Int. Conf Electron Comput Technol (ICECT2011), 8\u201311 April n","author":"Kumar","year":"2011"},{"key":"2021011807350541500_ref35","first-page":"134","article-title":"Cuckoo optimization algorithm based on the communication operator","volume":"35","author":"Qu","year":"2014","journal-title":"J. Chin. Comput. Syst."},{"key":"2021011807350541500_ref36","doi-asserted-by":"crossref","first-page":"777","DOI":"10.12785\/amis\/070248","article-title":"A novel discrete cuckoo searchalgorithm for spherical traveling salesman problem","volume":"7","author":"Ouyang","year":"2013","journal-title":"Appl. Math Inf. Sci."},{"key":"2021011807350541500_ref37","first-page":"68","article-title":"An e_cient optimization algorithm for structural software testing","volume":"9","author":"Srivastava","year":"2012","journal-title":"Int. J. Artif. Intell."},{"key":"2021011807350541500_ref38","doi-asserted-by":"crossref","first-page":"642","DOI":"10.1504\/IJMC.2011.042781","article-title":"Cuckoo search for data gathering in wireless sensor network","volume":"9","author":"Dhivya","year":"2011","journal-title":"Int. J. Mob. Commun."},{"key":"2021011807350541500_ref39","first-page":"68","article-title":"Self-adaptive step cuckoo search algorithm","volume":"49","author":"Zheng","year":"2013","journal-title":"Comput. Eng. Appl."},{"key":"2021011807350541500_ref40","first-page":"45","article-title":"Improved cuckoo algorithm based on adaptive step size","volume":"34","author":"Sun","year":"2018","journal-title":"J. Chifeng Univ."},{"key":"2021011807350541500_ref41","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.swevo.2016.03.001","article-title":"SHybrid self-adaptive cuckoo search for global optimization","volume":"29","author":"Zheng","year":"2016","journal-title":"Swarm Evol. Comput."},{"key":"2021011807350541500_ref42","doi-asserted-by":"crossref","first-page":"2184","DOI":"10.1016\/j.engappai.2013.06.016","article-title":"Backward Q-learning: The combination of Sarsa algorithm and Q-learning","volume":"26","author":"Wang","year":"2013","journal-title":"Eng. Appl. Artif. Intel."},{"key":"2021011807350541500_ref43","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1016\/j.asoc.2016.01.006","article-title":"A new reinforcement learning-based memetic particle swarm optimizer","volume":"43","author":"Hussein","year":"2016","journal-title":"Appl. Soft. Comput."},{"key":"2021011807350541500_ref44","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1016\/j.asoc.2016.08.032","article-title":"Cooperative learning for radial basis function networks using particle swarm optimization","volume":"49","author":"Alex","year":"2016","journal-title":"Appl. Soft. Comput."},{"key":"2021011807350541500_ref45","doi-asserted-by":"crossref","first-page":"6771","DOI":"10.1007\/s13369-017-2873-8","article-title":"Action-selection method for reinforcement learning based on cuckoo search algorithm","volume":"43","author":"Bilal","year":"2018","journal-title":"Arab J. Sci. Eng."},{"key":"2021011807350541500_ref46","first-page":"356","article-title":"A cuckoo searching algorithm using elite opposition- based learning and chaotic disturbance","volume":"64","author":"li","year":"2015","journal-title":"J. Wuhan Univ."},{"key":"2021011807350541500_ref47","doi-asserted-by":"crossref","first-page":"771","DOI":"10.1016\/j.asoc.2016.09.048","article-title":"Snap-drift cuckoo search: A novel cuckoo search optimization algorithm","volume":"52","author":"Rakhshani","year":"2017","journal-title":"Appl. Soft. Comput."},{"key":"2021011807350541500_ref48","article-title":"Problem solving with reinforcement learning","author":"Rummery","year":"1994"},{"key":"2021011807350541500_ref49","article-title":"Temporal credit assignment in reinforcement","author":"Sutton","year":"1984"},{"key":"2021011807350541500_ref50","first-page":"279","article-title":"Q-learning","volume":"v8","author":"Watkins","year":"1992","journal-title":"Machining"},{"key":"2021011807350541500_ref51","first-page":"3860","article-title":"Chaos-enhanced cuckoo search optimization algorithms for global optimization","volume":"40","author":"Huang","year":"2016","journal-title":"Appl. Soft. Compt."},{"key":"2021011807350541500_ref52","first-page":"566","article-title":"The cuckoo search algorithm based on Gaussian disturbance","volume":"4","author":"Wang","year":"2011","journal-title":"J. Xian Polytech. Univ."},{"key":"2021011807350541500_ref53","doi-asserted-by":"crossref","first-page":"646","DOI":"10.1109\/TEVC.2006.872133","article-title":"Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems","volume":"10","author":"Brest","year":"2017","journal-title":"IEEE Trans. Evol. Comput."},{"key":"2021011807350541500_ref54","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1109\/TEVC.2008.927706","article-title":"Differential evolution algorithm with strategy adaptation for global numerical optimization","volume":"13","author":"Qin","year":"2009","journal-title":"IEEE Trans Evol. Comput"},{"key":"2021011807350541500_ref55","first-page":"69","article-title":"A modified particle swarm optimizer","volume-title":"Proc. IEEE Int Conf Evol Comput","author":"Shi","year":"1998"},{"key":"2021011807350541500_ref56","doi-asserted-by":"crossref","DOI":"10.1109\/SIS.2003.1202264","article-title":"Fitness-distance-ratio based particle swarm optimization","author":"Peram","year":"2003"},{"key":"2021011807350541500_ref57","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1109\/TEVC.2010.2052054","article-title":"Orthogonal learning particle swarm optimization","volume":"15","author":"Zhan","year":"2011","journal-title":"IEEE Trans. Evol. Comput."}],"container-title":["The Computer Journal"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/comjnl\/article-pdf\/64\/1\/108\/35886281\/bxz149.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/comjnl\/article-pdf\/64\/1\/108\/35886281\/bxz149.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,1,18]],"date-time":"2021-01-18T14:01:01Z","timestamp":1610978461000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/comjnl\/article\/64\/1\/108\/5802861"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,11]]},"references-count":57,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,3,11]]},"published-print":{"date-parts":[[2021,1,19]]}},"URL":"https:\/\/doi.org\/10.1093\/comjnl\/bxz149","relation":{},"ISSN":["0010-4620","1460-2067"],"issn-type":[{"value":"0010-4620","type":"print"},{"value":"1460-2067","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,1]]},"published":{"date-parts":[[2020,3,11]]}}}