{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T01:17:23Z","timestamp":1773883043498,"version":"3.50.1"},"reference-count":75,"publisher":"EDP Sciences","issue":"5","license":[{"start":{"date-parts":[[2020,7,23]],"date-time":"2020-07-23T00:00:00Z","timestamp":1595462400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.edpsciences.org\/en\/authors\/copyright-and-licensing"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["71471092"],"award-info":[{"award-number":["71471092"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004731","name":"Zhejiang Natural Science Foundation","doi-asserted-by":"crossref","award":["LR17G010001"],"award-info":[{"award-number":["LR17G010001"]}],"id":[{"id":"10.13039\/501100004731","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Ningbo Science & Technology Bureau","award":["2014A35006"],"award-info":[{"award-number":["2014A35006"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["RAIRO-Oper. Res."],"accepted":{"date-parts":[[2019,8,14]]},"published-print":{"date-parts":[[2020,9]]},"abstract":"<jats:p>This paper studies a real-life container transportation problem with a wide planning horizon divided into multiple shifts. The trucks in this problem do not return to depot after every single shift but at the end of every two shifts. The mathematical model of the problem is first established, but it is unrealistic to solve this large scale problem with exact search methods. Thus, a Variable Neighbourhood Search algorithm with Reinforcement Learning (VNS-RLS) is thus developed. An urgency level-based insertion heuristic is proposed to construct the initial solution. Reinforcement learning is then used to guide the search in the local search improvement phase. Our study shows that the Sampling scheme in single solution-based algorithms does not significantly improve the solution quality but can greatly reduce the rate of infeasible solutions explored during the search. Compared to the exact search and the state-of-the-art algorithms, the proposed VNS-RLS produces promising results.<\/jats:p>","DOI":"10.1051\/ro\/2019080","type":"journal-article","created":{"date-parts":[[2019,8,29]],"date-time":"2019-08-29T18:51:40Z","timestamp":1567104700000},"page":"1467-1494","source":"Crossref","is-referenced-by-count":40,"title":["A variable neighborhood search algorithm with reinforcement learning for a real-life periodic vehicle routing problem with time windows and open routes"],"prefix":"10.1051","volume":"54","author":[{"given":"Binhui","family":"Chen","sequence":"first","affiliation":[]},{"given":"Rong","family":"Qu","sequence":"additional","affiliation":[]},{"given":"Ruibin","family":"Bai","sequence":"additional","affiliation":[]},{"given":"Wasakorn","family":"Laesanklang","sequence":"additional","affiliation":[]}],"member":"250","published-online":{"date-parts":[[2020,7,23]]},"reference":[{"key":"R1","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1016\/j.trb.2015.06.002","volume":"79","author":"Bai","year":"2015","journal-title":"Transp. Res. Part B: Methodol."},{"key":"R2","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1016\/S0305-0548(02)00051-5","volume":"30","author":"Baker","year":"2003","journal-title":"Comput. Oper. Res."},{"key":"R3","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1016\/S0377-2217(03)00238-8","volume":"157","author":"Brand\u00e3o","year":"2004","journal-title":"Eur. J. Oper. Res."},{"key":"R4","unstructured":"Br\u00e4ysy O. and Gendreau M., Metaheuristics for the vehicle routing problem with time windows. Report STF42 A1025 (2001)."},{"key":"R5","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1287\/trsc.1030.0057","volume":"39","author":"Br\u00e4ysy","year":"2005","journal-title":"Transp. Sci."},{"key":"R6","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.asoc.2015.03.026","volume":"32","author":"Brito","year":"2015","journal-title":"Appl. Soft Comput."},{"key":"R7","doi-asserted-by":"crossref","unstructured":"Burke E.K., Gendreau M., Ochoa G. and Walker J.D., Adaptive iterated local search for cross-domain optimisation. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation. ACM (2011) 1987\u20131994.","DOI":"10.1145\/2001576.2001843"},{"key":"R8","unstructured":"Chen J., Bai R., Qu R. and Kendall G., A task based approach for a real-world commodity routing problem. In: 2013 IEEE Workshop on Computational Intelligence in Production And Logistics Systems (CIPLS). IEEE (2013) 1\u20138."},{"key":"R9","doi-asserted-by":"crossref","unstructured":"Chen B., Qu R., Bai R. and Ishibuchi H., A variable neighbourhood search algorithm with compound neighbourhoods for VRPTW. In: Proceedings of the 5th International Conference on Operations Research and Enterprise Systems (ICORES 2016), Rome, Italy. SCITEPRESS (2016) 25\u201335.","DOI":"10.5220\/0005661800250035"},{"key":"R10","unstructured":"Chen B., Qu R. and Ishibuchi H., Variable-depth adaptive large meighbourhood search algorithm for open periodic vehicle routing problem with time windows. In: Proceedings of the International Conference on Harbor, Maritime and Multimodal Logistic Modelling and Simulation (HMS 2017), Barcelona, Spain (2017) 25\u201334."},{"key":"R11","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1287\/opre.12.4.568","volume":"12","author":"Clarke","year":"1964","journal-title":"Oper. Res."},{"key":"R12","doi-asserted-by":"crossref","first-page":"928","DOI":"10.1057\/palgrave.jors.2601163","volume":"52","author":"Cordeau","year":"2001","journal-title":"J. Oper. Res. Soc."},{"key":"R13","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1057\/palgrave.jors.2601707","volume":"55","author":"Cordeau","year":"2004","journal-title":"J. Oper. Res. Soc."},{"key":"R14","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/S0927-0507(06)14006-2","volume":"14","author":"Cordeau","year":"2007","journal-title":"Handbooks Oper. Res. Manage. Sci."},{"key":"R15","unstructured":"Danandeh A., Ghazanfari M., Tavakoli-Moghaddam R. and Alinaghian M., A swift heuristic algorithm based on capacitated clustering for the open periodic vehicle routing problem. In: Proceedings of the 9th WSEAS International Conference on Artificial intelligence, Knowledge Engineering and Data Bases, World Scientific and Engineering Academy and Society (WSEAS), Stevens Point, Wisconsin, USA (2010) 208\u2013214."},{"key":"R16","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1287\/mnsc.6.1.80","volume":"6","author":"Dantzig","year":"1959","journal-title":"Manage. Sci."},{"key":"R17","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1006\/jcph.1993.1010","volume":"104","author":"Dueck","year":"1993","journal-title":"J. Comput. Phys."},{"key":"R18","doi-asserted-by":"crossref","first-page":"1472","DOI":"10.1016\/j.cie.2009.05.009","volume":"57","author":"Eksioglu","year":"2009","journal-title":"Comput. Ind. Eng."},{"key":"R19","unstructured":"Eppen G. and Schrage L., Centralized ordering policies in a multi-warehouse system with lead times and random demand. Multi-Level Prod.\/Inventory Control Syst.: Theory Pract. In Vol. 16. North-Holland (1981) 51\u201367."},{"key":"R20","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1057\/palgrave.jors.2601817","volume":"56","author":"Fu","year":"2005","journal-title":"J. Oper. Res. Soc."},{"key":"R21","unstructured":"Gehring H. and Homberger J., A parallel hybrid evolutionary metaheuristic for the vehicle routing problem with time windows. In: Proceedings of EUROGEN99. Citeseer (1999) 57\u201364."},{"key":"R22","doi-asserted-by":"crossref","unstructured":"Gendreau M., Potvin J.-Y., Br\u00e4umlaysy O., Hasle G. and L\u00f8kketangen A., Metaheuristics for the vehicle routing problem and its extensions: a categorized bibliography. In: The Vehicle Routing Problem: Latest Advances and New Challenges. Springer, Boston, MA (2008) 143\u2013169.","DOI":"10.1007\/978-0-387-77778-8_7"},{"key":"R23","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1287\/opre.22.2.340","volume":"22","author":"Gillett","year":"1974","journal-title":"Oper. Res."},{"key":"R24","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/0305-0548(84)90007-8","volume":"11","author":"Golden","year":"1984","journal-title":"Comput. Oper. Res."},{"key":"R25","doi-asserted-by":"crossref","unstructured":"Golden B.L., Raghavan S. and Wasil E.A., The Vehicle Routing Problem: Latest Advances and New Challenges. In: Vol. 43. Springer Science & Business Media (2008).","DOI":"10.1007\/978-0-387-77778-8"},{"key":"R26","unstructured":"Guiyun L., An improved ant colony algorithm for open vehicle routing problem with time windows. In: Vol. 2 of 2009 International Conference on Information Management, Innovation Management and Industrial Engineering. IEEE (2009) 616\u2013619."},{"key":"R27","unstructured":"Guiyun L., Research on open vehicle routing problem with time windows based on improved genetic algorithm. In: International Conference on Computational Intelligence and Software Engineering, 2009. CiSE 2009. IEEE (2009) 1\u20135."},{"key":"R28","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1007\/s10479-009-0657-6","volume":"175","author":"Hansen","year":"2010","journal-title":"Ann. Oper. Res."},{"key":"R29","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1016\/j.ejor.2007.08.048","volume":"195","author":"Hemmelmayr","year":"2009","journal-title":"Eur. J. Oper. Res."},{"key":"R30","doi-asserted-by":"crossref","first-page":"3215","DOI":"10.1016\/j.cor.2012.04.007","volume":"39","author":"Hemmelmayr","year":"2012","journal-title":"Comput. Oper. Res."},{"key":"R31","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1057\/palgrave.jors.2601113","volume":"52","author":"Ioannou","year":"2001","journal-title":"J. Oper. Res. Soc."},{"key":"R32","first-page":"3","volume":"9","author":"Jin","year":"2005","journal-title":"Soft Comput. Fusion Found. Methodol. App."},{"key":"R33","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1016\/j.ejor.2007.05.055","volume":"189","author":"Jozefowiez","year":"2008","journal-title":"Eur. J. Oper. Res."},{"key":"R34","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/S0377-2217(97)00381-0","volume":"112","author":"Kaji","year":"1999","journal-title":"Eur. J. Oper. Res."},{"key":"R35","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1111\/j.1475-3995.2000.tb00200.x","volume":"7","author":"Laporte","year":"2000","journal-title":"Int. Trans. Oper. Res."},{"key":"R36","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1002\/net.3230110211","volume":"11","author":"Lenstra","year":"1981","journal-title":"Networks"},{"key":"R37","doi-asserted-by":"crossref","first-page":"1642","DOI":"10.1057\/palgrave.jors.2602345","volume":"58","author":"Letchford","year":"2007","journal-title":"J. Oper. Res. Soc."},{"key":"R38","doi-asserted-by":"crossref","first-page":"2918","DOI":"10.1016\/j.cor.2005.11.018","volume":"34","author":"Li","year":"2007","journal-title":"Comput. Oper. Res."},{"key":"R39","doi-asserted-by":"crossref","first-page":"2245","DOI":"10.1002\/j.1538-7305.1965.tb04146.x","volume":"44","author":"Lin","year":"1965","journal-title":"Bell Syst. Tech. J."},{"key":"R40","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1016\/j.ins.2015.12.022","volume":"339","author":"Lin","year":"2016","journal-title":"Inf. Sci."},{"key":"R41","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1016\/j.ejor.2012.01.061","volume":"220","author":"Liu","year":"2012","journal-title":"Eur. J. Oper. Res."},{"key":"R42","unstructured":"Lourens T., Using population-based incremental learning to optimize feasible distribution logistic solutions. Thesis, University of Stellenbosch, Stellenbosch (2005)."},{"key":"R43","unstructured":"G. Maps, Google maps. Accessed: 2018-05-11. https:\/\/www.google.co.uk\/maps\/@29.8715435,121.8372319,12z\/data=!3m1!4b1!4m2!6m1!1s1IPQurvRAx3x96-V7XEUw6h9kmFs (2018)."},{"key":"R44","doi-asserted-by":"crossref","first-page":"1028","DOI":"10.1016\/j.ejor.2006.02.030","volume":"183","author":"Mourgaya","year":"2007","journal-title":"Eur. J. Oper. Res."},{"key":"R45","unstructured":"Or I., Traveling Salesman-type Combinatorial Problems and Their Relation to the Logistics of Regional Blood Banking. Xerox University Microfilms (1976)."},{"key":"R46","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1016\/j.ejor.2009.05.017","volume":"202","author":"Park","year":"2010","journal-title":"Eur. J. Oper. Res."},{"key":"R47","doi-asserted-by":"crossref","unstructured":"Perwira Redi A., Maghfiroh M.F., Yu V.F., An improved variable neighborhood search for the open vehicle routing problem with time windows. In: 2013 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM). IEEE (2013) 1641\u20131645.","DOI":"10.1109\/IEEM.2013.6962688"},{"key":"R48","unstructured":"Pirkwieser S. and Raidl G.R., A variable neighborhood search for the periodic vehicle routing problem with time windows. In: Proceedings of the 9th EU\/meeting on Metaheuristics for Logistics and Vehicle Routing, Troyes, France (2008) 23\u201324."},{"key":"R49","doi-asserted-by":"crossref","first-page":"2403","DOI":"10.1016\/j.cor.2005.09.012","volume":"34","author":"Pisinger","year":"2007","journal-title":"Comput. Oper. Res."},{"key":"R50","doi-asserted-by":"crossref","first-page":"1433","DOI":"10.1057\/jors.1995.204","volume":"46","author":"Potvin","year":"1995","journal-title":"J. Oper. Res. Soc."},{"key":"R51","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1287\/ijoc.8.2.158","volume":"8","author":"Potvin","year":"1996","journal-title":"INFORMS J. Comput."},{"key":"R52","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.cor.2014.07.004","volume":"53","author":"Rahimi-Vahed","year":"2015","journal-title":"Comput. Oper. Res."},{"key":"R53","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1057\/palgrave.jors.2602143","volume":"58","author":"Repoussis","year":"2007","journal-title":"J. Oper. Res. Soc."},{"key":"R54","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1287\/trsc.1050.0135","volume":"40","author":"Ropke","year":"2006","journal-title":"Transp. Sci."},{"key":"R55","doi-asserted-by":"crossref","first-page":"750","DOI":"10.1016\/j.ejor.2004.09.004","volume":"171","author":"Ropke","year":"2006","journal-title":"Eur. J. Oper. Res."},{"key":"R56","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1287\/ijoc.4.2.146","volume":"4","author":"Savelsbergh","year":"1992","journal-title":"ORSA J. Comput."},{"key":"R57","doi-asserted-by":"crossref","unstructured":"Schopka K. and Kopfer H., An Adaptive Large Neighborhood Search for the Reverse Open Vehicle Routing Problem with Time Windows. Springer (2016) 243\u2013257.","DOI":"10.1007\/978-3-319-20863-3_18"},{"key":"R58","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1007\/s005000050009","volume":"1","author":"Smith","year":"1997","journal-title":"Soft Comput."},{"key":"R59","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1287\/opre.35.2.254","volume":"35","author":"Solomon","year":"1987","journal-title":"Oper. Res."},{"key":"R60","doi-asserted-by":"crossref","unstructured":"Soria Alcaraz J.A., Ochoa G., Carpio M. and Puga H., Evolvability metrics in adaptive operator selection. In: Proceedings of the 2014 Conference on Genetic and Evolutionary Computation. ACM (2014) 1327\u20131334.","DOI":"10.1145\/2576768.2598220"},{"key":"R61","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1287\/trsc.31.2.170","volume":"31","author":"Taillard","year":"1997","journal-title":"Transp. Sci."},{"key":"R62","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1051\/ro:2004029","volume":"38","author":"Tarantilis","year":"2004","journal-title":"RAIRO: OR"},{"key":"R63","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1057\/palgrave.jors.2601848","volume":"56","author":"Tarantilis","year":"2005","journal-title":"J. Oper. Res. Soc."},{"key":"R64","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1109\/TSMCB.2002.1049606","volume":"32","author":"Thathachar","year":"2002","journal-title":"IEEE Trans. Syst. Man Cybern. Part B: Cybern."},{"key":"R65","unstructured":"Thompson P.M. and Orlin J.B., The theory of cyclic transfers (1989)."},{"key":"R66","unstructured":"Thierens D., An adaptive pursuit strategy for allocating operator probabilities. In: Proceedings of the 7th Annual Conference on Genetic and evolutionary Computation. ACM (2005) 1539\u20131546."},{"key":"R67","doi-asserted-by":"crossref","unstructured":"Toth P. and Vigo D., The Vehicle Routing Problem. SIAM (2001).","DOI":"10.1137\/1.9780898718515"},{"key":"R68","doi-asserted-by":"crossref","unstructured":"Veerapen N., Maturana J. and Saubion F., An exploration-exploitation compromise-based adaptive operator selection for local search. In: Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation. ACM (2012) 1277\u20131284.","DOI":"10.1145\/2330163.2330340"},{"key":"R69","doi-asserted-by":"crossref","first-page":"658","DOI":"10.1016\/j.ejor.2013.09.045","volume":"234","author":"Vidal","year":"2014","journal-title":"Eur. J. Oper. Res."},{"key":"R70","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/0377-2217(96)00223-8","volume":"89","author":"Vigo","year":"1996","journal-title":"Eur. J. Oper. Res."},{"key":"R71","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/S0965-8564(00)00037-9","volume":"36","author":"Wang","year":"2002","journal-title":"Transp. Res. Part B: Methodol."},{"key":"R72","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1007\/s00291-007-0079-2","volume":"30","author":"Wieberneit","year":"2008","journal-title":"OR Spect."},{"key":"R73","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1016\/j.tre.2010.09.010","volume":"47","author":"Yu","year":"2011","journal-title":"Transp. Res. Part E: Logistics Transp. Rev."},{"key":"R74","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1016\/j.cor.2009.06.021","volume":"37","author":"Zachariadis","year":"2010","journal-title":"Comput. Oper. Res."},{"key":"R75","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1007\/s00291-010-0193-4","volume":"32","author":"Zhang","year":"2010","journal-title":"OR Spect."}],"container-title":["RAIRO - Operations Research"],"original-title":[],"link":[{"URL":"https:\/\/www.rairo-ro.org\/10.1051\/ro\/2019080\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,1,17]],"date-time":"2021-01-17T18:50:42Z","timestamp":1610909442000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.rairo-ro.org\/10.1051\/ro\/2019080"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,23]]},"references-count":75,"journal-issue":{"issue":"5"},"alternative-id":["ro180131"],"URL":"https:\/\/doi.org\/10.1051\/ro\/2019080","relation":{},"ISSN":["0399-0559","1290-3868"],"issn-type":[{"value":"0399-0559","type":"print"},{"value":"1290-3868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,23]]}}}