{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T20:44:40Z","timestamp":1775767480536,"version":"3.50.1"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,1,16]],"date-time":"2024-01-16T00:00:00Z","timestamp":1705363200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,1,16]],"date-time":"2024-01-16T00:00:00Z","timestamp":1705363200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Key Research and Development Project of Guangdong Province","award":["2021B0101420003"],"award-info":[{"award-number":["2021B0101420003"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2024,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Multimodal transportation is a modern way of cargo transportation. With the increasing demand for cargo transportation, higher requirements are being placed on multimodal transportation multi-objective routing optimization. In multimodal transportation multi-objective routing optimization, in response to the limitations of classical algorithms in solving large-scale problems with multiple nodes and modes of transport, the limitations of directed transportation networks in the application, and the uncertainty of transport time, this paper proposes an optimization framework based on multi-objective weighted sum <jats:italic>Q<\/jats:italic>-learning, combined with the proposed undirected multiple-node network, and characterizes the uncertainty of time with a positively skewed distribution. The undirected multiple-node transportation network can better simulate cargo transportation and characterize transfer information, facilitate the modification of origin and destination, and avoid suboptimal solutions due to the manual setting of wrong route directions. The network is combined with weighted sum <jats:italic>Q<\/jats:italic>-learning to solve multimodal transportation multi-objective routing optimization problems faster and better. When modeling the uncertainty of transport time, a positively skewed distribution is used. The three objectives of transport cost, carbon emission cost, and transport time were studied and compared with PSO, GA, AFO, NSGA-II, and MOPSO. The experimental results show that compared with PSO, GA, and AFO using a directed transportation network, the proposed method has a significant improvement in optimization results and running time, and the running time is shortened by 26 times. The proposed method can better solve the boundary of the Pareto front and dominate the partial solutions of NSGA-II and MOPSO. The effect of time uncertainty on the performance of the algorithm is more significant in transport orders with high time weight. With the increase in uncertainty, the reliability of the route decreases. The effectiveness of the proposed method is verified.<\/jats:p>","DOI":"10.1007\/s40747-023-01308-9","type":"journal-article","created":{"date-parts":[[2024,1,16]],"date-time":"2024-01-16T08:04:50Z","timestamp":1705392290000},"page":"3133-3152","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":41,"title":["Multimodal transportation routing optimization based on multi-objective Q-learning under time uncertainty"],"prefix":"10.1007","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9716-3970","authenticated-orcid":false,"given":"Tie","family":"Zhang","sequence":"first","affiliation":[]},{"given":"Jia","family":"Cheng","sequence":"additional","affiliation":[]},{"given":"Yanbiao","family":"Zou","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,1,16]]},"reference":[{"issue":"1","key":"1308_CR1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.tra.2003.06.001","volume":"38","author":"YM Bontekoning","year":"2004","unstructured":"Bontekoning YM, Macharis C, Trip JJ (2004) Is a new applied transportation research field emerging? A review of intermodal rail\u2013truck freight transport literature. Transp Res Part A Policy Pract 38(1):1\u201334","journal-title":"Transp Res Part A Policy Pract"},{"issue":"2","key":"1308_CR2","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1016\/S0377-2217(03)00161-9","volume":"153","author":"C Macharis","year":"2004","unstructured":"Macharis C, Bontekoning YM (2004) Opportunities for OR in intermodal freight transport research: a review. Eur J Oper Res 153(2):400\u2013416","journal-title":"Eur J Oper Res"},{"key":"1308_CR3","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.jfoodeng.2015.11.014","volume":"174","author":"M Bortolini","year":"2016","unstructured":"Bortolini M, Faccio M, Ferrari E et al (2016) Fresh food sustainable distribution: cost, delivery time and carbon footprint three-objective optimization. J Food Eng 174:56\u201367","journal-title":"J Food Eng"},{"issue":"3","key":"1308_CR4","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1057\/jors.2009.102","volume":"61","author":"J Bauer","year":"2010","unstructured":"Bauer J, Bektas T, Crainic TG (2010) Minimizing greenhouse gas emissions in intermodal freight transport: an application to rail service design. J Oper Res Soc 61(3):530\u2013542","journal-title":"J Oper Res Soc"},{"key":"1308_CR5","first-page":"1","volume":"2022","author":"CJ Zheng","year":"2022","unstructured":"Zheng CJ, Sun K, Gu YH et al (2022) Multimodal transport path selection of cold chain logistics based on improved particle swarm optimization algorithm. J Adv Transp 2022:1","journal-title":"J Adv Transp"},{"key":"1308_CR6","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.tre.2015.08.006","volume":"83","author":"HG Resat","year":"2015","unstructured":"Resat HG, Turkay M (2015) Design and operation of intermodal transportation network in the Marmara region of Turkey. Transp Res E Log 83:16\u201333","journal-title":"Transp Res E Log"},{"key":"1308_CR7","first-page":"1","volume":"2021","author":"H Zhang","year":"2021","unstructured":"Zhang H, Li Y, Zhang QP et al (2021) Route selection of multimodal transport based on China railway transportation. J Adv Transp 2021:1","journal-title":"J Adv Transp"},{"key":"1308_CR8","first-page":"248","volume":"2020","author":"J Jiang","year":"2020","unstructured":"Jiang J, Zhang D, Meng Q et al (2020) Regional multimodal logistics network design considering demand uncertainty and CO2 emission reduction target: a system-optimization approach. J Clean Prod 2020:248","journal-title":"J Clean Prod"},{"key":"1308_CR9","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1016\/j.cie.2018.03.041","volume":"119","author":"S Fazayeli","year":"2018","unstructured":"Fazayeli S, Eydi A, Kamalabadi IN (2018) Location-routing problem in multimodal transportation network with time windows and fuzzy demands: presenting a two-part genetic algorithm. Comput Ind Eng 119:233\u2013246","journal-title":"Comput Ind Eng"},{"issue":"19","key":"1308_CR10","first-page":"1","volume":"10","author":"H Liu","year":"2022","unstructured":"Liu H, Song G, Liu T et al (2022) Multitask emergency logistics planning under multimodal transportation. Mathematics 10(19):1","journal-title":"Mathematics"},{"issue":"3","key":"1308_CR11","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1504\/IJMIC.2013.052821","volume":"18","author":"D Xu","year":"2013","unstructured":"Xu D, Wenfeng L, Lanbo Z (2013) Ant colony optimisation for a resource-constrained shortest path problem with applications in multimodal transport. Int J Model Ident Control 18(3):268\u2013275","journal-title":"Int J Model Ident Control"},{"issue":"5","key":"1308_CR12","doi-asserted-by":"crossref","first-page":"4751","DOI":"10.1109\/TVT.2020.2979623","volume":"69","author":"Q Zhang","year":"2020","unstructured":"Zhang Q, Wu K, Shi Y (2020) Route planning and power management for PHEVs with reinforcement learning. IEEE Trans Veh Technol 69(5):4751\u20134762","journal-title":"IEEE Trans Veh Technol"},{"issue":"10","key":"1308_CR13","doi-asserted-by":"crossref","first-page":"11107","DOI":"10.1109\/TCYB.2021.3089179","volume":"52","author":"Y Xu","year":"2022","unstructured":"Xu Y, Fang M, Chen L et al (2022) Reinforcement learning with multiple relational attention for solving vehicle routing problems. IEEE Trans Cybern 52(10):11107\u201311120","journal-title":"IEEE Trans Cybern"},{"key":"1308_CR14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.trc.2022.103611","volume":"138","author":"S Feng","year":"2022","unstructured":"Feng S, Duan P, Ke J et al (2022) Coordinating ride-sourcing and public transport services with a reinforcement learning approach. Transp Res Part C Emerg Technol 138:1","journal-title":"Transp Res Part C Emerg Technol"},{"issue":"6","key":"1308_CR15","first-page":"1","volume":"39","author":"R Hu","year":"2020","unstructured":"Hu R, Xu J, Chen B et al (2020) TAP-net: transport-and-pack using reinforcement learning. ACM Trans Graph 39(6):1","journal-title":"ACM Trans Graph"},{"issue":"3","key":"1308_CR16","first-page":"279","volume":"8","author":"CJCH Watkins","year":"1992","unstructured":"Watkins CJCH, Dayan P (1992) Technical note: Q-learning. Mach Learn 8(3):279\u2013292","journal-title":"Mach Learn"},{"key":"1308_CR17","doi-asserted-by":"crossref","unstructured":"Jaakkola T, Jordan MI, Singh SP (1993) Convergence of stochastic iterative dynamic programming algorithms. In: Proceedings of the 6th international conference on neural information processing systems, pp 703\u2013710","DOI":"10.21236\/ADA276517"},{"issue":"3","key":"1308_CR18","first-page":"185","volume":"16","author":"JN Tsitsiklis","year":"1994","unstructured":"Tsitsiklis JN (1994) Asynchronous stochastic approximation and Q-learning. Mach Learn 16(3):185\u2013202","journal-title":"Mach Learn"},{"key":"1308_CR19","doi-asserted-by":"crossref","unstructured":"Baird L (1995) Residual algorithms: reinforcement learning with function approximation. Machine learning. In: Proceedings of the 12th international conference on machine learning, pp 30\u201337","DOI":"10.1016\/B978-1-55860-377-6.50013-X"},{"issue":"3","key":"1308_CR20","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1109\/TSMC.2014.2358639","volume":"45","author":"C Liu","year":"2015","unstructured":"Liu C, Xu X, Hu D (2015) Multiobjective reinforcement learning: a comprehensive overview. IEEE Trans Syst Man Cybern Syst 45(3):385\u2013398","journal-title":"IEEE Trans Syst Man Cybern Syst"},{"issue":"1","key":"1308_CR21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10458-021-09530-w","volume":"36","author":"CF Hayes","year":"2022","unstructured":"Hayes CF, Radulescu R, Bargiacchi E et al (2022) A practical guide to multi-objective reinforcement learning and planning. Autonomous Agents Multiagent Syst 36(1):1","journal-title":"Autonomous Agents Multiagent Syst"},{"issue":"2","key":"1308_CR22","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1109\/TITS.2011.2106158","volume":"12","author":"DCK Ngai","year":"2011","unstructured":"Ngai DCK, Yung NHC (2011) A multiple-goal reinforcement learning method for complex vehicle overtaking maneuvers. IEEE Trans Intell Transp Syst 12(2):509\u2013522","journal-title":"IEEE Trans Intell Transp Syst"},{"key":"1308_CR23","unstructured":"Zhao Y, Chen Q, Hu W et al (2010) Multi-objective reinforcement learning algorithm for MOSDMP in unknown environment. In: 8th world congress on intelligent control and automation (WCICA), pp 3190\u20133194"},{"issue":"1\u20132","key":"1308_CR24","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1007\/s10994-010-5232-5","volume":"84","author":"P Vamplew","year":"2011","unstructured":"Vamplew P, Dazeley R, Berry A et al (2011) Empirical evaluation methods for multiobjective reinforcement learning algorithms. Mach Learn 84(1\u20132):51\u201380","journal-title":"Mach Learn"},{"key":"1308_CR25","unstructured":"Zeng F, Zong Q, Sun Z et al (2010) Self-adaptive multi-objective optimization method design based on agent reinforcement learning for elevator group control systems. In: 8th world congress on intelligent control and automation (WCICA), pp 2577\u20132582"},{"issue":"15","key":"1308_CR26","first-page":"1","volume":"13","author":"X Zhang","year":"2021","unstructured":"Zhang X, Jin F-Y, Yuan X-M et al (2021) Low-carbon multimodal transportation path optimization under dual uncertainty of demand and time. Sustainability 13(15):1","journal-title":"Sustainability"},{"issue":"7","key":"1308_CR27","doi-asserted-by":"crossref","first-page":"2119","DOI":"10.1007\/s40815-020-00905-x","volume":"22","author":"Y Sun","year":"2020","unstructured":"Sun Y (2020) Fuzzy approaches and simulation-based reliability modeling to solve a road\u2013rail intermodal routing problem with soft delivery time windows when demand and capacity are uncertain. Int J Fuzzy Syst 22(7):2119\u20132148","journal-title":"Int J Fuzzy Syst"},{"issue":"1\u20132","key":"1308_CR28","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1016\/j.apm.2012.02.032","volume":"37","author":"M Ramezani","year":"2013","unstructured":"Ramezani M, Bashiri M, Tavakkoli-Moghaddam R (2013) A new multi-objective stochastic model for a forward\/reverse logistic network design with responsiveness and quality level. Appl Math Model 37(1\u20132):328\u2013344","journal-title":"Appl Math Model"},{"key":"1308_CR29","doi-asserted-by":"crossref","first-page":"789","DOI":"10.1016\/j.trb.2015.09.007","volume":"93","author":"E Demir","year":"2016","unstructured":"Demir E, Burgholzer W, Hrusovsky M et al (2016) A green intermodal service network design problem with travel time uncertainty. Transp Res Part B Methodol 93:789\u2013807","journal-title":"Transp Res Part B Methodol"},{"issue":"5","key":"1308_CR30","doi-asserted-by":"crossref","first-page":"751","DOI":"10.1016\/j.trc.2010.09.007","volume":"19","author":"A Juan","year":"2011","unstructured":"Juan A, Faulin J, Grasman S et al (2011) Using safety stocks and simulation to solve the vehicle routing problem with stochastic demands. Transp Res Part C Emerg Technol 19(5):751\u2013765","journal-title":"Transp Res Part C Emerg Technol"},{"key":"1308_CR31","doi-asserted-by":"crossref","first-page":"S3035","DOI":"10.1051\/ro\/2020110","volume":"55","author":"Y Peng","year":"2021","unstructured":"Peng Y, Yong P, Luo Y (2021) The route problem of multimodal transportation with timetable under uncertainty: multi-objective robust optimization model and heuristic approach. Rairo Oper Res 55:S3035\u2013S3050","journal-title":"Rairo Oper Res"},{"issue":"8","key":"1308_CR32","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1080\/03081060.2019.1675316","volume":"42","author":"A Baykasoglu","year":"2019","unstructured":"Baykasoglu A, Subulan K (2019) A fuzzy-stochastic optimization model for the intermodal fleet management problem of an international transportation company. Transp Plan Technol 42(8):777\u2013824","journal-title":"Transp Plan Technol"},{"key":"1308_CR33","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1016\/j.ijpe.2017.09.009","volume":"195","author":"A Haddadsisakht","year":"2018","unstructured":"Haddadsisakht A, Ryan SM (2018) Closed-loop supply chain network design with multiple transportation modes under stochastic demand and uncertain carbon tax. Int J Prod Econ 195:118\u2013131","journal-title":"Int J Prod Econ"},{"issue":"1","key":"1308_CR34","doi-asserted-by":"crossref","first-page":"91","DOI":"10.3390\/sym11010091","volume":"11","author":"Y Sun","year":"2019","unstructured":"Sun Y, Liang X, Li X et al (2019) A fuzzy programming method for modeling demand uncertainty in the capacitated road-rail multimodal routing problem with time windows. Symmetry 11(1):91","journal-title":"Symmetry"},{"key":"1308_CR35","doi-asserted-by":"crossref","unstructured":"Farahani A, Genga L, Dijkman R et al (2021) Online multimodal transportation planning using deep reinforcement learning. In: IEEE international conference on systems, man, and cybernetics (SMC), pp 1691\u20131698","DOI":"10.1109\/SMC52423.2021.9658943"},{"issue":"9","key":"1308_CR36","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.1016\/0362-546X(89)90096-5","volume":"13","author":"EN Barron","year":"1989","unstructured":"Barron EN, Ishii H (1989) The Bellman equation for minimizing the maximum cost. Nonlinear Anal Theory Methods Appl 13(9):1067\u20131090","journal-title":"Nonlinear Anal Theory Methods Appl"},{"key":"1308_CR37","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1613\/jair.3987","volume":"48","author":"DM Roijers","year":"2013","unstructured":"Roijers DM, Vamplew P, Whiteson S et al (2013) A survey of multi-objective sequential decision-making. J Artif Intell Res 48:67\u2013113","journal-title":"J Artif Intell Res"},{"key":"1308_CR38","doi-asserted-by":"crossref","first-page":"57969","DOI":"10.1109\/ACCESS.2018.2874184","volume":"6","author":"B Cao","year":"2018","unstructured":"Cao B, Sun K, Li T et al (2018) Trajectory modified in joint space for vibration suppression of manipulator. IEEE Access 6:57969\u201357980","journal-title":"IEEE Access"},{"issue":"1\u20132","key":"1308_CR39","doi-asserted-by":"crossref","first-page":"1253","DOI":"10.1007\/s00170-022-08796-y","volume":"120","author":"Y Yang","year":"2022","unstructured":"Yang Y, Xu H-Z, Li S-H et al (2022) Time-optimal trajectory optimization of serial robotic manipulator with kinematic and dynamic limits based on improved particle swarm optimization. Int J Adv Manuf Technol 120(1\u20132):1253\u20131264","journal-title":"Int J Adv Manuf Technol"},{"issue":"3","key":"1308_CR40","doi-asserted-by":"crossref","first-page":"1813","DOI":"10.3233\/JIFS-211214","volume":"42","author":"L Zhai","year":"2022","unstructured":"Zhai L, Feng S (2022) A novel evacuation path planning method based on improved genetic algorithm. J Intell Fuzzy Syst 42(3):1813\u20131823","journal-title":"J Intell Fuzzy Syst"},{"key":"1308_CR41","first-page":"232","volume":"2021","author":"Z Yang","year":"2021","unstructured":"Yang Z, Deng L, Wang Y et al (2021) Aptenodytes Forsteri optimization: algorithm and applications. Knowl Based Syst 2021:232","journal-title":"Knowl Based Syst"},{"key":"1308_CR42","doi-asserted-by":"crossref","first-page":"44862","DOI":"10.1109\/ACCESS.2019.2903910","volume":"7","author":"AF Zobaa","year":"2019","unstructured":"Zobaa AF (2019) Mixed-integer distributed ant colony multi-objective optimization of single-tuned passive harmonic filter parameters. IEEE Access 7:44862\u201344870","journal-title":"IEEE Access"},{"key":"1308_CR43","doi-asserted-by":"crossref","first-page":"2138","DOI":"10.1109\/ACCESS.2018.2886245","volume":"7","author":"S Thabit","year":"2019","unstructured":"Thabit S, Mohades A (2019) Multi-robot path planning based on multi-objective particle swarm optimization. IEEE Access 7:2138\u20132147","journal-title":"IEEE Access"},{"issue":"8","key":"1308_CR44","doi-asserted-by":"crossref","first-page":"8326","DOI":"10.1109\/TCYB.2021.3049712","volume":"52","author":"Z Wang","year":"2022","unstructured":"Wang Z, Zhen H-L, Deng J et al (2022) Multiobjective optimization-aided decision-making system for large-scale manufacturing planning. IEEE Trans Cybern 52(8):8326\u20138339","journal-title":"IEEE Trans Cybern"},{"key":"1308_CR45","first-page":"12518","volume":"37","author":"R Zheng","year":"2023","unstructured":"Zheng R, Wang Z (2023) A generalized scalarization method for evolutionary multi-objective optimization. Proc AAAI Conf Artif Intell 37:12518\u201312525","journal-title":"Proc AAAI Conf Artif Intell"},{"issue":"2","key":"1308_CR46","doi-asserted-by":"crossref","first-page":"474","DOI":"10.1109\/TCYB.2015.2403849","volume":"46","author":"Z Wang","year":"2016","unstructured":"Wang Z, Zhang Q, Zhou A et al (2016) Adaptive replacement strategies for MOEA\/D. IEEE Trans Cybern 46(2):474\u2013486","journal-title":"IEEE Trans Cybern"},{"issue":"6","key":"1308_CR47","doi-asserted-by":"crossref","first-page":"3103","DOI":"10.1109\/TCYB.2020.2977661","volume":"51","author":"K Li","year":"2021","unstructured":"Li K, Zhang T, Wang R (2021) Deep reinforcement learning for multiobjective optimization. IEEE Trans Cybern 51(6):3103\u20133114","journal-title":"IEEE Trans Cybern"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01308-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-023-01308-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01308-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,30]],"date-time":"2024-03-30T15:39:15Z","timestamp":1711813155000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-023-01308-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,16]]},"references-count":47,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,4]]}},"alternative-id":["1308"],"URL":"https:\/\/doi.org\/10.1007\/s40747-023-01308-9","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,16]]},"assertion":[{"value":"28 June 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 December 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 January 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}