{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T07:46:31Z","timestamp":1781077591897,"version":"3.54.1"},"reference-count":45,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2024,7,20]],"date-time":"2024-07-20T00:00:00Z","timestamp":1721433600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research &amp; Development Program of China","award":["2022YFD2401301"],"award-info":[{"award-number":["2022YFD2401301"]}]},{"name":"National Key Research &amp; Development Program of China","award":["42367054"],"award-info":[{"award-number":["42367054"]}]},{"name":"National Natural Science Foundation of China","award":["2022YFD2401301"],"award-info":[{"award-number":["2022YFD2401301"]}]},{"name":"National Natural Science Foundation of China","award":["42367054"],"award-info":[{"award-number":["42367054"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Robotic Mobile Fulfillment Systems (RMFSs) face challenges in handling large-scale orders and navigating complex environments, frequently encountering a series of intricate decision-making problems, such as order allocation, shelf selection, and robot scheduling. To address these challenges, this paper integrates Deep Reinforcement Learning (DRL) technology into an RMFS, to meet the needs of efficient order processing and system stability. This study focuses on three key stages of RMFSs: order allocation and sorting, shelf selection, and coordinated robot scheduling. For each stage, mathematical models are established and the corresponding solutions are proposed. Unlike traditional methods, DRL technology is introduced to solve these problems, utilizing a Genetic Algorithm and Ant Colony Optimization to handle decision making related to large-scale orders. Through simulation experiments, performance indicators\u2014such as shelf access frequency and the total processing time of the RMFS\u2014are evaluated. The experimental results demonstrate that, compared to traditional methods, our algorithms excel in handling large-scale orders, showcasing exceptional superiority, capable of completing approximately 110 tasks within an hour. Future research should focus on integrated decision-making modeling for each stage of RMFSs and designing efficient heuristic algorithms for large-scale problems, to further enhance system performance and efficiency.<\/jats:p>","DOI":"10.3390\/s24144713","type":"journal-article","created":{"date-parts":[[2024,7,22]],"date-time":"2024-07-22T14:45:53Z","timestamp":1721659553000},"page":"4713","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Optimizing Robotic Mobile Fulfillment Systems for Order Picking Based on Deep Reinforcement Learning"],"prefix":"10.3390","volume":"24","author":[{"given":"Zhenyi","family":"Zhu","sequence":"first","affiliation":[{"name":"School of Science, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3354-9089","authenticated-orcid":false,"given":"Sai","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Marine Resource Utilization in South China Sea, Hainan University, Haikou 570228, China"},{"name":"School of Ecology, Hainan University, Haikou 570228, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tuantuan","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Ecology, Hainan University, Haikou 570228, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Dai, W., Mou, C., Wu, J., and Ye, X. (2023, January 21\u201324). Diabetic retinopathy detection with enhanced vision transformers: The twins-pcpvt solution. Proceedings of the 2023 IEEE 3rd International Conference on Electronic Technology, Communication and Information (ICETCI), Qingdao, China.","DOI":"10.1109\/ICETCI57876.2023.10176810"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2396","DOI":"10.1109\/TCOMM.2023.3241326","article-title":"Joint spatio-temporal precoding for practical non-stationary wireless channels","volume":"71","author":"Zou","year":"2023","journal-title":"IEEE Trans. Commun."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"75742","DOI":"10.1109\/ACCESS.2022.3192026","article-title":"Fine segmentation on faces with masks based on a multistep iterative segmentation algorithm","volume":"10","author":"Zhang","year":"2022","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zheng, J., Li, W., Hong, J., Petersson, L., and Barnes, N. (2022, January 18\u201324). Towards open-set object detection and discovery. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPRW56347.2022.00441"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5617","DOI":"10.1109\/JIOT.2020.3030492","article-title":"Is image encoding beneficial for deep learning in finance?","volume":"9","author":"Wang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_6","first-page":"88","article-title":"The influence of digital transformation in enterprises on the dynamics of supply chain concentration: An empirical analysis of chinese a-share listed companies","volume":"1","author":"Xu","year":"2023","journal-title":"J. Organ. Technol. Entrep"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Fu, H., Liu, J., Dong, X., Chen, Z., and He, M. (2024). Evaluating the sustainable development goals within spatial planning for decision-making: A major function-oriented zone planning strategy in china. Land, 13.","DOI":"10.3390\/land13030390"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhou, L., Wang, M., and Zhou, N. (2024). Distributed federated learning-based deep learning model for privacy mri brain tumor detection. arXiv.","DOI":"10.62836\/jitp.2023.158"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"122262","DOI":"10.1016\/j.eswa.2023.122262","article-title":"Simultaneous allocation and sequencing of orders for robotic mobile fulfillment system using reinforcement learning algorithm","volume":"239","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_10","unstructured":"Zhou, L., Luo, Z., and Pan, X. (2024). Machine learning-based system reliability analysis with gaussian process regression. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Pan, X., Luo, Z., and Zhou, L. (2024). Navigating the landscape of distributed file systems: Architectures, implementations, and considerations. arXiv.","DOI":"10.62836\/iaet.v2i1.157"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1016\/j.apm.2022.06.036","article-title":"Optimization models for scheduling operations in robotic mobile fulfillment systems","volume":"111","author":"Teck","year":"2022","journal-title":"Appl. Math. Model."},{"key":"ref_13","first-page":"100128","article-title":"Decision rules for robotic mobile fulfillment systems","volume":"6","author":"Merschformann","year":"2019","journal-title":"Oper. Res. Perspect."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"102920","DOI":"10.1016\/j.tre.2022.102920","article-title":"Rack retrieval and repositioning optimization problem in robotic mobile fulfillment systems","volume":"167","author":"Zhuang","year":"2022","journal-title":"Transp. Res. Part E Logist. Transp. Rev."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"5212","DOI":"10.1021\/acssuschemeng.2c00179","article-title":"Kinetics and reaction mechanisms of acetic acid hydrodeoxygenation over pt and pt\u2013mo catalysts","volume":"10","author":"Zheng","year":"2022","journal-title":"ACS Sustain. Chem. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Chen, F., Luo, Z., Zhou, L., Pan, X., and Jiang, Y. (2024). Comprehensive survey of model compression and speed up for vision transformers. arXiv.","DOI":"10.62836\/jitp.v1i1.156"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1958","DOI":"10.1108\/K-09-2022-1205","article-title":"The optimal capacity decision of the catering merchant in omnichannel\u2014Service, production and delivery capacity","volume":"53","author":"Zhan","year":"2023","journal-title":"Kybernetes"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1016\/j.ejor.2017.03.053","article-title":"Parts-to-picker based order processing in a rack-moving mobile robots environment","volume":"262","author":"Boysen","year":"2017","journal-title":"Eur. J. Oper. Res."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.ejor.2020.05.032","article-title":"Introducing split orders and optimizing operational policies in robotic mobile fulfillment systems","volume":"288","author":"Xie","year":"2021","journal-title":"Eur. J. Oper. Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"105090","DOI":"10.1016\/j.cor.2020.105090","article-title":"Order allocation, rack allocation and rack sequencing for pickers in a mobile rack environment","volume":"125","author":"Valle","year":"2021","journal-title":"Comput. Oper. Res."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, S., and Tang, Z. (2023). Fabrication and control of porous structures via layer-by-layer assembly on pah\/paa polyelectrolyte coatings. Biomed. J. Sci. Tech. Res., 51.","DOI":"10.26717\/BJSTR.2023.51.008166"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"102789","DOI":"10.1016\/j.simpat.2023.102789","article-title":"An efficient multi-agent approach to order picking and robot scheduling in a robotic mobile fulfillment system","volume":"127","author":"Teck","year":"2023","journal-title":"Simul. Model. Pract. Theory"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3589","DOI":"10.1080\/00207543.2021.1926570","article-title":"Robotic mobile fulfillment systems: A mathematical modelling framework for e-commerce applications","volume":"60","author":"Gamache","year":"2022","journal-title":"Int. J. Prod. Res."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zou, Z., Amarasekara, I., and Dutta, A. (2024, January 26\u201329). Learning to decompose asymmetric channel kernels for generalized eigenwave multiplexing. Proceedings of the IEEE Conference on Computer Communications Workshops Proceedings, Paris, France.","DOI":"10.1109\/INFOCOM52122.2024.10621411"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/24725854.2018.1560517","article-title":"Inventory allocation in robotic mobile fulfillment systems","volume":"52","author":"Roy","year":"2020","journal-title":"IISE Trans."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"105467","DOI":"10.1016\/j.cor.2021.105467","article-title":"Joint optimization of order sequencing and rack scheduling in the robotic mobile fulfilment system","volume":"135","author":"Yang","year":"2021","journal-title":"Comput. Oper. Res."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1016\/j.ejor.2021.08.003","article-title":"Order picking optimization with rack-moving mobile robots and multiple workstations","volume":"300","author":"Zhuang","year":"2022","journal-title":"Eur. J. Oper. Res."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.tre.2018.11.005","article-title":"Robot-storage zone assignment strategies in mobile fulfillment systems","volume":"122","author":"Roy","year":"2019","journal-title":"Transp. Res. Part E Logist. Transp. Rev."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3789","DOI":"10.1287\/mnsc.2023.4858","article-title":"Courier dispatch in on-demand delivery","volume":"70","author":"Chen","year":"2024","journal-title":"Manag. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1109\/TEM.2016.2634540","article-title":"Bot-in-time delivery for robotic mobile fulfillment systems","volume":"64","author":"Yuan","year":"2017","journal-title":"IEEE Trans. Eng. Manag."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"4367","DOI":"10.1080\/00207543.2019.1652778","article-title":"Travel time models for the rack-moving mobile robot system","volume":"58","author":"Wang","year":"2020","journal-title":"Int. J. Prod. Res."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"108770","DOI":"10.1016\/j.asoc.2022.108770","article-title":"A bi-level memetic algorithm for the integrated order and vehicle scheduling in a rmfs","volume":"121","author":"Teck","year":"2022","journal-title":"Appl. Soft Comput."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"102087","DOI":"10.1016\/j.tre.2020.102087","article-title":"Robot scheduling for pod retrieval in a robotic mobile fulfillment system","volume":"142","author":"Gharehgozli","year":"2020","journal-title":"Transp. Res. Part E Logist. Transp. Rev."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"111541","DOI":"10.1016\/j.buildenv.2024.111541","article-title":"How does energy utilization affect rural sustainability development in traditional villages? re-examination from the coupling coordination degree of atmosphere-ecology-socioeconomics system","volume":"257","author":"Zhong","year":"2024","journal-title":"Build. Environ."},{"key":"ref_35","first-page":"42","article-title":"The reform of school education and teaching under the \u201cdouble reduction\u201d policy","volume":"4","author":"Chen","year":"2022","journal-title":"Sci. Soc. Res."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Xiong, C., Shukla, N., Xiong, W., and Zhu, S.-C. (2016, January 16\u201321). Robot learning with a spatial, temporal, and causal and-or graph. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden.","DOI":"10.1109\/ICRA.2016.7487364"},{"key":"ref_37","first-page":"21","article-title":"Efficacy and space optimization in industrial warehouses: An evaluation of paternoster continuous vertical conveyors","volume":"3","year":"2024","journal-title":"J. Eng. Manag. Syst. Eng."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zou, Z., Careem, M., Dutta, A., and Thawdar, N. (2022, January 16\u201320). Unified characterization and precoding for non-stationary channels. Proceedings of the ICC 2022-IEEE International Conference on Communications, Seoul, Republic of Korea.","DOI":"10.1109\/ICC45855.2022.9839118"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"107608","DOI":"10.1016\/j.enggeo.2024.107608","article-title":"Grey relation analysis and multiple criteria decision analysis method model for suitability evaluation of underground space development","volume":"338","author":"Ni","year":"2024","journal-title":"Eng. Geol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"100998","DOI":"10.1016\/j.aei.2019.100998","article-title":"Smart robotic mobile fulfillment system with dynamic conflict-free strategies considering cyber-physical integration","volume":"42","author":"Lee","year":"2019","journal-title":"Adv. Eng. Inform."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"116436","DOI":"10.1016\/j.oceaneng.2023.116436","article-title":"Ship encounter scenario generation for collision avoidance algorithm testing based on ais data","volume":"291","author":"Wang","year":"2024","journal-title":"Ocean. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"102578","DOI":"10.1016\/j.rcim.2023.102578","article-title":"A cyber-physical robotic mobile fulfillment system in smart manufacturing: The simulation aspect","volume":"83","author":"Keung","year":"2023","journal-title":"Robot.-Comput.-Integr. Manuf."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"13259","DOI":"10.1109\/ACCESS.2020.2966403","article-title":"A comprehensive framework for the design of modular robotic mobile fulfillment systems","volume":"8","author":"Wang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1116","DOI":"10.1016\/j.icte.2023.06.001","article-title":"Drl-based resource management in network slicing for vehicular applications","volume":"9","author":"Tairq","year":"2023","journal-title":"ICT Express"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.physb.2016.07.021","article-title":"Improved performance of dye-sensitized solar cell based on tio2 photoanode with fto glass and film both treated by ticl4","volume":"500","author":"Li","year":"2016","journal-title":"Phys. B Condens. Matter"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/14\/4713\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:20:23Z","timestamp":1760109623000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/14\/4713"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,20]]},"references-count":45,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["s24144713"],"URL":"https:\/\/doi.org\/10.3390\/s24144713","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,20]]}}}