{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T01:13:30Z","timestamp":1771636410192,"version":"3.50.1"},"reference-count":31,"publisher":"Institute for Operations Research and the Management Sciences (INFORMS)","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["INFORMS Journal on Applied Analytics"],"published-print":{"date-parts":[[2020,9]]},"abstract":"<jats:p> Order dispatching is instrumental to the marketplace engine of a large-scale ride-hailing platform, such as the DiDi platform, which continuously matches passenger trip requests to drivers at a scale of tens of millions per day. Because of the dynamic and stochastic nature of supply and demand in this context, the ride-hailing order-dispatching problem is challenging to solve for an optimal solution. Added to the complexity are considerations of system response time, reliability, and multiple objectives. In this paper, we describe how our approach to this optimization problem has evolved from a combinatorial optimization approach to one that encompasses a semi-Markov decision-process model and deep reinforcement learning. We discuss the various practical considerations of our solution development and real-world impact to the business. <\/jats:p>","DOI":"10.1287\/inte.2020.1047","type":"journal-article","created":{"date-parts":[[2020,9,24]],"date-time":"2020-09-24T13:00:37Z","timestamp":1600952437000},"page":"272-286","source":"Crossref","is-referenced-by-count":111,"title":["Ride-Hailing Order Dispatching at DiDi via Reinforcement Learning"],"prefix":"10.1287","volume":"50","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5383-4816","authenticated-orcid":false,"given":"Zhiwei (Tony)","family":"Qin","sequence":"first","affiliation":[{"name":"DiDi Labs, Mountain View, California 94043;"}]},{"given":"Xiaocheng","family":"Tang","sequence":"additional","affiliation":[{"name":"DiDi Labs, Mountain View, California 94043;"}]},{"given":"Yan","family":"Jiao","sequence":"additional","affiliation":[{"name":"DiDi Labs, Mountain View, California 94043;"}]},{"given":"Fan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Didi Chuxing, Beijing 100193, China"}]},{"given":"Zhe","family":"Xu","sequence":"additional","affiliation":[{"name":"Didi Chuxing, Beijing 100193, China"}]},{"given":"Hongtu","family":"Zhu","sequence":"additional","affiliation":[{"name":"Didi Chuxing, Beijing 100193, China"}]},{"given":"Jieping","family":"Ye","sequence":"additional","affiliation":[{"name":"Didi Chuxing, Beijing 100193, China"}]}],"member":"109","reference":[{"issue":"1","key":"B1","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/0025-5564(71)90051-4","volume":"10","author":"Albus JS","year":"1971","journal-title":"Math. Biosci."},{"issue":"3","key":"B2","doi-asserted-by":"crossref","first-page":"462","DOI":"10.1073\/pnas.1611675114","volume":"114","author":"Alonso-Mora J","year":"2017","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"B3","doi-asserted-by":"crossref","unstructured":"Bailey WA Jr , \nClark TD Jr   (1987) A simulation analysis of demand and fleet size effects on taxicab service rates.  \nThesen A , \nGrant H , \nKelton WD  , eds. Proc. 19th Conf. Winter Simulation (Association for Computing Machinery, New York), 838\u2013844.","DOI":"10.1145\/318371.318705"},{"key":"B4","doi-asserted-by":"crossref","DOI":"10.21236\/ADA280862","volume-title":"Advantage updating.","author":"Baird LC","year":"1993"},{"issue":"1","key":"B8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s004540010071","volume":"25","author":"Hales TC","year":"2001","journal-title":"Discrete Comput. Geometry"},{"key":"B9","doi-asserted-by":"crossref","unstructured":"Holler J , \nVuorio R , \nQin Z , \nTang X , \nJiao Y , \nJin T , \nSingh S , \nWang C , \nYe J   (2019) Deep reinforcement learning for multi-driver vehicle dispatching and repositioning problem.  \nWang J , \nShim K , \nWu X  , eds. 2019 IEEE Internat. Conf. Data Mining (ICDM) (Institute of Electrical and Electronics Engineers, Washington, DC), 1090\u20131095.","DOI":"10.1109\/ICDM.2019.00129"},{"key":"B10","doi-asserted-by":"crossref","unstructured":"Jindal I , \nQin ZT , \nChen X , \nNokleby M , \nYe J   (2018) Optimizing taxi carpool policies via reinforcement learning and spatio-temporal mining.  \nAbe N , \nLiu H , \nPu C , \nHu X , \nAhmed N , \nQiao M , \nSong Y , eds. 2018 IEEE Internat. Conf. Big Data (Big Data) (Institute of Electrical and Electronics Engineers, Washington, DC), 1417\u20131426.","DOI":"10.1109\/BigData.2018.8622481"},{"issue":"1","key":"B11","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1002\/nav.3800020109","volume":"2","author":"Kuhn HW","year":"1955","journal-title":"Naval Res. Logist. Quart."},{"key":"B12","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/j.procs.2016.04.112","volume":"83","author":"K\u00fcmmel M","year":"2016","journal-title":"Procedia Comput. Sci."},{"key":"B13","doi-asserted-by":"crossref","unstructured":"Li M , \nQin Z , \nJiao Y , \nYang Y , \nWang J , \nWang C , \nWu G , \nYe J   (2019) Efficient ridesharing order dispatching with mean field multi-agent reinforcement learning.  \nLiu L , \nWhite R  , eds. WWW\u201919 World Wide Web Conf. (Association for Computing Machinery, New York), 983\u2013994.","DOI":"10.1145\/3308558.3313433"},{"key":"B14","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.jpdc.2019.03.014","volume":"130","author":"Lopes PA","year":"2019","journal-title":"J. Parallel Distributed Comput."},{"issue":"2","key":"B15","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1109\/TASE.2016.2529580","volume":"13","author":"Miao F","year":"2016","journal-title":"IEEE Trans. Automation Sci. Engrg."},{"issue":"7540","key":"B16","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"Mnih V","year":"2015","journal-title":"Nature"},{"key":"B18","first-page":"9839","volume":"31","author":"Nazari M","year":"2018","journal-title":"Adv. Neural Inform. Processing Systems"},{"key":"B19","doi-asserted-by":"crossref","unstructured":"Oda T , \nJoe-Wong C   (2018) MOVI: A model-free approach to dynamic fleet management. IEEE INFOCOM 2018-IEEE Conf. Comput. Commun. (Institute of Electrical and Electronics Engineers, Washington, DC), 2708\u20132716.","DOI":"10.1109\/INFOCOM.2018.8485988"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1287\/stsy.2019.0037"},{"key":"B23","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.trc.2019.12.005","volume":"111","author":"Shou Z","year":"2020","journal-title":"Transportation Res. Part C: Emerging Tech."},{"issue":"7587","key":"B24","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","volume":"529","author":"Silver D","year":"2016","journal-title":"Nature"},{"issue":"1","key":"B25","first-page":"9","volume":"3","author":"Sutton RS","year":"1988","journal-title":"Machine Learn."},{"key":"B26","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton RS","year":"2018"},{"issue":"1","key":"B27","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/S0004-3702(99)00052-1","volume":"112","author":"Sutton RS","year":"1999","journal-title":"Artificial Intelligence"},{"key":"B28","doi-asserted-by":"crossref","unstructured":"Tang X , \nQin Z , \nZhang F , \nWang Z , \nXu Z , \nMa Y , \nZhu H , \nYe J   (2019) A deep value-network based approach for multi-driver order dispatching. Proc. 25th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 1780\u20131790.","DOI":"10.1145\/3292500.3330724"},{"issue":"1","key":"B29","first-page":"59","volume":"22","author":"Tsitsiklis JN","year":"1996","journal-title":"Machine Learn."},{"issue":"6","key":"B30","doi-asserted-by":"crossref","first-page":"731","DOI":"10.3390\/sym11060731","volume":"11","author":"Uher V","year":"2019","journal-title":"Symmetry"},{"key":"B32","doi-asserted-by":"crossref","unstructured":"Van Hasselt H , \nGuez A , \nSilver D   (2016) Deep reinforcement learning with double Q-learning. 30th AAAI Conf. Artificial Intelligence (Association for the Advancement of Artificial Intelligence, Menlo Park, CA), 2094\u20132100.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"B33","doi-asserted-by":"crossref","unstructured":"Verma T , \nVarakantham P , \nKraus S , \nLau HC   (2017) Augmenting decisions of taxi drivers through reinforcement learning for improving revenues. 27th Internat. Conf. Automated Planning Scheduling (Association for the Advancement of Artificial Intelligence, Menlo Park, CA), 409\u2013417.","DOI":"10.1609\/icaps.v27i1.13846"},{"key":"B34","first-page":"2692","volume":"28","author":"Vinyals O","year":"2015","journal-title":"Adv. Neural Inform. Processing Systems"},{"key":"B35","doi-asserted-by":"crossref","unstructured":"Wang Z , \nQin Z , \nTang X , \nYe J , \nZhu H   (2018) Deep reinforcement learning with knowledge transfer for online rides order dispatching. 2018 IEEE Internat. Conf. Data Mining (ICDM) (Institute of Electrical and Electronics Engineers, Washington, DC), 617\u2013626.","DOI":"10.1109\/ICDM.2018.00077"},{"key":"B36","doi-asserted-by":"crossref","unstructured":"Xu Z , \nLi Z , \nGuan Q , \nZhang D , \nLi Q , \nNan J , \nLiu C , \nBian W , \nYe J   (2018) Large-scale order dispatch in on-demand ride-hailing platforms: A learning and planning approach. Proc. 24th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 905\u2013913.","DOI":"10.1145\/3219819.3219824"},{"key":"B37","author":"Yan C","year":"2019","journal-title":"Naval Res. Logist."},{"key":"B38","doi-asserted-by":"crossref","unstructured":"Zhang L , \nHu T , \nMin Y , \nWu G , \nZhang J , \nFeng P , \nGong P , \nYe J   (2017) A taxi order dispatch model based on combinatorial optimization. Proc. 23rd ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 2151\u20132159.","DOI":"10.1145\/3097983.3098138"}],"container-title":["INFORMS Journal on Applied Analytics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubsonline.informs.org\/doi\/pdf\/10.1287\/inte.2020.1047","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,2]],"date-time":"2023-04-02T17:03:42Z","timestamp":1680455022000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubsonline.informs.org\/doi\/10.1287\/inte.2020.1047"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9]]},"references-count":31,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2020,9]]}},"alternative-id":["10.1287\/inte.2020.1047"],"URL":"https:\/\/doi.org\/10.1287\/inte.2020.1047","relation":{},"ISSN":["2644-0865","2644-0873"],"issn-type":[{"value":"2644-0865","type":"print"},{"value":"2644-0873","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9]]}}}