{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T23:04:50Z","timestamp":1782774290490,"version":"3.54.5"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2021,12,16]],"date-time":"2021-12-16T00:00:00Z","timestamp":1639612800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61976198 and 62022077"],"award-info":[{"award-number":["61976198 and 62022077"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Municipal Program on Science and Technology Research Project of Wuhu City","award":["2019yf05"],"award-info":[{"award-number":["2019yf05"]}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2021,12,31]]},"abstract":"<jats:p>Mobility prediction plays an important role in a wide range of location-based applications and services. However, there are three problems in the existing literature: (1) explicit high-order interactions of spatio-temporal features are not systemically modeled; (2) most existing algorithms place attention mechanisms on top of recurrent network, so they can not allow for full parallelism and are inferior to self-attention for capturing long-range dependence; (3) most literature does not make good use of long-term historical information and do not effectively model the long-term periodicity of users. To this end, we propose MoveNet and RLMoveNet. MoveNet is a self-attention-based sequential model, predicting each user\u2019s next destination based on her most recent visits and historical trajectory. MoveNet first introduces a cross-based learning framework for modeling feature interactions. With self-attention on both the most recent visits and historical trajectory, MoveNet can use an attention mechanism to capture the user\u2019s long-term regularity in a more efficient way. Based on MoveNet, to model long-term periodicity more effectively, we add the reinforcement learning layer and named RLMoveNet. RLMoveNet regards the human mobility prediction as a reinforcement learning problem, using the reinforcement learning layer as the regularization part to drive the model to pay attention to the behavior with periodic actions, which can help us make the algorithm more effective. We evaluate both of them with three real-world mobility datasets. MoveNet outperforms the state-of-the-art mobility predictor by around 10% in terms of accuracy, and simultaneously achieves faster convergence and over 4x training speedup. Moreover, RLMoveNet achieves higher prediction accuracy than MoveNet, which proves that modeling periodicity explicitly from the perspective of reinforcement learning is more effective.<\/jats:p>","DOI":"10.1145\/3469860","type":"journal-article","created":{"date-parts":[[2021,12,16]],"date-time":"2021-12-16T17:05:53Z","timestamp":1639674353000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Predicting Human Mobility with Reinforcement-Learning-Based Long-Term Periodicity Modeling"],"prefix":"10.1145","volume":"12","author":[{"given":"Shuo","family":"Tao","sequence":"first","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jingang","family":"Jiang","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Defu","family":"Lian","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kai","family":"Zheng","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Enhong","family":"Chen","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,12,16]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.5555\/862896.881068"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature04292"},{"key":"e_1_3_1_4_2","article-title":"Ask the right questions: Active question reformulation with reinforcement learning","author":"Buck Christian","year":"2017","unstructured":"Christian Buck, Jannis Bulian, Massimiliano Ciaramita, Wojciech Gajewski, Andrea Gesmundo, Neil Houlsby, and Wei Wang. 2017. Ask the right questions: Active question reformulation with reinforcement learning. arXiv preprint arXiv:1705.07830 (2017).","journal-title":"arXiv preprint arXiv:1705.07830"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-06605-9_16"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.3115\/981863.981904"},{"key":"e_1_3_1_7_2","first-page":"1052","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Chen Xinshi","year":"2019","unstructured":"Xinshi Chen, Shuang Li, Hui Li, Shaohua Jiang, Yuan Qi, and Le Song. 2019. Generative adversarial user model for reinforcement learning based recommendation system. In Proceedings of the International Conference on Machine Learning. PMLR, 1052\u20131061."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.5555\/2540128.2540504"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/2988450.2988454"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/2020408.2020579"},{"key":"e_1_3_1_11_2","article-title":"A sparse deep factorization machine for efficient CTR prediction","author":"Deng Wei","year":"2020","unstructured":"Wei Deng, Junwei Pan, Tian Zhou, Aaron Flores, and Guang Lin. 2020. A sparse deep factorization machine for efficient CTR prediction. arXiv preprint arXiv:2002.06987 (2020).","journal-title":"arXiv preprint arXiv:2002.06987"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186058"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.5555\/2832415.2832536"},{"key":"e_1_3_1_14_2","volume-title":"Proceedings of ICWSM\u201912","author":"Gao H.","year":"2012","unstructured":"H. Gao, J. Tang, and H. Liu. 2012. Exploring social-historical ties on location-based social networks. In Proceedings of ICWSM\u201912."},{"key":"e_1_3_1_15_2","volume-title":"Proceedings of the Mobile Data Challenge at the 10th International Conference on Pervasive Computing","author":"Gao H.","year":"2012","unstructured":"H. Gao, J. Tang, and H. Liu. 2012. Mobile location prediction in spatio-temporal context. In Proceedings of the Mobile Data Challenge at the 10th International Conference on Pervasive Computing."},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313610"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature06958"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.5555\/3172077.3172127"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.5555\/2997046.2997187"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080777"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2648584.2648589"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0966-6923(01)00019-9"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0308344101"},{"key":"e_1_3_1_24_2","first-page":"3543","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Jain Sarthak","year":"2019","unstructured":"Sarthak Jain and Byron C. Wallace. 2019. Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 3543\u20133556."},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3191746"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.5555\/3504035.3504131"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/2959100.2959134"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2018.00035"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1005259324588"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.5555\/3298483.3298548"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2629557"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623638"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/2487788.2487907"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-06608-0_9"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220023"},{"key":"e_1_3_1_36_2","article-title":"Deep reinforcement learning based recommendation with explicit user-item interactions modeling","author":"Liu Feng","year":"2018","unstructured":"Feng Liu, Ruiming Tang, Xutao Li, Weinan Zhang, Yunming Ye, Haokun Chen, Huifeng Guo, and Yuzhou Zhang. 2018. Deep reinforcement learning based recommendation with explicit user-item interactions modeling. arXiv preprint arXiv:1810.12027 (2018).","journal-title":"arXiv preprint arXiv:1810.12027"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.5555\/3015812.3015841"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/2370216.2370421"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/1557019.1557091"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2012.113"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186040"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2010.127"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.2352\/ISSN.2470-1173.2017.19.AVM-023"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1282"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939704"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330933"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature16961"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.1177170"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFCOM.2004.1357026"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.3115\/1220175.1220299"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295349"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3124749.3124754"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186066"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992696"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.5555\/3172077.3172324"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401147"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3041658"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.2014.2327053"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3133056"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/2093973.2093980"},{"key":"e_1_3_1_62_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-30146-0_21"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.14778\/2732939.2732949"},{"key":"e_1_3_1_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939793"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33015877"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.5555\/3015812.3015858"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240374"},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3185994"},{"key":"e_1_3_1_69_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2012.02.002"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330668"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3469860","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3469860","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:16Z","timestamp":1750188616000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3469860"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,16]]},"references-count":69,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,12,31]]}},"alternative-id":["10.1145\/3469860"],"URL":"https:\/\/doi.org\/10.1145\/3469860","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,16]]},"assertion":[{"value":"2020-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-12-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}