{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,14]],"date-time":"2026-11-14T08:36:49Z","timestamp":1794645409355,"version":"4.9.1"},"reference-count":35,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,2,14]],"date-time":"2022-02-14T00:00:00Z","timestamp":1644796800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Prediction of pedestrian crossing behavior is an important issue faced by the realization of autonomous driving. The current research on pedestrian crossing behavior prediction is mainly based on vehicle camera. However, the sight line of vehicle camera may be blocked by other vehicles or the road environment, making it difficult to obtain key information in the scene. Pedestrian crossing behavior prediction based on surveillance video can be used in key road sections or accident-prone areas to provide supplementary information for vehicle decision-making, thereby reducing the risk of accidents. To this end, we propose a pedestrian crossing behavior prediction network for surveillance video. The network integrates pedestrian posture, local context and global context features through a new cross-stacked gated recurrence unit (GRU) structure to achieve accurate prediction of pedestrian crossing behavior. Applied onto the surveillance video dataset from the University of California, Berkeley to predict the pedestrian crossing behavior, our model achieves the best results regarding accuracy, F1 parameter, etc. In addition, we conducted experiments to study the effects of time to prediction and pedestrian speed on the prediction accuracy. This paper proves the feasibility of pedestrian crossing behavior prediction based on surveillance video. It provides a reference for the application of edge computing in the safety guarantee of automatic driving.<\/jats:p>","DOI":"10.3390\/s22041467","type":"journal-article","created":{"date-parts":[[2022,2,14]],"date-time":"2022-02-14T20:58:03Z","timestamp":1644872283000},"page":"1467","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Prediction of Pedestrian Crossing Behavior Based on Surveillance Video"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3591-0982","authenticated-orcid":false,"given":"Xiao","family":"Zhou","sequence":"first","affiliation":[{"name":"School of Mechanical and Electronic Engineering, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongyu","family":"Ren","sequence":"additional","affiliation":[{"name":"School of Mechanical and Electronic Engineering, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tingting","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Mechanical and Electronic Engineering, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xingang","family":"Mou","sequence":"additional","affiliation":[{"name":"School of Mechanical and Electronic Engineering, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi","family":"He","sequence":"additional","affiliation":[{"name":"Intelligent Transport Systems Research Center, Wuhan University of Technology, Wuhan 430063, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3992-2312","authenticated-orcid":false,"given":"Ching-Yao","family":"Chan","sequence":"additional","affiliation":[{"name":"California Partners for Advanced Transportation Technology (PATH), University of California, Berkeley, Richmond, CA 94720-5800, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Fang, Z., V\u00e1zquez, D., and L\u00f3pez, A.M. (2017). On-board detection of pedestrian intentions. Sensors, 17.","DOI":"10.3390\/s17102193"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/j.trf.2021.04.001","article-title":"Safety of micro-mobility: Riders\u2019 psychological factors and risky behaviors of cargo TTWs in China","volume":"80","author":"He","year":"2021","journal-title":"Transp. Res. Part F Traffic Psychol. Behav."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Yang, D., Zhang, H., Yurtsever, E., Redmill, K., and \u00d6zg\u00fcner, \u00dc. (2021). Predicting pedestrian crossing intention with feature fusion and spatio-temporal attention. arXiv.","DOI":"10.1109\/TIV.2022.3162719"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Rasouli, A., Kotseruba, I., and Tsotsos, J.K. (2017, January 22\u201329). Are they going to cross? A benchmark dataset and baseline for pedestrian crosswalk behavior. Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV), Venice, Italy.","DOI":"10.1109\/ICCVW.2017.33"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3485","DOI":"10.1109\/LRA.2020.2976305","article-title":"Spatio-temporal relationship reasoning for pedestrian intent prediction","volume":"5","author":"Liu","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Tran, D., Bourdev, L., Fergus, R., Torresani, L., and Paluri, M. (2015, January 7\u201313). Learning spatio-temporal features with 3D convolutional networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.510"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Saleh, K., Hossny, M., and Nahavandi, S. (2019, January 20\u201324). Real-time intent prediction of pedestrians for autonomous ground vehicles via spatio-temporal densenet. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793991"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1016\/j.neucom.2019.12.091","article-title":"Spatio-temporal DenseNet for real-time intent prediction of pedestrians in urban traffic environments","volume":"386","author":"Saleh","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Lorenzo, J., Parra, I., Wirth, F., Stiller, C., Llorca, D.F., and Sotelo, M.A. (November, January 19). Rnn-based pedestrian crossing prediction using activity and pose-related features. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA.","DOI":"10.1109\/IV47402.2020.9304652"},{"key":"ref_10","unstructured":"Rasouli, A., Yau, T., Rohani, M., and Luo, J. (2020). Multi-modal hybrid architecture for pedestrian action prediction. arXiv."},{"key":"ref_11","unstructured":"Rasouli, A., Kotseruba, I., and Tsotsos, J.K. (2020). Pedestrian action anticipation using contextual feature fusion in stacked rnns. arXiv."},{"key":"ref_12","unstructured":"Rasouli, A., Rohani, M., and Luo, J. (2020). Pedestrian behavior prediction via multitask learning and categorical interaction modeling. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kotseruba, I., Rasouli, A., and Tsotsos, J.K. (2021, January 3\u20138). Benchmark for evaluating pedestrian action prediction. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV48630.2021.00130"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Piccoli, F., Balakrishnan, R., Perez, M.J., Sachdeo, M., Nunez, C., and Tang, M. (2020, January 1\u20134). Fussi-net: Fusion of spatio-temporal skeletons for intention prediction network. Proceedings of the 2020 54th Asilomar Conference on Signals Systems, and Computers, Pacific Grove, CA, USA.","DOI":"10.1109\/IEEECONF51394.2020.9443552"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"9457","DOI":"10.1109\/TVT.2016.2591558","article-title":"Interworking of DSRC and cellular network technologies for V2X communications: A survey","volume":"65","author":"Abboud","year":"2016","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5721","DOI":"10.1109\/TITS.2020.2991642","article-title":"Visualization analysis of intelligent vehicles research field based on mapping knowledge domain","volume":"22","author":"He","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_17","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_18","unstructured":"Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Shen, M., Habibi, G., and How, J.P. (2018, January 1\u20135). Transferable pedestrian motion prediction models at intersections. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593783"},{"key":"ref_20","unstructured":"Kipf, T.N., and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Lee, N., Choi, W., Vernaza, P., Choy, C.B., Torr, P.H., and Chandraker, M. (2017, January 21\u201326). Desire: Distant future prediction in dynamic scenes with interacting agents. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.233"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1109\/ACCESS.2017.2778011","article-title":"Action recognition in video sequences using deep bi-directional LSTM with CNN features","volume":"6","author":"Ullah","year":"2017","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Luong, M.T., Pham, H., and Manning, C.D. (2015). Effective approaches to attention-based neural machine translation. arXiv.","DOI":"10.18653\/v1\/D15-1166"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1522","DOI":"10.1109\/LRA.2018.2800780","article-title":"Predicting occupancy distributions of walking humans with convolutional neural networks","volume":"3","author":"Doellinger","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Sighencea, B.I., Stanciu, R.I., and C\u0103leanu, C.D. (2021). A review of deep learning-based methods for pedestrian trajectory prediction. Sensors, 21.","DOI":"10.3390\/s21227543"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Fang, Z., and L\u00f3pez, A.M. (2018, January 26\u201330). Is the pedestrian going to cross? Answering by 2d pose estimation. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500413"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Cadena PR, G., Yang, M., Qian, Y., and Wang, C. (2019, January 27\u201330). Pedestrian graph: Pedestrian crossing prediction based on 2d pose estimation and graph convolutional networks. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.","DOI":"10.1109\/ITSC.2019.8917118"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, Z., and Papanikolopoulos, N. (2020, January 25\u201329). Estimating pedestrian crossing states based on single 2D body pose. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341745"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Gesnouin, J., Pechberti, S., Bresson, G., Stanciulescu, B., and Moutarde, F. (2020). Predicting intentions of pedestrians from 2d skeletal pose sequences with a representation-focused multi-branch deep learning network. Algorithms, 13.","DOI":"10.3390\/a13120331"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cai, Y., Li, D., Zhou, X., and Mou, X. (2018). Robust drivable road region detection for fixed-route autonomous vehicles using map-fusion images. Sensors, 18.","DOI":"10.3390\/s18124158"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Cai, Y., Zhou, X., Li, D., Ming, Y., and Mou, X. (2018, January 1\u20133). Head pose estimation with neural networks from surveillant images. Proceedings of the Eleventh International Conference on Machine Vision (ICMV 2018), Munich, Germany.","DOI":"10.1117\/12.2523090"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Kotseruba, I., Rasouli, A., and Tsotsos, J.K. (2020, January 23). Do they want to cross? Understanding pedestrian intention for behavior prediction. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA.","DOI":"10.1109\/IV47402.2020.9304591"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Bhattacharyya, A., Fritz, M., and Schiele, B. (2018, January 18\u201323). Long-term on-board prediction of people in traffic scenes under uncertainty. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00441"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/4\/1467\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:19:19Z","timestamp":1760134759000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/4\/1467"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,14]]},"references-count":35,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["s22041467"],"URL":"https:\/\/doi.org\/10.3390\/s22041467","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,14]]}}}