{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T21:03:15Z","timestamp":1779310995828,"version":"3.51.4"},"reference-count":44,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,4,30]],"date-time":"2024-04-30T00:00:00Z","timestamp":1714435200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation","award":["CMMI-1739503 and CCF-2140154"],"award-info":[{"award-number":["CMMI-1739503 and CCF-2140154"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Cyber-Phys. Syst."],"published-print":{"date-parts":[[2024,4,30]]},"abstract":"<jats:p>\n            Autonomous vehicles are cyber-physical systems that combine embedded computing and deep learning with physical systems to perceive the world, predict future states, and safely control the vehicle through changing environments. The ability of an autonomous vehicle to accurately predict the motion of other road users across a wide range of diverse scenarios is critical for both motion planning and safety. However, existing motion prediction methods do not explicitly model contextual information about the environment, which can cause significant variations in performance across diverse driving scenarios. To address this limitation, we propose\n            <jats:bold>CASTNet<\/jats:bold>\n            : a dynamic, context-aware approach for motion prediction that (i) identifies the current driving context using a spatio-temporal model, (ii) adapts an ensemble of motion prediction models to fit the current context, and (iii) applies novel trajectory fusion methods to combine predictions output by the ensemble. This approach enables CASTNet to improve robustness by minimizing motion prediction error across diverse driving scenarios. CASTNet is highly modular and can be used with various existing image processing backbones and motion predictors. We demonstrate how CASTNet can improve both CNN-based and graph-learning-based motion prediction approaches and conduct ablation studies on the performance, latency, and model size for various ensemble architecture choices. In addition, we propose and evaluate several attention-based spatio-temporal models for context identification and ensemble selection. We also propose a modular trajectory fusion algorithm that effectively filters, clusters, and fuses the predicted trajectories output by the ensemble. On the nuScenes dataset, our approach demonstrates more robust and consistent performance across diverse, real-world driving contexts than state-of-the-art techniques.\n          <\/jats:p>","DOI":"10.1145\/3648622","type":"journal-article","created":{"date-parts":[[2024,2,16]],"date-time":"2024-02-16T12:32:45Z","timestamp":1708086765000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["CASTNet: A Context-Aware, Spatio-Temporal Dynamic Motion Prediction Ensemble for Autonomous Driving"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2074-1722","authenticated-orcid":false,"given":"Trier","family":"Mortlock","sequence":"first","affiliation":[{"name":"University of California, Irvine, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4974-1622","authenticated-orcid":false,"given":"Arnav","family":"Malawade","sequence":"additional","affiliation":[{"name":"University of California, Irvine, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-4023-2011","authenticated-orcid":false,"given":"Kohei","family":"Tsujio","sequence":"additional","affiliation":[{"name":"University of California, Irvine, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5390-0497","authenticated-orcid":false,"given":"Mohammad","family":"Al Faruque","sequence":"additional","affiliation":[{"name":"University of California, Irvine, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,5,15]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Estimation with Applications to Tracking and Navigation: Theory Algorithms and Software","author":"Bar-Shalom Yaakov","year":"2004","unstructured":"Yaakov Bar-Shalom, X. Rong Li, and Thiagalingam Kirubarajan. 2004. Estimation with Applications to Tracking and Navigation: Theory Algorithms and Software. John Wiley & Sons."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3375716"},{"key":"e_1_3_2_4_2","first-page":"329","volume-title":"Conference on Robot Learning","author":"Buhet Thibault","year":"2021","unstructured":"Thibault Buhet, Emilie Wirbel, Andrei Bursuc, and Xavier Perrotton. 2021. PLOP: Probabilistic polynomial objects trajectory prediction for autonomous driving. In Conference on Robot Learning. PMLR, 329\u2013338."},{"key":"e_1_3_2_5_2","first-page":"11621","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201920)","author":"Caesar Holger","year":"2020","unstructured":"Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. 2020. nuScenes: A multimodal dataset for autonomous driving. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201920). 11621\u201311631."},{"key":"e_1_3_2_6_2","article-title":"MultiPath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction","author":"Chai Yuning","year":"2019","unstructured":"Yuning Chai, Benjamin Sapp, Mayank Bansal, and Dragomir Anguelov. 2019. MultiPath: Multiple probabilistic anchor trajectory hypotheses for behavior prediction. In 3rd Conference on Robot Learning (CoRL\u201919).","journal-title":"3rd Conference on Robot Learning (CoRL\u201919)"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS51168.2021.9636252"},{"key":"e_1_3_2_8_2","first-page":"2090","volume-title":"IEEE International Conference on Robotics and Automation (ICRA\u201919)","author":"Cui Henggang","year":"2019","unstructured":"Henggang Cui, Vladan Radosavljevic, Fang-Chieh Chou, Tsung-Han Lin, Thi Nguyen, Tzu-Kuo Huang, Jeff Schneider, and Nemanja Djuric. 2019. Multimodal trajectory predictions for autonomous driving using deep convolutional networks. In IEEE International Conference on Robotics and Automation (ICRA\u201919). IEEE, 2090\u20132096."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIV.2018.2804159"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/IV48863.2021.9575718"},{"key":"e_1_3_2_11_2","first-page":"2095","volume-title":"IEEE\/CVF Winter Conference on Applications of Computer Vision","author":"Djuric Nemanja","year":"2020","unstructured":"Nemanja Djuric, Vladan Radosavljevic, Henggang Cui, Thi Nguyen, Fang-Chieh Chou, Tsung-Han Lin, Nitin Singh, and Jeff Schneider. 2020. Uncertainty-aware short-term motion prediction of traffic actors for autonomous driving. In IEEE\/CVF Winter Conference on Applications of Computer Vision. 2095\u20132104."},{"key":"e_1_3_2_12_2","first-page":"9710","volume-title":"IEEE\/CVF International Conference on Computer Vision","year":"2021","unstructured":"Scott Ettinger, Shuyang Cheng, Benjamin Caine, Chenxi Liu, Hang Zhao, Sabeek Pradhan, Yuning Chai, Ben Sapp, Charles R. Qi, Yin Zhou, Zoey Yang, Aurelien Chouard, Pei Sun, Jiquan Ngiam, Vijay Vasudevan, Alexander McCauley, Jonathon Shlens, and Dragomir Anguelov. 2021. Large scale interactive motion forecasting for autonomous driving: The Waymo Open Motion Dataset. In IEEE\/CVF International Conference on Computer Vision. 9710\u20139719."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/2823345"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9812253"},{"issue":"4","key":"e_1_3_2_15_2","doi-asserted-by":"crossref","first-page":"6915","DOI":"10.1109\/LRA.2021.3096157","article-title":"ObserveNet control: A vision-dynamics learning approach to predictive control in autonomous vehicles","volume":"6","author":"Ginerica Cosmin","year":"2021","unstructured":"Cosmin Ginerica, Mihai Zaha, Florin Gogianu, Lucian Busoniu, Bogdan Trasnea, and Sorin Grigorescu. 2021. ObserveNet control: A vision-dynamics learning approach to predictive control in autonomous vehicles. IEEE Robot. Automat. Lett. 6, 4 (2021), 6915\u20136922.","journal-title":"IEEE Robot. Automat. Lett."},{"issue":"4","key":"e_1_3_2_16_2","doi-asserted-by":"crossref","first-page":"3441","DOI":"10.1109\/LRA.2019.2926224","article-title":"NeuroTrajectory: A neuroevolutionary approach to local state trajectory learning for autonomous vehicles","volume":"4","author":"Grigorescu Sorin Mihai","year":"2019","unstructured":"Sorin Mihai Grigorescu, Bogdan Trasnea, Liviu Marina, Andrei Vasilcoi, and Tiberiu Cocias. 2019. NeuroTrajectory: A neuroevolutionary approach to local state trajectory learning for autonomous vehicles. IEEE Robot. Automat. Lett. 4, 4 (2019), 3441\u20133448.","journal-title":"IEEE Robot. Automat. Lett."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3470453"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_19_2","article-title":"Situation-aware environment perception using a multi-layer attention map","author":"Henning Matti","year":"2022","unstructured":"Matti Henning, Johannes Christian Muller, Fabian Gies, Michael Buchholz, and Klaus Dietmayer. 2022. Situation-aware environment perception using a multi-layer attention map. IEEE Trans. Intell. Vehic. 8, 1 (2022).","journal-title":"IEEE Trans. Intell. Vehic."},{"key":"e_1_3_2_20_2","first-page":"409","volume-title":"Conference on Robot Learning","author":"Houston John","year":"2021","unstructured":"John Houston, Guido Zuidhof, Luca Bergamini, Yawei Ye, Long Chen, Ashesh Jain, Sammy Omari, Vladimir Iglovikov, and Peter Ondruska. 2021. One thousand and one hours: Self-driving motion prediction dataset. In Conference on Robot Learning. PMLR, 409\u2013418."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8794282"},{"key":"e_1_3_2_22_2","volume-title":"Conference and Workshop on Neural Information Processing Systems (NEURIPS\u201917)","author":"Japuria Nikita","year":"2017","unstructured":"Nikita Japuria, Golnaz Habibi, and Jonathan How. 2017. CASNSC: A context-based approach for accurate pedestrian motion prediction at intersections. In Conference and Workshop on Neural Information Processing Systems (NEURIPS\u201917)."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3083918"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1186\/s40648-014-0001-z"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS45743.2020.9341392"},{"key":"e_1_3_2_26_2","volume-title":"ACM\/IEEE 13th International Conference on Cyber-Physical Systems (ICCPS\u201922)","author":"Malawade Arnav Vaibhav","year":"2022","unstructured":"Arnav Vaibhav Malawade, Trier Mortlock, and Mohammad Abdullah Al Faruque. 2022. HydraFusion: Context-aware selective sensor fusion for robust and efficient autonomous vehicle perception. In ACM\/IEEE 13th International Conference on Cyber-Physical Systems (ICCPS\u201922)."},{"key":"e_1_3_2_27_2","volume-title":"Design Automation Conference (DAC\u201922)","author":"Malawade Arnav Vaibhav","year":"2022","unstructured":"Arnav Vaibhav Malawade, Trier Robert Mortlock, and Mohammad Abdullah Al Faruque. 2022. EcoFusion: Energy-aware adaptive sensor fusion for efficient autonomous vehicle perception. In Design Automation Conference (DAC\u201922)."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3564273"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIV.2020.2991952"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3565270"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jsr.2020.10.005"},{"key":"e_1_3_2_32_2","first-page":"14074","volume-title":"IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201920)","author":"Phan-Minh Tung","year":"2020","unstructured":"Tung Phan-Minh, Elena Corina Grigore, Freddy Boulton, Oscar Beijbom, and Eric Wolff. 2020. CoverNet: Multimodal behavior prediction using trajectory sets. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201920). 14074\u201314083."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491243"},{"key":"e_1_3_2_34_2","first-page":"2821","volume-title":"IEEE\/CVF International Conference on Computer Vision","author":"Rhinehart Nicholas","year":"2019","unstructured":"Nicholas Rhinehart, Rowan McAllister, Kris Kitani, and Sergey Levine. 2019. PRECOG: Prediction conditioned on goals in visual multi-agent settings. In IEEE\/CVF International Conference on Computer Vision. 2821\u20132830."},{"key":"e_1_3_2_35_2","article-title":"Driver and pedestrian mutual awareness for path prediction and collision risk estimation","author":"Roth Markus","year":"2021","unstructured":"Markus Roth, Jork Stapel, Riender Happee, and Dariu M. Gavrila. 2021. Driver and pedestrian mutual awareness for path prediction and collision risk estimation. IEEE Trans. Intell. Vehic. 7, 4 (2021).","journal-title":"IEEE Trans. Intell. Vehic."},{"key":"e_1_3_2_36_2","first-page":"683","volume-title":"European Conference on Computer Vision (ECCV\u201920)","author":"Salzmann Tim","year":"2020","unstructured":"Tim Salzmann, Boris Ivanovic, Punarjay Chakravarty, and Marco Pavone. 2020. Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data. In European Conference on Computer Vision (ECCV\u201920). Springer, 683\u2013700."},{"issue":"4","key":"e_1_3_2_37_2","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1109\/TIV.2022.3188662","article-title":"Prediction-uncertainty-aware decision-making for autonomous vehicles","volume":"7","author":"Tang Xiaolin","year":"2022","unstructured":"Xiaolin Tang, Kai Yang, Hong Wang, Jiahang Wu, Yechen Qin, Wenhao Yu, and Dongpu Cao. 2022. Prediction-uncertainty-aware decision-making for autonomous vehicles. IEEE Trans. Intell. Vehic. 7, 4 (2022), 849\u2013862.","journal-title":"IEEE Trans. Intell. Vehic."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3461342"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-09823-4_34"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9812107"},{"key":"e_1_3_2_41_2","doi-asserted-by":"crossref","first-page":"100315","DOI":"10.1016\/j.trip.2021.100315","article-title":"Investigating heterogeneous car-following behaviors of different vehicle types, traffic densities and road types","volume":"9","author":"Wang Jinghua","year":"2021","unstructured":"Jinghua Wang, Zhao Zhang, Feng Liu, and Guangquan Lu. 2021. Investigating heterogeneous car-following behaviors of different vehicle types, traffic densities and road types. Transport. Res. Interdiscip. Perspect. 9 (2021), 100315.","journal-title":"Transport. Res. Interdiscip. Perspect."},{"key":"e_1_3_2_42_2","article-title":"Vehicle trajectory prediction by knowledge-driven LSTM network in urban environments","volume":"2020","author":"Wang Shaobo","year":"2020","unstructured":"Shaobo Wang, Pan Zhao, Biao Yu, Weixin Huang, and Huawei Liang. 2020. Vehicle trajectory prediction by knowledge-driven LSTM network in urban environments. J. Advanc. Transport. 2020, 1 (2020).","journal-title":"J. Advanc. Transport."},{"key":"e_1_3_2_43_2","first-page":"225","volume-title":"ACM\/IEEE 13th International Conference on Cyber-Physical Systems (ICCPS\u201922)","author":"Yang Yahan","year":"2022","unstructured":"Yahan Yang, Ramneet Kaur, Souradeep Dutta, and Insup Lee. 2022. Interpretable detection of distribution shifts in learning enabled cyber-physical systems. In ACM\/IEEE 13th International Conference on Cyber-Physical Systems (ICCPS\u201922). IEEE, 225\u2013235."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2983149"},{"key":"e_1_3_2_45_2","first-page":"895","volume-title":"Conference on Robot Learning","author":"Zhao Hang","year":"2021","unstructured":"Hang Zhao, Jiyang Gao, Tian Lan, Chen Sun, Ben Sapp, Balakrishnan Varadarajan, Yue Shen, Yi Shen, Yuning Chai, Cordelia Schmid, et\u00a0al. 2021. TNT: Target-driven trajectory prediction. In Conference on Robot Learning. PMLR, 895\u2013904."}],"container-title":["ACM Transactions on Cyber-Physical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3648622","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3648622","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:20Z","timestamp":1750287020000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3648622"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,30]]},"references-count":44,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,4,30]]}},"alternative-id":["10.1145\/3648622"],"URL":"https:\/\/doi.org\/10.1145\/3648622","relation":{},"ISSN":["2378-962X","2378-9638"],"issn-type":[{"value":"2378-962X","type":"print"},{"value":"2378-9638","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,30]]},"assertion":[{"value":"2023-02-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-04","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-15","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}