{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,21]],"date-time":"2025-11-21T15:09:47Z","timestamp":1763737787952,"version":"3.45.0"},"reference-count":56,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2025,11,21]],"date-time":"2025-11-21T00:00:00Z","timestamp":1763683200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"R&D Program funded by Ministry of Land, Infrastructure and Transport of Korean government","award":["RS-2023-00233952"],"award-info":[{"award-number":["RS-2023-00233952"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Autonomous driving in dense urban environments requires accurate trajectory forecasting supported by interpretable contextual evidence. This study presents a multimodal framework that performs driving scene context (DSC)-aware trajectory prediction while providing risk-aware explanations to reveal the contextual cues behind predicted motion. The framework integrates temporal object states\u2014trajectories, velocities, yaw angles, and motion status\u2014with semantic information from forward-facing camera imagery, and is composed of four modules: object behavioral feature extraction, scene context extraction, DSC-augmented trajectory prediction, and risk-aware reasoning using a multimodal large language model (MLLM). Experiments on the Rank2Tell dataset demonstrate the feasibility and applicability of the proposed approach, achieving an ADE of 10.972, an FDE of 13.701, and an RMSE of 8.782. Additional qualitative evaluation shows that DeepSeek-R1-Distill-Qwen-7B generates the most coherent and contextually aligned explanations among the tested models. These findings indicate that combining DSC-aware prediction with interpretable reasoning provides a practical and transparent solution for autonomous driving in complex urban environments.<\/jats:p>","DOI":"10.3390\/s25237112","type":"journal-article","created":{"date-parts":[[2025,11,21]],"date-time":"2025-11-21T14:17:23Z","timestamp":1763734643000},"page":"7112","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["DSC-LLM: Driving Scene Context Representation-Based Trajectory Prediction Framework with Risk Factor Reasoning Using LLMs"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-9088-7036","authenticated-orcid":false,"given":"Sunghun","family":"Kim","sequence":"first","affiliation":[{"name":"Department of AI and Big Data, Soonchunhyang University, 22 Soonchunhyang-ro, Asan 31538, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-1563-2353","authenticated-orcid":false,"given":"Joobin","family":"Jin","sequence":"additional","affiliation":[{"name":"Department of Future Convergence Technology, Soonchunhyang University, 22 Soonchunhyang-ro, Asan 31538, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-2029-8935","authenticated-orcid":false,"given":"Seokjun","family":"Hong","sequence":"additional","affiliation":[{"name":"Department of Future Convergence Technology, Soonchunhyang University, 22 Soonchunhyang-ro, Asan 31538, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9963-1416","authenticated-orcid":false,"given":"Dongho","family":"Ka","sequence":"additional","affiliation":[{"name":"ITS Team, Nota AI Inc., 521 Teheran-ro, Gangnam-gu, Seoul 06164, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6728-0727","authenticated-orcid":false,"given":"Hakjae","family":"Kim","sequence":"additional","affiliation":[{"name":"R&D Department, Class Act Co., Ltd., 123, Gasan Digital 2-ro, Geumcheon-gu, Seoul 08505, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0458-1573","authenticated-orcid":false,"given":"Byeongjoon","family":"Noh","sequence":"additional","affiliation":[{"name":"Department of AI and Big Data, Soonchunhyang University, 22 Soonchunhyang-ro, Asan 31538, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"103570","DOI":"10.1016\/j.trc.2022.103570","article-title":"A novel method of predictive collision risk area estimation for proactive pedestrian accident prevention system in urban surveillance infrastructure","volume":"137","author":"Noh","year":"2022","journal-title":"Transp. Res. Part C: Emerg. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"107938","DOI":"10.1016\/j.aap.2025.107938","article-title":"MineSim: A scenario-based simulation test system and benchmark for autonomous trucks in open-pit mines","volume":"213","author":"Chen","year":"2025","journal-title":"Accid. Anal. Prev."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Su, H., Arakawa, S., and Murata, M. (2025, January 22\u201325). Occlusion-Aware Planning for Connected and Automated Vehicles with Cooperative Perception at Unsignalized Intersection. Proceedings of the 2025 IEEE Intelligent Vehicles Symposium (IV), Cluj-Napoca, Romania.","DOI":"10.1109\/IV64158.2025.11097405"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Nie, J., Jiang, J., Li, Y., and Ercisli, S. (IEEE Trans. Intell. Transp. Syst., 2025). Generative AI-Enhanced Autonomous Driving: Innovating Decision-Making and Risk Assessment in Multi-Interactive Environments, IEEE Trans. Intell. Transp. Syst., early access.","DOI":"10.1109\/TITS.2025.3555774"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zeru, L., and In, K.J. (2025). Autonomous Earthwork Machinery for Urban Construction: A Review of Integrated Control, Fleet Coordination, and Safety Assurance. Buildings, 15.","DOI":"10.3390\/buildings15142570"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"4931","DOI":"10.1038\/s41467-024-48526-4","article-title":"A matched case-control analysis of autonomous vs human-driven vehicle accidents","volume":"15","author":"Ding","year":"2024","journal-title":"Nat. Commun."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"7339","DOI":"10.1109\/TITS.2023.3349198","article-title":"Evolutionary decision-making and planning for autonomous driving: A hybrid augmented intelligence framework","volume":"25","author":"Yuan","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"8334","DOI":"10.1109\/TITS.2024.3374342","article-title":"Behavioral intention prediction in driving scenes: A survey","volume":"25","author":"Fang","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4716","DOI":"10.1109\/TIP.2024.3445734","article-title":"Decouple ego-view motions for predicting pedestrian trajectory and intention","volume":"33","author":"Zhang","year":"2024","journal-title":"IEEE Trans. Image Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3126","DOI":"10.1109\/TPAMI.2025.3529259","article-title":"WAKE: Towards Robust and Physically Feasible Trajectory Prediction for Autonomous Vehicles With WAvelet and KinEmatics Synergy","volume":"47","author":"Wang","year":"2025","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"105115","DOI":"10.1016\/j.trc.2025.105115","article-title":"A learning-based interacting multiple model filter for trajectory prediction of small multirotor drones considering differential sequences","volume":"174","author":"Tang","year":"2025","journal-title":"Transp. Res. Part C: Emerg. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"128222","DOI":"10.1016\/j.eswa.2025.128222","article-title":"Customizable Multimodal Trajectory Prediction via Nodes of Interest Selection for Autonomous Vehicles","volume":"288","author":"Jiang","year":"2025","journal-title":"Expert Syst. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., and Savarese, S. (2016, January 27\u201330). Social lstm: Human trajectory prediction in crowded spaces. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.110"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"111800","DOI":"10.1016\/j.engappai.2025.111800","article-title":"EnTAIL: Evolutional temporal-aware interaction learning for motion forecasting","volume":"160","author":"Liu","year":"2025","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"108059","DOI":"10.1109\/ACCESS.2025.3581916","article-title":"Attention-Driven Lane Change Trajectory Prediction with Traffic Context in Urban Environments","volume":"13","author":"Hong","year":"2025","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"101603","DOI":"10.1109\/ACCESS.2024.3431437","article-title":"Explainable artificial intelligence for autonomous driving: A comprehensive overview and field guide for future research directions","volume":"12","author":"Atakishiyev","year":"2024","journal-title":"IEEE Access"},{"key":"ref_17","unstructured":"Rivera, E., L\u00fcbberstedt, J., Uhlemann, N., and Lienkamp, M. (March, January 28). Scenario Understanding of Traffic Scenes Through Large Visual Language Models. Proceedings of the Winter Conference on Applications of Computer Vision, Tucson, Arizona."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3331","DOI":"10.1109\/TIV.2024.3451350","article-title":"HazardVLM: A Video Language Model for Real-Time Hazard Description in Automated Driving Systems","volume":"10","author":"Xiao","year":"2025","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zeng, T., Wu, L., Shi, L., Zhou, D., and Guo, F. (2025, January 3\u20137). Are vision llms road-ready? A comprehensive benchmark for safety-critical driving video understanding. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, Toronto, ON, Canada.","DOI":"10.1145\/3711896.3737396"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhou, X., Liu, M., Yurtsever, E., Zagar, B.L., Zimmer, W., Cao, H., and Knoll, A.C. (IEEE Trans. Intell. Veh., 2024). Vision language models in autonomous driving: A survey and outlook, IEEE Trans. Intell. Veh., early access.","DOI":"10.1109\/TIV.2024.3402136"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kim, S., Hong, S., Jin, J., and Noh, B. (2025, January 1\u20133). Driving Scene Context-Augmented Trajectory Prediction with Risk-Aware Decision Reasoning Using Multimodal LLM. Proceedings of the EPIA Conference on Artificial Intelligence, Faro, Portugal.","DOI":"10.1007\/978-3-032-05179-0_15"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sachdeva, E., Agarwal, N., Chundi, S., Roelofs, S., Li, J., Kochenderfer, M., Choi, C., and Dariush, B. (2024, January 3\u20138). Rank2tell: A multimodal driving dataset for joint importance ranking and reasoning. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV57701.2024.00734"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"119888","DOI":"10.1016\/j.eswa.2023.119888","article-title":"A trajectory prediction method based on bayonet importance encoding and bidirectional lstm","volume":"223","author":"Guan","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_24","unstructured":"Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF models for sequence tagging. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"106432","DOI":"10.1109\/ACCESS.2024.3435741","article-title":"Predicting lane change and vehicle trajectory with driving micro-data and deep learning","volume":"12","author":"Wang","year":"2024","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"131639","DOI":"10.1109\/ACCESS.2023.3334622","article-title":"Vehicle trajectory prediction based on multivariate interaction modeling","volume":"11","author":"Sun","year":"2023","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Fu, Z., Jiang, K., Xie, C., Xu, Y., Huang, J., and Yang, D. (IEEE Trans. Intell. Veh., 2024). Summary and reflections on pedestrian trajectory prediction in the field of autonomous driving, IEEE Trans. Intell. Veh., early access.","DOI":"10.1109\/TIV.2024.3399327"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5571","DOI":"10.1109\/TIV.2024.3352180","article-title":"Social force embedded mixed graph convolutional network for multi-class trajectory prediction","volume":"9","author":"Du","year":"2024","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"13109","DOI":"10.1109\/TITS.2025.3588855","article-title":"Graph-Based Spatial-Temporal Attentive Network for Vehicle Trajectory Prediction in Automated Driving","volume":"26","author":"Benrachou","year":"2025","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"15087","DOI":"10.1109\/TITS.2025.3585259","article-title":"Self-Interaction Dynamic Graph Convolutional Network With Multiscale Time-Frequency Fusion for Vehicle Trajectory Prediction","volume":"26","author":"Wang","year":"2025","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Pu, Y., Li, Y., Xia, B., Wang, X., Qin, H., and Zhu, L. (2024, January 24\u201327). A Lightweight Lane-Guided Vector Transformer for Multi-Agent Trajectory Prediction in Autonomous Driving. Proceedings of the 2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC), Edmonton, AB, Canada.","DOI":"10.1109\/ITSC58415.2024.10919701"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"28864","DOI":"10.1109\/JIOT.2025.3567291","article-title":"Multi-Modal Vehicle Motion Prediction Based on Motion-Query Social Transformer Network for Internet of Vehicles","volume":"12","author":"Jiang","year":"2025","journal-title":"IEEE Internet Things J."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Cyranka, J., and Haponiuk, S. (2023). Unified Long-Term Time-Series Forecasting Benchmark. arXiv.","DOI":"10.2139\/ssrn.5002162"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Tang, X., Kan, M., Shan, S., Ji, Z., Bai, J., and Chen, X. (2024, January 16\u201322). Hpnet: Dynamic trajectory forecasting with historical prediction attention. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.01445"},{"key":"ref_35","first-page":"102851","article-title":"Vision-based mapping of lane semantics and topology for intelligent vehicles","volume":"111","author":"Tian","year":"2022","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"7381","DOI":"10.1109\/LRA.2024.3426386","article-title":"Semanticformer: Holistic and semantic traffic scene representation for trajectory prediction using knowledge graphs","volume":"9","author":"Sun","year":"2024","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_37","first-page":"19622","article-title":"Large language models are zero-shot time series forecasters","volume":"36","author":"Gruver","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Chen, L., Sinavski, O., H\u00fcnermann, J., Karnsund, A., Willmott, A.J., Birch, D., Maund, D., and Shotton, J. (2024, January 13\u201317). Driving with llms: Fusing object-level vector modality for explainable autonomous driving. Proceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), Atlanta, GA, USA.","DOI":"10.1109\/ICRA57147.2024.10611018"},{"key":"ref_39","unstructured":"Chib, P.S., and Singh, P. (2024). Lg-traj: Llm guided pedestrian trajectory prediction. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s44267-025-00074-1","article-title":"Enhancing human-centered dynamic scene understanding via multiple llms collaborated reasoning","volume":"3","author":"Zhang","year":"2025","journal-title":"Vis. Intell."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"8186","DOI":"10.1109\/LRA.2024.3440097","article-title":"Drivegpt4: Interpretable end-to-end autonomous driving via large language model","volume":"9","author":"Xu","year":"2024","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_42","unstructured":"Luo, X., Ding, F., Panda, R., Chen, R., Loo, J., and Zhang, S. (March, January 28). \u201c What\u2019s Happening\u201d\u2014A Human-centered Multimodal Interpreter Explaining the Actions of Autonomous Vehicles. Proceedings of the Winter Conference on Applications of Computer Vision, Tucson, Arizona."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"9780","DOI":"10.1109\/TITS.2023.3273547","article-title":"Nle-dm: Natural-language explanations for decision making of autonomous driving based on semantic scene understanding","volume":"24","author":"Feng","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"100170","DOI":"10.1016\/j.commtr.2025.100170","article-title":"Lc-llm: Explainable lane-change intention and trajectory predictions with large language models","volume":"5","author":"Peng","year":"2025","journal-title":"Commun. Transp. Res."},{"key":"ref_45","unstructured":"Sima, C., Renz, K., Chitta, K., Chen, L., Zhang, H., Xie, C., Bei\u00dfwenger, J., Luo, P., Geiger, A., and Li, H. (October, January 29). Drivelm: Driving with graph visual question answering. Proceedings of the European Conference on Computer Vision, Milano, Italy."},{"key":"ref_46","unstructured":"Chen, Z., Xu, S., Ye, M., Qian, Z., Zou, X., Yeung, D.Y., and Chen, Q. (October, January 29). Learning High-Resolution Vector Representation from Multi-camera Images for 3D Object Detection. Proceedings of the European Conference on Computer Vision, Milano, Italy."},{"key":"ref_47","first-page":"3","article-title":"Lora: Low-rank adaptation of large language models","volume":"1","author":"Hu","year":"2022","journal-title":"ICLR"},{"key":"ref_48","unstructured":"Tian, R., Li, B., Weng, X., Chen, Y., Schmerling, E., Wang, Y., Ivanovic, B., and Pavone, M. (2024). Tokenize the world into object-level knowledge to address long-tail events in autonomous driving. arXiv."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"8040","DOI":"10.1109\/TIV.2024.3406372","article-title":"Delving into multi-modal multi-task foundation models for road scene understanding: From learning paradigm perspectives","volume":"9","author":"Luo","year":"2024","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_50","unstructured":"Sadat, A., Casas, S., Ren, M., Wu, X., Dhawan, P., and Urtasun, R. (October, January 29). Perceive, predict, and plan: Safe motion planning through interpretable semantic representations. Proceedings of the European Conference on Computer Vision, Milano, Italy."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Sun, M., Li, J., Liu, J., and Yang, X. (2024, January 1\u20135). MTPI: Multimodal Trajectory Prediction for Autonomous Driving via Informer. Proceedings of the 2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C), Cambridge, UK.","DOI":"10.1109\/QRS-C63300.2024.00077"},{"key":"ref_52","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18\u201324). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"2513912","DOI":"10.1109\/TIM.2022.3192056","article-title":"Vehicle trajectory prediction based on intention-aware non-autoregressive transformer with multi-attention learning for Internet of Vehicles","volume":"71","author":"Chen","year":"2022","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_54","unstructured":"Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., and Bi, X. (2025). Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv."},{"key":"ref_55","unstructured":"Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., and Vaughan, A. (2024). The llama 3 herd of models. arXiv."},{"key":"ref_56","unstructured":"Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., and Saulnier, L. (2023). Mistral 7B. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/23\/7112\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,21]],"date-time":"2025-11-21T14:18:04Z","timestamp":1763734684000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/23\/7112"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,21]]},"references-count":56,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["s25237112"],"URL":"https:\/\/doi.org\/10.3390\/s25237112","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,21]]}}}