{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T20:21:56Z","timestamp":1773778916129,"version":"3.50.1"},"reference-count":48,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2024,10,6]],"date-time":"2024-10-06T00:00:00Z","timestamp":1728172800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["2020AAA0108105"],"award-info":[{"award-number":["2020AAA0108105"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["20220301033GX"],"award-info":[{"award-number":["20220301033GX"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["2105"],"award-info":[{"award-number":["2105"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["2023C034-3"],"award-info":[{"award-number":["2023C034-3"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Major Science and Technology Projects of Jilin Province and Changchun City","award":["2020AAA0108105"],"award-info":[{"award-number":["2020AAA0108105"]}]},{"name":"Major Science and Technology Projects of Jilin Province and Changchun City","award":["20220301033GX"],"award-info":[{"award-number":["20220301033GX"]}]},{"name":"Major Science and Technology Projects of Jilin Province and Changchun City","award":["2105"],"award-info":[{"award-number":["2105"]}]},{"name":"Major Science and Technology Projects of Jilin Province and Changchun City","award":["2023C034-3"],"award-info":[{"award-number":["2023C034-3"]}]},{"name":"Shanghai Automobile Industry Technology Development Foundation","award":["2020AAA0108105"],"award-info":[{"award-number":["2020AAA0108105"]}]},{"name":"Shanghai Automobile Industry Technology Development Foundation","award":["20220301033GX"],"award-info":[{"award-number":["20220301033GX"]}]},{"name":"Shanghai Automobile Industry Technology Development Foundation","award":["2105"],"award-info":[{"award-number":["2105"]}]},{"name":"Shanghai Automobile Industry Technology Development Foundation","award":["2023C034-3"],"award-info":[{"award-number":["2023C034-3"]}]},{"name":"Development and Reform Commission Foundation of Jilin Province","award":["2020AAA0108105"],"award-info":[{"award-number":["2020AAA0108105"]}]},{"name":"Development and Reform Commission Foundation of Jilin Province","award":["20220301033GX"],"award-info":[{"award-number":["20220301033GX"]}]},{"name":"Development and Reform Commission Foundation of Jilin Province","award":["2105"],"award-info":[{"award-number":["2105"]}]},{"name":"Development and Reform Commission Foundation of Jilin Province","award":["2023C034-3"],"award-info":[{"award-number":["2023C034-3"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Ensuring the safety of autonomous vehicles (AVs) in complex and high-risk traffic scenarios remains a critical unresolved challenge. Current AV planning methods exhibit limitations in generating robust driving trajectories that effectively avoid collisions, highlighting the urgent need for improved planning strategies to address these issues. This paper introduces a novel iterative optimization framework that incorporates safety-critical trajectory generation to enhance AV planning. The use of the HighD dataset, which is collected using the wide-area remote sensing capabilities of unmanned aerial vehicles (UAVs), is fundamental to the framework. Remote sensing enables large-scale real-time observation of traffic conditions, providing precise data on vehicle dynamics, road structures, and surrounding environments. To generate safety-critical trajectories, the decoder within the conditional variational auto-encoder (CVAE) is innovatively designed through a data-mechanism integration method, ensuring that these trajectories strictly adhere to vehicle kinematic constraints. Furthermore, two parallel CVAEs (Dual-CVAE) are trained collaboratively by a shared objective function to implicitly model the multi-vehicle interactions. Inspired by the concept of \u201clearning to collide\u201d, adversarial optimization is integrated into the Dual-CVAE (Adv. Dual-CVAE), facilitating efficient generation from normal to safety-critical trajectories. Building upon this, these generated trajectories are then incorporated into an iterative optimization framework, significantly enhancing the AV\u2019s planning ability to avoid collisions. This framework decomposes the optimization process into stages, initially addressing normal trajectories and progressively tackling more safety-critical and collision trajectories. Finally, comparative case studies of enhancing AV planning are conducted and the simulation results demonstrate that the proposed method can efficiently enhance AV planning by generating safety-critical trajectories.<\/jats:p>","DOI":"10.3390\/rs16193721","type":"journal-article","created":{"date-parts":[[2024,10,7]],"date-time":"2024-10-07T07:30:18Z","timestamp":1728286218000},"page":"3721","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Enhancing Planning for Autonomous Driving via an Iterative Optimization Framework Incorporating Safety-Critical Trajectory Generation"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-2061-6998","authenticated-orcid":false,"given":"Zhen","family":"Liu","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence, Jilin University, Changchun 130012, China"}]},{"given":"Hang","family":"Gao","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Jilin University, Changchun 130012, China"}]},{"given":"Yeting","family":"Lin","sequence":"additional","affiliation":[{"name":"SAIC Motor R&D Innovation Headquarters, Shanghai 200030, China"}]},{"given":"Xun","family":"Gong","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Jilin University, Changchun 130012, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,10,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1038\/s41586-023-05732-2","article-title":"Dense reinforcement learning for safety validation of autonomous vehicles","volume":"615","author":"Feng","year":"2023","journal-title":"Nature"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"748","DOI":"10.1038\/s41467-021-21007-8","article-title":"Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment","volume":"12","author":"Feng","year":"2021","journal-title":"Nat. Commun."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"493","DOI":"10.3390\/jcp3030025","article-title":"Autonomous vehicles: Sophisticated attacks, safety issues, challenges, open topics, blockchain, and future directions","volume":"3","author":"Giannaros","year":"2023","journal-title":"J. Cybersecur. Priv."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1213","DOI":"10.1109\/TITS.2020.3023668","article-title":"Testing scenario library generation for connected and automated vehicles: An adaptive framework","volume":"23","author":"Feng","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2062","DOI":"10.1109\/TIV.2023.3236150","article-title":"Integrated post-impact planning and active safety control for autonomous vehicles","volume":"8","author":"Wang","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.eng.2023.10.005","article-title":"Toward trustworthy decision-making for autonomous vehicles: A robust reinforcement learning approach with safety guarantees","volume":"33","author":"He","year":"2024","journal-title":"Engineering"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3692","DOI":"10.1109\/TIV.2023.3274536","article-title":"Motion planning for autonomous driving: The state of the art and future perspectives","volume":"8","author":"Teng","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Hanselmann, N., Renz, K., Chitta, K., Bhattacharyya, A., and Geiger, A. (2022). King: Generating safety-critical driving scenarios for robust imitation via kinematics gradients. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-031-19839-7_20"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Chowdhury, J., Shivaraman, V., Sundaram, S., and Sujit, P. (2024, January 20\u201327). Graph-based Prediction and Planning Policy Network (GP3Net) for scalable self-driving in dynamic environments using Deep Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i10.29043"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1016\/j.trc.2017.01.003","article-title":"Fine-tuning ADAS algorithm parameters for optimizing traffic safety and mobility in connected vehicle environment","volume":"76","author":"Liu","year":"2017","journal-title":"Transp. Res. Part C Emerg. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"972","DOI":"10.1109\/TCST.2019.2949540","article-title":"Optimization-based collision avoidance","volume":"29","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, J., Pun, A., Tu, J., Manivasagam, S., Sadat, A., Casas, S., Ren, M., and Urtasun, R. (2021, January 20\u201325). Advsim: Generating safety-critical scenarios for self-driving vehicles. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00978"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wang, Z., Li, X., Wei, D., Wang, L., and Huang, Y. (2024). Efficient Generation of Safety-Critical Scenarios Combining Dynamic and Static Scenario Parameters. IEEE Trans. Intell. Veh.","DOI":"10.1109\/TIV.2024.3402221"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"798","DOI":"10.1109\/TRO.2020.3036624","article-title":"Fail-safe motion planning for online verification of autonomous vehicles using convex optimization","volume":"37","author":"Pek","year":"2020","journal-title":"IEEE Trans. Robot."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"104211","DOI":"10.1016\/j.engappai.2021.104211","article-title":"Recent advances in motion and behavior planning techniques for software architecture of autonomous vehicles: A state-of-the-art survey","volume":"101","author":"Sharma","year":"2021","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"6971","DOI":"10.1109\/TITS.2023.3259322","article-title":"A survey on safety-critical driving scenario generation\u2014A methodological perspective","volume":"24","author":"Ding","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"105893","DOI":"10.1016\/j.engappai.2023.105893","article-title":"Search-based task and motion planning for hybrid systems: Agile autonomous vehicles","volume":"121","author":"Regolin","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhang, C., Tang, Z., Zhang, M., Wang, B., and Hou, L. (2022). Developing a more reliable aerial photography-based method for acquiring freeway traffic data. Remote Sens., 14.","DOI":"10.3390\/rs14092202"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Krajewski, R., Bock, J., Kloeker, L., and Eckstein, L. (2018, January 4\u20137). The highD dataset: A drone dataset of naturalistic vehicle trajectories on german highways for validation of highly automated driving systems. Proceedings of the 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569552"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Rempe, D., Philion, J., Guibas, L.J., Fidler, S., and Litany, O. (2022, January 18\u201324). Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01679"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"10333","DOI":"10.1109\/TITS.2021.3091477","article-title":"Adversarial evaluation of autonomous vehicles in lane-change scenarios","volume":"23","author":"Chen","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_22","unstructured":"Yan, X., Feng, S., Sun, H., and Liu, H.X. (2021). Distributionally consistent simulation of naturalistic driving environment for autonomous vehicle testing. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Abeysirigoonawardena, Y., Shkurti, F., and Dudek, G. (2019, January 20\u201324). Generating adversarial driving scenarios in high-fidelity simulators. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793740"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1007\/s10846-020-01215-y","article-title":"Trajectory planning of autonomous vehicles based on parameterized control optimization in dynamic on-road environments","volume":"100","author":"Zhu","year":"2020","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"344","DOI":"10.3390\/vehicles4020021","article-title":"Motion planning for autonomous vehicles based on sequential optimization","volume":"4","author":"Diachuk","year":"2022","journal-title":"Vehicles"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"5435","DOI":"10.1109\/TNNLS.2021.3084685","article-title":"Safe reinforcement learning with stability guarantee for motion planning of autonomous vehicles","volume":"32","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1109\/TITS.2020.3024655","article-title":"Survey of deep reinforcement learning for motion planning of autonomous vehicles","volume":"23","author":"Aradi","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Fisac, J.F., Bronstein, E., Stefansson, E., Sadigh, D., Sastry, S.S., and Dragan, A.D. (2019, January 20\u201324). Hierarchical game-theoretic planning for autonomous vehicles. Proceedings of the 2019 International conference on robotics and automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794007"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"4572","DOI":"10.1109\/TIV.2023.3321346","article-title":"A multi-vehicle game-theoretic framework for decision making and planning of autonomous vehicles in mixed traffic","volume":"8","author":"Yan","year":"2023","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"10248","DOI":"10.1109\/LRA.2022.3191241","article-title":"Efficient game-theoretic planning with prediction heuristic for socially-compliant autonomous driving","volume":"7","author":"Li","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_31","unstructured":"Pinto, L., Davidson, J., Sukthankar, R., and Gupta, A. (2017, January 6\u201311). Robust adversarial reinforcement learning. Proceedings of the International Conference on Machine Learning. PMLR, Sydney, Australia."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1782","DOI":"10.1109\/TCST.2017.2723574","article-title":"Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems","volume":"26","author":"Li","year":"2017","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2211","DOI":"10.1109\/TITS.2020.3035363","article-title":"Game-theoretic modeling of traffic in unsignalized intersection network for autonomous vehicle control verification and validation","volume":"23","author":"Tian","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Lopez-Martin, M., Carro, B., Sanchez-Esguevillas, A., and Lloret, J. (2017). Conditional variational autoencoder for prediction and feature recovery applied to intrusion detection in iot. Sensors, 17.","DOI":"10.3390\/s17091967"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zhang, Z., and Zhu, L. (2023). A review on unmanned aerial vehicle remote sensing: Platforms, sensors, data processing methods, and applications. Drones, 7.","DOI":"10.3390\/drones7060398"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"179424","DOI":"10.1109\/ACCESS.2020.3026658","article-title":"VNet: An end-to-end fully convolutional neural network for road extraction from high-resolution remote sensing data","volume":"8","author":"Abdollahi","year":"2020","journal-title":"IEEE Access"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1720","DOI":"10.1007\/s12083-020-00879-5","article-title":"Vehicles joint UAVs to acquire and analyze data for topology discovery in large-scale IoT systems","volume":"13","author":"Teng","year":"2020","journal-title":"Peer-Netw. Appl."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"9654","DOI":"10.1109\/JSTARS.2022.3216564","article-title":"A comprehensive review for typical applications based upon unmanned aerial vehicle platform","volume":"15","author":"Han","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Yao, H., Qin, R., and Chen, X. (2019). Unmanned aerial vehicle for remote sensing applications\u2014A review. Remote Sens., 11.","DOI":"10.3390\/rs11121443"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"9060797","DOI":"10.1155\/2019\/9060797","article-title":"Extracting vehicle trajectories using unmanned aerial vehicles in congested traffic conditions","volume":"2019","author":"Kim","year":"2019","journal-title":"J. Adv. Transp."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Jim\u00e9nez-Jim\u00e9nez, S.I., Ojeda-Bustamante, W., Marcial-Pablo, M.d.J., and Enciso, J. (2021). Digital terrain models generated with low-cost UAV photogrammetry: Methodology and accuracy. ISPRS Int. J. Geo-Inf., 10.","DOI":"10.3390\/ijgi10050285"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1080\/19427867.2019.1650430","article-title":"An evaluation of time-to-collision as a surrogate safety measure and a proposal of a new method for its application in safety analysis","volume":"12","author":"Nadimi","year":"2020","journal-title":"Transp. Lett."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Hochreiter, S. (1997). Long Short-term Memory. Neural Computation, MIT-Press.","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Kong, J., Pfeiffer, M., Schildbach, G., and Borrelli, F. (July, January 28). Kinematic and dynamic vehicle models for autonomous driving control design. Proceedings of the 2015 IEEE Intelligent Vehicles Symposium (IV), Seoul, Republic of Korea.","DOI":"10.1109\/IVS.2015.7225830"},{"key":"ref_45","unstructured":"Kingma, D.P. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"04021083","DOI":"10.1061\/JTEPBS.0000584","article-title":"Modeling car-following behavior on freeways considering driving style","volume":"147","author":"Sun","year":"2021","journal-title":"J. Transp. Eng. Part A Syst."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"3113","DOI":"10.1109\/TVT.2021.3063483","article-title":"Online vehicle velocity prediction using an adaptive radial basis function neural network","volume":"70","author":"Hou","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Feng, X., Cen, Z., Hu, J., and Zhang, Y. (2019, January 27\u201330). Vehicle trajectory prediction using intention-based conditional variational autoencoder. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.","DOI":"10.1109\/ITSC.2019.8917482"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/19\/3721\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:11:48Z","timestamp":1760112708000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/19\/3721"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,6]]},"references-count":48,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2024,10]]}},"alternative-id":["rs16193721"],"URL":"https:\/\/doi.org\/10.3390\/rs16193721","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,6]]}}}