{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T12:59:37Z","timestamp":1780491577963,"version":"3.54.1"},"reference-count":42,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2024,6,6]],"date-time":"2024-06-06T00:00:00Z","timestamp":1717632000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2021YFB2501403"],"award-info":[{"award-number":["2021YFB2501403"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Imitation learning has shown promise for autonomous driving, but suffers from covariate shift, where the policy performs poorly in unseen environments. DAgger is a popular approach that addresses this by leveraging expert demonstrations. However, DAgger\u2019s frequent visits to sub-optimal states can lead to several challenges. This paper proposes a novel DAgger framework that integrates Bayesian uncertainty estimation via mean field variational inference (MFVI) to address this issue. MFVI provides better-calibrated uncertainty estimates compared to prior methods. During training, the framework identifies both uncertain and critical states, querying the expert only for these states. This targeted data collection reduces the burden on the expert and improves data efficiency. Evaluations on the CARLA simulator demonstrate that our approach outperforms existing methods, highlighting the effectiveness of Bayesian uncertainty estimation and targeted data aggregation for imitation learning in autonomous driving.<\/jats:p>","DOI":"10.3390\/info15060336","type":"journal-article","created":{"date-parts":[[2024,6,6]],"date-time":"2024-06-06T06:40:26Z","timestamp":1717656026000},"page":"336","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Uncertainty-Driven Data Aggregation for Imitation Learning in Autonomous Vehicles"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-2978-1017","authenticated-orcid":false,"given":"Changquan","family":"Wang","sequence":"first","affiliation":[{"name":"Institute of Microelectronics of the Chinese Academy of Sciences, Beijing 100029, China"},{"name":"University of Chinese Academy of Sciences, Beijing 101408, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9390-9226","authenticated-orcid":false,"given":"Yun","family":"Wang","sequence":"additional","affiliation":[{"name":"Institute of Microelectronics of the Chinese Academy of Sciences, Beijing 100029, China"},{"name":"University of Chinese Academy of Sciences, Beijing 101408, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Codevilla, F., Miiller, M., L\u00f3pez, A., Koltun, V., and Dosovitskiy, A. (2018, January 21\u201325). End-to-end driving via conditional imitation learning. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460487"},{"key":"ref_2","unstructured":"Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., and Zhang, J. (2016). End to end learning for self-driving cars. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Prakash, A., Chitta, K., and Geiger, A. (2021, January 19\u201325). Multi-modal fusion transformer for end-to-end autonomous driving. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00700"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"537","DOI":"10.1109\/TITS.2020.3013234","article-title":"Multimodal end-to-end autonomous driving","volume":"23","author":"Xiao","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Yang, Z., Zhang, Y., Yu, J., Cai, J., and Luo, J. (2018, January 20\u201324). End-to-end multi-modal multi-task vehicle control for self-driving cars with visual perceptions. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.","DOI":"10.1109\/ICPR.2018.8546189"},{"key":"ref_6","unstructured":"Brantley, K., Sun, W., and Henaff, M. (2019, January 6\u20139). Disagreement-regularized imitation learning. Proceedings of the International Conference on Learning Representations, New Orleans, LO, USA."},{"key":"ref_7","first-page":"2914","article-title":"Toward the fundamental limits of imitation learning","volume":"33","author":"Rajaraman","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_8","unstructured":"Ross, S., Gordon, G., and Bagnell, D. (2011, January 11\u201313). A reduction of imitation learning and structured prediction to no-regret online learning. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, Fort Lauderdale, FL, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Mullins, G.E., Dress, A.G., Stankiewicz, P.G., Appler, J.D., and Gupta, S.K. (2018, January 21\u201325). Accelerated testing and evaluation of autonomous vehicles via imitation learning. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460965"},{"key":"ref_10","unstructured":"Ross, S., and Bagnell, J.A. (2014). Reinforcement and imitation learning via interactive no-regret learning. arXiv."},{"key":"ref_11","unstructured":"Sun, W., Venkatraman, A., Gordon, G.J., Boots, B., and Bagnell, J.A. (2017, January 6\u201311). Deeply aggrevated: Differentiable imitation learning for sequential prediction. Proceedings of the International Conference on Machine Learning, PMLR, Sydney, NSW, Australia."},{"key":"ref_12","unstructured":"Monfort, M., Johnson, M., Oliva, A., and Hofmann, K. (2017, January 8\u201312). Asynchronous data aggregation for training end to end visual control networks. Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, Sao Paulo, Brazil."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Prakash, A., Behl, A., Ohn-Bar, E., Chitta, K., and Geiger, A. (2020, January 14\u201319). Exploring data aggregation in policy learning for vision-based urban autonomous driving. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, DC, USA.","DOI":"10.1109\/CVPR42600.2020.01178"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Liniger, A., Dai, D., Yu, F., and Van Gool, L. (2021, January 11\u201317). End-to-end urban driving by imitating a reinforcement learning coach. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Virtual.","DOI":"10.1109\/ICCV48922.2021.01494"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, J., and Cho, K. (2016). Query-efficient imitation learning for end-to-end autonomous driving. arXiv.","DOI":"10.1609\/aaai.v31i1.10857"},{"key":"ref_16","unstructured":"Laskey, M., Powers, C., Joshi, R., Poursohi, A., and Goldberg, K. (2017). Learning robust bed making using deep imitation learning with dart. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Menda, K., Driggs-Campbell, K., and Kochenderfer, M.J. (2019, January 3\u20138). Ensembledagger: A bayesian approach to safe imitation learning. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.","DOI":"10.1109\/IROS40897.2019.8968287"},{"key":"ref_18","unstructured":"Menda, K., Driggs-Campbell, K., and Kochenderfer, M.J. (2017). Dropoutdagger: A bayesian approach to safe imitation learning. arXiv."},{"key":"ref_19","unstructured":"Gal, Y., and Ghahramani, Z. (2016, January 19\u201324). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. Proceedings of the International Conference on Machine Learning, PMLR, New York, NY, USA."},{"key":"ref_20","unstructured":"Cronrath, C., Jorge, E., Moberg, J., Jirstrand, M., and Lennartson, B. (2018, January 5). BAgger: A Bayesian algorithm for safe and query-efficient imitation learning. Proceedings of the Machine Learning in Robot Motion Planning\u2014IROS 2018 Workshop, Madrid, Spain."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Cui, Y., Isele, D., Niekum, S., and Fujimura, K. (2019, January 20\u201324). Uncertainty-aware data aggregation for deep imitation learning. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794025"},{"key":"ref_22","first-page":"4697","article-title":"Bayesian deep learning and a probabilistic perspective of generalization","volume":"33","author":"Wilson","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Michelmore, R., Wicker, M., Laurenti, L., Cardelli, L., Gal, Y., and Kwiatkowska, M. (August, January 31). Uncertainty quantification with statistical guarantees in end-to-end autonomous driving control. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9196844"},{"key":"ref_24","unstructured":"Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. (2015, January 6\u201311). Weight uncertainty in neural network. Proceedings of the International Conference on Machine Learning, PMLR, Lille, France."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1023\/A:1007665907178","article-title":"An introduction to variational methods for graphical models","volume":"37","author":"Jordan","year":"1999","journal-title":"Mach. Learn."},{"key":"ref_26","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13\u201315). CARLA: An Open Urban Driving Simulator. Proceedings of the 1st Annual Conference on Robot Learning, Mountain View, CA, USA."},{"key":"ref_27","unstructured":"Codevilla, F., Santana, E., L\u00f3pez, A.M., and Gaidon, A. (November, January 27). Exploring the limitations of behavior cloning for autonomous driving. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_28","unstructured":"Leaderboard, C. (2024, March 16). CARLA Leaderboard. Available online: https:\/\/leaderboard.carla.org\/."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3054912","article-title":"Imitation learning: A survey of learning methods","volume":"50","author":"Hussein","year":"2017","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"375","DOI":"10.2307\/2669318","article-title":"Estimation and inference via Bayesian simulation: An introduction to Markov chain Monte Carlo","volume":"44","author":"Jackman","year":"2000","journal-title":"Am. J. Political Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2008","DOI":"10.1109\/TPAMI.2018.2889774","article-title":"Advances in variational inference","volume":"41","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"6964","DOI":"10.1109\/JSEN.2022.3154980","article-title":"Camera, lidar, and radar sensor fusion based on bayesian neural network (clr-bnn)","volume":"22","author":"Ravindran","year":"2022","journal-title":"IEEE Sens. J."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Verstraete, D., Droguett, E., and Modarres, M. (2019). A deep adversarial approach based on multi-sensor fusion for semi-supervised remaining useful life prognostics. Sensors, 20.","DOI":"10.3390\/s20010176"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"11238","DOI":"10.1109\/JIOT.2020.2996564","article-title":"Indoor radio map construction and localization with deep Gaussian processes","volume":"7","author":"Wang","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1109\/TSMC.2022.3186639","article-title":"Vehicle trajectory prediction considering driver uncertainty and vehicle dynamics based on dynamic bayesian network","volume":"53","author":"Jiang","year":"2022","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Brechtel, S., Gindele, T., and Dillmann, R. (2014, January 8\u201311). Probabilistic decision-making under uncertainty for autonomous driving using continuous POMDPs. Proceedings of the 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), Qingdao, China.","DOI":"10.1109\/ITSC.2014.6957722"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hoel, C.J., Wolff, K., and Laine, L. (November, January 19). Tactical decision-making in autonomous driving by reinforcement learning with uncertainty estimation. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA.","DOI":"10.1109\/IV47402.2020.9304614"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Ma, J., Xie, H., Song, K., and Liu, H. (2021). A bayesian driver agent model for autonomous vehicles system based on knowledge-aware and real-time data. Sensors, 21.","DOI":"10.3390\/s21020331"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/S0001-4575(00)00019-1","article-title":"Extended time-to-collision measures for road traffic safety assessment","volume":"33","author":"Minderhoud","year":"2001","journal-title":"Accid. Anal. Prev."},{"key":"ref_40","unstructured":"Van der Horst, A.R.A. (1991). A Time-Based Analysis of Road User Behaviour in Normal and Critical Encounters, TU Delft Library."},{"key":"ref_41","first-page":"669","article-title":"Comparing time to collision and time headway as safety criteria","volume":"27","author":"Nejad","year":"2020","journal-title":"Pamukkale \u00dcniversitesi M\u00fchendislik Bilim. Derg."},{"key":"ref_42","first-page":"1254","article-title":"Can you trust your model\u2019s uncertainty? evaluating predictive uncertainty under dataset shift","volume":"32","author":"Ovadia","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/6\/336\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:54:43Z","timestamp":1760108083000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/6\/336"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,6]]},"references-count":42,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["info15060336"],"URL":"https:\/\/doi.org\/10.3390\/info15060336","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,6]]}}}