{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T17:33:43Z","timestamp":1787506423708,"version":"build-2736575974"},"reference-count":34,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T00:00:00Z","timestamp":1770076800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003052","name":"Ministry of Trade, Industry and Energy","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003052","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003661","name":"Korea Institute of Advancement of Technology","doi-asserted-by":"crossref","award":["P0026022"],"award-info":[{"award-number":["P0026022"]}],"id":[{"id":"10.13039\/501100003661","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Autonomous driving systems rely on vast and diverse datasets for robust object recognition. However, acquiring real-world data, especially for rare and hazardous scenarios, is prohibitively expensive and risky. While purely synthetic data offers flexibility, it often suffers from a significant reality gap due to discrepancies in visual fidelity and physics. To address these challenges, this paper proposes a novel real\u2013virtual fusion framework for efficiently generating highly realistic augmented image datasets for autonomous driving. Our methodology leverages real-world driving data from South Korea\u2019s K-City, synchronizing it with a digital twin environment in Morai Sim (v24.R2) through a robust look-up table and fine-tuned localization approach. We then seamlessly inject diverse virtual objects (e.g., pedestrians, vehicles, traffic lights) into real image backgrounds. A critical contribution is our focus on inconsistency mitigation, employing advanced techniques such as illumination matching during virtual object injection to minimize visual discrepancies. We evaluate the proposed approach through experiments. Our results show that this real\u2013virtual fusion strategy significantly bridges the reality gap, providing a cost-effective and safe solution for enriching autonomous driving datasets and improving the generalization capabilities of perception models.<\/jats:p>","DOI":"10.3390\/s26030987","type":"journal-article","created":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T12:46:11Z","timestamp":1770122771000},"page":"987","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Toward Realistic Autonomous Driving Dataset Augmentation: A Real\u2013Virtual Fusion Approach with Inconsistency Mitigation"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-3214-2255","authenticated-orcid":false,"given":"Sukwoo","family":"Jung","sequence":"first","affiliation":[{"name":"Contents Convergence Research Center, Korea Electronics Technology Institute, Seongnam-si 13449, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9565-2463","authenticated-orcid":false,"given":"Myeongseop","family":"Kim","sequence":"additional","affiliation":[{"name":"Contents Convergence Research Center, Korea Electronics Technology Institute, Seongnam-si 13449, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9709-2658","authenticated-orcid":false,"given":"Jean","family":"Oh","sequence":"additional","affiliation":[{"name":"Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7844-0100","authenticated-orcid":false,"given":"Jonghwa","family":"Kim","sequence":"additional","affiliation":[{"name":"MORAI Inc., Seoul 06168, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5048-9268","authenticated-orcid":false,"given":"Kyung-Taek","family":"Lee","sequence":"additional","affiliation":[{"name":"Contents Convergence Research Center, Korea Electronics Technology Institute, Seongnam-si 13449, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,3]]},"reference":[{"key":"ref_1","unstructured":"Yu, B., Yuan, C., Wan, Z., Tang, J., Kurdahi, F., and Liu, S. (2025). ADDT\u2014A Digital Twin Framework for Proactive Safety Validation in Autonomous Driving Systems. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Jung, S., Cho, Y., Kim, D., and Chang, M. (2019). Moving object detection from moving camera image sequences using an inertial measurement unit sensor. Appl. Sci., 10.","DOI":"10.3390\/app10010268"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1049","DOI":"10.1007\/s12541-021-00527-9","article-title":"Moving object detection with single moving camera and IMU sensor using mask R-CNN instance image segmentation","volume":"22","author":"Jung","year":"2021","journal-title":"Int. J. Precis. Eng. Manuf."},{"key":"ref_4","first-page":"3011","article-title":"Pose Tracking of Moving Sensor using Monocular Camera and IMU Sensor","volume":"15","author":"Jung","year":"2021","journal-title":"KSII Trans. Internet Inf. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wang, Z., Huang, X., and Hu, Z. (2025). Attention-Based LiDAR\u2013Camera Fusion for 3D Object Detection in Autonomous Driving. World Electr. Veh. J., 16.","DOI":"10.3390\/wevj16060306"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, C.Y., Bochkovskiy, A., and Liao, H.Y.M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv.","DOI":"10.1109\/CVPR52729.2023.00721"},{"key":"ref_7","unstructured":"Jocher, G., Chaurasia, A., and Qiu, J. (2025, December 23). Ultralytics YOLOv8, 2023. Available online: https:\/\/docs.ultralytics.com\/models\/yolov8\/."},{"key":"ref_8","unstructured":"Jocher, G., and Qiu, J. (2025, December 23). Ultralytics YOLO11, 2024. Available online: https:\/\/docs.ultralytics.com\/models\/yolo11\/."},{"key":"ref_9","unstructured":"Wang, A., Liu, L., Chen, H., Lin, Z., Han, J., and Ding, G. (2025). YOLOE: Real-Time Seeing Anything. arXiv."},{"key":"ref_10","unstructured":"Wang, Y., Zhao, N., and Lee, G.H. (2024, January 25\u201328). Syn-to-Real Unsupervised Domain Adaptation for Indoor 3D Object Detection. Proceedings of the 35th British Machine Vision Conference (BMVC), Glasgow, UK."},{"key":"ref_11","unstructured":"Kim, J., Lee, J., Han, G., Lee, D., Jeong, M., and Kim, J. (2025). SynAD: Enhancing Real-World End-to-End Autonomous Driving Models through Synthetic Data Integration. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Goedicke, D., Bremers, A.W., Lee, S., Bu, F., Yasuda, H., and Ju, W. (May, January 30). XR-OOM: MiXed Reality driving simulation with real cars for research and design. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA.","DOI":"10.1145\/3491102.3517704"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Reway, F., Huber, W., and Ribeiro, E.P. (2018, January 1\u201312). Test Methodology for Vision-Based ADAS Algorithms with an Automotive Camera-in-the-Loop. Proceedings of the 2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES), Madrid, Spain.","DOI":"10.1109\/ICVES.2018.8519598"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Perez-Segui, R., Arias-Perez, P., Melero-Deza, J., Fernandez-Cortizas, M., Perez-Saura, D., and Campoy, P. (2023). Bridging the Gap between Simulation and Real Autonomous UAV Flights in Industrial Applications. Aerospace, 10.","DOI":"10.3390\/aerospace10090814"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Cao, X., Chen, H., Gelbal, S.Y., Aksun-Guvenc, B., and Guvenc, L. (2023). Vehicle-in-Virtual-Environment (VVE) Method for Autonomous Driving System Development, Evaluation and Demonstration. Sensors, 23.","DOI":"10.3390\/s23115088"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5778","DOI":"10.1109\/TPAMI.2025.3553051","article-title":"Generating Inverse Feature Space for Class Imbalance in Point Cloud Semantic Segmentation","volume":"47","author":"Han","year":"2025","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"9188","DOI":"10.1109\/TCE.2025.3587291","article-title":"Improved Semantic Segmentation With Large-Scale Attention-Based Self-Supervised Few-Shot Learning","volume":"71","author":"Jing","year":"2025","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_18","unstructured":"He, Y., Chen, H., and Shi, W. (2024). An Advanced Framework for Ultra-Realistic Simulation and Digital Twinning for Autonomous Vehicles. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Barab\u00e1s, I., Iclodean, C., Beles, H., Antonya, C., Molea, A., and Scurt, F.B. (2025). An Approach to Modeling and Developing Virtual Sensors Used in the Simulation of Autonomous Vehicles. Sensors, 25.","DOI":"10.20944\/preprints202503.1400.v1"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liang, Z., Wang, J., Zhang, T., and Yong, X. (2025). DTTF-Sim: A Digital Twin-Based Simulation System for Continuous Autonomous Driving Testing. Sensors, 25.","DOI":"10.3390\/s25113447"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, T., Liu, H., Wang, W., and Wang, X. (2024). Virtual Tools for Testing Autonomous Driving: A Survey and Benchmark of Simulators, Datasets, and Competitions. Electronics, 13.","DOI":"10.3390\/electronics13173486"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Xu, M., Niyato, D., Chen, J., Zhang, H., Kang, J., Xiong, Z., Mao, S., and Han, Z. (2023). Generative AI-empowered Simulation for Autonomous Driving in Vehicular Mixed Reality Metaverses. arXiv.","DOI":"10.1109\/JSTSP.2023.3293650"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Niemeyer, M., and Geiger, A. (2021). GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields. arXiv.","DOI":"10.1109\/CVPR46437.2021.01129"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, L., Lian, Q., Wang, L., Ma, N., and Chen, Y.C. (2023). Lift3D: Synthesize 3D Training Data by Lifting 2D GAN to 3D Generative Radiance Field. arXiv.","DOI":"10.1109\/CVPR52729.2023.00040"},{"key":"ref_25","unstructured":"Imane, A., Gu\u00e9riau, M., and Ainouz, S. (October, January 18). Towards a Mixed-Reality framework for autonomous driving. Proceedings of the IEEE International Conference on Intelligent Transportation Systems (ITSC), Macau, China."},{"key":"ref_26","unstructured":"Baruffa, R., Pereira, J., Romet, P., Gechter, F., and Weiss, T. (November, January 19). Mixed Reality Autonomous Vehicle Simulation: Implementation of a Hardware-In-the-Loop Architecture at a Miniature Scale. Proceedings of the the IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NA, USA."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1046","DOI":"10.3390\/vehicles5030057","article-title":"A Vehicle Crash Simulator Using Digital Twin Technology for Synthesizing Simulation and Graphical Models","volume":"5","author":"Nam","year":"2023","journal-title":"Vehicles"},{"key":"ref_28","unstructured":"MORAI (2025, June 23). MORAI, Autonomous Vehicle Driving Simulator, Available online: https:\/\/www.morai.ai\/."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/38.946629","article-title":"Color transfer between images","volume":"21","author":"Reinhard","year":"2001","journal-title":"IEEE Comput. Graph. Appl."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yu, F., Chen, H., Wang, X., Xian, W., Chen, Y., Liu, F., Madhavan, V., and Darrell, T. (2020). BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning. arXiv.","DOI":"10.1109\/CVPR42600.2020.00271"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"8056","DOI":"10.1038\/s41598-023-35170-z","article-title":"Real time object detection using LiDAR and camera fusion for autonomous driving","volume":"13","author":"Liu","year":"2023","journal-title":"Sci. Rep."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Jung, S., Lee, Y.S., Lee, Y., and Lee, K. (2022). 3D reconstruction using 3D registration-based ToF-stereo fusion. Sensors, 22.","DOI":"10.3390\/s22218369"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Fang, J., Zuo, X., Zhou, D., Jin, S., Wang, S., and Zhang, L. (2021, January 20\u201325). LiDAR-Aug: A General Rendering-based Augmentation Framework for 3D Object Detection. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00468"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Jung, S., Yun, H., and Lee, K. (2023, January 11\u201313). Accurate Pose Estimation Method using ToF-Stereo Camera Data Fusion. Proceedings of the 2023 14th International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Republic of Korea.","DOI":"10.1109\/ICTC58733.2023.10392469"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/3\/987\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T05:16:18Z","timestamp":1770268578000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/3\/987"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,3]]},"references-count":34,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["s26030987"],"URL":"https:\/\/doi.org\/10.3390\/s26030987","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,3]]}}}