{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T03:32:52Z","timestamp":1784345572227,"version":"3.55.0"},"reference-count":44,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2025,6,20]],"date-time":"2025-06-20T00:00:00Z","timestamp":1750377600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Flemish Government"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Multimodal sensing is essential in order to reach the robustness required of autonomous vehicle perception systems. Infrared (IR) imaging is of particular interest due to its low cost and complementarity with traditional RGB sensors. However, the lack of IR data in many datasets and simulation tools limits the development and validation of sensor fusion algorithms that exploit this complementarity. To address this, we propose an augmentation method that synthesizes realistic IR data from RGB images using gradient-boosting decision trees. We demonstrate that this method is an effective alternative to traditional deep learning methods for image translation such as CNNs and GANs, particularly in data-scarce situations. The proposed approach generates high-quality synthetic IR, i.e., Near-Infrared (NIR) and thermal images from RGB images, enhancing datasets such as MS2, EPFL, and Freiburg. Our synthetic images exhibit good visual quality when evaluated using metrics such as R2, PSNR, SSIM, and LPIPS, achieving an R2 of 0.98 on the MS2 dataset and a PSNR of 21.3 dB on the Freiburg dataset. We also discuss the application of this method to synthetic RGB images generated by the CARLA simulator for autonomous driving. Our approach provides richer datasets with a particular focus on IR modalities for sensor fusion along with a framework for generating a wider variety of driving scenarios within urban driving datasets, which can help to enhance the robustness of sensor fusion algorithms.<\/jats:p>","DOI":"10.3390\/jimaging11070206","type":"journal-article","created":{"date-parts":[[2025,6,20]],"date-time":"2025-06-20T11:23:24Z","timestamp":1750418604000},"page":"206","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["RGB-to-Infrared Translation Using Ensemble Learning Applied to Driving Scenarios"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3318-6764","authenticated-orcid":false,"given":"Leonardo","family":"Ravaglia","sequence":"first","affiliation":[{"name":"Interuniversity Microelectronics Centre, Kapeldreef 75, 3001 Leuven, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0506-2617","authenticated-orcid":false,"given":"Roberto","family":"Longo","sequence":"additional","affiliation":[{"name":"Interuniversity Microelectronics Centre, Kapeldreef 75, 3001 Leuven, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kaili","family":"Wang","sequence":"additional","affiliation":[{"name":"Interuniversity Microelectronics Centre, Kapeldreef 75, 3001 Leuven, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2112-3475","authenticated-orcid":false,"given":"David","family":"Van Hamme","sequence":"additional","affiliation":[{"name":"Interuniversity Microelectronics Centre, Kapeldreef 75, 3001 Leuven, Belgium"},{"name":"Image Processing and Interpretation, Ghent University, 9000 Ghent, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julie","family":"Moeyersoms","sequence":"additional","affiliation":[{"name":"Interuniversity Microelectronics Centre, Kapeldreef 75, 3001 Leuven, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6925-8235","authenticated-orcid":false,"given":"Ben","family":"Stoffelen","sequence":"additional","affiliation":[{"name":"Interuniversity Microelectronics Centre, Kapeldreef 75, 3001 Leuven, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2969-3133","authenticated-orcid":false,"given":"Tom","family":"De Schepper","sequence":"additional","affiliation":[{"name":"Interuniversity Microelectronics Centre, Kapeldreef 75, 3001 Leuven, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Pendleton, S., Andersen, H., Du, X., Shen, X., Meghjani, M., Eng, Y., Rus, D., and Ang, M. (2017). Perception, planning, control, and coordination for autonomous vehicles. Machines, 5.","DOI":"10.3390\/machines5010006"},{"key":"ref_2","first-page":"101110","article-title":"Exploring the future: A meta-analysis of autonomous vehicle adoption and its impact on urban life and the healthcare sector","volume":"26","author":"Adnan","year":"2024","journal-title":"Transp. Res. Interdiscip. Perspect."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Sana, F., Azad, N.L., and Raahemifar, K. (2023). Autonomous vehicle decision-making and control in complex and unconventional scenarios\u2014A review. Machines, 11.","DOI":"10.3390\/machines11070676"},{"key":"ref_4","unstructured":"Huang, K., Shi, B., Li, X., Li, X., Huang, S., and Li, Y. (2022). Multi-modal sensor fusion for auto driving perception: A survey. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Marnissi, M.A., Fradi, H., Sahbani, A., and Amara, N.E.B. (2021, January 10\u201315). Thermal image enhancement using generative adversarial network for pedestrian detection. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.","DOI":"10.1109\/ICPR48806.2021.9412331"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1239","DOI":"10.1109\/TPAMI.2009.122","article-title":"Survey of pedestrian detection for advanced driver assistance systems","volume":"32","author":"Geronimo","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Aloufi, N., Alnori, A., and Basuhail, A. (2024). Enhancing Autonomous Vehicle Perception in Adverse Weather: A Multi Objectives Model for Integrated Weather Classification and Object Detection. Electronics, 13.","DOI":"10.3390\/electronics13153063"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Fadadu, S., Pandey, S., Hegde, D., Shi, Y., Chou, F.C., Djuric, N., and Vallespi-Gonzalez, C. (2022, January 3\u20138). Multi-view fusion of sensor data for improved perception and prediction in autonomous driving. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV51458.2022.00335"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Liu, Z., Tang, H., Amini, A., Yang, X., Mao, H., Rus, D.L., and Han, S. (June, January 29). Bevfusion: Multi-task multi-sensor fusion with unified bird\u2019s-eye view representation. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10160968"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J., Zhou, T., and Efros, A. (2017, January 21\u201326). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhu, J., Park, T., Isola, P., and Efros, A. (2017, January 22\u201329). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yi, Z., Zhang, H., Tan, P., and Gong, M. (2017, January 22\u201329). DualGAN: Unsupervised dual learning for image-to-image translation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.310"},{"key":"ref_13","unstructured":"Kim, T., Cha, M., Kim, H., Lee, J., and Kim, J. (2017, January 6\u201311). Learning to discover cross-domain relations with generative adversarial networks. Proceedings of the International Conference on Machine Learning, PMLR, Sydney, Australia."},{"key":"ref_14","unstructured":"Friedjungov\u00e1, M., Va\u0161ata, D., Chobola, T., and Ji\u0159ina, M. (2022, January 5\u20137). Unsupervised Latent Space Translation Network. Proceedings of the European Symposium on Artificial Neural Networks, Bruges, Belgium."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wang, T., Liu, M., Zhu, J., Tao, A., Kautz, J., and Catanzaro, B. (2018, January 18\u201323). High-resolution image synthesis and semantic manipulation with conditional GANs. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00917"},{"key":"ref_16","unstructured":"Liu, M., Breuel, T., and Kautz, J. (2017). Unsupervised image-to-image translation networks. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Huang, X., Liu, M.Y., Belongie, S., and Kautz, J. (2018, January 8\u201314). Multimodal unsupervised image-to-image translation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01219-9_11"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Jin, Y., Park, I., Song, H., Ju, H., Nalcakan, Y., and Kim, S. (2024). Pix2Next: Leveraging Vision Foundation Models for RGB to NIR Image Translation. arXiv.","DOI":"10.2139\/ssrn.4960430"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Yang, S., Sun, M., Lou, X., Yang, H., and Liu, D. (2024). Nighttime Thermal Infrared Image Translation Integrating Visible Images. Remote Sens., 16.","DOI":"10.3390\/rs16040666"},{"key":"ref_20","unstructured":"Jeon, H., Seo, J., Kim, T., Son, S., Lee, J., Choi, G., and Lim, Y. (2023). RainSD: Rain Style Diversification Module for Image Synthesis Enhancement using Feature-Level Style Distribution. arXiv."},{"key":"ref_21","unstructured":"Zhai, H., Jin, G., Yang, X., and Kang, G. (2024). ColorMamba: Towards High-quality NIR-to-RGB Spectral Translation with Mamba. arXiv."},{"key":"ref_22","unstructured":"Wang, Z., Colonnier, F., Zheng, J., Acharya, J., Jiang, W., and Huang, K. (November, January 29). Tirdet: Mono-modality thermal infrared object detection based on prior thermal-to-visible translation. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada."},{"key":"ref_23","first-page":"9","article-title":"Deep learning thermal image translation for night vision perception","volume":"12","author":"Liu","year":"2020","journal-title":"ACM Trans. Intell. Syst. Technol. (TIST)"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Pizzati, F., Charette, R.d., Zaccaria, M., and Cerri, P. (2020, January 1\u20135). Domain bridge for unpaired image-to-image translation and unsupervised domain adaptation. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA.","DOI":"10.1109\/WACV45572.2020.9093540"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Richardson, E., and Weiss, Y. (2021, January 10\u201315). The surprising effectiveness of linear unsupervised image-to-image translation. Proceedings of the 2020 25th International Conference on Pattern Recognition, Virtual Event.","DOI":"10.1109\/ICPR48806.2021.9413199"},{"key":"ref_26","first-page":"989","article-title":"Image processing and image mining using decision trees","volume":"25","author":"Lu","year":"2009","journal-title":"J. Inf. Sci. Eng."},{"key":"ref_27","unstructured":"Brown, M., and S\u00fcsstrunk, S. (2011, January 20\u201325). Multispectral SIFT for Scene Category Recognition. Proceedings of the Computer Vision and Pattern Recognition (CVPR11), Colorado Springs, CO, USA."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Shin, U., Park, J., and Kweon, I.S. (2023, January 17\u201324). Deep Depth Estimation From Thermal Image. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00107"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Vertens, J., Z\u00fcrn, J., and Burgard, W. (2020). HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images. arXiv.","DOI":"10.1109\/IROS45743.2020.9341192"},{"key":"ref_30","unstructured":"Wu, Y., Kirillov, A., Massa, F., Lo, W., and Girshick, R. (2024, June 01). Detectron2. Available online: https:\/\/github.com\/facebookresearch\/detectron2."},{"key":"ref_31","unstructured":"Chen, T., and Guestrin, C. (2024, April 24). XGBoost Documentation. Available online: https:\/\/xgboost.readthedocs.io."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"e623","DOI":"10.7717\/peerj-cs.623","article-title":"The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation","volume":"7","author":"Chicco","year":"2021","journal-title":"PeerJ Comput. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18\u201323). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00068"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Yue, G., Zhang, L., Zhang, J., Xu, Z., Wang, S., Zhou, T., Gong, Y., and Zhou, W. (2024, January 27\u201330). Subjective quality assessment of thermal infrared images. Proceedings of the 2024 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, UAE.","DOI":"10.1109\/ICIP51287.2024.10648145"},{"key":"ref_36","first-page":"73","article-title":"Study of subjective and objective quality assessment of infrared compressed images","volume":"73","author":"Zelmati","year":"2022","journal-title":"J. Electr. Eng."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lee, D.-G., Jeon, M.-H., Cho, Y., and Kim, A. (June, January 29). Edge-guided multi-domain RGB-to-TIR image translation for training vision tasks with challenging labels. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10161210"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2684","DOI":"10.1016\/j.procs.2023.01.241","article-title":"A machine learning-based comparative approach to predict the crop yield using supervised learning with regression models","volume":"218","author":"Panigrahi","year":"2023","journal-title":"Procedia Comput. Sci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"48451","DOI":"10.1109\/ACCESS.2020.2979348","article-title":"PiiGAN: Generative adversarial networks for pluralistic image inpainting","volume":"8","author":"Cai","year":"2020","journal-title":"IEEE Access"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"116087","DOI":"10.1016\/j.eswa.2021.116087","article-title":"Structural similarity index (SSIM) revisited: A data-driven approach","volume":"189","author":"Bakurov","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_41","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13\u201315). CARLA: An open urban driving simulator. Proceedings of the Conference on Robot Learning. PMLR, Mountain View, CA, USA."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Shaikh, Z.A., Van Hamme, D., Veelaert, P., and Philips, W. (2022). Probabilistic fusion for pedestrian detection from thermal and colour images. Sensors, 22.","DOI":"10.3390\/s22228637"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Dimitrievski, M., Van Hamme, D., Veelaert, P., and Philips, W. (2020). Cooperative multi-sensor tracking of vulnerable road users in the presence of missing detections. Sensors, 20.","DOI":"10.3390\/s20174817"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/7\/206\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:56:00Z","timestamp":1760032560000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/7\/206"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,20]]},"references-count":44,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2025,7]]}},"alternative-id":["jimaging11070206"],"URL":"https:\/\/doi.org\/10.3390\/jimaging11070206","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,20]]}}}