{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T16:27:18Z","timestamp":1776443238233,"version":"3.51.2"},"reference-count":34,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2022,7,6]],"date-time":"2022-07-06T00:00:00Z","timestamp":1657065600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF)","doi-asserted-by":"publisher","award":["2021R1I1A3055973"],"award-info":[{"award-number":["2021R1I1A3055973"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Soonchunhyang University Research Fund","award":["2021R1I1A3055973"],"award-info":[{"award-number":["2021R1I1A3055973"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Recently, the rapid development of convolutional neural networks (CNN) has consistently improved object detection performance using CNN and has naturally been implemented in autonomous driving due to its operational potential in real-time. Detecting moving targets to realize autonomous driving is an essential task for the safety of drivers and pedestrians, and CNN-based moving target detectors have shown stable performance in fair weather. However, there is a considerable drop in detection performance during poor weather conditions like hazy or foggy situations due to particles in the atmosphere. To ensure stable moving object detection, an image restoration process with haze removal must be accompanied. Therefore, this paper proposes an image dehazing network that estimates the current weather conditions and removes haze using the haze level to improve the detection performance under poor weather conditions due to haze and low visibility. Combined with the thermal image, the restored image is assigned to the two You Only Look Once (YOLO) object detectors, respectively, which detect moving targets independently and improve object detection performance using late fusion. The proposed model showed improved dehazing performance compared with the existing image dehazing models and has proved that images taken under foggy conditions, the poorest weather for autonomous driving, can be restored to normal images. Through the fusion of the RGB image restored by the proposed image dehazing network with thermal images, the proposed model improved the detection accuracy by up to 22% or above in a dense haze environment like fog compared with models using existing image dehazing techniques.<\/jats:p>","DOI":"10.3390\/s22145084","type":"journal-article","created":{"date-parts":[[2022,7,6]],"date-time":"2022-07-06T21:15:52Z","timestamp":1657142152000},"page":"5084","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Deep Multimodal Detection in Reduced Visibility Using Thermal Depth Estimation for Autonomous Driving"],"prefix":"10.3390","volume":"22","author":[{"given":"Sungan","family":"Yoon","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Soonchunhyang University, Asan 31538, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5162-1745","authenticated-orcid":false,"given":"Jeongho","family":"Cho","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Soonchunhyang University, Asan 31538, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11\u201314). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4224","DOI":"10.1109\/TII.2018.2822828","article-title":"Object Classification Using CNN-Based Fusion of Vision and LIDAR in Autonomous Vehicle Environment","volume":"14","author":"Gao","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Bechtel, M.G., Mcellhiney, E., Kim, M., and Yun, H. (2018, January 28\u201331). DeepPicar: A Low-Cost Deep Neural Network-Based Autonomous Car. Proceedings of the IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, Hakodate, Japan.","DOI":"10.1109\/RTCSA.2018.00011"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2353","DOI":"10.1109\/TITS.2017.2787101","article-title":"Vehicle tracking using surveillance with multimodal data fusion","volume":"19","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1964","DOI":"10.1109\/TITS.2018.2857510","article-title":"Dynamic vehicle detection with sparse point clouds based on PE-CPD","volume":"20","author":"Liu","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"He, Y., and Li, L. (2018, January 12\u201316). A Novel Multi-Source Vehicle Detection Algorithm based on Deep Learning. Proceedings of the IEEE International Conference on Signal Processing, Beijing, China.","DOI":"10.1109\/ICSP.2018.8652388"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yao, X., Zhang, Y., Yao, Y., Tian, J., Yang, C., Xu, Z., and Guan, Y. (2021, January 27\u201328). Traffic vehicle detection algorithm based on YOLOv3. Proceedings of the International Conference on Intelligent Transportation Big Data & Smart City, Xi\u2019an, China.","DOI":"10.1109\/ICITBS53129.2021.00020"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1023\/A:1016328200723","article-title":"Vision and the Atmosphere","volume":"48","author":"Narasimhan","year":"2002","journal-title":"Int. J. Comput. Vis."},{"key":"ref_14","first-page":"2341","article-title":"Single image haze removal using dark channel prior","volume":"33","author":"He","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3522","DOI":"10.1109\/TIP.2015.2446191","article-title":"A fast single image haze removal algorithm using color attenuation prior","volume":"24","author":"Zhu","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_16","unstructured":"Tarel, J.P., and Hautiere, N. (October, January 29). Fast Visibility Restoration from a Single Color or Gray Level Image. Proceedings of the IEEE International Conference on Computer Vision, Kyoto, Japan."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Berman, D., Treibitz, T., and Avidan, S. (2016, January 27\u201330). Non-local Image Dehazing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.185"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Xie, B., Guo, F., and Cai, Z. (2010, January 13\u201314). Improved single image dehazing using dark channel prior and multi-scale retinex. Proceedings of the International Conference on Intelligent System Design and Engineering Application, Changsha, China.","DOI":"10.1109\/ISDEA.2010.141"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1109\/LGRS.2018.2874084","article-title":"Haze and Thin Cloud Removal via Sphere Model Improved Dark Channel Prior","volume":"16","author":"Li","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Dong, H., Pan, J., Xiang, L., Hu, Z., Zhang, X., Wang, F., and Yang, M.H. (2020, January 13\u201319). Multi-scale boosted dehazing network with dense feature fusion. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00223"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Hong, M., Xie, Y., Li, C., and Qu, Y. (2020, January 13\u201319). Distilling image dehazing with heterogeneous task imitation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00352"},{"key":"ref_22","first-page":"11908","article-title":"FFA-Net: Feature fusion attention network for single image dehazing","volume":"34","author":"Qin","year":"2020","journal-title":"AAAI Tech. Track Vis."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"5187","DOI":"10.1109\/TIP.2016.2598681","article-title":"DehazeNet: An end-to-end system for single image haze removal","volume":"25","author":"Cai","year":"2016","journal-title":"IEEE Trans. Image Process."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, B., Peng, X., Wang, Z., Xu, J., and Feng, D. (2017, January 22\u201329). AOD-Net: All-in-one dehazing Network. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.511"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, H., and Patel, V.M. (2018, January 18\u201323). Densely connected pyramid dehazing network. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00337"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ma, F., and Karaman, S. (2018, January 21\u201325). Sparse-to-dense: Depth prediction from sparse depth samples and a single image. Proceedings of the IEEE International Conference on Robotics and Automation, Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8460184"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"H\u00e4ne, C., Zach, C., Lim, J., Ranganathan, A., and Pollefeys, M. (2011, January 25\u201330). Stereo depth map fusion for robot navigation. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6048261"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Muratov, Y.R., Nikiforov, M.B., Rusakov, A.B., and Gurov, V.S. (2015, January 14\u201318). Estimation of distance to objects by stereovision. Proceedings of the Mediterranean Conference on Embedded Computing, Budva, Montenegro.","DOI":"10.1109\/MECO.2015.7181890"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Godard, C., Aodha, O.M., and Brostow, G.J. (2017, January 21\u201326). Unsupervised Monocular Depth Estimation with Left-Right Consistency. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.699"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1109\/TITS.2020.2972974","article-title":"Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges","volume":"22","author":"Feng","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_32","unstructured":"(2021, October 01). FREE Teledyne FLIR Thermal Dataset for Algorithm Training. Available online: https:\/\/www.flir.com\/news-center\/camera-cores--components\/flir-open-source-starter-thermal-dataset-for-autonomous-vehicle-testing\/."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kim, Y., and Yim, C. (2020, January 19\u201322). Image Dehaze Method Using Depth Map Estimation Network Based on Atmospheric Scattering Model. Proceedings of the International Conference on Electronics, Information, and Communication, Barcelona, Spain.","DOI":"10.1109\/ICEIC49074.2020.9051031"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"492","DOI":"10.1109\/TIP.2018.2867951","article-title":"Benchmarking single-image dehazing and beyond","volume":"28","author":"Li","year":"2019","journal-title":"IEEE Trans. Image Process."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5084\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:43:33Z","timestamp":1760139813000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5084"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,6]]},"references-count":34,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["s22145084"],"URL":"https:\/\/doi.org\/10.3390\/s22145084","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,6]]}}}