{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T16:40:59Z","timestamp":1778258459094,"version":"3.51.4"},"reference-count":50,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2022,7,15]],"date-time":"2022-07-15T00:00:00Z","timestamp":1657843200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Autonomous driving requires robust and highly accurate perception technologies. Various deep learning algorithms based on only image processing satisfy this requirement, but few such algorithms are based on LiDAR. However, images are only one part of the perceptible sensors in an autonomous driving vehicle; LiDAR is also essential for the recognition of driving environments. The main reason why there exist few deep learning algorithms based on LiDAR is a lack of data. Recent translation technology using generative adversarial networks (GANs) has been proposed to deal with this problem. However, these technologies focus on only image-to-image translation, although a lack of data occurs more often with LiDAR than with images. LiDAR translation technology is required not only for data augmentation, but also for driving simulation, which allows algorithms to practice driving as if they were commanding a real vehicle, before doing so in the real world. In other words, driving simulation is a key technology for evaluating and verifying algorithms which are practically applied to vehicles. In this paper, we propose a GAN-based LiDAR translation algorithm for autonomous driving and driving simulation. It is the first LiDAR translation approach that can deal with various types of weather that are based on an empirical approach. We tested the proposed method on the JARI data set, which was collected under various adverse weather scenarios with diverse precipitation and visible distance settings. The proposed method was also applied to the real-world Spain data set. Our experimental results demonstrate that the proposed method can generate realistic LiDAR data under adverse weather conditions.<\/jats:p>","DOI":"10.3390\/s22145287","type":"journal-article","created":{"date-parts":[[2022,7,18]],"date-time":"2022-07-18T01:53:22Z","timestamp":1658109202000},"page":"5287","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":36,"title":["GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation"],"prefix":"10.3390","volume":"22","author":[{"given":"Jinho","family":"Lee","sequence":"first","affiliation":[{"name":"Emerging Design and Informatics Course, Graduate School of Interdisciplinary Information Studies, The University of Tokyo, 4 Chome-6-1 Komaba, Meguro City, Tokyo 153-0041, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daiki","family":"Shiotsuka","sequence":"additional","affiliation":[{"name":"Emerging Design and Informatics Course, Graduate School of Interdisciplinary Information Studies, The University of Tokyo, 4 Chome-6-1 Komaba, Meguro City, Tokyo 153-0041, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Toshiaki","family":"Nishimori","sequence":"additional","affiliation":[{"name":"Mitsubishi Heavy Industries Machinery Systems Ltd., 1-1, Wadasaki-cho 1-chome, Hyogo-ku, Kobe 652-8585, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kenta","family":"Nakao","sequence":"additional","affiliation":[{"name":"Mitsubishi Heavy Industries Ltd., 1-1, Wadasaki-cho 1-chome, Hyogo-ku, Kobe 652-8585, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shunsuke","family":"Kamijo","sequence":"additional","affiliation":[{"name":"The Institute of Industrial Science (IIS), The University of Tokyo, 4 Chome-6-1 Komaba, Meguro City, Tokyo 153-0041, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chen, X., Ma, H., Wan, J., Li, B., and Xia, T. (2017, January 21\u201326). Multi-view 3d object detection network for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.691"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., and Urtasun, R. (2016, January 27\u201330). Monocular 3d object detection for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.236"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1109\/TITS.2020.2972974","article-title":"Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges","volume":"22","author":"Feng","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Fidler, S., and Urtasun, R. (2016, January 27\u201330). Instance-level segmentation for autonomous driving with deep densely connected mrfs. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.79"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"De Brabandere, B., Neven, D., and Van Gool, L. (2017, January 21\u201326). Semantic instance segmentation for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.66"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Fu, H., Gong, M., Wang, C., Batmanghelich, K., and Tao, D. (2018, January 18\u201323). Deep ordinal regression network for monocular depth estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00214"},{"key":"ref_7","unstructured":"Godard, C., Mac Aodha, O., Firman, M., and Brostow, G.J. (November, January 27). Digging into self-supervised monocular depth estimation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wofk, D., Ma, F., Yang, T.J., Karaman, S., and Sze, V. (2019, January 20\u201324). Fastdepth: Fast monocular depth estimation on embedded systems. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794182"},{"key":"ref_9","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_10","unstructured":"El Sallab, A., Sobh, I., Zahran, M., and Essam, N. (2019). LiDAR sensor modeling and data augmentation with GANs for autonomous driving. arXiv."},{"key":"ref_11","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13\u201315). CARLA: An open urban driving simulator. Proceedings of the Conference on Robot Learning, Mountain View, CA, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1109\/TVT.2010.2090675","article-title":"From design to experiments of a 2-DOF vehicle driving simulator","volume":"60","author":"Arioui","year":"2010","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_13","unstructured":"Lee, W.-S., Kim, J.-H., and Cho, J.-H. (1998, January 20). A driving simulator as a virtual reality tool. Proceedings of the 1998 IEEE International Conference on Robotics and Automation (Cat. No. 98CH36146), Leuven, Belgium."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kim, I.I., McArthur, B., and Korevaar, E.J. (2001, January 5\u20138). Comparison of laser beam propagation at 785 nm and 1550 nm in fog and haze for optical wireless communications. Proceedings of the Optical Wireless Communications III. International Society for Optics and Photonics, Boston, MA, USA.","DOI":"10.1117\/12.417512"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"49","DOI":"10.5194\/ars-9-49-2011","article-title":"Influences of weather phenomena on automotive laser radar systems","volume":"9","author":"Rasshofer","year":"2011","journal-title":"Adv. Radio Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"183","DOI":"10.2478\/s11772-014-0190-2","article-title":"Comparison of 905 nm and 1550 nm semiconductor laser rangefinders\u2019 performance deterioration due to adverse environmental conditions","volume":"22","author":"Wojtanowski","year":"2014","journal-title":"Opto-Electron. Rev."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.measurement.2016.10.009","article-title":"Quantifying the influence of rain in LiDAR performance","volume":"95","author":"Filgueira","year":"2017","journal-title":"Measurement"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kutila, M., Pyyk\u00f6nen, P., Holzh\u00fcter, H., Colomb, M., and Duthon, P. (2018, January 4\u20137). Automotive LiDAR performance verification in fog and rain. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569624"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Jokela, M., Kutila, M., and Pyyk\u00f6nen, P. (2019). Testing and validation of automotive point-cloud sensors in adverse weather conditions. Appl. Sci., 9.","DOI":"10.3390\/app9112341"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Heinzler, R., Schindler, P., Seekircher, J., Ritter, W., and Stork, W. (2019, January 9\u201312). Weather influence and classification with automotive lidar sensors. Proceedings of the 2019 IEEE intelligent vehicles symposium (IV), Paris, France.","DOI":"10.1109\/IVS.2019.8814205"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"7064","DOI":"10.1109\/TVT.2020.2989148","article-title":"Full waveform LiDAR for adverse weather conditions","volume":"69","author":"Wallace","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"6670","DOI":"10.1109\/TITS.2020.2998077","article-title":"What happens for a ToF LiDAR in fog?","volume":"22","author":"Li","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Kutila, M., Pyyk\u00f6nen, P., Ritter, W., Sawade, O., and Sch\u00e4ufele, B. (2016, January 1\u20134). Automotive lidar sensor development scenarios for harsh weather conditions. Proceedings of the IEEE International Conference on Intelligent Transportation Systems (ITSC), Rio de Janeiro, Brazil.","DOI":"10.1109\/ITSC.2016.7795565"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"973","DOI":"10.1007\/s11263-018-1072-8","article-title":"Semantic foggy scene understanding with synthetic data","volume":"126","author":"Sakaridis","year":"2018","journal-title":"Int. J. Comput. Vis."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27\u201330). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hahner, M., Dai, D., Sakaridis, C., Zaech, J.N., and Van Gool, L. (2019, January 27\u201330). Semantic understanding of foggy scenes with purely synthetic data. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.","DOI":"10.1109\/ITSC.2019.8917518"},{"key":"ref_27","unstructured":"Wrenninge, M., and Unger, J. (2018). Synscapes: A photorealistic synthetic dataset for street scene parsing. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Sakaridis, C., Dai, D., and Van Gool, L. (2021, January 11\u201317). ACDC: The adverse conditions dataset with correspondences for semantic driving scene understanding. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.01059"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Bijelic, M., Gruber, T., Mannan, F., Kraus, F., Ritter, W., Dietmayer, K., and Heide, F. (2020, January 13\u201319). Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01170"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Hahner, M., Sakaridis, C., Dai, D., and Van Gool, L. (2021, January 11\u201317). Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.01500"},{"key":"ref_31","unstructured":"Yang, T., Li, Y., Ruichek, Y., and Yan, Z. (2021). Performance Modeling a Near-Infrared ToF LiDAR Under Fog: A Data-Driven Approach. IEEE Trans. Intell. Transp. Syst., 1\u201310."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Vargas Rivero, J.R., Gerbich, T., Buschardt, B., and Chen, J. (2021). Data Augmentation of Automotive LIDAR Point Clouds under Adverse Weather Situations. Sensors, 21.","DOI":"10.3390\/s21134503"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Milz, S., Rudiger, T., and Suss, S. (2018, January 8\u201314). Aerial ganeration: Towards realistic data augmentation using conditional gans. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, Germany.","DOI":"10.1007\/978-3-030-11012-3_5"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhu, J.-Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22\u201329). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The kitti dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_36","unstructured":"Wang, L., Goldluecke, B., and Anklam, C. (December, January 30). L2R GAN: LiDAR-to-radar translation. Proceedings of the Asian Conference on Computer Vision, Kyoto, Japan."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Li, R., Li, X., Fu, C.W., Cohen-Or, D., and Heng, P.A. (2019, January 27\u201328). Pu-gan: A point cloud upsampling adversarial network. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00730"},{"key":"ref_38","unstructured":"Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6\u201311). Wasserstein generative adversarial networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., and Paul Smolley, S. (2017, January 22\u201329). Least squares generative adversarial networks. Proceedings of the IEEE International Conference on computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.304"},{"key":"ref_40","unstructured":"Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., and Abbeel, P. (2016, January 5\u201310). Infogan: Interpretable representation learning by information maximizing generative adversarial nets. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_41","unstructured":"Mirza, M., and Osindero, S. (2014). Conditional generative adversarial nets. arXiv."},{"key":"ref_42","unstructured":"Radford, A., Metz, L., and Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv."},{"key":"ref_43","unstructured":"Zhang, H., Goodfellow, I., Metaxas, D., and Odena, A. (2019, January 10\u201315). Self-attention generative adversarial networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_45","unstructured":"Kim, T., Cha, M., Kim, H., Lee, J.K., and Kim, J. (2017, January 6\u201311). Learning to discover cross-domain relations with generative adversarial networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Yi, Z., Zhang, H., Tan, P., and Gong, M. (2017, January 22\u201329). Dualgan: Unsupervised dual learning for image-to-image translation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.310"},{"key":"ref_47","unstructured":"Liu, M.Y., Breuel, T., and Kautz, J. (2017, January 4\u20139). Unsupervised image-to-image translation networks. Proceedings of the 30th International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_48","unstructured":"Liu, M.Y., and Tuzel, O. (2016, January 5\u201310). Coupled generative adversarial networks. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016). Identity mappings in deep residual networks. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46493-0_38"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Tong, T., Li, G., Liu, X., and Gao, Q. (2017, January 22\u201329). Image super-resolution using dense skip connections. Proceedings of the IEEE International Conference on computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.514"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5287\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:50:57Z","timestamp":1760140257000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/14\/5287"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,15]]},"references-count":50,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["s22145287"],"URL":"https:\/\/doi.org\/10.3390\/s22145287","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,15]]}}}