{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T03:39:16Z","timestamp":1771645156017,"version":"3.50.1"},"reference-count":63,"publisher":"Springer Science and Business Media LLC","issue":"12","license":[{"start":{"date-parts":[[2022,9,22]],"date-time":"2022-09-22T00:00:00Z","timestamp":1663804800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,9,22]],"date-time":"2022-09-22T00:00:00Z","timestamp":1663804800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Karlsruher Institut f\u00fcr Technologie (KIT)"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Vis"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>A considerable amount of research is concerned with the generation of realistic sensor data. LiDAR point clouds are generated by complex simulations or learned generative models. The generated data is usually exploited to enable or improve downstream perception algorithms. Two major questions arise from these procedures: First, how to evaluate the realism of the generated data? Second, does more realistic data also lead to better perception performance? This paper addresses both questions and presents a novel metric to quantify the realism of LiDAR point clouds. Relevant features are learned from real-world and synthetic point clouds by training on a proxy classification task. In a series of experiments, we demonstrate the application of our metric to determine the realism of generated LiDAR data and compare the realism estimation of our metric to the performance of a segmentation model. We confirm that our metric provides an indication for the downstream segmentation performance.<\/jats:p>","DOI":"10.1007\/s11263-022-01676-8","type":"journal-article","created":{"date-parts":[[2022,9,22]],"date-time":"2022-09-22T09:02:36Z","timestamp":1663837356000},"page":"2962-2979","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A Realism Metric for Generated LiDAR Point Clouds"],"prefix":"10.1007","volume":"130","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0037-8460","authenticated-orcid":false,"given":"Larissa T.","family":"Triess","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4578-1616","authenticated-orcid":false,"given":"Christoph B.","family":"Rist","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7950-9915","authenticated-orcid":false,"given":"David","family":"Peter","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6190-7202","authenticated-orcid":false,"given":"J. Marius","family":"Z\u00f6llner","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,9,22]]},"reference":[{"key":"1676_CR1","unstructured":"Achlioptas, P., Diamanti, O., Mitliagkas, I. & Guibas, L. (2018). Learning representations and generative models for 3D point clouds. In Proceedings of the international conference on learning representations (ICLR) workshops."},{"key":"1676_CR2","unstructured":"Arjovsky, M., Chintala, S. & Bottou, L. (2017). Wasserstein GAN."},{"key":"1676_CR3","unstructured":"Arora, S., Risteski, A. & Zhang, Y. (2018). Do GANs learn the distribution? Some theory and empirics. In Proceedings of the international conference on learning representations (ICLR)."},{"key":"1676_CR4","unstructured":"Beutel, A., Chen, J., Zhao, Z. & Chi, E. H. (2017). Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations. In Workshop on fairness, accountability, and transparency in machine learning."},{"key":"1676_CR5","doi-asserted-by":"crossref","unstructured":"Borji, A. (2019). Pros and cons of GAN evaluation measures. In Computer vision and image understanding (CVIU), (pp. 41\u201365).","DOI":"10.1016\/j.cviu.2018.10.009"},{"key":"1676_CR6","doi-asserted-by":"crossref","unstructured":"Caccia, L., van Hoof, H., Courville, A. & Pineau, J. (2019). Deep generative modeling of LiDAR Data. In Proceedings of the IEEE international conference on intelligent robots and systems (IROS), (pp. 5034\u20135040).","DOI":"10.1109\/IROS40897.2019.8968535"},{"key":"1676_CR7","doi-asserted-by":"crossref","unstructured":"Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G. & Beijbom, O. (2020). nuScenes: A multimodal dataset for autonomous driving. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), (pp. 11618\u201311628).","DOI":"10.1109\/CVPR42600.2020.01164"},{"key":"1676_CR8","unstructured":"Chang, A.X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., Xiao, J., Yi, L. & Yu, F.(2015). ShapeNet: An information-rich 3D model repository."},{"key":"1676_CR9","doi-asserted-by":"crossref","unstructured":"Charles, R. Q., Su, H., Kaichun, M. & Guibas, L.J. (2017). PointNet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), (pp. 77\u201385).","DOI":"10.1109\/CVPR.2017.16"},{"key":"1676_CR10","unstructured":"Che, T., Li, Y., Jacob, A.P., Bengio, Y. & Li, W. (2017). Mode regularized generative adversarial networks. In Proceedings of the international conference on learning representations (ICLR)."},{"key":"1676_CR11","unstructured":"Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I. & Abbeel, P. (2016). InfoGAN: interpretable representation learning by information maximizing generative adversarial nets. In Advances in neural information processing systems (NIPS)."},{"key":"1676_CR12","doi-asserted-by":"crossref","unstructured":"Chicco, D. (2021). Siamese neural networks: an overview. In Artificial neural networks, (pp. 73\u201394).","DOI":"10.1007\/978-1-0716-0826-5_3"},{"key":"1676_CR13","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K. & Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), (pp. 248\u2013255).","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"1676_CR14","doi-asserted-by":"crossref","unstructured":"Dong, X. & Shen, J. (2018). Triplet loss in siamese network for object tracking. In Proceedings of the European conference on computer vision (ECCV), (pp. 472\u2013488).","DOI":"10.1007\/978-3-030-01261-8_28"},{"key":"1676_CR15","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A. & Koltun, V. (2017). CARLA: An open urban driving simulator. In Proceedings of the 1st annual conference on robot learning, (pp. 1\u201316)."},{"issue":"11","key":"1676_CR16","doi-asserted-by":"publisher","first-page":"1231","DOI":"10.1177\/0278364913491297","volume":"32","author":"A Geiger","year":"2013","unstructured":"Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013). Vision meets Robotics: The KITTI Dataset. International Journal of Robotics Research (IJRR), 32(11), 1231\u20131237.","journal-title":"International Journal of Robotics Research (IJRR)"},{"key":"1676_CR17","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. & Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems (NIPS)."},{"key":"1676_CR18","unstructured":"Gretton, A., Borgwardt, K. M., Rasch, M. J., Sch\u00f6lkopf, B. & Smola, A. (2012). A kernel two-sample test. Journal of Machine Learning Research (JMLR), (pp. 723\u2013773)."},{"key":"1676_CR19","doi-asserted-by":"crossref","unstructured":"Gurumurthy, S., Sarvadevabhatla, R. K. & Babu, R. V. (2017). DeLiGAN: Generative adversarial networks for diverse and limited data. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), (pp. 4941\u20134949).","DOI":"10.1109\/CVPR.2017.525"},{"key":"1676_CR20","unstructured":"Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B. & Hochreiter, S.(2017). GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in neural information processing systems (NIPS), (pp. 6629\u20136640)."},{"key":"1676_CR21","doi-asserted-by":"crossref","unstructured":"Hoffer, E. & Ailon, N. (2015). Deep metric learning using triplet network. In Similarity-based pattern recognition (SIMBAD), (pp. 84\u201392).","DOI":"10.1007\/978-3-319-24261-3_7"},{"key":"1676_CR22","doi-asserted-by":"crossref","unstructured":"Huang, X., Li, Y., Poursaeed, O., Hopcroft, J. & Belongie, S. (2017). Stacked generative adversarial networks. In Proceedings IEEE conference on computer vision and pattern recognition (CVPR), (pp. 1866\u20131875).","DOI":"10.1109\/CVPR.2017.202"},{"key":"1676_CR23","doi-asserted-by":"crossref","unstructured":"Hurl, B., Czarnecki, K. & Waslander, S.L. (2019). Precise Synthetic Image and LiDAR (PreSIL) Dataset for autonomous vehicle perception.","DOI":"10.1109\/IVS.2019.8813809"},{"key":"1676_CR24","unstructured":"Im, D.J., Kim, C.D., Jiang, H. & Memisevic, R. (2016). Generating images with recurrent adversarial networks."},{"key":"1676_CR25","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J. Y., Zhou, T. & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2017.632"},{"key":"1676_CR26","doi-asserted-by":"crossref","unstructured":"Johnson, J., Alahi, A. & Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. In Proceedings of the European conference on computer vision (ECCV).","DOI":"10.1007\/978-3-319-46475-6_43"},{"key":"1676_CR27","unstructured":"Khrulkov, V. & Oseledets, I. V. (2018). Geometry score: A method for comparing generative adversarial networks. In Proceedings of the international conference on machine learning (ICML)."},{"key":"1676_CR28","doi-asserted-by":"crossref","unstructured":"Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z. & Shi, W. (2017). Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), (pp. 105\u2013114).","DOI":"10.1109\/CVPR.2017.19"},{"key":"1676_CR29","doi-asserted-by":"crossref","unstructured":"Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z. & Shi, W. (2017). Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), (pp. 105\u2013114).","DOI":"10.1109\/CVPR.2017.19"},{"key":"1676_CR30","volume-title":"Testing statistical hypotheses","author":"EL Lehmann","year":"2006","unstructured":"Lehmann, E. L., & Romano, J. P. (2006). Testing statistical hypotheses. Springer."},{"key":"1676_CR31","doi-asserted-by":"crossref","unstructured":"Li, D., Ling, H., Kim, S. W., Kreis, K., Barriuso, A., Fidler, S. & Torralba, A. (2022). BigDatasetGAN: Synthesizing ImageNet with pixel-wise annotations. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR52688.2022.02064"},{"key":"1676_CR32","doi-asserted-by":"crossref","unstructured":"Lin, Z., Khetan, A., Fanti, G. & Oh, S. (2018). PacGAN: The power of two samples in generative adversarial networks. In Advances in neural information processing systems (NIPS), (pp. 324\u2013335).","DOI":"10.1109\/JSAIT.2020.2983071"},{"key":"1676_CR33","unstructured":"L\u00f6hdefink, J. & Fingscheidt, T. (2022). Improving performance of semantic segmentation CycleGANs by noise injection into the latent segmentation space."},{"key":"1676_CR34","unstructured":"Lucic, M., Kurach, K., Michalski, M., Gelly, S. & Bousquet, O. (2018). Are GANs created equal? A large-scale study. In Advances in Neural Information Processing Systems (NIPS), (pp. 698\u2013707)."},{"key":"1676_CR35","unstructured":"Mathieu, M.F., Zhao, J.J., Zhao, J., Ramesh, A., Sprechmann, P. & LeCun, Y. (2016). Disentangling factors of variation in deep representation using adversarial training. In Advances in Neural information processing systems (NIPS), (pp. 5047\u20135055)."},{"key":"1676_CR36","doi-asserted-by":"crossref","unstructured":"Milioto, A., Vizzo, I., Behley, J. & Stachniss, C. (2019). RangeNet++: Fast and accurate LiDAR semantic segmentation. In Proceedings IEEE international conference on intelligent robots and systems (IROS).","DOI":"10.1109\/IROS40897.2019.8967762"},{"key":"1676_CR37","unstructured":"Olsson, C., Bhupatiraju, S., Brown, T., Odena, A. & Goodfellow, I. (2018). Skill rating for generative models."},{"key":"1676_CR38","doi-asserted-by":"crossref","unstructured":"Park, T., Liu, M., Wang, T. & Zhu, J. (2019). Semantic image synthesis with spatially-adaptive normalization. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR).","DOI":"10.1109\/CVPR.2019.00244"},{"key":"1676_CR39","unstructured":"Qi, C. R., Yi, L., Su, H. & Guibas, L. J. (2017). PointNet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in neural information processing systems (NIPS)."},{"key":"1676_CR40","unstructured":"Radford, A., Metz, L. & Chintala, S. (2016). Unsupervised representation learning with deep convolutional generative adversarial networks."},{"key":"1676_CR41","doi-asserted-by":"crossref","unstructured":"Raff, E. & Sylvester, J. (2018). Gradient reversal against discrimination: A fair neural network learning approach. In Proceedings of IEEE international conference on data science and advanced analytics (DSAA), (pp. 189\u2013198).","DOI":"10.1109\/DSAA.2018.00029"},{"key":"1676_CR42","unstructured":"Richardson, E. & Weiss, Y. (2018). On GANs and GMMs. In Advances in neural information processing systems (NIPS), (pp. 5852\u20135863)."},{"key":"1676_CR43","unstructured":"Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X. & Chen, X. (2016). Improved techniques for training GANs. In Advances in neural information processing systems (NIPS), (pp. 2234\u20132242)."},{"key":"1676_CR44","unstructured":"Sallab, A. E., Sobh, I., Zahran, M. & Essam, N. (2019). LiDAR Sensor modeling and Data augmentation with GANs for Autonomous driving. In Proceedings of the international conference on machine learning (ICML) workshops."},{"key":"1676_CR45","unstructured":"Santurkar, S., Schmidt, L. & Madry, A. (2018). A classification-based study of covariate shift in GAN distributions. In Proceedings of the international conference on machine learning (ICML), (pp. 4480\u20134489)."},{"key":"1676_CR46","unstructured":"Scale AI: PandaSet (2020), https:\/\/pandaset.org"},{"key":"1676_CR47","doi-asserted-by":"crossref","unstructured":"Shu, D., Park, S. W. & Kwon, J. (2019). 3D Point cloud generative adversarial network based on tree structured graph convolutions. In Proceedings of the IEEE international conference on computer vision (ICCV), (pp. 3858\u20133867).","DOI":"10.1109\/ICCV.2019.00396"},{"key":"1676_CR48","unstructured":"Srivastava, A., Valkov, L., Russell, C., Gutmann, M. U. & Sutton, C. (2017). VEEGAN: Reducing mode collapse in GANs using implicit variational learning. In Advances in neural information processing systems (NIPS)"},{"key":"1676_CR49","unstructured":"Theis, L., van\u00a0den Oord, A. & Bethge, M. (2016). A note on the evaluation of generative models. In Proceedings of the international conference on learning representations (ICLR)."},{"key":"1676_CR50","unstructured":"Tolstikhin, I. O., Gelly, S., Bousquet, O., Simon-Gabriel, C. J. & Sch\u00f6lkopf, B. (2017). AdaGAN: Boosting generative models. In Advances in neural information processing systems (NIPS), (pp. 5424\u20135433)."},{"key":"1676_CR51","doi-asserted-by":"crossref","unstructured":"Triess, L. T., Dreissig, M., Rist, C. B. & Z\u00f6llner, J. M. (2021). A survey on deep domain adaptation for LiDAR perception. In Proceedings of IEEE intelligent vehicles symposium (IV) workshops.","DOI":"10.1109\/IVWorkshops54471.2021.9669228"},{"key":"1676_CR52","doi-asserted-by":"crossref","unstructured":"Triess, L. T., Peter, D., Baur, S. A., & Z\u00f6llner, J. M. (2021). Quantifying point cloud realism through adversarially learned latent representations. In Proceedings of the German conference on pattern recognition (GCPR).","DOI":"10.1007\/978-3-030-92659-5_44"},{"key":"1676_CR53","doi-asserted-by":"crossref","unstructured":"Triess, L. T., Peter, D., Rist, C. B., Enzweiler, M. & Z\u00f6llner, J. M. (2019). CNN-based synthesis of realistic high-resolution LiDAR data. In Proceedings of IEEE intelligent vehicles symposium (IV), (pp. 1512\u20131519)","DOI":"10.1109\/IVS.2019.8813771"},{"key":"1676_CR54","unstructured":"Wang, Y., Zhang, L. & van\u00a0de Weijer, J. (2016). Ensembles of generative adversarial networks. In Advances in neural information processing systems (NIPS) workshops."},{"issue":"4","key":"1676_CR55","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1109\/TIP.2003.819861","volume":"13","author":"Z Wang","year":"2004","unstructured":"Wang, Z., Bovik, A., Sheikh, H., & Simoncelli, E. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transaction on Image Processing (TIP), 13(4), 600\u2013612.","journal-title":"IEEE Transaction on Image Processing (TIP)"},{"key":"1676_CR56","doi-asserted-by":"crossref","unstructured":"Wu, B., Zhou, X., Zhao, S., Yue, X. & Keutzer, K. (2019). SqueezeSegV2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a LiDAR point cloud. In Proceedings of IEEE international conference on robotics and automation (ICRA).","DOI":"10.1109\/ICRA.2019.8793495"},{"key":"1676_CR57","unstructured":"Xiang, S. & Li, H. (2017). On the effects of batch and weight normalization in generative adversarial networks."},{"key":"1676_CR58","doi-asserted-by":"crossref","unstructured":"Xu, Q., Huang, G., Yuan, Y., Guo, C., Sun, Y., Wu, F. & Weinberger, K.Q. (2018). An empirical study on evaluation metrics of generative adversarial networks.","DOI":"10.1109\/BigData.2018.8622525"},{"key":"1676_CR59","doi-asserted-by":"crossref","unstructured":"Xu, Y., He, F., Du, B., Zhang, L. & Tao, D. (2021). Self-ensembling GAN for cross-domain semantic segmentation.","DOI":"10.1109\/TMM.2022.3229976"},{"key":"1676_CR60","unstructured":"Yang, J., Kannan, A., Batra, D. & Parikh, D. (2017). LR-GAN: Layered recursive generative adversarial networks for image generation. In Proceedings of the international conference on learning representations (ICLR)."},{"key":"1676_CR61","doi-asserted-by":"crossref","unstructured":"Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X. & Metaxas, D. (2017). StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks. In Proceedings of the IEEE international conference on computer vision (ICCV), (pp. 5908\u20135916).","DOI":"10.1109\/ICCV.2017.629"},{"key":"1676_CR62","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Song, Y. & Qi, H. (2018). Decoupled learning for conditional adversarial networks. In Proceedings of the IEEE winter conference on applications of computer vision (WACV), (pp. 700\u2013708).","DOI":"10.1109\/WACV.2018.00082"},{"key":"1676_CR63","unstructured":"Zhou, Z., Cai, H., Rong, S., Song, Y., Ren, K., Zhang, W., Wang, J. & Yu, Y. (2018). Activation maximization generative adversarial nets. In Proceedings of the international conference on learning representations (ICLR)."}],"container-title":["International Journal of Computer Vision"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-022-01676-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11263-022-01676-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-022-01676-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,20]],"date-time":"2023-02-20T01:29:02Z","timestamp":1676856542000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11263-022-01676-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,22]]},"references-count":63,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["1676"],"URL":"https:\/\/doi.org\/10.1007\/s11263-022-01676-8","relation":{},"ISSN":["0920-5691","1573-1405"],"issn-type":[{"value":"0920-5691","type":"print"},{"value":"1573-1405","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,22]]},"assertion":[{"value":"13 February 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 July 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 September 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}