{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T06:45:31Z","timestamp":1778222731776,"version":"3.51.4"},"reference-count":30,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2021,7,14]],"date-time":"2021-07-14T00:00:00Z","timestamp":1626220800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010680","name":"H2020 Transport","doi-asserted-by":"publisher","award":["856602"],"award-info":[{"award-number":["856602"]}],"id":[{"id":"10.13039\/100010680","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this work, we investigated two issues: (1) How the fusion of lidar and camera data can improve semantic segmentation performance compared with the individual sensor modalities in a supervised learning context; and (2) How fusion can also be leveraged for semi-supervised learning in order to further improve performance and to adapt to new domains without requiring any additional labelled data. A comparative study was carried out by providing an experimental evaluation on networks trained in different setups using various scenarios from sunny days to rainy night scenes. The networks were tested for challenging, and less common, scenarios where cameras or lidars individually would not provide a reliable prediction. Our results suggest that semi-supervised learning and fusion techniques increase the overall performance of the network in challenging scenarios using less data annotations.<\/jats:p>","DOI":"10.3390\/s21144813","type":"journal-article","created":{"date-parts":[[2021,7,14]],"date-time":"2021-07-14T10:13:42Z","timestamp":1626257622000},"page":"4813","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Lidar\u2013Camera Semi-Supervised Learning for Semantic Segmentation"],"prefix":"10.3390","volume":"21","author":[{"given":"Luca","family":"Caltagirone","sequence":"first","affiliation":[{"name":"Applied Artificial Intelligence Research Group, Department of Mechanics and Maritime Sciences, Chalmers University of Technology, 412 58 Gothenburg, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3692-0688","authenticated-orcid":false,"given":"Mauro","family":"Bellone","sequence":"additional","affiliation":[{"name":"Smart City Center of Excellence, Tallinn University of Technology, 12616 Tallinn, Estonia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0206-9186","authenticated-orcid":false,"given":"Lennart","family":"Svensson","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Chalmers University of Technology, 412 58 Gothenburg, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6679-637X","authenticated-orcid":false,"given":"Mattias","family":"Wahde","sequence":"additional","affiliation":[{"name":"Applied Artificial Intelligence Research Group, Department of Mechanics and Maritime Sciences, Chalmers University of Technology, 412 58 Gothenburg, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Raivo","family":"Sell","sequence":"additional","affiliation":[{"name":"Department of Mechanical and Industrial Engineering, Tallinn University of Technology, 12616 Tallinn, Estonia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1016\/j.tra.2016.09.010","article-title":"Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?","volume":"94","author":"Kalra","year":"2016","journal-title":"Transp. Res. Part A Policy Pract."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1007\/s10994-019-05855-6","article-title":"A survey on semi-supervised learning","volume":"109","author":"Hoos","year":"2020","journal-title":"Mach. Learn."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ouali, Y., Hudelot, C., and Tami, M. (2020, January 13\u201319). Semi-Supervised Semantic Segmentation with Cross-Consistency Training. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01269"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The pascal visual object classes (voc) challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Xiao, H., Wei, Y., Liu, Y., Zhang, M., and Feng, J. (2018, January 2\u20137). Transferable semi-supervised semantic segmentation. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.12250"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"41830","DOI":"10.1109\/ACCESS.2020.2975022","article-title":"Digging Into Pseudo Label: A Low-Budget Approach for Semi-Supervised Semantic Segmentation","volume":"8","author":"Chen","year":"2020","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1109\/TITS.2020.2972974","article-title":"Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges","volume":"22","author":"Feng","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Nartey, O.T., Yang, G., Asare, S.K., Wu, J., and Frempong, L.N. (2020). Robust semi-supervised traffic sign recognition via self-training and weakly-supervised learning. Sensors, 20.","DOI":"10.3390\/s20092684"},{"key":"ref_9","unstructured":"Zhu, Y., Zhang, Z., Wu, C., Zhang, Z., He, T., Zhang, H., Manmatha, R., Li, M., and Smola, A. (2020). Improving Semantic Segmentation via Self-Training. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27\u201330). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.patrec.2008.04.005","article-title":"Semantic object classes in video: A high-definition ground truth database","volume":"30","author":"Brostow","year":"2009","journal-title":"Pattern Recognit. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets robotics: The kitti dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_13","unstructured":"Gao, B., Pan, Y., Li, C., Geng, S., and Zhao, H. (2020). Are We Hungry for 3D LiDAR Data for Semantic Segmentation?. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2496","DOI":"10.1109\/TITS.2019.2919741","article-title":"Semantic segmentation of 3d lidar data in dynamic scene using semi-supervised learning","volume":"21","author":"Mei","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Fayyad, J., Jaradat, M.A., Gruyer, D., and Najjaran, H. (2020). Deep learning sensor fusion for autonomous vehicle perception and localization: A review. Sensors, 20.","DOI":"10.3390\/s20154220"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Bellone, M., Ismailogullari, A., M\u00fc\u00fcr, J., Nissin, O., Sell, R., and Soe, R.M. (2021). Autonomous driving in the real-world: The weather challenge in the Sohjoa Baltic project. Towards Connected and Autonomous Vehicle Highway, Springer.","DOI":"10.1007\/978-3-030-66042-0_9"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Xiao, F. (2020). GIQ: A generalized intelligent quality-based approach for fusing multi-source information. IEEE Trans. Fuzzy Syst.","DOI":"10.1109\/TFUZZ.2020.2991296"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.inffus.2016.02.005","article-title":"An intelligent quality-based approach to fusing multi-source probabilistic information","volume":"31","author":"Yager","year":"2016","journal-title":"Inf. Fusion"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Krishnamurthi, R., Kumar, A., Gopinathan, D., Nayyar, A., and Qureshi, B. (2020). An Overview of IoT Sensor Data Processing, Fusion, and Analysis Techniques. Sensors, 20.","DOI":"10.3390\/s20216076"},{"key":"ref_20","unstructured":"Li, H., Chen, Y., Zhang, Q., and Zhao, D. (2020). BiFNet: Bidirectional Fusion Network for Road Segmentation. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.robot.2018.11.002","article-title":"LIDAR\u2013camera fusion for road detection using fully convolutional neural networks","volume":"111","author":"Caltagirone","year":"2019","journal-title":"Robot. Auton. Syst."},{"key":"ref_22","unstructured":"Caltagirone, L., Svensson, L., Wahde, M., and Sanfridson, M. (2019). Lidar-Camera Co-Training for Semi-Supervised Road Detection. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., and Caine, B. (2019). Scalability in Perception for Autonomous Driving: Waymo Open Dataset. arXiv.","DOI":"10.1109\/CVPR42600.2020.00252"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Caltagirone, L., Scheidegger, S., Svensson, L., and Wahde, M. (2017, January 11\u201314). Fast LIDAR-based road detection using fully convolutional neural networks. Proceedings of the 2017 Ieee Intelligent Vehicles Symposium (iv), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995848"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhou, Y., and Tuzel, O. (2018, January 18\u201322). Voxelnet: End-to-end learning for point cloud based 3d object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00472"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"van Beers, F., Lindstr\u00f6m, A., Okafor, E., and Wiering, M.A. (2021, June 29). Deep Neural Networks with Intersection over Union Loss for Binary Image Segmentation. Available online: https:\/\/www.scitepress.org\/Papers\/2019\/73475\/73475.pdf.","DOI":"10.5220\/0007347504380445"},{"key":"ref_28","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Oliveira, G.L., Burgard, W., and Brox, T. (2016, January 9\u201314). Efficient deep models for monocular road segmentation. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759717"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/14\/4813\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:30:37Z","timestamp":1760164237000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/14\/4813"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,14]]},"references-count":30,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2021,7]]}},"alternative-id":["s21144813"],"URL":"https:\/\/doi.org\/10.3390\/s21144813","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,14]]}}}