{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T21:42:44Z","timestamp":1781386964623,"version":"3.54.1"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,11,15]],"date-time":"2024-11-15T00:00:00Z","timestamp":1731628800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Big Data"],"abstract":"<jats:p>Bird's-eye-view Semantic Segmentation (BEVSS) is a powerful and crucial component of planning and control systems in many autonomous vehicles. Current methods rely on end-to-end learning to train models, leading to indirectly supervised and inaccurate camera-to-BEV projections. We propose a novel method of supervising feature extraction with camera-view depth and segmentation information, which improves the quality of feature extraction and projection in the BEVSS pipeline. Our model, evaluated on the nuScenes dataset, shows a 3.8% improvement in Intersection-over-Union (IoU) for vehicle segmentation and a 30-fold reduction in depth error compared to baselines, while maintaining competitive inference times of 32 FPS. This method offers more accurate and reliable BEVSS for real-time autonomous driving systems. The codes and implementation details and code can be found at <jats:ext-link>https:\/\/github.com\/bluffish\/sucam<\/jats:ext-link>.<\/jats:p>","DOI":"10.3389\/fdata.2024.1431346","type":"journal-article","created":{"date-parts":[[2024,11,15]],"date-time":"2024-11-15T06:15:16Z","timestamp":1731651316000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Camera-view supervision for bird's-eye-view semantic segmentation"],"prefix":"10.3389","volume":"7","author":[{"given":"Bowen","family":"Yang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"LinLin","family":"Yu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feng","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,11,15]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01164","article-title":"\u201cNuscenes: a multimodal dataset for autonomous driving,\u201d","author":"Caesar","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B2","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","article-title":"Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","volume":"40","author":"Chen","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B3","article-title":"Rethinking monocular depth estimation with adversarial training","author":"Chen","year":"2018","journal-title":"arXiv preprint arXiv:1808.07528"},{"key":"B4","article-title":"Efficient and robust 2D-to-bev representation learning via geometry-guided kernel transformer","author":"Chen","year":"2022","journal-title":"arXiv preprint arXiv:2206.04584"},{"key":"B5","article-title":"\u201cDepth map prediction from a single image using a multi-scale deep network,\u201d","author":"Eigen","year":"2014","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.81","article-title":"\u201cRich feature hierarchies for accurate object detection and semantic segmentation,\u201d","author":"Girshick","year":"2014","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.699","article-title":"\u201cUnsupervised monocular depth estimation with left-right consistency,\u201d","author":"Godard","year":"2017","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00393","article-title":"\u201cDigging into self-supervised monocular depth estimation,\u201d","author":"Godard","year":"2019","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1968","DOI":"10.1109\/LRA.2022.3142418","article-title":"Bird's-eye-view panoptic segmentation using monocular frontal view images","volume":"7","author":"Gosala","year":"2022","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B10","first-page":"2759","article-title":"\u201cSimple-bev: what really matters for multi-sensor bev perception?\u201d","volume-title":"2023 IEEE International Conference on Robotics and Automation (ICRA)","author":"Harley","year":"2022"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90","article-title":"\u201cDeep residual learning for image recognition,\u201d","author":"He","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01499","article-title":"\u201cFiery: future instance prediction in bird's-eye view from surround monocular cameras,\u201d","author":"Hu","year":"2021","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"B13","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"arXiv preprint arXiv:1412.6980"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.00978","article-title":"\u201cSeabird: segmentation in bird's view with dice loss improves monocular 3D detection of large objects,\u201d","author":"Kumar","year":"2024","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B15","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1109\/3DV.2016.32","article-title":"\u201cDeeper depth prediction with fully convolutional residual networks,\u201d","volume-title":"2016 Fourth International Conference on 3D vision (3DV)","author":"Laina","year":"2016"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i2.25233","article-title":"\u201cBevdepth: acquisition of reliable depth for multi-view 3D object detection,\u201d","author":"Li","year":"2023","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.324","article-title":"\u201cFocal loss for dense object detection,\u201d","author":"Lin","year":"2017","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"B18","doi-asserted-by":"publisher","first-page":"2024","DOI":"10.1109\/TPAMI.2015.2505283","article-title":"Learning depth from single monocular images using deep convolutional neural fields","volume":"38","author":"Liu","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B19","doi-asserted-by":"crossref","first-page":"2774","DOI":"10.1109\/ICRA48891.2023.10160968","article-title":"\u201cBevfusion: multi-task multi-sensor fusion with unified bird's-eye view representation,\u201d","volume-title":"2023 IEEE International Conference on Robotics and Automation (ICRA)","author":"Liu","year":"2023"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00217","article-title":"\u201cRoi-10D: monocular lifting of 2D detection to 6D pose and metric shape,\u201d","author":"Manhardt","year":"2019","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B21","doi-asserted-by":"publisher","first-page":"4867","DOI":"10.1109\/LRA.2020.3004325","article-title":"Cross-view semantic segmentation for sensing surroundings","volume":"5","author":"Pan","year":"2020","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B22","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1007\/978-3-030-58568-6_12","article-title":"\u201cLift, splat, shoot: encoding images from arbitrary camera rigs by implicitly unprojecting to 3d,\u201d","volume-title":"Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV 16","author":"Philion","year":"2020"},{"key":"B23","first-page":"652","article-title":"\u201cPointnet: deep learning on point sets for 3D classification and segmentation,\u201d","author":"Qi","year":"","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B24","article-title":"\u201cPointnet++: deep hierarchical feature learning on point sets in a metric space,\u201d","author":"Qi","year":"","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01196","article-title":"\u201cVision transformers for dense prediction,\u201d","author":"Ranftl","year":"2021","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01115","article-title":"\u201cPredicting semantic map representations from images using pyramid occupancy networks,\u201d","author":"Roddick","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B27","article-title":"Orthographic feature transform for monocular 3D object detection","author":"Roddick","year":"2018","journal-title":"arXiv preprint arXiv:1811.08188"},{"key":"B28","first-page":"1161","article-title":"\u201cLearning depth from single monocular images,\u201d","author":"Saxena","year":"2005","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B29","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1007\/s11263-007-0071-y","article-title":"3-D depth reconstruction from a single still image","volume":"76","author":"Saxena","year":"2008","journal-title":"Int. J. Comput. Vis"},{"key":"B30","doi-asserted-by":"crossref","first-page":"464","DOI":"10.1109\/WACV.2017.58","article-title":"\u201cCyclical learning rates for training neural networks,\u201d","volume-title":"2017 IEEE Winter Conference on Applications of Computer Vision (WACV)","author":"Smith","year":"2017"},{"key":"B31","doi-asserted-by":"publisher","first-page":"497","DOI":"10.1109\/TPAMI.2023.3322549","article-title":"Sc-depthv3: robust self-supervised monocular depth estimation for dynamic scenes","volume":"46","author":"Sun","year":"2023","journal-title":"IEEE Trans. Patt. Anal. Mach. Intell"},{"key":"B32","first-page":"6105","article-title":"\u201cEfficientnet: rethinking model scaling for convolutional neural networks,\u201d","volume-title":"International Conference on Machine Learning","author":"Tan","year":"2019"},{"key":"B33","article-title":"M2bev: Multi-camera joint 3D detection and segmentation with unified birds-eye view representation","author":"Xie","year":"2022","journal-title":"arXiv preprint arXiv:2204.05088"},{"key":"B34","first-page":"2350","article-title":"\u201cMonocular depth estimation via geometry-guided attention network,\u201d","author":"Xu","year":"","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B35","article-title":"Cobevt: cooperative bird's eye view semantic segmentation with sparse transformers","author":"Xu","year":"","journal-title":"arXiv preprint arXiv:2207.02202"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01339","article-title":"\u201cCross-view transformers for real-time map-view semantic segmentation,\u201d","author":"Zhou","year":"2022","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00472","article-title":"\u201cVoxelnet: end-to-end learning for point cloud based 3D object detection,\u201d","author":"Zhou","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"}],"container-title":["Frontiers in Big Data"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2024.1431346\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,15]],"date-time":"2024-11-15T06:15:23Z","timestamp":1731651323000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2024.1431346\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,15]]},"references-count":37,"alternative-id":["10.3389\/fdata.2024.1431346"],"URL":"https:\/\/doi.org\/10.3389\/fdata.2024.1431346","relation":{},"ISSN":["2624-909X"],"issn-type":[{"value":"2624-909X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,15]]},"article-number":"1431346"}}