{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:54:44Z","timestamp":1760144084730,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2024,3,15]],"date-time":"2024-03-15T00:00:00Z","timestamp":1710460800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Mitacs Accelerate","award":["IT18048"],"award-info":[{"award-number":["IT18048"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>The application of large field-of-view (FoV) cameras equipped with fish-eye lenses brings notable advantages to various real-world computer vision applications, including autonomous driving. While deep learning has proven successful in conventional computer vision applications using regular perspective images, its potential in fish-eye camera contexts remains largely unexplored due to limited datasets for fully supervised learning. Semi-supervised learning comes as a potential solution to manage this challenge. In this study, we explore and benchmark two popular semi-supervised methods from the perspective image domain for fish-eye image segmentation. We further introduce FishSegSSL, a novel fish-eye image segmentation framework featuring three semi-supervised components: pseudo-label filtering, dynamic confidence thresholding, and robust strong augmentation. Evaluation on the WoodScape dataset, collected from vehicle-mounted fish-eye cameras, demonstrates that our proposed method enhances the model\u2019s performance by up to 10.49% over fully supervised methods using the same amount of labeled data. Our method also improves the existing image segmentation methods by 2.34%. To the best of our knowledge, this is the first work on semi-supervised semantic segmentation on fish-eye images. Additionally, we conduct a comprehensive ablation study and sensitivity analysis to showcase the efficacy of each proposed method in this research.<\/jats:p>","DOI":"10.3390\/jimaging10030071","type":"journal-article","created":{"date-parts":[[2024,3,15]],"date-time":"2024-03-15T14:05:25Z","timestamp":1710511525000},"page":"71","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["FishSegSSL: A Semi-Supervised Semantic Segmentation Framework for Fish-Eye Images"],"prefix":"10.3390","volume":"10","author":[{"given":"Sneha","family":"Paul","sequence":"first","affiliation":[{"name":"Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, QC H3G1M8, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zachary","family":"Patterson","sequence":"additional","affiliation":[{"name":"Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, QC H3G1M8, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7224-7940","authenticated-orcid":false,"given":"Nizar","family":"Bouguila","sequence":"additional","affiliation":[{"name":"Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, QC H3G1M8, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,15]]},"reference":[{"key":"ref_1","unstructured":"Yogamani, S., Hughes, C., Horgan, J., Sistu, G., Varley, P., O\u2019Dea, D., Uric\u00e1r, M., Milz, S., Simon, M., and Amende, K. (November, January 27). Woodscape: A multi-task, multi-camera fisheye dataset for autonomous driving. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Talaoubrid, H., Vert, M., Hayat, K., and Magnier, B. (2022). Human tracking in top-view fisheye images: Analysis of familiar similarity measures via hog and against various color spaces. J. Imaging, 8.","DOI":"10.3390\/jimaging8040115"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Deng, L., Yang, M., Qian, Y., Wang, C., and Wang, B. (2017, January 11\u201314). CNN based semantic segmentation for urban traffic scenes using fisheye camera. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995725"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kumar, V.R., Klingner, M., Yogamani, S., Milz, S., Fingscheidt, T., and Mader, P. (2021, January 5\u20139). Syndistnet: Self-supervised monocular fisheye camera distance estimation synergized with semantic segmentation for autonomous driving. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Virtual.","DOI":"10.1109\/WACV48630.2021.00011"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Shi, H., Li, Y., Yang, K., Zhang, J., Peng, K., Roitberg, A., Ye, Y., Ni, H., Wang, K., and Stiefelhagen, R. (2023). FishDreamer: Towards Fisheye Semantic Completion via Unified Image Outpainting and Segmentation. arXiv.","DOI":"10.1109\/CVPRW59228.2023.00684"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Khayretdinova, G., Apprato, D., and Gout, C. (2023). A Level Set-Based Model for Image Segmentation under Geometric Constraints and Data Approximation. J. Imaging, 10.","DOI":"10.3390\/jimaging10010002"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Apud Baca, J.G., Jantos, T., Theuermann, M., Hamdad, M.A., Steinbrener, J., Weiss, S., Almer, A., and Perko, R. (2021). Automated Data Annotation for 6-DoF AI-Based Navigation Algorithm Development. J. Imaging, 7.","DOI":"10.3390\/jimaging7110236"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1109\/TITS.2020.2972974","article-title":"Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges","volume":"22","author":"Feng","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Peng, X., Wang, K., Zhang, Z., Geng, N., and Zhang, Z. (2023). A Point-Cloud Segmentation Network Based on SqueezeNet and Time Series for Plants. J. Imaging, 9.","DOI":"10.3390\/jimaging9120258"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Valada, A., Oliveira, G.L., Brox, T., and Burgard, W. (2017, January 3\u20136). Deep multispectral semantic scene understanding of forested environments using multimodal fusion. Proceedings of the 2016 International Symposium on Experimental Robotics, Tokyo, Japan.","DOI":"10.1007\/978-3-319-50115-4_41"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2576","DOI":"10.1109\/LRA.2019.2904733","article-title":"RTFNet: RGB-thermal fusion network for semantic segmentation of urban scenes","volume":"4","author":"Sun","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Paul, S., Patterson, Z., and Bouguila, N. (2023, January 15\u201317). Semantic Segmentation Using Transfer Learning on Fisheye Images. Proceedings of the 22nd IEEE International Conference on Machine Learning and Applications (ICMLA), Jacksonville, FL, USA.","DOI":"10.1109\/ICMLA58977.2023.00068"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ye, Y., Yang, K., Xiang, K., Wang, J., and Wang, K. (2020, January 11\u201314). Universal semantic segmentation for fisheye urban driving images. Proceedings of the 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Toronto, ON, Canada.","DOI":"10.1109\/SMC42975.2020.9283099"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Blott, G., Takami, M., and Heipke, C. (2018, January 8\u201314). Semantic segmentation of fisheye images. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, Germany.","DOI":"10.1007\/978-3-030-11009-3_10"},{"key":"ref_15","unstructured":"Playout, C., Ahmad, O., Lecue, F., and Cheriet, F. (2021). Adaptable deformable convolutions for semantic segmentation of fisheye images in autonomous driving systems. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"S\u00e1ez, \u00c1., Bergasa, L.M., L\u00f3pez-Guill\u00e9n, E., Romera, E., Tradacete, M., G\u00f3mez-Hu\u00e9lamo, C., and Del Egido, J. (2019). Real-time semantic segmentation for fisheye urban driving images based on ERFNet. Sensors, 19.","DOI":"10.3390\/s19030503"},{"key":"ref_17","unstructured":"Tarvainen, A., and Valpola, H. (2017). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chen, X., Yuan, Y., Zeng, G., and Wang, J. (2021, January 20\u201325). Semi-supervised semantic segmentation with cross pseudo supervision. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00264"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1109\/JSTARS.2022.3225150","article-title":"Improved Swin Transformer-Based Semantic Segmentation of Postearthquake Dense Buildings in Urban Areas Using Remote Sensing Images","volume":"16","author":"Cui","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., and Torr, P.H. (2021, January 20\u201325). Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00681"},{"key":"ref_21","first-page":"4971","article-title":"Segvit: Semantic segmentation with plain vision transformers","volume":"35","author":"Zhang","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_22","first-page":"12077","article-title":"SegFormer: Simple and efficient design for semantic segmentation with transformers","volume":"34","author":"Xie","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Arsenali, B., Viswanath, P., and Novosel, J. (2019, January 27\u201328). Rotinvmtl: Rotation invariant multinet on fisheye images for autonomous driving applications. Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea.","DOI":"10.1109\/ICCVW.2019.00291"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"48480","DOI":"10.1109\/ACCESS.2023.3276985","article-title":"Surround-view Fisheye Camera Viewpoint Augmentation for Image Semantic Segmentation","volume":"11","author":"Cho","year":"2023","journal-title":"IEEE Access"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kumar, V.R., Hiremath, S.A., Bach, M., Milz, S., Witt, C., Pinard, C., Yogamani, S., and M\u00e4der, P. (August, January 31). Fisheyedistancenet: Self-supervised scale-aware distance estimation using monocular fisheye camera for autonomous driving. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197319"},{"key":"ref_26","unstructured":"Ramachandran, S., Sistu, G., McDonald, J., and Yogamani, S. (2021). Woodscape Fisheye Semantic Segmentation for Autonomous Driving\u2013CVPR 2021 OmniCV Workshop Challenge. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2830","DOI":"10.1109\/LRA.2021.3062324","article-title":"Omnidet: Surround view cameras based multi-task visual perception network for autonomous driving","volume":"6","author":"Kumar","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Schneider, L., Jasch, M., Fr\u00f6hlich, B., Weber, T., Franke, U., Pollefeys, M., and R\u00e4tsch, M. (2017, January 12\u201314). Multimodal neural networks: RGB-D for semantic segmentation and object detection. Proceedings of the Image Analysis: 20th Scandinavian Conference, SCIA 2017, Troms\u00f8, Norway. Proceedings, Part I 20.","DOI":"10.1007\/978-3-319-59126-1_9"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"Segnet: A deep convolutional encoder-decoder architecture for image segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Teichmann, M., Weber, M., Zoellner, M., Cipolla, R., and Urtasun, R. (2018, January 26\u201330). Multinet: Real-time joint semantic reasoning for autonomous driving. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500504"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3580","DOI":"10.1109\/LRA.2020.2978666","article-title":"Semantic segmentation with unsupervised domain adaptation under varying weather conditions for autonomous vehicles","volume":"5","author":"Erkent","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_33","unstructured":"Kalluri, T., Varma, G., Chandraker, M., and Jawahar, C. (November, January 27). Universal semi-supervised semantic segmentation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Huo, X., Xie, L., He, J., Yang, Z., Zhou, W., Li, H., and Tian, Q. (2021, January 20\u201325). ATSO: Asynchronous teacher-student optimization for semi-supervised image segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00129"},{"key":"ref_35","unstructured":"Hung, W.C., Tsai, Y.H., Liou, Y.T., Lin, Y.Y., and Yang, M.H. (2018). Adversarial learning for semi-supervised semantic segmentation. arXiv."},{"key":"ref_36","unstructured":"Novosel, J., Viswanath, P., and Arsenali, B. (2019, January 13). Boosting semantic segmentation with multi-task self-supervised learning for autonomous driving applications. Proceedings of the NeurIPS-Workshops, Vancouver, BC, Canada."},{"key":"ref_37","unstructured":"French, G., Laine, S., Aila, T., Mackiewicz, M., and Finlayson, G. (2019). Semi-supervised semantic segmentation needs strong, varied perturbations. arXiv."},{"key":"ref_38","first-page":"596","article-title":"Fixmatch: Simplifying semi-supervised learning with consistency and confidence","volume":"33","author":"Sohn","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_39","first-page":"18408","article-title":"Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling","volume":"34","author":"Zhang","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Gui, G., Zhao, Z., Qi, L., Zhou, L., Wang, L., and Shi, Y. (2023, January 2\u20133). Enhancing Sample Utilization through Sample Adaptive Augmentation in Semi-Supervised Learning. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Paris, France.","DOI":"10.1109\/ICCV51070.2023.01455"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1109\/TSMC.1979.4310076","article-title":"A threshold selection method from gray-level histograms","volume":"9","author":"Otsu","year":"1979","journal-title":"IEEE Trans. Syst. Man Cybern."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/10\/3\/71\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:14:27Z","timestamp":1760105667000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/10\/3\/71"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,15]]},"references-count":41,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["jimaging10030071"],"URL":"https:\/\/doi.org\/10.3390\/jimaging10030071","relation":{},"ISSN":["2313-433X"],"issn-type":[{"type":"electronic","value":"2313-433X"}],"subject":[],"published":{"date-parts":[[2024,3,15]]}}}