{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:32:18Z","timestamp":1760239938760,"version":"build-2065373602"},"reference-count":35,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2019,1,25]],"date-time":"2019-01-25T00:00:00Z","timestamp":1548374400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Spanish MINECO\/FEDER through the SmartElderlyCar project","award":["TRA2015-70501-C2-1-R"],"award-info":[{"award-number":["TRA2015-70501-C2-1-R"]}]},{"name":"DGT through the SERMON project","award":["SPIP2017-02305"],"award-info":[{"award-number":["SPIP2017-02305"]}]},{"name":"RoboCity2030-III-CM project (Rob\u00f3tica aplicada a la mejora de la calidad  de vida de los ciudadanos. fase  III), funded by Programas de actividades I+D (CAM) and cofunded by EU Structural Fund.","award":["S2013\/MIT-2748"],"award-info":[{"award-number":["S2013\/MIT-2748"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The interest in fisheye cameras has recently risen in the autonomous vehicles field, as they are able to reduce the complexity of perception systems while improving the management of dangerous driving situations. However, the strong distortion inherent to these cameras makes the usage of conventional computer vision algorithms difficult and has prevented the development of these devices. This paper presents a methodology that provides real-time semantic segmentation on fisheye cameras leveraging only synthetic images. Furthermore, we propose some Convolutional Neural Networks(CNN) architectures based on Efficient Residual Factorized Network(ERFNet) that demonstrate notable skills handling distortion and a new training strategy that improves the segmentation on the image borders. Our proposals are compared to similar state-of-the-art works showing an outstanding performance and tested in an unknown real world scenario using a fisheye camera integrated in an open-source autonomous electric car, showing a high domain adaptation capability.<\/jats:p>","DOI":"10.3390\/s19030503","type":"journal-article","created":{"date-parts":[[2019,1,25]],"date-time":"2019-01-25T11:30:00Z","timestamp":1548415800000},"page":"503","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":36,"title":["Real-Time Semantic Segmentation for Fisheye Urban Driving Images Based on ERFNet"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2606-470X","authenticated-orcid":false,"given":"\u00c1lvaro","family":"S\u00e1ez","sequence":"first","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, Campus Universitario, 28805 Alcal\u00e1 de Henares, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0087-3077","authenticated-orcid":false,"given":"Luis M.","family":"Bergasa","sequence":"additional","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, Campus Universitario, 28805 Alcal\u00e1 de Henares, Spain"}]},{"given":"Elena","family":"L\u00f3pez-Guill\u00e9n","sequence":"additional","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, Campus Universitario, 28805 Alcal\u00e1 de Henares, Spain"}]},{"given":"Eduardo","family":"Romera","sequence":"additional","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, Campus Universitario, 28805 Alcal\u00e1 de Henares, Spain"}]},{"given":"Miguel","family":"Tradacete","sequence":"additional","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, Campus Universitario, 28805 Alcal\u00e1 de Henares, Spain"}]},{"given":"Carlos","family":"G\u00f3mez-Hu\u00e9lamo","sequence":"additional","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, Campus Universitario, 28805 Alcal\u00e1 de Henares, Spain"}]},{"given":"Javier","family":"del Egido","sequence":"additional","affiliation":[{"name":"Electronics Department, University of Alcal\u00e1, Campus Universitario, 28805 Alcal\u00e1 de Henares, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2019,1,25]]},"reference":[{"key":"ref_1","unstructured":"Romera, E., Bergasa, L.M., and Arroyo, R. (arXiv, 2016). Can we unify monocular detectors for autonomous driving by using the pixel-wise semantic segmentation of CNNs?, arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.neucom.2015.09.116","article-title":"Deep learning for visual understanding: A review","volume":"187","author":"Guo","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"7068349","DOI":"10.1155\/2018\/7068349","article-title":"Deep learning for computer vision: A brief review","volume":"2018","author":"Voulodimos","year":"2018","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_4","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (arXiv, 2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems, arXiv."},{"key":"ref_5","unstructured":"Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., and Zhang, Z. (arXiv, 2015). Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems, arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Neuhold, G., Ollmann, T., Bul\u00f2, S.R., and Kontschieder, P. (2017, January 22\u201329). The mapillary vistas dataset for semantic understanding of street scenes. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.534"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27\u201330). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"S\u00e1ez, A., Bergasa, L.M., Romera, E., Guill\u00e9n, E.L., Barea, R., and Sanz, R. (2018, January 26\u201330). Cnn-based fisheye image real-time semantic segmentation. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500456"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Fremont, V., Bui, M.T., Boukerroui, D., and Letort, P. (2016). Vision-based people detection system for heavy machine applications. Sensors, 16.","DOI":"10.3390\/s16010128"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Varga, R., Costea, A., Florea, H., Giosan, I., and Nedevschi, S. (2017, January 16\u201319). Super-sensor for 360-degree environment perception: Point cloud segmentation using image features. Proceedings of the 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, Japan.","DOI":"10.1109\/ITSC.2017.8317846"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yin, X., Wang, X., Yu, J., Zhang, M., Fua, P., and Tao, D. (arXiv, 2018). Fisheyerecnet: A multi-context collaborative deep network for fisheye image rectification, arXiv.","DOI":"10.1007\/978-3-030-01249-6_29"},{"key":"ref_12","unstructured":"Deng, F., Zhu, X., and Ren, J. (2017, January 24\u201326). Object detection on panoramic images based on deep learning. Proceedings of the 2017 3rd International Conference on Control, Automation and Robotics (ICCAR), Nagoya, Japan."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Choi, D.Y., Choi, J.H., Choi, J.W., and Song, B.C. (2017, January 17\u201320). CNN-based pre-processing and multi-frame-based view transformation for fisheye camera-based avm system. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.","DOI":"10.1109\/ICIP.2017.8297048"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Qian, Y., Yang, M., Wang, C., and Wang, B. (2017, January 11\u201314). Self-adapting part-based pedestrian detection using a fish-eye camera. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995695"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Doll\u00e1r, P., Wojek, C., Schiele, B., and Perona, P. (2009, January 20\u201325). Pedestrian detection: A benchmark. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206631"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Felzenszwalb, P., McAllester, D., and Ramanan, D. (2008, January 23\u201328). A discriminatively trained, multiscale, deformable part model. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2008), Anchorage, AK, USA.","DOI":"10.1109\/CVPR.2008.4587597"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Deng, L., Yang, M., Qian, Y., Wang, C., and Wang, B. (2017, January 11\u201314). CNN based semantic segmentation for urban traffic scenes using fisheye camera. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995725"},{"key":"ref_18","unstructured":"Deng, L., Yang, M., Li, H., Li, T., Hu, B., and Wang, C. (arXiv, 2018). Restricted deformable convolution based road scene semantic segmentation using surround view cameras, arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ros, G., Sellart, L., Materzynska, J., Vazquez, D., and Lopez, A.M. (2016, January 27\u201330). The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.352"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Shotton, J., Winn, J., Rother, C., and Criminisi, A. (2006, January 7\u201313). Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. Proceedings of the European Conference on Computer Vision, Graz, Austria.","DOI":"10.1007\/11744023_1"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","article-title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","volume":"40","author":"Chen","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017, January 22\u201329). Deformable convolutional networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.89"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2014, January 6\u201312). Spatial pyramid pooling in deep convolutional networks for visual recognition. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10578-9_23"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Romera, E., Bergasa, L.M., Alvarez, J.M., and Trivedi, M. (2018, January 26\u201330). Train here, deploy there: Robust segmentation in unseen domains. Proceedings of the IEEE Conference on Intelligent Vehicles Symposium, Changshu, China.","DOI":"10.1109\/IVS.2018.8500561"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_26","unstructured":"Yu, F., and Koltun, V. (2016, January 2\u20134). Multi-scale context aggregation by dilated convolutions. Proceedings of the International Conference on Learning Representations (ICLR), San Juan, PR, USA."},{"key":"ref_27","unstructured":"Treml, M., Arjona-Medina, J., Unterthiner, T., Durgesh, R., Friedmann, F., Schuberth, P., Mayr, A., Heusel, M., Hofmarcher, M., and Widrich, M. (2016, January 9). Speeding up semantic segmentation for autonomous driving. Proceedings of the MLITS, NIPS Workshop, Barcelona, Spain."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Romera, E., Alvarez, J.M., Bergasa, L.M., and Arroyo, R. (2017, January 11\u201314). Efficient convnet for real-time semantic segmentation. Proceedings of the 2017 IEEE Intelligent Vehicles Symposium (IV), Los Angeles, CA, USA.","DOI":"10.1109\/IVS.2017.7995966"},{"key":"ref_29","unstructured":"Paszke, A., Chaurasia, A., Kim, S., and Culurciello, E. (arXiv, 2016). Enet: A deep neural network architecture for real-time semantic segmentation, arXiv."},{"key":"ref_30","unstructured":"Badrinarayanan, V., Handa, A., and Cipolla, R. (arXiv, 2015). Segnet: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling, arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Yu, F., Koltun, V., and Funkhouser, T. (2017, January 21\u201326). Dilated residual networks. Proceedings of the Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.75"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Tradacete, M., S\u00e1ez, \u00c1., Arango, J.F., Hu\u00e9lamo, C.G., Revenga, P., Barea, R., L\u00f3pez-Guill\u00e9n, E., and Bergasa, L.M. (2018, January 22\u201323). Positioning system for an electric autonomous vehicle based on the fusion of multi-gnss rtk and odometry by using an extented kalman filter. Proceedings of the Workshop of Physical Agents, Madrid, Spain.","DOI":"10.1007\/978-3-319-99885-5_2"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Murciego, E., Hu\u00e9lamo, C.G., Barea, R., Bergasa, L.M., Romera, E., Arango, J.F., Tradacete, M., and S\u00e1ez, \u00c1. (2018, January 22\u201323). Topological road mapping for autonomous driving applications. Proceedings of the Workshop of Physical Agents, Madrid, Spain.","DOI":"10.1007\/978-3-319-99885-5_18"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Barea, R., P\u00e9rez, C., Bergasa, L.M., L\u00f3pez-Guill\u00e9n, E., Romera, E., Molinos, E., Ocana, M., and L\u00f3pez, J. (2018, January 4\u20137). Vehicle detection and localization using 3d lidar point cloud and image semantic segmentation. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569962"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/3\/503\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:28:40Z","timestamp":1760185720000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/3\/503"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,25]]},"references-count":35,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2019,2]]}},"alternative-id":["s19030503"],"URL":"https:\/\/doi.org\/10.3390\/s19030503","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,1,25]]}}}