{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T08:16:17Z","timestamp":1782375377992,"version":"3.54.5"},"reference-count":35,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2019,7,22]],"date-time":"2019-07-22T00:00:00Z","timestamp":1563753600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Bayrische Forschungsstifung","award":["AZ-1310-17"],"award-info":[{"award-number":["AZ-1310-17"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Typically, lane departure warning systems rely on lane lines being present on the road.However, in many scenarios, e.g., secondary roads or some streets in cities, lane lines are eithernot present or not sufficiently well signaled. In this work, we present a vision-based method tolocate a vehicle within the road when no lane lines are present using only RGB images as input.To this end, we propose to fuse together the outputs of a semantic segmentation and a monoculardepth estimation architecture to reconstruct locally a semantic 3D point cloud of the viewed scene.We only retain points belonging to the road and, additionally, to any kind of fences or walls thatmight be present right at the sides of the road. We then compute the width of the road at a certainpoint on the planned trajectory and, additionally, what we denote as the fence-to-fence distance.Our system is suited to any kind of motoring scenario and is especially useful when lane lines arenot present on the road or do not signal the path correctly. The additional fence-to-fence distancecomputation is complementary to the road\u2019s width estimation. We quantitatively test our methodon a set of images featuring streets of the city of Munich that contain a road-fence structure, so asto compare our two proposed variants, namely the road\u2019s width and the fence-to-fence distancecomputation. In addition, we also validate our system qualitatively on the Stuttgart sequence of thepublicly available Cityscapes dataset, where no fences or walls are present at the sides of the road,thus demonstrating that our system can be deployed in a standard city-like environment. For thebenefit of the community, we make our software open source.<\/jats:p>","DOI":"10.3390\/s19143224","type":"journal-article","created":{"date-parts":[[2019,7,22]],"date-time":"2019-07-22T11:07:28Z","timestamp":1563793648000},"page":"3224","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["SemanticDepth: Fusing Semantic Segmentation and Monocular Depth Estimation for Enabling Autonomous Driving in Roads without Lane Lines"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5944-0938","authenticated-orcid":false,"given":"Pablo R.","family":"Palafox","sequence":"first","affiliation":[{"name":"Institute of Automotive Technology, Technical University of Munich, Boltzmannstr. 15, 85748 Garching bei M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9197-2849","authenticated-orcid":false,"given":"Johannes","family":"Betz","sequence":"additional","affiliation":[{"name":"Institute of Automotive Technology, Technical University of Munich, Boltzmannstr. 15, 85748 Garching bei M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6939-7315","authenticated-orcid":false,"given":"Felix","family":"Nobis","sequence":"additional","affiliation":[{"name":"Institute of Automotive Technology, Technical University of Munich, Boltzmannstr. 15, 85748 Garching bei M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6646-0585","authenticated-orcid":false,"given":"Konstantin","family":"Riedl","sequence":"additional","affiliation":[{"name":"Institute of Automotive Technology, Technical University of Munich, Boltzmannstr. 15, 85748 Garching bei M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus","family":"Lienkamp","sequence":"additional","affiliation":[{"name":"Institute of Automotive Technology, Technical University of Munich, Boltzmannstr. 15, 85748 Garching bei M\u00fcnchen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,7,22]]},"reference":[{"key":"ref_1","unstructured":"(2019, July 20). Roborace. Available online: https:\/\/roborace.com\/."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Betz, J., Wischnewski, A., Heilmeier, A., Nobis, F., Stahl, T., Hermansdorfer, L., and Lienkamp, M. (2018, January 12\u201313). What can we learn from autonomous level-5 motorsport?. Proceedings of the 9th International Munich Chassis Symposium, Munich, Germany.","DOI":"10.1007\/978-3-658-22050-1_12"},{"key":"ref_3","first-page":"4","article-title":"Lane detection techniques: A review","volume":"112","author":"Kaur","year":"2015","journal-title":"Int. J. Comput. Appl."},{"key":"ref_4","unstructured":"Saha, A., Roy, D.D., Alam, T., and Deb, K. (2012). Automated road lane detection for intelligent vehicles. Glob. J. Comput. Sci. Technol., 12, Available online: https:\/\/computerresearch.org\/index.php\/computer\/article\/view\/472\/472."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1109\/TITS.2007.908582","article-title":"Robust lane detection and tracking in challenging scenarios","volume":"9","author":"Kim","year":"2008","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Aly, M. (2008, January 4\u20136). Real time detection of lane markers in urban streets. Proceedings of the 2008 IEEE Intelligent Vehicles Symposium, Eindhoven, The Netherlands.","DOI":"10.1109\/IVS.2008.4621152"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Haloi, M., and Jayagopi, D.B. (July, January 28). A robust lane detection and departure warning system. Proceedings of the 2015 IEEE Intelligent Vehicles Symposium (IV), Seoul, Korea.","DOI":"10.1109\/IVS.2015.7225674"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhang, W. (2010, January 21\u201324). Lidar-based road and road-edge detection. Proceedings of the 2010 IEEE Intelligent Vehicles Symposium, La Jolla, CA, USA.","DOI":"10.1109\/IVS.2010.5548134"},{"key":"ref_9","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 2\u20138). Imagenet classification with deep convolutional neural networks. Proceedings of the Neural Information Processing Systems 2012, Lake Tahoe, NV, USA."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Shotton, J., Johnson, M., and Cipolla, R. (2008, January 23\u201328). Semantic texton forests for image categorization and segmentation. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.","DOI":"10.1109\/CVPR.2008.4587503"},{"key":"ref_12","unstructured":"Ciresan, D., Giusti, A., Gambardella, L.M., and Schmidhuber, J. (2012, January 2\u20138). Deep neural networks segment neuronal membranes in electron microscopy images. Proceedings of the Neural Information Processing Systems 2012, Lake Tahoe, NV, USA."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The pascal visual object classes (voc) challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_15","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_17","unstructured":"Chen, L.C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2016). Pyramid scene parsing network. arXiv.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_19","unstructured":"Paszke, A., Chaurasia, A., Kim, S., and Culurciello, E. (2016). Enet: A deep neural network architecture for real-time semantic segmentation. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Romera, E., Alvarez, J.M., Bergasa, L.M., and Arroyo, R. (2017, January 11\u201314). Efficient convnet for real-time semantic segmentation. Proceedings of the Intelligent Vehicles Symposium (IV), Redondo Beach, CA, USA.","DOI":"10.1109\/IVS.2017.7995966"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1023\/A:1014573219977","article-title":"A taxonomy and evaluation of dense two-frame stereo correspondence algorithms","volume":"47","author":"Scharstein","year":"2002","journal-title":"Int. J. Comput. Vis."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/0600000052","article-title":"Multi-view stereo: A tutorial","volume":"9","author":"Furukawa","year":"2015","journal-title":"Found. Trends\u00ae Comput. Graph. Vis."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ranftl, R., Vineet, V., Chen, Q., and Koltun, V. (2016, January 27\u201330). Dense monocular depth estimation in complex dynamic scenes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.440"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Abrams, A., Hawley, C., and Pless, R. (2012). Heliometric stereo: Shape from sun position. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-642-33709-3_26"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Godard, C., Mac Aodha, O., and Brostow, G.J. (2017, January 21\u201326). Unsupervised monocular depth estimation with left-right consistency. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.699"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Mayer, N., Ilg, E., Hausser, P., Fischer, P., Cremers, D., Dosovitskiy, A., and Brox, T. (2016, January 27\u201330). A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.438"},{"key":"ref_27","first-page":"2","article-title":"Stereo matching by training a convolutional neural network to compare image patches","volume":"17","author":"Zbontar","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jiao, J., Cao, Y., Song, Y., and Lau, R. (2018, January 8\u201314). Look deeper into depth: Monocular depth estimation with semantic booster and attention-driven loss. Proceedings of the European Conference on Computer Visio (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01267-0_4"},{"key":"ref_29","unstructured":"Ramirez, P.Z., Poggi, M., Tosi, F., Mattoccia, S., and Di Stefano, L. (2018, January 2\u20136). Geometry meets semantics for semi-supervised monocular depth estimation. Proceedings of the Asian Conference on Computer Vision, Perth, Australia."},{"key":"ref_30","unstructured":"Hazirbas, C., Ma, L., Domokos, C., and Cremers, D. (2016, January 20\u201324). Fusenet: Incorporating depth into semantic segmentation via fusion-based CNN architecture. Proceedings of the Asian Conference on Computer Vision, Taipei, Taiwan."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Lin, X., S\u00e1nchez-Escobedo, D., Casas, J.R., and Pard\u00e0s, M. (2019). Depth estimation and semantic segmentation from a single RGB image using a hybrid convolutional neural network. Sensors, 19.","DOI":"10.3390\/s19081795"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27\u201330). The Cityscapes Dataset for Semantic Urban Scene Understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Hartley, R., and Zisserman, A. (2003). Multiple View Geometry in Computer Vision, Cambridge University Press.","DOI":"10.1017\/CBO9780511811685"},{"key":"ref_34","unstructured":"Bradski, G., and Kaehler, A. (2008). Learning OpenCV: Computer Vision with the OpenCV Library, O\u2019Reilly Media, Inc."},{"key":"ref_35","unstructured":"Zhou, Q.Y., Park, J., and Koltun, V. (2018). Open3D: A Modern Library for 3D Data Processing. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/14\/3224\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:08:25Z","timestamp":1760188105000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/14\/3224"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,22]]},"references-count":35,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2019,7]]}},"alternative-id":["s19143224"],"URL":"https:\/\/doi.org\/10.3390\/s19143224","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,22]]}}}