{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,24]],"date-time":"2025-12-24T12:17:21Z","timestamp":1766578641454,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2020,2,27]],"date-time":"2020-02-27T00:00:00Z","timestamp":1582761600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Romanian Ministry of Research and Innovation, CNCS - UEFISCDI","award":["PN-III-P1-1.1-TE-2016-0440"],"award-info":[{"award-number":["PN-III-P1-1.1-TE-2016-0440"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Cameras are sensors that are available anywhere and to everyone, and can be placed easily inside vehicles. While stereovision setups of two or more synchronized cameras have the advantage of directly extracting 3D information, a single camera can be easily set up behind the windshield (like a dashcam), or above the dashboard, usually as an internal camera of a mobile phone placed there for navigation assistance. This paper presents a framework for extracting and tracking obstacle 3D data from the surrounding environment of a vehicle in traffic, using as a sensor a generic camera. The system combines the strength of Convolutional Neural Network (CNN)-based segmentation with a generic probabilistic model of the environment, the dynamic occupancy grid. The main contributions presented in this paper are the following: A method for generating the probabilistic measurement model from monocular images, based on CNN segmentation, which takes into account the particularities, uncertainties, and limitations of monocular vision; a method for automatic calibration of the extrinsic and intrinsic parameters of the camera, without the need of user assistance; the integration of automatic calibration and measurement model generation into a scene tracking system that is able to work with any camera to perceive the obstacles in real traffic. The presented system can be easily fitted to any vehicle, working standalone or together with other sensors, to enhance the environment perception capabilities and improve the traffic safety.<\/jats:p>","DOI":"10.3390\/s20051280","type":"journal-article","created":{"date-parts":[[2020,2,28]],"date-time":"2020-02-28T09:30:36Z","timestamp":1582882236000},"page":"1280","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["A Self-Calibrating Probabilistic Framework for 3D Environment Perception Using Monocular Vision"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8156-7313","authenticated-orcid":false,"given":"Razvan","family":"Itu","sequence":"first","affiliation":[{"name":"Technical University of Cluj-Napoca, St. Memorandumului 28, 400114 Cluj-Napoca, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4515-8114","authenticated-orcid":false,"given":"Radu Gabriel","family":"Danescu","sequence":"additional","affiliation":[{"name":"Technical University of Cluj-Napoca, St. Memorandumului 28, 400114 Cluj-Napoca, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,2,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2584","DOI":"10.1109\/TITS.2017.2658662","article-title":"Overview of Environment Perception for Intelligent Vehicles","volume":"18","author":"Zhu","year":"2017","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1109\/TITS.2017.2750080","article-title":"ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation","volume":"19","author":"Romera","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16\u201321). Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Xie, J., Girshick, R., and Farhadi, A. (2016, January 11\u201314). Deep3d: Fully automatic 2d-to-3d video conversion with deep convolutional neural networks. Proceedings of the European Conference on Computer Vision\u2014ECCV 2016, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46493-0_51"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Garg, R., Kumar, V., and Reid, I. (2016, January 11\u201314). Unsupervised CNN for single view depth estimation: Geometry to the rescue. Proceedings of the Computer Vision \u2013 ECCV 2016, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46484-8_45"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Godard, C., Aodha, O.M., and Brostow, G.J. (2020, February 26). Unsupervised Monocular Depth Estimation with Left-Right Consistency. Computer Vision and Pattern Recognition. Available online: http:\/\/openaccess.thecvf.com\/content_cvpr_2017\/html\/Godard_Unsupervised_Monocular_Depth_CVPR_2017_paper.html.","DOI":"10.1109\/CVPR.2017.699"},{"key":"ref_8","unstructured":"Casser, V., Pirk, S., Mahjourian, R., and Angelova, A. (2018, January 2\u20137). Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videos. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA."},{"key":"ref_9","unstructured":"Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., and Urtasun, R. (2020, February 26). Monocular 3D Object Detection for Autonomous Driving. Available online: https:\/\/www.cs.toronto.edu\/~urtasun\/publications\/chen_etal_cvpr16.pdf."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Mousavian, A., Anguelov, D., Flynn, J., and Kosecka, J. (2020, February 26). 3D Bounding Box Estimation Using Deep Learning and Geometry. Available online: https:\/\/zpascal.net\/cvpr2017\/Mousavian_3D_Bounding_Box_CVPR_2017_paper.pdf.","DOI":"10.1109\/CVPR.2017.597"},{"key":"ref_11","unstructured":"Roddick, T., Kendall, A., and Cipolla, R. (2019). Orthographic Feature Transform for Monocular 3D Object Detection. British Machine Vision Conference. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, Y., Chao, W.-L., Garg, D., Hariharan, B., Campbell, M., and Weinberger, K.Q. (2018). Pseudo-lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving. arXiv.","DOI":"10.1109\/CVPR.2019.00864"},{"key":"ref_13","unstructured":"(2019, December 16). Caltech Calibration Toolbox. Available online: http:\/\/www.vision.caltech.edu\/bouguetj\/calib_doc\/."},{"key":"ref_14","unstructured":"Wang, K., Huang, H., Li, Y., and Wang, F.Y. (2007, January 13\u201315). Research on lane-marking line based camera calibration. Proceedings of the 2007 IEEE International Conference on Vehicular Electronics and Safety, Beijing, China."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, D., Fang, B., Yang, W., Luo, X., and Tang, Y. (2014, January 18\u201319). Robust inverse perspective mapping based on vanishing point. Proceedings of the 2014 IEEE International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), Wuhan, China.","DOI":"10.1109\/SPAC.2014.6982733"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Russmusen, C. (2020, February 26). Texture-Based Vanishing Point Voting for Road Shape Estimation. Available online: http:\/\/www.bmva.org\/bmvc\/2004\/papers\/paper_261.pdf.","DOI":"10.5244\/C.18.7"},{"key":"ref_17","unstructured":"Kong, H., Audibert, J.-Y., and Ponce, J. (2020, February 26). Vanishing Point Detection for Road Detection. Available online: https:\/\/www.di.ens.fr\/willow\/pdfs\/cvpr09c.pdf."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.imavis.2005.09.023","article-title":"Inverse perspective mapping and optic flow: A calibration method and a quantitative analysis","volume":"24","author":"Tan","year":"2006","journal-title":"Image Vision Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1023\/A:1007927317325","article-title":"3d pose estimation by directly matching polyhedral models to gray value gradients","volume":"23","author":"Kollnig","year":"1997","journal-title":"Int. J. Comput. Vision"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Barth, A., and Franke, U. (2008, January 4\u20136). Where will the oncoming vehicle be the next second?. Proceedings of the IEEE Intelligent Vehicles Symposium, Eindhoven, The Netherlands.","DOI":"10.1109\/IVS.2008.4621210"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Hu, H.-N., Cai, Q.-Z., Wang, D., Lin, J., Sun, M., Kraehenbuehl, P., Darrell, T., and Yu, F. (2020, February 26). Joint Monocular 3D Vehicle Detection and Tracking. Available online: http:\/\/openaccess.thecvf.com\/content_ICCV_2019\/papers\/Hu_Joint_Monocular_3D_Vehicle_Detection_and_Tracking_ICCV_2019_paper.pdf.","DOI":"10.1109\/ICCV.2019.00549"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Danescu, R., Itu, R., and Petrovai, A. (2016). Generic Dynamic Environment Perception Using Smart Mobile Devices. Sensors, 16.","DOI":"10.3390\/s16101721"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1109\/TITS.2011.2158097","article-title":"Modeling and Tracking the Driving Environment with a Particle-Based Occupancy Grid","volume":"12","author":"Danescu","year":"2011","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27\u201330). The Cityscapes Dataset for Semantic Urban Scene Understanding. Proceedings of the Computer Vision and Pattern Recognition, Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_25","unstructured":"Yu, F., Xian, W., Chen, Y., Liu, F., Liao, M., Madhavan, V., and Darrell, T. (2018). BDD100K: A Diverse Driving Video Database with Scalable Annotation Tooling. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Neuhold, G., Ollmann, T., Bul\u00f2, S.R., and Kontschieder, P. (2017, January 22\u201329). The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.534"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Danescu, R., and Itu, R. (2019, January 3\u20136). Camera Calibration for CNN-based Generic Obstacle Detection. Proceedings of the 19th EPIA Conference on Artificial Intelligence, Vila Real, Portugal.","DOI":"10.1007\/978-3-030-30241-2_52"},{"key":"ref_28","unstructured":"(2019, December 18). Monocular Road Traffic Dataset. Available online: http:\/\/users.utcluj.ro\/~razvanitu\/dataset.html."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2020, February 26). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Available online: https:\/\/eccv2018.org\/openaccess\/content_ECCV_2018\/papers\/Liang-Chieh_Chen_Encoder-Decoder_with_Atrous_ECCV_2018_paper.pdf.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_30","unstructured":"Paszke, A., Chaurasia, A., Kim, S., and Culurciello, E. (2016). ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation. arXiv."},{"key":"ref_31","unstructured":"(2020, February 10). UGV Driver Assistant. Available online: https:\/\/play.google.com\/store\/apps\/details?id=com.infocomltd.ugvassistant&hl=en."},{"key":"ref_32","unstructured":"Chen, X., Kundu, K., Zhu, Y., Berneshawi, A.G., Ma, H., Fidler, S., and Urtasun, R. (2020, February 26). 3D Object Proposals for Accurate Object Class Detection. Available online: https:\/\/papers.nips.cc\/paper\/5644-3d-object-proposals-for-accurate-object-class-detection."},{"key":"ref_33","unstructured":"Li, B., Zhang, T., and Xia, T. (2016). Vehicle detection from 3d lidar using fully convolutional network. Robotics: Science and Systems. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/5\/1280\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:02:09Z","timestamp":1760173329000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/5\/1280"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,27]]},"references-count":33,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["s20051280"],"URL":"https:\/\/doi.org\/10.3390\/s20051280","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,2,27]]}}}