{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T00:47:08Z","timestamp":1775695628550,"version":"3.50.1"},"reference-count":68,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2019,5,3]],"date-time":"2019-05-03T00:00:00Z","timestamp":1556841600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, one solution for an end-to-end deep neural network for autonomous driving is presented. The main objective of our work was to achieve autonomous driving with a light deep neural network suitable for deployment on embedded automotive platforms. There are several end-to-end deep neural networks used for autonomous driving, where the input to the machine learning algorithm are camera images and the output is the steering angle prediction, but those convolutional neural networks are significantly more complex than the network architecture we are proposing. The network architecture, computational complexity, and performance evaluation during autonomous driving using our network are compared with two other convolutional neural networks that we re-implemented with the aim to have an objective evaluation of the proposed network. The trained model of the proposed network is four times smaller than the PilotNet model and about 250 times smaller than AlexNet model. While complexity and size of the novel network are reduced in comparison to other models, which leads to lower latency and higher frame rate during inference, our network maintained the performance, achieving successful autonomous driving with similar efficiency compared to autonomous driving using two other models. Moreover, the proposed deep neural network downsized the needs for real-time inference hardware in terms of computational power, cost, and size.<\/jats:p>","DOI":"10.3390\/s19092064","type":"journal-article","created":{"date-parts":[[2019,5,7]],"date-time":"2019-05-07T03:15:46Z","timestamp":1557198946000},"page":"2064","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":126,"title":["An End-to-End Deep Neural Network for Autonomous Driving Designed for Embedded Automotive Platforms"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6229-5823","authenticated-orcid":false,"given":"Jelena","family":"Koci\u0107","sequence":"first","affiliation":[{"name":"School of Electrical Engineering, University of Belgrade, 11120 Belgrade, Serbia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0633-324X","authenticated-orcid":false,"given":"Nenad","family":"Jovi\u010di\u0107","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, University of Belgrade, 11120 Belgrade, Serbia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6437-5408","authenticated-orcid":false,"given":"Vujo","family":"Drndarevi\u0107","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, University of Belgrade, 11120 Belgrade, Serbia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,5,3]]},"reference":[{"key":"ref_1","unstructured":"Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L., Monfort, M., Muller, U., and Zhang, J. (arXiv, 2016). End to end learning for self-driving cars, arXiv."},{"key":"ref_2","unstructured":"Bojarski, M., Yeres, P., Choromanska, A., Choromanski, K., Firner, B., Jackel, L., and Muller, U. (arXiv, 2017). Explaining how a deep neural network trained with end-to-end learning steers a car, arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Mehta, A., Adithya, S., and Anbumani, S. (arXiv, 2018). Learning end-to-end autonomous driving using guided auxiliary supervision, arXiv.","DOI":"10.1145\/3293353.3293364"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chen, Y., Wang, J., Li, J., Lu, C., Luo, Z., Xue, H., and Wang, C. (2018, January 18\u201323). LiDAR-Video Driving Dataset: Learning Driving Policies Effectively. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00615"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ramezani Dooraki, A., and Lee, D.-J. (2018). An End-to-End Deep Reinforcement Learning-Based Intelligent Agent Capable of Autonomous Exploration in Unknown Environments. Sensors, 18.","DOI":"10.3390\/s18103575"},{"key":"ref_6","unstructured":"Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation, Inc."},{"key":"ref_7","unstructured":"Udacity, Inc. (2018, November 05). Self-Driving Car Simulator. Available online: https:\/\/github.com\/udacity\/self-driving-car-sim."},{"key":"ref_8","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2017). Deep Learning, The MIT Press. Available online: https:\/\/www.deeplearningbook.org."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Aggarwal, C.C. (2018). Neural Networks and Deep Learning, Springer International Publishing.","DOI":"10.1007\/978-3-319-94463-0"},{"key":"ref_10","unstructured":"Chollet, F. (2018). Deep Learning with Python, Manning Publications."},{"key":"ref_11","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning, The MIT Press. [2nd ed.]."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1162\/neco.1989.1.4.541","article-title":"Backpropagation applied to handwritten zip code recognition","volume":"1","author":"LeCun","year":"1989","journal-title":"Neural Comput."},{"key":"ref_13","unstructured":"LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (2019, May 01). Gradient-Based Learning Applied to Document Recognition. Available online: http:\/\/yann.lecun.org\/exdb\/publis\/pdf\/lecun-01a.pdf."},{"key":"ref_14","unstructured":"Simard, D., Steinkraus, P.Y., and Platt, J.C. (2003, January 6). Best practices for convolutional neural networks applied to visual document analysis. Proceedings of the Seventh International Conference on Document Analysis and Recognition, Edinburgh, UK."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1285","DOI":"10.1109\/TMI.2016.2528162","article-title":"Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning","volume":"35","author":"Shin","year":"2016","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Pathak, D., Kr\u00e4henb\u00fchl, P., Donahue, J., Darrell, T., and Efros, A.A. (2016, January 27\u201330). Context Encoders: Feature Learning by Inpainting. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.278"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., and Li, F.-F. (2014, January 23\u201328). Large-Scale Video Classification with Convolutional Neural Networks. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.223"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1305","DOI":"10.3390\/rs9121305","article-title":"Prediction of Arctic Sea Ice Concentration Using a Fully Data Driven Deep Neural Network","volume":"9","author":"Chi","year":"2017","journal-title":"Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_20","unstructured":"Simonyan, K., and Zisserman, A. (arXiv, 2004). Very deep convolutional networks for large-scale image recognition, arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_23","unstructured":"Visin, F., Kastner, K., Cho, K., Matteucci, M., Courville, A., and Bengio, Y. (arXiv, 2015). Renet: A recurrent neural network based alternative to convolutional networks, arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018, January 18\u201323). Learning Transferable Architectures for Scalable Image Recognition. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00907"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Acuna, D., Ling, H., Kar, A., and Fidler, S. (2018, January 18\u201323). Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00096"},{"key":"ref_26","unstructured":"Wang, T.C., Liu, M.Y., Zhu, J.Y., Liu, G., Tao, A., Kautz, J., and Catanzaro, B. (2018, January 3\u20138). Video-to-video synthesis. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"ref_28","unstructured":"Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., and Lillicrap, T. (arXiv, 2017). Mastering chess and shogi by self-play with a general reinforcement learning algorithm, arXiv."},{"key":"ref_29","unstructured":"Amodei, D., Ananthanarayanan, S., Anubhai, R., Bai, J., Battenberg, E., Case, C., and Chen, J. (2016, January 19\u201324). Deep speech 2: End-to-end speech recognition in English and Mandarin. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Chen, Z., Zhang, T., and Ouyang, C. (2018). End-to-End Airplane Detection Using Transfer Learning in Remote Sensing Images. Remote Sens., 10.","DOI":"10.3390\/rs10010139"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1584","DOI":"10.3390\/app8091584","article-title":"End-To-End Convolutional Neural Network Model for Gear Fault Diagnosis Based on Sound Signals","volume":"8","author":"Yao","year":"2018","journal-title":"Appl. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kanade, T., Thorpe, C., and Whittaker, W. (1986, January 4\u20136). Autonomous land vehicle project at CMU. Proceedings of the 1986 ACM fourteenth annual conference on Computer science, Cincinnati, OH, USA.","DOI":"10.1145\/324634.325197"},{"key":"ref_33","unstructured":"Wallace, R. (1985, January 18\u201323). First results in robot road-following. Proceedings of the 9th international joint conference on Artificial intelligence, Los Angeles, CA, USA."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/S1474-6670(17)55320-3","article-title":"Autonomous High Speed Road Vehicle Guidance by Computer Vision","volume":"20","author":"Dickmanns","year":"1987","journal-title":"IFAC Proc. Vol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1002\/rob.20147","article-title":"Stanley: The Robot That Won the DARPA Grand Challenge","volume":"23","author":"Thrun","year":"2006","journal-title":"J. Field Robot."},{"key":"ref_36","unstructured":"Montemerlo, M., Thrun, S., Dahlkamp, H., Stavens, D., and Strohband, S. (2006, January 16\u201320). Winning the DARPA grand challenge with an AI robot. Proceedings of the 21st national conference on Artificial intelligence, Boston, MA, USA."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Buehler, M., Iagnemma, K., and Singh, S. (2009). The DARPA Urban Challenge: Autonomous Vehicles in City Traffic. Springer Tracts in Advanced Robotics, Springer.","DOI":"10.1007\/978-3-642-03991-1"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Xu, D., Jain, A., and Anguelov, D. (2018, January 18\u201323). PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00033"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"7258","DOI":"10.1109\/JSEN.2016.2598600","article-title":"Fast Occupancy Grid Filtering Using Grid Cell Clusters from LIDAR and Stereo Vision Sensor Data","volume":"16","author":"Oh","year":"2016","journal-title":"IEEE Sens. J."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1109\/TITS.2015.2479925","article-title":"Multiple Sensor Fusion and Classification for Moving Object Detection and Tracking","volume":"17","author":"Aycard","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Cho, H., Seo, Y., Vijaya Kumar, B.V.K., and Rajkumar, R.R. (June, January 31). A multi-sensor fusion system for moving object detection and tracking in urban driving environments. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.","DOI":"10.1109\/ICRA.2014.6907100"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Ravankar, A., Ravankar, A.A., Kobayashi, Y., Hoshino, Y., and Peng, C.-C. (2018). Path Smoothing Techniques in Robot Navigation: State-of-the-Art, Current and Future Challenges. Sensors, 18.","DOI":"10.3390\/s18093170"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Wei, K., and Ren, B. (2018). A Method on Dynamic Path Planning for Robotic Manipulator Autonomous Obstacle Avoidance Based on an Improved RRT Algorithm. Sensors, 18.","DOI":"10.3390\/s18020571"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Cai, W., Zhang, M., and Zheng, Y.R. (2017). Task Assignment and Path Planning for Multiple Autonomous Underwater Vehicles Using 3D Dubins Curves. Sensors, 17.","DOI":"10.3390\/s17071607"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1109\/TIV.2016.2578706","article-title":"A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles","volume":"1","author":"Paden","year":"2016","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1109\/TITS.2015.2498841","article-title":"A Review of Motion Planning Techniques for Automated Vehicles","volume":"17","author":"Nashashibi","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Sung, Y., Jin, Y., Kwak, J., Lee, S.-G., and Cho, K. (2018). Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing. Sustainability, 10.","DOI":"10.3390\/su10030816"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Koci\u0107, J., Jovi\u010di\u0107, N., and Drndarevi\u0107, V. (2018, January 21\u201323). Driver behavioral cloning using deep learning. Proceedings of the 17th International Symposium INFOTEH-JAHORINA (INFOTEH), East Sarajevo, Republika Srpska.","DOI":"10.1109\/INFOTEH.2018.8345542"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Navarro, A., Joerdening, J., Khalil, R., Brown, A., and Asher, Z. (2018). Development of an Autonomous Vehicle Control Strategy Using a Single Camera and Deep Neural Networks, SAE. SAE Technical Paper No. 2018-01-0035.","DOI":"10.4271\/2018-01-0035"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Riedmiller, M., Montemerlo, M., and Dahlkamp, H. (2007, January 11\u201313). Learning to Drive a Real Car in 20 Minutes. Proceedings of the 2007 Frontiers in the Convergence of Bioscience and Information Technologies, Jeju City, Korea.","DOI":"10.1109\/FBIT.2007.37"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Wu, K., Abolfazli Esfahani, M., Yuan, S., and Wang, H. (2018). Learn to Steer through Deep Reinforcement Learning. Sensors, 18.","DOI":"10.3390\/s18113650"},{"key":"ref_52","unstructured":"Shalev-Shwartz, S., Shammah, S., and Shashua, A. (arXiv, 2016). Safe, multi-agent, reinforcement learning for autonomous driving, arXiv."},{"key":"ref_53","unstructured":"Fridman, L., Jenik, B., and Terwilliger, J. (arXiv, 2018). Deeptraffic: Driving fast through dense traffic with deep reinforcement learning, arXiv."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Lee, K.W., Yoon, H.S., Song, J.M., and Park, K.R. (2018). Convolutional Neural Network-Based Classification of Driver\u2019s Emotion during Aggressive and Smooth Driving Using Multi-Modal Camera Sensors. Sensors, 18.","DOI":"10.3390\/s18040957"},{"key":"ref_55","unstructured":"Fridman, L. (arXiv, 2018). Human-Centered Autonomous Vehicle Systems: Principles of Effective Shared Autonomy, arXiv."},{"key":"ref_56","unstructured":"Teti, M., Barenholtz, E., Martin, S., and Hahn, W. (arXiv, 2018). A Systematic Comparison of Deep Learning Architectures in an Autonomous Vehicle, arXiv."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Koci\u0107, J., Jovi\u010di\u0107, N., and Drndarevi\u0107, V. (2018, January 20\u201321). Sensors and Sensor Fusion in Autonomous Vehicles. Proceedings of the 2018 26th Telecommunications Forum (TELFOR), Belgrade, Serbia.","DOI":"10.1109\/TELFOR.2018.8612054"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Iandola, F., and Keutzer, K. (2017, January 15\u201320). Keynote: small neural nets are beautiful: enabling embedded systems with small deep-neural-network architectures. Proceedings of the 2017 International Conference on Hardware\/Software Codesign and System Synthesis, Seoul, Korea.","DOI":"10.1145\/3125502.3125606"},{"key":"ref_59","unstructured":"Glorot, X., and Bengio, Y. (2010, January 13\u201315). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, Sardinia, Italy."},{"key":"ref_60","first-page":"94","article-title":"Computational complexity of neural networks: A survey","volume":"1994","author":"Orponen","year":"1994","journal-title":"Nord. J. Comput."},{"key":"ref_61","unstructured":"Raghu, M., Poole, B., Kleinberg, J., Ganguli, S., and Sohl Dickstein, J. (2017, January 6\u201311). On the expressive power of deep neural networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"2295","DOI":"10.1109\/JPROC.2017.2761740","article-title":"Efficient Processing of Deep Neural Networks: A Tutorial and Survey","volume":"105","author":"Sze","year":"2017","journal-title":"Proc. IEEE"},{"key":"ref_63","unstructured":"Koci\u0107, J. (2019, February 22). Model Shallow. Available online: https:\/\/www.youtube.com\/watch?v=w4UUz-gI7yw."},{"key":"ref_64","unstructured":"Gulli, A., and Pal, S. (2017). Deep Learning with Keras, Packt Publishing."},{"key":"ref_65","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference for Learning Representations, San Diego, CA, USA."},{"key":"ref_66","unstructured":"Koci\u0107, J. (2018, November 05). Model AlexNet. Available online: https:\/\/www.youtube.com\/watch?v=ICxRQfm5a_4."},{"key":"ref_67","unstructured":"Koci\u0107, J. (2018, November 05). Model PilotNet (NVIDIA Net). Available online: https:\/\/www.youtube.com\/watch?v=YPZRKh4xbm4."},{"key":"ref_68","unstructured":"Koci\u0107, J. (2018, November 05). Model J-Net. Available online: https:\/\/www.youtube.com\/watch?v=BB-YljzloWI."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/9\/2064\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:48:56Z","timestamp":1760186936000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/9\/2064"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,3]]},"references-count":68,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2019,5]]}},"alternative-id":["s19092064"],"URL":"https:\/\/doi.org\/10.3390\/s19092064","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5,3]]}}}