{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:33:36Z","timestamp":1760236416916,"version":"build-2065373602"},"reference-count":29,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2021,11,28]],"date-time":"2021-11-28T00:00:00Z","timestamp":1638057600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Structural and Investment Funds in the FEDER component, through the Operational Competitiveness and Internationalization Programme (COMPETE 2020)","award":["[Project n037902, funding reference POCI-01-0247-FEDER-037902"],"award-info":[{"award-number":["[Project n037902, funding reference POCI-01-0247-FEDER-037902"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Research about deep learning applied in object detection tasks in LiDAR data has been massively widespread in recent years, achieving notable developments, namely in improving precision and inference speed performances. These improvements have been facilitated by powerful GPU servers, taking advantage of their capacity to train the networks in reasonable periods and their parallel architecture that allows for high performance and real-time inference. However, these features are limited in autonomous driving due to space, power capacity, and inference time constraints, and onboard devices are not as powerful as their counterparts used for training. This paper investigates the use of a deep learning-based method in edge devices for onboard real-time inference that is power-effective and low in terms of space-constrained demand. A methodology is proposed for deploying high-end GPU-specific models in edge devices for onboard inference, consisting of a two-folder flow: study model hyperparameters\u2019 implications in meeting application requirements; and compression of the network for meeting the board resource limitations. A hybrid FPGA-CPU board is proposed as an effective onboard inference solution by comparing its performance in the KITTI dataset with computer performances. The achieved accuracy is comparable to the PC-based deep learning method with a plus that it is more effective for real-time inference, power limited and space-constrained purposes.<\/jats:p>","DOI":"10.3390\/s21237933","type":"journal-article","created":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T01:45:02Z","timestamp":1638323102000},"page":"7933","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Resource-Constrained Onboard Inference of 3D Object Detection and Localisation in Point Clouds Targeting Self-Driving Applications"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7075-3364","authenticated-orcid":false,"given":"Ant\u00f3nio","family":"Silva","sequence":"first","affiliation":[{"name":"Algoritmi Centre, University of Minho, 4800-058 Guimar\u00e3es, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9736-5812","authenticated-orcid":false,"given":"Duarte","family":"Fernandes","sequence":"additional","affiliation":[{"name":"Algoritmi Centre, University of Minho, 4800-058 Guimar\u00e3es, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5576-6175","authenticated-orcid":false,"given":"Rafael","family":"N\u00e9voa","sequence":"additional","affiliation":[{"name":"Algoritmi Centre, University of Minho, 4800-058 Guimar\u00e3es, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3287-3995","authenticated-orcid":false,"given":"Jo\u00e3o","family":"Monteiro","sequence":"additional","affiliation":[{"name":"Algoritmi Centre, University of Minho, 4800-058 Guimar\u00e3es, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3549-0754","authenticated-orcid":false,"given":"Paulo","family":"Novais","sequence":"additional","affiliation":[{"name":"Algoritmi Centre, University of Minho, 4800-058 Guimar\u00e3es, Portugal"}]},{"given":"Pedro","family":"Gir\u00e3o","sequence":"additional","affiliation":[{"name":"Bosch Company, 4700-113 Braga, Portugal"}]},{"given":"Tiago","family":"Afonso","sequence":"additional","affiliation":[{"name":"Bosch Company, 4700-113 Braga, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8257-0143","authenticated-orcid":false,"given":"Pedro","family":"Melo-Pinto","sequence":"additional","affiliation":[{"name":"Algoritmi Centre, University of Minho, 4800-058 Guimar\u00e3es, Portugal"},{"name":"Department of Engineering, University of Tr\u00e1s-os-Montes and Alto Douro, 5000-801 Vila Real, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Cosmas, K., and Kenichi, A. (2020). Utilization of FPGA for onboard inference of landmark localization in CNN-Based spacecraft pose estimation. Aerospace, 7.","DOI":"10.3390\/aerospace7110159"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"015001","DOI":"10.1088\/2632-2153\/aba042","article-title":"Compressing deep neural networks on FPGAs to binary and ternary precision with hls4ml","volume":"2","author":"Ngadiuba","year":"2020","journal-title":"Mach. Learn. Sci. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Sharma, H., Park, J., Amaro, E., Thwaites, B., Kotha, P., Gupta, A., Kim, J.K., Mishra, A., and Esmaeilzadeh, H. (2016, January 15\u201319). Dnnweaver: From high-level deep network models to fpga acceleration. Proceedings of the Workshop on Cognitive Architectures, Taipei, Taiwan.","DOI":"10.1109\/MICRO.2016.7783720"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1016\/j.inffus.2020.11.002","article-title":"Point-cloud based 3D object detection and classification methods for self-driving applications: A survey and taxonomy","volume":"68","author":"Fernandes","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1186\/s40537-021-00444-8","article-title":"Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions","volume":"8","author":"Alzubaidi","year":"2021","journal-title":"J. Big Data"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Xu, L., Zhou, X., Tao, Y., Liu, L., Yu, X., and Kumar, N. (2021). Intelligent Security Performance Prediction for IoT-Enabled Healthcare Networks Using Improved CNN. IEEE Trans. Ind. Inform.","DOI":"10.1109\/TII.2021.3082907"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zhou, Y., and Tuzel, O. (2017). VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection. arXiv.","DOI":"10.1109\/CVPR.2018.00472"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yan, Y., Mao, Y., and Li, B. (2018). Second: Sparsely embedded convolutional detection. Sensors, 18.","DOI":"10.3390\/s18103337"},{"key":"ref_9","unstructured":"Qi, C.R., Yi, L., Su, H., and Guibas, L.J. (2017, January 4\u20139). Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_10","first-page":"2647","article-title":"From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network","volume":"43","author":"Shi","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ye, M., Xu, S., and Cao, T. (2020, January 13\u201319). Hvnet: Hybrid voxel network for lidar based 3d object detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00170"},{"key":"ref_12","unstructured":"Graham, B., and van der Maaten, L. (2021, November 25). Submanifold Sparse Convolutional Networks. Available online: https:\/\/arxiv.org\/abs\/1706.01307."},{"key":"ref_13","unstructured":"Graham, B. (2021, November 25). Sparse 3D Convolutional Neural Networks. Available online: https:\/\/arxiv.org\/abs\/1505.02890."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Qi, C.R., Liu, W., Wu, C., Su, H., and Guibas, L.J. (2018, January 18\u201323). Frustum pointnets for 3d object detection from rgb-d data. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00102"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Meyer, G.P., Laddha, A., Kee, E., Vallespi-Gonzalez, C., and Wellington, C.K. (2019, January 16\u201317). Lasernet: An efficient probabilistic 3d object detector for autonomous driving. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01296"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Li, B., Zhang, T., and Xia, T. (2016). Vehicle detection from 3d lidar using fully convolutional network. arXiv.","DOI":"10.15607\/RSS.2016.XII.042"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chen, X., Ma, H., Wan, J., Li, B., and Xia, T. (2017, January 21\u201326). Multi-view 3d object detection network for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.691"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., and Beijbom, O. (2019, January 16\u201317). Pointpillars: Fast encoders for object detection from point clouds. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01298"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4196","DOI":"10.1109\/TCSI.2018.2840092","article-title":"Energy-Efficient Convolution Architecture Based on Rescheduled Dataflow","volume":"65","author":"Jo","year":"2018","journal-title":"IEEE Trans. Circuits Syst. I Regul. Pap."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"458","DOI":"10.1109\/JPROC.2018.2802438","article-title":"Onboard Processing with Hybrid and Reconfigurable Computing on Small Satellites","volume":"106","author":"George","year":"2018","journal-title":"Proc. IEEE"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Qiu, J., Wang, J., Yao, S., Guo, K., Li, B., Zhou, E., Yu, J., Tang, T., Xu, N., and Song, S. (2016, January 21\u201323). Going Deeper with Embedded FPGA Platform for Convolutional Neural Network. Proceedings of the 2016 ACM\/SIGDA International Symposium on Field-Programmable Gate Arrays, Monterey, CA, USA.","DOI":"10.1145\/2847263.2847265"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yang, Z., Yan, L., and Yuan, J. (2020, January 16\u201318). Design and Implementation of Driverless Perceptual System Based on CPU + FPGA. Proceedings of the 2020 5th International Conference on Control, Robotics and Cybernetics (CRC), Wuhan, China.","DOI":"10.1109\/CRC51253.2020.9253490"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1109\/LES.2017.2743247","article-title":"Tactics to directly map CNN graphs on embedded FPGAs","volume":"9","author":"Abdelouahab","year":"2017","journal-title":"IEEE Embed. Syst. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Kathail, V. (2020, January 23\u201325). Xilinx Vitis unified software platform. Proceedings of the 2020 ACM\/SIGDA International Symposium on Field-Programmable Gate Arrays, Seaside, CA, USA.","DOI":"10.1145\/3373087.3375887"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"P07027","DOI":"10.1088\/1748-0221\/13\/07\/P07027","article-title":"Fast inference of deep neural networks in FPGAs for particle physics","volume":"13","author":"Duarte","year":"2018","journal-title":"J. Instrum."},{"key":"ref_26","unstructured":"Xilinx (2021, July 10). DPUCZDX8G for Zynq UltraScale+ MPSoCs. Product Guide. Xilinx. Available online: http:\/\/aiweb.techfak.uni-bielefeld.de\/content\/bworld-robot-control-software\/."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Fernandes, D., Afonso, A., Gir\u00e3o, P., Gonzalez, D., Silva, A., N\u00e9voa, R., Novais, P., Monteiro, J., and Melo-Pinto, P. (2021). Real-time 3D Object Detection and SLAM Fusion in a Low-Cost LiDAR Test Vehicle Setup. Sensors, submitted.","DOI":"10.3390\/s21248381"},{"key":"ref_28","unstructured":"Krishnamoorthi, R. (2018). Quantizing deep Convolutional Networks for Efficient Inference: A Whitepaper. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Nagel, M., van Baalen, M., Blankevoort, T., and Welling, M. (2019). Data-Free Quantization Through Weight Equalization and Bias Correction. arXiv.","DOI":"10.1109\/ICCV.2019.00141"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/23\/7933\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:36:52Z","timestamp":1760168212000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/23\/7933"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,28]]},"references-count":29,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["s21237933"],"URL":"https:\/\/doi.org\/10.3390\/s21237933","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,11,28]]}}}