{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:32:49Z","timestamp":1760149969846,"version":"build-2065373602"},"reference-count":24,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2023,9,17]],"date-time":"2023-09-17T00:00:00Z","timestamp":1694908800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Structural and Investment Funds in the FEDER component through the Operational Competitiveness and Internationalization Programme","award":["047264-THEIA","LA\/P\/0063"],"award-info":[{"award-number":["047264-THEIA","LA\/P\/0063"]}]},{"name":"Portuguese funding agency, FCT-Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","award":["047264-THEIA","LA\/P\/0063"],"award-info":[{"award-number":["047264-THEIA","LA\/P\/0063"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>The increased demand for and use of autonomous driving and advanced driver assistance systems has highlighted the issue of abnormalities occurring within the perception layers, some of which may result in accidents. Recent publications have noted the lack of standardized independent testing formats and insufficient methods with which to analyze, verify, and qualify LiDAR (Light Detection and Ranging)-acquired data and their subsequent labeling. While camera-based approaches benefit from a significant amount of long-term research, images captured through the visible spectrum can be unreliable in situations with impaired visibility, such as dim lighting, fog, and heavy rain. A redoubled focus upon LiDAR usage would combat these shortcomings; however, research involving the detection of anomalies and the validation of gathered data is few and far between when compared to its counterparts. This paper aims to contribute to expand the knowledge on how to evaluate LiDAR data by introducing a novel method with the ability to detect these patterns and complement other performance evaluators while using a statistical approach. Although it is preliminary, the proposed methodology shows promising results in the evaluation of an algorithm\u2019s confidence score, the impact that weather and road conditions may have on data, and fringe cases in which the data may be insufficient or otherwise unusable.<\/jats:p>","DOI":"10.3390\/rs15184570","type":"journal-article","created":{"date-parts":[[2023,9,17]],"date-time":"2023-09-17T23:32:27Z","timestamp":1694993547000},"page":"4570","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["An Introduction to the Evaluation of Perception Algorithms and LiDAR Point Clouds Using a Copula-Based Outlier Detector"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1671-2884","authenticated-orcid":false,"given":"Nuno","family":"Reis","sequence":"first","affiliation":[{"name":"Departamento de Engenharia Eletrot\u00e9cnica e de Computadores, Faculdade de Engenharia, Universidade do Porto, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9160-9158","authenticated-orcid":false,"given":"Jos\u00e9","family":"Machado da Silva","sequence":"additional","affiliation":[{"name":"Departamento de Engenharia Eletrot\u00e9cnica e de Computadores, Faculdade de Engenharia, Universidade do Porto, 4200-465 Porto, Portugal"},{"name":"INESC TEC\u2014INESC Technology and Science, 4200-465 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6065-9358","authenticated-orcid":false,"given":"Miguel Velhote","family":"Correia","sequence":"additional","affiliation":[{"name":"Departamento de Engenharia Eletrot\u00e9cnica e de Computadores, Faculdade de Engenharia, Universidade do Porto, 4200-465 Porto, Portugal"},{"name":"INESC TEC\u2014INESC Technology and Science, 4200-465 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1016\/j.trpro.2020.03.003","article-title":"Traffic Accidents with Autonomous Vehicles: Type of Collisions, Manoeuvres and Errors of Conventional Vehicles\u2019 Drivers","volume":"45","year":"2020","journal-title":"Transp. Res. Procedia"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ma, Y., Yang, S., Lu, J., Feng, X., Yin, Y., and Cao, Y. (2022, January 25\u201327). Analysis of Autonomous Vehicles Accidents Based on DMV Reports. Proceedings of the 2022 China Automation Congress (CAC), Xiamen, China.","DOI":"10.1109\/CAC57257.2022.10056056"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Miethig, B., Liu, A., Habibi, S., and Mohrenschildt, M.v. (2019, January 19\u201321). Leveraging Thermal Imaging for Autonomous Driving. Proceedings of the 2019 IEEE Transportation Electrification Conference and Expo (ITEC), Detroit, MI, USA.","DOI":"10.1109\/ITEC.2019.8790493"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Bogdoll, D., Nitsche, M., and Zollner, J.M. (2022, January 19\u201320). Anomaly Detection in Autonomous Driving: A Survey. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), New Orleans, LA, USA.","DOI":"10.1109\/CVPRW56347.2022.00495"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Christian, G., Woodlief, T., and Elbaum, S. (2023, January 14\u201315). Generating Realistic and Diverse Tests for LiDAR-Based Perception Systems. Proceedings of the 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE), Melbourne, Australia.","DOI":"10.1109\/ICSE48619.2023.00217"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, Z., Zhao, Y., Botta, N., Ionescu, C., and Hu, X. (2020, January 17\u201320). COPOD: Copula-Based Outlier Detection. Proceedings of the 2020 IEEE International Conference on Data Mining (ICDM), Sorrento, Italy.","DOI":"10.1109\/ICDM50108.2020.00135"},{"key":"ref_7","first-page":"229","article-title":"Fonctions de r\u00e9partition \u00e0 n dimensions et leurs marges","volume":"8","author":"Sklar","year":"1959","journal-title":"Ann. L\u2019Isup"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"863","DOI":"10.1007\/s11219-021-09568-9","article-title":"Copula-Based Software Metrics Aggregation","volume":"29","author":"Ulan","year":"2021","journal-title":"Softw. Qual. J."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wifvat, K., Kumerow, J., and Shemyakin, A. (2020). Copula Model Selection for Vehicle Component Failures Based on Warranty Claims. Risks, 8.","DOI":"10.3390\/risks8020056"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1400","DOI":"10.1109\/TCAD.2011.2149522","article-title":"Estimation of Analog Parametric Test Metrics Using Copulas","volume":"30","author":"Bounceur","year":"2011","journal-title":"IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Liu, L., and Zuo, M.J. (2017, January 6\u20139). Copula-Based Time-Frequency Distribution Analysis for Planetary Gearbox Fault Detection. Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Cleveland, OH, USA.","DOI":"10.1115\/DETC2017-68060"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16\u201321). Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ribera, J., Guera, D., Chen, Y., and Delp, E.J. (2019, January 15\u201320). Locating objects without bounding boxes. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00664"},{"key":"ref_14","unstructured":"Dubuisson, M.P., and Jain, A.K. (1994, January 9\u201313). A modified Hausdorff distance for object matching. Proceedings of the 12th IEEE International Conference on Pattern Recognition, Jerusalem, Israel."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1109\/LRA.2022.3214791","article-title":"Lidar Upsampling With Sliced Wasserstein Distance","volume":"8","author":"Savkin","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Nazeri, B., and Crawford, M. (2021). Detection of Outliers in LiDAR Data Acquired by Multiple Platforms over Sorghum and Maize. Remote Sens., 13.","DOI":"10.3390\/rs13214445"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"131699","DOI":"10.1109\/ACCESS.2020.3009680","article-title":"Performance Analysis of 10 Models of 3D LiDARs for Automated Driving","volume":"8","author":"Lambert","year":"2020","journal-title":"IEEE Access"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1007\/s42154-021-00172-y","article-title":"A Review of Testing Object-Based Environment Perception for Safe Automated Driving","volume":"5","author":"Hoss","year":"2022","journal-title":"Automot. Innov."},{"key":"ref_19","unstructured":"(2018). Road Vehicles\u2014Functional Safety\u2014Part 1: Vocabulary. Standard No. ISO Standard No.26262-1:2018."},{"key":"ref_20","unstructured":"(2022). Road Vehicles\u2014Safety of the Intended Functionality. Standard No. ISO Standard No.21448:2022."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Horv\u00e1th, G., Kov\u00e1cs, E., Molontay, R., and Nov\u00e1czki, S. (2019). Copula-based anomaly scoring and localization for large-scale, high-dimensional continuous data. arXiv.","DOI":"10.1145\/3372274"},{"key":"ref_22","first-page":"1","article-title":"PyOD: A Python Toolbox for Scalable Outlier Detection","volume":"20","author":"Zhao","year":"2019","journal-title":"J. Mach. Learn. Res."},{"key":"ref_23","unstructured":"Sivaji12331 (2023, April 08). File:Empirical CDF, CDF and Confidence Interval Plots for Various Sample Sizes of Normal Distribution. 20 November 2019. Available online: https:\/\/commons.wikimedia.org\/wiki\/File:Empirical_CDF,_CDF_and_Confidence_Interval_plots_for_various_sample_sizes_of_Normal_Distribution.png."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1177\/0278364913491297","article-title":"Vision meets Robotics: The KITTI Dataset","volume":"32","author":"Geiger","year":"2013","journal-title":"Int. J. Robot. Res. (IJRR)"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/18\/4570\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:52:29Z","timestamp":1760129549000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/18\/4570"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,17]]},"references-count":24,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["rs15184570"],"URL":"https:\/\/doi.org\/10.3390\/rs15184570","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2023,9,17]]}}}