{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T14:13:29Z","timestamp":1762956809524,"version":"build-2065373602"},"reference-count":26,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2021,12,18]],"date-time":"2021-12-18T00:00:00Z","timestamp":1639785600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>ADAS and autonomous technologies in vehicles become more and more complex, which increases development time and expenses. This paper presents a new real-time ADAS multisensory validation system, which can speed up the development and implementation processes while lowering its cost. The proposed test system integrates a high-quality 3D CARLA simulator with a real-time-based automation platform. We present system experimental verifications on several types of sensors and testing system architectures. The first, open-loop experiment explains the real-time capabilities of the system based on the Mobileye 6 camera sensor detections. The second experiment runs a real-time closed-loop test of a lane-keeping algorithm (LKA) based on the Mobileye 6 line detection. The last experiment presents a simulation of Velodyne VLP-16 lidar, which runs a free space detection algorithm. Simulated lidar output is compared with the real lidar performance. We show that the platform generates reproducible results and allows closed-loop operation which, combined with a real-time collection of event information, promises good scalability toward complex ADAS or autonomous functionalities testing.<\/jats:p>","DOI":"10.3390\/s21248458","type":"journal-article","created":{"date-parts":[[2021,12,20]],"date-time":"2021-12-20T02:40:32Z","timestamp":1639968032000},"page":"8458","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Multisensory Testing Framework for Advanced Driver Assistant Systems Supported by High-Quality 3D Simulation"],"prefix":"10.3390","volume":"21","author":[{"given":"Pawe\u0142","family":"Jab\u0142o\u0144ski","sequence":"first","affiliation":[{"name":"Department of Robotics and Mechatronics, Faculty of Mechanical Engineering and Robotics, AGH University of Science and Technology, Mickiewicz Alley 30, 30-059 Cracow, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2320-1417","authenticated-orcid":false,"given":"Joanna","family":"Iwaniec","sequence":"additional","affiliation":[{"name":"Department of Robotics and Mechatronics, Faculty of Mechanical Engineering and Robotics, AGH University of Science and Technology, Mickiewicz Alley 30, 30-059 Cracow, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Micha\u0142","family":"Jab\u0142o\u0144ski","sequence":"additional","affiliation":[{"name":"Varroc Lighting Systems, s.r.o., Suvorovova 195, 742 42 \u0160enov u Nov\u00e9ho Ji\u010d\u00edna, Czech Republic"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,18]]},"reference":[{"key":"ref_1","unstructured":"(2021, October 20). Levels of Driving Automation Are Defined in New SAE International Standard J3016. Available online: https:\/\/www.sae.org\/news\/2019\/01\/sae-updates-j3016-automated-driving-graphic."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"63","DOI":"10.4271\/2016-01-8013","article-title":"Current approaches in HiL-based ADAS testing","volume":"9","author":"Feilhauer","year":"2016","journal-title":"SAE Int. J. Commer. Veh."},{"key":"ref_3","unstructured":"Peng, H., \u00c7akmakci, M., and Ulsoy, A.G. (2012). Automotive Control Systems, Cambridge University Press. Automotive Control-System Design Process."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1080\/00423110600563338","article-title":"Development of advanced driver assistance systems with vehicle hardware-in-the-loop simulations","volume":"44","author":"Gietelink","year":"2006","journal-title":"Veh. Syst. Dyn."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Galko, C., Rossi, R., and Savatier, X. (2014, January 14\u201317). Vehicle-Hardware-In-The-Loop system for ADAS prototyping and validation. Proceedings of the 2014 International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation (SAMOS XIV), Agios Konstantinos, Greece.","DOI":"10.1109\/SAMOS.2014.6893229"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Solmaz, S., and Holzinger, F. (2019, January 4\u20138). A Novel Testbench for Development, Calibration and Functional Testing of ADAS\/AD Functions. Proceedings of the 2019 IEEE International Conference on Connected Vehicles and Expo (ICCVE), Graz, Austria.","DOI":"10.1109\/ICCVE45908.2019.8965225"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"32","DOI":"10.4204\/EPTCS.269.4","article-title":"The CAT Vehicle Testbed: A Simulator with Hardware in the Loop for Autonomous Vehicle Applications","volume":"Volume 269","author":"Bhadani","year":"2018","journal-title":"Electronic Proceedings in Theoretical Computer Science"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kassem, N.S., Masoud, A.M., and Aly, A.M.B. (2021, January 23\u201325). Driver-in-the-Loop for computer-vision based ADAS testing. Proceedings of the 2021 3rd Novel Intelligent and Leading Emerging Sciences Conference (NILES), Giza, Egypt.","DOI":"10.1109\/NILES53778.2021.9600491"},{"key":"ref_9","unstructured":"Laurent, R., and Rouah, X. (2014, January 17). MBT and Test Automation for ADAS Validation. Proceedings of the UCAAT Conference, Munich, Germany."},{"key":"ref_10","unstructured":"Gechter, F., Dafflon, B., Gruer, P., and Koukam, A. (2014, January 12\u201316). Towards a Hybrid Real\/virtual Simulation of Autonomous Vehicles for Critical Scenarios. Proceedings of the Sixth International Conference on Advances in System Simulation, Nice, France."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Jeong, M., and Ko, B.C. (2018). Driver\u2019s Facial Expression Recognition in Real-Time for Safe Driving. Sensors, 18.","DOI":"10.3390\/s18124270"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kakani, V., Kim, H., Kumbham, M., Park, D., Jin, C.-B., and Nguyen, V.H. (2019). Feasible Self-Calibration of Larger Field-of-View (FOV) Camera Sensors for the Advanced Driver-Assistance System (ADAS). Sensors, 19.","DOI":"10.3390\/s19153369"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"66","DOI":"10.4271\/2016-01-0049","article-title":"A Framework for Virtual Testing of ADAS","volume":"9","author":"Zhou","year":"2016","journal-title":"SAE Int. J. Passeng. Cars-Electron. Electr. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"\u00c1lvaro, S., Bergasa, L.M., L\u00f3pez-Guill\u00e9n, E., Romera, E., Tradacete, M., G\u00f3mez-Hu\u00e9lamo, C., and Del Egido, J. (2019). Real-Time Semantic Segmentation for Fisheye Urban Driving Images Based on ERFNet. Sensors, 19.","DOI":"10.3390\/s19030503"},{"key":"ref_15","unstructured":"(2021, October 20). Software and Testing Solutions, Morphee\u2014Automation System. Available online: https:\/\/www.fev.com\/fileadmin\/user_upload\/FEV_MORPHEE_2017.pdf."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Genser, S., Muckenhuber, S., Solmaz, S., and Reckenzaun, J. (2021). Development and Experimental Validation of an Intelligent Camera Model for Automated Driving. Sensors, 21.","DOI":"10.3390\/s21227583"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Mohammed, A.S., Amamou, A., Ayevide, F.K., Kelouwani, S., Agbossou, K., and Zioui, N. (2020). The Perception System of Intelligent Ground Vehicles in All Weather Conditions: A Systematic Literature Review. Sensors, 20.","DOI":"10.3390\/s20226532"},{"key":"ref_18","unstructured":"(2021, July 21). Vactor, CANoe\u2014The Multibus Development and Test Tool for ECUs and Networks. Available online: https:\/\/assets.vector.com\/cms\/content\/products\/canoe\/canoe\/docs\/Fact%20Sheets\/CANoe_FactSheet_EN.pdf."},{"key":"ref_19","unstructured":"(2021, October 20). Vector, FDX\u2014Fast Data Exchange with CANoe. Available online: https:\/\/assets.vector.com\/cms\/content\/know-how\/_application-notes\/AN-AND-1-119_Fast_Data_Exchange_with_CANoe.pdf."},{"key":"ref_20","unstructured":"(2021, October 20). Interval Zero, Real-Time Operating System (RTOS) Platform Vision. Available online: https:\/\/www.intervalzero.com\/products\/rtos-platform-vision."},{"key":"ref_21","unstructured":"(2021, October 20). Interval Zero, Real-Time Operating System (RTOS) RTX on Windows Diagram. Available online: https:\/\/help.intervalzero.com\/product_help\/RTX64_4\/Content\/PROJECTS\/Subsystem\/RTSS_Architecture\/Adding_RTX_To_Win_Environment.htm."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"12153","DOI":"10.3390\/s120912153","article-title":"Recognition Stage for a Speed Supervisor Based on Road Sign Detection","volume":"12","author":"Carrasco","year":"2012","journal-title":"Sensors"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.measurement.2016.10.009","article-title":"Quantifying the influence of rain in LiDAR performance","volume":"95","author":"Filgueira","year":"2017","journal-title":"Measurement"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"S\u00e1nchez, D.E., and Pany, T. (2019, January 16\u201320). Analysis and Calibration of the VLP-16 LiDAR for Automotive Applications. Proceedings of the 32nd International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2019), Miami, FL, USA.","DOI":"10.33012\/2019.17093"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Keertheeswaran, M.P., and Nair, B.B. (2021). Vehicle Identification Using Automotive LIDAR VLP-16 and Deep Learning. Evol. Comput. Mob. Sustain. Netw., 779\u2013792.","DOI":"10.1007\/978-981-15-9647-6_60"},{"key":"ref_26","unstructured":"(2021, October 20). Smart Vehicle FEV Project. Available online: https:\/\/smart-vehicle.fev.com\/#adas."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/24\/8458\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:51:15Z","timestamp":1760169075000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/24\/8458"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,18]]},"references-count":26,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["s21248458"],"URL":"https:\/\/doi.org\/10.3390\/s21248458","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,12,18]]}}}