{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T11:43:22Z","timestamp":1774957402329,"version":"3.50.1"},"reference-count":59,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T00:00:00Z","timestamp":1700784000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Science Foundation Ireland","award":["13\/RC\/2094_P2"],"award-info":[{"award-number":["13\/RC\/2094_P2"]}]},{"name":"European Regional Development","award":["13\/RC\/2094_P2"],"award-info":[{"award-number":["13\/RC\/2094_P2"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>In advanced driver assistance systems (ADAS) or autonomous vehicle research, acquiring semantic information about the surrounding environment generally relies heavily on camera-based object detection. Image signal processors (ISPs) in cameras are generally tuned for human perception. In most cases, ISP parameters are selected subjectively and the resulting image differs depending on the individual who tuned it. While the installation of cameras on cars started as a means of providing a view of the vehicle\u2019s environment to the driver, cameras are increasingly becoming part of safety-critical object detection systems for ADAS. Deep learning-based object detection has become prominent, but the effect of varying the ISP parameters has an unknown performance impact. In this study, we analyze the performance of 14 popular object detection models in the context of changes in the ISP parameters. We consider eight ISP blocks: demosaicing, gamma, denoising, edge enhancement, local tone mapping, saturation, contrast, and hue angle. We investigate two raw datasets, PASCALRAW and a custom raw dataset collected from an advanced driver assistance system (ADAS) perspective. We found that varying from a default ISP degrades the object detection performance and that the models differ in sensitivity to varying ISP parameters. Finally, we propose a novel methodology that increases object detection model robustness via ISP variation data augmentation.<\/jats:p>","DOI":"10.3390\/jimaging9120260","type":"journal-article","created":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T03:43:59Z","timestamp":1700797439000},"page":"260","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Impact of ISP Tuning on Object Detection"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0186-0718","authenticated-orcid":false,"given":"Dara","family":"Molloy","sequence":"first","affiliation":[{"name":"School of Engineering, University of Galway, University Road, H91 TK33 Galway, Ireland"},{"name":"Ryan Institute, University of Galway, University Road, H91 TK33 Galway, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3678-7605","authenticated-orcid":false,"given":"Brian","family":"Deegan","sequence":"additional","affiliation":[{"name":"School of Engineering, University of Galway, University Road, H91 TK33 Galway, Ireland"},{"name":"Ryan Institute, University of Galway, University Road, H91 TK33 Galway, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8555-7868","authenticated-orcid":false,"given":"Darragh","family":"Mullins","sequence":"additional","affiliation":[{"name":"School of Engineering, University of Galway, University Road, H91 TK33 Galway, Ireland"},{"name":"Ryan Institute, University of Galway, University Road, H91 TK33 Galway, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Enda","family":"Ward","sequence":"additional","affiliation":[{"name":"Valeo, Tuam, Co., H54 Y276 Galway, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan","family":"Horgan","sequence":"additional","affiliation":[{"name":"Valeo, Tuam, Co., H54 Y276 Galway, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8383-2635","authenticated-orcid":false,"given":"Ciaran","family":"Eising","sequence":"additional","affiliation":[{"name":"School of Engineering, University of Galway, University Road, H91 TK33 Galway, Ireland"},{"name":"Department of Electronic and Computer Engineering, University of Limerick, Castletroy, V94 T9PX Limerick, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7494-8513","authenticated-orcid":false,"given":"Patrick","family":"Denny","sequence":"additional","affiliation":[{"name":"School of Engineering, University of Galway, University Road, H91 TK33 Galway, Ireland"},{"name":"Department of Electronic and Computer Engineering, University of Limerick, Castletroy, V94 T9PX Limerick, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Edward","family":"Jones","sequence":"additional","affiliation":[{"name":"School of Engineering, University of Galway, University Road, H91 TK33 Galway, Ireland"},{"name":"Ryan Institute, University of Galway, University Road, H91 TK33 Galway, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1477-4835","authenticated-orcid":false,"given":"Martin","family":"Glavin","sequence":"additional","affiliation":[{"name":"School of Engineering, University of Galway, University Road, H91 TK33 Galway, Ireland"},{"name":"Ryan Institute, University of Galway, University Road, H91 TK33 Galway, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,11,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3212","DOI":"10.1109\/TNNLS.2018.2876865","article-title":"Object Detection with Deep Learning: A Review","volume":"30","author":"Zhao","year":"2019","journal-title":"IEEE Trans. Neural Networks Learn. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Zhu, H., Wei, H., Li, B., Yuan, X., and Kehtarnavaz, N. (2020). A Review of Video Object Detection: Datasets, Metrics and Methods. Appl. Sci., 10.","DOI":"10.3390\/app10217834"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1016\/j.eng.2020.01.007","article-title":"A Survey of Accelerator Architectures for Deep Neural Networks","volume":"6","author":"Chen","year":"2020","journal-title":"Engineering"},{"key":"ref_4","unstructured":"Mishra, P., and Saroha, G. (2016, January 16\u201318). A Study on Video Surveillance System for Object Detection and Tracking. Proceedings of the 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wang, X., Li, B., Ma, H., and Luo, M. (2019, January 27\u201330). A fast quantity and position detection method based on monocular vision for a workpieces counting and sorting system. Proceedings of the 2019 Chinese Control Conference (CCC), Guangzhou, China.","DOI":"10.23919\/ChiCC.2019.8865361"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"103589","DOI":"10.1016\/j.robot.2020.103589","article-title":"Monocular vision-based gripping of objects","volume":"131","year":"2020","journal-title":"Robot. Auton. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.neucom.2021.01.110","article-title":"Monocular 3D object detection using dual quadric for autonomous driving","volume":"441","author":"Li","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yu, F., Chen, H., Wang, X., Xian, W., Chen, Y., Liu, F., Madhavan, V., and Darrell, T. (2020, January 13\u201319). BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00271"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Doll\u00e1r, P. (2015). Microsoft COCO: Common Objects in Context. arXiv.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Mosleh, A., Sharma, A., Onzon, E., Mannan, F., Robidoux, N., and Heide, F. (2020, January 13\u201319). Hardware-in-the-Loop End-to-End Optimization of Camera Image Processing Pipelines. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00755"},{"key":"ref_11","unstructured":"(2023, November 21). OpenISP: Image Signal Processor. Available online: https:\/\/github.com\/cruxopen\/openISP."},{"key":"ref_12","unstructured":"Jueqin, Q. (2023, November 21). Fast-openisp: A Faster Re-Implementation of OpenISP. Available online: https:\/\/github.com\/QiuJueqin\/fast-openISP."},{"key":"ref_13","first-page":"77440Z","article-title":"Image quality assessment and human visual system","volume":"Volume 7744","author":"Frossard","year":"2010","journal-title":"Proceedings of the Visual Communications and Image Processing"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Buckler, M., Jayasuriya, S., and Sampson, A. (2017). Reconfiguring the Imaging Pipeline for Computer Vision. arXiv.","DOI":"10.1109\/ICCV.2017.111"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2289","DOI":"10.1109\/TPAMI.2012.58","article-title":"A New In-Camera Imaging Model for Color Computer Vision and Its Application","volume":"34","author":"Kim","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"013019","DOI":"10.1117\/1.JEI.22.1.013019","article-title":"Image informative maps for component-wise estimating parameters of signal-dependent noise","volume":"22","author":"Uss","year":"2013","journal-title":"J. Electron. Imaging"},{"key":"ref_17","unstructured":"Yahiaoui, L., Horgan, J., Yogamani, S., Eising, C., and Deegan, B. (2018, January 29\u201331). Impact analysis and tuning strategies for camera Image Signal Processing parameters in Computer Vision. Proceedings of the 20th Irish Machine Vision and Image Processing Conference, Belfast, Ireland."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yahiaoui, L., Horgan, J., Deegan, B., Yogamani, S., Hughes, C., and Denny, P. (2019). Overview and Empirical Analysis of ISP Parameter Tuning for Visual Perception in Autonomous Driving. J. Imaging, 5.","DOI":"10.3390\/jimaging5100078"},{"key":"ref_19","unstructured":"Hansen, N. (2016). The CMA Evolution Strategy: A Tutorial. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hansen, P., Vilkin, A., Khrustalev, Y., Imber, J., Hanwell, D., Mattina, M., and Whatmough, P.N. (2021). ISP4ML: Understanding the Role of Image Signal Processing in Efficient Deep Learning Vision Systems. arXiv.","DOI":"10.1109\/ICPR48806.2021.9411985"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Robidoux, N., Seo, D.e., Ariza, F., Garcia Capel, L.E., Sharma, A., and Heide, F. (2021, January 20\u201325). End-to-end High Dynamic Range Camera Pipeline Optimization. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00623"},{"key":"ref_22","unstructured":"Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"art00014","DOI":"10.2352\/ISSN.2470-1173.2019.15.AVM-044","article-title":"Optimization of ISP parameters for object detection algorithms","volume":"31","author":"Yahiaoui","year":"2019","journal-title":"Electron. Imaging"},{"key":"ref_24","unstructured":"Omid-Zohoor, A., Ta, D., and Murmann, B. (2023, November 21). PASCALRAW: Raw image database for object detection. Available online: https:\/\/searchworks.stanford.edu\/view\/hq050zr7488."},{"key":"ref_25","unstructured":"Sekachev, B., Manovich, N., Zhiltsov, M., Zhavoronkov, A., Kalinin, D., Hoff, B., Kruchinin, D., Zankevich, A. (2020). OpenCV\/CVAT: V1.1.0, Github."},{"key":"ref_26","unstructured":"Jocher, G., Chaurasia, A., Stoken, A., Borovec, J., Kwon, Y., Xie, T., Fang, J., and Michael, K. (2022). Ultralytics\/YOLOv5: V6.1\u2014TensorRT, TensorFlow Edge TPU and OpenVINO Export and Inference, Github."},{"key":"ref_27","unstructured":"(2023, November 21). TorchVision: PyTorch\u2019s Computer Vision Library. Available online: https:\/\/github.com\/pytorch\/vision."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. arXiv.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1007\/s11263-013-0620-5","article-title":"Selective Search for Object Recognition","volume":"104","author":"Uijlings","year":"2013","journal-title":"Int. J. Comput. Vis."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015). Fast R-CNN. arXiv.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2016). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv.","DOI":"10.1109\/TPAMI.2016.2577031"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Redmon, J., and Farhadi, A. (2017, January 21\u201326). YOLO9000: Better, Faster, Stronger. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.690"},{"key":"ref_34","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wang, C.Y., Bochkovskiy, A., and Liao, H.Y.M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv.","DOI":"10.1109\/CVPR52729.2023.00721"},{"key":"ref_36","unstructured":"Jocher, G., Chaurasia, A., and Qiu, J. (2023, November 21). Ultralytics YOLOv8. Available online: https:\/\/github.com\/ultralytics\/ultralytics."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S.E., Fu, C., and Berg, A.C. (2016, January 11\u201314). SSD: Single Shot MultiBox Detector. Proceedings of the 2016 ECCV European Conference on Computer Vision, 14th European Conference, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2018). Focal Loss for Dense Object Detection. arXiv.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017). Feature Pyramid Networks for Object Detection. arXiv.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Tian, Z., Shen, C., Chen, H., and He, T. (2019, January 27\u20132). FCOS: Fully Convolutional One-Stage Object Detection. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.","DOI":"10.1109\/ICCV.2019.00972"},{"key":"ref_41","unstructured":"Lv, W., Zhao, Y., Xu, S., Wei, J., Wang, G., Cui, C., Du, Y., Dang, Q., and Liu, Y. (2023). DETRs Beat YOLOs on Real-time Object Detection. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23\u201328). End-to-End Object Detection with Transformers. Proceedings of the 2020 ECCV European Conference on Computer Vision, 16th European Conference, Glasgow, UK.","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"ref_43","unstructured":"(2023, November 21). PaddleClas. Available online: https:\/\/github.com\/PaddlePaddle\/PaddleClas."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Sch\u00f6berl, M., Schnurrer, W., Oberd\u00f6rster, A., F\u00f6ssel, S., and Kaup, A. (2010, January 26\u201329). Dimensioning of optical birefringent anti-alias filters for digital cameras. Proceedings of the 2010 IEEE International Conference on Image Processing, Hong Kong, China.","DOI":"10.1109\/ICIP.2010.5651784"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Li, X., Gunturk, B., and Zhang, L. (2008, January 28). Image demosaicing: A systematic survey. Proceedings of the Visual Communications and Image Processing 2008, San Jose, CA, USA.","DOI":"10.1117\/12.766768"},{"key":"ref_46","unstructured":"(2023, November 21). OpenCV: Open Computer Vision Library. Available online: https:\/\/opencv.org\/."},{"key":"ref_47","unstructured":"(2023, November 21). sRGB IEC 61966-2-1:1999. Available online: https:\/\/webstore.iec.ch\/publication\/6169."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Wang, G., Renshaw, D., Denyer, P., and Lu, M. (1991, January 27\u201331). CMOS video cameras. Proceedings of the Euro ASIC \u201991, Paris, France.","DOI":"10.1109\/EUASIC.1991.212885"},{"key":"ref_49","unstructured":"(2023, November 21). EMVA Working Group EMVA Standard 1288. Available online: https:\/\/zenodo.org\/records\/3951558."},{"key":"ref_50","unstructured":"Tomasi, C., and Manduchi, R. (1998, January 7). Bilateral filtering for gray and color images. Proceedings of the Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271), Bombay, India."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1479","DOI":"10.1109\/TIP.2005.852196","article-title":"Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization","volume":"14","author":"Chan","year":"2005","journal-title":"IEEE Trans. Image Process."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.1109\/83.862633","article-title":"Adaptive wavelet thresholding for image denoising and compression","volume":"9","author":"Chang","year":"2000","journal-title":"IEEE Trans. Image Process."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1016\/S0734-189X(87)80186-X","article-title":"Adaptive histogram equalization and its variations","volume":"39","author":"Pizer","year":"1987","journal-title":"Comput. Vision, Graph. Image Process."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1109\/83.826787","article-title":"Image enhancement via adaptive unsharp masking","volume":"9","author":"Polesel","year":"2000","journal-title":"IEEE Trans. Image Process."},{"key":"ref_55","unstructured":"Fairchild, M.D. (2004, January 9\u201312). Color Appearance Models: CIECAM02 and Beyond. Proceedings of the Tutorial Slides for IS&T\/SID 12th Color Imaging Conference, Scottsdale, AZ, USA."},{"key":"ref_56","unstructured":"Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., and Zisserman, A. (2023, November 21). The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. Available online: http:\/\/host.robots.ox.ac.uk\/pascal\/VOC\/."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Bolya, D., Foley, S., Hays, J., and Hoffman, J. (2020). TIDE: A General Toolbox for Identifying Object Detection Errors. arXiv.","DOI":"10.1007\/978-3-030-58580-8_33"},{"key":"ref_58","unstructured":"Molloy, D. (2023, November 21). ISP Object Detection Benchmark. Available online: https:\/\/zenodo.org\/records\/7802651."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on Image Data Augmentation for Deep Learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J. Big Data"}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/9\/12\/260\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:29:06Z","timestamp":1760131746000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/9\/12\/260"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,24]]},"references-count":59,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["jimaging9120260"],"URL":"https:\/\/doi.org\/10.3390\/jimaging9120260","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,24]]}}}