{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T12:31:10Z","timestamp":1762432270812,"version":"build-2065373602"},"reference-count":53,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2022,11,1]],"date-time":"2022-11-01T00:00:00Z","timestamp":1667260800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"German Ministry for Economic Affairs and Energy","award":["03SX481B","01IS17054"],"award-info":[{"award-number":["03SX481B","01IS17054"]}]},{"name":"Training Center Machine Learning, T\u00fcbingen","award":["03SX481B","01IS17054"],"award-info":[{"award-number":["03SX481B","01IS17054"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>An object detection pipeline comprises a camera that captures the scene and an object detector that processes these images. The quality of the images directly affects the performance of the object detector. Current works focus on independently improving the image quality or object detection models but neglect the importance of joint optimization of the two subsystems. This paper aims to tune the detection throughput and accuracy of existing object detectors in the remote sensing scenario by optimizing the input images tailored to the object detector. We empirically analyze the influence of two selected camera calibration parameters (camera distortion correction and gamma correction) and five image parameters (quantization, compression, resolution, color model, and additional channels) for these applications. For our experiments, we utilize three Unmanned Aerial Vehicle (UAV) data sets from different domains and a mixture of large and small state-of-the-art object detector models to provide an extensive evaluation of the influence of the pipeline parameters. Finally, we realize an object detection pipeline prototype on an embedded platform for a UAV and give a best practice recommendation for building object detection pipelines based on our findings. We show that not all parameters have an equal impact on detection accuracy and data throughput. Using a suitable compromise between parameters, we can achieve higher detection accuracy for lightweight object detection models while keeping the same data throughput.<\/jats:p>","DOI":"10.3390\/rs14215508","type":"journal-article","created":{"date-parts":[[2022,11,2]],"date-time":"2022-11-02T03:36:44Z","timestamp":1667360204000},"page":"5508","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Comprehensive Analysis of the Object Detection Pipeline on UAVs"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0473-5907","authenticated-orcid":false,"given":"Leon Amadeus","family":"Varga","sequence":"first","affiliation":[{"name":"Cognitive Systems, University of Tuebingen, 72074 T\u00fcbingen, Germany"}]},{"given":"Sebastian","family":"Koch","sequence":"additional","affiliation":[{"name":"Cognitive Systems, University of Tuebingen, 72074 T\u00fcbingen, Germany"}]},{"given":"Andreas","family":"Zell","sequence":"additional","affiliation":[{"name":"Cognitive Systems, University of Tuebingen, 72074 T\u00fcbingen, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,1]]},"reference":[{"key":"ref_1","unstructured":"Zhu, P., Wen, L., Du, D., Bian, X., Hu, Q., and Ling, H. (2020). Vision Meets Drones: Past, Present and Future. CoRR, abs\/2001.06303."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Varga, L.A., Kiefer, B., Messmer, M., and Zell, A. SeaDronesSee: A maritime benchmark for detecting humans in open water In Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 3\u20138 January 2022.","DOI":"10.1109\/WACV51458.2022.00374"},{"key":"ref_3","unstructured":"Ding, J., Xue, N., Xia, G., Bai, X., Yang, W., Yang, M.Y., Belongie, S.J., Luo, J., Datcu, M., and Pelillo, M. (2021). Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges. CoRR, abs\/2102.12219."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Yahyanejad, S., Misiorny, J., and Rinner, B. (2011, January 17\u201318). Lens distortion correction for thermal cameras to improve aerial imaging with small-scale UAVs. Proceedings of the 2011 IEEE International Symposium on Robotic and Sensors Environments (ROSE), Montreal, QC, Canada.","DOI":"10.1109\/ROSE.2011.6058528"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"161-1","DOI":"10.2352\/ISSN.2470-1173.2018.05.PMII-161","article-title":"Optimizing Image Acquisition Systems for Autonomous Driving","volume":"2018","author":"Blasinski","year":"2018","journal-title":"Electron. Imaging"},{"key":"ref_6","first-page":"505","article-title":"Modeling Camera Effects to Improve Visual Learning from Synthetic Data","volume":"Volume 11129","author":"Carlson","year":"2019","journal-title":"Proceedings of the Computer Vision\u2014ECCV 2018 Workshops"},{"key":"ref_7","unstructured":"Liu, Z., Lian, T., Farrell, J.E., and Wandell, B.A. (November, January 27). Soft Prototyping Camera Designs for Car Detection Based on a Convolutional Neural Network. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"10443","DOI":"10.1109\/ACCESS.2020.2965089","article-title":"Neural Network Generalization: The Impact of Camera Parameters","volume":"8","author":"Liu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Saad, K., and Schneider, S. (2019, January 4\u20138). Camera Vignetting Model and its Effects on Deep Neural Networks for Object Detection. Proceedings of the 2019 IEEE International Conference on Connected Vehicles and Expo (ICCVE), Graz, Austria.","DOI":"10.1109\/ICCVE45908.2019.8965233"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Secci, F., and Ceccarelli, A. (2020, January 12\u201315). On failures of RGB cameras and their effects in autonomous driving applications. Proceedings of the 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE), Coimbra, Portugal.","DOI":"10.1109\/ISSRE5003.2020.00011"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Buckler, M., Jayasuriya, S., and Sampson, A. (2017, January 22\u201329). Reconfiguring the Imaging Pipeline for Computer Vision. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.111"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Li, R., Wang, Y., Liang, F., Qin, H., Yan, J., and Fan, R. (2019, January 15\u201319). Fully Quantized Network for Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00292"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Cai, Y., Yao, Z., Dong, Z., Gholami, A., Mahoney, M.W., and Keutzer, K. (2020, January 13\u201319). ZeroQ: A Novel Zero Shot Quantization Framework. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01318"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","unstructured":"Bochkovskiy, A., Wang, C., and Liao, H.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. CoRR, abs\/2004.10934."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Tan, M., Pang, R., and Le, Q.V. (2020, January 13\u201319). EfficientDet: Scalable and Efficient Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01079"},{"key":"ref_17","unstructured":"Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., and Tian, Q. (November, January 27). CenterNet: Keypoint Triplets for Object Detection. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Varga, L.A., and Zell, A. (2021, January 11\u201317). Tackling the Background Bias in Sparse Object Detection via Cropped Windows. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV) Workshops, Montreal, BC, Canada.","DOI":"10.1109\/ICCVW54120.2021.00311"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1109\/MCOM.1977.1089500","article-title":"Quantization","volume":"15","author":"Gersho","year":"1977","journal-title":"IEEE Commun. Soc. Mag."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hunt, R. (2005). The Reproduction of Colour, Wiley.","DOI":"10.1002\/0470024275"},{"key":"ref_21","unstructured":"Cucchiara, R., Grana, C., Piccardi, M., Prati, A., and Sirotti, S. (2001, January 25\u201329). Improving shadow suppression in moving object detection with HSV color information. Proceedings of the ITSC 2001, 2001 IEEE Intelligent Transportation Systems. Proceedings (Cat. No.01TH8585), Oakland, CA, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Shuhua, L., and Gaizhi, G. (2010, January 21\u201324). The application of improved HSV color space model in image processing. Proceedings of the 2010 2nd International Conference on Future Computer and Communication, Wuhan, China.","DOI":"10.1109\/ICFCC.2010.5497299"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1938","DOI":"10.1109\/LGRS.2015.2439517","article-title":"Fast Multiclass Vehicle Detection on Aerial Images","volume":"12","author":"Liu","year":"2015","journal-title":"IEEE Geosci. Remote. Sens. Lett."},{"key":"ref_24","unstructured":"Tan, M., and Le, Q.V. (2019, January 9\u201315). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning, ICML, Long Beach, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Messmer, M., Kiefer, B., and Zell, A. (2021). Gaining Scale Invariance in UAV Bird\u2019s Eye View Object Detection by Adaptive Resizing. CoRR, abs\/2101.12694.","DOI":"10.1109\/ICPR56361.2022.9956122"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4026","DOI":"10.3390\/rs70404026","article-title":"Evaluating multispectral images and vegetation indices for precision farming applications from UAV images","volume":"7","author":"Candiago","year":"2015","journal-title":"Remote Sens."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.isprsjprs.2018.09.008","article-title":"UAV-based multispectral remote sensing for precision agriculture: A comparison between different cameras","volume":"146","author":"Deng","year":"2018","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, L., Zhang, H., Niu, Y., and Han, W. (2019). Mapping maize water stress based on UAV multispectral remote sensing. Remote Sens., 11.","DOI":"10.3390\/rs11060605"},{"key":"ref_29","unstructured":"Karasawa, T., Watanabe, K., Ha, Q., Tejero-de-Pablos, A., Ushiku, Y., and Harada, T. Multispectral Object Detection for Autonomous Vehicles. Proceedings of the Thematic Workshops \u201917: Proceedings of the on Thematic Workshops of ACM Multimedia 2017, Mountain View, CA, USA, 23\u201327 October 2017."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Vandersteegen, M., Beeck, K.V., and Goedem\u00e9, T. (2018, January 27\u201329). Real-Time Multispectral Pedestrian Detection with a Single-Pass Deep Neural Network. Proceedings of the Image Analysis and Recognition\u201415th International Conference, ICIAR 2018, Povoa de Varzim, Portugal.","DOI":"10.1007\/978-3-319-93000-8_47"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.inffus.2018.11.017","article-title":"Fusion of multispectral data through illumination-aware deep neural networks for pedestrian detection","volume":"50","author":"Guan","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ophoff, T., Beeck, K.V., and Goedem\u00e9, T. (2019). Exploring RGB + Depth Fusion for Real-Time Object Detection. Sensors, 19.","DOI":"10.3390\/s19040866"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"104042","DOI":"10.1016\/j.imavis.2020.104042","article-title":"Deep multimodal fusion for semantic image segmentation: A survey","volume":"105","author":"Zhang","year":"2021","journal-title":"Image Vis. Comput."},{"key":"ref_34","unstructured":"(2015). Photography\u2014Digital Cameras\u2014Geometric Distortion (GD) Measurements. Standard No. ISO17850:2015."},{"key":"ref_35","first-page":"444","article-title":"Decentering distortion of lenses","volume":"32","author":"Brown","year":"1966","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Poynton, C. (2012). Digital Video and HD: Algorithms and Interfaces, Elsevier.","DOI":"10.1016\/B978-0-12-391926-7.50063-1"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/S0167-2789(02)00734-0","article-title":"On the foundations of vision modeling: I. Weber\u2019s law and Weberized TV restoration","volume":"175","author":"Shen","year":"2003","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zafrir, O., Boudoukh, G., Izsak, P., and Wasserblat, M. (2019, January 13). Q8bert: Quantized 8 bit bert. Proceedings of the 2019 Fifth Workshop on Energy Efficient Machine Learning and Cognitive Computing\u2014NeurIPS Edition (EMC2-NIPS), Vancouver, BC, Canada.","DOI":"10.1109\/EMC2-NIPS53020.2019.00016"},{"key":"ref_39","unstructured":"Krishnamoorthi, R. (2018). Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. (2018, January 18\u201322). Quantization and training of neural networks for efficient integer-arithmetic-only inference. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00286"},{"key":"ref_41","unstructured":"Wang, C.W., Cheng, C.A., Cheng, C.J., Hu, H.N., Chu, H.K., and Sun, M. (2019, January 16\u201320). Augpod: Augmentation-oriented probabilistic object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshop on the Robotic Vision Probabilistic Object Detection Challenge, Long Beach, CA, USA."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhang, L., Zhang, Y., Zhang, Z., Shen, J., and Wang, H. (2019). Real-time water surface object detection based on improved faster R-CNN. Sensors, 19.","DOI":"10.3390\/s19163523"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1109\/TCE.2007.381734","article-title":"A Dynamic Histogram Equalization for Image Contrast Enhancement","volume":"53","author":"Kabir","year":"2007","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Lin, T., Maire, M., Belongie, S.J., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., and Zitnick, C.L. (2014, January 6\u201312). Microsoft COCO: Common Objects in Context. Proceedings of the 13th European Conference, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1364\/JOSA.41.000302","article-title":"The Near Infrared Absorption Spectrum of Liquid Water","volume":"41","author":"Curcio","year":"1951","journal-title":"J. Opt. Soc. Am."},{"key":"ref_46","first-page":"1","article-title":"PNG (Portable Network Graphics) Specification Version 1.0","volume":"2083","author":"Boutell","year":"1997","journal-title":"RFC"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1145\/103085.103089","article-title":"The JPEG Still Picture Compression Standard","volume":"34","author":"Wallace","year":"1991","journal-title":"Commun. ACM"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Newell, A., Yang, K., and Deng, J. (2016, January 11\u201314). Stacked hourglass networks for human pose estimation. Proceedings of the Computer Vision\u2014ECCV 2016\u201414th European Conference, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46484-8_29"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_50","unstructured":"Pailla, D.R., Kollerathu, V.A., and Chennamsetty, S.S. (2019). Object detection on aerial imagery using CenterNet. CoRR, abs\/1908.08244."},{"key":"ref_51","unstructured":"Tan, M., and Le, Q.V. (2021, January 18\u201324). EfficientNetV2: Smaller Models and Faster Training. Proceedings of the 38th International Conference on Machine Learning, ICML 2021, Virtual Event."},{"key":"ref_52","unstructured":"Kingma, D.P., and Ba, J.L. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Xie, S., Girshick, R., Doll\u00e1r, P., Tu, Z., and He, K. (2017, January 21\u201326). Aggregated residual transformations for deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.634"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/21\/5508\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:09:00Z","timestamp":1760144940000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/21\/5508"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,1]]},"references-count":53,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["rs14215508"],"URL":"https:\/\/doi.org\/10.3390\/rs14215508","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2022,11,1]]}}}