{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T11:04:42Z","timestamp":1784027082599,"version":"3.55.0"},"reference-count":37,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T00:00:00Z","timestamp":1783987200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Planning Project of Science and Technology of Yangzhou","award":["YZ2024219"],"award-info":[{"award-number":["YZ2024219"]}]},{"name":"Jiangsu Province Higher Education Basic Science Research General Project","award":["25KJB460029"],"award-info":[{"award-number":["25KJB460029"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The performance of visual detection models in industrial applications is strongly influenced by training dataset quality. Although imaging scheme design and algorithm optimization are often emphasized, systematic dataset quality evaluation remains insufficient. To address this gap, this study proposes a dataset quality evaluation framework for industrial object detection. It includes four dimensions and thirteen quantifiable indices: three for acquisition environment, two for image quality, three for dataset scale, and five for annotation quality. Normalization based on theoretical maximum scores is used to reduce biases caused by different score ranges, and dimension weights are assigned using Taguchi orthogonal experiments. Validation is performed on five public and three self-constructed datasets using YOLOv12n and RT-DETR-R18. A positive correlation trend is observed between the proposed scores and detection accuracy, with PLCC\/SRCC\/Kendall\u2019s tau values of 0.685\/0.850\/0.764 and 0.656\/0.826\/0.691, respectively. After second-level weight optimization, the correlations increase to 0.775\/0.922\/0.837 and 0.748\/0.898\/0.764. Corresponding p-values and 95% confidence intervals are reported to quantify statistical uncertainty. Sensitivity analysis and ablation comparisons further verify the robustness and necessity of the proposed multidimensional framework. The proposed framework provides a quantifiable method and practical acquisition guidelines for improving industrial image dataset quality.<\/jats:p>","DOI":"10.3390\/s26144465","type":"journal-article","created":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T10:12:53Z","timestamp":1784023973000},"page":"4465","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["An Image Dataset Quality Evaluation System for Industrial Object Detection Tasks"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-0828-0150","authenticated-orcid":false,"given":"Shengguo","family":"Zhu","sequence":"first","affiliation":[{"name":"School of Mechanical Engineering, Yangzhou University, Yangzhou 225127, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-3897-7457","authenticated-orcid":false,"given":"Yunxi","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Yangzhou University, Yangzhou 225127, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5222-440X","authenticated-orcid":false,"given":"Enhui","family":"Lu","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Yangzhou University, Yangzhou 225127, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8772-248X","authenticated-orcid":false,"given":"Xinglong","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Yangzhou University, Yangzhou 225127, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9003-8042","authenticated-orcid":false,"given":"Jian","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Advanced Design and Manufacture for Vehicle Body, Hunan University, Changsha 410082, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,7,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1007\/s40684-021-00343-6","article-title":"State of the Art in Defect Detection Based on Machine Vision","volume":"9","author":"Ren","year":"2022","journal-title":"Int. J. Precis. Eng. Manuf. Green Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/s10462-026-11502-z","article-title":"Quality over Quantity: A Data-Centric Survey of Annotation Errors in Object Detection Datasets","volume":"59","author":"Hussain","year":"2026","journal-title":"Artif. Intell. Rev."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Liu, H., Zhao, D., and Li, Y. (2026). YOLOv12-VSD: A Transfer-Learning-Assisted Real-Time Detection Algorithm for Vehicle Surface Defects. Sensors, 26.","DOI":"10.3390\/s26092905"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Padilla, R., Passos, W.L., Dias, T.L.B., Netto, S.L., and Da Silva, E.A.B. (2021). A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit. Electronics, 10.","DOI":"10.3390\/electronics10030279"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"128102","DOI":"10.1016\/j.neucom.2024.128102","article-title":"Development and Challenges of Object Detection: A Survey","volume":"598","author":"Li","year":"2024","journal-title":"Neurocomputing"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"107717","DOI":"10.1016\/j.engappai.2023.107717","article-title":"A Systematic Review of Deep Learning Approaches for Surface Defect Detection in Industrial Applications","volume":"130","author":"Ameri","year":"2024","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"115060","DOI":"10.1016\/j.measurement.2024.115060","article-title":"WSS-YOLO: An Improved Industrial Defect Detection Network for Steel Surface Defects","volume":"236","author":"Lu","year":"2024","journal-title":"Measurement"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Liang, T., Liu, Z., Shi, L., Yang, H., Ma, N., and Wang, H. (2026). SPR-DETR: DETR with Self-Supervised Learning and Position Relation Modeling for UAV-Based Catenary Support Component Detection in Electrified Railways. Sensors, 26.","DOI":"10.3390\/s26103077"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"122669","DOI":"10.1016\/j.eswa.2023.122669","article-title":"DsP-YOLO: An Anchor-Free Network with DsPAN for Small Object Detection of Multiscale Defects","volume":"241","author":"Zhang","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Majeed, A., and Hwang, S.O. (2024). A Data-Centric AI Paradigm for Socio-Industrial and Global Challenges. Electronics, 13.","DOI":"10.3390\/electronics13112156"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Mohanty, S.P., Hughes, D.P., and Salath\u00e9, M. (2016). Using Deep Learning for Image-Based Plant Disease Detection. Front. Plant Sci., 7.","DOI":"10.3389\/fpls.2016.01419"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"102858","DOI":"10.1016\/j.displa.2024.102858","article-title":"Subjective and Objective Quality Evaluation for Industrial Images","volume":"85","author":"Zhou","year":"2024","journal-title":"Displays"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1007\/s10851-015-0626-4","article-title":"Evaluation Method, Dataset Size or Dataset Content: How to Evaluate Algorithms for Image Matching?","volume":"55","author":"Kanwal","year":"2016","journal-title":"J. Math. Imaging Vis."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3388","DOI":"10.1109\/TPAMI.2020.2981890","article-title":"Imbalance Problems in Object Detection: A Review","volume":"43","author":"Oksuz","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"B\u00e4r, A., Uhrig, J., Umesh, J.P., Cordts, M., and Fingscheidt, T. (2023, January 17\u201324). A Novel Benchmark for Refinement of Noisy Localization Labels in Autolabeled Datasets for Object Detection. Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada.","DOI":"10.1109\/CVPRW59228.2023.00399"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Murrugarra-Llerena, J., and Jung, C.R. (March, January 26). Noise-Aware Evaluation of Object Detectors. Proceedings of the 2025 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Tucson, AZ, USA.","DOI":"10.1109\/WACV61041.2025.00903"},{"key":"ref_17","first-page":"17","article-title":"The key issues and evaluation methods for constructing agricultural pest and disease image datasets: A review","volume":"5","author":"Guan","year":"2023","journal-title":"Smart Agric."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"108910","DOI":"10.1016\/j.optlaseng.2025.108910","article-title":"Advancing Visual Roughness Measurement: A Full-Reference Imaging Approach for Light Source Interference Elimination","volume":"188","author":"Lu","year":"2025","journal-title":"Opt. Lasers Eng."},{"key":"ref_19","first-page":"876","article-title":"A Morphological Filter Chip Using a Modified Decoding Function","volume":"47","author":"Ong","year":"2000","journal-title":"IEEE Trans. Circuits Syst. II"},{"key":"ref_20","first-page":"134","article-title":"Detection of straw coverage based on multi-threshold and multi-target UAV image segmentation optimization algorithm","volume":"36","author":"Liu","year":"2020","journal-title":"Trans. Chin. Soc. Agric. Eng."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1001","DOI":"10.1007\/s10845-010-0393-4","article-title":"Artificial Bee Colony Algorithm for Large-Scale Problems and Engineering Design Optimization","volume":"23","author":"Akay","year":"2012","journal-title":"J. Intell. Manuf."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1007\/s00530-014-0446-1","article-title":"Mean Opinion Score (MOS) Revisited: Methods and Applications, Limitations and Alternatives","volume":"22","author":"Streijl","year":"2016","journal-title":"Multimed. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Venkatanath, N., Praneeth, D., Bh, M.C., Channappayya, S.S., and Medasani, S.S. (March, January 27). Blind Image Quality Evaluation Using Perception Based Features. Proceedings of the 2015 Twenty First National Conference on Communications (NCC), Mumbai, India.","DOI":"10.1109\/NCC.2015.7084843"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1080\/00031305.2021.2004922","article-title":"Myths About Linear and Monotonic Associations: Pearson\u2019s r, Spearman\u2019s \u03c1, and Kendall\u2019s \u03c4","volume":"76","author":"Zhan","year":"2022","journal-title":"Am. Stat."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.triboint.2018.02.033","article-title":"Designing Indices to Measure Surface Roughness Based on the Color Distribution Statistical Matrix (CDSM)","volume":"122","author":"Lu","year":"2018","journal-title":"Tribol. Int."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"440","DOI":"10.1016\/j.physa.2016.01.056","article-title":"Measuring mixing patterns in complex networks by Spearman rank correlation coefficient","volume":"451","author":"Zhang","year":"2016","journal-title":"Phys. A"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Cao, J., Bao, W., Shang, H., Yuan, M., and Cheng, Q. (2023). GCL-YOLO: A GhostConv-Based Lightweight YOLO Network for UAV Small Object Detection. Remote Sens., 15.","DOI":"10.3390\/rs15204932"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Xie, Z., and Zou, X. (2024). MFAD-RTDETR: A Multi-Frequency Aggregate Diffusion Feature Flow Composite Model for Printed Circuit Board Defect Detection. Electronics, 13.","DOI":"10.3390\/electronics13173557"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"16548","DOI":"10.1109\/TITS.2024.3437770","article-title":"Multi-Object Detection for Daily Road Maintenance Inspection with UAV Based on Improved YOLOv8","volume":"25","author":"Zhu","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"An, N., Yang, Z., Wan, L., Li, J., and Wang, Y. (2026). FLF-RCNN: A Fine-Tuned Lightweight Faster RCNN for Precise and Efficient Industrial Quality Inspection. Sensors, 26.","DOI":"10.3390\/s26061768"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Tschirschwitz, D.E., and Rodehorst, V. (March, January 26). Label Convergence: Defining an Upper Performance Bound in Object Recognition through Contradictory Annotations. Proceedings of the 2025 IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV), Tucson, AZ, USA.","DOI":"10.1109\/WACV61041.2025.00666"},{"key":"ref_32","unstructured":"Xu, M., Bai, Y., and Ghanem, B. (2019, January 16\u201320). Missing Labels in Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Caroppo, A., Diraco, G., and Leone, A. (2026). A Systematic Review of Available Multispectral UAV Image Datasets for Precision Agriculture Applications. Remote Sens., 18.","DOI":"10.3390\/rs18040659"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Sun, Y., Yan, H., Shang, Z., and Yang, M. (2025). MCH-YOLOv12: Research on Surface Defect Detection Algorithm for Aluminum Profiles Based on Improved YOLOv12. Sensors, 25.","DOI":"10.3390\/s25175389"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/s10921-025-01251-4","article-title":"A Lightweight RT-DETR Model for Metal Surface Defect Detection Using Multi-Scale Network and Additive Attention Mechanism","volume":"44","author":"Hao","year":"2025","journal-title":"J. Nondestruct. Eval."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1389","DOI":"10.1080\/09537287.2016.1225999","article-title":"A Conceptual Hybrid Framework for Industrial Process Improvement: Integrating Taguchi Methods, Shainin System and Six Sigma","volume":"27","author":"Prashar","year":"2016","journal-title":"Prod. Plan. Control"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/S0924-0136(02)01036-1","article-title":"Die Casting Process Optimization Using Taguchi Methods","volume":"135","author":"Syrcos","year":"2003","journal-title":"J. Mater. Process. Technol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/14\/4465\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T10:16:57Z","timestamp":1784024217000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/14\/4465"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,14]]},"references-count":37,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["s26144465"],"URL":"https:\/\/doi.org\/10.3390\/s26144465","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,14]]}}}