{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T19:29:08Z","timestamp":1773775748469,"version":"3.50.1"},"reference-count":46,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,6,4]],"date-time":"2024-06-04T00:00:00Z","timestamp":1717459200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science and Technology Council, Taiwan","award":["NSTC 112-2221-E-324-012-MY2"],"award-info":[{"award-number":["NSTC 112-2221-E-324-012-MY2"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The integrity of product assembly in the precision assembly industry significantly influences the quality of the final products. During the assembly process, products may acquire assembly defects due to personnel oversight. A severe assembly defect could impair the product\u2019s normal function and potentially cause loss of life or property for the user. For workpiece defect inspection, there is limited discussion on the simultaneous detection of the primary kinds of assembly anomaly (missing parts, misplaced parts, foreign objects, and extra parts). However, these assembly anomalies account for most customer complaints in the traditional hand tool industry. This is because no equipment can comprehensively inspect major assembly defects, and inspections rely solely on professionals using simple tools and their own experience. Thus, this study proposes an automated visual inspection system to achieve defect inspection in hand tool assembly. This study samples the work-in-process from three assembly stations in the ratchet wrench assembly process; an investigation of 28 common assembly defect types is presented, covering the 4 kinds of assembly anomaly in the assembly operation; also, this study captures sample images of various assembly defects for the experiments. First, the captured images are filtered to eliminate surface reflection noise from the workpiece; then, a circular mask is given at the assembly position to extract the ROI area; next, the filtered ROI images are used to create a defect-type label set using manual annotation; after this, the R-CNN series network models are applied to object feature extraction and classification; finally, they are compared with other object detection models to identify which inspection model has the better performance. The experimental results show that, if each station uses the best model for defect inspection, it can effectively detect and classify defects. The average defect detection rate (1-\u03b2) of each station is 92.64%, the average misjudgment rate (\u03b1) is 6.68%, and the average correct classification rate (CR) is 88.03%.<\/jats:p>","DOI":"10.3390\/s24113635","type":"journal-article","created":{"date-parts":[[2024,6,4]],"date-time":"2024-06-04T07:39:49Z","timestamp":1717486789000},"page":"3635","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Utilizing Deep Learning for Defect Inspection in Hand Tool Assembly"],"prefix":"10.3390","volume":"24","author":[{"given":"Hong-Dar","family":"Lin","sequence":"first","affiliation":[{"name":"Department of Industrial Engineering and Management, Chaoyang University of Technology, Taichung 413310, Taiwan"}]},{"given":"Cheng-Kai","family":"Jheng","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering and Management, Chaoyang University of Technology, Taichung 413310, Taiwan"}]},{"given":"Chou-Hsien","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Civil, Architectural, and Environmental Engineering, The University of Texas at Austin, Austin, TX 78712-0273, USA"}]},{"given":"Hung-Tso","family":"Chang","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering and Management, Chaoyang University of Technology, Taichung 413310, Taiwan"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"104453","DOI":"10.1016\/j.engfailanal.2020.104453","article-title":"Research of an aircraft engine cylinder assembly integrity assessment-Thermomechanical FEM analysis","volume":"111","year":"2020","journal-title":"Eng. Fail. Anal."},{"key":"ref_2","first-page":"389","article-title":"Integrity assessment of an aircraft cylinder assembly with a crack","volume":"56","year":"2022","journal-title":"Mater. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chen, Y., Ding, Y., Zhao, F., Zhang, E., Wu, Z., and Shao, L. (2021). Surface defect detection methods for industrial products: A review. Appl. Sci., 11.","DOI":"10.3390\/app11167657"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"183192","DOI":"10.1109\/ACCESS.2020.3029127","article-title":"A review and analysis of automatic optical inspection and quality monitoring methods in electronic industry","volume":"8","author":"Ebayyeh","year":"2020","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1016\/j.jare.2021.03.015","article-title":"A review on modern defect detection models using DCNNs-Deep convolutional neural networks","volume":"35","author":"Tulbure","year":"2022","journal-title":"J. Adv. Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1016\/j.future.2018.05.072","article-title":"A unified framework for automated inspection of handheld safety critical devices in production assemblies","volume":"88","author":"Rehmat","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.compind.2014.10.006","article-title":"Automated visual inspection in the semiconductor industry: A survey","volume":"66","author":"Huang","year":"2015","journal-title":"Comput. Ind."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3260","DOI":"10.1016\/j.eswa.2010.09.012","article-title":"Machine learning approach for automated visual inspection of machine components","volume":"38","author":"Ravikumar","year":"2011","journal-title":"Expert Syst. Appl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1016\/j.promfg.2020.02.072","article-title":"An embedded machine vision system for an in-line quality check of assembly processes","volume":"42","author":"Frustaci","year":"2020","journal-title":"Procedia Manuf."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1016\/j.jmsy.2017.03.007","article-title":"Inspection of assembly error with effect on throat and incidence for turbine blades","volume":"43","author":"Chen","year":"2017","journal-title":"J. Manuf. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1016\/j.rcim.2014.10.004","article-title":"Vision-based assembly and inspection system for golf club heads","volume":"32","author":"Natsagdorj","year":"2015","journal-title":"Robot. Comput. Integr. Manuf."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1016\/S0360-8352(97)00115-0","article-title":"Automated misplaced component inspection for printed circuit boards","volume":"33","author":"Enke","year":"1997","journal-title":"Comput. Ind. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1016\/j.eng.2020.09.017","article-title":"A portable noncontact profile scanning system for aircraft assembly","volume":"15","author":"Liang","year":"2021","journal-title":"Engineering"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"914","DOI":"10.1016\/j.promfg.2017.07.195","article-title":"Virtual commissioning of camera-based quality assurance systems for mixed model assembly lines","volume":"11","author":"Piero","year":"2017","journal-title":"Procedia Manuf."},{"key":"ref_15","first-page":"234","article-title":"Multi-angle automotive fuse box detection and assembly method based on machine vision","volume":"145","author":"Wang","year":"2019","journal-title":"Microprocess. Microsyst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.vrih.2019.12.002","article-title":"A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection","volume":"2","author":"Zheng","year":"2020","journal-title":"Virtual Real. Intell. Hardw."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Trevor, D., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"107346","DOI":"10.1016\/j.knosys.2021.107346","article-title":"Stereo priori RCNN based car detection on point level for autonomous driving","volume":"229","author":"Tao","year":"2021","journal-title":"Knowl.-Based Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1016\/j.procs.2019.01.232","article-title":"RCNN-based foreign object detection for securing power transmission lines (RCNN4SPTL)","volume":"147","author":"Zhang","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"102844","DOI":"10.1016\/j.autcon.2019.102844","article-title":"Quasi-autonomous bolt-loosening detection method using vision-based deep learning and image processing","volume":"105","author":"Huynh","year":"2019","journal-title":"Autom. Constr."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.eswa.2018.07.020","article-title":"Pedestrian detection based on fast R-CNN in nighttime by fusing deep convolutional features of successive images","volume":"114","author":"Kim","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"109742","DOI":"10.1016\/j.measurement.2021.109742","article-title":"An optimized railway fastener detection method based on modified Fast R-CNN","volume":"182","author":"Bai","year":"2021","journal-title":"Measurement"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"102898","DOI":"10.1016\/j.scs.2021.102898","article-title":"Automated rail surface crack analytics using deep data-driven models and transfer learning","volume":"70","author":"Zheng","year":"2021","journal-title":"Sustain. Cities Soc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"3328","DOI":"10.1016\/j.matpr.2021.07.249","article-title":"Detection and localization of cotton based on deep neural networks","volume":"80","author":"Annapoorna","year":"2023","journal-title":"Mater. Today Proc."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"104190","DOI":"10.1016\/j.engappai.2021.104190","article-title":"Underwater target detection based on Faster R-CNN and adversarial occlusion network","volume":"10","author":"Zeng","year":"2021","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"104009","DOI":"10.1016\/j.autcon.2021.104009","article-title":"Quantitative loosening detection of threaded fasteners using vision-based deep learning and geometric imaging theory","volume":"133","author":"Gong","year":"2022","journal-title":"Autom. Constr."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Dollar, P., and Girshick, R. (2017, January 22\u201329). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"109316","DOI":"10.1016\/j.measurement.2021.109316","article-title":"Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN","volume":"178","author":"Xu","year":"2021","journal-title":"Measurement"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1016\/j.cja.2020.09.024","article-title":"A mask R-CNN based method for inspecting cable brackets in aircraft","volume":"34","author":"Zhao","year":"2020","journal-title":"Chin. J. Aeronaut."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhad, A. (2016, January 27\u201330). You Only Look Once: Unified, real-time object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_33","unstructured":"Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y. (2020). YOLOv4: Optimal speed and accuracy of object detection. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"103596","DOI":"10.1016\/j.autcon.2021.103596","article-title":"Real-time railroad track components inspection based on the improved YOLOv4 framework","volume":"125","author":"Guo","year":"2021","journal-title":"Autom. Constr."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/j.jmsy.2021.11.004","article-title":"Real-time precise object segmentation using a pixel-wise coarse-fine method with deep learning for automated manufacturing","volume":"62","author":"Cho","year":"2022","journal-title":"J. Manuf. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"116178","DOI":"10.1016\/j.eswa.2021.116178","article-title":"Gaussian-IoU loss: Better learning for bounding box regression on PCB component detection","volume":"190","author":"Liu","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"4709","DOI":"10.1016\/j.matpr.2020.10.301","article-title":"Automated inspection system for automobile bearing seals","volume":"46","author":"Ashwini","year":"2021","journal-title":"Mater. Today Proc."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"11085","DOI":"10.1016\/j.ceramint.2021.12.328","article-title":"Ceramic tile surface defect detection based on deep learning","volume":"48","author":"Guang","year":"2022","journal-title":"Ceram. Int."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"103807","DOI":"10.1016\/j.micpro.2020.103807","article-title":"Machine vision based online detection of PCB defect","volume":"82","author":"Liu","year":"2021","journal-title":"Microprocess. Microsyst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"100933","DOI":"10.1016\/j.aei.2019.100933","article-title":"One class based feature learning approach for defect detection using deep autoencoders","volume":"42","author":"Mujeeb","year":"2019","journal-title":"Adv. Eng. Inform."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"104242","DOI":"10.1016\/j.engappai.2021.104242","article-title":"Computer vision detection of foreign objects in coal processing using attention CNN","volume":"101","author":"Zhang","year":"2021","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"102237","DOI":"10.1016\/j.rcim.2021.102237","article-title":"Comparison of visual servoing technologies for robotized aerospace structural assembly and inspection","volume":"73","author":"Villani","year":"2022","journal-title":"Robot. Comput. Integr. Manuf."},{"key":"ref_43","unstructured":"Gonzalez, R.C., and Woods, R.E. (2018). Digital Image Processing, Pearson Education. [4th ed.]."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Li, F.-F. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_45","unstructured":"Montgomery, D.C. (2019). Design and Analysis of Experiments, John Wiley & Sons, Inc.. [10th ed.]."},{"key":"ref_46","unstructured":"Montgomery, D.C., Runger, G.C., and Hubele, N.F. (2010). Engineering Statistics, Wiley. [5th ed.]."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/11\/3635\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:53:32Z","timestamp":1760108012000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/11\/3635"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,4]]},"references-count":46,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["s24113635"],"URL":"https:\/\/doi.org\/10.3390\/s24113635","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,4]]}}}