{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T15:30:59Z","timestamp":1772206259160,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,5,28]],"date-time":"2024-05-28T00:00:00Z","timestamp":1716854400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003995","name":"Anhui Provincial Natural Science Foundation","doi-asserted-by":"publisher","award":["2108085ME166"],"award-info":[{"award-number":["2108085ME166"]}],"id":[{"id":"10.13039\/501100003995","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003995","name":"Anhui Provincial Natural Science Foundation","doi-asserted-by":"publisher","award":["KJ2021A0408"],"award-info":[{"award-number":["KJ2021A0408"]}],"id":[{"id":"10.13039\/501100003995","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Natural Science Research Project of Universities in Anhui Province","award":["2108085ME166"],"award-info":[{"award-number":["2108085ME166"]}]},{"name":"Natural Science Research Project of Universities in Anhui Province","award":["KJ2021A0408"],"award-info":[{"award-number":["KJ2021A0408"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In order to improve the efficiency and accuracy of multitarget detection of soldering defects on surface-mounted components in Printed Circuit Board (PCB) fabrication, we propose a sample generation method using Stable Diffusion Model and ControlNet, as well as a defect detection method based on the Swin Transformer. The method consists of two stages: First, high-definition original images collected in industrial production and the corresponding prompts are input to Stable Diffusion Model and ControlNet for automatic generation of nonindependent samples. Subsequently, we integrate Swin Transformer as the backbone into the Cascade Mask R-CNN to improve the quality of defect features extracted from the samples for accurate detection box localization and segmentation. Instead of segmenting individual components on the PCB, the method inspects all components in the field of view simultaneously over a larger area. The experimental results demonstrate the effectiveness of our method in scaling up nonindependent sample datasets, thereby enabling the generation of high-quality datasets. The method accurately recognizes targets and detects defect types when performing multitarget inspection on printed circuit boards. The analysis against other models shows that our improved defect detection and segmentation method improves the Average Recall (AR) by 2.8% and the mean Average Precision (mAP) by 1.9%.<\/jats:p>","DOI":"10.3390\/s24113473","type":"journal-article","created":{"date-parts":[[2024,5,28]],"date-time":"2024-05-28T07:38:39Z","timestamp":1716881919000},"page":"3473","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Automatic PCB Sample Generation and Defect Detection Based on ControlNet and Swin Transformer"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9113-6143","authenticated-orcid":false,"given":"Yulong","family":"Liu","sequence":"first","affiliation":[{"name":"School of Mechanical Engineering, Anhui University of Technology, Maanshan 243032, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4537-2898","authenticated-orcid":false,"given":"Hao","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Anhui University of Technology, Maanshan 243032, China"}]},{"given":"Youzhi","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Anhui University of Technology, Maanshan 243032, China"}]},{"given":"Xiaoming","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Anhui University of Technology, Maanshan 243032, China"}]},{"given":"Xiujuan","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Anhui University of Technology, Maanshan 243032, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,5,28]]},"reference":[{"key":"ref_1","unstructured":"Li, C., and Wu, Z. (September, January 30). Process management system analysis and design of SMT reflow soldering process. Proceedings of the 2005 6th International Conference on Electronic Packaging Technology, Shenzhen, China."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1109\/TCPMT.2015.2501284","article-title":"A New IC Solder Joint Inspection Method for an Automatic Optical Inspection System Based on an Improved Visual Background Extraction Algorithm","volume":"6","author":"Cai","year":"2016","journal-title":"IEEE Trans. Compon. Packag. Manuf. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22\u201329). Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1145\/3422622","article-title":"Generative adversarial networks","volume":"63","author":"Goodfellow","year":"2020","journal-title":"Commun. ACM"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022, January 18\u201324). High-Resolution Image Synthesis with Latent Diffusion Models. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhang, L., Rao, A., and Agrawala, M. (2023). Adding Conditional Control to Text-to-Image Diffusion Models, Cornell University Library.","DOI":"10.1109\/ICCV51070.2023.00355"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., and Dollar, P. (2017, January 22\u201329). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Cai, Z., and Vasconcelos, N. (2018, January 18\u201322). Cascade r-cnn: Delving into high quality object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00644"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Liu, G. (2020, January 18\u201320). Surface Defect Detection Methods Based on Deep Learning: A Brief Review. Proceedings of the 2020 2nd International Conference on Information Technology and Computer Application (ITCA), Guangzhou, China.","DOI":"10.1109\/ITCA52113.2020.00049"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chen, Y., Ding, Y., Zhao, F., Zhang, E., Wu, Z., and Shao, L. (2021). Surface Defect Detection Methods for Industrial Products: A Review. Appl. Sci., 11.","DOI":"10.3390\/app11167657"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1109\/TCPMT.2023.3344096","article-title":"Automatic Solder Defect Detection in Electronic Components Using Transformer Architecture","volume":"14","author":"Liu","year":"2024","journal-title":"IEEE Trans. Compon. Packag. Manuf. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11\u201317). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_14","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention Is All You Need. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature Pyramid Networks for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liu, J., Wu, H., Liu, Y., and Wang, J. (2022, January 18\u201320). Automatic Generation and Detection Method of LCD Samples Based on Deep Learning. Proceedings of the 2022 5th World Conference on Mechanical Engineering and Intelligent Manufacturing (WCMEIM), Ma\u2019anshan, China.","DOI":"10.1109\/WCMEIM56910.2022.10021421"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2021.3087826","article-title":"Auto-Annotated Deep Segmentation for Surface Defect Detection","volume":"70","author":"Tsai","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_18","first-page":"6840","article-title":"Denoising diffusion probabilistic models","volume":"33","author":"Ho","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_19","first-page":"8780","article-title":"Diffusion models beat gans on image synthesis","volume":"34","author":"Dhariwal","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_20","unstructured":"Song, J., Meng, C., and Ermon, S. (2020). Denoising diffusion implicit models. arXiv."},{"key":"ref_21","unstructured":"Kingma, D.P., and Welling, M. (2013). Auto-Encoding Variational Bayes, Cornell University Library."},{"key":"ref_22","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021). Learning Transferable Visual Models from Natural Language Supervision, Cornell University Library."},{"key":"ref_23","unstructured":"Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., and Chen, M. (2021). GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models, Cornell University Library."},{"key":"ref_24","unstructured":"Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. (2022). Hierarchical text-conditional image generation with clip latents. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Seyed, K.S.G., Ayan, B.K., Mahdavi, S.S., and Lopes, R.G. (2022). Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding, Cornell University Library.","DOI":"10.1145\/3528233.3530757"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Saharia, C., Chan, W., Chang, H., Lee, C., Ho, J., Salimans, T., and Norouzi, M. (2022, January 7\u201311). Palette: Image-to-image diffusion models. Proceedings of the ACM SIGGRAPH 2022 Conference Proceedings, Vancouver, BC, Canada.","DOI":"10.1145\/3528233.3530757"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhao, Z., Bai, H., Zhu, Y., Zhang, J., Xu, S., Zhang, Y., Zhang, K., Meng, D., Timofte, R., and Gool, L.V. (2023). DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion, Cornell University Library.","DOI":"10.1109\/ICCV51070.2023.00742"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Soukup, D., and Huber-M\u00f6rk, R. (2014). Convolutional Neural Networks for Steel Surface Defect Detection from Photometric Stereo Images, Springer International Publishing.","DOI":"10.1007\/978-3-319-14249-4_64"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"109316","DOI":"10.1016\/j.measurement.2021.109316","article-title":"Automatic defect detection and segmentation of tunnel surface using modified Mask R-CNN","volume":"178","author":"Xu","year":"2021","journal-title":"Meas. J. Int. Meas. Confed."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1109\/TCPMT.2019.2952393","article-title":"Solder Joint Recognition Using Mask R-CNN Method","volume":"10","author":"Wu","year":"2020","journal-title":"IEEE Trans. Compon. Packag. Manuf. Technol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1109\/TSM.2023.3245606","article-title":"Watermark Detection in CMOS Image Sensors Using Cosine-Convolutional Semantic Networks","volume":"36","author":"Solorzano","year":"2023","journal-title":"IEEE Trans. Semicond. Manuf."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Fan, C.M., Liu, T.J., and Liu, K.H. (June, January 27). SUNet: Swin Transformer UNet for Image Denoising. Proceedings of the 2022 IEEE International Symposium on Circuits and Systems (ISCAS), Austin, TX, USA.","DOI":"10.1109\/ISCAS48785.2022.9937486"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zunair, H., and Hamza, A.B. (2021). Sharp U-Net: Depthwise convolutional network for biomedical image segmentation. Comput. Biol. Med., 136.","DOI":"10.1016\/j.compbiomed.2021.104699"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"129480","DOI":"10.1109\/ACCESS.2022.3228206","article-title":"PCB Defect Detection Method Based on Transformer-YOLO","volume":"10","author":"Chen","year":"2022","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1007\/s11263-007-0090-8","article-title":"LabelMe: A database and web-based tool for image annotation","volume":"77","author":"Russell","year":"2008","journal-title":"Int. J. Comput. Vis."},{"key":"ref_36","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Liu, S., and Deng, W. (2015, January 3\u20136). Very deep convolutional neural network based image classification using small training sample size. Proceedings of the 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ACPR.2015.7486599"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_39","unstructured":"Loshchilov, I., and Hutter, F. (2019). Decoupled Weight Decay Regularization, Cornell University Library."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/11\/3473\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:49:27Z","timestamp":1760107767000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/11\/3473"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,28]]},"references-count":39,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["s24113473"],"URL":"https:\/\/doi.org\/10.3390\/s24113473","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,28]]}}}