{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T07:28:17Z","timestamp":1772782097629,"version":"3.50.1"},"reference-count":32,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2020,9,29]],"date-time":"2020-09-29T00:00:00Z","timestamp":1601337600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Metallography is the study of the structure of metals and alloys. Metallographic analysis can be regarded as a detection tool to assist in identifying a metal or alloy, to evaluate whether an alloy is processed correctly, to inspect multiple phases within a material, to locate and characterize imperfections such as voids or impurities, or to find the damaged areas of metallographic images. However, the defect detection of metallography is evaluated by human experts, and its automatic identification is still a challenge in almost every real solution. Deep learning has been applied to different problems in computer vision since the proposal of AlexNet in 2012. In this study, we propose a novel convolutional neural network architecture for metallographic analysis based on a modified residual neural network (ResNet). Multi-scale ResNet (M-ResNet), the modified method, improves efficiency by utilizing multi-scale operations for the accurate detection of objects of various sizes, especially small objects. The experimental results show that the proposed method yields an accuracy of 85.7% (mAP) in recognition performance, which is higher than existing methods. As a consequence, we propose a novel system for automatic defect detection as an application for metallographic analysis.<\/jats:p>","DOI":"10.3390\/s20195593","type":"journal-article","created":{"date-parts":[[2020,9,29]],"date-time":"2020-09-29T20:56:22Z","timestamp":1601412982000},"page":"5593","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["A Study of Defect Detection Techniques for Metallographic Images"],"prefix":"10.3390","volume":"20","author":[{"given":"Wei-Hung","family":"Wu","sequence":"first","affiliation":[{"name":"Department of Mechatoronics Engineering, National Changhua University of Education, Changhua City 50007, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jen-Chun","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Telecommunication Engineering, National Kaohsiung University of Science and Technology, Kaohsiung City 80778, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yi-Ming","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Mechatoronics Engineering, National Changhua University of Education, Changhua City 50007, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition CVPR, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016, January 11\u201314). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_3","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition CVPR, Las Vegas, NV, USA."},{"key":"ref_4","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition CVPR, Las Vegas, NV, USA."},{"key":"ref_5","unstructured":"Bochkovskiy, A., Wang, C.Y., and Mark Liao, H.Y. (2020, January 14\u201319). YOLOv4: Optimal speed and accuracy of object detection. Proceedings of the Conference on Computer Vision and Pattern Recognition CVPR, Nanjing, China."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/S1044-5803(01)00128-0","article-title":"Computer-aided versus manual grain size assessment in a single phase material","volume":"46","author":"Latala","year":"2001","journal-title":"Mater. Charact."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"959","DOI":"10.1016\/S0892-6875(98)00083-1","article-title":"Analysis of ore particles based on textural pattern recognition","volume":"11","author":"Petersen","year":"1998","journal-title":"Miner. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1016\/S0921-5093(01)01892-5","article-title":"An image processing method for morphology characterization and pitting corrosion evaluation","volume":"334","author":"Codaro","year":"2002","journal-title":"Mater. Sci. Eng. A"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"964","DOI":"10.1016\/j.matchar.2009.03.014","article-title":"Application of digital image processing for automatic detection of characterization of cracked constituent particles\/inclusions in wrought aluminum alloys","volume":"60","author":"Lee","year":"2009","journal-title":"Mater. Charact."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Han, F., Zou, J., Ai, Y., Xu, C., Liu, S., and Liu, S. (2019, January 11\u201315). Image classification and analysis during the additive manufacturing process based on deep convolutional neural networks. Proceedings of the 20th International Conference on Electronic Packaging Technology(ICEPT), Hong Kong, China.","DOI":"10.1109\/ICEPT47577.2019.245264"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1007\/978-3-030-50347-5_26","article-title":"Detecting defects in materials using deep convolutional neural networks","volume":"Volume 12131","author":"Boyadjian","year":"2020","journal-title":"Lecture Notes in Computer Science (LNCS)"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.engappai.2019.06.007","article-title":"Segmentation of casting defect regions for the extraction of microstructural properties","volume":"85","author":"Lin","year":"2019","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"7068349","DOI":"10.1155\/2018\/7068349","article-title":"Deep learning for computer vision: A brief review","volume":"2018","author":"Voulodimos","year":"2018","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.neucom.2015.09.116","article-title":"Deep learning for visual understanding: A review","volume":"187","author":"Guo","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1007\/s11263-013-0620-5","article-title":"Selective search for object recognition","volume":"104","author":"Uijlings","year":"2013","journal-title":"Int. J. Comput. Vis."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015). Fast R-CNN. arXiv.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_17","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation Inc."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Redmon, J., and Farhadi, A. (2017, January 21\u201326). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.690"},{"key":"ref_20","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation Inc."},{"key":"ref_21","unstructured":"(2020, September 25). ASM Micrograph Database. Available online: https:\/\/www.asminternational.org\/materials-resources\/online-databases\/-\/journal_content\/56\/10192\/15468789\/DATABASE."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R.B., He, K., Hariharan, B., and Belongie, S.J. (2017, January 21\u201326). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Bulo, S.R., Porzi, L., and Kontschieder, P. (2018, January 18\u201323). In-place activated batchnorm for memory-optimized training of DNNs. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00591"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhao, H., Zhang, Y., Liu, S., Shi, J., Loy, C.C., Lin, D., and Jia, J. (2018, January 8\u201314). PSANet: Point-wise spatial attention network for scene parsing. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01240-3_17"},{"key":"ref_25","unstructured":"Yu, F., and Koltun, V. (2016, January 2\u20134). Multi-scale context aggregation by dilated convolutions. Proceedings of the International Conference on Learning Representations (ICLR), San Juan, PR, USA."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","article-title":"The pascal visual object classes (voc) challenge","volume":"88","author":"Everingham","year":"2010","journal-title":"Int. J. Comput. Vis."},{"key":"ref_27","unstructured":"Chen, S., Abhinav, S., Saurabh, S., and Abhinav, G. (2017, January 22\u201329). Revisting unreasonable effectiveness of data in deep learning era. Proceedings of the International Conference on Computer Vision (ICCV), Venice, Italy."},{"key":"ref_28","unstructured":"Long, X., Deng, K., Wang, G., Zhang, Y., Dang, Q., Gao, Y., Shen, H., and Ren, J. (2020). PP-YOLO: An effective and efficient implementation of object detector. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"128837","DOI":"10.1109\/ACCESS.2019.2939201","article-title":"A survey of deep learning-based object detection","volume":"7","author":"Jiao","year":"2019","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wei, S., Su, H., Ming, J., Wang, C., Yan, M., Kumar, D., Shi, J., and Zhang, X. (2020). Precise and robust ship detection for high-resolution SAR imagery based on HR-SDNet. Remote Sens., 12.","DOI":"10.3390\/rs12010167"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on image data augmentation for deep learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J. Big Data"},{"key":"ref_32","unstructured":"Antoniou, A., Storkey, A., and Edwards, H. (2017). Data augmentation generative adversarial networks. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5593\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:14:57Z","timestamp":1760177697000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5593"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,29]]},"references-count":32,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2020,10]]}},"alternative-id":["s20195593"],"URL":"https:\/\/doi.org\/10.3390\/s20195593","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,29]]}}}