{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T03:37:10Z","timestamp":1774064230385,"version":"3.50.1"},"reference-count":43,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2020,5,6]],"date-time":"2020-05-06T00:00:00Z","timestamp":1588723200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61773104"],"award-info":[{"award-number":["61773104"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Key R&amp;D Program of China","award":["2017YFB0306400"],"award-info":[{"award-number":["2017YFB0306400"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>The automatic generation of language description is an important task in the intelligent analysis of aluminum alloy metallographic images, and is crucial for the high-quality development of the non-ferrous metals manufacturing industry. In this paper, we propose a methodological framework to generate the language description for aluminum alloy metallographic images. The framework consists of two parts: feature extraction and classification. In the process of feature extraction, we used ResNet (residual network) and CNN (convolutional neural network) to extract visual features from metallographic images. Meanwhile, we used LSTM (long short term memory), FastText, and TextCNN to extract language text features from questions. Then, we implemented a fusion strategy to integrate these two features. Finally, we used the fused features as the input of the classification network. This framework turns the description generation problem into a classification task, which greatly simplifies the generation process of language description and provides a new idea for the description of metallographic images. Based on this basic framework, we implemented seven different methods to generate the language description of aluminum alloy metallographic images, and their performance comparisons are given. To verify the effectiveness of this framework, we built the aluminum alloy metallographic image dataset. A large number of experimental results show that this framework can effectively accomplish the given tasks.<\/jats:p>","DOI":"10.3390\/sym12050771","type":"journal-article","created":{"date-parts":[[2020,5,7]],"date-time":"2020-05-07T04:46:07Z","timestamp":1588826767000},"page":"771","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Framework of Specific Description Generation for Aluminum Alloy Metallographic Image Based on Visual and Language Information Fusion"],"prefix":"10.3390","volume":"12","author":[{"given":"Dali","family":"Chen","sequence":"first","affiliation":[{"name":"College of Information Science and Engineering, Northeastern University, Shenyang 110819, China"}]},{"given":"Yang","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Northeastern University, Shenyang 110819, China"}]},{"given":"Shixin","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Northeastern University, Shenyang 110819, China"}]},{"given":"Fang","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Materials Science and Engineering, Northeastern University, Shenyang 110819, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7422-5988","authenticated-orcid":false,"given":"Yangquan","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Engineering, University of California, Merced, CA 95343, USA"}]}],"member":"1968","published-online":{"date-parts":[[2020,5,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1016\/j.actamat.2012.10.044","article-title":"Superior light metals by texture engineering: Optimized aluminum and magnesium alloys for automotive applications","volume":"61","author":"Hirsch","year":"2013","journal-title":"Acta Mater."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1016\/j.jmst.2018.09.004","article-title":"A review of selective laser melting of aluminum alloys: Processing, microstructure, property and developing trends","volume":"35","author":"Zhang","year":"2019","journal-title":"J. Mater. Sci. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Kadleckova, M., Minarik, A., Smolka, P., Mracek, A., Wrzecionko, E., Novak, L., Musilova, L., and Gajdosik, R. (2019). Preparation of Textured Surfaces on Aluminum-Alloy Substrates. Materials, 12.","DOI":"10.3390\/ma12010109"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/S0921-5093(99)00674-7","article-title":"Recent development in aluminium alloys for aerospace applications","volume":"280","author":"Heinz","year":"2000","journal-title":"Mater. Sci. Eng. A"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1016\/S0300-9440(01)00133-3","article-title":"Inorganic\/organic hybrid coatings for aircraft aluminum alloy substrates","volume":"41","author":"Du","year":"2001","journal-title":"Progress Org. Coat."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1038\/nature23894","article-title":"3D printing of high-strength aluminium alloys","volume":"549","author":"Martin","year":"2017","journal-title":"Nature"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/S1044-5803(97)00154-X","article-title":"Metallographic Methods for Revealing the Multiphase Microstructure of TRIP-Assisted Steels","volume":"40","author":"Girault","year":"1998","journal-title":"Mater. Charact."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1007\/BF02648419","article-title":"Porosity formation in AI9 Wt Pct Si3 Wt Pct Cu alloy systems: Metallographic observations","volume":"27","author":"Roy","year":"1996","journal-title":"Metall. Mater. Trans. A"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/S1359-6454(00)00335-9","article-title":"A metallographic and quantitative analysis of the influence of stacking fault energy on shock-hardening in Cu and Cu\u2013Al alloys","volume":"49","author":"Rohatgi","year":"2001","journal-title":"Acta Mater."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/S1044-5803(97)80024-1","article-title":"Microstructural evolution during solidification of austenitic stainless steel weld metals: A color metallographic and electron microprobe analysis study","volume":"38","author":"Rajasekhar","year":"1997","journal-title":"Mater. Charact."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Tamadon, A., Pons, D.J., Sued, K., and Clucas, D. (2017). Development of Metallographic Etchants for the Microstructure Evolution of A6082-T6 BFSW Welds. Metals, 7.","DOI":"10.3390\/met7100423"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3864","DOI":"10.3390\/ma8073864","article-title":"New Analysis Method Application in Metallographic Images through the Construction of Mosaics via Speeded up Robust Features and Scale Invariant Feature Transform","volume":"8","author":"Moreira","year":"2015","journal-title":"Materials"},{"key":"ref_13","first-page":"1419","article-title":"Intelligent system for prediction of mechanical properties of material based on metallographic images","volume":"22","author":"Paulic","year":"2015","journal-title":"Teh. Vjesn.\u2014Tech. Gaz."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1007\/s11106-015-9705-8","article-title":"Metallographic Analysis and Microstructural Image Processing of Sandblasting Nozzles Produced by Powder Metallurgy Methods","volume":"54","author":"Povstyanoi","year":"2015","journal-title":"Powder Metall. Metal Ceram."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1016\/j.commatsci.2016.05.034","article-title":"Image driven machine learning methods for microstructure recognition","volume":"123","author":"Chowdhury","year":"2016","journal-title":"Comput. Mater. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/j.commatsci.2015.08.011","article-title":"A computer vision approach for automated analysis and classification of microstructural image data","volume":"110","author":"DeCost","year":"2015","journal-title":"Comput. Mater. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1016\/j.commatsci.2018.03.004","article-title":"Advanced microstructure classification by data mining methods","volume":"148","author":"Gola","year":"2018","journal-title":"Comput. Mater. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.cageo.2018.03.010","article-title":"A method for automatic grain segmentation of multi-angle cross-polarized microscopic images of sandstone","volume":"115","author":"Jiang","year":"2018","journal-title":"Comput. Geosci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"644","DOI":"10.1016\/j.ndteint.2009.05.002","article-title":"Evaluation of multilayer perceptron and self-organizing map neural network topologies applied on microstructure segmentation from metallographic images","volume":"42","author":"Cortez","year":"2009","journal-title":"NDT E Int."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2078","DOI":"10.1038\/s41598-018-20438-6","article-title":"Pattern recognition with machine learning on optical microscopy images of typical metallurgical microstructures","volume":"8","author":"Bulgarevich","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1002\/jemt.20870","article-title":"Automatic evaluation of nickel alloy secondary phases from sem images","volume":"74","author":"Silva","year":"2011","journal-title":"Microsc. Res. Tech."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1016\/j.eswa.2012.07.062","article-title":"Computer techniques towards the automatic characterization of graphite particles in metallographic images of industrial materials","volume":"40","author":"Papa","year":"2013","journal-title":"Expert Syst. Appl."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"Lecun","year":"2015","journal-title":"Nature"},{"key":"ref_24","unstructured":"Bengio, Y., Goodfellow, J., and Courville, A. (2016). Deep Learning, The MIT Press."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2128","DOI":"10.1038\/s41598-018-20037-5","article-title":"Advanced steel microstructure classification by deep learning methods","volume":"8","author":"Azimi","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Ma, B., Ban, X., Huang, H., Chen, Y., Liu, W., and Zhi, Y. (2018). Deep learning-based image segmentation for al-la alloy microscopic images. Symmetry, 10.","DOI":"10.3390\/sym10040107"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"33035","DOI":"10.1117\/1.JEI.28.3.033035","article-title":"Aluminum alloy microstructural segmentation method based on simple noniterative clustering and adaptive density-based spatial clustering of applications with noise","volume":"28","author":"Zhang","year":"2019","journal-title":"J. Electron. Imaging"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1016\/j.matdes.2017.12.049","article-title":"New methods for automatic quantification of microstructural features using digital image processing","volume":"141","author":"Campbell","year":"2018","journal-title":"Mater. Design"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"012011","DOI":"10.1088\/1757-899X\/179\/1\/012011","article-title":"Automated microstructural analysis of titanium alloys using digital image processing","volume":"179","author":"Campbell","year":"2017","journal-title":"IOP Conf. Ser. Mater. Sci. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"25001","DOI":"10.1088\/2051-672X\/aab73b","article-title":"Algorithm Based on Regional Separation for Automatic Grain Boundary Extraction Using Improved Mean Shift Method","volume":"6","author":"Zhenying","year":"2018","journal-title":"Surf. Topogr. Metrol. Prop."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1016\/S0924-0136(01)01057-3","article-title":"Evaluating creep in metals by grain boundary extraction using directional wavelets and mathematical morphology","volume":"117","author":"Journaux","year":"2001","journal-title":"J. Mater. Process. Technol."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1109\/TPAMI.2016.2598339","article-title":"Deep Visual-Semantic Alignments for Generating Image Descriptions","volume":"39","author":"Karpathy","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1465","DOI":"10.1109\/TIP.2011.2175739","article-title":"Rotation-Invariant Image and Video Description with Local Binary Pattern Features","volume":"21","author":"Zhao","year":"2011","journal-title":"IEEE Trans. Image Process."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2891","DOI":"10.1109\/TPAMI.2012.162","article-title":"BabyTalk: Understanding and Generating Simple Image Descriptions","volume":"35","author":"Kulkarni","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C.L., and Parikh, D. (2015). Vqa: Visual question answering. Int. J. Comput. Vis.","DOI":"10.1109\/ICCV.2015.279"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.cviu.2017.05.001","article-title":"Visual Question Answering: A Survey of Methods and Datasets","volume":"163","author":"Wu","year":"2017","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_37","unstructured":"Kazemi, V., and Elqursh, A. (2017). Show, Ask, Attend, and Answer: A Strong Baseline for Visual Question Answering. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep Residual Learning for Image Recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kwan, C., Chou, B., Yang, J., Rangamani, A., Tran, T., Zhang, J., and Etienne-Cummings, R. (2019). Deep Learning-Based Target Tracking and Classification for Low Quality Videos Using Coded Aperture Cameras. Sensors, 19.","DOI":"10.3390\/s19173702"},{"key":"ref_40","first-page":"167","article-title":"Deep Learning Based Target Tracking and Classification for Infrared Videos Using Compressive","volume":"10","author":"Kwan","year":"2019","journal-title":"J. Signal Inf. Process."},{"key":"ref_41","unstructured":"Gers, F. (2001). Long Short-Term Memory in Recurrent Neural Networks. [Ph.D. Thesis, Swiss Federal Institute of Technology]."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Joulin, A., Grave, E., Bojanowski, P., and Mikolov, T. (2017). Bag of Tricks for Efficient Text Classification. European Association of Computational Linguistics (EACL), Association for Computational Linguistics.","DOI":"10.18653\/v1\/E17-2068"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014). Convolutional Neural Networks for Sentence Classification. arXiv.","DOI":"10.3115\/v1\/D14-1181"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/12\/5\/771\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:26:06Z","timestamp":1760174766000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/12\/5\/771"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,6]]},"references-count":43,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2020,5]]}},"alternative-id":["sym12050771"],"URL":"https:\/\/doi.org\/10.3390\/sym12050771","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,5,6]]}}}