{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T16:12:21Z","timestamp":1781021541245,"version":"3.54.1"},"reference-count":43,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,5,15]],"date-time":"2022-05-15T00:00:00Z","timestamp":1652572800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Key Research and Development Plan of Shanxi Province","award":["201903D111002"],"award-info":[{"award-number":["201903D111002"]}]},{"name":"Key Research and Development Plan of Shanxi Province","award":["2021Y607"],"award-info":[{"award-number":["2021Y607"]}]},{"name":"Key Research and Development Plan of Shanxi Province","award":["JCKY2021210B073"],"award-info":[{"award-number":["JCKY2021210B073"]}]},{"name":"Key Research and Development Plan of Shanxi Province","award":["202102010101002"],"award-info":[{"award-number":["202102010101002"]}]},{"name":"Key Research and Development Plan of Shanxi Province","award":["201901D211242"],"award-info":[{"award-number":["201901D211242"]}]},{"name":"Shanxi Province Graduate Student Innovation Project","award":["201903D111002"],"award-info":[{"award-number":["201903D111002"]}]},{"name":"Shanxi Province Graduate Student Innovation Project","award":["2021Y607"],"award-info":[{"award-number":["2021Y607"]}]},{"name":"Shanxi Province Graduate Student Innovation Project","award":["JCKY2021210B073"],"award-info":[{"award-number":["JCKY2021210B073"]}]},{"name":"Shanxi Province Graduate Student Innovation Project","award":["202102010101002"],"award-info":[{"award-number":["202102010101002"]}]},{"name":"Shanxi Province Graduate Student Innovation Project","award":["201901D211242"],"award-info":[{"award-number":["201901D211242"]}]},{"name":"National Basic Research Program","award":["201903D111002"],"award-info":[{"award-number":["201903D111002"]}]},{"name":"National Basic Research Program","award":["2021Y607"],"award-info":[{"award-number":["2021Y607"]}]},{"name":"National Basic Research Program","award":["JCKY2021210B073"],"award-info":[{"award-number":["JCKY2021210B073"]}]},{"name":"National Basic Research Program","award":["202102010101002"],"award-info":[{"award-number":["202102010101002"]}]},{"name":"National Basic Research Program","award":["201901D211242"],"award-info":[{"award-number":["201901D211242"]}]},{"name":"Key Research and Development Plan of Shanxi Province","award":["201903D111002"],"award-info":[{"award-number":["201903D111002"]}]},{"name":"Key Research and Development Plan of Shanxi Province","award":["2021Y607"],"award-info":[{"award-number":["2021Y607"]}]},{"name":"Key Research and Development Plan of Shanxi Province","award":["JCKY2021210B073"],"award-info":[{"award-number":["JCKY2021210B073"]}]},{"name":"Key Research and Development Plan of Shanxi Province","award":["202102010101002"],"award-info":[{"award-number":["202102010101002"]}]},{"name":"Key Research and Development Plan of Shanxi Province","award":["201901D211242"],"award-info":[{"award-number":["201901D211242"]}]},{"name":"Shanxi Province Science Foundation for Youths","award":["201903D111002"],"award-info":[{"award-number":["201903D111002"]}]},{"name":"Shanxi Province Science Foundation for Youths","award":["2021Y607"],"award-info":[{"award-number":["2021Y607"]}]},{"name":"Shanxi Province Science Foundation for Youths","award":["JCKY2021210B073"],"award-info":[{"award-number":["JCKY2021210B073"]}]},{"name":"Shanxi Province Science Foundation for Youths","award":["202102010101002"],"award-info":[{"award-number":["202102010101002"]}]},{"name":"Shanxi Province Science Foundation for Youths","award":["201901D211242"],"award-info":[{"award-number":["201901D211242"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>AMC (automatic modulation classification) plays a vital role in spectrum monitoring and electromagnetic abnormal signal detection. Up to now, few studies have focused on the complementarity between features of different modalities and the importance of the feature fusion mechanism in the AMC method. This paper proposes a dual-modal feature fusion convolutional neural network (DMFF-CNN) for AMC to use the complementarity between different modal features fully. DMFF-CNN uses the gram angular field (GAF) image coding and intelligence quotient (IQ) data combined with CNN. Firstly, the original signal is converted into images by GAF, and the GAF images are used as the input of ResNet50. Secondly, it is converted into IQ data and as the complex value network (CV-CNN) input to extract features. Furthermore, a dual-modal feature fusion mechanism (DMFF) is proposed to fuse the dual-modal features extracted by GAF-ResNet50 and CV-CNN. The fusion feature is used as the input of DMFF-CNN for model training to achieve AMC of multi-type signals. In the evaluation stage, the advantages of the DMFF mechanism proposed in this paper and the accuracy improvement compared with other feature fusion algorithms are discussed. The experiment shows that our method performs better than others, including some state-of-the-art methods, and has superior robustness at a low signal-to-noise ratio (SNR), and the average classification accuracy of the dataset signals reaches 92.1%. The DMFF-CNN proposed in this paper provides a new path for the AMC field.<\/jats:p>","DOI":"10.3390\/e24050700","type":"journal-article","created":{"date-parts":[[2022,5,15]],"date-time":"2022-05-15T09:48:22Z","timestamp":1652608102000},"page":"700","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Electromagnetic Modulation Signal Classification Using Dual-Modal Feature Fusion CNN"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8715-3939","authenticated-orcid":false,"given":"Jiansheng","family":"Bai","sequence":"first","affiliation":[{"name":"State Key Lab for Electronic Testing Technology, North University of China, Taiyuan 030051, China"},{"name":"School of Information and Communication Engineering, North University of China, Taiyuan 030051, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinjie","family":"Yao","sequence":"additional","affiliation":[{"name":"State Key Lab for Electronic Testing Technology, North University of China, Taiyuan 030051, China"},{"name":"School of Information and Communication Engineering, North University of China, Taiyuan 030051, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juncheng","family":"Qi","sequence":"additional","affiliation":[{"name":"State Key Lab for Electronic Testing Technology, North University of China, Taiyuan 030051, China"},{"name":"School of Information and Communication Engineering, North University of China, Taiyuan 030051, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liming","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Lab for Electronic Testing Technology, North University of China, Taiyuan 030051, China"},{"name":"School of Information and Communication Engineering, North University of China, Taiyuan 030051, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1049\/iet-com:20050176","article-title":"Survey of automatic modulation classification techniques: Classical approaches and new trends","volume":"1","author":"Dobre","year":"2007","journal-title":"IET Commun."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3079","DOI":"10.1109\/JSEN.2017.2688240","article-title":"Optimized Distributed Automatic Modulation Classification in Wireless Sensor Networks Using Information Theoretic Measures","volume":"17","author":"Hakimi","year":"2017","journal-title":"IEEE Sens. J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"10760","DOI":"10.1109\/TVT.2018.2868698","article-title":"Automatic Modulation Classification: A Deep Learning Enabled Approach","volume":"67","author":"Meng","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1109\/TSMCC.2010.2076347","article-title":"Likelihood-Ratio Approaches to Automatic Modulation Classification","volume":"41","author":"Xu","year":"2010","journal-title":"IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.)"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1109\/TWC.2016.2623716","article-title":"Low complexity automatic modulation classification based on order-statistics","volume":"16","author":"Han","year":"2017","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/LCOMM.2011.112311.112006","article-title":"Cyclostationarity-Based Robust Algorithms for QAM Signal Identification","volume":"16","author":"Dobre","year":"2011","journal-title":"IEEE Commun. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1109\/TIFS.2011.2159000","article-title":"Modulation Recognition in Continuous Phase Modulation Using Approximate Entropy","volume":"6","author":"Pawar","year":"2011","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1986","DOI":"10.1049\/el.2014.2700","article-title":"Modulation recognition of MAPSK signals using template matching","volume":"50","author":"Deng","year":"2014","journal-title":"Electron. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1049\/iet-com.2015.1124","article-title":"On classifiers for blind feature-based automatic modulation classification over multiple-input\u2013multiple-output channels","volume":"10","author":"Kharbech","year":"2016","journal-title":"IET Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1109\/LWC.2017.2764078","article-title":"Automatic Modulation Recognition for Phase Shift Keying Signals With Compressive Measurements","volume":"7","author":"Xie","year":"2017","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2595","DOI":"10.1109\/COMST.2018.2846401","article-title":"Deep Learning for Intelligent Wireless Networks: A Comprehensive Survey","volume":"20","author":"Mao","year":"2018","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"58113","DOI":"10.1109\/ACCESS.2019.2913945","article-title":"Deep Neural Network Compression Technique Towards Efficient Digital Signal Modulation Recognition in Edge Device","volume":"7","author":"Tu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"114002","DOI":"10.1016\/j.enconman.2021.114002","article-title":"A deep learning-based evolutionary model for short-term wind speed forecasting: A case study of the Lillgrund offshore wind farm","volume":"236","author":"Neshat","year":"2021","journal-title":"Energy Convers. Manag."},{"key":"ref_14","first-page":"694","article-title":"Time series classification of cryptocurrency price trend based on a recurrent LSTM neural network","volume":"15","author":"Kwon","year":"2019","journal-title":"J. Inf. Processing Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13321-017-0226-y","article-title":"Deep-learning: Investigating deep neural networks hyper-parameters and comparison of performance to shallow methods for modeling bioactivity data","volume":"9","author":"Koutsoukas","year":"2017","journal-title":"J. Cheminform."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"14155","DOI":"10.1007\/s00500-021-06222-1","article-title":"Deep learning-based gas identification and quantification with auto-tuning of hyper-parameters","volume":"25","author":"Pareek","year":"2021","journal-title":"Soft Comput."},{"key":"ref_17","unstructured":"Peng, S., Sun, S., and Yao, Y.D. (2021). A Survey of Modulation Classification Using Deep Learning: Signal Representation and Data Preprocessing. IEEE Trans. Neural Netw. Learn. Syst., 1\u201319."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1626","DOI":"10.1109\/LSP.2017.2752459","article-title":"Automatic Modulation Classification Using Deep Learning Based on Sparse Autoencoders With Nonnegativity Constraints","volume":"24","author":"Ali","year":"2017","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"63760","DOI":"10.1109\/ACCESS.2019.2916833","article-title":"Deep Learning in Digital Modulation Recognition Using High Order Cumulants","volume":"7","author":"Xie","year":"2019","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lee, S.H., Kim, K.-Y., Kim, J.H., and Shin, Y. (2019, January 11\u201313). Effective Feature-Based Automatic Modulation Classification Method Using DNN Algorithm. Proceedings of the 2019 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Okinawa, Japan.","DOI":"10.1109\/ICAIIC.2019.8669036"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1529","DOI":"10.1109\/LCOMM.2019.2927348","article-title":"Classification of Spectrally Efficient Constant Envelope Modulations Based on Radial Basis Function Network and Deep Learning","volume":"23","author":"Shah","year":"2019","journal-title":"IEEE Commun. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"104591","DOI":"10.1109\/ACCESS.2019.2932266","article-title":"Particle Swarm Optimization-Based Deep Neural Network for Digital Modulation Recognition","volume":"7","author":"Shi","year":"2019","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1275","DOI":"10.1109\/LCOMM.2020.2980840","article-title":"Automatic Modulation Classification Based on Constellation Density Using Deep Learning","volume":"24","author":"Kumar","year":"2020","journal-title":"IEEE Commun. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1667","DOI":"10.1109\/LPT.2017.2742553","article-title":"Modulation Format Recognition and OSNR Estimation Using CNN-Based Deep Learning","volume":"29","author":"Wang","year":"2017","journal-title":"IEEE Photon-Technol. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ma, J., Lin, S.-C., Gao, H., and Qiu, T. (2019, January 20\u201324). Automatic Modulation Classification Under Non-Gaussian Noise: A Deep Residual Learning Approach. Proceedings of the ICC 2019\u20142019 IEEE International Conference on Communications (ICC), Shanghai, China.","DOI":"10.1109\/ICC.2019.8761426"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Li, Y., Shao, G., and Wang, B. (2019, January 24\u201326). Automatic Modulation Classification Based on Bispectrum and CNN. Proceedings of the 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing, China.","DOI":"10.1109\/ITAIC.2019.8785692"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1498","DOI":"10.1109\/ACCESS.2019.2960775","article-title":"Data Augmentation for Deep Learning-Based Radio Modulation Classification","volume":"8","author":"Huang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1038","DOI":"10.1109\/LCOMM.2020.2970922","article-title":"CNN-Based Automatic Modulation Classification for Beyond 5G Communications","volume":"24","author":"Hermawan","year":"2020","journal-title":"IEEE Commun. Lett."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Mossad, O.S., ElNainay, M., and Torki, M. (2019, January 24\u201328). Deep Convolutional Neural Network with Multi-Task Learning Scheme for Modulations Recognition. Proceedings of the 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC), Tangier, Morocco.","DOI":"10.1109\/IWCMC.2019.8766665"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"895","DOI":"10.1049\/el.2019.1789","article-title":"Convolutional neural network and multi-feature fusion for automatic modulation classification","volume":"55","author":"Wu","year":"2019","journal-title":"Electron. Lett."},{"key":"ref_31","unstructured":"Mao, Y., Dong, Y.-Y., Sun, T., Rao, X., and Dong, C.-X. (2021). Attentive Siamese Networks for Automatic Modulation Classification Based on Multitiming Constellation Diagrams. IEEE Trans. Neural Netw. Learn. Syst., 1\u201315."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1049\/iet-rsn.2018.5549","article-title":"Automatic modulation classification based on joint feature map and convolutional neural network","volume":"13","author":"Wang","year":"2019","journal-title":"IET Radar, Sonar Navig."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1109\/TSIPN.2019.2900201","article-title":"Automatic Modulation Classification Using Convolutional Neural Network With Features Fusion of SPWVD and BJD","volume":"5","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Signal Inf. Process. Over Netw."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhang, M., Zeng, Y., Han, Z., and Gong, Y. (2018, January 25\u201328). Automatic Modulation Recognition Using Deep Learning Architectures. Proceedings of the 2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Kalamata, Greece.","DOI":"10.1109\/SPAWC.2018.8446021"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"4074","DOI":"10.1109\/TVT.2019.2900460","article-title":"Data-Driven Deep Learning for Automatic Modulation Recognition in Cognitive Radios","volume":"68","author":"Wang","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Hiremath, S.M., Behura, S., Kedia, S., Deshmukh, S., and Patra, S.K. (2019, January 20\u201323). Deep Learning-Based Modulation Classification Using Time and Stockwell Domain Channeling. Proceedings of the 2019 National Conference on Communications (NCC), Bangalore, India.","DOI":"10.1109\/NCC.2019.8732258"},{"key":"ref_37","unstructured":"Wang, Z., and Oates, T. (2015, January 25\u201330). Encoding Time Series as Images for Visual Inspection and Classification Using Tiled Convolutional Neural Networks. Proceedings of the Workshops at the Twenty-Ninth Aaai Conference on Artificial Intelligence, Austin, TX, USA."},{"key":"ref_38","unstructured":"Wang, Z., and Oates, T. (2015, January 25\u201331). Imaging time-series to improve classification and imputation. Proceedings of the 24th International Conference on Artificial Intelligence, Menlo Park, CA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.neucom.2018.04.068","article-title":"The optimally designed dynamic memory networks for targeted sentiment classification","volume":"309","author":"Zhang","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zhang, H., Yu, L., Chen, Y., and Wei, Y. (2021). Fast Complex-Valued CNN for Radar Jamming Signal Recognition. Remote Sens., 13.","DOI":"10.3390\/rs13152867"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"42841","DOI":"10.1109\/ACCESS.2020.2978094","article-title":"Deep Learning-Based Automatic Modulation Recognition Method in the Presence of Phase Offset","volume":"8","author":"Shi","year":"2020","journal-title":"IEEE Access"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1757","DOI":"10.1109\/LWC.2021.3078878","article-title":"Deep Learning Based Modulation Recognition With Multi-Cue Fusion","volume":"10","author":"Wang","year":"2021","journal-title":"IEEE Wirel. Commun. Lett."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/5\/700\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:11:02Z","timestamp":1760137862000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/5\/700"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,15]]},"references-count":43,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["e24050700"],"URL":"https:\/\/doi.org\/10.3390\/e24050700","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,15]]}}}