{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T10:40:02Z","timestamp":1780915202927,"version":"3.54.1"},"reference-count":33,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,5,13]],"date-time":"2025-05-13T00:00:00Z","timestamp":1747094400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>The number of music of different genres is increasing year by year, and manual classification is costly and requires professionals in the field of music to manually design features, some of which lack the generality of music genre classification. Deep learning has had a large number of scientific research results in the field of music classification, but the existing deep learning methods still have the problems of insufficient extraction of music feature information, low accuracy rate of music genres, loss of time series information, and slow training. To address the problem that different music durations affect the accuracy of music genre classification, we form a Log Mel spectrum with music audio data of different cut durations. After discarding incomplete audio, we design data enhancement with different slicing durations and verify its effect on accuracy and training time through comparison experiments. Based on this, the audio signal is divided into frames, windowed and short-time Fourier transformed, and then the Log Mel spectrum is obtained by using the Mel filter and logarithmic compression. Aiming at the problems of loss of time information, insufficient feature extraction, and low classification accuracy in music genre classification, firstly, we propose a Res-Transformer model that fuses the residual network with the Transformer coding layer. The model consists of two branches, the left branch is an improved residual network, which enhances the spectral feature extraction ability and network expression ability and realizes the dimensionality reduction; the right branch uses four Transformer coding layers to extract the time-series information of the Log Mel spectrum. The output vectors of the two branches are spliced and input into the classifier to realize music genre classification. Then, to further improve the classification accuracy of the model, we propose the TS-Resformer model based on the Res-Transformer model, combined with different attention mechanisms, and design the time-frequency attention mechanism, which employs different scales of filters to fully extract the low-level music features from the two dimensions of time and frequency as the input to the time-frequency attention mechanism, respectively. Finally, experiments show that the accuracy of this method is 90.23% on the FMA-small dataset, which is an improvement in classification accuracy compared with the classical model.<\/jats:p>","DOI":"10.3389\/fnbot.2025.1568811","type":"journal-article","created":{"date-parts":[[2025,5,13]],"date-time":"2025-05-13T05:31:08Z","timestamp":1747114268000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["TS-Resformer: a model based on multimodal fusion for the classification of music signals"],"prefix":"10.3389","volume":"19","author":[{"given":"Yilin","family":"Zhang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,5,13]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"188","DOI":"10.14569\/IJACSA.2017.080844","article-title":"Automatic music genres classification using machine learning","volume":"8","author":"Ali","year":"2017","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref2","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1109\/ICSIPA.2009.5478635","article-title":"Enhanced polyphonic music genre classification using high level features","volume-title":"2009 IEEE international conference on signal and image processing applications","author":"Arabi","year":"2009"},{"key":"ref3","first-page":"96","article-title":"A novel approach of automatic music genre classification based on timbrai texture and rhythmic content features. 16th international conference on advanced communication technology","author":"Baniya","year":"2014"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"616","DOI":"10.3390\/fractalfract8110616","article-title":"Optimizing fractional-order convolutional neural networks for groove classification in music using differential evolution","volume":"8","author":"Chen","year":"2024","journal-title":"Fractal Fract."},{"key":"ref5","doi-asserted-by":"crossref","first-page":"2392","DOI":"10.1109\/ICASSP.2017.7952585","article-title":"Convolutional recurrent neural networks for music classification","volume-title":"2017 IEEE international conference on acoustics, speech and signal processing (ICASSP)","author":"Choi","year":"2017"},{"key":"ref6","first-page":"5","volume-title":"Pattern classification","author":"Duda","year":"2012"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"62460","DOI":"10.1109\/ACCESS.2021.3074713","article-title":"Fully automatic model based on se-resnet for bone age assessment","volume":"9","author":"He","year":"2021","journal-title":"IEEE Acc."},{"key":"ref8","author":"Khasgiwala","year":"2021"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1007\/s12539-023-00579-7","article-title":"Classification of glomerular pathology images in children using convolutional neural networks with improved SE-ResNet module","volume":"15","author":"Kong","year":"2023","journal-title":"Interdiscip. Sci.: Comput. Life Sci."},{"key":"ref10","first-page":"55","article-title":"Music genre classification: looking for the perfect network","volume-title":"International conference on computational science","author":"Kostrzewa","year":"2021"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"20157","DOI":"10.1007\/s11042-022-14252-6","article-title":"Music genre classification based on fusing audio and lyric information","volume":"82","author":"Li","year":"2023","journal-title":"Multimed. Tools Appl."},{"key":"ref12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2022\/3022767","article-title":"Music score recognition method based on deep learning","volume":"2022","author":"Lin","year":"2022","journal-title":"Comput. Intell. Neurosci."},{"key":"ref13","doi-asserted-by":"publisher","first-page":"104209","DOI":"10.1016\/j.medengphy.2024.104209","article-title":"A novel diagnosis method combined dual-channel SE-res net with expert features for inter-patient heartbeat classification","volume":"130","author":"Liu","year":"2024","journal-title":"Med. Eng. Phys."},{"key":"ref14","first-page":"11","article-title":"Mel frequency cepstral coefficients for music modeling","volume":"270","author":"Logan","year":"2000","journal-title":"Ismir. Plymouth"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"5017","DOI":"10.1007\/s11042-021-11584-7","article-title":"End-to-end music emotion variation detection using iteratively reconstructed deep features","volume":"81","author":"Orjesek","year":"2022","journal-title":"Multimed. Tools Appl."},{"key":"ref16","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1186\/s13636-023-00278-7","article-title":"AAM: a dataset of artificial audio multitracks for diverse music information retrieval tasks","volume":"2023","author":"Ostermann","year":"2023","journal-title":"Eurasip J. Audio Speech Music Process."},{"key":"ref17","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1109\/CJECE.2020.2970144","article-title":"Neural network music genre classification","volume":"43","author":"Pelchat","year":"2020","journal-title":"Can. J. Electr. Comput. Eng."},{"key":"ref18","doi-asserted-by":"publisher","first-page":"118636","DOI":"10.1016\/j.eswa.2022.118636","article-title":"Holistic approaches to music genre classification using efficient transfer and deep learning techniques","volume":"211","author":"Prabhakar","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref19","author":"Sarkar","year":"2015"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"1659","DOI":"10.1007\/s41870-019-00285-y","article-title":"Music instrument recognition using deep convolutional neural networks","volume":"14","author":"Solanki","year":"2022","journal-title":"Int. J. Inf. Technol."},{"key":"ref21","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1145\/2390848.2390851","article-title":"An analysis of the gtzan music genre dataset. Proceedings of the second international ACM workshop on music information retrieval with user-centered and multimodal strategies","author":"Sturm","year":"2012","journal-title":"Nara"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1109\/TSA.2002.800560","article-title":"Musical genre classification of audio signals","volume":"10","author":"Tzanetakis","year":"2002","journal-title":"IEEE Trans. Speech Audio Proc."},{"key":"ref23","first-page":"21","article-title":"Music genre classification using neural networks with data augmentation","volume":"1","author":"Vaibhavi","year":"2021","journal-title":"J. Innov. Sci. Sustain. Technol"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"22719","DOI":"10.1038\/s41598-023-49869-6","article-title":"A novel hybrid model integrating MFCC and acoustic parameters for voice disorder detection","volume":"13","author":"Verma","year":"2023","journal-title":"Sci. Rep."},{"key":"ref25","doi-asserted-by":"publisher","first-page":"10211","DOI":"10.1007\/s11042-023-16024-2","article-title":"Parallel attention of representation global time-frequency correlation for music genre classification","volume":"83","author":"Wen","year":"2024","journal-title":"Multimed. Tools Appl."},{"key":"ref26","author":"Xi","year":"2004"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"13527","DOI":"10.1007\/s11042-023-15277-1","article-title":"Music genre classification based on res-gated CNN and attention mechanism","volume":"83","author":"Xie","year":"2024","journal-title":"Multimed. Tools Appl."},{"key":"ref28","doi-asserted-by":"publisher","first-page":"19629","DOI":"10.1109\/ACCESS.2020.2968170","article-title":"Parallel recurrent convolutional neural networks-based music genre classification method for mobile devices","volume":"8","author":"Yang","year":"2020","journal-title":"IEEE Access"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"260","DOI":"10.3390\/su17010260","article-title":"Can urban internet development attract labor force? Evidence from Chinese Cities","volume":"17","author":"Zhai","year":"2025","journal-title":"Sustainability"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TNNLS.2024.3437676","article-title":"Convolutional dynamically convergent differential neural network for brain signal classification","author":"Zhang","year":"2024","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref31","doi-asserted-by":"crossref","first-page":"3304","DOI":"10.21437\/Interspeech.2016-1236","article-title":"Improved music genre classification with convolutional neural networks","volume-title":"Interspeech","author":"Zhang","year":"2016"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"107122","DOI":"10.1016\/j.neunet.2025.107122","article-title":"VPT: video portraits transformer for realistic talking face generation","volume":"184","author":"Zhang","year":"2025","journal-title":"Neural Netw."},{"key":"ref33","first-page":"155","article-title":"Music genre classification with transformer classifier. Proceedings of the 2020 4th international conference on digital signal processing","author":"Zhuang","year":"2020","journal-title":"Xiamen IEEE"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2025.1568811\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,13]],"date-time":"2025-05-13T05:31:10Z","timestamp":1747114270000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2025.1568811\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,13]]},"references-count":33,"alternative-id":["10.3389\/fnbot.2025.1568811"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2025.1568811","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,13]]},"article-number":"1568811"}}