{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T09:13:38Z","timestamp":1775639618543,"version":"3.50.1"},"reference-count":30,"publisher":"PeerJ","license":[{"start":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T00:00:00Z","timestamp":1775606400000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:p>\n                    With the continuous growth of intelligent music generation and the increasing demand for human\u2013computer interaction, realizing music composition driven by natural language descriptions has become a significant research direction in this field. However, existing approaches still exhibit limitations in semantic alignment between text and music, audio quality of generation, and controllability, making it difficult to balance high fidelity with semantic consistency. To address this issue, this study proposes a text and music-based generation net (TMG-Net) framework that integrates Transformer, Audio Transformer, contrastive learning, and diffusion-based generation for text-driven music generation tasks. The framework employs a Transformer to capture the semantic features of text and time\u2013frequency information of audio, and achieves cross-modal alignment through contrastive learning. On this basis, a conditional diffusion model is introduced to generate Mel-spectrograms, which are subsequently reconstructed into high-fidelity music\n                    <jats:italic>via<\/jats:italic>\n                    a vocoder, thereby enhancing both the naturalness and semantic consistency of the generated music. Experiments conducted on the MusicCaps and Song Describer Dataset public benchmarks demonstrate that TMG-Net significantly outperforms representative methods such as MuseGAN, Restyle-MusicVAE, and Mustango across three key metrics\u2014Fr\u00e9chet Audio Distance (FAD), Contrastive Language-Audio Pretraining (CLAP) score, and Recall at 10 (R@10)\u2014while approaching the performance of MusicLLM. These results indicate that TMG-Net can effectively align with textual semantics while ensuring audio quality, offering a novel technological pathway and application potential for intelligent music creation and interactive performance.\n                  <\/jats:p>","DOI":"10.7717\/peerj-cs.3749","type":"journal-article","created":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T08:37:53Z","timestamp":1775637473000},"page":"e3749","source":"Crossref","is-referenced-by-count":0,"title":["Cross-modal deep collaborative networks for intelligent music composition and interactive performance"],"prefix":"10.7717","volume":"12","author":[{"given":"Junyu","family":"Li","sequence":"first","affiliation":[{"name":"NanJing XiaoZhuang University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yang","family":"Yu","sequence":"additional","affiliation":[{"name":"Nanjing Foreign Language School Fangshan Campus, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yiyan","family":"Shao","sequence":"additional","affiliation":[{"name":"Nanjing Normal University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8310-6251","authenticated-orcid":true,"given":"Adeel Ashraf","family":"Cheema","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence and Data Science, National University of Computer and Emerging Sciences, Islamabad, Islamabad, Pakistan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"4443","published-online":{"date-parts":[[2026,4,8]]},"reference":[{"key":"10.7717\/peerj-cs.3749\/ref-1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2301.11325","article-title":"MusicLM: generating music from text","author":"Agostinelli","year":"2023"},{"key":"10.7717\/peerj-cs.3749\/ref-2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2409.09214","article-title":"Seed-music: a unified framework for high quality and controlled music generation","author":"Bai","year":"2024"},{"issue":"1","key":"10.7717\/peerj-cs.3749\/ref-3","doi-asserted-by":"publisher","first-page":"118190","DOI":"10.1016\/j.eswa.2022.118190","article-title":"A systematic review of artificial intelligence-based music generation: scope, applications, and future trends","volume":"209","author":"Civit","year":"2022","journal-title":"Expert Systems with Applications"},{"key":"10.7717\/peerj-cs.3749\/ref-4","doi-asserted-by":"publisher","first-page":"110743","DOI":"10.1016\/j.dib.2024.110743","article-title":"SunoCaps: a novel dataset of text-prompt based AI-generated music with emotion annotations","volume":"55","author":"Civit","year":"2024","journal-title":"Data in Brief"},{"issue":"1","key":"10.7717\/peerj-cs.3749\/ref-5","doi-asserted-by":"publisher","first-page":"9096207","DOI":"10.1155\/2022\/9096207","article-title":"The auxiliary role of artificial intelligence in the intelligent creation and conducting of symphony","volume":"2022","author":"Dong","year":"2022","journal-title":"Advances in Multimedia"},{"issue":"1","key":"10.7717\/peerj-cs.3749\/ref-6","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1609\/aaai.v32i1.11312","article-title":"MuseGAN: multi-track sequential generative adversarial networks for symbolic music generation and accompaniment","volume":"32","author":"Dong","year":"2018","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"10.7717\/peerj-cs.3749\/ref-7","doi-asserted-by":"publisher","first-page":"7616","DOI":"10.48550\/arXiv.2202.09729","article-title":"It\u2019s raw! Audio generation with state-space models","author":"Goel","year":"2022"},{"issue":"6","key":"10.7717\/peerj-cs.3749\/ref-8","doi-asserted-by":"publisher","first-page":"3625","DOI":"10.1007\/s00530-023-01144-4","article-title":"Audio-text retrieval based on contrastive learning and collaborative attention mechanism","volume":"29","author":"Hu","year":"2023","journal-title":"Multimedia Systems"},{"issue":"1","key":"10.7717\/peerj-cs.3749\/ref-9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3597493","article-title":"A survey on deep learning for symbolic music generation: representations, algorithms, evaluations, and challenges","volume":"56","author":"Ji","year":"2023","journal-title":"ACM Computing Surveys"},{"key":"10.7717\/peerj-cs.3749\/ref-10","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2407.15060","article-title":"MusiConGen: rhythm and chord control for transformer-based text-to-music generation","author":"Lan","year":"2024"},{"key":"10.7717\/peerj-cs.3749\/ref-11","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2205.05448","article-title":"Symphony generation with permutation invariant language model","author":"Liu","year":"2022"},{"key":"10.7717\/peerj-cs.3749\/ref-12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-3-031-73195-2_1","article-title":"SmartControl: enhancing ControlNet for handling rough visual conditions","author":"Liu","year":"2024"},{"key":"10.7717\/peerj-cs.3749\/ref-13","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2311.10057","article-title":"The Song Describer Dataset: a corpus of audio captions for music-and-language evaluation","author":"Manco","year":"2023"},{"key":"10.7717\/peerj-cs.3749\/ref-14","doi-asserted-by":"publisher","first-page":"430","DOI":"10.1109\/SP61157.2025.00085","article-title":"Harmonycloak: making music unlearnable for generative AI","author":"Meerza","year":"2025"},{"key":"10.7717\/peerj-cs.3749\/ref-15","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.naacl-long.459","article-title":"MustanGO: toward controllable text-to-music generation","author":"Melechovsky","year":"2023"},{"key":"10.7717\/peerj-cs.3749\/ref-16","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1611.09904","article-title":"C-RNN-GAN: continuous recurrent neural networks with adversarial training","author":"Mogren","year":"2016"},{"key":"10.7717\/peerj-cs.3749\/ref-17","doi-asserted-by":"publisher","first-page":"17031","DOI":"10.1109\/access.2023.3244620","article-title":"Music deep learning: deep learning methods for music signal processing\u2014a review of the state-of-the-art","volume":"11","author":"Moysis","year":"2023","journal-title":"IEEE Access"},{"issue":"1","key":"10.7717\/peerj-cs.3749\/ref-18","doi-asserted-by":"publisher","first-page":"408","DOI":"10.1609\/aaai.v35i1.16117","article-title":"Symbolic music generation with transformer-GANs","volume":"35","author":"Muhamed","year":"2021","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"10.7717\/peerj-cs.3749\/ref-19","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2503.01183","article-title":"DiffRhythm: blazingly fast and embarrassingly simple end-to-end full-length song generation with latent diffusion","author":"Ning","year":"2025"},{"key":"10.7717\/peerj-cs.3749\/ref-20","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1145\/3511047.3536412","article-title":"ReStyle-MusicVAE: enhancing user control of deep generative music models with expert labeled anchors","author":"Prvulovic","year":"2022"},{"key":"10.7717\/peerj-cs.3749\/ref-21","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1007\/978-3-031-29956-8_17","article-title":"GTR-CTRL: instrument and genre conditioning for guitar-focused music generation with transformers","author":"Sarmento","year":"2023"},{"issue":"15","key":"10.7717\/peerj-cs.3749\/ref-22","doi-asserted-by":"publisher","first-page":"8504","DOI":"10.3390\/app15158504","article-title":"Enhancing MusicGen with prompt tuning","volume":"15","author":"Shin","year":"2025","journal-title":"Applied Sciences"},{"key":"10.7717\/peerj-cs.3749\/ref-23","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1609.03499","article-title":"WaveNet: a generative model for raw audio","author":"Van Den Oord","year":"2016"},{"issue":"8","key":"10.7717\/peerj-cs.3749\/ref-24","doi-asserted-by":"publisher","first-page":"656","DOI":"10.3390\/info16080656","article-title":"From tools to creators: a review on the development and application of artificial intelligence music generation","volume":"16","author":"Wei","year":"2025","journal-title":"Information"},{"key":"10.7717\/peerj-cs.3749\/ref-25","doi-asserted-by":"publisher","first-page":"2692","DOI":"10.1109\/taslp.2024.3399026","article-title":"Music ControlNet: multiple time-varying controls for music generation","volume":"32","author":"Wu","year":"2024","journal-title":"IEEE\/ACM Transactions on Audio, Speech, and Language Processing"},{"key":"10.7717\/peerj-cs.3749\/ref-26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2409.09256","article-title":"Audio-text retrieval with transformer-based hierarchical alignment and disentangled cross-modal representation","author":"Xin","year":"2024"},{"issue":"2","key":"10.7717\/peerj-cs.3749\/ref-27","doi-asserted-by":"publisher","first-page":"126419","DOI":"10.1016\/j.energy.2022.126419","article-title":"A dual-scale deep learning model based on ELM-BiLSTM and improved reptile search algorithm for wind power prediction","volume":"266","author":"Xiong","year":"2023","journal-title":"Energy"},{"issue":"4","key":"10.7717\/peerj-cs.3749\/ref-28","doi-asserted-by":"publisher","first-page":"3191","DOI":"10.1007\/s00521-022-07863-5","article-title":"Conditional hybrid GAN for melody generation from lyrics","volume":"35","author":"Yu","year":"2023","journal-title":"Neural Computing and Applications"},{"issue":"10","key":"10.7717\/peerj-cs.3749\/ref-29","doi-asserted-by":"publisher","first-page":"101768","DOI":"10.1016\/j.tsc.2025.101768","article-title":"Compositional tools based on artificial intelligence for choral artistic education: enhancing creative skills in choral arrangements","volume":"56","author":"Zhang","year":"2025","journal-title":"Thinking Skills and Creativity"},{"key":"10.7717\/peerj-cs.3749\/ref-30","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1007\/978-3-031-19836-6_11","article-title":"Quantized GAN for complex music generation from dance videos","author":"Zhu","year":"2022"}],"container-title":["PeerJ Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/peerj.com\/articles\/cs-3749.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3749.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3749.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3749.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T08:37:57Z","timestamp":1775637477000},"score":1,"resource":{"primary":{"URL":"https:\/\/peerj.com\/articles\/cs-3749"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,8]]},"references-count":30,"alternative-id":["10.7717\/peerj-cs.3749"],"URL":"https:\/\/doi.org\/10.7717\/peerj-cs.3749","archive":["CLOCKSS","LOCKSS","Portico"],"relation":{},"ISSN":["2376-5992"],"issn-type":[{"value":"2376-5992","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,8]]},"article-number":"e3749"}}