{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T15:33:02Z","timestamp":1750174382414},"reference-count":25,"publisher":"Cambridge University Press (CUP)","issue":"3","license":[{"start":{"date-parts":[[2021,5,5]],"date-time":"2021-05-05T00:00:00Z","timestamp":1620172800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":["cambridge.org"],"crossmark-restriction":true},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2022,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Song lyrics contain repeated patterns that have been proven to facilitate automated lyrics segmentation, with the final goal of detecting the building blocks (e.g., chorus, verse) of a song text. Our contribution in this article is twofold. First, we introduce a convolutional neural network (CNN)-based model that learns to segment the lyrics based on their repetitive text structure. We experiment with novel features to reveal different kinds of repetitions in the lyrics, for instance based on phonetical and syntactical properties. Second, using a novel corpus where the song text is synchronized to the audio of the song, we show that the text and audio modalities capture complementary structure of the lyrics and that combining both is beneficial for lyrics segmentation performance. For the purely text-based lyrics segmentation on a dataset of 103k lyrics, we achieve an F-score of 67.4%, improving on the state of the art (59.2% F-score). On the synchronized text\u2013audio dataset of 4.8k songs, we show that the additional audio features improve segmentation performance to 75.3% F-score, significantly outperforming the purely text-based approaches.<\/jats:p>","DOI":"10.1017\/s1351324921000024","type":"journal-article","created":{"date-parts":[[2021,5,5]],"date-time":"2021-05-05T07:26:36Z","timestamp":1620199596000},"page":"317-336","update-policy":"http:\/\/dx.doi.org\/10.1017\/policypage","source":"Crossref","is-referenced-by-count":3,"title":["Lyrics segmentation via bimodal text\u2013audio representation"],"prefix":"10.1017","volume":"28","author":[{"given":"Michael","family":"Fell","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yaroslav","family":"Nechaev","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gabriel","family":"Meseguer-Brocal","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elena","family":"Cabrio","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fabien","family":"Gandon","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Geoffroy","family":"Peeters","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2021,5,5]]},"reference":[{"key":"S1351324921000024_ref1","doi-asserted-by":"publisher","DOI":"10.1109\/SMAP.2013.15"},{"key":"S1351324921000024_ref11","volume-title":"Deep Learning","author":"Goodfellow","year":"2016"},{"key":"S1351324921000024_ref8","unstructured":"Fell, M. , Nechaev, Y. , Cabrio, E. and Gandon, F. (2018). Lyrics segmentation: textual macrostructure detection using convolutions. In Proceedings of the 27th International Conference on Computational Linguistics, pp. 2044\u20132054."},{"key":"S1351324921000024_ref23","doi-asserted-by":"publisher","DOI":"10.1017\/S0261143000001227"},{"key":"S1351324921000024_ref7","author":"Fell","year":"2014"},{"key":"S1351324921000024_ref9","first-page":"452","volume":"1","author":"Foote","year":"2000"},{"key":"S1351324921000024_ref21","first-page":"38","article-title":"The double metaphone search algorithm","volume":"18","author":"Philips","year":"2000","journal-title":"C\/C++ Users Journal"},{"key":"S1351324921000024_ref20","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"S1351324921000024_ref22","first-page":"399","author":"Snoek","year":"2005"},{"key":"S1351324921000024_ref13","doi-asserted-by":"publisher","DOI":"10.1145\/1101149.1101255"},{"key":"S1351324921000024_ref25","unstructured":"Watanabe, K. , Matsubayashi, Y. , Orita, N. , Okazaki, N. , Inui, K. , Fukayama, S. , Nakano, T. , Smith, J. and Goto, M. (2016). Modeling discourse segments in lyrics using repeated patterns. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 1959\u20131969."},{"key":"S1351324921000024_ref2","volume-title":"Interpreting Popular Music","author":"Brackett","year":"1995"},{"key":"S1351324921000024_ref19","first-page":"1833","article-title":"Permutation tests for studying classifier performance","volume":"11","author":"Ojala","year":"2010","journal-title":"Journal of Machine Learning Research"},{"key":"S1351324921000024_ref5","doi-asserted-by":"publisher","DOI":"10.1109\/TASSP.1980.1163420"},{"key":"S1351324921000024_ref14","unstructured":"Mayer, R. and Rauber, A. (2011). Musical genre classification by ensembles of audio and lyrics features. In Proceedings of the 12th International Conference on Music Information Retrieval, pp. 675\u2013680."},{"key":"S1351324921000024_ref3","doi-asserted-by":"publisher","DOI":"10.1109\/ISCAS.2009.5118096"},{"key":"S1351324921000024_ref12","first-page":"707","article-title":"Binary codes capable of correcting deletions, insertions, and reversals","volume":"10","author":"Levenshtein","year":"1966","journal-title":"Soviet Physics Doklady"},{"key":"S1351324921000024_ref4","unstructured":"Cohen-Hadria, A. and Peeters, G. (2017). Music structure boundaries estimation using multiple self-similarity matrices as input depth of convolutional neural networks. In AES International Conference Semantic Audio 2017."},{"key":"S1351324921000024_ref6","unstructured":"Devlin, J. , Chang, M.-W. , Lee, K. and Toutanova, K. (2018). Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805."},{"key":"S1351324921000024_ref24","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems 30","author":"Vaswani","year":"2017"},{"key":"S1351324921000024_ref17","author":"Meseguer-Brocal","year":"2017"},{"key":"S1351324921000024_ref15","doi-asserted-by":"publisher","DOI":"10.25080\/Majora-7b98e3ed-003"},{"key":"S1351324921000024_ref18","unstructured":"Mihalcea, R. and Strapparava, C. (2012). Lyrics, music, and emotions. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, pp. 590\u2013599."},{"key":"S1351324921000024_ref16","unstructured":"Meseguer-Brocal, G. , Cohen-Hadria, A. and Peeters, G. (2018). DALI: a large dataset of synchronized audio, lyrics and notes, automatically created using teacher-student machine learning paradigm. In ISMIR Paris, France."},{"key":"S1351324921000024_ref10","author":"Fujishima","year":"1999"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324921000024","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,25]],"date-time":"2022-04-25T12:32:53Z","timestamp":1650889973000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324921000024\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,5]]},"references-count":25,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,5]]}},"alternative-id":["S1351324921000024"],"URL":"https:\/\/doi.org\/10.1017\/s1351324921000024","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"value":"1351-3249","type":"print"},{"value":"1469-8110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,5]]},"assertion":[{"value":"\u00a9 The Author(s), 2021. Published by Cambridge University Press","name":"copyright","label":"Copyright","group":{"name":"copyright_and_licensing","label":"Copyright and Licensing"}}]}}