{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T11:12:25Z","timestamp":1740136345669,"version":"3.37.3"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2022,12,23]],"date-time":"2022-12-23T00:00:00Z","timestamp":1671753600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,12,23]],"date-time":"2022-12-23T00:00:00Z","timestamp":1671753600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["POCI-01-0145-FEDER-031380"],"award-info":[{"award-number":["POCI-01-0145-FEDER-031380"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006752","name":"Universidade do Porto","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100006752","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Pers Ubiquit Comput"],"published-print":{"date-parts":[[2023,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We advance Mixmash-AIS, a multimodal optimization music mashup creation model for loop recombination at scale. Our motivation is to (1) tackle current scalability limitations in state-of-the-art (brute force) computational mashup models while enforcing the (2) compatibility of audio loops and (3) a pool of diverse mashups that can accommodate user preferences. To this end, we adopt the artificial immune system (AIS) opt-aiNet algorithm to efficiently compute a population of compatible and diverse music mashups from loop recombinations. Optimal mashups result from local minima in a feature space representing harmonic, rhythmic, and spectral musical audio compatibility. We objectively assess the compatibility, diversity, and computational performance of Mixmash-AIS generated mashups compared to a standard genetic algorithm (GA) and a brute force (BF) approach. Furthermore, we conducted a perceptual test to validate the objective evaluation function within Mixmash-AIS in capturing <jats:italic>user enjoyment<\/jats:italic> of the computer-generated loop mashups. Our results show that while the GA stands as the most efficient algorithm, the AIS opt-aiNet outperforms both the GA and BF approaches in terms of compatibility and diversity. Our listening test has shown that Mixmash-AIS objective evaluation function significantly captures the perceptual compatibility of loop mashups (<jats:bold><jats:italic>p<\/jats:italic><\/jats:bold> &lt; .<jats:bold>001<\/jats:bold>).<\/jats:p>","DOI":"10.1007\/s00779-022-01702-z","type":"journal-article","created":{"date-parts":[[2022,12,23]],"date-time":"2022-12-23T07:02:46Z","timestamp":1671778966000},"page":"1793-1809","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Leveraging compatibility and diversity in computer-aided music mashup creation"],"prefix":"10.1007","volume":"27","author":[{"given":"Gon\u00e7alo","family":"Bernardo","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3884-2687","authenticated-orcid":false,"given":"Gilberto","family":"Bernardes","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,12,23]]},"reference":[{"issue":"2","key":"1702_CR1","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1080\/07393180701262685","volume":"24","author":"J Shiga","year":"2007","unstructured":"Shiga J (2007) Copy-and-persist: the logic of mash-up culture. Crit Stud Media Commun 24 (2):93\u2013114. https:\/\/doi.org\/10.1080\/07393180701262685","journal-title":"Crit Stud Media Commun"},{"issue":"4","key":"1702_CR2","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1080\/09298215.2019.1659828","volume":"48","author":"D Keller","year":"2019","unstructured":"Keller D, Schiavoni F, Lazzarini V (2019) Ubiquitous music: perspectives and challenges. J New Music Res 48(4):309\u2013315","journal-title":"J New Music Res"},{"key":"1702_CR3","doi-asserted-by":"publisher","unstructured":"Goto M (2012) Grand challenges in music information research. In: M\u00fcller M, Goto M, Schedl M (eds) Multimodal Music Processing. Dagstuhl Follow-Ups, vol. 3, pp. 217\u2013226. Schloss Dagstuhl\u2013Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany. https:\/\/doi.org\/10.4230\/DFU.Vol3.11041.217","DOI":"10.4230\/DFU.Vol3.11041.217"},{"key":"1702_CR4","unstructured":"Andersen K, Knees P (2016) Conversations with expert users in music retrieval and research challenges for creative MIR. In: Proceedings of the International Society for Music Information Retrieval Conference, pp. 122\u2013128"},{"issue":"12","key":"1702_CR5","doi-asserted-by":"publisher","first-page":"1726","DOI":"10.1109\/TASLP.2014.2347135","volume":"22","author":"MEP Davies","year":"2014","unstructured":"Davies MEP, Hamel P, Yoshii K, Goto M (2014) Automashupper: automatic creation of multi-song music mashups. IEEE\/ACM Transactions on Audio Speech and Language Processing 22(12): 1726\u20131737. https:\/\/doi.org\/10.1109\/TASLP.2014.2347135","journal-title":"IEEE\/ACM Transactions on Audio Speech and Language Processing"},{"key":"1702_CR6","doi-asserted-by":"crossref","unstructured":"Ma\u00e7\u00e3s C, Rodrigues A, Bernardes G, Machado P (2018) Mixmash: a visualisation system for musical mashup creation. In: 2018 22Nd international conference information visualisation (IV), pp. 471\u2013477","DOI":"10.1109\/iV.2018.00088"},{"key":"1702_CR7","unstructured":"Lee C, Lin Y, Yao Z, Lee F, Wu J (2015) Automatic mashup creation by considering both vertical and horizontal mashabilities. In: Proceedings of the International Society for Music Information Retrieval Conference, pp. 399\u2013405"},{"issue":"2","key":"1702_CR8","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1076\/jnmr.30.2.159.7114","volume":"30","author":"M Goto","year":"2002","unstructured":"Goto M (2002) An audio-based real-time beat tracking system for music with or without drum-sounds. J New Music Res 30(2):159\u2013171. https:\/\/doi.org\/10.1076\/jnmr.30.2.159.7114","journal-title":"J New Music Res"},{"key":"1702_CR9","unstructured":"Chen B-Y, Smith JB, Yang Y-H (2020) Neural loop combiner: neural network models for assessing the compatibility of loops. In: Proceedings of the International Society for Music Information Retrieval Conference, pp. 424\u2013431"},{"key":"1702_CR10","doi-asserted-by":"crossref","unstructured":"Huang J, Wang J-C, Smith JBL, Song X, Wang Y (2021) Modeling the compatibility of stem tracks to generate music mashups. arXiv:2103.14208","DOI":"10.1609\/aaai.v35i1.16092"},{"key":"1702_CR11","unstructured":"Wong K-C (2015) Evolutionary multimodal optimization: a short survey. arXiv:1508.00457"},{"key":"1702_CR12","doi-asserted-by":"publisher","unstructured":"De Castro LN, Timmis J (2002) An artificial immune network for multimodal function optimization. In: Proceedings of the 2002 Congress on Evolutionary Computation. CEC\u201902 (Cat. No.02TH8600), vol. 1, pp. 699\u20137041. https:\/\/doi.org\/10.1109\/CEC.2002.1007011","DOI":"10.1109\/CEC.2002.1007011"},{"key":"1702_CR13","doi-asserted-by":"publisher","first-page":"100484","DOI":"10.1016\/j.swevo.2018.12.010","volume":"50","author":"M Caetano","year":"2019","unstructured":"Caetano M, Zacharakis A, Barbancho I, Tard\u00f3n LJ (2019) Leveraging diversity in computer-aided musical orchestration with an artificial immune system for multi-modal optimization. Swarm Evol Comput 50:100484. https:\/\/doi.org\/10.1016\/j.swevo.2018.12.010","journal-title":"Swarm Evol Comput"},{"key":"1702_CR14","doi-asserted-by":"crossref","unstructured":"Abreu J, Caetano M, Penha R (2016) Computer-aided musical orchestration using an artificial immune system. In: Johnson C, Ciesielski V, Correia J, Machado P (eds) Evolutionary and biologically inspired music, Sound, Art and Design, pp 1-16","DOI":"10.1007\/978-3-319-31008-4_1"},{"key":"1702_CR15","unstructured":"Ramires A, Bernardes G, Davies M, Serra X (2020) Tiv.lib: an open-source library for the tonal description of musical audio. In: Proceedings of the 23rd International Conference on Digital Audio Effects, pp 304\u2013309"},{"key":"1702_CR16","unstructured":"Lidy T, Rauber A (2005) Evaluation of feature extractors and psycho-acoustic transformations for music genre classification. In: Proceedings of the International Society for Music Information Retrieval Conference, pp 34\u201341"},{"key":"1702_CR17","doi-asserted-by":"publisher","unstructured":"Davies MEP, Hamel P, Yoshii K, Goto M (2014) Automashupper: automatic creation of multi-song music mashups, vol 22, pp 1726\u20131737, DOI https:\/\/doi.org\/10.1109\/TASLP.2014.2347135","DOI":"10.1109\/TASLP.2014.2347135"},{"key":"1702_CR18","unstructured":"Puckette M (1996) Pure data: another integrated computer music environment. In: Proceedings of the International Computer Music Conference, pp 37\u201341"},{"key":"1702_CR19","doi-asserted-by":"crossref","unstructured":"Bernardo G, Bernardes G (2021) Leveraging compatibility and diversity in computational music mashup creation. In: Audio mostly 2021, pp 248\u2013255","DOI":"10.1145\/3478384.3478424"},{"key":"1702_CR20","unstructured":"Bernardes G, Davies MEP, Guedes C Aramaki M, Davies MEP, Kronland-Martinet R, Ystad S (eds) (2018) Music technology with swing. Springer, Cham"},{"key":"1702_CR21","doi-asserted-by":"crossref","unstructured":"Griffin G, Kim Y, Turnbull D (2010) Beat-sync-mash-coder: a web application for real-time creation of beat-synchronous music mashups. In: Proceedings of the International Conference on Acoustics, Speech, & Signal Processing, pp. 2\u20135","DOI":"10.1109\/ICASSP.2010.5495743"},{"key":"1702_CR22","unstructured":"Fujishima T (1999) Real-time chord recognition of musical sound: a system using common lisp music. In: Proceedings of the International Computer Music Conference, pp 464\u2013 467"},{"issue":"5","key":"1702_CR23","doi-asserted-by":"publisher","first-page":"136","DOI":"10.3390\/app6050123","volume":"6","author":"RB Gebhardt","year":"2016","unstructured":"Gebhardt RB, Davies MEP, Seeber BU (2016) Psychoacoustic approaches for harmonic music mixing. Appl Sci 6(5):136","journal-title":"Appl Sci"},{"key":"1702_CR24","unstructured":"P\u00e9rez Fern\u00e1ndez M (2020) Harmonic compatibility for loops in electronic music. Master\u2019s thesis, Universitat Pompeu Fabra"},{"key":"1702_CR25","doi-asserted-by":"publisher","unstructured":"Harrison P, Pearce M (2020) Simultaneous consonance in music perception and composition, vol 127, pp 216\u2013244. https:\/\/doi.org\/10.1037\/rev0000169","DOI":"10.1037\/rev0000169"},{"key":"1702_CR26","doi-asserted-by":"crossref","unstructured":"St\u00f6ter F-R, Liutkus A, Ito N (2018) The 2018 signal separation evaluation campaign. In: International Conference on Latent Variable Analysis and Signal Separation, pp 293\u2013305. Springer","DOI":"10.1007\/978-3-319-93764-9_28"},{"key":"1702_CR27","unstructured":"Jansson A, Humphrey E, Montecchio N, Bittner R, Kumar A, Weyde T (2017) Singing voice separation with deep u-net convolutional networks. In: Proceedings of International Society for Music Information Retrieval Conference, pp 745\u2013751"},{"key":"1702_CR28","volume-title":"Evolutionary optimization algorithms","author":"D Simon","year":"2013","unstructured":"Simon D (2013) Evolutionary optimization algorithms. Wiley, Hoboken, New Jersey"},{"key":"1702_CR29","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1016\/j.ins.2011.12.016","volume":"194","author":"K-C Wong","year":"2012","unstructured":"Wong K-C, Wu C-H, Mok RKP, Peng C, Zhang Z (2012) Evolutionary multimodal optimization using the principle of locality. Inf Sci 194:138\u2013170. https:\/\/doi.org\/10.1016\/j.ins.2011.12.016","journal-title":"Inf Sci"},{"key":"1702_CR30","unstructured":"Savic D (2002) Single-objective vs. multiobjective optimisation for integrated decision support. In: Proceedings of the First Biennial Meeting of the International Environmental Modelling and Software Society, vol. 1, pp. 7\u201312"},{"key":"1702_CR31","doi-asserted-by":"publisher","first-page":"100543","DOI":"10.1016\/j.swevo.2019.05.012","volume":"50","author":"M Navarro-C\u00e1ceres","year":"2019","unstructured":"Navarro-C\u00e1ceres M, Caetano M, Bernardes G, de Castro LN (2019) Chordais: an assistive system for the generation of chord progressions with an artificial immune system. Swarm Evol Comput 50:100543. https:\/\/doi.org\/10.1016\/j.swevo.2019.05.012","journal-title":"Swarm Evol Comput"},{"key":"1702_CR32","doi-asserted-by":"crossref","unstructured":"Lampropoulos AS, Sotiropoulos DN, Tsihrintzis GA (2010) A music recommender based on artificial immune systems. In: Intelligent interactive multimedia systems and services. Springer, Berlin Heidelberg, pp 167\u2013179","DOI":"10.1007\/978-3-642-14619-0_17"},{"key":"1702_CR33","unstructured":"Gallagher M (2009) The music tech dictionary: a glossary of audio-related terms and technologies. Cengage Learning Ptr, Boston, MA USA"},{"issue":"1","key":"1702_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1525\/mp.2001.19.1.1","volume":"19","author":"D Huron","year":"2001","unstructured":"Huron D (2001) Tone and voice: a derivation of the rules of voice-leading from perceptual principles. Music Percept 19(1):1\u201364","journal-title":"Music Percept"},{"key":"1702_CR35","unstructured":"Bernardes G, Davies ME, Guedes C (2017) A perceptually-motivated harmonic compatibility method for music mixing. In: Procedings of the symposium on computer music multidisciplinary research, pp. 104\u2013115"},{"key":"1702_CR36","doi-asserted-by":"crossref","unstructured":"Pampalk E, Rauber A, Merkl D (2002) Content-based organization and visualization of music archives. In: Proceedings of the Tenth ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, pp 570\u2013579","DOI":"10.1145\/641007.641121"},{"issue":"2","key":"1702_CR37","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1076\/jnmr.32.2.193.16745","volume":"32","author":"A Rauber","year":"2003","unstructured":"Rauber A, Pampalk E, Merkl D (2003) The som-enhanced jukebox: organization and visualization of music collections based on perceptual models. Journal of New Music Research 32(2):193\u2013210. https:\/\/doi.org\/10.1076\/jnmr.32.2.193.16745","journal-title":"Journal of New Music Research"},{"issue":"2","key":"1702_CR38","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1121\/1.1908630","volume":"33","author":"E Zwicker","year":"1961","unstructured":"Zwicker E (1961) Subdivision of the audible frequency range into critical bands (frequenzgruppen). The Journal of the Acoustical Society of America 33(2):248\u2013248. https:\/\/doi.org\/10.1121\/1.1908630","journal-title":"The Journal of the Acoustical Society of America"},{"key":"1702_CR39","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-2728-1","volume-title":"Human psychophysics, vol 3","author":"W Yost","year":"1993","unstructured":"Yost W, Popper A, Fay R (1993) Human psychophysics, vol 3. Springer, New York"},{"issue":"5","key":"1702_CR40","doi-asserted-by":"publisher","first-page":"2755","DOI":"10.1121\/1.4777600","volume":"110","author":"P Vassilakis","year":"2001","unstructured":"Vassilakis P (2001) Auditory roughness estimation of complex spectra\u2014roughness degrees and dissonance ratings of harmonic intervals revisited. J Acoust Soc Am 110(5):2755\u20132755","journal-title":"J Acoust Soc Am"},{"key":"1702_CR41","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1080\/09298215.2016.1182192","volume":"45","author":"G Bernardes","year":"2016","unstructured":"Bernardes G, Cocharro D, Caetano M, Guedes C, Davies MEP (2016) A multi-level tonal interval space for modelling pitch relatedness and musical consonance. J New Music Res 45:243\u2013260","journal-title":"J New Music Res"},{"issue":"2","key":"1702_CR42","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1177\/102986490901300202","volume":"13","author":"P Holtz","year":"2009","unstructured":"Holtz P (2009) What\u2019s your music? Subjective theories of music-creating artists. Music Sci 13(2):207\u2013230","journal-title":"Music Sci"},{"key":"1702_CR43","unstructured":"Fonseca E, Pons Puig J, Favory X, Font Corbera F, Bogdanov D, Ferraro A, Oramas S, Porter A, Serra X (2017) Freesound datasets: a platform for the creation of open audio datasets. In: Hu X, Cunningham SJ, Turnbull D, Duan Z (eds) Proceedings of the 18th ISMIR Conference; 2017 Oct 23-27; Suzhou, China.[Canada]: International Society for Music Information Retrieval; 2017. P. 486-93. International Society for Music Information Retrieval (ISMIR)"}],"container-title":["Personal and Ubiquitous Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00779-022-01702-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00779-022-01702-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00779-022-01702-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,20]],"date-time":"2023-10-20T05:04:35Z","timestamp":1697778275000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00779-022-01702-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,23]]},"references-count":43,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,10]]}},"alternative-id":["1702"],"URL":"https:\/\/doi.org\/10.1007\/s00779-022-01702-z","relation":{},"ISSN":["1617-4909","1617-4917"],"issn-type":[{"type":"print","value":"1617-4909"},{"type":"electronic","value":"1617-4917"}],"subject":[],"published":{"date-parts":[[2022,12,23]]},"assertion":[{"value":"3 January 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 November 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 December 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Conflict of interest"}}]}}