{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T04:24:49Z","timestamp":1784521489149,"version":"3.55.0"},"reference-count":111,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2023,3,21]],"date-time":"2023-03-21T00:00:00Z","timestamp":1679356800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,21]],"date-time":"2023-03-21T00:00:00Z","timestamp":1679356800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2023,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Deep learning methods are recognised as state-of-the-art for many applications of machine learning. Recently, deep learning methods have emerged as a solution to the task of automatic music generation (AMG) using symbolic tokens in a target style, but their superiority over non-deep learning methods has not been demonstrated. Here, we conduct a listening study to comparatively evaluate several music generation systems along six musical dimensions: stylistic success, aesthetic pleasure, repetition or self-reference, melody, harmony, and rhythm. A range of models, both deep learning algorithms and other methods, are used to generate 30-s excerpts in the style of Classical string quartets and classical piano improvisations. Fifty participants with relatively high musical knowledge rate unlabelled samples of computer-generated and human-composed excerpts for the six musical dimensions. We use non-parametric Bayesian hypothesis testing to interpret the results, allowing the possibility of finding meaningful<jats:italic>non<\/jats:italic>-differences between systems\u2019 performance. We find that the strongest deep learning method, a reimplemented version of Music Transformer, has equivalent performance to a non-deep learning method, MAIA Markov, demonstrating that to date, deep learning does not outperform other methods for AMG. We also find there still remains a significant gap between any algorithmic method and human-composed excerpts.<\/jats:p>","DOI":"10.1007\/s10994-023-06309-w","type":"journal-article","created":{"date-parts":[[2023,3,21]],"date-time":"2023-03-21T21:17:15Z","timestamp":1679433435000},"page":"1785-1822","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["Deep learning\u2019s shallow gains: a comparative evaluation of algorithms for automatic music generation"],"prefix":"10.1007","volume":"112","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8709-8829","authenticated-orcid":false,"given":"Zongyu","family":"Yin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1330-7346","authenticated-orcid":false,"given":"Federico","family":"Reuben","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3146-5401","authenticated-orcid":false,"given":"Susan","family":"Stepney","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7880-5093","authenticated-orcid":false,"given":"Tom","family":"Collins","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,3,21]]},"reference":[{"issue":"3","key":"6309_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2967506","volume":"14","author":"K Agres","year":"2016","unstructured":"Agres, K., Forth, J., & Wiggins, G. A. (2016). Evaluation of musical creativity and musical metacreation systems. Computers in Entertainment (CIE), 14(3), 1\u201333.","journal-title":"Computers in Entertainment (CIE)"},{"issue":"6","key":"6309_CR2","doi-asserted-by":"publisher","first-page":"1200","DOI":"10.1016\/j.matcom.2009.04.012","volume":"80","author":"G Aguilera","year":"2010","unstructured":"Aguilera, G., Gal\u00e1n, J. L., Madrid, R., Mart\u00ednez, A. M., Padilla, Y., & Rodr\u00edguez, P. (2010). Automated generation of contrapuntal musical compositions using probabilistic logic in derive. Mathematics and Computers in Simulation, 80(6), 1200\u20131211.","journal-title":"Mathematics and Computers in Simulation"},{"key":"6309_CR3","first-page":"25","volume":"17","author":"M Allan","year":"2005","unstructured":"Allan, M., & Williams, C. K. (2005). Harmonising chorales by probabilistic inference. Advances in neural information processing systems, 17, 25\u201332.","journal-title":"Advances in neural information processing systems"},{"issue":"5","key":"6309_CR4","doi-asserted-by":"publisher","first-page":"997","DOI":"10.1037\/0022-3514.43.5.997","volume":"43","author":"TM Amabile","year":"1982","unstructured":"Amabile, T. M. (1982). Social psychology of creativity: A consensual assessment technique. Journal of Personality and Social Psychology, 43(5), 997.","journal-title":"Journal of Personality and Social Psychology"},{"issue":"2","key":"6309_CR5","doi-asserted-by":"publisher","first-page":"175","DOI":"10.2307\/1575226","volume":"22","author":"C Ames","year":"1989","unstructured":"Ames, C. (1989). The Markov process as a compositional model: A survey and tutorial. Leonardo, 22(2), 175\u2013187.","journal-title":"Leonardo"},{"issue":"2","key":"6309_CR6","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1162\/comj.2010.34.2.25","volume":"34","author":"T Anders","year":"2010","unstructured":"Anders, T., & Miranda, E. R. (2010). Constraint application with higher-order programming for modeling music theories. Computer Music Journal, 34(2), 25\u201338.","journal-title":"Computer Music Journal"},{"issue":"2","key":"6309_CR7","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1162\/comj.2009.33.2.48","volume":"33","author":"C Ariza","year":"2009","unstructured":"Ariza, C. (2009). The interrogator as critic: The turing test and the evaluation of generative music systems. Computer Music Journal, 33(2), 48\u201370.","journal-title":"Computer Music Journal"},{"key":"6309_CR8","volume-title":"Statistics for psychology","author":"A Aron","year":"2013","unstructured":"Aron, A., Coups, E. J., & Aron, E. N. (2013). Statistics for psychology (6th ed.). London: Pearson.","edition":"6"},{"key":"6309_CR9","first-page":"366","volume-title":"Bol processor grammars","author":"B Bel","year":"1992","unstructured":"Bel, B., & Kippen, J. (1992). Bol processor grammars (pp. 366\u2013400). Cambridge, MA: MIT Press."},{"key":"6309_CR10","unstructured":"Bengio, S., Vinyals, O., Jaitly, N., & Shazeer, N. (2015). Scheduled sampling for sequence prediction with recurrent neural networks. In: Proceedings of the 28th international conference on neural information processing systems (Vol. 1, pp. 1171\u20131179)"},{"issue":"1","key":"6309_CR11","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1016\/j.cognition.2005.11.007","volume":"100","author":"E Bigand","year":"2006","unstructured":"Bigand, E., & Poulin-Charronnat, B. (2006). Are we experienced listeners? A review of the musical capacities that do not depend on formal musical training. Cognition, 100(1), 100\u2013130.","journal-title":"Cognition"},{"key":"6309_CR12","volume-title":"The creative mind: Myths and mechanisms","author":"MA Boden","year":"1990","unstructured":"Boden, M. A. (1990). The creative mind: Myths and mechanisms. London: Weidenfield and Nicholson."},{"key":"6309_CR13","unstructured":"Cambridge University Faculty of Music: Music Tripos courses. Cambridge, UK (2010)"},{"key":"6309_CR14","first-page":"26","volume":"102","author":"H Cohen","year":"1999","unstructured":"Cohen, H. (1999). Colouring without seeing: A problem in machine creativity. AISB Quarterly, 102, 26\u201335.","journal-title":"AISB Quarterly"},{"key":"6309_CR15","unstructured":"Collins, T. E. (2011). Improved methods for pattern discovery in music, with applications in automated stylistic composition. Ph.D. thesis, The Open University"},{"key":"6309_CR16","doi-asserted-by":"crossref","unstructured":"Collins, T., & Laney, R. (2017). Computer-generated stylistic compositions with long-term repetitive and phrasal structure. Journal of Creative Music Systems 1(2)","DOI":"10.5920\/JCMS.2017.02"},{"key":"6309_CR17","unstructured":"Collins, T., Arzt, A., Flossmann, S., & Widmer, G. (2013). SIARCT-CFP: Improving precision and the discovery of inexact musical patterns in point-set representations. In: Proceedings of the international society for music information retrieval conference (ISMIR) (pp. 549\u2013554)"},{"issue":"4","key":"6309_CR18","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1111\/j.1740-9713.2011.00519.x","volume":"8","author":"T Collins","year":"2011","unstructured":"Collins, T., Laney, R., Willis, A., & Garthwaite, P. H. (2011). Chopin, mazurkas and Markov: Making music in style with statistics. Significance, 8(4), 154\u2013159.","journal-title":"Significance"},{"issue":"1","key":"6309_CR19","first-page":"16","volume":"30","author":"T Collins","year":"2016","unstructured":"Collins, T., Laney, R., Willis, A., & Garthwaite, P. H. (2016). Developing and evaluating computational models of musical style. AI EDAM, 30(1), 16\u201343.","journal-title":"AI EDAM"},{"key":"6309_CR20","unstructured":"Collins, T., Thurlow, J., Laney, R., Willis, A., & Garthwaite, P. (2010). A comparative evaluation of algorithms for discovering translational patterns in baroque keyboard works. In Proceedings of the international society for music information retrieval conference (ISMIR) (pp. 3\u20138)"},{"key":"6309_CR21","unstructured":"Colton, S., Pease, A., Corneli, J., Cook, M., & Llano, T. (2014). Assessing progress in building autonomously creative systems. In: ICCC (pp. 137\u2013145). Ljubljana"},{"issue":"1","key":"6309_CR22","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1080\/09298219508570672","volume":"24","author":"D Conklin","year":"1995","unstructured":"Conklin, D., & Witten, I. H. (1995). Multiple viewpoint systems for music prediction. Journal of New Music Research, 24(1), 51\u201373.","journal-title":"Journal of New Music Research"},{"key":"6309_CR23","unstructured":"Cope, D.: Experiments in musical intelligence (Vol.\u00a012). AR editions (1996)"},{"key":"6309_CR24","volume-title":"Computer models of musical creativity","author":"D Cope","year":"2005","unstructured":"Cope, D. (2005). Computer models of musical creativity. Cambridge: MIT Press."},{"key":"6309_CR25","doi-asserted-by":"crossref","unstructured":"De\u00a0Boom, C., Laere, S. V., Verbelen, T., & Dhoedt, B. (2019). Rhythm, chord and melody generation for lead sheets using recurrent neural networks. In Joint European conference on machine learning and knowledge discovery in databases (pp. 454\u2013461). Springer","DOI":"10.1007\/978-3-030-43887-6_38"},{"issue":"2","key":"6309_CR26","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1111\/jaac.12636","volume":"77","author":"R De Clercq","year":"2019","unstructured":"De Clercq, R. (2019). Aesthetic pleasure explained: De Clercq. The Journal of Aesthetics and Art Criticism, 77(2), 121\u2013132. https:\/\/doi.org\/10.1111\/jaac.12636","journal-title":"The Journal of Aesthetics and Art Criticism"},{"key":"6309_CR27","unstructured":"Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: Human language technologies (Vol. 1 (Long and Short Papers), pp. 4171\u20134186)"},{"key":"6309_CR28","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1214\/aoms\/1177697203","volume":"41","author":"JM Dickey","year":"1970","unstructured":"Dickey, J. M., & Lientz, B. (1970). The weighted likelihood ratio, sharp hypotheses about chances, the order of a Markov chain. The Annals of Mathematical Statistics, 41, 214\u2013226.","journal-title":"The Annals of Mathematical Statistics"},{"key":"6309_CR29","doi-asserted-by":"publisher","first-page":"781","DOI":"10.3389\/fpsyg.2014.00781","volume":"5","author":"Z Dienes","year":"2014","unstructured":"Dienes, Z. (2014). Using Bayes to get the most out of non-significant results. Frontiers in Psychology, 5, 781.","journal-title":"Frontiers in Psychology"},{"key":"6309_CR30","unstructured":"Donahue, C., Mao, H. H., Li, Y. E., Cottrell, G. W., & McAuley, J. (2019). LakhNES: Improving multi-instrumental music generation with cross-domain pre-training. In Proceedings of the international society for music information retrieval conference (ISMIR)"},{"key":"6309_CR31","doi-asserted-by":"crossref","unstructured":"Dong, H. W., Hsiao, W. Y., Yang, L. C., & Yang, Y. H. (2018). MuseGAN: Multi-track sequential generative adversarial networks for symbolic music generation and accompaniment. In Proceedings of the 32nd AAAI conference on artificial intelligence","DOI":"10.1609\/aaai.v32i1.11312"},{"issue":"1\u20132","key":"6309_CR32","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1016\/0743-1066(90)90055-A","volume":"8","author":"K Ebcio\u011flu","year":"1990","unstructured":"Ebcio\u011flu, K. (1990). An expert system for harmonizing chorales in the style of JS Bach. The Journal of Logic Programming, 8(1\u20132), 145\u2013185.","journal-title":"The Journal of Logic Programming"},{"key":"6309_CR33","doi-asserted-by":"crossref","unstructured":"Eck, D., & Schmidhuber, J. (2002). Finding temporal structure in music: Blues improvisation with LSTM recurrent networks. In Proceedings of the 12th IEEE workshop on neural networks for signal processing (pp. 747\u2013756). IEEE","DOI":"10.1109\/NNSP.2002.1030094"},{"key":"6309_CR34","unstructured":"Eigenfeldt, A., & Pasquier, P. (2010). Realtime generation of harmonic progressions using controlled Markov selection. In Proceedings of ICCC-X-computational creativity conference (pp. 16\u201325)"},{"key":"6309_CR35","doi-asserted-by":"publisher","first-page":"513","DOI":"10.1613\/jair.3908","volume":"48","author":"JD Fern\u00e1ndez","year":"2013","unstructured":"Fern\u00e1ndez, J. D., & Vico, F. (2013). AI methods in algorithmic composition: A comprehensive survey. Journal of Artificial Intelligence Research, 48, 513\u2013582.","journal-title":"Journal of Artificial Intelligence Research"},{"key":"6309_CR36","volume-title":"Conceptual spaces: The geometry of thought","author":"P Gardenfors","year":"2004","unstructured":"Gardenfors, P. (2004). Conceptual spaces: The geometry of thought. Cambridge: MIT press."},{"key":"6309_CR37","volume-title":"A classic turn of phrase: Music and the psychology of convention","author":"R Gjerdingen","year":"1988","unstructured":"Gjerdingen, R. (1988). A classic turn of phrase: Music and the psychology of convention. Philadelphia, PA: University of Pennsylvania Press."},{"key":"6309_CR38","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780195313710.001.0001","volume-title":"Music in the galant style","author":"R Gjerdingen","year":"2007","unstructured":"Gjerdingen, R. (2007). Music in the galant style. Oxford, UK: Oxford University Press."},{"key":"6309_CR39","volume-title":"Research in psychology: Methods and design","author":"CJ Goodwin","year":"2016","unstructured":"Goodwin, C. J., & Goodwin, K. A. (2016). Research in psychology: Methods and design (8th ed.). New York, NY: Wiley.","edition":"8"},{"key":"6309_CR40","unstructured":"Grachten, M., & Widmer, G. (2011). Explaining musical expression as a mixture of basis functions. In Proceedings of the 8th sound and music computing conference (SMC 2011)"},{"key":"6309_CR41","doi-asserted-by":"crossref","unstructured":"Graves, A., Mohamed, A. R., Hinton, G. (2013). Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing (pp. 6645\u20136649). IEEE","DOI":"10.1109\/ICASSP.2013.6638947"},{"issue":"4","key":"6309_CR42","doi-asserted-by":"publisher","first-page":"995","DOI":"10.1007\/s00521-018-3868-4","volume":"32","author":"G Hadjeres","year":"2020","unstructured":"Hadjeres, G., & Nielsen, F. (2020). Anticipation-RNN: Enforcing unary constraints in sequence generation, with application to interactive music generation. Neural Computing and Applications, 32(4), 995\u20131005.","journal-title":"Neural Computing and Applications"},{"key":"6309_CR43","unstructured":"Hadjeres, G., Pachet, F., & Nielsen, F. (2017). Deepbach: A steerable model for Bach chorales generation. In International conference on machine learning (pp. 1362\u20131371). PMLR"},{"key":"6309_CR44","unstructured":"Hawthorne, C., Elsen, E., Song, J., Roberts, A., Simon, I., Raffel, C., Engel, J., Oore, S., & Eck, D. (2018). Onsets and frames: Dual-objective piano transcription. In Proceedings of the international society for music information retrieval conference (ISMIR)"},{"key":"6309_CR45","unstructured":"Hawthorne, C., Stasyuk, A., Roberts, A., Simon, I., Huang, C. Z. A., Dieleman, S., Elsen, E., Engel, J., & Eck, D. (2019). Enabling factorized piano music modeling and generation with the MAESTRO dataset. In International conference on learning representations. https:\/\/openreview.net\/forum?id=r1lYRjC9F7."},{"key":"6309_CR46","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770\u2013778)","DOI":"10.1109\/CVPR.2016.90"},{"issue":"2","key":"6309_CR47","doi-asserted-by":"publisher","first-page":"180","DOI":"10.1093\/ml\/59.2.180","volume":"59","author":"SA Hedges","year":"1978","unstructured":"Hedges, S. A. (1978). Dice music in the eighteenth century. Music & Letters, 59(2), 180\u2013187.","journal-title":"Music & Letters"},{"issue":"4","key":"6309_CR48","doi-asserted-by":"publisher","first-page":"510","DOI":"10.1109\/TAFFC.2017.2737984","volume":"10","author":"D Herremans","year":"2017","unstructured":"Herremans, D., & Chew, E. (2017). Morpheus: Generating structured music with constrained patterns and tension. IEEE Transactions on Affective Computing, 10(4), 510\u2013523.","journal-title":"IEEE Transactions on Affective Computing"},{"issue":"2","key":"6309_CR49","doi-asserted-by":"publisher","first-page":"246","DOI":"10.2307\/1415746","volume":"48","author":"K Hevner","year":"1936","unstructured":"Hevner, K. (1936). Experimental studies of the elements of expression in music. The American Journal of Psychology, 48(2), 246\u2013268.","journal-title":"The American Journal of Psychology"},{"key":"6309_CR50","unstructured":"Hild, H., Feulner, J., & Menzel, W. (1991). Harmonet: A neural net for harmonizing chorales in the style of JS Bach. In Proceedings of the 4th international conference on neural information processing systems (pp. 267\u2013274)"},{"key":"6309_CR51","unstructured":"Hiller\u00a0Jr, L. A., & Isaacson, L. M. (1957). Musical composition with a high speed digital computer. In Audio Engineering society convention 9. Audio Engineering Society"},{"key":"6309_CR52","volume-title":"Fluid concepts and creative analogies: Computer models of the fundamental mechanisms of thought","author":"D Hofstadter","year":"1995","unstructured":"Hofstadter, D. (1995). Fluid concepts and creative analogies: Computer models of the fundamental mechanisms of thought. New York: Basic Books."},{"key":"6309_CR53","unstructured":"Huang, C. Z. A., Vaswani, A., Uszkoreit, J., Simon, I., Hawthorne, C., Shazeer, N., Dai, A. M., Hoffman, M. D., Dinculescu, M., & Eck, D. (2018). Music transformer: Generating music with long-term structure. In International conference on learning representations"},{"key":"6309_CR54","unstructured":"Janssen, B., Collins, T., & Ren, I. Y. (2019). Algorithmic ability to predict the musical future: Datasets and evaluation. In Proceedings of the international society for music information retrieval conference (ISMIR) (pp. 208\u2013215)"},{"key":"6309_CR55","doi-asserted-by":"crossref","unstructured":"Johnson, D. D. (2017). Generating polyphonic music using tied parallel networks. In International conference on evolutionary and biologically inspired music and art (pp. 128\u2013143). Springer","DOI":"10.1007\/978-3-319-55750-2_9"},{"issue":"3","key":"6309_CR56","doi-asserted-by":"publisher","first-page":"246","DOI":"10.1007\/s12559-012-9156-1","volume":"4","author":"A Jordanous","year":"2012","unstructured":"Jordanous, A. (2012). A standardised procedure for evaluating creative systems: Computational creativity evaluation based on what it is to be creative. Cognitive Computation, 4(3), 246\u2013279.","journal-title":"Cognitive Computation"},{"key":"6309_CR57","doi-asserted-by":"crossref","unstructured":"Jordanous, A. (2019). Evaluating evaluation: Assessing progress and practices in computational creativity research. In Computational creativity (pp. 211\u2013236). Springer","DOI":"10.1007\/978-3-319-43610-4_10"},{"key":"6309_CR58","volume-title":"The concise Oxford dictionary of music","author":"M Kennedy","year":"2004","unstructured":"Kennedy, M., & Bourne, J. (2004). The concise Oxford dictionary of music. Oxford: OUP Oxford."},{"key":"6309_CR59","unstructured":"Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980."},{"issue":"3","key":"6309_CR60","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1016\/0010-0285(79)90016-1","volume":"11","author":"CL Krumhansl","year":"1979","unstructured":"Krumhansl, C. L. (1979). The psychological representation of musical pitch in a tonal context. Cognitive Psychology, 11(3), 346\u2013374.","journal-title":"Cognitive Psychology"},{"key":"6309_CR61","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781139087759","volume-title":"Bayesian cognitive modeling: A practical course","author":"MD Lee","year":"2014","unstructured":"Lee, M. D., & Wagenmakers, E. J. (2014). Bayesian cognitive modeling: A practical course. Cambridge: Cambridge University Press."},{"key":"6309_CR62","unstructured":"Liang, F. (2006). Bachbot: Automatic composition in the style of Bach chorales. Master\u2019s thesis, University of Cambridge"},{"key":"6309_CR63","unstructured":"Lim, H., Rhyu, S., & Lee, K. (2017). Chord generation from symbolic melody using BLSTM networks. In Proceedings of the international society for music information retrieval conference (ISMIR)"},{"issue":"5","key":"6309_CR64","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1016\/j.jmp.2011.06.001","volume":"55","author":"T Lodewyckx","year":"2011","unstructured":"Lodewyckx, T., Kim, W., Lee, M. D., Tuerlinckx, F., Kuppens, P., & Wagenmakers, E. J. (2011). A tutorial on Bayes factor estimation with the product space method. Journal of Mathematical Psychology, 55(5), 331\u2013347.","journal-title":"Journal of Mathematical Psychology"},{"key":"6309_CR65","doi-asserted-by":"crossref","unstructured":"Louie, R., Coenen, A., Huang, C. Z., Terry, M., & Cai, C. J. (2020). Novice-AI music co-creation via AI-steering tools for deep generative models. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1\u201313)","DOI":"10.1145\/3313831.3376739"},{"key":"6309_CR66","doi-asserted-by":"crossref","unstructured":"Mao, H. H., Shin, T., & Cottrell, G. (2018). Deepj: Style-specific music generation. In 2018 IEEE 12th international conference on semantic computing (ICSC) (pp. 377\u2013382). IEEE","DOI":"10.1109\/ICSC.2018.00077"},{"key":"6309_CR67","doi-asserted-by":"crossref","unstructured":"McCormack, J., & Lomas, A. (2020). Understanding aesthetic evaluation using deep learning. In International conference on computational intelligence in music, sound, art and design (part of EvoStar) (pp. 118\u2013133). Springer","DOI":"10.1007\/978-3-030-43859-3_9"},{"key":"6309_CR68","unstructured":"McKay, C., Cumming, J., & Fujinaga, I. (2018). Jsymbolic 2.2: Extracting features from symbolic music for use in musicological and mir research. In Proceedings of the international society for music information retrieval conference (ISMIR) (pp. 348\u2013354)"},{"key":"6309_CR69","unstructured":"Mehri, S., Kumar, K., Gulrajani, I., Kumar, R., Jain, S., Sotelo, J., Courville, A., & Bengio, Y. (2017). SampleRNN: An unconditional end-to-end neural audio generation model. In 5th international conference on learning representations (ICLR 2017)"},{"issue":"2\u20133","key":"6309_CR70","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1080\/09540099408915726","volume":"6","author":"MC Mozer","year":"1994","unstructured":"Mozer, M. C. (1994). Neural network music composition by prediction: Exploring the benefits of psychoacoustic constraints and multi-scale processing. Connection Science, 6(2\u20133), 247\u2013280.","journal-title":"Connection Science"},{"key":"6309_CR71","doi-asserted-by":"crossref","unstructured":"Navarro, M., Caetano, M., Bernardes, G., Castro, L. N. D., Corchado, J. M. (2015). Automatic generation of chord progressions with an artificial immune system. In International conference on evolutionary and biologically inspired music and art (pp. 175\u2013186). Springer","DOI":"10.1007\/978-3-319-16498-4_16"},{"key":"6309_CR72","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-211-75540-2","volume-title":"Algorithmic composition: Paradigms of automated music generation","author":"G Nierhaus","year":"2009","unstructured":"Nierhaus, G. (2009). Algorithmic composition: Paradigms of automated music generation. Berlin: Springer."},{"key":"6309_CR73","volume-title":"Markov chains","author":"JR Norris","year":"1998","unstructured":"Norris, J. R., & Norris, J. R. (1998). Markov chains (Vol. 2). Cambridge: Cambridge University Press."},{"key":"6309_CR74","doi-asserted-by":"crossref","unstructured":"O\u2019Mahony, N., Campbell, S., Carvalho, A., Harapanahalli, S., Hernandez, G. V., Krpalkova, L., Riordan, D., Walsh, J. (2019). Deep learning vs. traditional computer vision. In Science and information conference (pp. 128\u2013144). Springer","DOI":"10.1007\/978-3-030-17795-9_10"},{"key":"6309_CR75","first-page":"1","volume":"32","author":"S Oore","year":"2018","unstructured":"Oore, S., Simon, I., Dieleman, S., Eck, D., & Simonyan, K. (2018). This time with feeling: Learning expressive musical performance. Neural Computing and Applications, 32, 1\u201313.","journal-title":"Neural Computing and Applications"},{"key":"6309_CR76","unstructured":"Pachet, F., & Roy, P. (2020). Plagiarism risk detector and interface. US Patent application number US 2020\/0372882 A1"},{"key":"6309_CR77","unstructured":"Papadopoulos, G., & Wiggins, G. (1999). AI methods for algorithmic composition: A survey, a critical view and future prospects. In AISB symposium on musical creativity (Vol. 124, pp. 110\u2013117). Edinburgh, UK"},{"issue":"2","key":"6309_CR78","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1177\/102986490200600203","volume":"6","author":"M Pearce","year":"2002","unstructured":"Pearce, M., Meredith, D., & Wiggins, G. (2002). Motivations and methodologies for automation of the compositional process. Musicae Scientiae, 6(2), 119\u2013147.","journal-title":"Musicae Scientiae"},{"key":"6309_CR79","unstructured":"Pearce, M., & Wiggins, G. (2001). Towards a framework for the evaluation of machine compositions. In Proceedings of the AISB\u201901 symposium on artificial intelligence and creativity in the arts and sciences (pp. 22\u201332). Citeseer"},{"key":"6309_CR80","unstructured":"Pearce, M. T., & Wiggins, G. A. (2007). Evaluating cognitive models of musical composition. In Proceedings of the 4th international joint workshop on computational creativity (pp. 73\u201380). Goldsmiths, University of London"},{"key":"6309_CR81","doi-asserted-by":"crossref","unstructured":"Popper, K. (2005). The logic of scientific discovery. Routledge, London, UK. Original work published 1959","DOI":"10.4324\/9780203994627"},{"key":"6309_CR82","doi-asserted-by":"crossref","unstructured":"Popper, K. (2014). Conjectures and refutations: The growth of scientific knowledge. Routledge, London, UK. Original work published 1963","DOI":"10.4324\/9780203538074"},{"key":"6309_CR83","doi-asserted-by":"crossref","unstructured":"Quick, D., & Hudak, P. (2013). Grammar-based automated music composition in Haskell. In Proceedings of the first ACM SIGPLAN workshop on Functional art, music, modeling and design (pp. 59\u201370)","DOI":"10.1145\/2505341.2505345"},{"key":"6309_CR84","doi-asserted-by":"crossref","unstructured":"Rafael, B., Oertl, S., Affenzeller, M., & Wagner, S. (2009). Using heuristic optimization for segmentation of symbolic music. In International conference on computer aided systems theory (pp. 641\u2013648). Springer","DOI":"10.1007\/978-3-642-04772-5_83"},{"key":"6309_CR85","unstructured":"Roberts, A., Engel, J., Raffel, C., Hawthorne, C., & Eck, D. (2018). A hierarchical latent vector model for learning long-term structure in music. In International conference on machine learning (pp. 4364\u20134373)"},{"key":"6309_CR86","doi-asserted-by":"crossref","unstructured":"Rodr\u00edguez-L\u00f3pez, M., & Volk, A. (2013). Symbolic segmentation: A corpus-based analysis of melodic phrases. In International symposium on computer music multidisciplinary research (pp. 548\u2013557). Springer","DOI":"10.1007\/978-3-319-12976-1_33"},{"key":"6309_CR87","volume-title":"The classical style: Haydn, Mozart, Beethoven","author":"C Rosen","year":"1997","unstructured":"Rosen, C. (1997). The classical style: Haydn, Mozart, Beethoven. London: WW Norton & Company."},{"issue":"2","key":"6309_CR88","doi-asserted-by":"publisher","first-page":"225","DOI":"10.3758\/PBR.16.2.225","volume":"16","author":"JN Rouder","year":"2009","unstructured":"Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225\u2013237.","journal-title":"Psychonomic Bulletin & Review"},{"issue":"2","key":"6309_CR89","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1037\/0003-066X.54.2.93","volume":"54","author":"N Schwarz","year":"1999","unstructured":"Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54(2), 93.","journal-title":"American Psychologist"},{"issue":"1","key":"6309_CR90","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1016\/0022-2836(81)90087-5","volume":"147","author":"TF Smith","year":"1981","unstructured":"Smith, T. F., & Waterman, M. S. (1981). Identification of common molecular subsequences. Journal of Molecular Biology, 147(1), 195\u2013197.","journal-title":"Journal of Molecular Biology"},{"issue":"1","key":"6309_CR91","doi-asserted-by":"publisher","first-page":"52","DOI":"10.2307\/40285282","volume":"2","author":"MJ Steedman","year":"1984","unstructured":"Steedman, M. J. (1984). A generative grammar for jazz chord sequences. Music Perception, 2(1), 52\u201377.","journal-title":"Music Perception"},{"key":"6309_CR92","volume-title":"Combinatorics of experimental design","author":"AP Street","year":"1986","unstructured":"Street, A. P., & Street, D. J. (1986). Combinatorics of experimental design. Oxford: Oxford University Press, Inc."},{"issue":"1","key":"6309_CR93","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1080\/09298215.2018.1515233","volume":"48","author":"BL Sturm","year":"2019","unstructured":"Sturm, B. L., Ben-Tal, O., Monaghan, \u00da., Collins, N., Herremans, D., Chew, E., Hadjeres, G., Deruty, E., & Pachet, F. (2019). Machine learning research that matters for music creation: A case study. Journal of New Music Research, 48(1), 36\u201355.","journal-title":"Journal of New Music Research"},{"key":"6309_CR94","unstructured":"Sturm, B., Santos, J. F., & Korshunova, I. (2015). Folk music style modelling by recurrent neural networks with long short term memory units. In Proceedings of the international society for music information retrieval conference (ISMIR)"},{"key":"6309_CR95","unstructured":"Tan, H. H., & Herremans, D. (2020). Music fadernets: Controllable music generation based on high-level features via low-level feature modelling. In Proceedings of the international society for music information retrieval conference (ISMIR)"},{"key":"6309_CR96","unstructured":"Thickstun, J., Harchaoui, Z., Foster, D. P., & Kakade, S. M.: Coupled recurrent models for polyphonic music composition. In Proceedings of the international society for music information retrieval conference (ISMIR)"},{"issue":"4","key":"6309_CR97","doi-asserted-by":"publisher","first-page":"27","DOI":"10.2307\/3679551","volume":"13","author":"PM Todd","year":"1989","unstructured":"Todd, P. M. (1989). A connectionist approach to algorithmic composition. Computer Music Journal, 13(4), 27\u201343.","journal-title":"Computer Music Journal"},{"key":"6309_CR98","unstructured":"Torrance, E. P. (1998). Torrance tests of creative thinking: Norms-technical manual: Figural (streamlined) forms A & B. Scholastic Testing Service."},{"key":"6309_CR99","doi-asserted-by":"crossref","unstructured":"Toshniwal, S., Kannan, A., Chiu, C. C., Wu, Y., Sainath, T. N., & Livescu, K. (2018). A comparison of techniques for language model integration in encoder\u2013decoder speech recognition. In 2018 IEEE spoken language technology workshop (SLT) (pp. 369\u2013375). IEEE.","DOI":"10.1109\/SLT.2018.8639038"},{"key":"6309_CR100","unstructured":"van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016). Wavenet: A generative model for raw audio. arXiv:1609.03499"},{"key":"6309_CR101","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/02664763.2019.1709053","volume":"47","author":"J van Doorn","year":"2020","unstructured":"van Doorn, J., Ly, A., Marsman, M., & Wagenmakers, E. J. (2020). Bayesian rank-based hypothesis testing for the rank sum test, the signed rank test, and spearman\u2019s $$\\rho$$. Journal of Applied Statistics, 47, 1\u201323.","journal-title":"Journal of Applied Statistics"},{"key":"6309_CR102","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, \u0141., & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 5998\u20136008)"},{"issue":"3","key":"6309_CR103","doi-asserted-by":"publisher","first-page":"158","DOI":"10.1016\/j.cogpsych.2009.12.001","volume":"60","author":"EJ Wagenmakers","year":"2010","unstructured":"Wagenmakers, E. J., Lodewyckx, T., Kuriyal, H., & Grasman, R. (2010). Bayesian hypothesis testing for psychologists: A tutorial on the Savage\u2013Dickey method. Cognitive Psychology, 60(3), 158\u2013189.","journal-title":"Cognitive Psychology"},{"issue":"1","key":"6309_CR104","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1076\/jnmr.31.1.37.8103","volume":"31","author":"G Widmer","year":"2002","unstructured":"Widmer, G. (2002). Machine discoveries: A few simple, robust local expression principles. Journal of New Music Research, 31(1), 37\u201350.","journal-title":"Journal of New Music Research"},{"issue":"2","key":"6309_CR105","first-page":"1","volume":"8","author":"G Widmer","year":"2016","unstructured":"Widmer, G. (2016). Getting closer to the essence of music: The con espressione manifesto. ACM Transactions on Intelligent Systems and Technology (TIST), 8(2), 1\u201313.","journal-title":"ACM Transactions on Intelligent Systems and Technology (TIST)"},{"issue":"1","key":"6309_CR106","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1093\/llc\/fqm025","volume":"23","author":"GA Wiggins","year":"2008","unstructured":"Wiggins, G. A. (2008). Computer models of musical creativity: A review of computer models of musical creativity by David Cope. Literary and Linguistic Computing, 23(1), 109\u2013116.","journal-title":"Literary and Linguistic Computing"},{"key":"6309_CR107","volume-title":"Formalized music: Thought and mathematics in composition","author":"I Xenakis","year":"1992","unstructured":"Xenakis, I. (1992). Formalized music: Thought and mathematics in composition (Vol. 6). New York: Pendragon Press."},{"key":"6309_CR108","unstructured":"Yang, L. C., Chou, S. Y., & Yang, Y. H.: MidiNet: A convolutional generative adversarial network for symbolic-domain music generation. In Proceedings of the international society for music information retrieval conference (ISMIR) (pp. 324\u2013331)"},{"issue":"9","key":"6309_CR109","doi-asserted-by":"publisher","first-page":"4773","DOI":"10.1007\/s00521-018-3849-7","volume":"32","author":"LC Yang","year":"2020","unstructured":"Yang, L. C., & Lerch, A. (2020). On the evaluation of generative models in music. Neural Computing and Applications, 32(9), 4773\u20134784.","journal-title":"Neural Computing and Applications"},{"key":"6309_CR110","doi-asserted-by":"crossref","unstructured":"Yin, Z., Reuben, F., Stepney, S., Collins, T. (2021). A good algorithm does not steal\u2013it imitates: The originality report as a means of measuring when a music generation algorithm copies too much. In Artificial intelligence in music, sound, art and design: 10th international conference, EvoMUSART 2021, Held as Part of EvoStar 2021, Virtual Event, April 7\u20139, 2021, proceedings 10 (pp. 360\u2013375). Springer International Publishing.","DOI":"10.1007\/978-3-030-72914-1_24"},{"key":"6309_CR111","first-page":"340","volume":"3","author":"Z Yin","year":"2022","unstructured":"Yin, Z., Reuben, F., Stepney, S., & Collins, T. (2022). Measuring when a music generation algorithm copies too much: The originality report, cardinality score, and symbolic fingerprinting by geometric hashing. Springer Nature Computer Science, 3, 340.","journal-title":"Springer Nature Computer Science"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-023-06309-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-023-06309-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-023-06309-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,9]],"date-time":"2023-12-09T09:00:12Z","timestamp":1702112412000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-023-06309-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,21]]},"references-count":111,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,5]]}},"alternative-id":["6309"],"URL":"https:\/\/doi.org\/10.1007\/s10994-023-06309-w","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,21]]},"assertion":[{"value":"25 June 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 October 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 January 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 March 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The last author is a co-founder of the music collective Music Artificial Intelligence Algorithms, Inc. As such, he has an interest in the strong performance of the MAIA Markov algorithm. To avoid this bias affecting the outcome of the study, we ran the listening study blind and did everything one could reasonably expect to maximise the performance of outputs from other algorithms. Apart from this, the authors have no relevant financial or non-financial interests to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The listening study has been approved by the Physical Sciences Ethics Committee of the University of York.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"Participants involved in the listening study are informed their provided data will be used anonymously for this work.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"All authors consent to the publication of this work.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}]}}