{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T13:13:03Z","timestamp":1740143583173,"version":"3.37.3"},"reference-count":52,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,9,14]],"date-time":"2021-09-14T00:00:00Z","timestamp":1631577600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,9,14]],"date-time":"2021-09-14T00:00:00Z","timestamp":1631577600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Lang Resources &amp; Evaluation"],"published-print":{"date-parts":[[2022,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The optimal innovation hypothesis (OIH) offers a good aesthetic and cognitive reference for the kind of linguistic creativity where minimal variations can have a strong effect on the audience and realize an overall intended goal. The same approach can be the basis for creative systems. The question is how to concretely evaluate not only their quality, but also their pragmatic effect. This paper describes the original evaluations of two systems based on the OIH, one automatically yielding witty headlines for incoming news, the other producing song parodies, varying the song lyrics to evoke a given concept. The goal is to bring attention to the importance of evaluating the pragmatic potential of creative systems, in addition to the quality of their output, and to demonstrate how such evaluations can be done.<\/jats:p>","DOI":"10.1007\/s10579-021-09560-6","type":"journal-article","created":{"date-parts":[[2021,9,14]],"date-time":"2021-09-14T02:03:26Z","timestamp":1631585006000},"page":"451-476","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Pragmatic evaluations of automated linguistic creativity"],"prefix":"10.1007","volume":"56","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2422-5055","authenticated-orcid":false,"given":"Lorenzo","family":"Gatti","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Oliviero","family":"Stock","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carlo","family":"Strapparava","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"G\u00f6zde","family":"\u00d6zbal","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,9,14]]},"reference":[{"issue":"4","key":"9560_CR1","doi-asserted-by":"publisher","first-page":"434","DOI":"10.2501\/S0021849906060491","volume":"46","author":"D Allan","year":"2006","unstructured":"Allan, D. (2006). Effects of popular music in advertising on attention and memory. Journal of Advertising Research, 46(4), 434\u2013444.","journal-title":"Journal of Advertising Research"},{"key":"9560_CR3","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324920000236","author":"K Alnajjar","year":"2020","unstructured":"Alnajjar, K., & Toivonen, H. (2020). Computational generation of slogans. Natural Language Engineering. https:\/\/doi.org\/10.1017\/S1351324920000236","journal-title":"Natural Language Engineering"},{"key":"9560_CR2","unstructured":"Alnajjar, K., Lepp\u00e4nen, L. & Toivonen, H. (2019). No time like the present: Methods for generating colourful and factual multilingual news headlines. In Proceedings of the 10th International Conference on Computational Creativity (ICCC 2019)."},{"key":"9560_CR4","doi-asserted-by":"publisher","first-page":"768","DOI":"10.1007\/s11390-017-1758-3","volume":"32","author":"A Ayana","year":"2017","unstructured":"Ayana, Shen, S. Q., Lin, Y., Tu, C. C., Zhao, Y., Liu, Z., & Sun, M. (2017). Recent advances on neural headline generation. Journal of Computer Science and Technology, 32, 768\u2013784.","journal-title":"Journal of Computer Science and Technology"},{"key":"9560_CR5","unstructured":"Barbieri, G. , Pachet, F. , Roy, P. & Degli\u00a0Esposti, M. (2012). Markov constraints for generating lyrics with style. In Proceedings of the 20th European Conference on Artificial Intelligence (ECAI 2012) (Vol.\u00a0242, pp. 115\u2013120)."},{"key":"9560_CR6","volume-title":"Web 1T 5-gram version 1 LDC2006T13. DVD","author":"T Brants","year":"2006","unstructured":"Brants, T., & Franz, A. (2006). Web 1T 5-gram version 1 LDC2006T13. DVD. Linguistic Data Consortium."},{"key":"9560_CR7","doi-asserted-by":"crossref","unstructured":"Braun, D., Reiter, E. & Siddharthan, A. (2015). Creating textual driver feedback from telemetric data. In Proceedings of the 15th European workshop on Natural Language Generation (ENLG 2015) (pp. 156\u2013165).","DOI":"10.18653\/v1\/W15-4726"},{"key":"9560_CR8","doi-asserted-by":"crossref","unstructured":"Clark, E., Ross, A. S., Tan, C., Ji, Y. & Smith, N. A. (2018). Creative writing with a machine in the loop: Case studies on slogans and stories. In Proceedings of the 23rd international conference on Intelligent User Interfaces (IUI \u201918) (pp. 329\u2013340).","DOI":"10.1145\/3172944.3172983"},{"key":"9560_CR9","unstructured":"Colton, S., Charnley, J. W. & Pease, A.(2012). Computational creativity theory: The FACE and IDEA descriptive models. In Proceedings of the 2nd International Conference on Computational Creativity (ICCC 2011) (pp. 90\u201395)."},{"issue":"3","key":"9560_CR10","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1609\/aimag.v30i3.2257","volume":"30","author":"S Colton","year":"2009","unstructured":"Colton, S., de M\u00e1ntaras, R. L., & Stock, O. (2009). Computational creativity: Coming of age. AI Magazine, 30(3), 11\u201314.","journal-title":"AI Magazine"},{"key":"9560_CR11","unstructured":"Colton, S., Pease, A., Corneli, J., Cook, M. & Llano, T. (2014). Assessing progress in building autonomously creative systems. In Proceedings of the 5th International Conference on Computational Creativity (ICCC 2014) (pp. 137\u2013145)."},{"key":"9560_CR12","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1016\/j.artint.2017.11.005","volume":"256","author":"M Eppe","year":"2018","unstructured":"Eppe, M., Maclean, E., Confalonieri, R., Kutz, O., Schorlemmer, M., Plaza, E., & K\u00fchnberger, K. U. (2018). A computational framework for conceptual blending. Artificial Intelligence, 256, 105\u2013129. https:\/\/doi.org\/10.1016\/j.artint.2017.11.005","journal-title":"Artificial Intelligence"},{"issue":"2","key":"9560_CR13","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1207\/s15516709cog2202_1","volume":"22","author":"G Fauconnier","year":"1998","unstructured":"Fauconnier, G., & Turner, M. (1998). Conceptual integration networks. Cognitive Science, 22(2), 133\u2013187.","journal-title":"Cognitive Science"},{"key":"9560_CR14","volume-title":"The way we think: Conceptual blending and the mind\u2019s hidden complexities","author":"G Fauconnier","year":"2008","unstructured":"Fauconnier, G., & Turner, M. (2008). The way we think: Conceptual blending and the mind\u2019s hidden complexities. Basic Books."},{"key":"9560_CR15","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/7287.001.0001","volume-title":"WordNet","author":"C Fellbaum","year":"1998","unstructured":"Fellbaum, C. (1998). WordNet. Wiley."},{"key":"9560_CR16","unstructured":"Fish, S. (2009). Headline art. The New York Times. Retrieved March, 10, 2021, from http:\/\/web.archive.org\/web\/20210310114310\/"},{"key":"9560_CR17","unstructured":"Gatti, L., \u00d6zbal, G., Guerini, M., Stock, O. & Strapparava, C. (2015). Slogans are not forever: Adapting linguistic expressions to the news. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI 2015) (pp. 2452\u20132458)."},{"key":"9560_CR18","doi-asserted-by":"crossref","unstructured":"Gatti, L., \u00d6zbal, G., Guerini, M., Stock, O. & Strapparava, C. (2016). Heady-Lines: A creative generator of newspaper headlines. In Companion publication of the 2016 International Conference on Intelligent User Interfaces (IUI \u201916) (pp. 79\u201383).","DOI":"10.1145\/2876456.2879469"},{"key":"9560_CR19","doi-asserted-by":"crossref","unstructured":"Gatti, L., \u00d6zbal, G., Stock, O. & Strapparava, C. (2017a). Automatic generation of lyrics parodies. In Proceedings of the 25th ACM Multimedia conference (ACMMM-2017) (pp. 485\u2013491).","DOI":"10.1145\/3123266.3123410"},{"key":"9560_CR20","doi-asserted-by":"crossref","unstructured":"Gatti, L., \u00d6zbal, G., Stock, O. & Strapparava, C. (2017b). To sing like a mockingbird. In Proceedings of the 16th conference of the European chapter of the Association for Computational Linguistics (EACL-2017).","DOI":"10.18653\/v1\/E17-2048"},{"issue":"2","key":"9560_CR21","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1207\/s15327868ms1902_2","volume":"19","author":"R Giora","year":"2004","unstructured":"Giora, R., Fein, O., Kronrod, A., Elnatan, I., Shuval, N., & Zur, A. (2004). Weapons of mass distraction: Optimal innovation and pleasure ratings. Metaphor and Symbol, 19(2), 115\u2013141.","journal-title":"Metaphor and Symbol"},{"issue":"1","key":"9560_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/10926488.2017.1272934","volume":"32","author":"R Giora","year":"2017","unstructured":"Giora, R., Givoni, S., Heruti, V., & Fein, O. (2017). The role of defaultness in affecting pleasure: The optimal innovation hypothesis revisited. Metaphor and Symbol, 32(1), 1\u201318.","journal-title":"Metaphor and Symbol"},{"key":"9560_CR23","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/9780262018579.001.0001","volume-title":"Lexical analysis: Norms and exploitations","author":"P Hanks","year":"2013","unstructured":"Hanks, P. (2013). Lexical analysis: Norms and exploitations. Mit Press."},{"key":"9560_CR24","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/978-3-319-43610-4_10","volume-title":"Computational creativity: The philosophy and engineering of autonomously creative systems","author":"A Jordanous","year":"2019","unstructured":"Jordanous, A. (2019). Evaluating evaluation: Assessing progress and practices in computational creativity research. In T. Veale & F. A. Cardoso (Eds.), Computational creativity: The philosophy and engineering of autonomously creative systems (pp. 211\u2013236). Springer. https:\/\/doi.org\/10.1007\/978-3-319-43610-4_10"},{"key":"9560_CR25","unstructured":"Kenmochi, H. Ohshita, H. (2007). VOCALOID\u2014Commercial singing synthesizer based on sample concatenation. In Proceedings of the 8th annual conference of the International Speech Communication Association (INTERSPEECH\u201907) (pp. 4009\u20134010)."},{"issue":"3","key":"9560_CR26","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1080\/17513472.2017.1373561","volume":"11","author":"C Lamb","year":"2017","unstructured":"Lamb, C., Brown, D. G., & Clarke, C. L. (2017). A taxonomy of generative poetry techniques. Journal of Mathematics and the Arts, 11(3), 159\u2013179.","journal-title":"Journal of Mathematics and the Arts"},{"key":"9560_CR27","unstructured":"Li, B., Zook, A., Davis, N. & Riedl, M. O. (2012). Goal-driven conceptual blending: A computational approach for creativity. In Proceedings of the 2012 International Conference on Computational Creativity (ICCC 2012) (pp. 3\u201316)."},{"key":"9560_CR28","doi-asserted-by":"crossref","unstructured":"Machado, P. Cardoso, A. (1998). Computing aesthetics. In Proceedings of the 12th Brazilian Symposium on Artificial Intelligence (SBIA \u201998) (pp. 219\u2013228).","DOI":"10.1007\/10692710_23"},{"key":"9560_CR29","doi-asserted-by":"crossref","unstructured":"Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S. & McClosky, D. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of the demo track of the 52nd annual meeting of the Association for Computational Linguistics (ACL 2014) (pp. 55\u201360).","DOI":"10.3115\/v1\/P14-5010"},{"key":"9560_CR30","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1007\/978-3-319-43610-4_5","volume-title":"Computational creativity: The philosophy and engineering of autonomously creative systems","author":"P Martins","year":"2019","unstructured":"Martins, P., Pereira, F. C., & Cardoso, A. F. (2019). The nuts and bolts of conceptual blending: Multidomain concept creation with Divago. In T. Veale & F. A. Cardoso (Eds.), Computational creativity: The philosophy and engineering of autonomously creative systems (pp. 91\u2013119). Cham: Springer. https:\/\/doi.org\/10.1007\/978-3-319-43610-4_5"},{"key":"9560_CR31","unstructured":"Mihalcea, R. Strapparava, C. (2012). Lyrics, music, and emotions. In Proceedings of the 2012 joint conference on Empirical Methods on Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL \u201912) (pp. 590\u2013599)."},{"key":"9560_CR32","unstructured":"Mikolov, T , Sutskever, I., Chen, K., Corrado, G. S. & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th conference on advances in Neural Information Processing Systems (NIPS 2013) (pp. 3111\u20133119)."},{"key":"9560_CR33","doi-asserted-by":"crossref","unstructured":"Munigala, V., Mishra, A., Tamilselvam, S. G., Khare, S., Dasgupta, R. & Sankaran, A. (2018). PersuAIDE an adaptive persuasive text generation system for fashion domain. In Proceedings of ACM the Web conference (WWW \u201918) (pp. 335\u2013342).","DOI":"10.1145\/3184558.3186345"},{"key":"9560_CR35","doi-asserted-by":"crossref","unstructured":"\u00d6zbal, G. & Strapparava, C. (2013). Namelette: A tasteful supporter for creative naming. In Companion publication of the 2013 international conference on Intelligent User Interfaces (IUI \u201913) (pp. 55\u201356).","DOI":"10.1145\/2451176.2451196"},{"key":"9560_CR34","unstructured":"\u00d6zbal, G., Pighin, D. & Strapparava, C. (2013). BrainSup: Brainstorming support for creative sentence generation. In Proceedings of the 51st annual meeting of the Association for Computational Linguistics (ACL 2013) (pp. 1446\u20131455)."},{"key":"9560_CR36","volume-title":"English GigaWord fifth edition, LDC2011T07 DVD","author":"R Parker","year":"2011","unstructured":"Parker, R., Graff, D., Kong, J., Chen, K., & Maeda, K. (2011). English GigaWord fifth edition, LDC2011T07 DVD. Linguistic Data Consortium."},{"key":"9560_CR37","doi-asserted-by":"crossref","unstructured":"P\u00e9rez y P\u00e9rez, R. & Sharples, M. (2004). Three computer-based models of storytelling: BRUTUS, MINSTREL and MEXICA. Knowledge-based systems, 17(1), 15\u201329.","DOI":"10.1016\/S0950-7051(03)00048-0"},{"key":"9560_CR38","unstructured":"Pianta, E., Girardi, C. & Zanoli, R. (2008). The TextPro tool suite. In Proceedings of the 6th international conference Language Resources and Evaluation\u2013 (LREC 08) (pp. 2603\u20132607)."},{"issue":"1","key":"9560_CR39","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1016\/S0004-3702(02)00370-3","volume":"144","author":"E Reiter","year":"2003","unstructured":"Reiter, E., Robertson, R., & Osman, L. M. (2003). Lessons from a failure: Generating tailored smoking cessation letters. Artificial Intelligence, 144(1), 41\u201358.","journal-title":"Artificial Intelligence"},{"issue":"1","key":"9560_CR40","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1007\/s11023-007-9066-2","volume":"17","author":"GD Ritchie","year":"2007","unstructured":"Ritchie, G. D. (2007). Some empirical criteria for attributing creativity to a computer program. Minds and Machines, 17(1), 67\u201399.","journal-title":"Minds and Machines"},{"issue":"2","key":"9560_CR41","first-page":"67","volume":"21","author":"GD Ritchie","year":"2006","unstructured":"Ritchie, G. D., Manurung, R., Pain, H., Waller, A., & O\u2019Mara, D. (2006). The STANDUP interactive riddle builder. IEEE Intelligent Systems, 21(2), 67\u201369.","journal-title":"IEEE Intelligent Systems"},{"key":"9560_CR42","volume-title":"The CMU pronouncing dictionary release 07b","author":"A Rudnicky","year":"2014","unstructured":"Rudnicky, A. (2014). The CMU pronouncing dictionary release 07b. Carnegie Mellon University."},{"key":"9560_CR43","unstructured":"SPIN staff. (2007). Alanis Morissette \u2018My Humps\u2019 Video. SPIN. Retrieved March, 10, 2021, from https:\/\/www.spin.com\/2007\/04\/alanis-morissette-my-humps-video\/"},{"key":"9560_CR44","unstructured":"Stock, O. Strapparava, C. (2003). Getting serious about the development of computational humor. In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI-03) (Vol.\u00a03, pp. 59\u201364)."},{"key":"9560_CR45","unstructured":"Toma\u0161ic, P., Znidar\u0161ic, M. & Papa, G. (2014). Implementation of a slogan generator. In Proceedings of 5th International Conference on Computational Creativity (ICCC 2014) (pp. 340\u2013343)."},{"key":"9560_CR46","volume-title":"The Oxford thesaurus: An AZ dictionary of synonyms","author":"L Urdang","year":"1993","unstructured":"Urdang, L. (1993). The Oxford thesaurus: An AZ dictionary of synonyms. Clarendon Press."},{"key":"9560_CR47","unstructured":"Veale, T. (2014). Coming good and breaking bad: Generating transformative character arcs for use in compelling stories. In Proceedings of the 5th International Conference on Computational Creativity (ICCC 2014)."},{"key":"9560_CR48","doi-asserted-by":"crossref","unstructured":"Veale, T. (2016) Round up the usual suspects: Knowledge-based metaphor generation. In Proceedings of the 4th workshop on Metaphor in NLP (Metaphor 2016) (pp. 34\u201341).","DOI":"10.18653\/v1\/W16-1105"},{"key":"9560_CR49","doi-asserted-by":"publisher","unstructured":"Veale, T. (2019). From conceptual mash-ups to badass blends: A robust computational model of conceptual blending. In T.\u00a0Veale & F. A.\u00a0Cardoso (Eds.), Computational creativity: The philosophy and engineering of autonomously creative systems (pp. 71\u201389). Springer. https:\/\/doi.org\/10.1007\/978-3-319-43610-4\\_4","DOI":"10.1007\/978-3-319-43610-4\\_4"},{"key":"9560_CR50","doi-asserted-by":"publisher","unstructured":"Williams, S. & Reiter, E. (2008). Generating basic skills reports for low-skilled readers. Natural Language Engineering, 14(4), 495\u2013525. https:\/\/doi.org\/10.1017\/S1351324908004725","DOI":"10.1017\/S1351324908004725"},{"key":"9560_CR51","doi-asserted-by":"crossref","unstructured":"Xu, S., Yang, S. & Lau, F. C. M. (2010). Keyword extraction and headline generation using novel word features. In Proceedings of the 24th conference on Artificial Intelligence (AAAI-10) (pp. 1461\u20131466).","DOI":"10.1609\/aaai.v24i1.7511"},{"key":"9560_CR52","unstructured":"Zhou, L. & Hovy, E. (2003). Headline summarization at ISI. In Proceedings of the Document Understanding Conference (DUC-2003)."}],"container-title":["Language Resources and Evaluation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-021-09560-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10579-021-09560-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-021-09560-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T03:42:19Z","timestamp":1673235739000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10579-021-09560-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,14]]},"references-count":52,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,6]]}},"alternative-id":["9560"],"URL":"https:\/\/doi.org\/10.1007\/s10579-021-09560-6","relation":{},"ISSN":["1574-020X","1574-0218"],"issn-type":[{"type":"print","value":"1574-020X"},{"type":"electronic","value":"1574-0218"}],"subject":[],"published":{"date-parts":[[2021,9,14]]},"assertion":[{"value":"2 September 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 September 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}