{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T20:06:34Z","timestamp":1783454794754,"version":"3.55.0"},"reference-count":103,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2023,8,1]],"date-time":"2023-08-01T00:00:00Z","timestamp":1690848000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,1]],"date-time":"2023-08-01T00:00:00Z","timestamp":1690848000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["JP19K20289"],"award-info":[{"award-number":["JP19K20289"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["JP18H03339"],"award-info":[{"award-number":["JP18H03339"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004721","name":"The University of Tokyo","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004721","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Multimed Tools Appl"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Creating impressive video content such as movies and advertisements is a very important yet challenging task in business that requires both a sense of creativity and a lot of experience. Even professionals cannot necessarily invoke the impressions and emotions that they have aimed at. Many video advertisements are created and then disappear without giving a large impact on viewers. This paper presents a large-scale dataset of television (TV) advertisements that consists of 14,490 videos. The impressions of each video such as the recognition rate and interestingness rate are from the results of questionnaires answered by 620 participants. We also present a baseline for predicting the impression effects of TV advertisements by using visual and audio information, metadata such as broadcasting pattern, business category, the popularity of the casts, and text information including texts appearing on videos and narrations in audios. We predict four impressions of the viewers: 1) how much participants remember the video afterward, 2) how much they feel like buying the product\/service, 3) how much they become interested in the product\/service, and 4) how much they like the content of the advertisement itself. By combining images, audio, metadata, cast data, and text data, our baseline method is able to predict such impressions with a correlation of 0.69-0.82, much better than using a single-modal feature such as visual data or audio data only. This paper also gives some possible applications such as estimating the importance scores of each key frame, which gives us informative insights about how to make the advertisement content more impressive.<\/jats:p>","DOI":"10.1007\/s11042-023-14704-7","type":"journal-article","created":{"date-parts":[[2023,8,1]],"date-time":"2023-08-01T08:02:29Z","timestamp":1690876949000},"page":"18779-18802","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A large-scale television advertising dataset for detailed impression analysis"],"prefix":"10.1007","volume":"83","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8758-0471","authenticated-orcid":false,"given":"Li","family":"Tao","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shunsuke","family":"Nakamura","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xueting","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tatsuya","family":"Kawahara","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gen","family":"Tamura","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Toshihiko","family":"Yamasaki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,8,1]]},"reference":[{"key":"14704_CR1","unstructured":"Aaker DA, Norris D (1982) Characteristics of tv commercials perceived as informative. J Advert Res"},{"key":"14704_CR2","doi-asserted-by":"crossref","unstructured":"Adamov AZ, Adali E (2016) Opinion mining and sentiment analysis for contextual online-advertisement. In: 2016 IEEE 10Th international conference on application of information and communication technologies (AICT), IEEE, pp 1\u20133","DOI":"10.1109\/ICAICT.2016.7991682"},{"key":"14704_CR3","doi-asserted-by":"publisher","first-page":"16173","DOI":"10.1109\/ACCESS.2017.2690342","volume":"5","author":"T Al-Moslmi","year":"2017","unstructured":"Al-Moslmi T, Omar N, Abdullah S, Albared M (2017) Approaches to cross-domain sentiment analysis: a systematic literature review. IEEE Access 5:16173\u201316192","journal-title":"IEEE Access"},{"key":"14704_CR4","doi-asserted-by":"crossref","unstructured":"Aytar Y, Vondrick C, Torralba A (2016) Soundnet: Learning sound representations from unlabeled video. Adv Neural Inf Process Syst 29","DOI":"10.1109\/CVPR.2016.18"},{"issue":"Supplement C","key":"14704_CR5","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1016\/j.neuroimage.2017.01.063","volume":"149","author":"WA Bainbridge","year":"2017","unstructured":"Bainbridge WA, Dilks DD, Oliva A (2017) Memorability: a stimulus-driven perceptual neural signature distinctive from memory. Neuroimage 149 (Supplement C):141\u2013152","journal-title":"Neuroimage"},{"key":"14704_CR6","unstructured":"Bakshi RK, Kaur N, Kaur R, Kaur G (2016) Opinion mining and sentiment analysis. In: 2016 3rd international conference on computing for sustainable global development (INDIACom), pp 452\u2013455"},{"issue":"4","key":"14704_CR7","doi-asserted-by":"publisher","first-page":"396","DOI":"10.1109\/TAFFC.2017.2661284","volume":"9","author":"Y Baveye","year":"2017","unstructured":"Baveye Y, Chamaret C, Dellandr\u00e9a E, Chen L (2017) Affective video content analysis: a multidisciplinary insight. IEEE Trans Affect Comput 9(4):396\u2013409","journal-title":"IEEE Trans Affect Comput"},{"key":"14704_CR8","doi-asserted-by":"crossref","unstructured":"Bazzani L, Bergamo A, Anguelov D, Torresani L (2016) Self-taught object localization with deep networks. In: 2016 IEEE Winter conference on applications of computer vision (WACV), IEEE, pp 1\u20139","DOI":"10.1109\/WACV.2016.7477688"},{"key":"14704_CR9","doi-asserted-by":"crossref","unstructured":"Borth D, Ji R, Chen T, Breuel T, Chang SF (2013) Large-scale visual sentiment ontology and detectors using adjective noun pairs. In: Proceedings of the 21st ACM international conference on Multimedia, pp 223\u2013232","DOI":"10.1145\/2502081.2502282"},{"key":"14704_CR10","doi-asserted-by":"crossref","unstructured":"Bylinskii Z, Kim NW, O\u2019Donovan P, Alsheikh S, Madan S, Pfister H, Durand F, Russell B, Hertzmann A (2017) Learning visual importance for graphic designs and data visualizations. In: Proceedings of the 30th Annual ACM symposium on user interface software and technology, pp 57\u201369","DOI":"10.1145\/3126594.3126653"},{"key":"14704_CR11","doi-asserted-by":"crossref","unstructured":"Bylinskii Z, Kim NW, O\u2019Donovan P, Alsheikh S, Madan S, Pfister H, Durand F, Russell B, Hertzmann A (2017) Learning visual importance for graphic designs and data visualizations. In: Proceedings of the 30th Annual ACM symposium on user interface software and technology, pp 57\u201369","DOI":"10.1145\/3126594.3126653"},{"key":"14704_CR12","doi-asserted-by":"crossref","unstructured":"Cakir E, Heittola T, Huttunen H, Virtanen T (2015) Polyphonic sound event detection using multi label deep neural networks. In: 2015 International joint conference on neural networks (IJCNN), IEEE, pp 1\u20137","DOI":"10.1109\/IJCNN.2015.7280624"},{"key":"14704_CR13","doi-asserted-by":"crossref","unstructured":"Carreira J, Zisserman A (2017) Quo vadis, action recognition? a new model and the kinetics dataset. In: proceedings of the IEEE conference on computer vision and pattern Recognition (CVPR), pp 6299\u20136308","DOI":"10.1109\/CVPR.2017.502"},{"key":"14704_CR14","doi-asserted-by":"crossref","unstructured":"Chae Y, Nakazawa M, Stenger B (2018) Enhancing product images for click-through rate improvement. In: 2018 25Th IEEE international conference on image processing (ICIP), IEEE, pp 1428\u20131432","DOI":"10.1109\/ICIP.2018.8451513"},{"key":"14704_CR15","unstructured":"Chen H, Li J, Zhang F, Li Y, Wang H (2015) 3d model-based continuous emotion recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1836\u20131845"},{"key":"14704_CR16","doi-asserted-by":"crossref","unstructured":"Chen J, Sun B, Li H, Lu H, Hua XS (2016) Deep ctr prediction in display advertising. In: Proceedings of the 24th ACM international conference on multimedia, p 811\u2013820","DOI":"10.1145\/2964284.2964325"},{"issue":"9","key":"14704_CR17","doi-asserted-by":"publisher","first-page":"2701","DOI":"10.1109\/TIP.2015.2421437","volume":"24","author":"P Chiranjeevi","year":"2015","unstructured":"Chiranjeevi P, Gopalakrishnan V, Moogi P (2015) Neutral face classification using personalized appearance models for fast and robust emotion detection. IEEE Trans Image Process 24(9):2701\u20132711","journal-title":"IEEE Trans Image Process"},{"key":"14704_CR18","doi-asserted-by":"crossref","unstructured":"Cohendet R, Demarty CH, Duong NQK, Engilberge M (2019) Videomem: Constructing, analyzing, predicting short-term and long-term video memorability. In: Proceedings of the IEEE\/CVF international conference on computer vision (ICCV)","DOI":"10.1109\/ICCV.2019.00262"},{"key":"14704_CR19","doi-asserted-by":"crossref","unstructured":"Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE Computer society conference on computer vision and pattern recognition, vol. 1. IEEE, pp 886\u2013893","DOI":"10.1109\/CVPR.2005.177"},{"key":"14704_CR20","unstructured":"Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies (naacl-hlt), association for computational linguistics, pp 4171\u20134186"},{"key":"14704_CR21","doi-asserted-by":"crossref","unstructured":"Dubey R, Peterson J, Khosla A, Yang MH, Ghanem B (2015) What makes an object memorable?. In: Proceedings of the ieee international conference on computer vision, pp 1089\u20131097","DOI":"10.1109\/ICCV.2015.130"},{"issue":"5","key":"14704_CR22","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1109\/MIS.2017.3711649","volume":"32","author":"M Ebrahimi","year":"2017","unstructured":"Ebrahimi M, Yazdavar AH, Sheth A (2017) Challenges of sentiment analysis for dynamic events. IEEE Intell Syst 32(5):70\u201375","journal-title":"IEEE Intell Syst"},{"key":"14704_CR23","doi-asserted-by":"crossref","unstructured":"Ekman P, Friesen WV, O\u2019sullivan M, Chan A, Diacoyanni-Tarlatzis I, Heider K, et al. (1987) Universals and cultural differences in the judgments of facial expressions of emotion. In: Journal of personality and social psychology, vol 53. American psychological association","DOI":"10.1037\/\/0022-3514.53.4.712"},{"key":"14704_CR24","doi-asserted-by":"crossref","unstructured":"Eyben F, Weninger F, Gross F, Schuller B (2013) Recent developments in opensmile, the munich open-source multimedia feature extractor. In: Proceedings of the 21st ACM international conference on Multimedia, pp 835\u2013838","DOI":"10.1145\/2502081.2502224"},{"key":"14704_CR25","doi-asserted-by":"crossref","unstructured":"Fabian Benitez-Quiroz C, Srinivasan R, Martinez AM (2016) Emotionet: an accurate, real-time algorithm for the automatic annotation of a million facial expressions in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5562\u20135570","DOI":"10.1109\/CVPR.2016.600"},{"key":"14704_CR26","doi-asserted-by":"crossref","unstructured":"Fajtl J, Argyriou V, Monekosso D, Remagnino P (2018) Amnet: Memorability estimation with attention. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6363\u20136372","DOI":"10.1109\/CVPR.2018.00666"},{"key":"14704_CR27","doi-asserted-by":"crossref","unstructured":"Fei M, Jiang W, Mao W (2017) Creating memorable video summaries that satisfy the user fs intention for taking the videos. Neurocomputing","DOI":"10.1016\/j.neucom.2017.10.030"},{"key":"14704_CR28","doi-asserted-by":"crossref","unstructured":"Fire M, Schler J (2017) Exploring online ad images using a deep convolutional neural network approach. In: 2017 IEEE International conference on internet of things (ithings) and IEEE green computing and communications (greencom) and IEEE cyber, physical and social computing (CPSCom) and IEEE smart data (smartdata)","DOI":"10.1109\/iThings-GreenCom-CPSCom-SmartData.2017.160"},{"key":"14704_CR29","unstructured":"Gregov J (2008) The impact of actor presence in product placements on brand and ad attitudes and purchase intention, and the role of recognition and recall. In: Annual meeting of the NCA 94th annual convention"},{"issue":"8","key":"14704_CR30","doi-asserted-by":"publisher","first-page":"1692","DOI":"10.1109\/TCYB.2014.2358647","volume":"45","author":"J Han","year":"2014","unstructured":"Han J, Chen C, Shao L, Hu X, Han J, Liu T (2014) Learning computational models of video memorability from fmri brain imaging. IEEE transactions on cybernetics 45(8):1692\u20131703","journal-title":"IEEE transactions on cybernetics"},{"key":"14704_CR31","unstructured":"Hannun A, Case C, Casper J, Catanzaro B, Diamos G, Elsen E, et al. (2014) Deep speech:, Scaling up end-to-end speech recognition. arXiv:1412.5567"},{"key":"14704_CR32","doi-asserted-by":"crossref","unstructured":"Hara K, Kataoka H, Satoh Y (2018) Can spatiotemporal 3d CNNS retrace the history of 2d CNNS and imagenet. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, pp 18\u201322","DOI":"10.1109\/CVPR.2018.00685"},{"key":"14704_CR33","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"14704_CR34","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"14704_CR35","doi-asserted-by":"crossref","unstructured":"He X, Deng K, Wang X, Li Y, Zhang Y, Wang M (2020) Lightgcn: Simplifying and powering graph convolution network for recommendation. In: Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp 639\u2013648","DOI":"10.1145\/3397271.3401063"},{"key":"14704_CR36","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1016\/j.patcog.2017.03.003","volume":"68","author":"S Hou","year":"2017","unstructured":"Hou S, Chen L, Tao D, Zhou S, Liu W, Zheng Y (2017) Multi-layer multi-view topic model for classifying advertising video. Pattern Recogn 68:66\u201381","journal-title":"Pattern Recogn"},{"key":"14704_CR37","doi-asserted-by":"crossref","unstructured":"Hussain Z, Zhang M, Zhang X, Ye K, Thomas C, Agha Z, Ong N, Kovashka A (2017) Automatic understanding of image and video advertisements. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1705\u20131715","DOI":"10.1109\/CVPR.2017.123"},{"key":"14704_CR38","doi-asserted-by":"crossref","unstructured":"Hussain Z, Zhang M, Zhang X, Ye K, Thomas C, Agha Z, Ong N, Kovashka A (2017) Automatic understanding of image and video advertisements. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1705\u20131715","DOI":"10.1109\/CVPR.2017.123"},{"key":"14704_CR39","doi-asserted-by":"crossref","unstructured":"Hutto C, Gilbert E (2014) Vader: a parsimonious rule-based model for sentiment analysis of social media text. In: Proceedings of the international AAAI conference on web and social media, vol 8","DOI":"10.1609\/icwsm.v8i1.14550"},{"key":"14704_CR40","doi-asserted-by":"crossref","unstructured":"Ikeda J, Seshime H, Wang X, Yamasaki T (2021) Predicting online video advertising effects with multimodal deep learning. In: International conference on pattern recognition, pp 2995\u20133002","DOI":"10.1109\/ICPR48806.2021.9412666"},{"issue":"6","key":"14704_CR41","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1109\/TMM.2010.2051871","volume":"12","author":"G Irie","year":"2010","unstructured":"Irie G, Satou T, Kojima A, Yamasaki T, Aizawa K (2010) Affective audio-visual words and latent topic driving model for realizing movie affective scene classification. IEEE Trans Multimedia 12(6):523\u2013535","journal-title":"IEEE Trans Multimedia"},{"issue":"1","key":"14704_CR42","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1109\/TPAMI.2012.59","volume":"35","author":"S Ji","year":"2012","unstructured":"Ji S, Xu W, Yang M (2012) Yu, k.: 3d convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell 35 (1):221\u2013231","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"14704_CR43","doi-asserted-by":"publisher","unstructured":"Kar A, Mavin P, Ghaturle Y, Vani M (2017) What makes a video memorable?. In: 2017 IEEE International conference on data science and advanced analytics (DSAA), pp 373\u2013381, https:\/\/doi.org\/10.1109\/DSAA.2017.37","DOI":"10.1109\/DSAA.2017.37"},{"issue":"8","key":"14704_CR44","doi-asserted-by":"publisher","first-page":"1668","DOI":"10.1109\/TASLP.2017.2678164","volume":"25","author":"L Kaushik","year":"2017","unstructured":"Kaushik L, Sangwan A, Hansen JH (2017) Automatic sentiment detection in naturalistic audio. IEEE\/ACM Trans Audio Speech Lang Process 25 (8):1668\u20131679","journal-title":"IEEE\/ACM Trans Audio Speech Lang Process"},{"key":"14704_CR45","unstructured":"Kay W, Carreira J, Simonyan K, Zhang B, Hillier C, Vijayanarasimhan S, et al. (2017) The kinetics human action video dataset. arXiv:1705.06950 1\u201322"},{"key":"14704_CR46","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1016\/j.imavis.2017.01.012","volume":"65","author":"H Kaya","year":"2017","unstructured":"Kaya H, G\u00fcrp\u0131nar F, Salah AA (2017) Video-based emotion recognition in the wild using deep transfer learning and score fusion. Image Vision Comput 65:66\u201375","journal-title":"Image Vision Comput"},{"key":"14704_CR47","doi-asserted-by":"crossref","unstructured":"Khosla A, Raju AS, Torralba A, Oliva A (2015) Understanding and predicting image memorability at a large scale. In: Proceedings of the IEEE international conference on computer vision, pp 2390\u20132398","DOI":"10.1109\/ICCV.2015.275"},{"key":"14704_CR48","doi-asserted-by":"crossref","unstructured":"Kim S, Patra KAE, Kim A, Lee KP, Segev A, Lee U (2017) Sensors know which photos are memorable. In: Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems, pp 2706\u20132713","DOI":"10.1145\/3027063.3053198"},{"key":"14704_CR49","doi-asserted-by":"crossref","unstructured":"Kosti R, Alvarez JM, Recasens A, Lapedriza A (2017) Emotic: Emotions in context dataset. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 61\u201369","DOI":"10.1109\/CVPRW.2017.285"},{"key":"14704_CR50","doi-asserted-by":"crossref","unstructured":"Kosti R, Alvarez JM, Recasens A, Lapedriza A (2017) Emotion recognition in context. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1667\u20131675","DOI":"10.1109\/CVPR.2017.212"},{"key":"14704_CR51","doi-asserted-by":"crossref","unstructured":"Kuehne H, Jhuang H, Garrote E, Poggio T, Serre T (2011) Hmdb: a large video database for human motion recognition. In: International conference on computer vision (ICCV), IEEE, pp 2556\u20132563","DOI":"10.1109\/ICCV.2011.6126543"},{"issue":"10","key":"14704_CR52","doi-asserted-by":"publisher","first-page":"2336","DOI":"10.1109\/TCYB.2015.2392156","volume":"45","author":"T Li","year":"2015","unstructured":"Li T, Ni B, Xu M, Wang M, Gao Q, Yan S (2015) Data-driven affective filtering for images and videos. IEEE Trans Cybern 45(10):2336\u20132349","journal-title":"IEEE Trans Cybern"},{"key":"14704_CR53","doi-asserted-by":"crossref","unstructured":"Liang Y, Liu W, Liu K, Ma H (2018) Automatic generation of textual advertisement for video advertising. In: 2018 IEEE Fourth international conference on multimedia big data (bigMM), IEEE, pp 1\u20135","DOI":"10.1109\/BigMM.2018.8499465"},{"key":"14704_CR54","first-page":"289","volume":"29","author":"J Lu","year":"2016","unstructured":"Lu J, Yang J, Batra D, Parikh D (2016) Hierarchical question-image co-attention for visual question answering. Adv Neural Inf Process Syst 29:289\u2013297","journal-title":"Adv Neural Inf Process Syst"},{"key":"14704_CR55","doi-asserted-by":"crossref","unstructured":"Lu X, Lin Z, Jin H, Yang J, Wang JZ (2014) Rapid: Rating pictorial aesthetics using deep learning. In: Proceedings of the 22nd ACM international conference on Multimedia, pp 457\u2013466","DOI":"10.1145\/2647868.2654927"},{"key":"14704_CR56","doi-asserted-by":"crossref","unstructured":"Mathews A, Xie L, He X (2016) Senticap: Generating image descriptions with sentiments. In: Proceedings of the AAAI conference on artificial intelligence, vol 30","DOI":"10.1609\/aaai.v30i1.10475"},{"issue":"3","key":"14704_CR57","doi-asserted-by":"publisher","first-page":"314","DOI":"10.1109\/TAFFC.2016.2531664","volume":"8","author":"A Mencattini","year":"2016","unstructured":"Mencattini A, Martinelli E, Ringeval F, Schuller B, Di Natale C (2016) Continuous estimation of emotions in speech by dynamic cooperative speaker models. IEEE Trans Affect Comput 8(3):314\u2013327","journal-title":"IEEE Trans Affect Comput"},{"issue":"1","key":"14704_CR58","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1109\/TAFFC.2017.2713783","volume":"10","author":"F Noroozi","year":"2017","unstructured":"Noroozi F, Marjanovic M, Njegus A, Escalera S, Anbarjafari G (2017) Audio-visual emotion recognition in video clips. IEEE Trans Affect Comput 10(1):60\u201375","journal-title":"IEEE Trans Affect Comput"},{"key":"14704_CR59","doi-asserted-by":"crossref","unstructured":"Nwana AO, Avestimehr S, Chen T (2013) A latent social approach to youtube popularity prediction. In: 2013 IEEE Global communications conference (GLOBECOM), IEEE, pp 3138\u20133144","DOI":"10.1109\/GLOCOM.2013.6831554"},{"issue":"1","key":"14704_CR60","first-page":"131","volume":"6","author":"G Okada","year":"2018","unstructured":"Okada G, Yonezawa T, Kurita K, Tsumura N (2018) Monitoring emotion by remote measurement of physiological signals using an rgb camera. ITE Trans Media Technol Appl 6(1):131\u2013137","journal-title":"ITE Trans Media Technol Appl"},{"key":"14704_CR61","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1016\/j.neucom.2018.12.025","volume":"332","author":"Y Pan","year":"2019","unstructured":"Pan Y, He F, Yu H (2019) A novel enhanced collaborative autoencoder with knowledge distillation for top-n recommender systems. Neurocomputing 332:137\u2013148","journal-title":"Neurocomputing"},{"issue":"3","key":"14704_CR62","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11704-019-8123-3","volume":"14","author":"Y Pan","year":"2020","unstructured":"Pan Y, He F, Yu H (2020) A correlative denoising autoencoder to model social influence for top-n recommender system. Front Comput Sci 14(3):1\u201313","journal-title":"Front Comput Sci"},{"issue":"4","key":"14704_CR63","doi-asserted-by":"publisher","first-page":"2259","DOI":"10.1007\/s11280-020-00793-z","volume":"23","author":"Y Pan","year":"2020","unstructured":"Pan Y, He F, Yu H (2020) Learning social representations with deep autoencoder for recommender system. World Wide Web 23(4):2259\u20132279","journal-title":"World Wide Web"},{"key":"14704_CR64","doi-asserted-by":"crossref","unstructured":"Pang B, Lee L, Vaithyanathan S (2002) Thumbs up?: Sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 conference on empirical methods in natural language processing, pp 79\u201386","DOI":"10.3115\/1118693.1118704"},{"key":"14704_CR65","first-page":"103","volume":"107116","author":"KW Park","year":"2021","unstructured":"Park KW, Ha JW, Lee J, Kwon S, Kim KM, Zhang BT (2021) M2fn: Multi-step modality fusion for advertisement image assessment. Appl Soft Comput 107116:103","journal-title":"Appl Soft Comput"},{"key":"14704_CR66","doi-asserted-by":"crossref","unstructured":"Peng KC, Chen T, Sadovnik A, Gallagher AC (2015) A mixed bag of emotions: Model, predict, and transfer emotion distributions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 860\u2013868","DOI":"10.1109\/CVPR.2015.7298687"},{"key":"14704_CR67","doi-asserted-by":"crossref","unstructured":"Perera S, Tal A, Zelnik-Manor L (2019) Is image memorability prediction solved?. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR) workshops","DOI":"10.1109\/CVPRW.2019.00108"},{"issue":"4","key":"14704_CR68","doi-asserted-by":"publisher","first-page":"344","DOI":"10.1511\/2001.28.344","volume":"89","author":"R Plutchik","year":"2001","unstructured":"Plutchik R (2001) The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. Am Sci 89(4):344\u2013350","journal-title":"Am Sci"},{"key":"14704_CR69","doi-asserted-by":"crossref","unstructured":"Qiu Z, Yao T, Mei T (2017) Learning spatio-temporal representation with pseudo-3d residual networks. In: Proceedings of the IEEE international conference on computer vision, pp 5533\u20135541","DOI":"10.1109\/ICCV.2017.590"},{"key":"14704_CR70","doi-asserted-by":"crossref","unstructured":"Redi M, O\u2019Hare N, Schifanella R, Trevisiol M, Jaimes A (2014) 6 seconds of sound and vision: Creativity in micro-videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4272\u20134279","DOI":"10.1109\/CVPR.2014.544"},{"issue":"3","key":"14704_CR71","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211\u2013252","journal-title":"Int J Comput Vis"},{"issue":"12","key":"14704_CR72","doi-asserted-by":"publisher","first-page":"6006","DOI":"10.1109\/TIP.2017.2726010","volume":"26","author":"B Ryu","year":"2017","unstructured":"Ryu B, Rivera AR, Kim J, Chae O (2017) Local directional ternary pattern for facial expression recognition. IEEE Trans Image Process 26 (12):6006\u20136018","journal-title":"IEEE Trans Image Process"},{"key":"14704_CR73","doi-asserted-by":"crossref","unstructured":"Salamon J, Jacoby C, Bello JP (2014) A dataset and taxonomy for urban sound research. In: Proceedings of the 22nd ACM international conference on Multimedia, pp 1041\u20131044","DOI":"10.1145\/2647868.2655045"},{"key":"14704_CR74","unstructured":"Saleh B, Dontcheva M, Hertzmann A, Liu Z (2015) Learning style similarity for searching infographics. In: Proceedings of the 41st graphics interface conference, canadian information processing society, pp 59\u201364"},{"key":"14704_CR75","doi-asserted-by":"crossref","unstructured":"Shekhar S, Singal D, Singh H, Kedia M, Shetty A (2017) Show and recall: Learning what makes videos memorable. In: Proceedings of the IEEE international conference on computer vision workshops, pp 2730\u20132739","DOI":"10.1109\/ICCVW.2017.321"},{"key":"14704_CR76","doi-asserted-by":"crossref","unstructured":"Shukla A, Gullapuram SS, Katti H, Yadati K, Kankanhalli M, Subramanian R (2017) Affect recognition in ads with application to computational advertising. In: Proceedings of the 25th ACM international conference on Multimedia, pp 1148\u20131156","DOI":"10.1145\/3123266.3123444"},{"key":"14704_CR77","doi-asserted-by":"crossref","unstructured":"Shukla A, Gullapuram SS, Katti H, Yadati K, Kankanhalli M, Subramanian R (2017) Evaluating content-centric vs. user-centric ad affect recognition. In: Proceedings of the 19th ACM international conference on multimodal interaction, pp 402\u2013410","DOI":"10.1145\/3136755.3136796"},{"key":"14704_CR78","doi-asserted-by":"crossref","unstructured":"Siarohin A, Zen G, Majtanovic C, Alameda-Pineda X, Ricci E, Sebe N (2017) How to make an image more memorable? a deep style transfer approach proceedings of the 2017 ACM on international conference on Multimedia Retrieval, pp 322-329. Association for Computing Machinery, New York","DOI":"10.1145\/3078971.3078986"},{"key":"14704_CR79","unstructured":"Soomro K, Zamir AR, Shah M (2012). arXiv: preprint1212.0402 pp 1\u20137"},{"issue":"3","key":"14704_CR80","doi-asserted-by":"publisher","first-page":"513","DOI":"10.1109\/TMM.2017.2751969","volume":"20","author":"N Takahashi","year":"2017","unstructured":"Takahashi N, Gygli M, Van Gool L (2017) Aenet: Learning deep audio features for video analysis. IEEE Trans Multimedia 20(3):513\u2013524","journal-title":"IEEE Trans Multimedia"},{"key":"14704_CR81","doi-asserted-by":"publisher","unstructured":"Tao L, Wang X, Yamasaki T (2020) Motion representation using residual frames with 3d CNN. In: 2020 IEEE International conference on image processing (ICIP), pp 1786\u20131790, https:\/\/doi.org\/10.1109\/ICIP40778.2020.9191133","DOI":"10.1109\/ICIP40778.2020.9191133"},{"key":"14704_CR82","doi-asserted-by":"publisher","first-page":"9231","DOI":"10.1109\/TIP.2021.3124156","volume":"30","author":"L Tao","year":"2021","unstructured":"Tao L, Wang X, Yamasaki T (2021) Rethinking motion representation: Residual frames with 3d convnets. IEEE Trans Image Process 30:9231\u20139244. https:\/\/doi.org\/10.1109\/TIP.2021.3124156","journal-title":"IEEE Trans Image Process"},{"key":"14704_CR83","doi-asserted-by":"crossref","unstructured":"Tran D, Bourdev L, Fergus R, Torresani L, Paluri M (2015) Learning spatiotemporal features with 3d convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 4489\u20134497","DOI":"10.1109\/ICCV.2015.510"},{"key":"14704_CR84","doi-asserted-by":"crossref","unstructured":"Tran D, Wang H, Torresani L, Ray J, LeCun Y, Paluri M (2018) A closer look at spatiotemporal convolutions for action recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6450\u20136459","DOI":"10.1109\/CVPR.2018.00675"},{"key":"14704_CR85","unstructured":"Van den Oord A, Dieleman S, Schrauwen B (2013) Deep content-based music recommendation. Adv Neural Inf Process Sys 26"},{"key":"14704_CR86","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30"},{"key":"14704_CR87","unstructured":"Vaughn R (1980) How advertising works: A planning model Advertising research"},{"issue":"3","key":"14704_CR88","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1109\/TAFFC.2015.2415212","volume":"6","author":"JC Wang","year":"2015","unstructured":"Wang JC, Lee YS, Chin YH, Chen YR, Hsieh WC (2015) Hierarchical dirichlet process mixture model for music emotion recognition. IEEE Trans Affect Comput 6(3):261\u2013271","journal-title":"IEEE Trans Affect Comput"},{"issue":"1","key":"14704_CR89","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1109\/TAFFC.2015.2392101","volume":"6","author":"K Wang","year":"2015","unstructured":"Wang K, An N, Li BN, Zhang Y, Li L (2015) Speech emotion recognition using fourier parameters. IEEE Trans Affect Comput 6(1):69\u201375","journal-title":"IEEE Trans Affect Comput"},{"key":"14704_CR90","doi-asserted-by":"crossref","unstructured":"Wilber MJ, Fang C, Jin H, Hertzmann A, Collomosse J, Belongie S (2017) Bam! the behance artistic media dataset for recognition beyond photography. In: Proceedings of the IEEE international conference on computer vision, pp 1202\u20131211","DOI":"10.1109\/ICCV.2017.136"},{"issue":"1","key":"14704_CR91","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1142\/S1793351X20400048","volume":"14","author":"B Xia","year":"2020","unstructured":"Xia B, Seshime H, Wang X, Yamasaki T (2020) Click-through rate prediction of online banners featuring multimodal analysis. Int J Semant Comput 14(1):71\u201391","journal-title":"Int J Semant Comput"},{"key":"14704_CR92","doi-asserted-by":"crossref","unstructured":"Xia B, Wang X, Yamasaki T, Aizawa K, Seshime H (2019) Deep neural network-based click-through rate prediction using multimodal features of online banners. In: IEEE International conference on multimedia big data, pp 162\u2013170","DOI":"10.1109\/BigMM.2019.00-29"},{"key":"14704_CR93","doi-asserted-by":"crossref","unstructured":"Xie S, Sun C, Huang J, Tu Z, Murphy K (2018) Rethinking spatiotemporal feature learning: Speed-accuracy trade-offs in video classification. In: Proceedings of the european conference on computer vision (ECCV), pp 305\u2013321","DOI":"10.1007\/978-3-030-01267-0_19"},{"issue":"2","key":"14704_CR94","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1109\/TAFFC.2016.2622690","volume":"9","author":"B Xu","year":"2016","unstructured":"Xu B, Fu Y, Jiang YG, Li B, Sigal L (2016) Heterogeneous knowledge transfer in video emotion recognition, attribution and summarization. IEEE Trans Affect Comput 9(2):255\u2013270","journal-title":"IEEE Trans Affect Comput"},{"issue":"1","key":"14704_CR95","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1109\/TMM.2013.2282128","volume":"16","author":"K Yadati","year":"2013","unstructured":"Yadati K, Katti H, Kankanhalli M (2013) Cavva: Computational affective video-in-video advertising. IEEE Trans Multimed 16(1):15\u201323","journal-title":"IEEE Trans Multimed"},{"issue":"4","key":"14704_CR96","doi-asserted-by":"publisher","first-page":"432","DOI":"10.1109\/TAFFC.2014.2364581","volume":"5","author":"S Yang","year":"2014","unstructured":"Yang S, Kafai M, An L, Bhanu B (2014) Zapping index: using smile to measure advertisement zapping likelihood. IEEE Trans Affect Comput 5 (4):432\u2013444","journal-title":"IEEE Trans Affect Comput"},{"key":"14704_CR97","doi-asserted-by":"crossref","unstructured":"Yashima T, Okazaki N, Inui K, Yamaguchi K, Okatani T (2016) Learning to describe e-commerce images from noisy online data. In: Asian conference on computer vision, Springer, pp 85\u2013100","DOI":"10.1007\/978-3-319-54193-8_6"},{"key":"14704_CR98","doi-asserted-by":"crossref","unstructured":"Ying R, He R, Chen K, Eksombatchai P, Hamilton WL, Leskovec J (2018) Graph convolutional neural networks for web-scale recommender systems. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 974\u2013983","DOI":"10.1145\/3219819.3219890"},{"key":"14704_CR99","doi-asserted-by":"crossref","unstructured":"You Q, Jin H, Luo J (2017) Visual sentiment analysis by attending on local image regions. In: Proceedings of the AAAI conference on artificial intelligence, vol 31","DOI":"10.1609\/aaai.v31i1.10501"},{"issue":"2","key":"14704_CR100","doi-asserted-by":"publisher","first-page":"520","DOI":"10.1109\/TII.2016.2605629","volume":"13","author":"H Zhang","year":"2016","unstructured":"Zhang H, Cao X, Ho JK, Chow TW (2016) Object-level video advertising: an optimization framework. IEEE Trans Ind Inform 13(2):520\u2013531","journal-title":"IEEE Trans Ind Inform"},{"issue":"6","key":"14704_CR101","doi-asserted-by":"publisher","first-page":"1576","DOI":"10.1109\/TMM.2017.2766843","volume":"20","author":"S Zhang","year":"2017","unstructured":"Zhang S, Zhang S, Huang T, Gao W (2017) Speech emotion recognition using deep convolutional neural network and discriminant temporal pyramid matching. IEEE Trans Multimedia 20(6):1576\u20131590","journal-title":"IEEE Trans Multimedia"},{"issue":"4","key":"14704_CR102","first-page":"1","volume":"37","author":"N Zhao","year":"2018","unstructured":"Zhao N, Cao Y, Lau RW (2018) What characterizes personalities of graphic designs? ACM Trans Graph (TOG) 37(4):1\u201315","journal-title":"ACM Trans Graph (TOG)"},{"issue":"4","key":"14704_CR103","doi-asserted-by":"publisher","first-page":"524","DOI":"10.1109\/TAFFC.2017.2747553","volume":"10","author":"Q Zhen","year":"2017","unstructured":"Zhen Q, Huang D, Drira H, Amor BB, Wang Y, Daoudi M (2017) Magnifying subtle facial motions for effective 4d expression recognition. IEEE Trans Affect Comput 10(4):524\u2013536","journal-title":"IEEE Trans Affect Comput"}],"container-title":["Multimedia Tools and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-023-14704-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11042-023-14704-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-023-14704-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,15]],"date-time":"2024-02-15T10:34:01Z","timestamp":1707993241000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11042-023-14704-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,1]]},"references-count":103,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,2]]}},"alternative-id":["14704"],"URL":"https:\/\/doi.org\/10.1007\/s11042-023-14704-7","relation":{},"ISSN":["1573-7721"],"issn-type":[{"value":"1573-7721","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,1]]},"assertion":[{"value":"26 April 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 April 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 February 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 August 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflict of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Conflict of Interests"}}]}}