{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,26]],"date-time":"2025-10-26T14:38:09Z","timestamp":1761489489665},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2014,11,22]],"date-time":"2014-11-22T00:00:00Z","timestamp":1416614400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Multimed Tools Appl"],"published-print":{"date-parts":[[2016,1]]},"DOI":"10.1007\/s11042-014-2369-4","type":"journal-article","created":{"date-parts":[[2014,11,21]],"date-time":"2014-11-21T11:09:49Z","timestamp":1416568189000},"page":"1301-1331","update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Result diversification in social image retrieval: a benchmarking framework"],"prefix":"10.1007","volume":"75","author":[{"given":"Bogdan","family":"Ionescu","sequence":"first","affiliation":[]},{"given":"Adrian","family":"Popescu","sequence":"additional","affiliation":[]},{"given":"Anca-Livia","family":"Radu","sequence":"additional","affiliation":[]},{"given":"Henning","family":"M\u00fcller","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2014,11,22]]},"reference":[{"key":"2369_CR1","doi-asserted-by":"crossref","DOI":"10.1145\/1498759.1498766","volume-title":"Diversifying search results","author":"R Agrawal","year":"2009","unstructured":"Agrawal R, Gollapudi S, Halverson A, Ieong S (2009) Diversifying search results. ACM International Conference on Web Search and Data Mining, Barcelona, Spain"},{"key":"2369_CR2","doi-asserted-by":"crossref","unstructured":"Arandjelovic R, Zisserman AA (2012) Three things everyone should know to improve object retrieval, IEEE Conference on Computer Vision and Pattern Recognition","DOI":"10.1109\/CVPR.2012.6248018"},{"key":"2369_CR3","unstructured":"Armagan A, Popescu A, Duygulu P (2013) MUCKE Participation at Retrieving Diverse Social Images Task of MediaEval 2013, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"key":"2369_CR4","doi-asserted-by":"crossref","unstructured":"Ballan L, Bertini M, Uricchio T, Del Bimbo A (2014) Data-driven approaches for social image and video tagging, Multimedia Tools and Applications, doi: 10.1007\/s11042-014-1976-4","DOI":"10.1007\/s11042-014-1976-4"},{"key":"2369_CR5","unstructured":"Bursuc A, Zaharia T (2013) ARTEMIS @ MediaEval 2013: A Content-Based Image Clustering Method for Public Image Repositories, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"issue":"1","key":"2369_CR6","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1109\/TKDE.2011.190","volume":"25","author":"X-Q Cheng","year":"2013","unstructured":"Cheng X-Q, Du P, Guo J, Zhu X, Chen Y (2013) Ranking on data manifold with sink points. IEEE Trans Knowl Data Eng 25(1):177\u2013191","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"1","key":"2369_CR7","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"XX","author":"J Cohen","year":"1960","unstructured":"Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Stud Meas XX(1):37\u201346","journal-title":"Educ Psychol Stud Meas"},{"issue":"4","key":"2369_CR8","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1037\/h0026256","volume":"70","author":"J Cohen","year":"1968","unstructured":"Cohen J (1968) Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychol Bull 70(4):213\u2013220","journal-title":"Psychol Bull"},{"key":"2369_CR9","unstructured":"Corney D, Martin C, G\u00f6ker A, Spyromitros-Xioufis E, Papadopoulos S, Kompatsiaris Y, Aiello L, Thomee B (2013) SocialSensor: Finding Diverse Images at MediaEval 2013, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"key":"2369_CR10","doi-asserted-by":"crossref","unstructured":"Dang V, Croft W B (2012) Diversity by Proportionality: An Election-based Approach to Search Result Diversification. ACM International Conference on Research and Development in Information Retrieval, Oregon, USA, pp 65\u201374","DOI":"10.1145\/2348283.2348296"},{"issue":"2","key":"2369_CR11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1348246.1348248","volume":"40","author":"R Datta","year":"2008","unstructured":"Datta R, Joshi D, Li J, Wang JZ (2008) Image Retrieval: Ideas, Influences, and Trends of the New Age. ACM Comput Surv 40(2):1\u201360","journal-title":"ACM Comput Surv"},{"key":"2369_CR12","doi-asserted-by":"crossref","unstructured":"Deselaers T, Gass T, Dreuw P, Ney H (2009) Jointly Optimising Relevance and Diversity in Image Retrieval, ACM International Conference on Image and Video Retrieval","DOI":"10.1145\/1646396.1646443"},{"key":"2369_CR13","unstructured":"Escalante HJ, Morales-Reyes A (2013) TIA-INAOE\u2019s Approach for the 2013 Retrieving Diverse Social Images Task, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"key":"2369_CR14","unstructured":"Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2012) The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. URL http:\/\/www.pascal-network.org\/challenges\/VOC\/voc2012\/workshop\/index.html"},{"key":"2369_CR15","unstructured":"Garc\u00eda Seco de Herrera A, Kalpathy-Cramer J, Demner Fushman D, Antani S, M\u00fcller H (2013) Overview of the ImageCLEF 2013 medical tasks. Working Notes of CLEF 2013 Cross Language Evaluation Forum, Valencia, Spain"},{"key":"2369_CR16","unstructured":"Huang Z, Hu B, Cheng H, Shen H, Liu H, Zhou X (2010) Mining near-duplicate graph for cluster-based reranking of web video search results, vol 28. ACM Transactions on Information Systems, USA, pp 22:1-22:27"},{"key":"2369_CR17","unstructured":"Jain N, Hare J, Samangooei S, Preston J, Davies J, Dupplaw D, Lewis P (2013) Experiments in Diversifying Flickr Result Sets, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"key":"2369_CR18","unstructured":"Ionescu B, Men\u00e9ndez M, M\u00fcller H, Popescu A (2013) Retrieving Diverse Social Images at MediaEval 2013: Objectives, Dataset and Evaluation, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"key":"2369_CR19","doi-asserted-by":"crossref","unstructured":"Ionescu B, Popescu A, M\u00fcller H, Men\u00e9ndez M, Radu A-L (2014) Benchmarking Result Diversification In Social Image Retrieval. IEEE International Conference on Image Processing, Paris, France , pp 27\u201330","DOI":"10.1109\/ICIP.2014.7025621"},{"key":"2369_CR20","doi-asserted-by":"crossref","unstructured":"Ionescu B, Radu A-L, Men\u00e9ndez M, M\u00fcller H, Popescu A, Loni B (2014) Div400: A Social Image Retrieval Result Diversification Dataset. ACM Multimedia Systems, Singapore, pp 19\u201321","DOI":"10.1145\/2557642.2563670"},{"issue":"11","key":"2369_CR21","first-page":"1870","volume":"30","author":"Y Jing","year":"2008","unstructured":"Jing Y, Baluja S (2008) Visualrank: Applying pagerank to large-scale image search. IEEE Trans Pattern Anal Mach Intell 30(11):1870\u20131890","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2369_CR22","unstructured":"Kuoman C, Tollari S, Detyniecki M (2013) UPMC at MediaEval 2013: Relevance by Text and Diversity by Visual Clustering, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"key":"2369_CR23","unstructured":"Lehman A (2005) Jmp For Basic Univariate And Multivariate Statistics: A Step-by-step Guide, Cary, NC: SAS Press. pp 123. ISBN 1-59047-576-3"},{"issue":"7","key":"2369_CR24","doi-asserted-by":"crossref","first-page":"1310","DOI":"10.1109\/TMM.2009.2030598","volume":"11","author":"X Li","year":"2009","unstructured":"Li X, Snoek CGM, Worring M (2009) Learning social tag relevance by neighbor voting. IEEE Trans Multimedia 11(7):1310\u20131322","journal-title":"IEEE Trans Multimedia"},{"key":"2369_CR25","doi-asserted-by":"crossref","unstructured":"McGinty L, Smyth B (2003) On the role of diversity in conversational recommender systems, International Conference on Case-Based Reasoning, pp 276\u2013290","DOI":"10.1007\/3-540-45006-8_23"},{"key":"2369_CR26","unstructured":"Over P, Awad G, Michel M, Fiscus J, Sanders G, Kraaij W, Smeaton A.F, Qu\u00e9enot G (2013) TRECVID 2013 \u2013 An Overview of the Goals, Tasks, Data, Evaluation Mechanisms and Metrics, Proceedings of TRECVID 2013, URL http:\/\/www-nlpir.nist.gov\/projects\/tvpubs\/tv13.papers\/tv13overview.pdf . NIST, USA"},{"key":"2369_CR27","unstructured":"Paramita ML, Sanderson M, Clough P (2009) Diversity in Photo Retrieval: Overview of the ImageCLEF Photo Task 2009. ImageCLEF"},{"key":"2369_CR28","doi-asserted-by":"crossref","unstructured":"Perdoch M, Chum O, Matas JJ (2009) Efficient Representation of Local Geometry for Large Scale Object Retrieval, IEEE Conference on Computer Vision and Pattern Recognition","DOI":"10.1109\/CVPR.2009.5206529"},{"key":"2369_CR29","unstructured":"Popescu A (2013) CEA LISTs Participation at the MediaEval 2013 Retrieving Diverse Social Images Task, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"key":"2369_CR30","doi-asserted-by":"crossref","unstructured":"Popescu A, Grefenstette G (2011) Social Media Driven Image Retrieval. ACM ICMR, Trento, Italy, pp 17\u201320","DOI":"10.1145\/1991996.1992029"},{"key":"2369_CR31","doi-asserted-by":"crossref","unstructured":"Priyatharshini R, Chitrakala S (2013) Association Based Image Retrieval: A Survey, Mobile Communication and Power Engineering. Springer Commun Comput Inf Sci 296:17\u201326","DOI":"10.1007\/978-3-642-35864-7_3"},{"key":"2369_CR32","unstructured":"Radu A-L, Boteanu B, Ple\u015f O, Ionescu B (2013) LAPI @ Retrieving Diverse Social Images Task 2013: Qualitative Photo Retrieval using Multimedia Content, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"key":"2369_CR33","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1007\/978-3-319-04114-8_3","volume":"8325","author":"A-L Radu","year":"2014","unstructured":"Radu A-L, Ionescu B, Men\u00e9ndez M, St\u00f6ttinger J, Giunchiglia F, De Angeli A (2014) A hybrid machine- crowd approach to photo retrieval result diversification. Multimedia Model, Ireland, LNCS 8325:25\u201336","journal-title":"Multimedia Model, Ireland, LNCS"},{"key":"2369_CR34","unstructured":"Randolph JJ (2005) Free-Marginal Multirater Kappa (multirater \u03bafree): an Alternative to Fleiss Fixed-Marginal Multirater Kappa. Joensuu Learning and Instruction Symposium"},{"issue":"4","key":"2369_CR35","doi-asserted-by":"crossref","first-page":"921","DOI":"10.1109\/TMM.2013.2237896","volume":"15","author":"S Rudinac","year":"2013","unstructured":"Rudinac S, Hanjalic A, Larson MA (2013) Generating visual summaries of geographic areas using community-contributed images. IEEE Trans Multimedia 15(4):921\u2013932","journal-title":"IEEE Trans Multimedia"},{"issue":"12","key":"2369_CR36","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1109\/34.895972","volume":"22","author":"AWM Smeulders","year":"2000","unstructured":"Smeulders A W M, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349\u20131380","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2369_CR37","doi-asserted-by":"crossref","unstructured":"Spyromitros-Xioufis E, Papadopoulos S, Kompatsiaris I, Tsoumakas G, Vlahavas I, Tsoumakas G, Vlahavas I (2012) An empirical study on the combination of SURF features with VLAD vectors for image search. International Workshop on Image Analysis for Multimedia Interactive Services, Dublin, Ireland","DOI":"10.1109\/WIAMIS.2012.6226771"},{"key":"2369_CR38","unstructured":"Szu\u030bcs G, Par\u00f3czi Z, Vincz DM (2013) BMEMTM at MediaEval 2013 Retrieving Diverse Social Images Task: Analysis of Text and Visual Information, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"key":"2369_CR39","doi-asserted-by":"crossref","unstructured":"Taneva B, Kacimi M, Weikum G (2010) Gathering and Ranking Photos of Named Entities with High Precision, High Recall, and Diversity, ACM Web Search and Data Mining, pp 431\u2013440","DOI":"10.1145\/1718487.1718541"},{"key":"2369_CR40","doi-asserted-by":"crossref","unstructured":"Tsikrika TT, Garc\u00eda Seco de Herrera A, M\u00fcller H (2011) Assessing the Scholarly Impact of ImageCLEF, Springer Lecture Notes in Computer Science (LNCS):95\u2013106","DOI":"10.1007\/978-3-642-23708-9_12"},{"issue":"3","key":"2369_CR41","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1109\/MMUL.2012.17","volume":"19","author":"T Tsikrika","year":"2012","unstructured":"Tsikrika T, Kludas J, Popescu A (2012) Building reliable and reusable test collections for image retrieval: the wikipedia task at imageclef. IEEE Multimedia 19(3):24\u201333","journal-title":"IEEE Multimedia"},{"key":"2369_CR42","unstructured":"Vandersmissen B., Tomar A., Godin F., De Neve W., Van de Walle R (2013) Ghent University-iMinds at MediaEval 2013 Diverse Images: Relevance-Based Hierarchical Clustering, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"key":"2369_CR43","doi-asserted-by":"crossref","unstructured":"van Leuken RH, Garcia L, Olivares X, Van Zwol R (2009) Visual Diversification of Image Search Results. ACM World Wide Web, pp 341\u2013350","DOI":"10.1145\/1526709.1526756"},{"key":"2369_CR44","doi-asserted-by":"crossref","unstructured":"Vee E, Srivastava U, Shanmugasundaram J, Bhat P, Yahia SA (2008) Efficient computation of diverse query results, IEEE International Conference on Data Engineering, pp 228\u2013236","DOI":"10.1109\/ICDE.2008.4497431"},{"key":"2369_CR45","doi-asserted-by":"crossref","unstructured":"Vieira MR, Razente HL, Barioni MCN, Hadjieleftheriou M, Srivastava D, Traina Jr C, Tsotras VJ (2011) On Query Result Diversification, vol 11-16. IEEE International Conference on Data Engineering, Hannover, Germany, pp 1163\u20131174","DOI":"10.1109\/ICDE.2011.5767846"},{"key":"2369_CR46","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1111\/j.1467-9639.1980.tb00395.x","volume":"2","author":"D Wilkie","year":"1980","unstructured":"Wilkie D (1980) Pictorial representation of kendall\u2019s, rank correlation coefficient. Teach Stat 2:76\u201378","journal-title":"Teach Stat"},{"key":"2369_CR47","unstructured":"Yanai K, Nga DH (2013) UEC, Tokyo at MediaEval 2013 Retrieving Diverse Social Images Task, Working Notes Proceedings MediaEval 2013 Workshop, Eds. M Larson, X Anguera, T Reuter, G J F Jones, B Ionescu, M Schedl, T Piatrik, C Hauff, M Soleymani, co-located with ACM Multimedia, Barcelona, Spain, October 18-19, CEUR-WS.org, ISSN 1613-0073, Vol 1043, http:\/\/ceur-ws.org\/Vol-1043\/"},{"key":"2369_CR48","unstructured":"Zhu X, Goldberg A, Gael JV, Andrzejewski D (2007) Improving Diversity in Ranking using Absorbing Random Walks, Human Language Technologies: The Annual Conference of the North American Chapter of the Association for Computational Linguistics"}],"container-title":["Multimedia Tools and Applications"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-014-2369-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s11042-014-2369-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-014-2369-4","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,8,17]],"date-time":"2019-08-17T17:19:01Z","timestamp":1566062341000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s11042-014-2369-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,11,22]]},"references-count":48,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2016,1]]}},"alternative-id":["2369"],"URL":"https:\/\/doi.org\/10.1007\/s11042-014-2369-4","relation":{},"ISSN":["1380-7501","1573-7721"],"issn-type":[{"value":"1380-7501","type":"print"},{"value":"1573-7721","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,11,22]]}}}