{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,20]],"date-time":"2025-10-20T12:10:40Z","timestamp":1760962240009,"version":"build-2065373602"},"reference-count":22,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2013,1,24]],"date-time":"2013-01-24T00:00:00Z","timestamp":1358985600000},"content-version":"vor","delay-in-days":389,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc of Assoc for Info"],"published-print":{"date-parts":[[2012,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Image retrieval systems are generally based on the notion of image similarity: they compute similarity scores between the images of the database and a query (image or text), and organize the images according to these scores. However, this notion is ill\u2010defined, and the collections used to train and evaluate image retrieval systems are based on similarity judgments that rely on simplistic, non\u2010realistic, assumptions. This paper addresses the issue of the definition of image similarity, and more precisely the two following questions: do humans assess image similarity in the same way? Is it possible to define reference similarity judgments that would correspond to the perception of most users? An experiment is proposed, in which human subjects are assigned two tasks that fall in principle to the system: rating the similarity of images and ranking images according to a reference image. The data provided by the subjects is analyzed quantitatively to the light of the two aforementioned questions. Results show that the subjects do not have collective strategies of similarity assessment, but that a satisfying consensus can be found individually on the data samples used in the experiments. Based on this, methods to define reference similarity scores and rankings are proposed, that can be used on a larger scale to produce realistic ground truths for the evaluation of image retrieval systems. This study is a first step towards a general, realistic, definition of the notion of image similarity in the context of image retrieval.<\/jats:p>","DOI":"10.1002\/meet.14504901180","type":"journal-article","created":{"date-parts":[[2013,1,24]],"date-time":"2013-01-24T10:49:23Z","timestamp":1359024563000},"page":"1-10","source":"Crossref","is-referenced-by-count":3,"title":["Image similarity as assessed by users: A quantitative study"],"prefix":"10.1002","volume":"49","author":[{"given":"Pierre","family":"Tirilly","sequence":"first","affiliation":[]},{"given":"Xiangming","family":"Mu","sequence":"additional","affiliation":[]},{"given":"Chunsheng","family":"Huang","sequence":"additional","affiliation":[]},{"given":"Iris","family":"Xie","sequence":"additional","affiliation":[]},{"given":"Wooseob","family":"Jeong","sequence":"additional","affiliation":[]},{"given":"Jin","family":"Zhang","sequence":"additional","affiliation":[]}],"member":"311","published-online":{"date-parts":[[2013,1,24]]},"reference":[{"key":"e_1_2_7_2_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1995.10476603"},{"issue":"1","key":"e_1_2_7_3_1","article-title":"The Bayesian image retrieval system, PicHunter: theory, implementation and psychological experiments","volume":"9","author":"Cox I. J.","year":"2009","journal-title":"IEEE Transactions on Image Processing"},{"key":"e_1_2_7_4_1","doi-asserted-by":"publisher","DOI":"10.1137\/S0895480102412856"},{"key":"e_1_2_7_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2005.09.012"},{"key":"e_1_2_7_6_1","doi-asserted-by":"crossref","unstructured":"Frese T. Bouman C. A. andAllebach J. P.(1997).A methodology for designing image similarity metrics based on human visual system models. InProceedings of SPIE Human Vision and Electronic Imaging II pages 472\u2013483 San Jose CA United States.","DOI":"10.1117\/12.274545"},{"key":"e_1_2_7_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2004.03.002"},{"key":"e_1_2_7_8_1","doi-asserted-by":"crossref","unstructured":"Huiskes M. J.andLew M. S.(2008).The MIR Flickr retrieval evaluation. InProceedings of the ACM Conference on Multimedia Information Retrieval (MIR) pages39\u201343 Vancouver BC Canada.","DOI":"10.1145\/1460096.1460104"},{"key":"e_1_2_7_9_1","doi-asserted-by":"crossref","unstructured":"Jaimes A. Jaimes R. andChang S.\u2010f.(2000).A conceptual framework for indexing visual information at multiple levels. Inin proceedings of SPIE Internet Imaging pages2\u201315 San Jose CA United States.","DOI":"10.1117\/12.373443"},{"key":"e_1_2_7_10_1","doi-asserted-by":"crossref","unstructured":"J\u00e9gou H. Douze M. andSchmid C.(2008).Hamming embedding and weak geometric consistency for large scale image search. InProceedings of the European Conference on Computer Vision (ECCV) volume 1 pages 304\u2013317 Marseille France.","DOI":"10.1007\/978-3-540-88682-2_24"},{"key":"e_1_2_7_11_1","doi-asserted-by":"crossref","unstructured":"Kennedy L. Slaney M. andWeinberger K.(2009).Reliable tags using image similarity: mining specificity and expertise from large\u2010scale multimedia databases. InProceedings of the 1st workshop on Web\u2010scale multimedia corpus (WSMC) pages 17\u201324 Beijing China.","DOI":"10.1145\/1631135.1631139"},{"issue":"284","key":"e_1_2_7_12_1","doi-asserted-by":"crossref","first-page":"814","DOI":"10.1080\/01621459.1958.10501481","article-title":"Ordinal measures of association","volume":"23","author":"Krsukal W. H.","year":"1958","journal-title":"Journal of the American Statistical Association"},{"key":"e_1_2_7_13_1","doi-asserted-by":"crossref","unstructured":"Liu H. Xie X. Tang X. Li Z.\u2010W. andMa W.\u2010Y.(2004).Effective browsing of web image search results. InProceedings of the ACM International Workshop on Multimedia Information Retrieval (MIR) pages 84\u201390 New York NY USA.","DOI":"10.1145\/1026711.1026726"},{"key":"e_1_2_7_14_1","doi-asserted-by":"crossref","unstructured":"Makadia A. Pavlovic V. andKumar S.(2008).A new baseline for image annotation. InProceedings of the European Conference on Computer Vision (ECCV) pages 316\u2013329 Marseille France.","DOI":"10.1007\/978-3-540-88690-7_24"},{"key":"e_1_2_7_15_1","doi-asserted-by":"crossref","unstructured":"M\u00fcller H. Marchand\u2010Maillet S. andPun T.(2002).The truth about corel\u2010evaluation in image retrieval. InProceedings of the Conference on Image and Video Retrieval (CIVR) pages 38\u201349 London UK.","DOI":"10.1007\/3-540-45479-9_5"},{"key":"e_1_2_7_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1119766.1119769"},{"key":"e_1_2_7_17_1","first-page":"29","volume-title":"Dataset Issues in Object Recognition","author":"Ponce J.","year":"2006"},{"key":"e_1_2_7_18_1","doi-asserted-by":"crossref","unstructured":"Rodden K. Basalaj W. Sinclair D. andWood K.(2001).Does organisation by similarity assist image browsing?InProceedings of the ACM SIGCHI conference Seattle WA United States.","DOI":"10.1145\/365024.365097"},{"key":"e_1_2_7_19_1","unstructured":"Rogowitz B. E. Frese T. Smith J. R. Bouman C. A. andKalin E.(1998).Perceptual image similarity experiments. InProceedings of the SPIE Human Vision and electronic Imaging III San Jose CA United States."},{"key":"e_1_2_7_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-007-0090-8"},{"key":"e_1_2_7_21_1","doi-asserted-by":"crossref","unstructured":"Scassellati B. Alexopoulos S. andFlickner M.(1994).Retrieving images by 2d shape: a comparison of computation methods with human perceptual judgments. InProceedings if SPIE Storage and Retrieval for Image and Video Databases pages 2\u201314 San Jose CA United States.","DOI":"10.1117\/12.171777"},{"key":"e_1_2_7_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.895972"},{"key":"e_1_2_7_23_1","unstructured":"Squire D. M.andPun T.(1997).A comparison of human and machine assessments of image similarity for the organization of image databases. InProceedings of the Scandinavian conference on image analysis Lappeenranta Finland."}],"container-title":["Proceedings of the American Society for Information Science and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fmeet.14504901180","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fmeet.14504901180","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/meet.14504901180","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,20]],"date-time":"2025-10-20T11:34:51Z","timestamp":1760960091000},"score":1,"resource":{"primary":{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/10.1002\/meet.14504901180"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,1]]},"references-count":22,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2012,1]]}},"alternative-id":["10.1002\/meet.14504901180"],"URL":"https:\/\/doi.org\/10.1002\/meet.14504901180","archive":["Portico"],"relation":{},"ISSN":["0044-7870","1550-8390"],"issn-type":[{"type":"print","value":"0044-7870"},{"type":"electronic","value":"1550-8390"}],"subject":[],"published":{"date-parts":[[2012,1]]}}}