{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,26]],"date-time":"2025-03-26T09:29:12Z","timestamp":1742981352355,"version":"3.40.3"},"publisher-location":"Berlin, Heidelberg","reference-count":36,"publisher":"Springer Berlin Heidelberg","isbn-type":[{"type":"print","value":"9783642151804"},{"type":"electronic","value":"9783642151811"}],"license":[{"start":{"date-parts":[[2010,1,1]],"date-time":"2010-01-01T00:00:00Z","timestamp":1262304000000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2010]]},"DOI":"10.1007\/978-3-642-15181-1_3","type":"book-chapter","created":{"date-parts":[[2010,8,20]],"date-time":"2010-08-20T12:43:45Z","timestamp":1282308225000},"page":"45-61","source":"Crossref","is-referenced-by-count":0,"title":["Creating Realistic Topics for Image Retrieval Evaluation"],"prefix":"10.1007","author":[{"given":"Henning","family":"M\u00fcller","sequence":"first","affiliation":[]}],"member":"297","reference":[{"key":"3_CR1","doi-asserted-by":"crossref","unstructured":"Caputo B, Pronobis A, Jensfelt P (2010) Overview of the CLEF 2009 robot vision task. In: Multilingual Information Access Evaluation Vol. II Multimedia Experiments: Proceedings of the 10th Workshop of the Cross\u2013Language Evaluation Forum (CLEF 2009), Revised Selected Papers. Lecture Notes in Computer Science (LNCS). Springer","DOI":"10.1007\/978-3-642-15751-6_11"},{"key":"3_CR2","unstructured":"Cleverdon CW (1962) Report on the testing and analysis of an investigation into the comparative efficiency of indexing systems. Tech. rep., Aslib Cranfield Research Project, Cranfield, USA"},{"key":"3_CR3","series-title":"Lecture Notes in Computer Science (LNCS)","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1007\/11519645_59","volume-title":"Multilingual Information Access for Text, Speech and Images: Result of the fifth CLEF evaluation campaign","author":"PD Clough","year":"2005","unstructured":"Clough PD, M\u00fcller H, Sanderson M (2005) The CLEF cross\u2013language image retrieval track (ImageCLEF) 2004. In: Peters C, Clough PD, Gonzalo J, Jones GJF, Kluck M, Magnini B (eds) Multilingual Information Access for Text, Speech and Images: Result of the fifth CLEF evaluation campaign. Lecture Notes in Computer Science (LNCS), vol 3491. Springer, Bath, UK, pp 597\u2013613"},{"key":"3_CR4","doi-asserted-by":"crossref","unstructured":"Clough PD, M\u00fcller H, Deselaers T, Grubinger M, Lehmann TM, Jensen J, Hersh W (2006) The CLEF 2005 cross\u2013language image retrieval track. In: Cross\u2013Language Evaluation Forum (CLEF 2005). Lecture Notes in Computer Science (LNCS). Springer, pp 535\u2013557","DOI":"10.1007\/11878773_60"},{"issue":"1","key":"3_CR5","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1007\/s11263-006-0007-y","volume":"74","author":"T Deselaers","year":"2007","unstructured":"Deselaers T, M\u00fcller H, Clough PD, Ney H, Lehmann TM (2007) The CLEF 2005 automatic medical image annotation task. International Journal in Computer Vision 74(1):51\u201358","journal-title":"International Journal in Computer Vision"},{"issue":"2","key":"3_CR6","doi-asserted-by":"publisher","first-page":"126","DOI":"10.1108\/eb026946","volume":"51","author":"PGB Enser","year":"1995","unstructured":"Enser PGB (1995) Pictorial information retrieval. Journal of Documentation 51(2):126\u2013170","journal-title":"Journal of Documentation"},{"key":"3_CR7","doi-asserted-by":"crossref","unstructured":"Everingham M, Zisserman A, Williams CKI, van Gool L, Allan M, Bishop CM, Chapelle O, Dalal N, Deselaers T, Dorko G, Duffner S, Eichhorn J, Farquhar JDR, Fritz M, Garcia C, Griffiths T, Jurie F, Keysers D, Koskela M, Laaksonen J, Larlus D, Leibe B, Meng H, Ney H, Schiele B, Schmid C, Seemann E, Shawe-Taylor J, Storkey A, Szedmak S, Triggs B, Ulusoy I, Viitaniemi V, Zhang J (2006) The 2005 PASCAL visual object classes challenge. In: Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment (PASCAL Workshop 05). Lecture Notes in Artificial Intelligence (LNAI), vol. 3944. Southampton, UK, pp 117\u2013176","DOI":"10.1007\/11736790_8"},{"issue":"2","key":"3_CR8","doi-asserted-by":"crossref","first-page":"63","DOI":"10.28945\/578","volume":"3","author":"A Goodrum","year":"2000","unstructured":"Goodrum A (2000) Image information retrieval: An overview of current research. Informing Science 3(2):63\u201366","journal-title":"Informing Science"},{"key":"3_CR9","unstructured":"Grubinger M, Clough PD (2007) On the creation of query topics for ImageCLEFphoto. In: MUSCLE\/ImageCLEF workshop 2007, Budapest, Hungary"},{"key":"3_CR10","series-title":"Lecture Notes in Computer Science (LNCS)","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1007\/978-3-540-85760-0_57","volume-title":"CLEF 2007 Proceedings","author":"M Grubinger","year":"2008","unstructured":"Grubinger M, Clough P, Hanbury A, M\u00fcller H (2008) Overview of the ImageCLEF 2007 photographic retrieval task. In: CLEF 2007 Proceedings. Lecture Notes in Computer Science (LNCS), vol 5152. Springer, Budapest, Hungary, pp 433\u2013444"},{"key":"3_CR11","doi-asserted-by":"crossref","unstructured":"G\u00fcld MO, Kohnen M, Keysers D, Schubert H, Wein BB, Bredno J, Lehmann TM (2002) Quality of DICOM header information for image categorization. In: International Symposium on Medical Imaging. SPIE Procweedings, vol 4685, San Diego, CA, USA, pp 280\u2013287","DOI":"10.1117\/12.467017"},{"key":"3_CR12","doi-asserted-by":"crossref","unstructured":"Gunther NJ, Beretta G (2001) A benchmark for image retrieval using distributed systems over the Internet: BIRDS\u2013I. Tech. rep., HP Labs, Palo Alto, Tech. Rep. HPL\u20132000\u2013162, San Jose","DOI":"10.1117\/12.411898"},{"key":"3_CR13","doi-asserted-by":"crossref","unstructured":"Harman D (1992) Overview of the first Text REtrieval Conference (TREC\u20131). In: Proceedings of the first Text REtrieval Conference (TREC\u20131), Washington DC, USA, pp 1\u201320","DOI":"10.6028\/NIST.SP.500-207"},{"key":"3_CR14","unstructured":"Hersh W, Jensen J, M\u00fcller H, Gorman P, Ruch P (2005) A qualitative task analysis for developing an image retrieval test collection. In: ImageCLEF\/MUSCLE workshop on image retrieval evaluation, Vienna, Austria, pp 11\u201316"},{"key":"3_CR15","doi-asserted-by":"crossref","unstructured":"J\u00f6rgensen C (1999) Retrieving the unretrievable in electronic imaging systems: emotions, themes and stories. In: Rogowitz B, Pappas TN (eds) Human Vision and Electronic Imaging IV. SPIEProc, vol 3644, San Jose, California, USA. (SPIE Photonics West Conference)","DOI":"10.1117\/12.348455"},{"key":"3_CR16","doi-asserted-by":"crossref","unstructured":"Law-To J, Joly LCA, Laptev I, Buisson O, Nozha Boujemaa VGB, Stentifordl F (2007) Video copy detection: a comparative study. In: Proceedings of the 6th ACM international conference on Image and video retrieval. ACM press, pp 371\u2013378","DOI":"10.1145\/1282280.1282336"},{"key":"3_CR17","unstructured":"Lestari\u00a0Paramita M, Sanderson M, Clough P (2010) Diversity in Photo Retrieval: Overview of the ImageCLEFphoto Task 2009. In: Peters C, Tsikrika T, M\u00fcller H, Kalpathy-Cramer J, Jones JFG, Gonzalo J, Caputo B (eds) Multilingual Information Access Evaluation Vol. II Multimedia Experiments: Proceedings of the 10th Workshop of the Cross\u2013Language Evaluation Forum (CLEF 2009), Revised Selected Papers. Lecture Notes in Computer Science (LNCS), Corfu, Greece"},{"key":"3_CR18","series-title":"Electronic Workshops in Computing","volume-title":"The Challenge of Image Retrieval, A Workshop and Symposium on Image Retrieval","author":"M Markkula","year":"1998","unstructured":"Markkula M, Sormunen E (1998) Searching for photos \u2014 journalists\u2019 practices in pictorial IR. In: Eakins JP, Harper DJ, Jose J (eds) The Challenge of Image Retrieval, A Workshop and Symposium on Image Retrieval. Electronic Workshops in Computing. The British Computer Society, Newcastle upon Tyne"},{"issue":"9","key":"3_CR19","doi-asserted-by":"publisher","first-page":"810","DOI":"10.1002\/(SICI)1097-4571(199709)48:9<810::AID-ASI6>3.0.CO;2-U","volume":"48","author":"S Mizzaro","year":"1997","unstructured":"Mizzaro S (1997) Relevance: The whole (hi)story. Journal of the American Society for Information Science 48(9):810\u2013832","journal-title":"Journal of the American Society for Information Science"},{"issue":"5","key":"3_CR20","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1016\/S0167-8655(00)00118-5","volume":"22","author":"H M\u00fcller","year":"2001","unstructured":"M\u00fcller H, M\u00fcller W, Squire DM, Marchand-Maillet S, Pun T (2001) Performance evaluation in content\u2013based image retrieval: Overview and proposals. PRL 22(5):593\u2013601. Special Issue on Image and Video Indexing","journal-title":"PRL"},{"key":"3_CR21","series-title":"Lecture Notes in Computer Science (LNCS)","isbn-type":"print","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1007\/3-540-45479-9_5","volume-title":"Proceedings of the International Conference on the Challenge of Image and Video Retrieval (CIVR 2002)","author":"H M\u00fcller","year":"2002","unstructured":"M\u00fcller H, Marchand-Maillet S, Pun T (2002) The truth about Corel \u2013 Evaluation in image retrieval. In: Lew MS, Sebe N, Eakins JP (eds) Proceedings of the International Conference on the Challenge of Image and Video Retrieval (CIVR 2002). Lecture Notes in Computer Science (LNCS), vol 2383. Springer, London, England, pp 38\u201349. 3-540-43899-8","ISBN":"https:\/\/id.crossref.org\/isbn\/3540438998"},{"issue":"1","key":"3_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ijmedinf.2003.11.024","volume":"73","author":"H M\u00fcller","year":"2004","unstructured":"M\u00fcller H, Michoux N, Bandon D, Geissbuhler A (2004) A review of content\u2013based image retrieval systems in medicine\u2014clinical benefits and future directions. International Journal of Medical Informatics 73(1):1\u201323","journal-title":"International Journal of Medical Informatics"},{"key":"3_CR23","unstructured":"M\u00fcller H, Despont-Gros C, Hersh W, Jensen J, Lovis C, Geissbuhler A (2006) Health care professionals\u2019 image use and search behaviour. In: Proceedings of the Medical Informatics Europe Conference (MIE 2006). IOS Press, Studies in Health Technology and Informatics, Maastricht, The Netherlands, pp 24\u201332"},{"key":"3_CR24","unstructured":"M\u00fcller H, Boyer C, Gaudinat A, Hersh W, Geissbuhler A (2007) Analyzing web log files of the Health On the Net HONmedia search engine to define typical image search tasks for image retrieval evaluation. In: MedInfo 2007. IOS press, Studies in Health Technology and Informatics, vol 12, Brisbane, Australia, pp 1319\u20131323"},{"key":"3_CR25","series-title":"Lecture Notes in Computer Science (LNCS)","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1007\/978-3-540-68555-5","volume-title":"CLEF 2007 Proceedings","author":"H M\u00fcller","year":"2008","unstructured":"M\u00fcller H, Deselaers T, Kim E, Kalpathy-Cramer J, Deserno TM, Clough PD, Hersh W (2008) Overview of the ImageCLEFmed 2007 medical retrieval and annotation tasks. In: CLEF 2007 Proceedings. Lecture Notes in Computer Science (LNCS), vol 5152. Springer, Budapest, Hungary, pp 473\u2013491"},{"key":"3_CR26","first-page":"523","volume-title":"Medical Informatics Europe (MIE2008)","author":"H M\u00fcller","year":"2008","unstructured":"M\u00fcller H, Kalpathy-Cramer J, Hersh W, Geissbuhler A (2008) Using Medline queries to generate image retrieval tasks for benchmarking. In: Medical Informatics Europe (MIE2008). IOS press, Gothenburg, Sweden, pp 523\u2013528"},{"key":"3_CR27","doi-asserted-by":"crossref","unstructured":"M\u00fcller H, Kalpathy-Cramer J, Eggel I, Bedrick S, Said R, Bakke B, Kahn\u00a0Jr. CE, Hersh W (2009) Overview of the CLEF 2009 medical image retrieval track. In: Working Notes of CLEF 2009, Corfu, Greece","DOI":"10.1007\/978-3-642-15751-6_8"},{"key":"3_CR28","unstructured":"Nowak S, Dunker P (2009) Overview of the CLEF 2009 Large\u2013Scale Visual Concept Detection and Annotation Task. In: Peters C, Tsikrika T, M\u00fcller H, Kalpathy-Cramer J, Jones J, Gonzalo J, Caputo B (eds) Multilingual Information Access Evaluation Vol. II Multimedia Experiments: Proceedings of the 10th Workshop of the Cross\u2013Language Evaluation Forum (CLEF 2009), Revised Selected Papers. LNCS, Corfu, Greece"},{"key":"3_CR29","unstructured":"Radhouani S, Kalpathy-Cramer J, Bedrick S, Hersh W (2009) Medical image retrieval, a user study. Tech. rep., Medical Inforamtics and Outcome Research, OHSU, Portland, OR, USA"},{"key":"3_CR30","series-title":"Lecture Notes in Computer Science (LNCS)","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1007\/3-540-45691-0_3","volume-title":"Report on the CLEF Conference 2001 (Cross Language Evaluation Forum)","author":"J Savoy","year":"2002","unstructured":"Savoy J (2002) Report on CLEF\u20132001 experiments. In: Report on the CLEF Conference 2001 (Cross Language Evaluation Forum). Lecture Notes in Computer Science (LNCS), vol 2406. Springer, Darmstadt, Germany, pp 27\u201343"},{"key":"3_CR31","doi-asserted-by":"crossref","unstructured":"Smeaton AF, Kraaij W, Over P (2003) TRECVID 2003: An overview. In: Proceedings of the TRECVID 2003 conference","DOI":"10.1145\/1027527.1027678"},{"issue":"12","key":"3_CR32","doi-asserted-by":"publisher","first-page":"1349","DOI":"10.1109\/34.895972","volume":"22","author":"AWM Smeulders","year":"2000","unstructured":"Smeulders AWM, Worring M, Santini S, Gupta A, Jain R (2000) Content\u2013based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12):1349\u20131380","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"3_CR33","unstructured":"Sparck\u00a0Jones K, van Rijsbergen C (1975) Report on the need for and provision of an ideal information retrieval test collection. British Library Research and Development Report 5266, Computer Laboratory, University of Cambridge"},{"key":"3_CR34","doi-asserted-by":"crossref","unstructured":"Tsikrika T, Kludas J (2009) Overview of the wikipediaMM task at ImageCLEF 2008. In: Peters C, Giampiccolo D, Ferro N, Petras V, Gonzalo J, Pe\u00f1as A, Deselaers T, Mandl T, Jones G, Kurimo M (eds) Evaluating Systems for Multilingual and Multimodal Information Access \u2014 9th Workshop of the Cross\u2013Language Evaluation Forum. Lecture Notes in Computer Science (LNCS), Aarhus, Denmark","DOI":"10.1007\/978-3-642-04447-2_66"},{"key":"3_CR35","doi-asserted-by":"crossref","unstructured":"Voorhees EM, Harmann D (2000) Overview of the ninth Text REtrieval Conference (TREC\u20139). In: The Ninth Text Retrieval Conference, Gaithersburg, MD, USA, pp 1\u201313","DOI":"10.6028\/NIST.SP.500-249"},{"key":"3_CR36","doi-asserted-by":"crossref","unstructured":"Westerveld T, van Zwol R (2007) The INEX 2006 Multimedia track. In: Fuhr N, Lalmas M, Trotman A (eds) Comparative Evaluation of XML Information Retrieval Systems, Proceedings of the 5th International Workshop of the Initiative for the Evaluation of XML Retrieval (INEX 2006), Revised Selected Papers. Lecture Notes in Computer Science (LNCS), vol 4518. Springer, pp 331\u2013344","DOI":"10.1007\/978-3-540-73888-6_33"}],"container-title":["The Information Retrieval Series","ImageCLEF"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-642-15181-1_3","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,18]],"date-time":"2023-02-18T04:05:17Z","timestamp":1676693117000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-642-15181-1_3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010]]},"ISBN":["9783642151804","9783642151811"],"references-count":36,"URL":"https:\/\/doi.org\/10.1007\/978-3-642-15181-1_3","relation":{},"ISSN":["1387-5264"],"issn-type":[{"type":"print","value":"1387-5264"}],"subject":[],"published":{"date-parts":[[2010]]}}}