{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T20:30:13Z","timestamp":1773520213729,"version":"3.50.1"},"reference-count":28,"publisher":"Association for Computing Machinery (ACM)","issue":"2","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGIR Forum"],"published-print":{"date-parts":[[2025,12,1]]},"abstract":"<jats:p>This report presents the sixteenth edition of the Conference and Labs of the Evaluation Forum (CLEF 2025), held from September 9 to 12, 2025, in Madrid, Spain, and hosted by the National Distance Education University (UNED). CLEF 2025 continued its hybrid format, combining a scientific conference with an evaluation forum. The conference programme featured keynote talks by Sameer Antani, Jos\u00e9 Hern\u00e1ndez-Orallo, and Joanna Bryson, alongside peer-reviewed research papers, poster sessions, and lab presentations. The evaluation forum comprised 14 labs: BioASQ, CheckThat!, ELOQUENT, eRisk, EXIST, ImageCLEF, JOKER, LifeCLEF, LongEval, PAN, QuantumCLEF, SimpleText, TalentCLEF, and Touch\u00e9. Each lab addresses diverse challenges in multilingual, multimodal, and interactive information access across a variety of domains and media.<\/jats:p>\n                  <jats:p>\n                    <jats:bold>Date:<\/jats:bold>\n                    9\u201312 September, 2025.\n                  <\/jats:p>\n                  <jats:p>\n                    <jats:bold>Website:<\/jats:bold>\n                    https:\/\/clef2025.clef-initiative.eu\/.\n                  <\/jats:p>","DOI":"10.1145\/3799914.3799930","type":"journal-article","created":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T18:19:03Z","timestamp":1772648343000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Report on the 16th Conference and Labs of the Evaluation Forum (CLEF 2025): Experimental IR Meets Multilinguality, Multimodality, and Interaction"],"prefix":"10.1145","volume":"59","author":[{"given":"Jorge","family":"Carrillo-de-Albornoz","sequence":"first","affiliation":[{"name":"UNED, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guglielmo","family":"Faggioli","sequence":"additional","affiliation":[{"name":"University of Padua, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nicola","family":"Ferro","sequence":"additional","affiliation":[{"name":"University of Padua, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alba Garc\u00eda Seco","family":"de Herrera","sequence":"additional","affiliation":[{"name":"UNED, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Julio","family":"Gonzalo","sequence":"additional","affiliation":[{"name":"UNED, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Laura","family":"Plaza","sequence":"additional","affiliation":[{"name":"UNED, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Josiane","family":"Mothe","sequence":"additional","affiliation":[{"name":"Universit\u00e9 de Toulouse, IRIT, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Florina","family":"Piroi","sequence":"additional","affiliation":[{"name":"TU Wien, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paolo","family":"Rosso","sequence":"additional","affiliation":[{"name":"Universitat Polit\u00e8cnica de Val\u00e8ncia, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Damiano","family":"Spina","sequence":"additional","affiliation":[{"name":"RMIT University, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,3,4]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","unstructured":"Firoj Alam Julia Maria Stru\u00df Tanmoy Chakraborty Stefan Dietze Salim Hafid Katerina Korre Arianna Muti Preslav Nakov Federico Ruggeri Sebastian Schellhammer Vinay Setty Megha Sundriyal Konstantin Todorov and V. Venktesh. Overview of the CLEF-2025 CheckThat! lab: Subjectivity fact-checking claim normalization and retrieval. In Carrillo-de Albornoz et al. [2025] pages 199\u2013223. ISBN 978-3-032-04354-2. URL https:\/\/link.springer.com\/book\/10.1007\/978-3-032-04354-2. 10.1007\/978-3-032-04354-2_13","DOI":"10.1007\/978-3-032-04354-2_13"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","unstructured":"Christine Bauer Ben Carterette Nicola Ferro Norbert Fuhr Joeran Beel Timo Breuer Charles L. A. Clarke Anita Crescenzi Gianluca Demartini Giorgio Maria Di Nunzio Laura Dietz Guglielmo Faggioli Bruce Ferwerda Maik Fr\u00f6be Matthias Hagen Allan Hanbury Claudia Hauff Dietmar Jannach Noriko Kando Evangelos Kanoulas Bart P. Knijnenburg Udo Kruschwitz Meijie Li Maria Maistro Lien Michiels Andrea Papenmeier Martin Potthast Paolo Rosso Alan Said Philipp Schaer Christin Seifert Damiano Spina Benno Stein Nava Tintarev Julian Urbano Henning Wachsmuth Martijn C. Willemsen and Justin Zobel. Report on the Dagstuhl seminar on frontiers of information access experimentation for research and education. 57(1) December 2023a. ISSN 0163-5840. 10.1145\/3636341.3636351","DOI":"10.1145\/3636341.3636351"},{"key":"e_1_2_1_3_1","volume-title":"Dagstuhl Reports","volume":"13","author":"Bauer Christine","year":"2023","unstructured":"Christine Bauer, Ben A. Carterette, Nicola Ferro, Norbert Fuhr, and Guglielmo Faggioli, editors. Report from Dagstuhl Seminar 23031: Frontiers of Information Access Experimentation for Research and Education, Dagstuhl Reports, Volume 13, Number 1, 2023b. Schloss Dagstuhl-Leibniz-Zentrum f\u00fcr Informatik, Germany."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-032-04354-2_21"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","unstructured":"Matteo Cancellieri Alaa El-Ebshihy Tobias Fink Maik Fr\u00f6be Petra Galu\u0161\u010d\u00e1kov\u00e1 Gabriela Gonzalez-Saez Lorraine Goeuriot David Iommi J\u00fcri Keller Petr Knoth Philippe Mulhem Florina Piroi David Pride and Philipp Schaer. LongEval at CLEF 2025: Longitudinal evaluation of ir systems on web and scientific data. In Carrillo-de Albornoz et al. [2025] pages 363\u2013387. ISBN 978-3-032-04354-2. URL https:\/\/link.springer.com\/book\/10.1007\/978-3-032-04354-2. 10.1007\/978-3-032-04354-2_20","DOI":"10.1007\/978-3-032-04354-2_20"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-85251-1"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-032-04354-2"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-032-04354-2_23"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-032-04354-2_18"},{"key":"e_1_2_1_10_1","volume-title":"CLEF 2021 Working Notes, 2021. CEUR Workshop Proceedings (CEUR-WS.org). URL http:\/\/ceur-ws.org\/Vol-2936\/.","author":"Faggioli Guglielmo","unstructured":"Guglielmo Faggioli, Nicola Ferro, Alexis Joly, Maria Maistro, and Florina Piroi, editors. CLEF 2021 Working Notes, 2021. CEUR Workshop Proceedings (CEUR-WS.org). URL http:\/\/ceur-ws.org\/Vol-2936\/."},{"key":"e_1_2_1_11_1","volume-title":"CLEF 2025 Working Notes, volume 4038 of CEUR Workshop Proceedings. CEUR Workshop Proceedings (CEUR-WS.org)","author":"Faggioli Guglielmo","year":"2025","unstructured":"Guglielmo Faggioli, Nicola Ferro, Paolo Rosso, and Damiano Spina, editors. CLEF 2025 Working Notes, volume 4038 of CEUR Workshop Proceedings. CEUR Workshop Proceedings (CEUR-WS.org), Madrid, Spain, September 2025. URL https:\/\/ceur-ws.org\/Vol-4038\/."},{"key":"e_1_2_1_12_1","first-page":"45","volume-title":"Martin Braschler, Jacques Savoy, Andreas Rauber, Henning M\u00fcller, David E","author":"Ferro Nicola","year":"2019","unstructured":"Nicola Ferro. What Happened in CLEF... For a While? In Fabio Crestani, Martin Braschler, Jacques Savoy, Andreas Rauber, Henning M\u00fcller, David E. Losada, Gundula Heinatz B\u00fcrki, Linda Cappellato, and Nicola Ferro, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Tenth International Conference of the CLEF Association (CLEF 2019), pages 3\u201345. Lecture Notes in Computer Science (LNCS) 11696, Springer, Heidelberg, Germany, 2019."},{"key":"e_1_2_1_13_1","first-page":"57","volume-title":"P. Mulhem, G. Qu\u00e9not, D. Schwab, L. Soulier, G. M. Di Nunzio","author":"Ferro Nicola","year":"2024","unstructured":"Nicola Ferro. What Happened in CLEF... For Another While? In L. Goeuriot, P. Mulhem, G. Qu\u00e9not, D. Schwab, L. Soulier, G. M. Di Nunzio, P. Galu\u0161\u010d\u00e1kov\u00e1, A. Garc\u00eda Seco de Herrera, G. Faggioli, and N. Ferro, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Fifteenth International Conference of the CLEF Association (CLEF 2024) - Part I, pages 3\u201357. Lecture Notes in Computer Science (LNCS) 14958, Springer, Heidelberg, Germany, 2024."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-22948-1"},{"issue":"1","key":"e_1_2_1_15_1","first-page":"175","volume":"53","author":"Ferro Nicola","unstructured":"Nicola Ferro and Gianmaria Silvello. 3.5K runs, 5K topics, 3M assessments and 70M measures: What trends in 10 years of Adhoc-ish CLEF? Information Processing & Management, 53(1): 175\u2013202, January 2017.","journal-title":"Management"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","unstructured":"Luis Gasco Hermenegildo Fabregat Laura Garc\u00eda-Sardi\u00f1a Paula Estrella Daniel Deniz Alvaro Rodrigo and Rabih Zbib. Overview of the TalentCLEF 2025: Skill and job title intelligence for human capital management. In Carrillo-de Albornoz et al. [2025] pages 464\u2013485. ISBN 978-3-032-04354-2. URL https:\/\/link.springer.com\/book\/10.1007\/978-3-032-04354-2. 10.1007\/978-3-032-04354-2_24","DOI":"10.1007\/978-3-032-04354-2_24"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-88720-8"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","unstructured":"Bogdan Ionescu Henning M\u00fcller Dan-Cristian Stanciu Alexandra-Georgiana Andrei Ahmed-khan Radzhabov Yuri Prokopchuk Liviu-Daniel \u015etefan Mihai Gabriel Constantin Mihai Dogariu Vassili Kovalev Hendrik Damm Johannes R\u00fcckert Asma Ben Abacha Alba Garc\u00eda Seco de Herrera Christoph M. Friedrich Louise Bloch Raphael Br\u00fcngel Ahmad Idrissi-Yaghir Henning Sch\u00e4fer Cynthia Sabrina Schmidt Tabea M. G. Pakull Benjamin Bracke Obioma Pelka Bahadir Eryilmaz Helmut Becker Wen-Wai Yim Noel Codella Roberto Andres Novoa Josep Malvehy Dimitar Dimitrov Rocktim Jyoti Das Zhuohan Xie Ming Shan Hee Preslav Nakov Ivan Koychev Steven A. Hicks Sushant Gautam Michael A. Riegler Vajira Thambawita P\u00e5l Halvorsen Diandra Fabre C\u00e9cile Macaire Benjamin Lecouteux Didier Schwab Martin Potthast Maximilian Heinrich Johannes Kiesel Moritz Wolter Sharat Anand and Benno Stein. Overview of ImageCLEF 2025: Multimedia retrieval in medical social media and content recommendation applications. In Carrillo-de Albornoz et al. [2025] pages 290\u2013314. ISBN 978-3-032-04354-2. URL https:\/\/link.springer.com\/book\/10.1007\/978-3-032-04354-2. 10.1007\/978-3-032-04354-2_17","DOI":"10.1007\/978-3-032-04354-2_17"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-032-04354-2_14"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-032-04354-2_25"},{"key":"e_1_2_1_21_1","volume-title":"Larsen. The Scholarly Impact of CLEF 2010-2017","volume":"41","author":"Birger","unstructured":"Birger Larsen. The Scholarly Impact of CLEF 2010-2017. In N. Ferro and C. Peters, editors, Information Retrieval Evaluation in a Changing World - Lessons Learned from 20 Years of CLEF, volume 41 of The Information Retrieval Series, pages 547\u2013554. Springer International Publishing, Germany, 2019."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","unstructured":"Anastasios Nentidis Georgios Katsimpras Anastasia Krithara Martin Krallinger Miguel Rodr\u00edguez-Ortega Eduard Rodriguez-L\u00f3pez Natalia Loukachevitch Andrey Sakhovskiy Elena Tutubalina Dimitris Dimitriadis Grigorios Tsoumakas George Giannakoulas Alexandra Bekiaridou Athanasios Samaras Giorgio Maria Di Nunzio Nicola Ferro Stefano Marchesin Marco Martinelli Gianmaria Silvello and Georgios Paliouras. Overview of BioASQ 2025: The thirteenth BioASQ challenge on large-scale biomedical semantic indexing and question answering. In Carrillo-de Albornoz et al. [2025] pages 173\u2013198. ISBN 978-3-032-04354-2. URL https:\/\/link.springer.com\/book\/10.1007\/978-3-032-04354-2. 10.1007\/978-3-032-04354-2_12","DOI":"10.1007\/978-3-032-04354-2_12"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-032-04354-2_15"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","unstructured":"Andrea Pasin Maurizio Ferrari Dacrema Washington Cunha Marcos Andr\u00e9 Gon\u00e7alves Paolo Cremonesi and Nicola Ferro. Overview of QuantumCLEF 2025: The second quantum computing challenge for information retrieval and recommender systems at CLEF. In Carrillo-de Albornoz et al. [2025] pages 412\u2013435. ISBN 978-3-032-04354-2. URL https:\/\/link.springer.com\/book\/10.1007\/978-3-032-04354-2. 10.1007\/978-3-032-04354-2_22","DOI":"10.1007\/978-3-032-04354-2_22"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","unstructured":"Luk\u00e1\u0161 Picek Stefan Kahl Herv\u00e9 Go\u00ebau Luk\u00e1\u0161 Adam Th\u00e9o Larcher Cesar Leblanc Maximilien Servajean Kl\u00e1ra Janou\u0161kov\u00e1 Ji\u0159\u00ed Matas Vojt\u011bch \u010cerm\u00e1k Kostas Papafitsoros Robert Planqu\u00e9 Willem-Pier Vellinga Holger Klinck Tom Denton Juan Sebasti\u00e1n Ca\u00f1as Giulio Martellucci Fabrice Vinatier Pierre Bonnet and Alexis Joly. Overview of LifeCLEF 2025: Challenges on species presence prediction and identification and individual animal identification. In Carrillo-de Albornoz et al. [2025] pages 338\u2013362. ISBN 978-3-032-04354-2. URL https:\/\/link.springer.com\/book\/10.1007\/978-3-032-04354-2. 10.1007\/978-3-032-04354-2_19","DOI":"10.1007\/978-3-032-04354-2_19"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-032-04354-2_16"},{"key":"e_1_2_1_27_1","first-page":"106","volume-title":"Multilingual and Multimodal Information Access Evaluation. Proceedings of the Second International Conference of the Cross-Language Evaluation Forum (CLEF","author":"Tsikrika Theodora","year":"2011","unstructured":"Theodora Tsikrika, Alba Garcia Seco de Herrera, and Henning M\u00fcller. Assessing the Scholarly Impact of ImageCLEF. In P. Forner, J. Gonzalo, J. Kek\u00e4l\u00e4inen, M. Lalmas, and M. de Rijke, editors, Multilingual and Multimodal Information Access Evaluation. Proceedings of the Second International Conference of the Cross-Language Evaluation Forum (CLEF 2011), pages 95\u2013106. Lecture Notes in Computer Science (LNCS) 6941, Springer, Heidelberg, Germany, 2011."},{"key":"e_1_2_1_28_1","first-page":"12","volume-title":"Proceedings of the Fourth International Conference of the CLEF Initiative (CLEF","author":"Tsikrika Theodora","year":"2013","unstructured":"Theodora Tsikrika, Birger Larsen, Henning M\u00fcller, Stefan Endrullis, and Erhard Rahm. The Scholarly Impact of CLEF (2000\u20132009). In P. Forner, H. M\u00fcller, R. Paredes, P. Rosso, and B. Stein, editors, Information Access Evaluation meets Multilinguality, Multimodality, and Visualization. Proceedings of the Fourth International Conference of the CLEF Initiative (CLEF 2013), pages 1\u201312. Lecture Notes in Computer Science (LNCS) 8138, Springer, Heidelberg, Germany, 2013."}],"container-title":["ACM SIGIR Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3799914.3799930","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T17:19:19Z","timestamp":1773508759000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3799914.3799930"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,1]]},"references-count":28,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,12,1]]}},"alternative-id":["10.1145\/3799914.3799930"],"URL":"https:\/\/doi.org\/10.1145\/3799914.3799930","relation":{},"ISSN":["0163-5840"],"issn-type":[{"value":"0163-5840","type":"print"}],"subject":[],"published":{"date-parts":[[2025,12,1]]},"assertion":[{"value":"2026-03-04","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}