{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T20:30:34Z","timestamp":1773520234454,"version":"3.50.1"},"reference-count":30,"publisher":"Association for Computing Machinery (ACM)","issue":"2","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGIR Forum"],"published-print":{"date-parts":[[2025,12,1]]},"abstract":"<jats:p>This event report summarizes the eighteenth round of the NII Testbeds and Community for Information Access Research (NTCIR-18), held on June 10\u201313, 2025 in Tokyo, Japan. NTCIR-18 organized seven core tasks (AEOLLM, FairWeb-2, FinArg-2, Lifelog-6, MedNLP-CHAT, RadNLP, Transfer-2) and three pilot tasks (HIDDEN-RAD, SUSHI, U4), spanning evaluation of generative LLMs, fair ranking, temporal reasoning in finance, multimodal lifelog retrieval, safety assessment for medical dialogue, bilingual radiology staging, resource transfer for dense retrieval, causal explanation in radiology, search over archival metadata, and table-centric QA over annual reports. Across 178 registrations from 113 teams worldwide, participants submitted runs and analyses that combined traditional IR pipelines with LLM-centric methods. This report outlines each task's motivation, data, and methodology, and summarize key findings, including the complementary roles of LLM-based and feature-based evaluators, trade-offs and mitigations in fairness-aware ranking, the importance of structure-aware approaches for tables, and the persistent challenges of sparse metadata and clinical reasoning.<\/jats:p>\n                  <jats:p>\n                    <jats:bold>Date:<\/jats:bold>\n                    10\u201313 June 2025.\n                  <\/jats:p>\n                  <jats:p>\n                    <jats:bold>Website:<\/jats:bold>\n                    https:\/\/research.nii.ac.jp\/ntcir\/ntcir-18\/.\n                  <\/jats:p>","DOI":"10.1145\/3799914.3799924","type":"journal-article","created":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T18:19:03Z","timestamp":1772648343000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Report on the 18th Round of NII Testbeds and Community for Information Access Research (NTCIR-18)"],"prefix":"10.1145","volume":"59","author":[{"given":"Chung-Chi","family":"Chen","sequence":"first","affiliation":[{"name":"National Institute of Advanced Industrial Science and Technology, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qingyao","family":"Ai","sequence":"additional","affiliation":[{"name":"Tsinghua University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shoko","family":"Wakamiya","sequence":"additional","affiliation":[{"name":"Nara Institute of Science and Technology, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Makoto P.","family":"Kato","sequence":"additional","affiliation":[{"name":"University of Tsukuba, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yiqun","family":"Liu","sequence":"additional","affiliation":[{"name":"Tsinghua University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Charles L. A.","family":"Clarke","sequence":"additional","affiliation":[{"name":"University of Waterloo, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Noriko","family":"Kando","sequence":"additional","affiliation":[{"name":"National Institute of Informatics, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,3,4]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Aramaki Eiji","year":"2025","unstructured":"Eiji Aramaki, Shoko Wakamiya, Shuntaro Yada, Shohei Hisada, Tomohiro Nishiyama, Lenard Paulo Tamayo, Jingnan Xiao, Axalia Levenchaud, Pierre Zweigenbaum, Christoph Otto, Jerycho Pasniczek, Philippe Thomas, Nathan Pohl, Wiebke Duettmann, Lisa Raithel, and Roland Roller. Ntcir-18 mednlp-chat determining medical, ethical and legal risks in patient-doctor conversations: Task overview. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_2_1","first-page":"15","volume-title":"Proceedings of the 17th NTCIR Conference on Evaluation of Information Access Technologies","author":"Chen Chung-Chi","year":"2023","unstructured":"Chung-Chi Chen, Chin-Yi Lin, Chr-Jr Chiu, Hen-Hsen Huang, Alaa Alhamzeh, Yu-Lieh Huang, Hiroya Takamura, and Hsin-Hsi Chen. Overview of the ntcir-17 finarg-1 task: Fine-grained argument understanding in financial analysis. In Proceedings of the 17th NTCIR Conference on Evaluation of Information Access Technologies, pages 12\u201315, 2023."},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Chen Chung-Chi","year":"2025","unstructured":"Chung-Chi Chen, Qingyao Ai, and Shoko Wakamiya. Overview of ntcir-18. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025a."},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Chen Chung-Chi","year":"2025","unstructured":"Chung-Chi Chen, Chin-Yi Lin, Cheng-Chih Chiu, Hen-Hsen Huang, Alaa Alhamzeh, Yu-Lieh Huang, Hiroya Takamura, and Hsin-Hsi Chen. Overview of the ntcir-18 finarg-2 task: Temporal inference of financial arguments. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025b."},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Chen Junjie","year":"2025","unstructured":"Junjie Chen, Haitao Li, Zhumin Chu, Yiqun Liu, and Qingyao Ai. Overview of the ntcir-18 automatic evaluation of llms (aeollm) task. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025c."},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Cho Ju-Min","year":"2025","unstructured":"Ju-Min Cho, Ho-Jin Yi, Myung-Kyu Kim, Se-Jin Jeong, and Seung-Hoon Na. Optimizing causality-based radiology reporting with retrieval-augmented and structured reasoning approaches for the ntcir-18 hidden-rad task. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Choi Key-Sun","year":"2025","unstructured":"Key-Sun Choi and You-Sang Cho. Overview of the ntcir-18 hidden-rad task: Hidden causality inclusion in radiology report generation. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Fujimaki Haruki","year":"2025","unstructured":"Haruki Fujimaki and Makoto P Kato. Kasys at the ntcir-18 sushi task. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Joho Hideo","year":"2025","unstructured":"Hideo Joho, Atsushi Keyaki, Yuuki Tachioka, and Shuhei Yamamoto. Overview of the ntcir-18 transfer-2 task. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_10_1","volume-title":"Evaluation of information access technologies with asian languages at ntcir workshop","author":"Kando Noriko","year":"2006","unstructured":"Noriko Kando. Evaluation of information access technologies with asian languages at ntcir workshop. 2006."},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Kim Yumi","year":"2025","unstructured":"Yumi Kim, Meen Chul Kim, and Jongwook Lee. Knuir at the ntcir-18 aeollm: Automatic evaluation of llms. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Kimura Yasutomo","year":"2025","unstructured":"Yasutomo Kimura, Sato Eisaku, Kazuma Kadowaki, and Hokuto Ototake. Overview of the ntcir-18 u4 task. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Mizuguchi Riku","year":"2025","unstructured":"Riku Mizuguchi, Takeshi Yamazaki, and Shuhei Yamamoto. Ymx2l at the ntcir-18 transfer-2 task. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_14_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Nakamura Yuta","year":"2024","unstructured":"Yuta Nakamura, Koji Fujimoto, Eiji Aramaki, Shouhei Hanaoka, and Shuntaro Yada. Ntcir-18 radnlp 2024 overview: Dataset and solutions for automated lung cancer staging. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Oard Douglas W","year":"2025","unstructured":"Douglas W Oard, Shashank Bhardwaj, and Emi Ishita. Biting into sushi: The university of maryland at ntcir-18. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the NTCIR-18 Conference","author":"Ohara Aoi","year":"2025","unstructured":"Aoi Ohara, Nanami Murata, Ami Yuge, and Rei Noguchi. Tusnlp at the ntcir-18 mednlp-chat task: Utilization of external medical knowledge and hybrid approach of bert and chatgpt. In Proceedings of the NTCIR-18 Conference, 2025."},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Ranjit Mercy","year":"2025","unstructured":"Mercy Ranjit, Rahul Kumar, Shaury Srivastav, Anirban Porya, and Tanuja Ganu. Rad-phi3 at the ntcir-18 hidden-rad: Hidden causality inclusion in radiology reports with multimodal small language models. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Rossetto Luca","year":"2025","unstructured":"Luca Rossetto. vitrivr-engine at the ntcir-18 lifelog-6 task. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-15-5554-1"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Suzuki Tokinori","year":"2025","unstructured":"Tokinori Suzuki, Douglas Oard, Shashank Bhardwaj, Emi Ishita, and Yoichi Tomiura. Ntcir-18 sushi pilot task overview. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Tachioka Yuuki","year":"2025","unstructured":"Yuuki Tachioka and Yasunori Terao. ditlab at the ntcir-18 transfer-2 task. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Takasago So","year":"2025","unstructured":"So Takasago and Tomoyoshi Akiba. Akbl at ntcir-18 u4 tableretrieval and tableqa. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the NTCIR-18 Conference","author":"Tamayo Lenard Paulo V","year":"2025","unstructured":"Lenard Paulo V Tamayo, Sa'idah Zahrotul Jannah, Axalia Levenchaud, Mohamad Alnajjar, Shaowen Peng, Shoko Wakamiya, and Eiji Aramaki. Naistym at the ntcir-18 mednlp-chat: Classifying patient-chatbot conversations with objective and subjective assessments using prompting techniques. In Proceedings of the NTCIR-18 Conference, 2025."},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Tanaka Koji","year":"2025","unstructured":"Koji Tanaka, Daiki Shirafuji, and Tatsuhiko Saito. Whiteme at u4 shared task: Hybrid retrieval with table-structured clues for economic table qa. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Tao Sijie","year":"2025","unstructured":"Sijie Tao, Tetsuya Sakai, Junjie Wang, Hanpei Fang, Yuxiang Zhang, Haitao Li, Yiteng Tu, Nuo Chen, and Maria Maistro. Overview of the ntcir-18 fairweb-2 task. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_26_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Tran Quang-Linh","year":"2025","unstructured":"Quang-Linh Tran, Binh T Nguyen, Gareth JF Jones, and Cathal Gurrin. Dcu memoriease at the ntcir-18 lifelog 6 task. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the NTCIR-18 Conference","author":"Supranes Michael Van","year":"2025","unstructured":"Michael Van Supranes, Martin Augustine Borlongan, Joseph Ryan Lansangan, Genelyn Ma Sarte, Shaowen Peng, Shoko Wakamiya, and Eiji Aramaki. Upxsocio at ntcir-18 mednlp-chat task: Similarity-based few-shot example selection for prompt-based detection. In Proceedings of the NTCIR-18 Conference, 2025."},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Won Youngseob","year":"2025","unstructured":"Youngseob Won, Younggyun Hahm, Chanhyuk Yoon, and Seong Tae Kim. Teddysum at the ntcir-18 hidden-rad task: Using rag and tree-of-thought for causal explanation generation. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Yamagishi Yosuke","year":"2025","unstructured":"Yosuke Yamagishi, Ryosuke Tomiyama, and Yui Ueda. Uty at the ntcir-18 radnlp 2024 task: Possibilities and limitations of a hybrid rule-based and llm approach for lung cancer tnm classification. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies","author":"Zhou Liting","year":"2025","unstructured":"Liting Zhou and Cathal Gurrin. Overview of the ntcir-18 lifelog-6 task. In Proceedings of the 18th NTCIR Conference on Evaluation of Information Access Technologies, 2025."}],"container-title":["ACM SIGIR Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3799914.3799924","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T17:19:50Z","timestamp":1773508790000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3799914.3799924"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,1]]},"references-count":30,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,12,1]]}},"alternative-id":["10.1145\/3799914.3799924"],"URL":"https:\/\/doi.org\/10.1145\/3799914.3799924","relation":{},"ISSN":["0163-5840"],"issn-type":[{"value":"0163-5840","type":"print"}],"subject":[],"published":{"date-parts":[[2025,12,1]]},"assertion":[{"value":"2026-03-04","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}