{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:12:54Z","timestamp":1750219974466,"version":"3.41.0"},"reference-count":13,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2022,12,1]],"date-time":"2022-12-01T00:00:00Z","timestamp":1669852800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGIR Forum"],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:p>This is a report on the NTCIR-16 conference held online in June 2022. NTCIR is a series of parallel and collective evaluation efforts designed to enhance research on diverse information access technologies, including, but not limited to, cross-language and multimedia information access, question-answering, text mining, and summarization. 53 active research groups from 20 countries\/regions participated in one or more of the 10 different tasks in NTCIR-16. This report introduces the highlights of the conference and describes the scope and task designs of 10 tasks organized at NTCIR-16.<\/jats:p>\n          <jats:p>\n            <jats:bold>Date:<\/jats:bold>\n            14--17 June, 2022.\n          <\/jats:p>\n          <jats:p>\n            <jats:bold>Website:<\/jats:bold>\n            https:\/\/research.nii.ac.jp\/ntcir\/ntcir-16\/.\n          <\/jats:p>","DOI":"10.1145\/3582900.3582911","type":"journal-article","created":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T23:13:31Z","timestamp":1675206811000},"page":"1-8","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Report on the 16th Round of NII Testbeds and Community for Information Access Research (NTCIR-16)"],"prefix":"10.1145","volume":"56","author":[{"given":"Takehiro","family":"Yamamoto","sequence":"first","affiliation":[{"name":"University of Hyogo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhicheng","family":"Dou","sequence":"additional","affiliation":[{"name":"Renmin University of China, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Noriko","family":"Kando","sequence":"additional","affiliation":[{"name":"National Institute of Informatics, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Charles L. A.","family":"Clarke","sequence":"additional","affiliation":[{"name":"University of Waterloo, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Makoto P.","family":"Kato","sequence":"additional","affiliation":[{"name":"University of Tsukuba, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yiqun","family":"Liu","sequence":"additional","affiliation":[{"name":"Tsinghua University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,1,31]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies","author":"Chen Chung-Chi","year":"2022","unstructured":"Chung-Chi Chen , Hen-Hsen Huang , Yu-Lieh Huang , Hiroya Takamura , and Hsin-Hsi Chen . Overview of the NTCIR-16 FinNum-3 task: Investor's and manager's fine-grained claim detection . In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies , 2022 a. Chung-Chi Chen, Hen-Hsen Huang, Yu-Lieh Huang, Hiroya Takamura, and Hsin-Hsi Chen. Overview of the NTCIR-16 FinNum-3 task: Investor's and manager's fine-grained claim detection. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, 2022a."},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","first-page":"2485","DOI":"10.1145\/3357384.3358158","volume-title":"Proceedings of the 28th ACM International Conference on Information and Knowledge Management","author":"Chen Jia","year":"2019","unstructured":"Jia Chen , Jiaxin Mao , Yiqun Liu , Min Zhang , and Shaoping Ma. Tiangong-st : A new dataset with large-scale refined real-world web search sessions . In Proceedings of the 28th ACM International Conference on Information and Knowledge Management , pages 2485 -- 2488 , 2019 . Jia Chen, Jiaxin Mao, Yiqun Liu, Min Zhang, and Shaoping Ma. Tiangong-st: A new dataset with large-scale refined real-world web search sessions. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 2485--2488, 2019."},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies","author":"Chen Jia","year":"2022","unstructured":"Jia Chen , Weihao Wu , Jiaxin Mao , Beining Wang , Fan Zhang , and Yiqun Liu . Overview of the NTCIR-16 session search (SS) task . In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies , 2022 b. Jia Chen, Weihao Wu, Jiaxin Mao, Beining Wang, Fan Zhang, and Yiqun Liu. Overview of the NTCIR-16 session search (SS) task. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, 2022b."},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies","author":"Healy Graham","year":"2022","unstructured":"Graham Healy , Tu-Khiem Le , Mai Boi Quach , Minh-Triet Tran , Thanh-Binh Nguyen , and Cathal Gurrin . Overview of the NTCIR-16 RCIR task . In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies , 2022 . Graham Healy, Tu-Khiem Le, Mai Boi Quach, Minh-Triet Tran, Thanh-Binh Nguyen, and Cathal Gurrin. Overview of the NTCIR-16 RCIR task. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, 2022."},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies","author":"Kando Noriko","year":"2022","unstructured":"Noriko Kando , Charles L. A. Clarke , Makoto P. Kato , Yiqun Liu , Takehiro Yamamoto , and Zhicheng Dou , editors. Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies , 2022 . Noriko Kando, Charles L. A. Clarke, Makoto P. Kato, Yiqun Liu, Takehiro Yamamoto, and Zhicheng Dou, editors. Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, 2022."},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies","author":"Kato Makoto P.","year":"2022","unstructured":"Makoto P. Kato , Hiroaki Ohshima , Ying-Hsang Liu , Hsin-Liang Chen , and Yu Nakano . Overview of the NTCIR-16 Data Search 2 task . In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies , 2022 . Makoto P. Kato, Hiroaki Ohshima, Ying-Hsang Liu, Hsin-Liang Chen, and Yu Nakano. Overview of the NTCIR-16 Data Search 2 task. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, 2022."},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies","author":"Kimura Yasutomo","year":"2022","unstructured":"Yasutomo Kimura , Hideyuki Shibuki , Hokuto Ototake , Yuzu Uchida , Keiichi Takamaru , Madoka Ishioroshi , Kazuma Kadowaki , Masaharu Yoshioka , Tomoyosi Akiba , Yasuhiro Ogawa , Minoru Sasaki , Ken-Ichi Yokote , Tatsunori Mori , Kenji Araki , Teruko Mitamura , and Satoshi Sekine . Overview of the NTCIR-16 QA Lab-PoliInfo-3 task . In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies , 2022 . Yasutomo Kimura, Hideyuki Shibuki, Hokuto Ototake, Yuzu Uchida, Keiichi Takamaru, Madoka Ishioroshi, Kazuma Kadowaki, Masaharu Yoshioka, Tomoyosi Akiba, Yasuhiro Ogawa, Minoru Sasaki, Ken-Ichi Yokote, Tatsunori Mori, Kenji Araki, Teruko Mitamura, and Satoshi Sekine. Overview of the NTCIR-16 QA Lab-PoliInfo-3 task. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, 2022."},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies","author":"Sakai Tetsuya","year":"2022","unstructured":"Tetsuya Sakai , Sijie Tao , Zhumin Chu , Maria Maistro , Yujing Li , Nuo Chen , Nicola Ferro , Junjie Wang , Ian Soboroff , and Yiqun Liu . Overview of the NTCIR-16 WeWantWeb with CENTRE (WWW-4) task . In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies , 2022 . Tetsuya Sakai, Sijie Tao, Zhumin Chu, Maria Maistro, Yujing Li, Nuo Chen, Nicola Ferro, Junjie Wang, Ian Soboroff, and Yiqun Liu. Overview of the NTCIR-16 WeWantWeb with CENTRE (WWW-4) task. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, 2022."},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies","author":"Tao Sijie","year":"2022","unstructured":"Sijie Tao and Tetsuya Sakai . Overview of the NTCIR-16 dialogue evaluation (DialEval-2) task . In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies , 2022 . Sijie Tao and Tetsuya Sakai. Overview of the NTCIR-16 dialogue evaluation (DialEval-2) task. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, 2022."},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies","author":"Yada Shuntaro","year":"2022","unstructured":"Shuntaro Yada , Yuta Nakamura , Shoko Wakamiya , and Eiji Aramaki . Real-MedNLP : Overview of real document-based medical natural language processing task . In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies , 2022 . Shuntaro Yada, Yuta Nakamura, Shoko Wakamiya, and Eiji Aramaki. Real-MedNLP: Overview of real document-based medical natural language processing task. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, 2022."},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the NTCIR-16 Conference","author":"Yamamoto Takehiro","year":"2022","unstructured":"Takehiro Yamamoto and Zhicheng Dou . Overview of NTCIR-16 . In Proceedings of the NTCIR-16 Conference , 2022 . Takehiro Yamamoto and Zhicheng Dou. Overview of NTCIR-16. In Proceedings of the NTCIR-16 Conference, 2022."},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies","author":"Zhao Yurou","year":"2022","unstructured":"Yurou Zhao , Zechun Niu , Feng Wang , Jiaxin Mao , Qingyao Ai , Yang Tao , Junqi Zhang , and Yiqun Liu . Overview of the NTCIR-16 unbiased learning to rank evaluation (ULTRE) task . In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies , 2022 . Yurou Zhao, Zechun Niu, Feng Wang, Jiaxin Mao, Qingyao Ai, Yang Tao, Junqi Zhang, and Yiqun Liu. Overview of the NTCIR-16 unbiased learning to rank evaluation (ULTRE) task. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, 2022."},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies","author":"Zhou Liting","year":"2022","unstructured":"Liting Zhou , Cathal Gurrin , Graham Healy , Hideo Joho , Thanh-Binh Nguyen , Rami Albatal , and Frank Hopfgartner . Overview of the NTCIR-16 Lifelog-4 task . In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies , 2022 . Liting Zhou, Cathal Gurrin, Graham Healy, Hideo Joho, Thanh-Binh Nguyen, Rami Albatal, and Frank Hopfgartner. Overview of the NTCIR-16 Lifelog-4 task. In Proceedings of the 16th NTCIR Conference on Evaluation of Information Access Technologies, 2022."}],"container-title":["ACM SIGIR Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3582900.3582911","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3582900.3582911","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:20Z","timestamp":1750182560000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3582900.3582911"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12]]},"references-count":13,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["10.1145\/3582900.3582911"],"URL":"https:\/\/doi.org\/10.1145\/3582900.3582911","relation":{},"ISSN":["0163-5840"],"issn-type":[{"type":"print","value":"0163-5840"}],"subject":[],"published":{"date-parts":[[2022,12]]},"assertion":[{"value":"2023-01-31","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}