{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T11:47:36Z","timestamp":1774352856645,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":51,"publisher":"ACM","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,7,13]]},"DOI":"10.1145\/3726302.3730294","type":"proceedings-article","created":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T01:21:38Z","timestamp":1752456098000},"page":"3346-3356","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Reproducing NevIR: Negation in Neural Information Retrieval"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-5210-6709","authenticated-orcid":false,"given":"Coen","family":"van den Elsen","sequence":"first","affiliation":[{"name":"University of Amsterdam, Amsterdam, Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-9414-4402","authenticated-orcid":false,"given":"Francien","family":"Barkhof","sequence":"additional","affiliation":[{"name":"University of Amsterdam, Amsterdam, Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-2665-4881","authenticated-orcid":false,"given":"Thijmen","family":"Nijdam","sequence":"additional","affiliation":[{"name":"University of Amsterdam, Amsterdam, Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2383-4557","authenticated-orcid":false,"given":"Simon","family":"Lupart","sequence":"additional","affiliation":[{"name":"University of Amsterdam, Amsterdam, Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9447-4172","authenticated-orcid":false,"given":"Mohammad","family":"Aliannejadi","sequence":"additional","affiliation":[{"name":"University of Amsterdam, Amsterdam, Netherlands"}]}],"member":"320","published-online":{"date-parts":[[2025,7,13]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"Payal Bajaj Daniel Campos Nick Craswell Li Deng Jianfeng Gao Xiaodong Liu Rangan Majumder Andrew McNamara Bhaskar Mitra Tri Nguyen et al. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268 (2016)."},{"key":"e_1_3_2_1_2_1","volume-title":"Language Models are Few-Shot Learners. arxiv","author":"Brown Tom B.","year":"2005","unstructured":"Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. arxiv: 2005.14165 [cs.CL] https:\/\/arxiv.org\/abs\/2005.14165"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1006\/jbin.2001.1029"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"crossref","unstructured":"Jianlv Chen Shitao Xiao Peitian Zhang Kun Luo Defu Lian and Zheng Liu. 2024. BGE M3-Embedding: Multi-Lingual Multi-Functionality Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. arxiv: 2402.03216 [cs.CL] https:\/\/arxiv.org\/abs\/2402.03216","DOI":"10.18653\/v1\/2024.findings-acl.137"},{"key":"e_1_3_2_1_5_1","unstructured":"Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle Aiesha Letman Akhil Mathur Alan Schelten Amy Yang Angela Fan et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)."},{"key":"e_1_3_2_1_6_1","volume-title":"SPLADE v2: Sparse lexical and expansion model for information retrieval. arXiv preprint arXiv:2109.10086","author":"Formal Thibault","year":"2021","unstructured":"Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and St\u00e9phane Clinchant. 2021. SPLADE v2: Sparse lexical and expansion model for information retrieval. arXiv preprint arXiv:2109.10086 (2021)."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531857"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.emnlp-main.491"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.203"},{"key":"e_1_3_2_1_10_1","volume-title":"Javier \u00c1lvez, Itziar Gonzalez-Dios, and German Rigau.","author":"Garc\u00eda-Ferrero Iker","year":"2023","unstructured":"Iker Garc\u00eda-Ferrero, Bego na Altuna, Javier \u00c1lvez, Itziar Gonzalez-Dios, and German Rigau. 2023. This is not a dataset: A large negation benchmark to challenge large language models. arXiv preprint arXiv:2310.15941 (2023)."},{"key":"e_1_3_2_1_11_1","volume-title":"Proc workshop on text analysis and search for bioinformatics, ACM SIGIR.","author":"Goldin Ilya","year":"2003","unstructured":"Ilya Goldin and Wendy W Chapman. 2003. Learning to detect negation with 'not'in medical texts. In Proc workshop on text analysis and search for bioinformatics, ACM SIGIR."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2009.05.002"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462891"},{"key":"e_1_3_2_1_14_1","volume-title":"An analysis of negation in natural language understanding corpora. arXiv preprint arXiv:2203.08929","author":"Hossain Md Mosharaf","year":"2022","unstructured":"Md Mosharaf Hossain, Dhivya Chinnappa, and Eduardo Blanco. 2022. An analysis of negation in natural language understanding corpora. arXiv preprint arXiv:2203.08929 (2022)."},{"key":"e_1_3_2_1_15_1","unstructured":"Hamish Ivison Yizhong Wang Valentina Pyatkin Nathan Lambert Matthew Peters Pradeep Dasigi Joel Jang David Wadden Noah A Smith Iz Beltagy et al. 2023. Camels in a changing climate: Enhancing lm adaptation with tulu 2. arXiv preprint arXiv:2311.10702 (2023)."},{"key":"e_1_3_2_1_16_1","volume-title":"Guillaume Lample, Marc'Aurelio Ranzato, Gabriel Synnaeve, and Nicolas Usunier.","author":"Jiang Albert Q.","year":"2023","unstructured":"Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Guillaume Lample, Marc'Aurelio Ranzato, Gabriel Synnaeve, and Nicolas Usunier. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825 (2023). https:\/\/arxiv.org\/abs\/2310.06825"},{"key":"e_1_3_2_1_17_1","first-page":"6769","article-title":"Dense passage retrieval for open-domain question answering. In EMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference (EMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference)","author":"Karpukhin Vladimir","year":"2020","unstructured":"Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen Tau Yih. 2020. Dense passage retrieval for open-domain question answering. In EMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference (EMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference). Association for Computational Linguistics (ACL), 6769-6781. Publisher Copyright: textcopyright 2020 Association for Computational Linguistics; 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020 ; Conference date: 16-11-2020 Through 20-11-2020.","journal-title":"Association for Computational Linguistics (ACL)"},{"key":"e_1_3_2_1_18_1","volume-title":"Proceedings of naacL-HLT","volume":"1","author":"Ming-Wei Chang Jacob Devlin","year":"2019","unstructured":"Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, Vol. 1. Minneapolis, Minnesota."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401075"},{"key":"e_1_3_2_1_20_1","volume-title":"Quantifying the Carbon Emissions of Machine Learning. arXiv preprint arXiv:1910.09700","author":"Lacoste Alexandre","year":"2019","unstructured":"Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the Carbon Emissions of Machine Learning. arXiv preprint arXiv:1910.09700 (2019)."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531833"},{"key":"e_1_3_2_1_22_1","unstructured":"Carlos Lassance Herv\u00e9 D\u00e9jean Thibault Formal and St\u00e9phane Clinchant. 2024. SPLADE-v3: New baselines for SPLADE. arxiv: 2403.06789 [cs.IR] https:\/\/arxiv.org\/abs\/2403.06789"},{"key":"e_1_3_2_1_23_1","volume-title":"Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281","author":"Li Zehan","year":"2023","unstructured":"Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281 (2023)."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.423"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626772.3657951"},{"key":"e_1_3_2_1_26_1","unstructured":"Xueguang Ma Xinyu Zhang Ronak Pradeep and Jimmy Lin. 2023. Zero-Shot Listwise Document Reranking with a Large Language Model. arxiv: 2305.02156 [cs.IR] https:\/\/arxiv.org\/abs\/2305.02156"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","unstructured":"Chaitanya Malaviya Peter Shaw Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2023. QUEST: A Retrieval Dataset of Entity-Seeking Queries with Implicit Set Operations. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) Anna Rogers Jordan Boyd-Graber and Naoaki Okazaki (Eds.). Association for Computational Linguistics Toronto Canada 14032-14047. doi:10.18653\/v1\/2023.acl-long.784","DOI":"10.18653\/v1\/2023.acl-long.784"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324920000534"},{"key":"e_1_3_2_1_29_1","unstructured":"Niklas Muennighoff Hongjin Su Liang Wang Nan Yang Furu Wei Tao Yu Amanpreet Singh and Douwe Kiela. 2024. Generative Representational Instruction Tuning. arxiv: 2402.09906 [cs.CL]"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.63"},{"key":"e_1_3_2_1_31_1","unstructured":"OpenAI. 2023. GPT-4 Technical Report. CoRR Vol. abs\/2303.08774 (2023). doi:10.48550\/ARXIV.2303.08774 showeprint[arXiv]2303.08774"},{"key":"e_1_3_2_1_32_1","first-page":"188","article-title":"NegBio: a high-performance tool for negation and uncertainty detection in radiology reports","volume":"2018","author":"Peng Yifan","year":"2018","unstructured":"Yifan Peng, Xiaosong Wang, Le Lu, Mohammadhadi Bagheri, Ronald Summers, and Zhiyong Lu. 2018. NegBio: a high-performance tool for negation and uncertainty detection in radiology reports. AMIA Summits on Translational Science Proceedings, Vol. 2018 (2018), 188.","journal-title":"AMIA Summits on Translational Science Proceedings"},{"key":"e_1_3_2_1_33_1","volume-title":"Rankvicuna: Zero-shot listwise document reranking with open-source large language models. arXiv preprint arXiv:2309.15088","author":"Pradeep Ronak","year":"2023","unstructured":"Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. 2023. Rankvicuna: Zero-shot listwise document reranking with open-source large language models. arXiv preprint arXiv:2309.15088 (2023)."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-naacl.97"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.466"},{"key":"e_1_3_2_1_36_1","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, Vol. 21, 140 (2020), 1-67.","journal-title":"Journal of machine learning research"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.598"},{"key":"e_1_3_2_1_38_1","volume-title":"Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv preprint arXiv:1908.10084","author":"Reimers N","year":"2019","unstructured":"N Reimers. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv preprint arXiv:1908.10084 (2019)."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1410"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000019"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.naacl-main.272"},{"key":"e_1_3_2_1_42_1","volume-title":"Negation-instance based evaluation of end-to-end negation resolution. arXiv preprint arXiv:2109.10013","author":"Sineva Elizaveta","year":"2021","unstructured":"Elizaveta Sineva, Stefan Gr\u00fcnewald, Annemarie Friedrich, and Jonas Kuhn. 2021. Negation-instance based evaluation of end-to-end negation resolution. arXiv preprint arXiv:2109.10013 (2021)."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.923"},{"key":"e_1_3_2_1_44_1","volume-title":"AMIA Annual Symposium Proceedings","volume":"2018","author":"Taylor Stuart J","year":"2018","unstructured":"Stuart J Taylor and Sanda M Harabagiu. 2018. The role of a deep-learning method for negation detection in patient cohort identification from electroencephalography reports. In AMIA Annual Symposium Proceedings, Vol. 2018. 1018."},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000203"},{"key":"e_1_3_2_1_46_1","volume-title":"Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672","author":"Wang Liang","year":"2024","unstructured":"Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Multilingual e5 text embeddings: A technical report. arXiv preprint arXiv:2402.05672 (2024)."},{"key":"e_1_3_2_1_47_1","volume-title":"Dawn Lawrie, Ashwin Paranjape, Yuhao Zhang, and Jack Hessel.","author":"Weller Orion","year":"2024","unstructured":"Orion Weller, Benjamin Van Durme, Dawn Lawrie, Ashwin Paranjape, Yuhao Zhang, and Jack Hessel. 2024a. Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models. arxiv: 2409.11136 [cs.IR] https:\/\/arxiv.org\/abs\/2409.11136"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.eacl-long.139"},{"key":"e_1_3_2_1_49_1","unstructured":"An Yang Baosong Yang Binyuan Hui Bo Zheng Bowen Yu Chang Zhou Chengpeng Li Chengyuan Li Dayiheng Liu Fei Huang Guanting Dong Haoran Wei Huan Lin Jialong Tang Jialin Wang Jian Yang Jianhong Tu Jianwei Zhang Jianxin Ma Jianxin Yang Jin Xu Jingren Zhou Jinze Bai Jinzheng He Junyang Lin Kai Dang Keming Lu Keqin Chen Kexin Yang Mei Li Mingfeng Xue Na Ni Pei Zhang Peng Wang Ru Peng Rui Men Ruize Gao Runji Lin Shijie Wang Shuai Bai Sinan Tan Tianhang Zhu Tianhao Li Tianyu Liu Wenbin Ge Xiaodong Deng Xiaohuan Zhou Xingzhang Ren Xinyu Zhang Xipin Wei Xuancheng Ren Xuejing Liu Yang Fan Yang Yao Yichang Zhang Yu Wan Yunfei Chu Yuqiong Liu Zeyu Cui Zhenru Zhang Zhifang Guo and Zhihao Fan. 2024. Qwen2 Technical Report. arxiv: 2407.10671 [cs.CL] https:\/\/arxiv.org\/abs\/2407.10671"},{"key":"e_1_3_2_1_50_1","volume-title":"ExcluIR: Exclusionary Neural Information Retrieval. arXiv preprint arXiv:2404.17288","author":"Zhang Wenhao","year":"2024","unstructured":"Wenhao Zhang, Mengqi Zhang, Shiguang Wu, Jiahuan Pei, Zhaochun Ren, Maarten de Rijke, Zhumin Chen, and Pengjie Ren. 2024. ExcluIR: Exclusionary Neural Information Retrieval. arXiv preprint arXiv:2404.17288 (2024)."},{"key":"e_1_3_2_1_51_1","unstructured":"Yutao Zhu Huaying Yuan Shuting Wang Jiongnan Liu Wenhan Liu Chenlong Deng Haonan Chen Zhicheng Dou and Ji-Rong Wen. [n. d.]. Large Language Models for Information Retrieval: A Survey. ([n.d.])."}],"event":{"name":"SIGIR '25: The 48th International ACM SIGIR Conference on Research and Development in Information Retrieval","location":"Padua Italy","acronym":"SIGIR '25","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3726302.3730294","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T10:00:59Z","timestamp":1755856859000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3726302.3730294"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,13]]},"references-count":51,"alternative-id":["10.1145\/3726302.3730294","10.1145\/3726302"],"URL":"https:\/\/doi.org\/10.1145\/3726302.3730294","relation":{},"subject":[],"published":{"date-parts":[[2025,7,13]]},"assertion":[{"value":"2025-07-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}