{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T00:42:31Z","timestamp":1780965751464,"version":"3.54.1"},"reference-count":126,"publisher":"MIT Press - Journals","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational Linguistics"],"published-print":{"date-parts":[[2020,3]]},"abstract":"<jats:p> Negation is a universal linguistic phenomenon with a great qualitative impact on natural language processing applications. The availability of corpora annotated with negation is essential to training negation processing systems. Currently, most corpora have been annotated for English, but the presence of languages other than English on the Internet, such as Chinese or Spanish, is greater every day. In this study, we present a review of the corpora annotated with negation information in several languages with the goal of evaluating what aspects of negation have been annotated and how compatible the corpora are. We conclude that it is very difficult to merge the existing corpora because we found differences in the annotation schemes used, and most importantly, in the annotation guidelines: the way in which each corpus was tokenized and the negation elements that have been annotated. Differently than for other well established tasks like semantic role labeling or parsing, for negation there is no standard annotation scheme nor guidelines, which hampers progress in its treatment. <\/jats:p>","DOI":"10.1162\/coli_a_00371","type":"journal-article","created":{"date-parts":[[2020,1,2]],"date-time":"2020-01-02T18:51:49Z","timestamp":1577991109000},"page":"1-52","source":"Crossref","is-referenced-by-count":16,"title":["Corpora Annotated with Negation: An Overview"],"prefix":"10.1162","volume":"46","author":[{"given":"Salud Mar\u00eda","family":"Jim\u00e9nez-Zafra","sequence":"first","affiliation":[{"name":"SINAI, Computer Science Department, CEATIC, Universidad de                         Ja\u00e9n."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roser","family":"Morante","sequence":"additional","affiliation":[{"name":"CLTL Lab, Computational Linguistics, VU University Amsterdam."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mar\u00eda","family":"Teresa Mart\u00edn-Valdivia","sequence":"additional","affiliation":[{"name":"SINAI, Computer Science Department, CEATIC, Universidad de                         Ja\u00e9n."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"L. Alfonso","family":"Ure\u00f1a-L\u00f3pez","sequence":"additional","affiliation":[{"name":"SINAI, Computer Science Department, CEATIC, Universidad de                         Ja\u00e9n."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","reference":[{"key":"bib1","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-014-0373-3"},{"key":"bib2","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2010.003228"},{"key":"bib3","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-1806"},{"key":"bib4","first-page":"43","volume-title":"IWSW","author":"Amores Mario","year":"2016"},{"key":"bib5","volume-title":"Text Mining for Biology and Biomedicine","author":"Ananiadou Sophia","year":"2006"},{"key":"bib6","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00099"},{"key":"bib7","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-28604-9_30"},{"key":"bib8","first-page":"3768","volume-title":"Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016)","author":"Banjade Rajendra","year":"2016"},{"key":"bib9","first-page":"3196","volume-title":"LREC 2012, Eighth International Conference on Language Resources and Evaluation","author":"Basile Valerio","year":"2012"},{"key":"bib10","first-page":"581","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","author":"Blanco Eduardo","year":"2011"},{"key":"bib11","first-page":"228","volume-title":"Proceedings of the Twenty-Fourth International FLAIRS Conference","author":"Blanco Eduardo","year":"2011"},{"key":"bib12","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324913000041"},{"key":"bib13","first-page":"1","volume-title":"Fourth Workshop on Building and Evaluating Resources for Health and Biomedical Text Processing (BIOTxtM 2014)","author":"Bokharaeian Behrouz","year":"2014"},{"key":"bib14","first-page":"49","volume":"51","author":"Bokharaeian Behrouz","year":"2013","journal-title":"Procesamiento del Lenguaje Natural"},{"key":"bib15","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2015.2475761"},{"key":"bib16","first-page":"143","volume-title":"Proceedings of the 5th Linguistic Annotation Workshop","author":"Caselli Tommaso","year":"2011"},{"key":"bib17","first-page":"76","volume-title":"Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH)","author":"Castilho de","year":"2016"},{"key":"bib18","doi-asserted-by":"publisher","DOI":"10.1006\/jbin.2001.1029"},{"key":"bib19","doi-asserted-by":"publisher","DOI":"10.1006\/jbin.2001.1029"},{"key":"bib20","doi-asserted-by":"publisher","DOI":"10.3115\/977035.977081"},{"key":"bib21","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-09891-3_34"},{"key":"bib22","first-page":"115","volume-title":"Proceedings of the Fifth Workshop on Building and Evaluating Resources for Biomedical Text Mining (BioTxtM2016)","author":"Cotik Viviana","year":"2016"},{"key":"bib23","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-2921"},{"key":"bib24","first-page":"51","volume-title":"Proceedings of the Workshop on Negation and Speculation in Natural Language Processing","author":"Councill Isaac","year":"2010"},{"key":"bib25","first-page":"51","volume-title":"Proceedings of the Workshop on Negation and Speculation in Natural Language Processing","author":"Councill Isaac G.","year":"2010"},{"key":"bib26","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23533"},{"key":"bib27","doi-asserted-by":"publisher","DOI":"10.1002\/asi.23533"},{"issue":"7","key":"bib28","first-page":"1398","volume":"63","author":"D\u00edaz Cruz","year":"2012","journal-title":"Journal of the Association for Information Science and Technology"},{"key":"bib29","first-page":"53","volume-title":"SemBEaR 2017","author":"D\u00edaz Cruz","year":"2017"},{"key":"bib30","doi-asserted-by":"publisher","DOI":"10.3115\/1557769.1557781"},{"issue":"906","key":"bib31","first-page":"1","volume":"219","author":"Dalianis Hercules","year":"2009","journal-title":"Women"},{"key":"bib32","first-page":"3071","volume-title":"the Seventh International Conference on Language Resources and Evaluation, LREC 2010","author":"Dalianis Hercules","year":"2010"},{"key":"bib33","first-page":"1","volume-title":"Proceedings of the Asia Pacific Finance Association Annual Conference (APFA)","volume":"35","author":"Das Sanjiv","year":"2001"},{"key":"bib34","doi-asserted-by":"publisher","DOI":"10.5815\/ijisa.2017.04.02"},{"key":"bib35","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-1810"},{"key":"bib36","first-page":"43","volume-title":"Proceedings of NEGES 2018: Workshop on Negation in Spanish, CEUR Workshop Proceedings","volume":"2174","author":"Fabregat Hermenegildo","year":"2018"},{"key":"bib37","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1047"},{"key":"bib38","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-1804"},{"key":"bib39","first-page":"1","volume-title":"Proceedings of the Fourteenth Conference on Computational Natural Language Learning\u2014Shared Task","author":"Farkas Rich\u00e1rd","year":"2010"},{"key":"bib40","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324900002370"},{"key":"bib41","first-page":"85","volume-title":"Proceedings of the 11th International Workshop on Treebanks and Linguistic Theories","author":"Flickinger Dan","year":"2012"},{"key":"bib42","first-page":"1","volume-title":"Proceedings of ACM-SIGIR 2003","author":"Goldin I. M.","year":"2003"},{"key":"bib43","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2009.05.002"},{"key":"bib44","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2013.07.011"},{"key":"bib45","doi-asserted-by":"publisher","DOI":"10.1126\/science.aaa8685"},{"key":"bib46","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-024-0881-2_1"},{"key":"bib47","first-page":"68","volume-title":"Proceedings of the ACL 2010 Conference Short Papers","author":"Ide Nancy","year":"2010"},{"key":"bib48","doi-asserted-by":"publisher","DOI":"10.1145\/1645953.1646241"},{"key":"bib49","first-page":"35","volume-title":"Proceedings of NEGES 2018: Workshop on Negation in Spanish","volume":"2174","author":"Jim\u00e9nez-Zafra Salud Mar\u00eda","year":"2018"},{"issue":"62","key":"bib50","first-page":"21","author":"Jim\u00e9nez-Zafra Salud Mar\u00eda","year":"2019","journal-title":"Procesamiento del Lenguaje Natural"},{"key":"bib51","first-page":"42","volume-title":"ExProM 2016","author":"Jim\u00e9nez-Zafra Salud Mar\u00eda","year":"2016"},{"key":"bib52","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2017.2693968"},{"key":"bib53","first-page":"37","volume":"54","author":"Jim\u00e9nez-Zafra Salud Mar\u00eda","year":"2015","journal-title":"Procesamiento del Lenguaje Natural"},{"key":"bib54","first-page":"915","volume-title":"Proceedings of the 27th International Conference on Computational Linguistics","author":"Jim\u00e9nez-Zafra Salud Mar\u00eda","year":"2018"},{"key":"bib55","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-017-9391-x"},{"key":"bib56","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2017.2693968"},{"key":"bib57","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2016.11.009"},{"key":"bib58","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8640.2006.00277.x"},{"key":"bib59","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btg1023"},{"key":"bib60","doi-asserted-by":"publisher","DOI":"10.3115\/1572340.1572342"},{"key":"bib61","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-9-10"},{"key":"bib62","first-page":"1","author":"Kolhatkar V.","year":"2019","journal-title":"Corpus Pragmatics"},{"key":"bib63","first-page":"3190","volume-title":"LREC","author":"Konstantinova Natalia","year":"2012"},{"key":"bib64","first-page":"139","volume-title":"RANLP Student Research Workshop","author":"Konstantinova Natalia","year":"2011"},{"key":"bib65","doi-asserted-by":"publisher","DOI":"10.1109\/ICDMW.2012.23"},{"key":"bib66","first-page":"1","author":"Lazib Lydia","year":"2018","journal-title":"Frontiers of Computer Science"},{"key":"bib67","first-page":"410","volume-title":"Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1","author":"Li Fangtao","year":"2012"},{"key":"bib68","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-2085"},{"key":"bib69","first-page":"671","volume-title":"Proceedings of the 23rd International Conference on Computational Linguistics","author":"Li Junhui","year":"2010"},{"key":"bib71","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781139084789"},{"key":"bib72","first-page":"49","volume-title":"Proceedings of NEGES 2018: Workshop on Negation in Spanish, CEUR Workshop Proceedings","volume":"2174","author":"Loharja Henry","year":"2018"},{"issue":"36","key":"bib73","first-page":"43","volume":"5","author":"Marimon Montserrat","year":"2017","journal-title":"SemBEaR 2017"},{"key":"bib74","first-page":"41","volume":"57","author":"Mart\u00ed M. Ant\u00f3nia","year":"2016","journal-title":"Procesamiento del Lenguaje Natural"},{"key":"bib75","first-page":"1743","volume-title":"LREC","author":"Matsuyoshi Suguru","year":"2014"},{"key":"bib76","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2015.02.010"},{"key":"bib77","first-page":"260","volume-title":"First Italian Conference on Computational Linguistics","author":"Minard Anne-Lyse","year":"2014"},{"key":"bib78","doi-asserted-by":"publisher","DOI":"10.4000\/books.aaccademia.1934"},{"key":"bib79","first-page":"25","volume":"56","author":"Miranda Carlos Henriquez","year":"2016","journal-title":"Procesamiento del Lenguaje Natural"},{"key":"bib80","first-page":"663","volume-title":"Medinfo","author":"Mitchell Kevin J.","year":"2004"},{"key":"bib81","first-page":"1429","volume-title":"Proceedings of the Seventh Conference on International Language Resources and Evaluation (LREC\u201910)","author":"Morante Roser","year":"2010"},{"key":"bib82","first-page":"265","volume-title":"Proceedings of the First Joint Conference on Lexical and Computational Semantics (*SEM)","author":"Morante Roser","year":"2012"},{"key":"bib83","first-page":"1563","volume-title":"Proceedings of the Eighth International Conference on Language Resources and Evaluation","author":"Morante Roser","year":"2012"},{"key":"bib84","doi-asserted-by":"publisher","DOI":"10.3115\/1613715.1613805"},{"key":"bib85","first-page":"1","author":"Morante Roser","year":"2011","journal-title":"Computational Linguistics and Psycholinguistics Technical Report Series, CTRS-003"},{"key":"bib86","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00095"},{"key":"bib87","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-010-0201-1_9"},{"key":"bib88","first-page":"31","volume-title":"Proceedings of CLEF 2014","author":"Mowery Danielle L.","year":"2014"},{"key":"bib89","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2001.0080598"},{"key":"bib90","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2009.07.007"},{"key":"bib91","doi-asserted-by":"publisher","DOI":"10.3115\/1289189.1289260"},{"key":"bib92","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2015.06.016"},{"key":"bib93","first-page":"2473","volume-title":"Proceedings of the Language Resources and Evaluation Conference (LREC 2012)","author":"Padr\u00f3 Llu\u00eds","year":"2012"},{"key":"bib94","doi-asserted-by":"publisher","DOI":"10.1162\/0891201053630264"},{"key":"bib95","doi-asserted-by":"publisher","DOI":"10.3115\/1118693.1118704"},{"key":"bib96","first-page":"1","volume-title":"Proceedings of the AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications","author":"Polanyi Livia","year":"2004"},{"key":"bib97","doi-asserted-by":"publisher","DOI":"10.1007\/1-4020-4102-0_1"},{"key":"bib98","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-8-50"},{"key":"bib99","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1078"},{"key":"bib100","doi-asserted-by":"publisher","DOI":"10.1109\/BIBM.2018.8621261"},{"key":"bib101","first-page":"294","volume-title":"Proceedings of the First Joint Conference on Lexical and Computational Semantics","author":"Rosenberg Sabine","year":"2012"},{"key":"bib102","first-page":"45","volume":"8","author":"Sandoval Antonio Moreno","year":"2013","journal-title":"Revista Iberoamericana de Ling\u00fc\u00edstica: RIL"},{"key":"bib103","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-009-9089-9"},{"key":"bib104","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2009.001560"},{"key":"bib105","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2011.04.005"},{"key":"bib106","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/2041-1480-2-1","volume":"2","author":"Skeppstedt Maria","year":"2011","journal-title":"Journal of Biomedical Semantics"},{"key":"bib107","first-page":"1631","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing","author":"Socher Richard","year":"2013"},{"key":"bib108","first-page":"1","volume-title":"AMIA Summits on Translational Science Proceedings","volume":"2012","author":"Sohn Sunghwan","year":"2012"},{"key":"bib109","first-page":"1","volume-title":"IJCAI-Workshop on Replicability and Reproducibility in Natural Language Processing: Adaptative Methods, Resources and Software","author":"Stricker Vanesa","year":"2015"},{"key":"bib110","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00172"},{"key":"bib111","doi-asserted-by":"publisher","DOI":"10.3115\/1572306.1572314"},{"key":"bib112","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00098"},{"key":"bib113","first-page":"427","volume-title":"Proceedings of the 5th Conference on Language Resources and Evaluation (LREC\u201906)","author":"Taboada Maite","year":"2006"},{"key":"bib114","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00049"},{"key":"bib115","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-010-0201-1_1"},{"key":"bib116","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000203"},{"key":"bib117","doi-asserted-by":"publisher","DOI":"10.1197\/jamia.M2950"},{"key":"bib118","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00126"},{"key":"bib119","first-page":"13","volume":"50","author":"Vilares David","year":"2013","journal-title":"Procesamiento del Lenguaje Natural"},{"key":"bib120","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324913000181"},{"key":"bib121","first-page":"28","volume-title":"Proceedings of the Workshop on Negation and Speculation in Natural Language Processing","author":"Vincze Veronika","year":"2010"},{"issue":"11","key":"bib122","first-page":"1","volume":"9","author":"Vincze Veronika","year":"2008","journal-title":"BMC Bioinformatics"},{"key":"bib123","first-page":"60","volume-title":"Proceedings of the Workshop on Negation and Speculation in Natural Language Processing","author":"Wiegand Michael","year":"2010"},{"key":"bib124","doi-asserted-by":"publisher","DOI":"10.3115\/1220575.1220619"},{"key":"bib125","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0112774"},{"key":"bib126","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-015-5101-2"},{"key":"bib127","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1187"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/coli_a_00371","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:28:31Z","timestamp":1615584511000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/46\/1\/1-52\/93383"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3]]},"references-count":126,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,3]]}},"alternative-id":["10.1162\/coli_a_00371"],"URL":"https:\/\/doi.org\/10.1162\/coli_a_00371","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3]]}}}