{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2022,8,25]],"date-time":"2022-08-25T19:51:02Z","timestamp":1661457062861},"reference-count":54,"publisher":"Cambridge University Press (CUP)","issue":"3","license":[{"start":{"date-parts":[[2019,5,15]],"date-time":"2019-05-15T00:00:00Z","timestamp":1557878400000},"content-version":"unspecified","delay-in-days":14,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Nat. Lang. Eng."],"published-print":{"date-parts":[[2019,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Approaches to building temporal information extraction systems typically rely on large, manually annotated corpora. Thus, porting these systems to new languages requires acquiring large corpora of manually annotated documents in the new languages. Acquiring such corpora is difficult owing to the complexity of temporal information extraction annotation. One strategy for addressing this difficulty is to reduce or eliminate the need for manually annotated corpora through annotation projection. This technique utilizes a temporal information extraction system for a source language (typically English) to automatically annotate the source language side of a parallel corpus. It then uses automatically generated word alignments to project the annotations, thereby creating noisily annotated target language training data. We developed an annotation projection technique for producing target language temporal information extraction systems. We carried out an English (source) to French (target) case study wherein we compared a French temporal information extraction system built using annotation projection with one built using a manually annotated French corpus. While annotation projection has been applied to building other kinds of Natural Language Processing tools (e.g., Named Entity Recognizers), to our knowledge, this is the first paper examining annotation projection as applied to temporal information extraction where no manual corrections of the target language annotations were made. We found that, even using manually annotated data to build a temporal information extraction system, <jats:italic>F<\/jats:italic>-scores were relatively low (&lt;0.35), which suggests that the problem is challenging even with manually annotated data. Our annotation projection approach performed well (relative to the system built from manually annotated data) on some aspects of temporal information extraction (e.g., event\u2013document creation time temporal relation prediction), but it performed poorly on the other kinds of temporal relation prediction (e.g., event\u2013event and event\u2013time).<\/jats:p>","DOI":"10.1017\/s1351324919000044","type":"journal-article","created":{"date-parts":[[2019,5,15]],"date-time":"2019-05-15T09:24:28Z","timestamp":1557912268000},"page":"385-403","source":"Crossref","is-referenced-by-count":4,"title":["Annotation projection for temporal information extraction"],"prefix":"10.1017","volume":"25","author":[{"given":"Chris R.","family":"Giannella","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ransom K.","family":"Winder","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joseph P.","family":"Jubinski","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2019,5,15]]},"reference":[{"key":"S1351324919000044_ref42","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1063"},{"key":"S1351324919000044_ref38","first-page":"99","volume-title":"Proceedings of the 9th Language Technologies Conference","author":"Skukan","year":"2014"},{"key":"S1351324919000044_ref26","first-page":"1385","volume-title":"Proceedings of the 24th AAAI Conference on Artificial Intelligence","author":"Ling","year":"2010"},{"key":"S1351324919000044_ref23","doi-asserted-by":"publisher","DOI":"10.5715\/jnlp.22.171"},{"key":"S1351324919000044_ref34","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-011-9416-7"},{"key":"S1351324919000044_ref49","first-page":"57","volume-title":"Proceedings of the 5th International Workshop on Semantic Evaluation (SemEval-10) as part of the 48th Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Verhagen","year":"2010"},{"key":"S1351324919000044_ref35","first-page":"44","volume-title":"Proceedings of the 1st Italian Conference on Computational Linguistics (CLiC-it) and the 4th International Workshop EVALITA","author":"Mirza","year":"2014"},{"key":"S1351324919000044_ref18","first-page":"38","volume-title":"Proceedings of the 17th Conference on Computational Natural Language Learning (CoNLL)","author":"He","year":"2013"},{"key":"S1351324919000044_ref15","first-page":"291","article-title":"Large-scale Bayesian logistic regression for text categorization","volume":"49","author":"Genkin","year":"2007","journal-title":"Technometrics (American Statistical Association and the American Society for Quality)"},{"key":"S1351324919000044_ref13","first-page":"1996","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Ganchev","year":"2013"},{"key":"S1351324919000044_ref36","first-page":"308","volume-title":"Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL)","author":"Mirza","year":"2014"},{"key":"S1351324919000044_ref17","first-page":"1386","volume-title":"Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)","author":"Gouws","year":"2015"},{"key":"S1351324919000044_ref12","volume-title":"Annotation of Temporal Relations Using Markov Logic Networks and Temporal Centering","author":"Fairholm","year":"2014"},{"key":"S1351324919000044_ref11","first-page":"3762","volume-title":"Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC)","author":"Forascu","year":"2012"},{"key":"S1351324919000044_ref10","first-page":"677","volume-title":"Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing (EMNLP) and Computational Natural Language Learning (CoNLL)","author":"Do","year":"2012"},{"key":"S1351324919000044_ref8","first-page":"918","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)","author":"D\u2019souza","year":"2013"},{"key":"S1351324919000044_ref22","first-page":"579","volume-title":"Proceedings of the 52nd Annual Meetings of the Association for Computational Linguistics","author":"Kozhevnikov","year":"2014"},{"key":"S1351324919000044_ref9","first-page":"600","volume-title":"Proceedings of the 49th Annual Meeting of the Association of Computational Linguistics (ACL)","author":"Das","year":"2011"},{"key":"S1351324919000044_ref7","first-page":"671","volume-title":"Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Costa","year":"2010"},{"key":"S1351324919000044_ref6","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00182"},{"key":"S1351324919000044_ref20","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33983-7_20"},{"key":"S1351324919000044_ref44","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00205"},{"key":"S1351324919000044_ref2","first-page":"130","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2","author":"Bittar","year":"2011"},{"key":"S1351324919000044_ref16","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324914000060"},{"key":"S1351324919000044_ref32","first-page":"4417","volume-title":"Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC)","author":"Minard","year":"2016"},{"key":"S1351324919000044_ref40","first-page":"321","volume-title":"Proceedings of the 5th International Workshop on Semantic Evaluation (SemEval-10) as part of the 48th Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Str\u00f6tgen","year":"2010"},{"key":"S1351324919000044_ref48","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-009-9086-z"},{"key":"S1351324919000044_ref19","doi-asserted-by":"publisher","DOI":"10.1145\/1017068.1017072"},{"key":"S1351324919000044_ref3","first-page":"1","volume-title":"Proceedings of the 8th Workshop on Statistical Machine Translation","author":"Bojar","year":"2013"},{"key":"S1351324919000044_ref45","unstructured":"The Apache Software Foundation. (2016). Apache Lucene 6.0.0 documentation. April 7. Available at https:\/\/lucene.apache.org\/core\/6_0_0\/index.html (accessed 26 May 2016)."},{"key":"S1351324919000044_ref41","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-012-9179-y"},{"key":"S1351324919000044_ref27","first-page":"284","volume-title":"Proceedings of the 5th International Workshop on Semantic Evaluation (SemEval-10) as part of the 48th Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Llorens","year":"2010"},{"key":"S1351324919000044_ref46","first-page":"261","volume-title":"Proceedings of the Recent Advances in Natural Language Processing (RANLP)","author":"Torbati","year":"2013"},{"key":"S1351324919000044_ref31","unstructured":"McCallum A. (2002). Available at http:\/\/mallet.cs.umass.edu (accessed 16 July 2013)."},{"key":"S1351324919000044_ref24","first-page":"358","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL) and the 7th International Joint Conference on Natural Language Processing (IJCNLP)","author":"Laparra","year":"2015"},{"key":"S1351324919000044_ref14","first-page":"369","volume-title":"Proceedings of the 47th Annual Meeting of the Association of Computational Linguistics (ACL)","author":"Ganchev","year":"2009"},{"key":"S1351324919000044_ref1","first-page":"10","volume-title":"Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval-13) as part of the 51st Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Bethard","year":"2013"},{"key":"S1351324919000044_ref33","doi-asserted-by":"publisher","DOI":"10.1613\/jair.3693"},{"key":"S1351324919000044_ref43","doi-asserted-by":"publisher","DOI":"10.2200\/S00721ED1V01Y201606HLT036"},{"key":"S1351324919000044_ref21","first-page":"279","volume-title":"Proceedings of the 19th Conference on Computational Language Learning (CoNLL)","author":"Jeong","year":"2015"},{"key":"S1351324919000044_ref4","first-page":"787","volume-title":"Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval-15) as part of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Caselli","year":"2015"},{"key":"S1351324919000044_ref50","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00165"},{"key":"S1351324919000044_ref47","first-page":"1","volume-title":"Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval-13) as part of the 51st Annual Meeting of the Association for Computational Linguistics (ACL)","author":"UzZaman","year":"2013"},{"key":"S1351324919000044_ref52","first-page":"1","volume-title":"Proceedings of the 1st International Conference on Human Language Technology Research","author":"Yarowsky","year":"2001"},{"key":"S1351324919000044_ref5","first-page":"173","volume-title":"Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL) \u2013 Interactive Poster and Demonstration Session","author":"Chambers","year":"2007"},{"key":"S1351324919000044_ref53","first-page":"405","volume-title":"Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics (ACL) and the 4th International Joint Conference on Natural Language Processing (IJCNLP) of the Asian Federation of Natural Language Processing (AFNLP)","author":"Yoshikawa","year":"2009"},{"key":"S1351324919000044_ref29","first-page":"39","volume-title":"Proceedings of the 1st Italian Conference on Computational Linguistics (CLiC-it) and the 4th International Workshop EVALITA","author":"Manfredi","year":"2014"},{"key":"S1351324919000044_ref30","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-5010"},{"key":"S1351324919000044_ref54","first-page":"450","volume-title":"Proceedings of the 26th International Conference on Computational Linguistics (COLING)","author":"Zennaki","year":"2016"},{"key":"S1351324919000044_ref39","first-page":"489","volume-title":"Proceedings of the 3rd International Joint Conference on Natural Language Processing (IJCNLP)","author":"Spreyer","year":"2008"},{"key":"S1351324919000044_ref28","first-page":"151","volume-title":"Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)","author":"Luong","year":"2015"},{"key":"S1351324919000044_ref51","volume-title":"Proceedings of the 2nd Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL)","author":"Yarowski","year":"2001"},{"key":"S1351324919000044_ref25","first-page":"104","volume-title":"Proceeding of the Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics (HLT-NAACL)","author":"Liang","year":"2006"},{"key":"S1351324919000044_ref37","first-page":"3239","volume-title":"9th International Conference on Language Resources and Evaluation (LREC)","author":"Moriceau","year":"2014"}],"container-title":["Natural Language Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1351324919000044","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,5,15]],"date-time":"2019-05-15T09:24:41Z","timestamp":1557912281000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1351324919000044\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5]]},"references-count":54,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2019,5]]}},"alternative-id":["S1351324919000044"],"URL":"https:\/\/doi.org\/10.1017\/s1351324919000044","relation":{},"ISSN":["1351-3249","1469-8110"],"issn-type":[{"value":"1351-3249","type":"print"},{"value":"1469-8110","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5]]}}}