{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T23:02:48Z","timestamp":1786230168231,"version":"build-2736575974"},"reference-count":103,"publisher":"MIT Press","issue":"3","license":[{"start":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T00:00:00Z","timestamp":1715817600000},"content-version":"vor","delay-in-days":136,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>While summarization has been extensively researched in natural language processing (NLP), cross-lingual cross-temporal summarization (CLCTS) is a largely unexplored area that has the potential to improve cross-cultural accessibility and understanding. This article comprehensively addresses the CLCTS task, including dataset creation, modeling, and evaluation. We (1) build the first CLCTS corpus with 328 instances for hDe-En (extended version with 455 instances) and 289 for hEn-De (extended version with 501 instances), leveraging historical fiction texts and Wikipedia summaries in English and German; (2) examine the effectiveness of popular transformer end-to-end models with different intermediate fine-tuning tasks; (3) explore the potential of GPT-3.5 as a summarizer; and (4) report evaluations from humans, GPT-4, and several recent automatic evaluation metrics. Our results indicate that intermediate task fine-tuned end-to-end models generate bad to moderate quality summaries while GPT-3.5, as a zero-shot summarizer, provides moderate to good quality outputs. GPT-3.5 also seems very adept at normalizing historical text. To assess data contamination in GPT-3.5, we design an adversarial attack scheme in which we find that GPT-3.5 performs slightly worse for unseen source documents compared to seen documents. Moreover, it sometimes hallucinates when the source sentences are inverted against its prior knowledge with a summarization accuracy of 0.67 for plot omission, 0.71 for entity swap, and 0.53 for plot negation. Overall, our regression results of model performances suggest that longer, older, and more complex source texts (all of which are more characteristic for historical language variants) are harder to summarize for all models, indicating the difficulty of the CLCTS task. Regarding evaluation, we observe that both the GPT-4 and BERTScore correlate moderately with human evaluations, implicating great potential for future improvement.<\/jats:p>","DOI":"10.1162\/coli_a_00519","type":"journal-article","created":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T18:50:30Z","timestamp":1715885430000},"page":"1001-1047","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":10,"title":["Cross-lingual Cross-temporal Summarization: Dataset, Models, Evaluation"],"prefix":"10.1162","volume":"50","author":[{"given":"Ran","family":"Zhang","sequence":"first","affiliation":[{"name":"Natural Language Learning Group (NLLG), School of Business Informatics and Mathematics, University of Mannheim. ran.zhang@uni-mannheim.de"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jihed","family":"Ouni","sequence":"additional","affiliation":[{"name":"Fachbereich Informatik, Technische Universit\u00e4t Darmstadt. jihed.ouni@stud.tu-darmstadt.de"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steffen","family":"Eger","sequence":"additional","affiliation":[{"name":"Natural Language Learning Group (NLLG), School of Business Informatics and Mathematics, University of Mannheim. steffen.eger@uni-mannheim.de"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2024,9,1]]},"reference":[{"key":"2024092014251823700_bib1","first-page":"365","article-title":"bokstaffua, bokstaffwa, bokstafwa, bokstaua, bokstawa... towards lexical link-up for a corpus of old Swedish","volume-title":"KONVENS","author":"Adesam","year":"2012"},{"key":"2024092014251823700_bib2","first-page":"216","article-title":"Genre classification using character networks","volume-title":"2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS)","author":"Agarwal","year":"2021"},{"key":"2024092014251823700_bib3","doi-asserted-by":"publisher","first-page":"6910","DOI":"10.18653\/v1\/2021.acl-long.538","article-title":"Cross-lingual abstractive summarization with limited parallel resources","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Bai","year":"2021"},{"key":"2024092014251823700_bib4","doi-asserted-by":"publisher","first-page":"1087","DOI":"10.1145\/3477495.3532071","article-title":"Unifying cross-lingual summarization and machine translation with compression rate","volume-title":"Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Bai","year":"2022"},{"key":"2024092014251823700_bib5","first-page":"67","article-title":"Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source LLMs","volume-title":"Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Balloccu","year":"2024"},{"key":"2024092014251823700_bib6","doi-asserted-by":"publisher","first-page":"675","DOI":"10.18653\/v1\/2023.ijcnlp-main.45","article-title":"A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity","volume-title":"Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Bang","year":"2023"},{"key":"2024092014251823700_bib7","article-title":"Vard2: A tool for dealing with spelling variation in historical corpora","volume-title":"Postgraduate Conference in Corpus Linguistics","author":"Baron","year":"2008"},{"key":"2024092014251823700_bib8","doi-asserted-by":"publisher","first-page":"358","DOI":"10.18653\/v1\/2023.eacl-main.27","article-title":"UScore: An effective approach to fully unsupervised evaluation metrics for machine translation","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","author":"Belouadi","year":"2023"},{"key":"2024092014251823700_bib9","article-title":"Longformer: The long-document transformer","author":"Beltagy","year":"2020","journal-title":"arXiv preprint arXiv:2004.05150"},{"key":"2024092014251823700_bib10","doi-asserted-by":"publisher","first-page":"249","DOI":"10.18653\/v1\/2021.inlg-1.24","article-title":"The ReproGen shared task on reproducibility of human evaluations in NLG: Overview and results","volume-title":"Proceedings of the 14th International Conference on Natural Language Generation","author":"Belz","year":"2021"},{"key":"2024092014251823700_bib11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18653\/v1\/2023.insights-1.1","article-title":"Missing information, unresponsive authors, experimental flaws: The impossibility of assessing the reproducibility of previous human evaluations in NLP","volume-title":"The Fourth Workshop on Insights from Negative Results in NLP","author":"Belz","year":"2023"},{"key":"2024092014251823700_bib12","article-title":"Unlimiformer: Long-range transformers with unlimited length input","volume":"36","author":"Bertsch","year":"2024","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2024092014251823700_bib13","doi-asserted-by":"publisher","first-page":"9347","DOI":"10.18653\/v1\/2020.emnlp-main.751","article-title":"Re-evaluating evaluation in text summarization","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Bhandari","year":"2020"},{"key":"2024092014251823700_bib14","first-page":"3","article-title":"Automatic normalization of historical texts using distance measures and the Norma tool","volume-title":"Proceedings of the Second Workshop on Annotation of Corpora for Research in the Humanities (ACRH-2)","author":"Bollmann","year":"2012"},{"key":"2024092014251823700_bib15","first-page":"3885","article-title":"A large-scale comparison of historical text normalization systems","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Bollmann","year":"2019"},{"key":"2024092014251823700_bib16","doi-asserted-by":"publisher","first-page":"332","DOI":"10.18653\/v1\/P17-1031","article-title":"Learning attention for historical text normalization by learning to pronounce","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Bollmann","year":"2017"},{"key":"2024092014251823700_bib17","first-page":"131","article-title":"Improving historical spelling normalization with bi-directional LSTMs and multi-task learning","volume-title":"Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers","author":"Bollmann","year":"2016"},{"key":"2024092014251823700_bib18","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2024092014251823700_bib19","doi-asserted-by":"publisher","first-page":"6220","DOI":"10.18653\/v1\/2020.acl-main.554","article-title":"Jointly learning to align and summarize for neural cross-lingual summarization","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Cao","year":"2020"},{"key":"2024092014251823700_bib20","doi-asserted-by":"publisher","first-page":"706","DOI":"10.18653\/v1\/2021.findings-emnlp.61","article-title":"Rethinking why intermediate-task fine-tuning works","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021","author":"Chang","year":"2021"},{"key":"2024092014251823700_bib21","doi-asserted-by":"publisher","first-page":"404","DOI":"10.18653\/v1\/2021.acl-long.34","article-title":"A training-free and reference-free summarization evaluation metric via centrality-weighted relevance and self-referenced redundancy","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Chen","year":"2021"},{"key":"2024092014251823700_bib22","doi-asserted-by":"publisher","first-page":"2965","DOI":"10.18653\/v1\/2022.emnlp-main.192","article-title":"Reproducibility issues for BERT-based evaluation metrics","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Chen","year":"2022"},{"key":"2024092014251823700_bib23","doi-asserted-by":"publisher","first-page":"804","DOI":"10.1162\/tacl_a_00576","article-title":"Menli: Robust evaluation metrics from natural language inference","volume":"11","author":"Chen","year":"2023","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2024092014251823700_bib24","doi-asserted-by":"publisher","first-page":"15607","DOI":"10.18653\/v1\/2023.acl-long.870","article-title":"Can large language models be an alternative to human evaluations?","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Chiang","year":"2023"},{"key":"2024092014251823700_bib25","doi-asserted-by":"publisher","first-page":"5881","DOI":"10.18653\/v1\/2021.naacl-main.470","article-title":"Sliding selector network with dynamic memory for extractive summarization of long documents","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Cui","year":"2021"},{"key":"2024092014251823700_bib26","article-title":"LongNet: Scaling transformers to 1,000,000,000 tokens","author":"Ding","year":"2023","journal-title":"arXiv preprint arXiv:2307.02486"},{"key":"2024092014251823700_bib27","doi-asserted-by":"publisher","first-page":"52","DOI":"10.18653\/v1\/P16-2009","article-title":"On the linearity of semantic change: Investigating meaning variation via dynamic graph models","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Eger","year":"2016"},{"issue":"1","key":"2024092014251823700_bib28","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1515\/pralin-2016-0004","article-title":"A comparison of four character-level string-to-string translation models for (ocr) spelling error correction","volume":"105","author":"Eger","year":"2016","journal-title":"The Prague Bulletin of Mathematical Linguistics"},{"key":"2024092014251823700_bib29","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1162\/tacl_a_00373","article-title":"SummEval: Re-evaluating summarization evaluation","volume":"9","author":"Fabbri","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"issue":"1","key":"2024092014251823700_bib30","first-page":"4839","article-title":"Beyond English-centric multilingual machine translation","volume":"22","author":"Fan","year":"2021","journal-title":"Journal of Machine Learning Research"},{"key":"2024092014251823700_bib31","article-title":"Human-like summarization evaluation with ChatGPT","author":"Gao","year":"2023","journal-title":"arXiv preprint arXiv:2304.02554"},{"key":"2024092014251823700_bib32","doi-asserted-by":"publisher","first-page":"1347","DOI":"10.18653\/v1\/2020.acl-main.124","article-title":"SUPERT: Towards new frontiers in unsupervised evaluation metrics for multi-document summarization","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Gao","year":"2020"},{"issue":"5","key":"2024092014251823700_bib33","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1016\/j.tics.2019.02.003","article-title":"How efficiency shapes human language","volume":"23","author":"Gibson","year":"2019","journal-title":"Trends in Cognitive Sciences"},{"key":"2024092014251823700_bib34","doi-asserted-by":"publisher","first-page":"3029","DOI":"10.1109\/TASLP.2020.3037401","article-title":"A divide-and-conquer approach to the summarization of long documents","volume":"28","author":"Gidiotis","year":"2020","journal-title":"IEEE\/ACM Transactions on Audio, Speech, and Language Processing"},{"key":"2024092014251823700_bib35","doi-asserted-by":"publisher","first-page":"3960","DOI":"10.18653\/v1\/2020.acl-main.365","article-title":"Analysing lexical semantic change with contextualised word representations","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Giulianelli","year":"2020"},{"key":"2024092014251823700_bib36","article-title":"News summarization and evaluation in the era of gpt-3","author":"Goyal","year":"2022","journal-title":"arXiv preprint arXiv:2209.12356"},{"key":"2024092014251823700_bib37","doi-asserted-by":"publisher","first-page":"444","DOI":"10.18653\/v1\/2022.emnlp-main.29","article-title":"SNaC: Coherence error detection for narrative summarization","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Goyal","year":"2022"},{"key":"2024092014251823700_bib38","doi-asserted-by":"publisher","first-page":"708","DOI":"10.18653\/v1\/N18-1065","article-title":"Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Grusky","year":"2018"},{"key":"2024092014251823700_bib39","doi-asserted-by":"publisher","first-page":"6507","DOI":"10.18653\/v1\/2022.acl-long.450","article-title":"MemSum: Extractive summarization of long documents using multi-step episodic Markov decision processes","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Gu","year":"2022"},{"key":"2024092014251823700_bib40","doi-asserted-by":"publisher","first-page":"1489","DOI":"10.18653\/v1\/P16-1141","article-title":"Diachronic word embeddings reveal statistical laws of semantic change","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Hamilton","year":"2016"},{"key":"2024092014251823700_bib41","doi-asserted-by":"publisher","first-page":"5095","DOI":"10.18653\/v1\/2023.acl-long.279","article-title":"Z-code++: A pre-trained language model optimized for abstractive summarization","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"He","year":"2023"},{"key":"2024092014251823700_bib42","doi-asserted-by":"publisher","first-page":"1419","DOI":"10.18653\/v1\/2021.naacl-main.112","article-title":"Efficient attentions for long document summarization","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Huang","year":"2021"},{"issue":"12","key":"2024092014251823700_bib43","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3571730","article-title":"Survey of hallucination in natural language generation","volume":"55","author":"Ji","year":"2023","journal-title":"ACM Computing Surveys"},{"key":"2024092014251823700_bib44","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1002\/9781405166348.ch18","article-title":"Diachronic morphology","author":"Joseph","year":"2017","journal-title":"The Handbook of Morphology"},{"key":"2024092014251823700_bib45","first-page":"109","article-title":"Exploring diachronic syntactic shifts with dependency length: The case of scientific English","volume-title":"Proceedings of the Fourth Workshop on Universal Dependencies (UDW 2020)","author":"Juzek","year":"2020"},{"key":"2024092014251823700_bib46","doi-asserted-by":"publisher","first-page":"419","DOI":"10.18653\/v1\/2023.wmt-1.41","article-title":"Large language models effectively leverage document-level context for literary translation, but critical errors persist","volume-title":"Proceedings of the Eighth Conference on Machine Translation","author":"Karpinska","year":"2023"},{"issue":"8","key":"2024092014251823700_bib47","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3545176","article-title":"An empirical survey on long document summarization: Datasets, models, and metrics","volume":"55","author":"Koh","year":"2022","journal-title":"ACM Computing Surveys"},{"key":"2024092014251823700_bib48","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1162\/tacl_a_00453","article-title":"SummaC: Re-visiting NLI-based models for inconsistency detection in summarization","volume":"10","author":"Laban","year":"2022","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2024092014251823700_bib49","doi-asserted-by":"publisher","first-page":"4034","DOI":"10.18653\/v1\/2020.findings-emnlp.360","article-title":"WikiLingua: A new benchmark dataset for cross-lingual abstractive summarization","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Ladhak","year":"2020"},{"key":"2024092014251823700_bib50","doi-asserted-by":"publisher","first-page":"102762","DOI":"10.1016\/j.lingua.2019.102762","article-title":"Is dependency distance experiencing a process of minimization? A diachronic study based on the State of the Union addresses","volume":"239","author":"Lei","year":"2020","journal-title":"Lingua"},{"key":"2024092014251823700_bib51","doi-asserted-by":"publisher","first-page":"100541","DOI":"10.1016\/j.mlwa.2024.100541","article-title":"ChatGPT: A meta-analysis after 2.5 months","volume":"16","author":"Leiter","year":"2024","journal-title":"Machine Learning with Applications"},{"key":"2024092014251823700_bib52","volume-title":"The History of the English Paragraph","author":"Lewis","year":"1894"},{"key":"2024092014251823700_bib53","doi-asserted-by":"publisher","first-page":"7871","DOI":"10.18653\/v1\/2020.acl-main.703","article-title":"BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Lewis","year":"2020"},{"key":"2024092014251823700_bib54","doi-asserted-by":"publisher","first-page":"2088","DOI":"10.18653\/v1\/2022.acl-long.148","article-title":"A variational hierarchical model for neural cross-lingual summarization","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Liang","year":"2022"},{"key":"2024092014251823700_bib55","first-page":"74","article-title":"ROUGE: A package for automatic evaluation of summaries","volume-title":"Text Summarization Branches Out","author":"Lin","year":"2004"},{"issue":"2","key":"2024092014251823700_bib56","doi-asserted-by":"publisher","first-page":"159","DOI":"10.17791\/jcs.2008.9.2.159","article-title":"Dependency distance as a metric of language comprehension difficulty","volume":"9","author":"Liu","year":"2008","journal-title":"Journal of Cognitive Science"},{"key":"2024092014251823700_bib57","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1016\/j.plrev.2017.03.002","article-title":"Dependency distance: A new perspective on syntactic patterns in natural languages","volume":"21","author":"Liu","year":"2017","journal-title":"Physics of Life Reviews"},{"key":"2024092014251823700_bib58","doi-asserted-by":"publisher","first-page":"2511","DOI":"10.18653\/v1\/2023.emnlp-main.153","article-title":"G-eval: NLG evaluation using gpt-4 with better human alignment","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Liu","year":"2023"},{"key":"2024092014251823700_bib59","doi-asserted-by":"publisher","first-page":"2109","DOI":"10.18653\/v1\/2022.naacl-main.153","article-title":"Reference-free summarization evaluation via semantic correlation and compression ratio","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Liu","year":"2022"},{"key":"2024092014251823700_bib60","doi-asserted-by":"publisher","first-page":"2890","DOI":"10.18653\/v1\/2022.acl-long.207","article-title":"BRIO: Bringing order to abstractive summarization","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Liu","year":"2022"},{"key":"2024092014251823700_bib61","doi-asserted-by":"publisher","first-page":"7284","DOI":"10.18653\/v1\/2020.acl-main.650","article-title":"Semi-supervised contextual historical text normalization","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Makarov","year":"2020"},{"key":"2024092014251823700_bib62","doi-asserted-by":"publisher","first-page":"6026","DOI":"10.18653\/v1\/2021.acl-long.470","article-title":"Long-span summarization via local attention and content selection","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Manakul","year":"2021"},{"key":"2024092014251823700_bib63","first-page":"380","article-title":"Using of Jaccard Coefficient for keywords similarity","volume-title":"Proceedings of the International Multiconference of Engineers and Computer Scientists","author":"Niwattanakul","year":"2013"},{"key":"2024092014251823700_bib64","doi-asserted-by":"publisher","first-page":"2025","DOI":"10.18653\/v1\/N19-1204","article-title":"A robust abstractive system for cross-lingual summarization","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Ouyang","year":"2019"},{"key":"2024092014251823700_bib65","doi-asserted-by":"publisher","first-page":"3123","DOI":"10.18653\/v1\/2021.eacl-main.273","article-title":"Summarising historical text in modern languages","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume","author":"Peng","year":"2021"},{"key":"2024092014251823700_bib66","unstructured":"Pettersson, Eva\n          . 2016. Spelling Normalisation and Linguistic Analysis of Historical Text for Information Extraction. Ph.D. thesis, Acta Universitatis Upsaliensis."},{"key":"2024092014251823700_bib67","doi-asserted-by":"publisher","first-page":"5093","DOI":"10.18653\/v1\/P19-1502","article-title":"Studying summarization evaluation metrics in the appropriate scoring range","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Peyrard","year":"2019"},{"key":"2024092014251823700_bib68","doi-asserted-by":"publisher","first-page":"9308","DOI":"10.18653\/v1\/2020.emnlp-main.748","article-title":"On extractive and abstractive neural document summarization with transformer language models","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Pilault","year":"2020"},{"key":"2024092014251823700_bib69","doi-asserted-by":"publisher","first-page":"392","DOI":"10.18653\/v1\/W15-3049","article-title":"chrF: Character n-gram F-score for automatic MT evaluation","volume-title":"Proceedings of the Tenth Workshop on Statistical Machine Translation","author":"Popovi\u0107","year":"2015"},{"key":"2024092014251823700_bib70","first-page":"70","article-title":"Edit transducers for spelling variation in old spanish","volume-title":"Proceedings of the Workshop on Computational Historical Linguistics at NODALIDA 2013. NEALT Proc. Series","author":"Porta","year":"2013"},{"key":"2024092014251823700_bib71","doi-asserted-by":"publisher","first-page":"101","DOI":"10.18653\/v1\/2020.acl-demos.14","article-title":"Stanza: A Python natural language processing toolkit for many human languages","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations","author":"Qi","year":"2020"},{"key":"2024092014251823700_bib72","doi-asserted-by":"publisher","first-page":"4504","DOI":"10.18653\/v1\/2022.acl-long.309","article-title":"SummaReranker: A multi-task mixture-of-experts re-ranking framework for abstractive summarization","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Ravaut","year":"2022"},{"key":"2024092014251823700_bib73","article-title":"Vard versus word: A comparison of the ucrel variant detector and modern spellcheckers on english historical corpora","author":"Rayson","year":"2005","journal-title":"Corpus Linguistics 2005"},{"key":"2024092014251823700_bib74","doi-asserted-by":"publisher","first-page":"4512","DOI":"10.18653\/v1\/2020.emnlp-main.365","article-title":"Making monolingual sentence embeddings multilingual using knowledge distillation","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Reimers","year":"2020"},{"key":"2024092014251823700_bib75","doi-asserted-by":"publisher","first-page":"720","DOI":"10.18653\/v1\/N18-2113","article-title":"Evaluating historical text normalization systems: How well do they generalize?","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)","author":"Robertson","year":"2018"},{"key":"2024092014251823700_bib76","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1075\/scl.85.10rud","article-title":"Variation of sentence length across time and genre","author":"Rudnicka","year":"2018","journal-title":"Diachronic Corpora, Genre, and Language Change"},{"key":"2024092014251823700_bib77","doi-asserted-by":"publisher","first-page":"10776","DOI":"10.18653\/v1\/2023.findings-emnlp.722","article-title":"NLP evaluation in trouble: On the need to measure LLM data contamination for each benchmark","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2023","author":"Sainz","year":"2023"},{"issue":"7","key":"2024092014251823700_bib78","doi-asserted-by":"publisher","first-page":"3751","DOI":"10.1007\/s41870-022-00937-6","article-title":"Leveraging genre classification with RNN for book recommendation","volume":"14","author":"Saraswat","year":"2022","journal-title":"International Journal of Information Technology"},{"key":"2024092014251823700_bib79","doi-asserted-by":"publisher","first-page":"8051","DOI":"10.18653\/v1\/2020.emnlp-main.647","article-title":"MLSUM: The multilingual summarization corpus","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Scialom","year":"2020"},{"key":"2024092014251823700_bib80","doi-asserted-by":"publisher","first-page":"57","DOI":"10.25080\/Majora-92bf1922-011","article-title":"Statsmodels: Econometric and statistical modeling with python","volume-title":"Proceedings of the Python in Science Conference","author":"Seabold","year":"2010"},{"key":"2024092014251823700_bib81","doi-asserted-by":"publisher","first-page":"4215","DOI":"10.18653\/v1\/2023.findings-emnlp.278","article-title":"Large language models are not yet human-level evaluators for abstractive summarization","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2023","author":"Shen","year":"2023"},{"key":"2024092014251823700_bib82","volume-title":"Analytics of Literature: A Manual for the Objective Study of English Prose and Poetry","author":"Sherman","year":"1893"},{"key":"2024092014251823700_bib83","article-title":"Comparing abstractive summaries generated by ChatGPT to real summaries through blinded reviewers and text classification algorithms","author":"Soni","year":"2023","journal-title":"arXiv preprint arXiv:2303.17650"},{"key":"2024092014251823700_bib84","first-page":"3008","article-title":"Multi-task learning for cross-lingual abstractive summarization","volume-title":"Proceedings of the Thirteenth Language Resources and Evaluation Conference","author":"Takase","year":"2022"},{"key":"2024092014251823700_bib85","article-title":"Multilingual translation with extensible multilingual pretraining and fine-tuning","author":"Tang","year":"2020","journal-title":"arXiv preprint arXiv:2008.00401"},{"key":"2024092014251823700_bib86","doi-asserted-by":"publisher","first-page":"9882","DOI":"10.18653\/v1\/2022.emnlp-main.672","article-title":"Exploring document-level literary machine translation with parallel paragraphs from world literature","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Thai","year":"2022"},{"key":"2024092014251823700_bib87","doi-asserted-by":"publisher","first-page":"1503","DOI":"10.18653\/v1\/N18-1136","article-title":"Identifying semantic divergences in parallel text without annotations","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Vyas","year":"2018"},{"key":"2024092014251823700_bib88","first-page":"917","article-title":"Cross-language document summarization based on machine translation quality prediction","volume-title":"Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics","author":"Wan","year":"2010"},{"key":"2024092014251823700_bib89","doi-asserted-by":"publisher","first-page":"5008","DOI":"10.18653\/v1\/2020.acl-main.450","article-title":"Asking and answering questions to evaluate the factual consistency of summaries","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Wang","year":"2020"},{"key":"2024092014251823700_bib90","doi-asserted-by":"publisher","first-page":"12","DOI":"10.18653\/v1\/2023.newsum-1.2","article-title":"Zero-shot cross-lingual summarization via large language models","volume-title":"Proceedings of the 4th New Frontiers in Summarization Workshop","author":"Wang","year":"2023"},{"key":"2024092014251823700_bib91","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1016\/j.langsci.2016.09.006","article-title":"The effects of genre on dependency distance and dependency direction","volume":"59","author":"Wang","year":"2017","journal-title":"Language Sciences"},{"key":"2024092014251823700_bib92","doi-asserted-by":"publisher","first-page":"272","DOI":"10.18653\/v1\/2022.acl-short.30","article-title":"When to use multi-task learning vs intermediate fine-tuning for pre-trained encoder transfer learning","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Weller","year":"2022"},{"key":"2024092014251823700_bib93","article-title":"Exploring the limits of ChatGPT for query or aspect-based text summarization","author":"Yang","year":"2023","journal-title":"arXiv preprint arXiv:2302.08081"},{"key":"2024092014251823700_bib94","doi-asserted-by":"publisher","first-page":"118","DOI":"10.18653\/v1\/D15-1012","article-title":"Phrase-based compressive cross-language summarization","volume-title":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","author":"Yao","year":"2015"},{"key":"2024092014251823700_bib95","first-page":"27263","article-title":"BARTScore: Evaluating generated text as text generation","volume-title":"Advances in Neural Information Processing Systems","author":"Yuan","year":"2021"},{"key":"2024092014251823700_bib96","first-page":"17283","article-title":"Big bird: Transformers for longer sequences","volume":"33","author":"Zaheer","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2024092014251823700_bib97","doi-asserted-by":"publisher","first-page":"3270","DOI":"10.18653\/v1\/2023.findings-emnlp.214","article-title":"Extractive summarization via ChatGPT for faithful summary generation","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2023","author":"Zhang","year":"2023"},{"key":"2024092014251823700_bib98","first-page":"11328","article-title":"Pegasus: Pre-training with extracted gap-sentences for abstractive summarization","volume-title":"International Conference on Machine Learning","author":"Zhang","year":"2020"},{"key":"2024092014251823700_bib99","article-title":"BERTScore: Evaluating text generation with BERT","volume-title":"International Conference on Learning Representations","author":"Zhang","year":"2020"},{"key":"2024092014251823700_bib100","doi-asserted-by":"publisher","first-page":"563","DOI":"10.18653\/v1\/D19-1053","article-title":"MoverScore: Text generation evaluating with contextualized embeddings and earth mover distance","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Zhao","year":"2019"},{"key":"2024092014251823700_bib101","doi-asserted-by":"publisher","first-page":"3865","DOI":"10.18653\/v1\/2023.eacl-main.278","article-title":"DiscoScore: Evaluating text generation with BERT and discourse coherence","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","author":"Zhao","year":"2023"},{"key":"2024092014251823700_bib102","doi-asserted-by":"publisher","first-page":"103307","DOI":"10.1016\/j.lingua.2022.103307","article-title":"Investigating diachronic change in dependency distance of modern English: A genre-specific perspective","volume":"272","author":"Zhu","year":"2022","journal-title":"Lingua"},{"key":"2024092014251823700_bib103","doi-asserted-by":"publisher","first-page":"3054","DOI":"10.18653\/v1\/D19-1302","article-title":"NCLS: Neural cross-lingual summarization","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Zhu","year":"2019"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/coli\/article-pdf\/50\/3\/1001\/2470940\/coli_a_00519.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/coli\/article-pdf\/50\/3\/1001\/2470940\/coli_a_00519.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T14:25:41Z","timestamp":1726842341000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/50\/3\/1001\/121095\/Cross-lingual-Cross-temporal-Summarization-Dataset"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":103,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,9,1]]},"published-print":{"date-parts":[[2024,9,1]]}},"URL":"https:\/\/doi.org\/10.1162\/coli_a_00519","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}