{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T05:00:10Z","timestamp":1775710810894,"version":"3.50.1"},"reference-count":111,"publisher":"MIT Press","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational Linguistics"],"published-print":{"date-parts":[[2017,12]]},"abstract":"<jats:p>In this article, we explore the potential of using sentence-level discourse structure for machine translation evaluation. We first design discourse-aware similarity measures, which use all-subtree kernels to compare discourse parse trees in accordance with the Rhetorical Structure Theory (RST). Then, we show that a simple linear combination with these measures can help improve various existing machine translation evaluation metrics regarding correlation with human judgments both at the segment level and at the system level. This suggests that discourse information is complementary to the information used by many of the existing evaluation metrics, and thus it could be taken into account when developing richer evaluation metrics, such as the WMT-14 winning combined metric DiscoTK<jats:sub>party<\/jats:sub>. We also provide a detailed analysis of the relevance of various discourse elements and relations from the RST parse trees for machine translation evaluation. In particular, we show that (i) all aspects of the RST tree are relevant, (ii) nuclearity is more useful than relation type, and (iii) the similarity of the translation RST tree to the reference RST tree is positively correlated with translation quality.<\/jats:p>","DOI":"10.1162\/coli_a_00298","type":"journal-article","created":{"date-parts":[[2017,9,11]],"date-time":"2017-09-11T15:00:35Z","timestamp":1505142035000},"page":"683-722","source":"Crossref","is-referenced-by-count":10,"title":["Discourse Structure in Machine Translation Evaluation"],"prefix":"10.1162","volume":"43","author":[{"given":"Shafiq","family":"Joty","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Nanyang Technological University."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Francisco","family":"Guzm\u00e1n","sequence":"additional","affiliation":[{"name":"Applied Machine Learning Group, Facebook."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Llu\u00eds","family":"M\u00e0rquez","sequence":"additional","affiliation":[{"name":"ALT, QCRI, HBKU, Qatar Foundation, HBKU Research Complex B1, P.O. Box 5825, Doha, Qatar."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Preslav","family":"Nakov","sequence":"additional","affiliation":[{"name":"ALT, QCRI, HBKU, Qatar Foundation, HBKU Research Complex B1, P.O. Box 5825, Doha, Qatar."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"bib2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1188"},{"key":"bib3","doi-asserted-by":"crossref","unstructured":"Becher, Viktor. 2011. When and why do translators add connectives? A corpus-based study. Target. International Journal of Translation Studies, 23(1):26\u201347.","DOI":"10.1075\/target.23.1.02bec"},{"key":"bib4","unstructured":"Ben, Guosheng, Deyi Xiong, Zhiyang Teng, Yajuan L\u00fc, and Qun Liu. 2013. Bilingual lexical cohesion trigger model for document-level machine translation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 382\u2013386, Sofia."},{"key":"bib5","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-3302"},{"key":"bib6","doi-asserted-by":"crossref","unstructured":"Bojar, Ond\u0159ej, Yvette Graham, Amir Kamran, and Milo\u0161 Stanojevi\u0107. 2016. Results of the WMT16 metrics shared task. In Proceedings of the First Conference on Machine Translation, pages 199\u2013231, Berlin.","DOI":"10.18653\/v1\/W16-2302"},{"key":"bib7","unstructured":"Brown, Peter F., Vincent J. Della Pietra, Stephen A. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263\u2013311."},{"key":"bib8","doi-asserted-by":"publisher","DOI":"10.3115\/1626355.1626373"},{"key":"bib9","doi-asserted-by":"publisher","DOI":"10.3115\/1626394.1626403"},{"key":"bib10","doi-asserted-by":"crossref","unstructured":"Callison-Burch, Chris, Philipp Koehn, Christof Monz, Kay Peterson, Mark Przybocki, and Omar Zaidan. 2010. Findings of the 2010 joint workshop on statistical machine translation and metrics for machine translation. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, pages 17\u201353, Uppsala.","DOI":"10.3115\/1626431.1626433"},{"key":"bib11","unstructured":"Callison-Burch, Chris, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2012. Findings of the 2012 workshop on statistical machine translation. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 10\u201351, Montr\u00e9al."},{"key":"bib12","doi-asserted-by":"crossref","unstructured":"Callison-Burch, Chris, Philipp Koehn, Christof Monz, and Josh Schroeder. 2009. Findings of the 2009 workshop on statistical machine translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 1\u201328, Athens.","DOI":"10.3115\/1626431.1626433"},{"key":"bib13","unstructured":"Callison-Burch, Chris, Philipp Koehn, Christof Monz, and Omar Zaidan. 2011. Findings of the 2011 workshop on statistical machine translation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 22\u201364, Edinburgh."},{"key":"bib14","unstructured":"Callison-Burch, Chris, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the role of BLEU in machine translation research. In Proceedings of 11th Conference of the European Chapter of the Association for Computational Linguistics, pages 249\u2013256, Trento."},{"key":"bib16","doi-asserted-by":"publisher","DOI":"10.3115\/1621969.1621974"},{"key":"bib17","unstructured":"Carpuat, Marine and Michel Simard. 2012. The trouble with SMT consistency. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 442\u2013449, Montreal."},{"key":"bib18","unstructured":"Carpuat, Marine and Dekai Wu. 2007. Improving statistical machine translation using word sense disambiguation. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 61\u201372, Prague."},{"key":"bib19","unstructured":"Cartoni, Bruno, Sandrine Zufferey, Thomas Meyer, and Andrei Popescu-Belis. 2011. How comparable are parallel corpora? Measuring the distribution of general vocabulary and connectives. In Proceedings of the 4th Workshop on Building and Using Comparable Corpora: Comparable Corpora and the Web, pages 78\u201386, Portland, OR."},{"key":"bib20","unstructured":"Chan, Yee Seng, Hwee Tou Ng, and David Chiang. 2007. Word sense disambiguation improves statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 33\u201340, Prague."},{"key":"bib21","doi-asserted-by":"crossref","unstructured":"Chiang, David. 2005. A hierarchical phrase-based model for statistical machine translation. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 263\u2013270, Ann Arbor, MI.","DOI":"10.3115\/1219840.1219873"},{"key":"bib23","doi-asserted-by":"crossref","unstructured":"Collins, Michael and Nigel Duffy. 2001. Convolution kernels for natural language. In Neural Information Processing Systems, pages 625\u2013632, Vancouver.","DOI":"10.7551\/mitpress\/1120.003.0085"},{"key":"bib24","unstructured":"Coughlin, Deborah. 2003. Correlating automated and human assessments of machine translation quality. In Proceedings of Machine Translation Summit IX, pages 23\u201327, New Orleans, LA."},{"key":"bib25","unstructured":"Culy, Christopher and Susanne Z. Riehemann. 2003. The limits of n-gram translation evaluation metrics. In Proceedings of Machine Translation Summit IX, pages 1\u20138, New Orleans, LA."},{"key":"bib26","unstructured":"Danlos, Laurence. 2009. D-STAG: A discourse analysis formalism based on synchronous tags. TAL, 50(1):111\u2013143."},{"key":"bib27","unstructured":"Denkowski, Michael and Alon Lavie. 2011. Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 85\u201391, Edinburgh."},{"key":"bib28","unstructured":"Denkowski, Michael and Alon Lavie. 2012. Challenges in predicting machine translation utility for human post-editors. In Proceedings of the Tenth Conference of the Association for Machine Translation in the Americas, San Diego, CA."},{"key":"bib30","doi-asserted-by":"crossref","unstructured":"Galley, Michel, Mark Hopkins, Kevin Knight, and Daniel Marcu. 2004. What's in a translation rule? In Proceedings of the 2004 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technology, pages 273\u2013280, San Diego, CA.","DOI":"10.21236\/ADA460212"},{"key":"bib31","doi-asserted-by":"publisher","DOI":"10.3115\/1626355.1626393"},{"key":"bib32","doi-asserted-by":"publisher","DOI":"10.3115\/1626431.1626479"},{"key":"bib33","doi-asserted-by":"publisher","DOI":"10.2478\/v10108-010-0022-6"},{"key":"bib34","unstructured":"Gim\u00e9nez, Jes\u00fas and Llu\u00eds M\u00e0rquez. 2010b. Linguistic measures for automatic machine translation evaluation. Machine Translation, 24(1):77\u201386."},{"key":"bib35","unstructured":"Gim\u00e9nez, Jes\u00fas, Llu\u00eds M\u00e0rquez, Elisabet Comelles, Irene Castell\u00f3n, and Victoria Arranz. 2010. Document-level automatic MT evaluation based on discourse representations. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, pages 333\u2013338, Uppsala."},{"key":"bib36","unstructured":"Gong, Zhengxian, Min Zhang, and Guodong Zhou. 2011. Cache-based document-level statistical machine translation. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 909\u2013919, Edinburgh."},{"key":"bib37","unstructured":"Gonz\u00e0lez, Meritxell, Jes\u00fas Gim\u00e9nez, and Llu\u00eds M\u00e0rquez. 2012. A graphical interface for MT evaluation and error analysis. In Proceedings of the ACL 2012 System Demonstrations, pages 139\u2013144, Jeju Island."},{"key":"bib38","unstructured":"Guillou, Liane. 2012. Improving pronoun translation for statistical machine translation. In Proceedings of the Student Research Workshop at the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 1\u201310, Avignon."},{"key":"bib39","unstructured":"Guillou, Liane. 2013. Analysing lexical consistency in translation. In Proceedings of the Workshop on Discourse in Machine Translation, pages 10\u201318, Sofia."},{"key":"bib40","doi-asserted-by":"crossref","unstructured":"Guillou, Liane, Christian Hardmeier, Preslav Nakov, Sara Stymne, J\u00f6rg Tiedemann, Yannick Versley, Mauro Cettolo, Bonnie Webber, and Andrei Popescu-Belis. 2016. Findings of the 2016 WMT shared task on cross-lingual pronoun prediction. In Proceedings of the First Conference on Machine Translation, pages 525\u2013542, Berlin.","DOI":"10.18653\/v1\/W16-2345"},{"key":"bib41","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1027"},{"key":"bib42","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-1065"},{"key":"bib43","doi-asserted-by":"crossref","unstructured":"Guzm\u00e1n, Francisco, Shafiq Joty, Llu\u00eds M\u00e0rquez, and Preslav Nakov. 2015. Pairwise neural machine translation evaluation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 805\u2013814, Beijing.","DOI":"10.3115\/v1\/P15-1078"},{"key":"bib44","unstructured":"Hajlaoui, Najeh. 2013. Are ACT's scores increasing with better translation quality? In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 408\u2013413, Sofia."},{"key":"bib45","unstructured":"Hajlaoui, Najeh and Andrei Popescu-Belis. 2012. Translating English discourse connectives into Arabic: a corpus-based analysis and an evaluation metric. In Fourth Workshop on Computational Approaches to Arabic Script-based Languages at Proceedings of the Tenth Biennial Conference of the Association for Machine Translation in the Americas."},{"key":"bib47","doi-asserted-by":"crossref","unstructured":"Hardmeier, Christian. 2012. Discourse in statistical machine translation. A survey and a case study. Discours. Revue de linguistique, psycholinguistique et informatique, 11(8726).","DOI":"10.4000\/discours.8726"},{"key":"bib49","unstructured":"Hardmeier, Christian and Marcello Federico. 2010. Modelling pronominal anaphora in statistical machine translation. In Proceedings of the International Workshop on Spoken Language Translation, pages 283\u2013289, Paris."},{"key":"bib50","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W15-2501"},{"key":"bib51","unstructured":"Hardmeier, Christian, Joakim Nivre, and J\u00f6rg Tiedemann. 2012. Document-wide decoding for phrase-based statistical machine translation. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1179\u20131190, Jeju Island."},{"key":"bib52","doi-asserted-by":"publisher","DOI":"10.1207\/s15516709cog0301_4"},{"key":"bib53","unstructured":"Hopkins, Mark and Jonathan May. 2011. Tuning as ranking. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Edinburgh."},{"key":"bib54","doi-asserted-by":"publisher","DOI":"10.1023\/A:1025510524115"},{"key":"bib55","unstructured":"Joty, Shafiq, Giuseppe Carenini, and Raymond T. Ng. 2012. A novel discriminative framework for sentence-level discourse analysis. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 904\u2013915, Jeju Island."},{"key":"bib56","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00226"},{"key":"bib57","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-3352"},{"key":"bib59","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/30.1-2.81"},{"key":"bib60","doi-asserted-by":"crossref","unstructured":"Koehn, Philipp, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, pages 48\u201354, Edmonton, Canada.","DOI":"10.3115\/1073445.1073462"},{"key":"bib61","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729694"},{"key":"bib62","doi-asserted-by":"publisher","DOI":"10.3115\/1626355.1626389"},{"key":"bib63","doi-asserted-by":"publisher","DOI":"10.1007\/s10590-009-9059-4"},{"key":"bib64","unstructured":"Le Nagard, Ronan and Philipp Koehn. 2010. Aiding pronoun translation with co-reference resolution. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, pages 252\u2013261, Uppsala."},{"key":"bib65","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1220"},{"key":"bib66","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-2047"},{"key":"bib67","unstructured":"Lin, Chin-Yew. 2004. ROUGE: A package for automatic evaluation of summaries. In Proceedings of Workshop on Text Summarization Branches Out, pages 74\u201381, Barcelona."},{"key":"bib68","unstructured":"Liu, Ding and Daniel Gildea. 2005. Syntactic features for evaluation of machine translation. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization, pages 25\u201332, Ann Arbor, MI."},{"key":"bib69","unstructured":"Lo, Chi kiu, Anand Karthik Tumuluru, and Dekai Wu. 2012. Fully automatic semantic MT evaluation. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 243\u2013252, Montr\u00e9al."},{"key":"bib70","unstructured":"Lo, Chi kiu and Dekai Wu. 2011. Meant: An inexpensive, high-accuracy, semi-automatic metric for evaluating translation utility based on semantic roles. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 220\u2013229, Portland, OR."},{"key":"bib71","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/E14-1017"},{"key":"bib72","doi-asserted-by":"crossref","unstructured":"Mach\u00e1\u010dek, Matou\u0161 and Ond\u0159ej Bojar. 2013. Results of the WMT13 metrics shared task. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 45\u201351, Sofia.","DOI":"10.3115\/v1\/W14-3336"},{"key":"bib73","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-3336"},{"key":"bib74","doi-asserted-by":"publisher","DOI":"10.1515\/text.1.1988.8.3.243"},{"key":"bib75","unstructured":"Marcu, Daniel, Lynn Carlson, and Maki Watanabe. 2000. The automatic translation of discourse structures. In Proceedings of the 1st North American Chapter of the Association for Computational Linguistics Conference, pages 9\u201317, Seattle, WA."},{"key":"bib76","unstructured":"Meyer, Thomas. 2011. Disambiguating temporal-contrastive connectives for machine translation. In Proceedings of the ACL 2011 Student Session, pages 46\u201351, Portland, OR."},{"key":"bib78","unstructured":"Meyer, Thomas and Lucie Pol\u00e1kov\u00e1. 2013. Machine translation with many manually labeled discourse connectives. In Proceedings of the Workshop on Discourse in Machine Translation, pages 43\u201350, Sofia."},{"key":"bib79","unstructured":"Meyer, Thomas and Andrei Popescu-Belis. 2012. Using sense-labeled discourse connectives for statistical machine translation. In Proceedings of the Joint Workshop on Exploiting Synergies between Information Retrieval and Machine Translation (ESIRMT) and Hybrid Approaches to Machine Translation (HyTra), pages 129\u2013138, Avignon."},{"key":"bib80","unstructured":"Meyer, Thomas, Andrei Popescu-Belis, Najeh Hajlaoui, and Andrea Gesmundo. 2012. Machine translation of labeled discourse connectives. In Proceedings of the Tenth Biennial Conference of the Association for Machine Translation in the Americas (AMTA), San Diego, CA."},{"key":"bib81","unstructured":"Meyer, Thomas and Bonnie Webber. 2013. Implicitation of discourse connectives in (machine) translation. In Proceedings of the Workshop on Discourse in Machine Translation, pages 19\u201326, Sofia."},{"key":"bib82","doi-asserted-by":"crossref","unstructured":"Moschitti, Alessandro. 2006. Efficient convolution kernels for dependency and constituent syntactic trees. In 17th European Conference on Machine Learning, pages 318\u2013329, Berlin.","DOI":"10.1007\/11871842_32"},{"key":"bib83","doi-asserted-by":"crossref","unstructured":"Moschitti, Alessandro. 2008. Kernel methods, syntax and semantics for relational text categorization. In Proceedings of the 17th ACM Conference on Information and Knowledge Management, pages 253\u2013262, Napa Valley, CA.","DOI":"10.1145\/1458082.1458118"},{"key":"bib84","unstructured":"Nadejde, Maria, Philip Williams, and Philipp Koehn. 2013. Edinburgh's syntax-based machine translation systems. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 170\u2013176, Sofia."},{"key":"bib85","unstructured":"Nov\u00e1k, Michal, Anna Nedoluzhko, and Zden\u011bk \u017dabokrtsk\u00fd. 2013. Translation of \u201cit\u201d in a deep syntax framework. In Proceedings of the Workshop on Discourse in Machine Translation, pages 51\u201359, Sofia."},{"key":"bib86","doi-asserted-by":"publisher","DOI":"10.3115\/1075096.1075117"},{"key":"bib87","doi-asserted-by":"crossref","unstructured":"Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the Association for Computational Linguistics, Philadelphia, PA.","DOI":"10.3115\/1073083.1073135"},{"key":"bib88","doi-asserted-by":"publisher","DOI":"10.1098\/rspl.1895.0041"},{"key":"bib89","unstructured":"Pitler, Emily and Ani Nenkova. 2009. Using syntax to disambiguate explicit discourse connectives in text. In Proceedings of the Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing Conference, pages 13\u201316, Suntec."},{"key":"bib90","unstructured":"Popescu-Belis, Andrei, Thomas Meyer, Jeevanthi Liyanapathirana, Bruno Cartoni, and Sandrine Zufferey. 2012. Discourse-level annotation over Europarl for machine translation: Connectives and pronouns. In Proceedings of the Eighth International Conference on Language Resources and Evaluation, pages 2716\u20132720, Istanbul."},{"key":"bib91","doi-asserted-by":"publisher","DOI":"10.3115\/1626355.1626362"},{"key":"bib92","unstructured":"Prasad, Rashmi, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn discourse treebank 2.0. In Proceedings of the Sixth International Conference on Language Resources and Evaluation, pages 921\u2013950, Marrakech."},{"key":"bib93","unstructured":"Punyakanok, Vasin and Dan Roth. 2001. The use of classifiers in sequential inference. In Advances in Neural Information Processing Systems 14, pages 995\u20131001, Vancouver."},{"key":"bib94","doi-asserted-by":"crossref","unstructured":"Quirk, Chris, Arul Menezes, and Colin Cherry. 2005. Dependency treelet translation: Syntactically informed phrasal SMT. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 271\u2013279, Ann Arbor, MI.","DOI":"10.3115\/1219840.1219874"},{"key":"bib95","doi-asserted-by":"publisher","DOI":"10.3115\/1119176.1119178"},{"key":"bib96","unstructured":"Smola, Alex and S.v.n. Vishwanathan. 2003. Fast kernels for string and tree matching. In Advances in Neural Information Processing Systems 15, pages 585\u2013592, Vancouver."},{"key":"bib97","unstructured":"Snover, Matthew, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Biennial Conference of the Association for Machine Translation in the Americas (AMTA), pages 223\u2013231, Cambridge, MA."},{"key":"bib98","doi-asserted-by":"publisher","DOI":"10.2307\/1412159"},{"key":"bib99","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W15-3031"},{"key":"bib100","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-2015"},{"key":"bib101","unstructured":"Sutton, Charles, Andrew McCallum, and Khashayar Rohanimanesh. 2007. Dynamic conditional random fields: Factorized probabilistic models for labeling and segmenting sequence data. Journal of Machine Learning Research (JMLR), 8:693\u2013723."},{"key":"bib102","doi-asserted-by":"publisher","DOI":"10.1177\/1461445606061881"},{"key":"bib103","doi-asserted-by":"publisher","DOI":"10.1145\/322139.322143"},{"key":"bib104","unstructured":"Taira, Hirotoshi, Katsuhito Sudoh, and Masaaki Nagata. 2012. Zero pronoun resolution can improve the quality of j-e translation. In Proceedings of the Sixth Workshop on Syntax, Semantics and Structure in Statistical Translation, pages 111\u2013118, Jeju Island."},{"key":"bib105","unstructured":"Tiedemann, J\u00f6rg. 2010a. Context adaptation in statistical machine translation using models with exponentially decaying cache. In Proceedings of the 2010 Workshop on Domain Adaptation for Natural Language Processing, pages 8\u201315, Uppsala."},{"key":"bib106","unstructured":"Tiedemann, J\u00f6rg. 2010b. To cache or not to cache? Experiments with adaptive models in statistical machine translation. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, pages 189\u2013194, Uppsala."},{"key":"bib107","unstructured":"Tu, Mei, Yu Zhou, and Chengqing Zong. 2013. A novel translation framework based on rhetorical structure theory. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 370\u2013374, Sofia."},{"key":"bib108","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-1080"},{"key":"bib109","unstructured":"Ture, Ferhan, Douglas W. Oard, and Philip Resnik. 2012. Encouraging consistent translation choices. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 417\u2013426, Montr\u00e9al."},{"key":"bib110","doi-asserted-by":"publisher","DOI":"10.3115\/1220575.1220672"},{"key":"bib111","unstructured":"Voigt, Rob and Dan Jurafsky. 2012. Towards a literary machine translation: The role of referential cohesion. In Proceedings of the NAACL-HLT 2012 Workshop on Computational Linguistics for Literature, pages 18\u201325, Montr\u00e9al."},{"key":"bib112","unstructured":"Watanabe, Taro, Jun Suzuki, Hajime Tsukada, and Hideki Isozaki. 2007. Online large-margin training for statistical machine translation. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 764\u2013773, Prague."},{"key":"bib113","doi-asserted-by":"publisher","DOI":"10.1207\/s15516709cog2805_6"},{"key":"bib117","unstructured":"Wong, Billy T. M. and Chunyu Kit. 2012. Extending machine translation evaluation metrics with lexical cohesion to document level. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1060\u20131068, Jeju Island."},{"key":"bib118","doi-asserted-by":"crossref","unstructured":"Wong, Billy T. M., Cecilia F. K. Pun, Chunyu Kit, and Jonathan J. Webster. 2011. Lexical cohesion for evaluation of machine translation at document level. In Proceedings of the 7th International Conference on Natural Language Processing and Knowledge Engineering, pages 238\u2013242, Tokushima.","DOI":"10.1109\/NLPKE.2011.6138201"},{"key":"bib119","doi-asserted-by":"crossref","unstructured":"Wu, Dekai and Pascale Fung. 2009. Semantic roles for SMT: A hybrid two-pass model. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 13\u201316, Boulder, CO.","DOI":"10.3115\/1620853.1620858"},{"key":"bib120","doi-asserted-by":"crossref","unstructured":"Xiong, Deyi, Yang Ding, Min Zhang, and Chew Lim Tan. 2013. Lexical chain based cohesion models for document-level statistical machine translation. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1563\u20131573, Seattle, WA.","DOI":"10.18653\/v1\/D13-1163"},{"key":"bib121","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2015.2395254"},{"key":"bib122","doi-asserted-by":"publisher","DOI":"10.3115\/992730.992783"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/COLI_a_00298","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T17:42:38Z","timestamp":1750873358000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/43\/4\/683-722\/1580"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,12]]},"references-count":111,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2017,12]]}},"alternative-id":["10.1162\/COLI_a_00298"],"URL":"https:\/\/doi.org\/10.1162\/coli_a_00298","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,12]]}}}