{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:52:02Z","timestamp":1750308722440,"version":"3.41.0"},"reference-count":32,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2013,6,1]],"date-time":"2013-06-01T00:00:00Z","timestamp":1370044800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000185","name":"Defense Advanced Research Projects Agency","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000185","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004316","name":"International Business Machines Corporation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004316","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2013,6]]},"abstract":"<jats:p>Today's Statistical Machine Translation (SMT) systems require high-quality human translations for parameter tuning, in addition to large bitexts for learning the translation units. This parameter tuning usually involves generating translations at different points in the parameter space and obtaining feedback against human-authored reference translations as to how good the translations. This feedback then dictates what point in the parameter space should be explored next. To measure this feedback, it is generally considered wise to have multiple (usually 4) reference translations to avoid unfair penalization of translation hypotheses which could easily happen given the large number of ways in which a sentence can be translated from one language to another. However, this reliance on multiple reference translations creates a problem since they are labor intensive and expensive to obtain. Therefore, most current MT datasets only contain a single reference. This leads to the problem of reference sparsity. In our previously published research, we had proposed the first paraphrase-based solution to this problem and evaluated its effect on Chinese-English translation. In this article, we first present extended results for that solution on additional source languages. More importantly, we present a novel way to generate \u201ctargeted\u201d paraphrases that yields substantially larger gains (up to 2.7 BLEU points) in translation quality when compared to our previous solution (up to 1.6 BLEU points). In addition, we further validate these improvements by supplementing with human preference judgments obtained via Amazon Mechanical Turk.<\/jats:p>","DOI":"10.1145\/2483669.2483673","type":"journal-article","created":{"date-parts":[[2013,7,1]],"date-time":"2013-07-01T12:27:28Z","timestamp":1372681648000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Generating targeted paraphrases for improved translation"],"prefix":"10.1145","volume":"4","author":[{"given":"Nitin","family":"Madnani","sequence":"first","affiliation":[{"name":"Educational Testing Service, Princeton, NJ"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bonnie J.","family":"Dorr","sequence":"additional","affiliation":[{"name":"University of Maryland, College Park, MD"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2013,7]]},"reference":[{"doi-asserted-by":"publisher","key":"e_1_2_2_1_1","DOI":"10.3115\/1219840.1219914"},{"doi-asserted-by":"publisher","key":"e_1_2_2_2_1","DOI":"10.5555\/92858.92860"},{"doi-asserted-by":"publisher","key":"e_1_2_2_3_1","DOI":"10.5555\/972470.972474"},{"doi-asserted-by":"publisher","key":"e_1_2_2_4_1","DOI":"10.5555\/1613715.1613743"},{"volume-title":"Proceedings of the 3rd Workshop on Statistical Machine Translation. 70--106","author":"Callison-Burch C.","unstructured":"Callison-Burch , C. , Fordyce , C. , Koehn , P. , Monz , C. , and Schroeder , J . 2008. Further meta-evaluation of machine translation . In Proceedings of the 3rd Workshop on Statistical Machine Translation. 70--106 . Callison-Burch, C., Fordyce, C., Koehn, P., Monz, C., and Schroeder, J. 2008. Further meta-evaluation of machine translation. In Proceedings of the 3rd Workshop on Statistical Machine Translation. 70--106.","key":"e_1_2_2_5_1"},{"volume-title":"Proceedings of the 3rd Workshop on Statistical Machine Translation.","author":"Callison-Burch C.","unstructured":"Callison-Burch , C. , Koehn , P. , Monz , C. , Schroeder , J. , and Fordyce , C. S. , Eds . 2008 . Proceedings of the 3rd Workshop on Statistical Machine Translation. Callison-Burch, C., Koehn, P., Monz, C., Schroeder, J., and Fordyce, C. S., Eds. 2008. Proceedings of the 3rd Workshop on Statistical Machine Translation.","key":"e_1_2_2_6_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_7_1","DOI":"10.3115\/1220835.1220838"},{"unstructured":"Chen S. F. and Goodman J. 1998. An empirical study of smoothing techniques for language modeling. Tech. rep. Computer Science Group Harvard University.  Chen S. F. and Goodman J. 1998. An empirical study of smoothing techniques for language modeling. Tech. rep. Computer Science Group Harvard University.","key":"e_1_2_2_8_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_9_1","DOI":"10.1162\/coli.2007.33.2.201"},{"doi-asserted-by":"publisher","key":"e_1_2_2_10_1","DOI":"10.3115\/1220575.1220673"},{"volume-title":"Proceedings of the NAACL Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk. 66--70","author":"Denkowski M.","unstructured":"Denkowski , M. , Al-Haj , H. , and Lavie , A . 2010. Turker-assisted paraphrasing for english-arabic machine translation . In Proceedings of the NAACL Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk. 66--70 . Denkowski, M., Al-Haj, H., and Lavie, A. 2010. Turker-assisted paraphrasing for english-arabic machine translation. In Proceedings of the NAACL Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk. 66--70.","key":"e_1_2_2_11_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_12_1","DOI":"10.3115\/1220835.1220893"},{"key":"e_1_2_2_13_1","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).","author":"Koehn P.","year":"2004","unstructured":"Koehn , P. 2004 . Statistical significance tests for machine translation evaluation . In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Koehn, P. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)."},{"key":"e_1_2_2_14_1","volume-title":"Proceedings of the Machine Translation Summit.","author":"Koehn P.","year":"2005","unstructured":"Koehn , P. 2005 . Europarl: A parallel corpus for statistical machine translation . In Proceedings of the Machine Translation Summit. Koehn, P. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of the Machine Translation Summit."},{"volume-title":"Statistical Machine Translation","author":"Koehn P.","unstructured":"Koehn , P. 2010. Statistical Machine Translation . Cambridge University Press . Koehn, P. 2010. Statistical Machine Translation. Cambridge University Press.","key":"e_1_2_2_15_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_16_1","DOI":"10.3115\/1073445.1073462"},{"doi-asserted-by":"publisher","key":"e_1_2_2_17_1","DOI":"10.5555\/974305.974328"},{"volume-title":"Proceedings of the 2nd Workshop on Statistical Machine Translation. 228--231","author":"Lavie A.","unstructured":"Lavie , A. and Agarwal , A . 2007. METEOR: An automatic metric for mt evaluation with high levels of correlation with human judgments . In Proceedings of the 2nd Workshop on Statistical Machine Translation. 228--231 . Lavie, A. and Agarwal, A. 2007. METEOR: An automatic metric for mt evaluation with high levels of correlation with human judgments. In Proceedings of the 2nd Workshop on Statistical Machine Translation. 228--231.","key":"e_1_2_2_18_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_19_1","DOI":"10.1145\/1380584.1380586"},{"volume-title":"Proceedings of the 2nd Workshop on Statistical Machine Translation.","author":"Madnani N.","unstructured":"Madnani , N. , Ayan , N. F. , Resnik , P. , and Dorr , B. J . 2007. Using paraphrases for parameter tuning in statistical machine translation . In Proceedings of the 2nd Workshop on Statistical Machine Translation. Madnani, N., Ayan, N. F., Resnik, P., and Dorr, B. J. 2007. Using paraphrases for parameter tuning in statistical machine translation. In Proceedings of the 2nd Workshop on Statistical Machine Translation.","key":"e_1_2_2_21_1"},{"volume-title":"Proceedings of the Conference of the Association for Machine Translation in the Americas (AMTA).","author":"Madnani N.","unstructured":"Madnani , N. , Resnik , P. , Dorr , B. J. , and Schwartz , R . 2008. Are multiple reference translations necessary&quest; Investigating the value of paraphrased reference translations in parameter optimization . In Proceedings of the Conference of the Association for Machine Translation in the Americas (AMTA). Madnani, N., Resnik, P., Dorr, B. J., and Schwartz, R. 2008. Are multiple reference translations necessary&quest; Investigating the value of paraphrased reference translations in parameter optimization. In Proceedings of the Conference of the Association for Machine Translation in the Americas (AMTA).","key":"e_1_2_2_22_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_23_1","DOI":"10.3115\/1118693.1118711"},{"volume-title":"Proceedings of the Conference of the Association for Computational Linguistics (ACL). 192--199","author":"Mi H.","unstructured":"Mi , H. , Huang , L. , and Liu , Q . 2008. Forest-based translation . In Proceedings of the Conference of the Association for Computational Linguistics (ACL). 192--199 . Mi, H., Huang, L., and Liu, Q. 2008. Forest-based translation. In Proceedings of the Conference of the Association for Computational Linguistics (ACL). 192--199.","key":"e_1_2_2_24_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_25_1","DOI":"10.1145\/974740.974741"},{"doi-asserted-by":"publisher","key":"e_1_2_2_26_1","DOI":"10.3115\/1075096.1075117"},{"unstructured":"Olive J. 2005. Global autonomous language exploitation. DARPA\/IPTO Proposer Information Pamphlet.  Olive J. 2005. Global autonomous language exploitation. DARPA\/IPTO Proposer Information Pamphlet.","key":"e_1_2_2_27_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_28_1","DOI":"10.3115\/112405.112416"},{"doi-asserted-by":"publisher","key":"e_1_2_2_29_1","DOI":"10.3115\/1073083.1073135"},{"doi-asserted-by":"publisher","key":"e_1_2_2_30_1","DOI":"10.1162\/coli_a_00015"},{"volume-title":"Proceedings of the Conference of the Association for Machine Translation in the Americas (AMTA).","author":"Snover M.","unstructured":"Snover , M. , Dorr , B. J. , Schwartz , R. , Micciulla , L. , and Makhoul , J . 2006. A study of translation edit rate with targeted human annotation . In Proceedings of the Conference of the Association for Machine Translation in the Americas (AMTA). Snover, M., Dorr, B. J., Schwartz, R., Micciulla, L., and Makhoul, J. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the Conference of the Association for Machine Translation in the Americas (AMTA).","key":"e_1_2_2_31_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_32_1","DOI":"10.1007\/s10590-009-9062-9"},{"volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). 52--61","author":"Zaidan O. F.","unstructured":"Zaidan , O. F. and Callison-Burch , C . 2009. Feasibility of human-in-the-loop minimum error rate training . In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). 52--61 . Zaidan, O. F. and Callison-Burch, C. 2009. Feasibility of human-in-the-loop minimum error rate training. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). 52--61.","key":"e_1_2_2_33_1"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2483669.2483673","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2483669.2483673","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T20:14:36Z","timestamp":1750277676000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2483669.2483673"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,6]]},"references-count":32,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2013,6]]}},"alternative-id":["10.1145\/2483669.2483673"],"URL":"https:\/\/doi.org\/10.1145\/2483669.2483673","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"type":"print","value":"2157-6904"},{"type":"electronic","value":"2157-6912"}],"subject":[],"published":{"date-parts":[[2013,6]]},"assertion":[{"value":"2011-02-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2011-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-07-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}