{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T13:12:55Z","timestamp":1740143575894,"version":"3.37.3"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2016,2,9]],"date-time":"2016-02-09T00:00:00Z","timestamp":1454976000000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Lang Resources & Evaluation"],"published-print":{"date-parts":[[2017,12]]},"DOI":"10.1007\/s10579-016-9335-x","type":"journal-article","created":{"date-parts":[[2016,2,9]],"date-time":"2016-02-09T10:37:21Z","timestamp":1455014241000},"page":"1053-1084","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Ebaluatoia: crowd evaluation for English\u2013Basque machine translation"],"prefix":"10.1007","volume":"51","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3719-9167","authenticated-orcid":false,"given":"Nora","family":"Aranberri","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gorka","family":"Labaka","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arantza","family":"D\u00edaz de Ilarraza","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kepa","family":"Sarasola","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2016,2,9]]},"reference":[{"key":"9335_CR1","unstructured":"Agirre, E., Atutxa, A., Labaka, G., Lersundi, M., Mayor, A., & Sarasola, K. (2009). Use of rich linguistic information to translate prepositions and grammar cases to Basque. In L. M\u00e0rquez & H. Somers (Eds.), Proceedings of the XIII conference of the European association for machine translation (EAMT 2009) (pp. 58-65). Barcelona, May 14\u201315. 2009."},{"key":"9335_CR2","unstructured":"Alegria, I., Cabezon, U., Fernandez de Beto\u00f1o, U., Labaka, G., Mayor, A., Sarasola, K., & Zubiaga, A. (2013). Reciprocal enrichment between basque Wikipedia and machine translators. In I. Gurevych & J. Kim (Eds.), The people\u2019s web meets NLP: Collaboratively constructed language resources, Springer, Book series Theory and applications of natural language processing, E. Hovy, M. Johnson & G. Hirst (eds.)."},{"key":"9335_CR3","unstructured":"Al-Haj, H. & Lavie, A. (2010). The impact of Arabic morphological segmentation on broad-coverage English-to-Arabic statistical machine translation. In Proceedings of the ninth conference of the association for machine translation in the Americas, Denver, Colorado, October 31\u2013November 4, 2010."},{"key":"9335_CR4","unstructured":"Aranberri, N., Labaka, G., Diaz de Ilarraza, A. & Sarasola, K. (2014). Comparison of post-editing productivity between professional translators and lay users. In Proceedings of the third workshop on post-editing technology and practice (WPTP 2014) (pp. 20\u201333). Vancouver, Canada, October 22\u201326, 2014."},{"key":"9335_CR50","unstructured":"Aranberri, N., Labaka, G., D\u00edaz de Ilarraza, A. & Sarasola, K. (2015). Exploiting portability to build an RBMT prototype for a new source language. In Proceedings of the XVIII annual conference of the European association for machine translation (EAMT 2015) (pp. 3\u201310). Antalya, Turkey, May 11\u201313, 2015."},{"key":"9335_CR5","unstructured":"Aranberri, N., & O\u2019Brien, S. (2009). Evaluating MT output for -ing forms: A study of four target languages. In W. Daelemans & V. Hoste (Eds.), Linguistica antverpiensia new series, Themes in Translation Studies (Vol. 8, pp. 105\u2013122)."},{"key":"9335_CR6","doi-asserted-by":"crossref","unstructured":"Arrieta, K., D\u00edaz de Ilarraza, A., Hern\u00e1ez, I., Iturraspe, U., Leturia, I., Navas, E., & Sarasola, K. (2008). AnHitz, development and integration of language, speech and visual technologies for Basque. In Second international symposium on Universal communication (pp. 338\u2013343). Osaka, Japan.","DOI":"10.1109\/ISUC.2008.43"},{"key":"9335_CR7","unstructured":"Bojar, O., Buck, C., Callison-Burch, C., Federmann, C., Haddow, B., Koehn, P. et al. (2013). Findings of the 2013 workshop on statistical machine translation. In Proceedings of the 8th workshop on statistical machine translation, WMT-2013, Sofia, Bulgaria (pp. 1\u201344). Sofia, Bulgaria."},{"key":"9335_CR8","doi-asserted-by":"crossref","unstructured":"Bojar, O., Buck, C. Federmann, C., Haddow, B., Koehn, P., Mach\u00e1c\u0115k, M. et al. (2014). Findings of the 2014 workshop on statistical machine translation. In Proceedings of the ninth workshop on statistical machine translation, Balrimore, USA, June (pp. 12\u201358). Association for Computational Linguistics.","DOI":"10.3115\/v1\/W14-3302"},{"key":"9335_CR9","doi-asserted-by":"crossref","unstructured":"Callison-Burch, C., Fordyce, C., Koehn, P., Monz, C., & Schroeder, J. (2007). (Meta-)evaluation of machine translation. In Proceedings of the second workshop on statistical machine translation, June 2007 (pp. 136\u2013158). Prague, Czech Republic.","DOI":"10.3115\/1626355.1626373"},{"key":"9335_CR10","doi-asserted-by":"crossref","unstructured":"Callison-Burch, C., Fordyce, C., Koehn, P., Monz, C., & Schroeder, J. (2008). Further meta-evaluation of machine translation. In Proceedings of the third workshop on statistical machine translation, June 2008 (pp. 70\u2013106). Columbus, Ohio.","DOI":"10.3115\/1626394.1626403"},{"key":"9335_CR11","unstructured":"Callison-Burch, C., Koehn, P., Monz, C., Post, M. Soricut, R., & Specia, L. (2012). Findings of the 2012 workshop on statistical machine translation. In Proceedings of the seventh workshop on statistical machine translation, Montr\u00e9al, Canada, June (pp. 10\u201351). Association for Computational Linguistics."},{"key":"9335_CR12","unstructured":"Callison-Burch, C., Koehn, P., Monz, C., Peterson, K., Przybocky, M., & Zaidan, O. (2010). Findings of the 2010 joint workshop on statistical machine translation and metrics for machine translation. In Proceedings of the Joint 5th workshop on statistical machine translation and MetricsMATR (pp. 17\u201353). Uppsala, Sweden, 15\u201316 July 2010."},{"key":"9335_CR13","doi-asserted-by":"crossref","unstructured":"Callison-Burch, C., Koehn, P., Monz, C., & Schroeder, J. (2009). Findings of the 2009 workshop on statistical machine translation. In Proceedings of the fourth workshop on statistical machine translation, March 30\u201331, 2009, Athens, Greece (pp. 1\u201328).","DOI":"10.3115\/1626431"},{"key":"9335_CR14","unstructured":"Callison-Burch, C., Koehn, P., Monz, C., & Zaidan, O. (2011). Findings of the 2011 workshop on statistical machine translation. In Proceedings of WMT (pp. 22\u201364). Edinburgh, Scotland, UK."},{"issue":"1","key":"9335_CR15","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurment, 20(1), 37\u201346.","journal-title":"Educational and Psychological Measurment"},{"key":"9335_CR16","unstructured":"Espa\u00f1a-Bonet, C., Labaka, G., Diaz de Ilarraza, A., M\u00e0rquez, L., & Sarasola, K. (2011). Hybrid machine translation guided by a rule\u2013based system. In Proceedings of the thirteenth machine translation Summit [organized by the] Asia-Pacific association for machine translation (AAMT2011) (pp. 554\u2013561). Xiamen, China."},{"key":"9335_CR17","doi-asserted-by":"crossref","first-page":"130","DOI":"10.2478\/v10108-012-0006-9","volume":"98","author":"C Federmann","year":"2012","unstructured":"Federmann, C. (2012). Appraise: An OpenSource toolkit for manual evaluation of MT output. The Prague Bulletin of Mathematical Linguistics, 98, 130\u2013134.","journal-title":"The Prague Bulletin of Mathematical Linguistics"},{"key":"9335_CR18","unstructured":"Fishel, M., Kaalep, H., & Muischnek, K. (2007). Estonian-english statistical machine translation: the first results. In Proceedings of the sixteenth Nordic conference of computational linguistics (NODALIDA-2007), Tartu, Estonia, May 2007."},{"key":"9335_CR19","doi-asserted-by":"crossref","unstructured":"Habash, N., & Sadat, F. (2006). Arabic preprocessing schemes for statistical machine translation. In Proceedings of the human language technology conference of the North American chapter of the ACL (pp. 49\u201352). New York, NY, USA, June 2006.","DOI":"10.3115\/1614049.1614062"},{"key":"9335_CR20","doi-asserted-by":"crossref","unstructured":"Khadivi, S., & Ney, H. (2005). Automatic filtering of bilingual corpora for statistical machine translation. In Proceedings 10th international conference on application of natural language to information systems, NLDB 2005 (pp. 263\u2013274). Springer, LNCS, Alicante, Spain, June 2005.","DOI":"10.1007\/11428817_24"},{"key":"9335_CR21","unstructured":"Khalilov, M., Fonollosa, J. A. R., Skadina, I., Bralitis, E., & Pretkalnina, L. (2010a). Towards improving english-latvian translation: A system comparison and a new rescoring feature. In Proceedings of the seventh international conference on language resources and evaluation (LREC\u201910) (pp. 1719\u20131725). Valetta, Malta, May 2010."},{"key":"9335_CR22","unstructured":"Khalilov, M., Pretkalnin, L., Kuvaldina, N., & Pereseina, V. (2010b). SMT of latvian, lithuanian and estonian languages: A comparative study. In Proceedings of the fourth international conference: human language technologies\u2014The baltic perspective. Riga, Latvia, October 7\u20138, 2010."},{"key":"9335_CR23","doi-asserted-by":"crossref","unstructured":"Klein, D., & Manning, C. D. (2003). Accurate unlexicalized parsing. In Proceedings of the 41st meeting of the association for computational linguistics (pp. 423\u2013430).","DOI":"10.3115\/1075096.1075150"},{"key":"9335_CR51","doi-asserted-by":"crossref","unstructured":"Koehn, P., & Monz, C. (2006). Manual and automatic evaluation of machine translation between European languages. In Proceedings of the workshop on statistical machine translation (pp. 102\u2013121). New York, June 2006.","DOI":"10.3115\/1654650.1654666"},{"key":"9335_CR24","doi-asserted-by":"crossref","unstructured":"Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., et al. (2007). Moses: Open source toolkit for statistical machine translation. In Annual meeting of the association for computational linguistics (ACL), demonstration session, Prague, Czech Republic, June 2007.","DOI":"10.3115\/1557769.1557821"},{"key":"9335_CR25","unstructured":"Labaka, G. (2010). EUSMT: Incorporating linguistic information into SMT for a morphologically rich language. Its use in SMT-RBMT-EBMT hybridation. Ph.D. Thesis. University of the Basque Country."},{"issue":"2","key":"9335_CR26","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1007\/s10590-014-9153-0","volume":"28","author":"G Labaka","year":"2014","unstructured":"Labaka, G., Espa\u00f1a-Bonet, C., M\u00e0rquez, L., & Sarasola, K. (2014). A hybrid machine translation architecture guided by syntax. Machine Translation Journal, 28(2), 91\u2013125.","journal-title":"Machine Translation Journal"},{"key":"9335_CR27","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","volume":"33","author":"JR Landis","year":"1977","unstructured":"Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159\u2013174.","journal-title":"Biometrics"},{"key":"9335_CR28","doi-asserted-by":"crossref","unstructured":"Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S. J., & McClosky, D. (2014). The stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations (pp. 55\u201360).","DOI":"10.3115\/v1\/P14-5010"},{"issue":"1","key":"9335_CR29","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1007\/s10590-011-9092-y","volume":"25","author":"A Mayor","year":"2011","unstructured":"Mayor, A., Alegria, I., D\u00edaz de Ilarraza, A., Labaka, G., Lersundi, M., & Sarasola, K. (2011). Matxin, an open-source rule-based machine translation system for Basque. Machine Translation Journal, 25(1), 53\u201382.","journal-title":"Machine Translation Journal"},{"key":"9335_CR30","unstructured":"Mitchell, L., Roturier, J., Silva, D. (2014). Using the ACCEPT framework to conduct an online community-based translation evaluation study. In Proceedings of the seventeenth annual conference of the European association for machine translation (EAMT), June 2014, Dubrovnik, Croatia."},{"key":"9335_CR31","unstructured":"Naradowsky, J., & Toutanova, K. (2011). Unsupervised bilingual morpheme segmentation and alignment with context-rich hidden semi-Markov models. In Proceedings of the 49th annual meeting of the association for computational linguistics (pp. 895\u2013904). Portland, Oregon, June 19\u201324, 2011."},{"key":"9335_CR32","doi-asserted-by":"crossref","unstructured":"Och, F. J. (2003). Minimum error rate training in statistical machine translation. In Proceedings of the 41st annual meeting of the association for computational linguistics (ACL) (pp. 160\u2013167). Sapporo, Japan, July.","DOI":"10.3115\/1075096.1075117"},{"key":"9335_CR33","doi-asserted-by":"crossref","unstructured":"Oflazer, K., & El-Kahlout, I. D. (2007). Exploring different representation units in English-to-Turkish statistical machine translation. In Proceedings of the second workshop on statistical machine translation (pp. 25\u201332). Prague, Czech Republic.","DOI":"10.3115\/1626355.1626359"},{"key":"9335_CR34","doi-asserted-by":"crossref","first-page":"7","DOI":"10.2478\/v10108-010-0010-x","volume":"93","author":"M Plitt","year":"2010","unstructured":"Plitt, M., & Masselot, F. (2010). A productivity test of statistical machine translation post-editing in a typical localisation context. Prague Bulletin of Mathematical Linguistics, 93, 7\u201316.","journal-title":"Prague Bulletin of Mathematical Linguistics"},{"key":"9335_CR35","unstructured":"Roturier, J., Mitchell, L., & Silva, D. (2013). The ACCEPT post-editing environment: A flexible and customisable online tool to perform and analyse machine translation post-editing. In Proceedings of machine translation summit XIV workshop on post-editing technology and practice, September 2013, Nice, France."},{"key":"9335_CR36","unstructured":"San Vicente, I,. & Manterola, I. (2012). PaCo2: A fully automated tool for gathering parallel corpora from the web. In Proceedings of the eight international conference on language resources and evaluation (LREC\u201912), May 23\u201325, 2012, Istanbul, Turkey."}],"container-title":["Language Resources and Evaluation"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10579-016-9335-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-016-9335-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-016-9335-x","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-016-9335-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,5,30]],"date-time":"2019-05-30T14:21:22Z","timestamp":1559226082000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10579-016-9335-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,2,9]]},"references-count":38,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2017,12]]}},"alternative-id":["9335"],"URL":"https:\/\/doi.org\/10.1007\/s10579-016-9335-x","relation":{},"ISSN":["1574-020X","1574-0218"],"issn-type":[{"type":"print","value":"1574-020X"},{"type":"electronic","value":"1574-0218"}],"subject":[],"published":{"date-parts":[[2016,2,9]]}}}