{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T05:18:06Z","timestamp":1778217486299,"version":"3.51.4"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2013,4,21]],"date-time":"2013-04-21T00:00:00Z","timestamp":1366502400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Lang Resources & Evaluation"],"published-print":{"date-parts":[[2014,3]]},"DOI":"10.1007\/s10579-013-9226-3","type":"journal-article","created":{"date-parts":[[2013,4,20]],"date-time":"2013-04-20T05:44:17Z","timestamp":1366436657000},"page":"65-92","source":"Crossref","is-referenced-by-count":21,"title":["Evaluating and automating the annotation of a learner corpus"],"prefix":"10.1007","volume":"48","author":[{"given":"Alexandr","family":"Rosen","sequence":"first","affiliation":[]},{"given":"Jirka","family":"Hana","sequence":"additional","affiliation":[]},{"given":"Barbora","family":"\u0160tindlov\u00e1","sequence":"additional","affiliation":[]},{"given":"Anna","family":"Feldman","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2013,4,21]]},"reference":[{"key":"9226_CR1","first-page":"161","volume":"7","author":"G. Abuhakema","year":"2009","unstructured":"Abuhakema, G., Feldman, A., & Fitzpatrick, E. (2009). ARIDA: An Arabic interlanguage database and its applications: A pilot study. Journal of the National Council of Less Commonly Taught Languages (NCOLCTL) 7, 161\u2013184.","journal-title":"Journal of the National Council of Less Commonly Taught Languages (NCOLCTL)"},{"key":"9226_CR2","unstructured":"Bed\u0159ichov\u00e1, Z., \u0160ebesta, K., \u0160kodov\u00e1, S., & \u0160ormov\u00e1, K. (2011). Podoba a vyu\u017eit\u00ed korpusu jinojazy\u010dn\u00fdch a romsk\u00fdch mluv\u010d\u00edch \u010de\u0161tiny: CZESL a ROMi [Form and utilization of a corpus of non-native and Romany speakers of Czech: CZESL and ROMi]. In F. \u010cerm\u00e1k (Ed.), Korpusov\u00e1 lingvistika Praha 2011: 2 - V\u00fdzkum a v\u00fdstavba korpus $$\\mathring{\\rm u}$$ , \u00dastav \u010cesk\u00e9ho n\u00e1rodn\u00edho korpusu, Nakladatelstv\u00ed Lidov\u00e9 noviny, Praha, Studie z korpusov\u00e9 lingvistiky, vol 15 (pp. 93\u2013104)."},{"key":"9226_CR3","unstructured":"Brants, T. (2000). TnT\u2014A statistical part-of-speech tagger. In Proceedings of the sixth applied natural language processing (ANLP-2000). WA: Seattle."},{"issue":"1","key":"9226_CR5","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J. Cohen","year":"1960","unstructured":"Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37\u201346.","journal-title":"Educational and Psychological Measurement"},{"key":"9226_CR4","unstructured":"de Cock, S. (2003). Recurrent sequences of words in native speaker and advanced learner spoken and written english. PhD thesis, Universit\u00e9 catholique de Louvain, Louvain-la-Neuve."},{"key":"9226_CR16","doi-asserted-by":"crossref","unstructured":"de Haan, P. (2000). Tagging non-native English with the TOSCA-ICLE tagger. In C. Mair & M. Hundt (Eds.), Corpus linguistics and linguistic theory. Papers from the twentieth international conference on English language research on computerized corpora (ICAME 20), (pp. 69\u201380). Freiburg im Breisgau 1999, Rodopi, Amsterdam.","DOI":"10.1163\/9789004490758_007"},{"key":"9226_CR28","unstructured":"de M\u00f6nnink, I. (2000). Parsing a learner corpus?. In C. Mair, M. Hundt (Eds.), Corpus linguistics and linguistic theory. Papers from the twentieth international conference on English language research on computerized corpora (ICAME 20), (pp. 81\u201390). Amsterdam: Freiburg im Breisgau 1999, Rodopi."},{"key":"9226_CR6","unstructured":"Dickinson, M. (2010). Generating learner-like morphological errors in Russian. In Proceedings of the 23nd international conference on computational linguistics (COLING-10). Beijing. http:\/\/jones.ling.indiana.edu\/~mdickinson\/papers\/dickinson-coling10.html ."},{"key":"9226_CR7","first-page":"83","volume":"19","author":"A. D\u00edaz-Negrillo","year":"2006","unstructured":"D\u00edaz-Negrillo, A., & Fern\u00e1ndez-Dom\u00ednguez, J. (2006). Error tagging systems for learner corpora. Resla, 19, 83\u2013102.","journal-title":"Resla"},{"key":"9226_CR8","unstructured":"D\u00edaz-Negrillo, A., Meurers, D., Valera, S., & Wunsch, H. (2010). Towards interlanguage POS annotation for effective learner corpora in SLA and FLT. Language Forum, 36(1\u20132), 139\u2013154. http:\/\/purl.org\/dm\/papers\/diaz-negrillo-et-al-09.html , special Issue on Corpus Linguistics for Teaching and Learning. In Honour of John Sinclair."},{"key":"9226_CR9","unstructured":"Fitzpatrick, E., & Seegmiller, S. (2001). The montclair electronic language learner database. In: Proceedings of the international conference on computing and information technologies (ICCIT)."},{"key":"9226_CR10","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1163\/9789004333772_013","volume-title":"Applied Corpus Linguistics: A Multidimensional Perspective.","author":"E. Fitzpatrick","year":"2004","unstructured":"Fitzpatrick, E., & Seegmiller, S. (2004). The Montclair electronic language database project. In U. Connor & T. A. Upton (Eds.), Applied corpus linguistics: A multidimensional perspective (pp. 223\u2013238). Amsterdam: Rodopi."},{"key":"9226_CR11","unstructured":"Flor, M., & Futagi, Y. (2011). Automatic correction of non-word misspellings and generation of learner language corpora. In Learner corpus research 2011\u201320\u00a0years of learner corpus research: Looking back, moving ahead, Centre for English Corpus Linguistics. Universit\u00e9 catholique de Louvain, Louvain-la-Neuve."},{"key":"9226_CR12","unstructured":"Granger, S. (1999). Use of tenses by advanced EFL learners: Evidence from error-tagged computer corpus. In H. Hasselg\u00e5rd, S. Oksefjell (Eds.), Out of corpora \u2014Studies in Honour of Stig Johansson. Amsterdam: Atlanta. http:\/\/hdl.handle.net\/2078.1\/76322 ."},{"issue":"3","key":"9226_CR13","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1558\/cj.v20i3.465-480","volume":"20","author":"S. Granger","year":"2003","unstructured":"Granger, S. (2003a). Error-tagged learner corpora and call: A promising synergy. CALICO Journal, 20(3), 465\u2013480.","journal-title":"CALICO Journal"},{"key":"9226_CR14","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1558\/cj.v20i3.465-480","volume":"20","author":"S. Granger","year":"2003","unstructured":"Granger, S. (2003b) Error-tagged learner corpora and CALL: A promising synergy. CALICO Journal, 20, 465\u2013480.","journal-title":"CALICO journal"},{"key":"9226_CR15","unstructured":"Granger, S. (2008). Learner corpora. In A. L\u00fcdeling & M. Kyt\u00f6 (Eds.), Corpus linguistics. An International Handbook, HSK 29. 1., vol. 1 (pp. 259\u2013274). Berlin: Mouton De Gruyter."},{"key":"9226_CR17","volume-title":"Disambiguation of Rich Inflection (Computational Morphology of Czech)","author":"J. Haji\u010d","year":"2004","unstructured":"Haji\u010d, J. (2004). Disambiguation of rich inflection (computational morphology of Czech). Prague: Charles University Press."},{"key":"9226_CR18","unstructured":"Hana, J., Rosen, A., \u0160kodov\u00e1, S., & \u0160tindlov\u00e1, B. (2010). Error-tagged learner corpus of Czech. In Proceedings of the fourth linguistic annotation workshop. Uppsala, Sweden: Association for Computational Linguistics. http:\/\/utkl.ff.cuni.cz\/~rosen\/public\/hanaetal_law2010.pdf ."},{"key":"9226_CR19","unstructured":"Hana, J., Rosen, A., \u0160tindlov\u00e1, B., & J\u00e4ger, P. (2012). Building a learner corpus. In N. Calzolari, K. Choukri, T. Declerck, M. U. Do\u011fan, B. Maegaard, J. Mariani, J. Odijk & S. Piperidis (Eds.), Proceedings of the eight international conference on language resources and evaluation (LREC\u201912). Istanbul, Turkey: European Language Resources Association (ELRA)."},{"key":"9226_CR20","first-page":"13","volume":"91","author":"T. Jel\u00ednek","year":"2008","unstructured":"Jel\u00ednek, T. (2008). Nov\u00e9 zna\u010dkov\u00e1n\u00ed v \u010cesk\u00e9m n\u00e1rodn\u00edm korpusu [A new tagging system in the Czech National Corpus]. Na\u0161e \u0159e\u010d 91, 13\u201320.","journal-title":"Na\u0161e \u0159e\u010d"},{"key":"9226_CR21","unstructured":"Jel\u00ednek, T., & Petkevi\u010d, V. (2011). Syst\u00e9m jazykov\u00e9ho zna\u010dkov\u00e1n\u00ed korpus $$\\mathring{\\rm u}$$ sou\u010dasn\u00e9 psan\u00e9 \u010de\u0161tiny [A system of linguistic markup of corpora of contemporary written Czech]. In V. Petkevi\u010d & A. Rosen (Eds.), Korpusov\u00e1 lingvistika Praha 2011: 3 \u2013 Gramatika a zna\u010dkov\u00e1n\u00ed korpus $$\\mathring{\\rm u}$$ , \u00dastav \u010cesk\u00e9ho n\u00e1rodn\u00edho korpusu, Nakladatelstv\u00ed Lidov\u00e9 noviny, vol. 16, (pp. 154\u2013170). Praha: Studie z korpusov\u00e9 lingvistiky."},{"key":"9226_CR22","doi-asserted-by":"crossref","unstructured":"Jel\u00ednek, T., \u0160tindlov\u00e1, B., Rosen, A., & Hana, J. (2012). Combining manual and automatic annotation of a learner corpus. In P. Sojka, A. Hor\u00e1k, I. Kope\u010dek & K. Pala (Eds.), Text, speech and dialogue\u2014Proceedings of the 15th international conference TSD 2012, no. 7499 in Lecture Notes in Computer Science, (pp. 127\u2013134). Springer.","DOI":"10.1007\/978-3-642-32790-2_15"},{"key":"9226_CR23","unstructured":"Kisselev, O. (2013). Russian learner corpus of academic writing: Design, development and applications: The American Association for Corpus Linguistics (AACL 2013), January 18\u201320, 2013, San Diego State University, San Diego, US."},{"key":"9226_CR24","first-page":"xiv","volume-title":"Learner English on Computer","author":"G. Leech","year":"1998","unstructured":"Leech, G. (1998). Preface. In S. Granger (Ed.), Learner English on computer (pp. xiv\u2013xx). London: Addison Wesley Longman."},{"key":"9226_CR25","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1075\/scl.17.07len","volume-title":"Corpora and Language Learners","author":"A. Le\u0144ko-Szyma\u0144ska","year":"2004","unstructured":"Le\u0144ko-Szyma\u0144ska, A. (2004). Demonstratives as anaphora markers in advanced learners\u2019 English. In G. Aston SBDS (Ed.), Corpora and language learners (pp. 89\u2013107). Amsterdam: John Benjamins."},{"key":"9226_CR26","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1515\/9783484970342.2.119","volume-title":"Fortgeschrittene Lernervariet\u00e4ten","author":"A. L\u00fcdeling","year":"2008","unstructured":"L\u00fcdeling, A. (2008). Mehrdeutigkeiten und Kategorisierung: Probleme bei der Annotation von Lernerkorpora. In P. Grommes, M. Walter (Eds.) Fortgeschrittene Lernervariet\u00e4ten (pp. 119\u2013140). T\u00fcbingen: Niemeyer."},{"key":"9226_CR27","doi-asserted-by":"crossref","unstructured":"Meurers, D. (2009). On the automatic analysis of learner language: Introduction to the special issue. CALICO Journal 26(3), 469\u2013473. http:\/\/purl.org\/dm\/papers\/meurers-09.html .","DOI":"10.1558\/cj.v26i3.469-473"},{"key":"9226_CR29","doi-asserted-by":"crossref","DOI":"10.1075\/scl.14","volume-title":"Collocations in a Learner Corpus","author":"N. Nesselhauf","year":"2005","unstructured":"Nesselhauf, N. (2005). Collocations in a learner corpus. Amsterdam: John Benjamins."},{"key":"9226_CR30","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1111\/j.1540-4781.2007.00541.x","volume":"91","author":"A. Pavlenko","year":"2007","unstructured":"Pavlenko, A., & Hasko, V. (2007). Russian emotion vocabulary in American learners\u2019 narratives. The Modern Language Journal 91, 213\u2013234.","journal-title":"The Modern Language Journal"},{"key":"9226_CR31","first-page":"81","volume":"26","author":"N.A. Pravec","year":"2002","unstructured":"Pravecm, N. A. (2002). Survey of learner corpora. ICAME Journal 26, 81\u2013114.","journal-title":"ICAME Journal"},{"key":"9226_CR32","unstructured":"Richter, M. (2010). Pokro\u010dil\u00fd korektor \u010de\u0161tiny [An advanced spell checker of Czech]. Master\u2019s thesis, Faculty of Mathematics and Physics, Charles University, Prague."},{"key":"9226_CR33","first-page":"41","volume-title":"Learner English on Computer","author":"H. Ringbom","year":"1998","unstructured":"Ringbom, H. (1998). Vocabulary frequencies in advanced learner English: A cross-linguistic approach. In S. Granger (Ed.), Learner English on computer (pp. 41\u201352). Harlow: Longman."},{"key":"9226_CR34","unstructured":"Rozovskaya, A., & Roth, D. (2010). Annotating ESL errors: Challenges and rewards. In Proceedings of NAACL\u201910 workshop on innovative use of NLP for building educational applications. University of Illinois at Urbana-Champ. http:\/\/cogcomp.cs.illinois.edu\/page\/publication_view\/212 ."},{"key":"9226_CR35","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1515\/iral.1972.10.1-4.209","volume":"10","author":"L. Selinker","year":"1972","unstructured":"Selinker, L. (1972). Interlanguage. IRAL 10, 209\u2013231.","journal-title":"IRAL"},{"key":"9226_CR36","unstructured":"Spoustov\u00e1, D., Haji\u010d, J., Votrubec, J., Krbec, P., & Kv\u011bto\u0148, P. (2007). The best of two worlds: Cooperation of statistical and rule-based taggers for Czech. In Proceedings of the workshop on Balto-Slavonic natural language processing 2007 (pp. 67\u201374). Praha, Czechia: Association for Computational Linguistics."},{"key":"9226_CR37","first-page":"135","volume":"7","author":"M. Stritar","year":"2009","unstructured":"Stritar, M. (2009). Slovene as a foreign language: The pilot learner corpus perspective. Slovenski jezik \u2013 Slovene Linguistic Studies 7, 135\u2013152.","journal-title":"Slovenski jezik \u2013 Slovene Linguistic Studies"},{"key":"9226_CR38","first-page":"11","volume":"1","author":"K. \u0160ebesta","year":"2010","unstructured":"\u0160ebesta, K. (2010). Korpusy \u010de\u0161tiny a osvojov\u00e1n\u00ed jazyka [Corpora of Czech and language acquistion]. Studie z aplikovan\u00e9 lingvistiky\/Studies in Applied Linguistics 1, 11\u201334.","journal-title":"Studie z aplikovan\u00e9 lingvistiky\/Studies in Applied Linguistics"},{"key":"9226_CR39","unstructured":"\u0160tindlov\u00e1, B. (2011). Evaluace chybov\u00e9 anotace v \u017e\u00e1kovsk\u00e9m korpusu \u010de\u0161tiny [Evaluation of error mark-up in a learner corpus of Czech]. PhD thesis, Charles University, Faculty of Arts, Prague."},{"key":"9226_CR40","unstructured":"\u0160tindlov\u00e1, B., \u0160kodov\u00e1, S., Hana, J., & Rosen, A. (2012a). CzeSL\u2014An error tagged corpus of Czech as a second language. In P. P\u0119zik (Ed.), PALC 2011\u2014Practical applications in language and computers, L\u00f3d\u017c 13\u201315 April 2011. Peter Lang, \u0141\u00f3d\u017a Studies in Language."},{"key":"9226_CR41","unstructured":"\u0160tindlov\u00e1, B., \u0160kodov\u00e1, S., Hana, J., & Rosen, A. (2012b). A learner corpus of Czech: Current state and future directions. In S. Granger, G. Gilquin & F. Meunier (Eds.), Twenty years of learner corpus research: Looking back, moving ahead. Corpora and language in use\u2014Proceedings 1. Louvain-la-Neuve: Presses Universitaires de Louvain (in print)."},{"key":"9226_CR42","unstructured":"\u0160tindlov\u00e1, B., \u0160kodov\u00e1, S., Rosen, A., & Hana, J. (2012c). Annotating foreign learners\u2019 Czech. In M. Zikov\u00e1 & M. Do\u010dekal (Eds.), Slavic languages in formal grammar. Proceedings of FDSL 8.5, Brno 2010 (pp. 205\u2013219). Frankfurt am Main: Peter Lang."},{"key":"9226_CR43","doi-asserted-by":"crossref","unstructured":"Tetreault, J., & Chodorow, M. (2008). Native judgements of non-native usage: Experiments in preposition error detection. In COLING workshop on human judgements in computational linguistics. Manchester.","DOI":"10.3115\/1611628.1611633"},{"key":"9226_CR44","unstructured":"Van Rooy, B., & Sch\u00e4fer, L. (2003). An evaluation of three POS taggers for the tagging of the Tswana Learner English Corpus. In D. Archer, P. Rayson, A. Wilson & T. McEnery (Eds.), Proceedings of the corpus linguistics 2003 conference (pp. 835\u2013844). Lancaster: UCREL, Lancaster University."},{"key":"9226_CR45","unstructured":"Votrubec, J. (2006). Morphological tagging based on averaged perceptron. In WDS\u201906 proceedings of contributed papers (pp. 191\u2013195). Praha, Czechia: Matfyzpress, Charles University."},{"key":"9226_CR46","volume-title":"Phrasal verbs. German and Italian learners of English compared","author":"B. Waibel","year":"2008","unstructured":"Waibel, B. (2008). Phrasal verbs. German and Italian learners of English compared. Saarbr\u00fccken: VDM."},{"key":"9226_CR47","unstructured":"Xiao, R. (2008). Well-known and influential corpora. In A. L\u00fcdeling & M. Kyt\u00f6 (Eds.), Corpus linguistics. An international handbook, handbooks of linguistics and communication science [HSK] 29.1, vol. 1 (pp. 383\u2013457). Berlin: Mouton de Gruyter."}],"container-title":["Language Resources and Evaluation"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-013-9226-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10579-013-9226-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-013-9226-3","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,9]],"date-time":"2024-05-09T08:55:37Z","timestamp":1715244937000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10579-013-9226-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,4,21]]},"references-count":47,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2014,3]]}},"alternative-id":["9226"],"URL":"https:\/\/doi.org\/10.1007\/s10579-013-9226-3","relation":{},"ISSN":["1574-020X","1574-0218"],"issn-type":[{"value":"1574-020X","type":"print"},{"value":"1574-0218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,4,21]]}}}