{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T10:18:37Z","timestamp":1753438717195},"reference-count":0,"publisher":"IOS Press","license":[{"start":{"date-parts":[[2020,9,15]],"date-time":"2020-09-15T00:00:00Z","timestamp":1600128000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,9,15]]},"abstract":"<jats:p>Recently, large pre-trained language models, such as BERT, have reached state-of-the-art performance in many natural language processing tasks, but for many languages, including Estonian, BERT models are not yet available. However, there exist several multilingual BERT models that can handle multiple languages simultaneously and that have been trained also on Estonian data. In this paper, we evaluate four multilingual models\u2014multilingual BERT, multilingual distilled BERT, XLM and XLM-RoBERTa\u2014on several NLP tasks including POS and morphological tagging, NER and text classification. Our aim is to establish a comparison between these multilingual BERT models and the existing baseline neural models for these tasks. Our results show that multilingual BERT models can generalise well on different Estonian NLP tasks outperforming all baselines models for POS and morphological tagging and text classification, and reaching the comparable level with the best baseline for NER, with XLM-RoBERTa achieving the highest results compared with other multilingual models.<\/jats:p>","DOI":"10.3233\/faia200597","type":"book-chapter","created":{"date-parts":[[2020,9,22]],"date-time":"2020-09-22T11:54:15Z","timestamp":1600775655000},"source":"Crossref","is-referenced-by-count":1,"title":["Evaluating Multilingual BERT for Estonian"],"prefix":"10.3233","author":[{"given":"Claudia","family":"Kittask","sequence":"first","affiliation":[{"name":"Institute of Computer Science, University of Tartu, Estonia"}]},{"given":"Kirill","family":"Milintsevich","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, University of Tartu, Estonia"}]},{"given":"Kairit","family":"Sirts","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, University of Tartu, Estonia"}]}],"member":"7437","container-title":["Frontiers in Artificial Intelligence and Applications","Human Language Technologies \u2013 The Baltic Perspective"],"original-title":[],"link":[{"URL":"http:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/FAIA200597","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,9,22]],"date-time":"2020-09-22T11:54:16Z","timestamp":1600775656000},"score":1,"resource":{"primary":{"URL":"http:\/\/ebooks.iospress.nl\/doi\/10.3233\/FAIA200597"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,15]]},"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/faia200597","relation":{},"ISSN":["0922-6389","1879-8314"],"issn-type":[{"value":"0922-6389","type":"print"},{"value":"1879-8314","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,15]]}}}