{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,10]],"date-time":"2025-09-10T22:48:30Z","timestamp":1757544510672,"version":"3.40.3"},"publisher-location":"Cham","reference-count":36,"publisher":"Springer Nature Switzerland","isbn-type":[{"type":"print","value":"9783031404979"},{"type":"electronic","value":"9783031404986"}],"license":[{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023]]},"DOI":"10.1007\/978-3-031-40498-6_9","type":"book-chapter","created":{"date-parts":[[2023,8,22]],"date-time":"2023-08-22T23:02:34Z","timestamp":1692745354000},"page":"94-104","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Mono- and Multilingual GPT-3 Models for\u00a0Hungarian"],"prefix":"10.1007","author":[{"given":"Zijian Gy\u0151z\u0151","family":"Yang","sequence":"first","affiliation":[]},{"given":"L\u00e1szl\u00f3 J\u00e1nos","family":"Laki","sequence":"additional","affiliation":[]},{"given":"Tam\u00e1s","family":"V\u00e1radi","sequence":"additional","affiliation":[]},{"given":"G\u00e1bor","family":"Pr\u00f3sz\u00e9ky","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,8,23]]},"reference":[{"key":"9_CR1","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1007\/978-3-319-10816-2_31","volume-title":"Text, Speech and Dialogue","author":"V Benko","year":"2014","unstructured":"Benko, V.: Aranea: yet another family of (comparable) web corpora. In: Sojka, P., Hor\u00e1k, A., Kope\u010dek, I., Pala, K. (eds.) TSD 2014. LNCS, pp. 247\u2013256. Springer International Publishing, Cham (2014). https:\/\/doi.org\/10.1007\/978-3-319-10816-2_31"},{"key":"9_CR2","unstructured":"Benko, V.: Compatible sketch grammars for comparable corpora. In: Abel, A., Vettori, C., Ralli, N. (eds.) Proceedings of the 16th EURALEX International Congress, pp. 417\u2013430. EURAC research, Bolzano, Italy (2014)"},{"key":"9_CR3","doi-asserted-by":"crossref","unstructured":"Black, S., et al.: GPT-NeoX-20B: an open-source autoregressive language model. In: Proceedings of the ACL Workshop on Challenges & Perspectives in Creating Large Language Models (2022). https:\/\/arxiv.org\/abs\/2204.06745","DOI":"10.18653\/v1\/2022.bigscience-1.9"},{"key":"9_CR4","doi-asserted-by":"publisher","unstructured":"Brodersen, K.H., Ong, C.S., Stephan, K.E., Buhmann, J.M.: The balanced accuracy and its posterior distribution. In: 2010 20th International Conference on Pattern Recognition, pp. 3121\u20133124 (2010). https:\/\/doi.org\/10.1109\/ICPR.2010.764","DOI":"10.1109\/ICPR.2010.764"},{"key":"9_CR5","unstructured":"Brown, T., et al.: Language models are few-shot learners. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 1877\u20131901. Curran Associates, Inc. (2020)"},{"key":"9_CR6","unstructured":"Chenghao Fan, Z.L., Tian, J.: Chinese-vicuna: a Chinese instruction-following llama-based model (2023). https:\/\/github.com\/Facico\/Chinese-Vicuna"},{"key":"9_CR7","unstructured":"Chowdhery, A., et al.: PaLM: scaling language modeling with pathways (2022)"},{"key":"9_CR8","unstructured":"Feldmann, \u00c1., et al.: HILBERT, magyar nyelv\u0171 BERT-large modell tan\u00edt\u00e1sa felh\u0151 k\u00f6rnyezetben. In: XVII. Magyar Sz\u00e1m\u00edt\u00f3g\u00e9pes Nyelv\u00e9szeti Konferencia, pp. 29\u201336. Szegedi Tudom\u00e1nyegyetem, Informatikai Int\u00e9zet, Szeged, Magyarorsz\u00e1g (2021)"},{"key":"9_CR9","unstructured":"Gao, L., et al.: The Pile: an 800GB dataset of diverse text for language modeling (2020)"},{"key":"9_CR10","unstructured":"He, J., Qiu, J., Zeng, A., Yang, Z., Zhai, J., Tang, J.: FastMoE: a fast mixture-of-expert training system (2021)"},{"key":"9_CR11","unstructured":"Indig, B.: K\u00f6z\u00f6s crawlnak is egy korpusz a v\u00e9ge - Korpusz\u00e9p\u00edt\u00e9s a CommonCrawl.hu domainj\u00e1b\u00f3l. In: Vincze, V. (ed.) XIV. Magyar Sz\u00e1m\u00edt\u00f3g\u00e9pes Nyelv\u00e9szeti Konferencia (MSZNY 2018), p. 125\u2013134. Szegedi Tudom\u00e1nyegyetem Informatikai Int\u00e9zet, Szegedi Tudom\u00e1nyegyetem Informatikai Tansz\u00e9kcsoport, Szeged (2018)"},{"key":"9_CR12","unstructured":"Jakub\u00ed\u010dek, M., Kilgarriff, A., Kov\u00e1\u0159, V., Rychl\u00fd, P., Suchomel, V.: The TenTen corpus family. In: 7th International Corpus Linguistics Conference CL 2013, pp. 125\u2013127. Lancaster (2013)"},{"key":"9_CR13","doi-asserted-by":"publisher","unstructured":"Kudo, T., Richardson, J.: SentencePiece: a simple and language independent subword tokenizer and detokenizer for neural text processing. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 66\u201371. Association for Computational Linguistics, Brussels, Belgium (2018). https:\/\/doi.org\/10.18653\/v1\/D18-2012, https:\/\/aclanthology.org\/D18-2012","DOI":"10.18653\/v1\/D18-2012"},{"key":"9_CR14","unstructured":"Ligeti-Nagy, N., et al.: HuLU: magyar nyelv\u0171 benchmark adatb\u00e1zis ki\u00e9p\u00edt\u00e9se a neur\u00e1lis nyelvmodellek ki\u00e9rt\u00e9kel\u00e9se c\u00e9lj\u00e1b\u00f3l. In: XVIII. Magyar Sz\u00e1m\u00edt\u00f3g\u00e9pes Nyelv\u00e9szeti Konferencia, pp. 431\u2013446. JATEPress, Szeged (2022)"},{"key":"9_CR15","unstructured":"Lison, P., Tiedemann, J.: OpenSubtitles2016: extracting large parallel corpora from movie and TV subtitles. In: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pp. 923\u2013929. European Language Resources Association (ELRA), Portoro\u017e, Slovenia (2016)"},{"key":"9_CR16","unstructured":"LM-SYS: Fastchat (vicuna: An open-source chatbot) (2023). https:\/\/github.com\/lm-sys\/FastChat"},{"key":"9_CR17","unstructured":"Nemeskey, D.M.: Natural Language Processing Methods for Language Modeling. Ph.D. thesis, E\u00f6tv\u00f6s Lor\u00e1nd University (2020)"},{"key":"9_CR18","unstructured":"Nemeskey, D.M.: Introducing huBERT. In: XVII. Magyar Sz\u00e1m\u00edt\u00f3g\u00e9pes Nyelv\u00e9szeti Konferencia, pp. 3\u201314. Szegedi Tudom\u00e1nyegyetem, Informatikai Int\u00e9zet, Szeged, Magyarorsz\u00e1g (2021)"},{"key":"9_CR19","unstructured":"Nov\u00e1k, A., Sikl\u00f3si, B., Oravecz, C.: A new integrated open-source morphological analyzer for Hungarian. In: Calzolari, N., et al. (eds.) Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). European Language Resources Association (ELRA), Paris, France (2016)"},{"key":"9_CR20","unstructured":"OpenAI: GPT-4 Technical Report (2023)"},{"key":"9_CR21","unstructured":"Oravecz, C., V\u00e1radi, T., Sass, B.: The Hungarian Gigaword corpus. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2014), pp. 1719\u20131723. European Language Resources Association (ELRA), Reykjavik, Iceland (2014)"},{"key":"9_CR22","unstructured":"Ouyang, L., et al.: Training language models to follow instructions with human feedback (2022)"},{"key":"9_CR23","unstructured":"Rychl\u00fd, P.: Manatee\/Bonito - a modular corpus manager. In: 1st Workshop on Recent Advances in Slavonic Natural Language Processing, pp. 65\u201370. Masarykova univerzita, Brno (2007)"},{"key":"9_CR24","unstructured":"Shazeer, N.: GLU Variants Improve Transformer (2020)"},{"key":"9_CR25","unstructured":"Shoeybi, M., Patwary, M., Puri, R., LeGresley, P., Casper, J., Catanzaro, B.: Megatron-LM: training multi-billion parameter language models using model parallelism (2019)"},{"key":"9_CR26","unstructured":"Smith, S., et al.: Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model (2022)"},{"key":"9_CR27","unstructured":"Su, J., Lu, Y., Pan, S., Murtadha, A., Wen, B., Liu, Y.: RoFormer: enhanced transformer with rotary position embedding (2021)"},{"key":"9_CR28","unstructured":"Suchomel, V., Pomik\u00e1lek, J.: Efficient web crawling for large text corpora. In: Kilgarriff, A., Sharoff, S. (eds.) Proceedings of the seventh Web as Corpus Workshop (WAC7), pp. 39\u201343. Lyon (2012)"},{"key":"9_CR29","unstructured":"Taori, R., et al.: Stanford alpaca: an instruction-following llama model (2023). https:\/\/github.com\/tatsu-lab\/stanford_alpaca"},{"key":"9_CR30","unstructured":"Touvron, H., et al.: LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)"},{"key":"9_CR31","unstructured":"Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp. 5998\u20136008. Curran Associates, Inc. (2017)"},{"key":"9_CR32","unstructured":"Wang, B., Komatsuzaki, A.: GPT-J-6B: a 6 billion parameter autoregressive language model (2021). https:\/\/github.com\/kingoflolz\/mesh-transformer-jax"},{"key":"9_CR33","doi-asserted-by":"crossref","unstructured":"Wang, Y., et al.: Self-instruct: aligning language model with self generated instructions (2022)","DOI":"10.18653\/v1\/2023.acl-long.754"},{"key":"9_CR34","doi-asserted-by":"publisher","unstructured":"Xu, B.: NLP Chinese corpus: large scale Chinese corpus for NLP (2019). https:\/\/doi.org\/10.5281\/zenodo.3402023","DOI":"10.5281\/zenodo.3402023"},{"key":"9_CR35","unstructured":"Yang, Z.G., et al.: J\u00f6nnek a nagyok! BERT-large, GPT-2 \u00e9s GPT-3 nyelvmodellek magyar nyelvre. In: XIX. Magyar Sz\u00e1m\u00edt\u00f3g\u00e9pes Nyelv\u00e9szeti Konferencia (MSZNY 2023), pp. 247\u2013262. Szegedi Tudom\u00e1nyegyetem, Informatikai Int\u00e9zet, Szeged, Hungary (2023)"},{"key":"9_CR36","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1016\/j.aiopen.2021.06.001","volume":"2","author":"S Yuan","year":"2021","unstructured":"Yuan, S., et al.: Wudaocorpora: a super large-scale Chinese corpora for pre-training language models. AI Open 2, 65\u201368 (2021). https:\/\/doi.org\/10.1016\/j.aiopen.2021.06.001","journal-title":"AI Open"}],"container-title":["Lecture Notes in Computer Science","Text, Speech, and Dialogue"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-40498-6_9","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,22]],"date-time":"2023-08-22T23:03:59Z","timestamp":1692745439000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-40498-6_9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"ISBN":["9783031404979","9783031404986"],"references-count":36,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-40498-6_9","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"type":"print","value":"0302-9743"},{"type":"electronic","value":"1611-3349"}],"subject":[],"published":{"date-parts":[[2023]]},"assertion":[{"value":"23 August 2023","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"TSD","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Conference on Text, Speech, and Dialogue","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Pilsen","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Czech Republic","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2023","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"4 September 2023","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"6 September 2023","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"26","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"tsd2023","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/www.kiv.zcu.cz\/tsd2023\/index.php","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Double-blind","order":1,"name":"type","label":"Type","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"CMS & back-office system","order":2,"name":"conference_management_system","label":"Conference Management System","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"64","order":3,"name":"number_of_submissions_sent_for_review","label":"Number of Submissions Sent for Review","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"31","order":4,"name":"number_of_full_papers_accepted","label":"Number of Full Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"0","order":5,"name":"number_of_short_papers_accepted","label":"Number of Short Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"48% - The value is computed by the equation \"Number of Full Papers Accepted \/ Number of Submissions Sent for Review * 100\" and then rounded to a whole number.","order":6,"name":"acceptance_rate_of_full_papers","label":"Acceptance Rate of Full Papers","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3","order":7,"name":"average_number_of_reviews_per_paper","label":"Average Number of Reviews per Paper","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"2.56","order":8,"name":"average_number_of_papers_per_reviewer","label":"Average Number of Papers per Reviewer","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"No","order":9,"name":"external_reviewers_involved","label":"External Reviewers Involved","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}}]}}