{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:12:32Z","timestamp":1760058752671,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2025,4,28]],"date-time":"2025-04-28T00:00:00Z","timestamp":1745798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Project of the Research Center for Trusted Artificial Intelligence of the Ivannikov Institute for System Programming of the Russian Academy of Sciences"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>The decision-making process to rule R&amp;D relies on information related to current trends in particular research areas. In this work, we investigated how one can use large language models (LLMs) to transfer the dataset and its annotation from one language to another. This is crucial since sharing knowledge between different languages could boost certain underresourced directions in the target language, saving lots of effort in data annotation or quick prototyping. We experiment with English and Russian pairs, translating the DEFT (Definition Extraction from Texts) corpus. This corpus contains three layers of annotation dedicated to term-definition pair mining, which is a rare annotation type for Russian. The presence of such a dataset is beneficial for the natural language processing methods of trend analysis in science since the terms and definitions are the basic blocks of any scientific field. We provide a pipeline for the annotation transfer using LLMs. In the end, we train the BERT-based models on the translated dataset to establish a baseline.<\/jats:p>","DOI":"10.3390\/bdcc9050116","type":"journal-article","created":{"date-parts":[[2025,4,28]],"date-time":"2025-04-28T06:23:32Z","timestamp":1745821412000},"page":"116","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Transferring Natural Language Datasets Between Languages Using Large Language Models for Modern Decision Support and Sci-Tech Analytical Systems"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-5010-6487","authenticated-orcid":false,"given":"Dmitrii","family":"Popov","sequence":"first","affiliation":[{"name":"Federal Research Center \u201cComputer Science and Control\u201d of the Russian Academy of Sciences (FRC CSC RAS), Moscow 119333, Russia"},{"name":"Faculty of Physics and Mathematics and Natural Sciences, RUDN University, Moscow 117198, Russia"},{"name":"Institute for Information Transmission Problems of the Russian Academy of Sciences (IITP RAS), Moscow 127051, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-6797-9292","authenticated-orcid":false,"given":"Egor","family":"Terentev","sequence":"additional","affiliation":[{"name":"Federal Research Center \u201cComputer Science and Control\u201d of the Russian Academy of Sciences (FRC CSC RAS), Moscow 119333, Russia"},{"name":"Faculty of Physics and Mathematics and Natural Sciences, RUDN University, Moscow 117198, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-6676-7255","authenticated-orcid":false,"given":"Danil","family":"Serenko","sequence":"additional","affiliation":[{"name":"Federal Research Center \u201cComputer Science and Control\u201d of the Russian Academy of Sciences (FRC CSC RAS), Moscow 119333, Russia"},{"name":"Faculty of Physics and Mathematics and Natural Sciences, RUDN University, Moscow 117198, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3113-3765","authenticated-orcid":false,"given":"Ilya","family":"Sochenkov","sequence":"additional","affiliation":[{"name":"Federal Research Center \u201cComputer Science and Control\u201d of the Russian Academy of Sciences (FRC CSC RAS), Moscow 119333, Russia"},{"name":"Institute for Information Transmission Problems of the Russian Academy of Sciences (IITP RAS), Moscow 127051, Russia"},{"name":"Ivannikov Institute for System Programming of the Russian Academy of Sciences (ISP RAS), Moscow 109004, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-6994-151X","authenticated-orcid":false,"given":"Igor","family":"Buyanov","sequence":"additional","affiliation":[{"name":"Federal Research Center \u201cComputer Science and Control\u201d of the Russian Academy of Sciences (FRC CSC RAS), Moscow 119333, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"11898","DOI":"10.1109\/TEM.2023.3306569","article-title":"Identifying and Visualizing Trends in Science, Technology, and Innovation Using SciBERT","volume":"71","author":"Lobanova","year":"2024","journal-title":"IEEE Trans. Eng. Manag."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Lobanova, P.A., Kuzminov, I.F., Karatetskaia, E.Y., Sabidaeva, E.A., and Anpilogov, V.V. (2023). Trend Detection Using NLP as a Mechanism of Decision Support. Sci. Tech. Inf. Process., 88\u201398.","DOI":"10.3103\/S0147688223050106"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"69","DOI":"10.17323\/2500-2597.2018.1.69.78","article-title":"Mapping the Research Landscape of Agricultural Sciences","volume":"12","author":"Devyatkin","year":"2018","journal-title":"Foresight Sti Gov."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Spala, S., Miller, N.A., Yang, Y., Dernoncourt, F., and Dockhorn, C. (2019, January 1). DEFT: A corpus for definition extraction in free- and semi-structured text. Proceedings of the 13th Linguistic Annotation Workshop, Florence, Italy.","DOI":"10.18653\/v1\/W19-4015"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Spala, S., Miller, N.A., Dernoncourt, F., and Dockhorn, C. (2020). SemEval-2020 Task 6: Definition Extraction from Free Text with the DEFT Corpus. arXiv.","DOI":"10.18653\/v1\/2020.semeval-1.41"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhu, W., Liu, H., Dong, Q., Xu, J., Kong, L., Chen, J., Li, L., and Huang, S. (2023). Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis. arXiv.","DOI":"10.18653\/v1\/2024.findings-naacl.176"},{"key":"ref_7","unstructured":"Zhou, W., Zhang, S., Gu, Y., Chen, M., and Poon, H. (2023). UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition. arXiv."},{"key":"ref_8","unstructured":"Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., and Vaughan, A. (2024). The Llama 3 Herd of Models. arXiv."},{"key":"ref_9","unstructured":"Klavans, J.L., and Muresan, S. (2001, January 3\u20137). Evaluation of the DEFINDER system for fully automatic glossary construction. Proceedings of the AMIA Symposium, Washington, DC, USA."},{"key":"ref_10","unstructured":"Navigli, R., and Velardi, P. (2010, January 13). Learning Word-Class Lattices for Definition and Hypernym Extraction. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"145","DOI":"10.5715\/jnlp.6.3_145","article-title":"The C-value\/NC-value domain-independent method for multi-word term extraction","volume":"6","author":"Frantzi","year":"1999","journal-title":"J. Nat. Lang. Process."},{"key":"ref_12","unstructured":"Sun, Y., and Zhuge, H. (2023). Discovering Patterns of Definitions and Methods from Scientific Documents. arXiv."},{"key":"ref_13","unstructured":"Collard, J., de Paiva, V.C.V., and Subrahmanian, E. (2023). Parmesan: Mathematical concept extraction for education. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kang, D., Head, A., Sidhu, R., Lo, K., Weld, D.S., and Hearst, M.A. (2020). Document-Level Definition Detection in Scholarly Documents: Existing Models, Error Analyses, and Future Directions. arXiv.","DOI":"10.18653\/v1\/2020.sdp-1.22"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_16","unstructured":"Vaswani, A., Shazeer, N.M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4\u20139). Attention is All you Need. Proceedings of the Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_17","unstructured":"Navigli, R., Velardi, P., and Ruiz-Mart\u00ednez, J.M. (2010, January 17\u201323). An Annotated Dataset for Extracting Definitions and Hypernyms from the Web. Proceedings of the International Conference on Language Resources and Evaluation, Valletta, Malta."},{"key":"ref_18","unstructured":"Martin-Boyle, A., Head, A., Lo, K., Sidhu, R., Hearst, M.A., and Kang, D. (2023). Complex Mathematical Symbol Definition Structures: A Dataset and Model for Coordination Resolution in Definition Extraction. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Bruches, E.P., Pauls, A., Batura, T., and Isachenko, V. (2020, January 14\u201315). Entity Recognition and Relation Extraction from Scientific and Technical Texts in Russian. Proceedings of the 2020 Science and Artificial Intelligence conference (S.A.I.ence) Novosibirsk, Russia.","DOI":"10.1109\/S.A.I.ence50533.2020.9303196"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"He, X., Lin, Z.W., Gong, Y., Jin, A., Zhang, H., Lin, C., Jiao, J., Yiu, S.M., Duan, N., and Chen, W. (2024, January 16\u201321). AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Mexico City, Mexico.","DOI":"10.18653\/v1\/2024.naacl-industry.15"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1007\/s42001-024-00345-9","article-title":"Open-Source LLMs for Text Annotation: A Practical Guide for Model Setting and Fine-Tuning","volume":"8","author":"Alizadeh","year":"2023","journal-title":"J. Comput. Soc. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Tian, Y., Ravichander, A., Qin, L., Bras, R.L., Marjieh, R., Peng, N., Choi, Y., Griffiths, T.L., and Brahman, F. MacGyver: Are Large Language Models Creative Problem Solvers? In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Mexico City, Mexico, 16\u201321 June 2024.","DOI":"10.18653\/v1\/2024.naacl-long.297"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"14350","DOI":"10.1109\/ACCESS.2024.3358206","article-title":"Socially Aware Synthetic Data Generation for Suicidal Ideation Detection Using Large Language Models","volume":"12","author":"Ghanadian","year":"2024","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Frei, J., and Kramer, F. (2023). Annotated dataset creation through large language models for non-english medical NLP. J. Biomed. Inform., 145.","DOI":"10.1016\/j.jbi.2023.104478"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Lhoest, Q., del Moral, A.V., Jernite, Y., Thakur, A., von Platen, P., Patil, S., Chaumond, J., Drame, M., Plu, J., and Tunstall, L. (2021). Datasets: A Community Library for Natural Language Processing. arXiv.","DOI":"10.18653\/v1\/2021.emnlp-demo.21"},{"key":"ref_26","unstructured":"Tkachenko, M., Malyuk, M., Holmanyuk, A., and Liubimov, N. (2025, April 01). Label Studio: Data Labeling Software, 2020\u20132024. Open Source Software. Available online: https:\/\/github.com\/HumanSignal\/label-studio."},{"key":"ref_27","unstructured":"Wei, J., Wang, X., Schuurmans, D., Bosma, M., hsin Chi, E.H., Xia, F., Le, Q., and Zhou, D. (2022). Chain of Thought Prompting Elicits Reasoning in Large Language Models. arXiv."},{"key":"ref_28","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 6\u201312). Bleu: A Method for Automatic Evaluation of Machine Translation. Proceedings of the Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Feng, F., Yang, Y., Cer, D.M., Arivazhagan, N., and Wang, W. (2022, January 22\u201327). Language-agnostic BERT Sentence Embedding. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Dublin, Ireland.","DOI":"10.18653\/v1\/2022.acl-long.62"},{"key":"ref_30","unstructured":"Dao, X.Q., and Le, N.B. (2023). Investigating the Effectiveness of ChatGPT in Mathematical Reasoning and Problem Solving: Evidence from the Vietnamese National High School Graduation Examination. arXiv."},{"key":"ref_31","unstructured":"Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv."},{"key":"ref_32","unstructured":"Kuratov, Y., and Arkhipov, M. (2019). Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language. arXiv."},{"key":"ref_33","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv."}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/5\/116\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:23:16Z","timestamp":1760030596000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/9\/5\/116"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,28]]},"references-count":33,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5]]}},"alternative-id":["bdcc9050116"],"URL":"https:\/\/doi.org\/10.3390\/bdcc9050116","relation":{},"ISSN":["2504-2289"],"issn-type":[{"type":"electronic","value":"2504-2289"}],"subject":[],"published":{"date-parts":[[2025,4,28]]}}}