{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T07:25:21Z","timestamp":1740122721054,"version":"3.37.3"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"17-18","license":[{"start":{"date-parts":[[2024,7,2]],"date-time":"2024-07-02T00:00:00Z","timestamp":1719878400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,7,2]],"date-time":"2024-07-02T00:00:00Z","timestamp":1719878400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Institute for Information and Communications Technology Planning & Evaluation"},{"DOI":"10.13039\/501100014188","name":"Ministry of Science and ICT","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100014188","id-type":"DOI","asserted-by":"crossref"}]},{"name":"ICT Creative Consilience program"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Intell"],"published-print":{"date-parts":[[2024,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Document-level relation extraction (RE) aims to predict the relational facts between two given entities from a document. Unlike widespread research on document-level RE in English, Korean document-level RE research is still at the very beginning due to the absence of a dataset. To accelerate the studies, we present    (<jats:bold>T<\/jats:bold>oward Document-Level <jats:bold>R<\/jats:bold>elation <jats:bold>E<\/jats:bold>xtraction in <jats:bold>K<\/jats:bold>orean) dataset constructed from Korean encyclopedia documents written by the domain experts. We provide detailed statistical analyses for our large-scale dataset and human evaluation results suggest the assured quality of   . Also, we introduce the document-level RE model that considers the named entity-type while considering the Korean language\u2019s properties. In the experiments, we demonstrate that our proposed model outperforms the baselines and conduct qualitative analysis.<\/jats:p>","DOI":"10.1007\/s10489-024-05605-9","type":"journal-article","created":{"date-parts":[[2024,7,2]],"date-time":"2024-07-02T11:02:04Z","timestamp":1719918124000},"page":"8681-8701","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A large-scale dataset for korean document-level relation extraction from encyclopedia texts"],"prefix":"10.1007","volume":"54","author":[{"given":"Suhyune","family":"Son","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jungwoo","family":"Lim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seonmin","family":"Koo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinsung","family":"Kim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Younghoon","family":"Kim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Youngsik","family":"Lim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dongseok","family":"Hyun","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9269-1157","authenticated-orcid":false,"given":"Heuiseok","family":"Lim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,7,2]]},"reference":[{"key":"5605_CR1","doi-asserted-by":"crossref","unstructured":"Hendrickx I, Kim SN, Kozareva Z, Nakov P, S\u00e9aghdha D\u00d3, Pad\u00f3 S, Pennacchiotti M, Romano L, Szpakowicz S (2010) Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the 5th International Workshop on Semantic Evaluation, pp. 33\u201338","DOI":"10.3115\/1621969.1621986"},{"key":"5605_CR2","unstructured":"Shen Y, Huang XJ (2016) Attention-based convolutional neural network for semantic relation extraction. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 2526\u20132536"},{"key":"5605_CR3","doi-asserted-by":"crossref","unstructured":"Zhang Y, Zhong V, Chen D, Angeli G, Manning CD (2017) Position-aware attention and supervised data improve slot filling. In: Conference on Empirical Methods in Natural Language Processing","DOI":"10.18653\/v1\/D17-1004"},{"key":"5605_CR4","unstructured":"Zeng D, Liu K, Lai S, Zhou G, Zhao J (2014) Relation classification via convolutional deep neural network. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 2335\u20132344. Dublin City University and Association for Computational Linguistics, Dublin, Ireland. https:\/\/aclanthology.org\/C14-1220"},{"key":"5605_CR5","unstructured":"Soares LB, FitzGerald N, Ling J, Kwiatkowski T (2019) Matching the blanks: Distributional similarity for relation learning. arXiv:1906.03158"},{"key":"5605_CR6","doi-asserted-by":"crossref","unstructured":"Ye D, Lin Y, Li P, Sun M (2022) Packed levitated marker for entity and relation extraction. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4904\u20134917","DOI":"10.18653\/v1\/2022.acl-long.337"},{"key":"5605_CR7","doi-asserted-by":"crossref","unstructured":"Ru D, Sun C, Feng J, Qiu L, Zhou H, Zhang W, Yu Y, Li L (2021) Learning logic rules for document-level relation extraction. arXiv:2111.05407","DOI":"10.18653\/v1\/2021.emnlp-main.95"},{"key":"5605_CR8","doi-asserted-by":"crossref","unstructured":"Yu J, Yang D, Tian S (2022) Relation-specific attentions over entity mentions for enhanced document-level relation extraction. arXiv:2205.14393","DOI":"10.18653\/v1\/2022.naacl-main.109"},{"key":"5605_CR9","doi-asserted-by":"publisher","unstructured":"Yao Y, Ye D, Li P, Han X, Lin Y, Liu Z, Liu Z, Huang L, Zhou J, Sun M (2019) Doc RED: A large-scale document-level relation extraction dataset. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 764\u2013777. Association for Computational Linguistics, Florence, Italy. https:\/\/doi.org\/10.18653\/v1\/P19-1074, https:\/\/aclanthology.org\/P19-1074","DOI":"10.18653\/v1\/P19-1074"},{"key":"5605_CR10","doi-asserted-by":"publisher","first-page":"14612","DOI":"10.1609\/aaai.v35i16.17717","volume":"35","author":"W Zhou","year":"2021","unstructured":"Zhou W, Huang K, Ma T, Huang J (2021) Document-level relation extraction with adaptive thresholding and localized context pooling. Proceedings of the AAAI Conference on Artificial Intelligence 35:14612\u201314620","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"5605_CR11","doi-asserted-by":"publisher","first-page":"14149","DOI":"10.1609\/aaai.v35i16.17665","volume":"35","author":"B Xu","year":"2021","unstructured":"Xu B, Wang Q, Lyu Y, Zhu Y, Mao Z (2021) Entity structure within and throughout: Modeling mention dependencies for document-level relation extraction. Proceedings of the AAAI Conference on Artificial Intelligence 35:14149\u201314157","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"5605_CR12","doi-asserted-by":"crossref","unstructured":"Giorgi J, Bader GD, Wang B (2022) A sequence-to-sequence approach for document-level relation extraction. arXiv:2204.01098","DOI":"10.18653\/v1\/2022.bionlp-1.2"},{"key":"5605_CR13","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2023.110428","volume":"267","author":"Q Sun","year":"2023","unstructured":"Sun Q, Zhang K, Huang K, Xu T, Li X, Liu Y (2023) Document-level relation extraction with two-stage dynamic graph attention networks. Knowl-Based Syst 267:110428","journal-title":"Knowl-Based Syst"},{"key":"5605_CR14","unstructured":"Nam S, Lee M, Kim D, Han K, Kim K, Yoon S, Kim Ek, Choi KS (2020) Effective crowdsourcing of multiple tasks for comprehensive knowledge extraction. In: Proceedings of the 12th Language Resources and Evaluation Conference, pp. 212\u2013219"},{"key":"5605_CR15","doi-asserted-by":"crossref","unstructured":"Jung J, Jung S, Roh Yh (2022) Sequential alignment methods for ensemble part-of-speech tagging. In: 2022 IEEE International Conference on Big Data and Smart Computing (BigComp), pp. 175\u2013181. IEEE","DOI":"10.1109\/BigComp54360.2022.00042"},{"key":"5605_CR16","volume-title":"A Syntactic and Semantic Study of Korean Auxiliaries: A Grammaticalization Perspective","author":"Oh Ss","year":"1998","unstructured":"Ss Oh (1998) A Syntactic and Semantic Study of Korean Auxiliaries: A Grammaticalization Perspective. University of Hawai\u2019i at Manoa, Honolulu, HI"},{"key":"5605_CR17","doi-asserted-by":"crossref","unstructured":"Lee S, Jang TY, Seo J (2002) The grammatical function analysis between korean adnoun clause and noun phrase by using support vector machines. In: COLING 2002: The 19th International Conference on Computational Linguistics","DOI":"10.3115\/1072228.1072247"},{"key":"5605_CR18","doi-asserted-by":"publisher","unstructured":"Yu J, Yang D, Tian S (2022) Relation-specific attentions over entity mentions for enhanced document-level relation extraction. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1523\u20131529. Association for Computational Linguistics, Seattle, United States. https:\/\/doi.org\/10.18653\/v1\/2022.naacl-main.109, https:\/\/aclanthology.org\/2022.naacl-main.109","DOI":"10.18653\/v1\/2022.naacl-main.109"},{"key":"5605_CR19","doi-asserted-by":"publisher","first-page":"14167","DOI":"10.1609\/aaai.v35i16.17667","volume":"35","author":"W Xu","year":"2021","unstructured":"Xu W, Chen K, Zhao T (2021) Document-level relation extraction with reconstruction. Proceedings of the AAAI Conference on Artificial Intelligence 35:14167\u201314175","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"5605_CR20","unstructured":"Jiang F, Niu J, Mo S, Fan S (2022) Key mention pairs guided document-level relation extraction. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 1904\u20131914. International Committee on Computational Linguistics, Gyeongju, Republic of Korea. https:\/\/aclanthology.org\/2022.coling-1.165"},{"key":"5605_CR21","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1016\/j.neunet.2023.11.062","volume":"171","author":"R Han","year":"2024","unstructured":"Han R, Peng T, Wang B, Liu L, Tiwari P, Wan X (2024) Document-level relation extraction with relation correlations. Neural Netw 171:14\u201324","journal-title":"Neural Netw"},{"key":"5605_CR22","unstructured":"Huang X, Yang H, Chen Y, Zhao J, Liu K, Sun W, Zhao Z (2022) Document-level relation extraction via pair-aware and entity-enhanced representation learning. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 2418\u20132428. International Committee on Computational Linguistics, Gyeongju, Republic of Korea. https:\/\/aclanthology.org\/2022.coling-1.213"},{"key":"5605_CR23","doi-asserted-by":"crossref","unstructured":"Tan Q, He R, Bing L, Ng HT (2022) Document-level relation extraction with adaptive focal loss and knowledge distillation. In: Findings of ACL. https:\/\/aclanthology.org\/2022.findings-acl.132","DOI":"10.18653\/v1\/2022.findings-acl.132"},{"key":"5605_CR24","doi-asserted-by":"publisher","unstructured":"Xiao Y, Zhang Z, Mao Y, Yang C, Han J (2022) SAIS: Supervising and augmenting intermediate steps for document-level relation extraction. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2395\u20132409. Association for Computational Linguistics, Seattle, United States . https:\/\/doi.org\/10.18653\/v1\/2022.naacl-main.171, https:\/\/aclanthology.org\/2022.naacl-main.171","DOI":"10.18653\/v1\/2022.naacl-main.171"},{"issue":"23","key":"5605_CR25","doi-asserted-by":"publisher","first-page":"11971","DOI":"10.3390\/app122311971","volume":"12","author":"M Jeong","year":"2022","unstructured":"Jeong M, Suh H, Lee H, Lee JH (2022) A named entity and relationship extraction method from trouble-shooting documents in korean. Appl Sci 12(23):11971","journal-title":"Appl Sci"},{"key":"5605_CR26","unstructured":"Kwak S, Kim B, Lee JS (2013) Triplet extraction using korean dependency parsing result. In: Annual Conference on Human and Language Technology, pp. 86\u201389. Human and Language Technology"},{"key":"5605_CR27","unstructured":"Kim B, Lee JS (2016) Extracting spatial entities and relations in Korean text. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 2389\u20132396. The COLING 2016 Organizing Committee, Osaka, Japan. https:\/\/aclanthology.org\/C16-1225"},{"issue":"23","key":"5605_CR28","doi-asserted-by":"publisher","first-page":"11472","DOI":"10.3390\/app112311472","volume":"11","author":"Y Hur","year":"2021","unstructured":"Hur Y, Son S, Shim M, Lim J, Lim H (2021) K-epic: Entity-perceived context representation in korean relation extraction. Appl Sci 11(23):11472","journal-title":"Appl Sci"},{"key":"5605_CR29","unstructured":"Park S, Kim S, Moon J, Cho WI, Cho K, Han J, Park J, Song C, Kim J, Song Y et al (2021) Klue: Korean language understanding evaluation. In: Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS 2021). Advances in Neural Information Processing Systems"},{"key":"5605_CR30","unstructured":"Kim G, Kim J, Son J, Lim HS (2022) Kochet: A korean cultural heritage corpus for entity-related tasks. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 3496\u20133505"},{"key":"5605_CR31","doi-asserted-by":"crossref","unstructured":"Yang S, Choi M, Cho Y, Choo J (2023) Histred: A historical document-level relation extraction dataset. arXiv:2307.04285","DOI":"10.18653\/v1\/2023.acl-long.180"},{"key":"5605_CR32","unstructured":"Heo H, Ko H, Kim S, Han G, Park J, Park K (2021) PORORO: Platform Of neuRal mOdels for natuRal language prOcessing. https:\/\/github.com\/kakaobrain\/pororo"},{"key":"5605_CR33","unstructured":"Clark K, Luong MT, Le QV, Manning CD (2019) Electra: Pre-training text encoders as discriminators rather than generators. In: International Conference on Learning Representations"},{"key":"5605_CR34","doi-asserted-by":"crossref","unstructured":"Chia YK, Bing L, Aljunied SM, Si L, Poria S (2022) A dataset for hyper-relational extraction and a cube-filling approach. arXiv:2211.10018","DOI":"10.18653\/v1\/2022.emnlp-main.688"},{"issue":"5","key":"5605_CR35","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1037\/h0031619","volume":"76","author":"JL Fleiss","year":"1971","unstructured":"Fleiss JL (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76(5):378","journal-title":"Psychol Bull"},{"key":"5605_CR36","doi-asserted-by":"crossref","unstructured":"Lin TY, Goyal P, Girshick R, He K, Doll\u00e1r P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980\u20132988","DOI":"10.1109\/ICCV.2017.324"},{"key":"5605_CR37","doi-asserted-by":"crossref","unstructured":"Jia R, Wong C, Poon H (2019) Document-level $$ n $$-ary relation extraction with multiscale representation learning. arXiv:1904.02347","DOI":"10.18653\/v1\/N19-1370"},{"key":"5605_CR38","unstructured":"Zheng H, Fu J, Zha ZJ, Luo J (2019) Learning deep bilinear transformation for fine-grained image representation. Advances in Neural Information Processing Systems. 32"},{"key":"5605_CR39","doi-asserted-by":"crossref","unstructured":"Tang Y, Huang J, Wang G, He X, Zhou B (2020) Orthogonal relation transforms with graph context modeling for knowledge graph embedding. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2713\u20132722","DOI":"10.18653\/v1\/2020.acl-main.241"},{"key":"5605_CR40","unstructured":"Loshchilov I, Hutter F (2017) Decoupled weight decay regularization. arXiv:1711.05101"},{"key":"5605_CR41","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171\u20134186"},{"key":"5605_CR42","doi-asserted-by":"crossref","unstructured":"Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Davison J, Shleifer S, Platen P, Ma C, Jernite Y, Plu J, Xu C, Scao TL, Gugger S, Drame M, Lhoest Q, Rush AM (2020) Transformers: State-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38\u201345. Association for Computational Linguistics, Online. https:\/\/www.aclweb.org\/anthology\/2020.emnlp-demos.6","DOI":"10.18653\/v1\/2020.emnlp-demos.6"}],"container-title":["Applied Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-024-05605-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10489-024-05605-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-024-05605-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T12:43:58Z","timestamp":1723034638000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10489-024-05605-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,2]]},"references-count":42,"journal-issue":{"issue":"17-18","published-print":{"date-parts":[[2024,9]]}},"alternative-id":["5605"],"URL":"https:\/\/doi.org\/10.1007\/s10489-024-05605-9","relation":{},"ISSN":["0924-669X","1573-7497"],"issn-type":[{"type":"print","value":"0924-669X"},{"type":"electronic","value":"1573-7497"}],"subject":[],"published":{"date-parts":[[2024,7,2]]},"assertion":[{"value":"8 June 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 July 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"The authors declare that they have no conflict of interest regarding the research carried out and the data produced. The authors are aware of and consent to the publication of data resulting from this research.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical and informed consent for data used"}}]}}