{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T15:25:01Z","timestamp":1777562701803,"version":"3.51.4"},"reference-count":16,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,2,28]],"date-time":"2022-02-28T00:00:00Z","timestamp":1646006400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,2,28]],"date-time":"2022-02-28T00:00:00Z","timestamp":1646006400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["18H0333808"],"award-info":[{"award-number":["18H0333808"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Rev Socionetwork Strat"],"published-print":{"date-parts":[[2022,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>A legal textual entailment task is a task to recognize entailment between a law article and its statements. In the Competition on Legal Information Extraction\/Entailment (COLIEE), this task is designed as a task to confirm the entailment of a yes\/no answer from the given civil code article(s). Based on the development of deep-learning-based natural language processing tools such as bidirectional encoder representations from transformers (BERT), many participants in the task used such tools, and the best performance system of COLIEE 2020 was a BERT-based system. However, because of the limitation of the size of training data provided by the task organizer, training such tools to adapt to the variability of the questions is difficult. In this paper, we propose a data-augmentation method to make training data using civil code articles for understanding the syntactic structure of the questions and articles for entailment. Our BERT-based ensemble system, which uses this augmentation method, achieves the best performance (accuracy = 0.7037) in Task 4 of COLIEE 2021. We also introduce the results of additional experiments to discuss the characteristics of the proposed method.<\/jats:p>","DOI":"10.1007\/s12626-022-00104-0","type":"journal-article","created":{"date-parts":[[2022,2,28]],"date-time":"2022-02-28T04:26:41Z","timestamp":1646022401000},"page":"175-196","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Data-Augmentation Method for BERT-based Legal Textual Entailment Systems in COLIEE Statute Law Task"],"prefix":"10.1007","volume":"16","author":[{"given":"Yasuhiro","family":"Aoki","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2096-1218","authenticated-orcid":false,"given":"Masaharu","family":"Yoshioka","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Youta","family":"Suzuki","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,2,28]]},"reference":[{"key":"104_CR1","doi-asserted-by":"publisher","unstructured":"Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171\u20134186. Association for Computational Linguistics, Minneapolis, Minnesota. https:\/\/doi.org\/10.18653\/v1\/N19-1423. https:\/\/www.aclweb.org\/anthology\/N19-1423.","DOI":"10.18653\/v1\/N19-1423"},{"key":"104_CR2","unstructured":"Evans, R., Saxton, D., Amos, D., Kohli, P., & Grefenstette, E. (2018). Can neural networks understand logical entailment? In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. https:\/\/openreview.net\/forum?id=SkZxCk-0Z."},{"key":"104_CR3","unstructured":"Kano, Y., Kim, M. Y., Goebel, R., & Satoh, K. (2017). Overview of coliee 2017. In: K.\u00a0Satoh, M.Y. Kim, Y.\u00a0Kano, R.\u00a0Goebel, T.\u00a0Oliveira (eds.) COLIEE 2017. 4th Competition on Legal Information Extraction and Entailment, EPiC Series in Computing, vol.\u00a047, pp. 1\u20138. EasyChair."},{"key":"104_CR4","unstructured":"Kim, M. Y., Goebel, R., Kano, Y., & Satoh, K. (2016). Coliee-2016: Evaluation of the competition on legal information extraction and entailment. In: The Proceedings of the 10th International Workshop on Juris-Informatics (JURISIN2016). Paper 11"},{"key":"104_CR5","doi-asserted-by":"publisher","first-page":"282","DOI":"10.1007\/978-3-319-50953-2_20","volume-title":"New Frontiers in Artificial Intelligence","author":"MY Kim","year":"2017","unstructured":"Kim, M. Y., Xu, Y., & Goebel, R. (2017). Applying a convolutional neural network to legal question answering. In M. Otake, S. Kurahashi, Y. Ota, K. Satoh, & D. Bekki (Eds.), New Frontiers in Artificial Intelligence (pp. 282\u2013294). Cham: Springer International Publishing."},{"key":"104_CR6","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems, pp. 3111\u20133119."},{"issue":"11","key":"104_CR7","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1145\/219717.219748","volume":"38","author":"GA Miller","year":"1995","unstructured":"Miller, G. A. (1995). WordNet: A lexical database for English. Communications of the ACM, 38(11), 39\u201341.","journal-title":"Communications of the ACM"},{"key":"104_CR8","doi-asserted-by":"publisher","unstructured":"Min, J., McCoy, R. T., Das, D., Pitler, E., & Linzen, T. (2020). Syntactic data augmentation increases robustness to inference heuristics. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2339\u20132352. Association for Computational Linguistics, Online. https:\/\/doi.org\/10.18653\/v1\/2020.acl-main.212. https:\/\/www.aclweb.org\/anthology\/2020.acl-main.212.","DOI":"10.18653\/v1\/2020.acl-main.212"},{"key":"104_CR9","unstructured":"Nguyen, H. T., Vuong, H. Y. T., Nguyen, P. M., Dang, B. T., Bui, Q. M., Vu, S. T., Nguyen, C. M., Tran, V., Satoh, K., & Nguyen, M. L. (2020). Jnlp team: Deep learning for legal processing. In: The Proceedings of the 14th International Workshop on Juris-Informatics (JURISIN2020), pp. 195\u2013208. The Japanese Society of Artificial Intelligence."},{"key":"104_CR10","unstructured":"Rabelo, J., Goebel, R., Kim, M. Y., Kano, Y., Yoshioka, M., & Satoh, K. (2021). Summary of the competition on legal information extraction\/entailment (coliee). In: Proceedings of the COLIEE Workshop in ICAIL, pp. 1\u20137."},{"key":"104_CR11","unstructured":"Rabelo, J., Kim, M. Y., Goebel, R., Yoshioka, M., Kano, Y., & Satoh, K. (2020). COLIEE2020:methods for legal document retrieval and entailment. In: The Proceedings of the 14th International Workshop on Juris-Informatics (JURISIN2020), pp. 114\u2013127. The Japanese Society of Artificial Intelligence."},{"key":"104_CR12","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1007\/978-3-030-58790-1_3","volume-title":"New Frontiers in Artificial Intelligence","author":"J Rabelo","year":"2020","unstructured":"Rabelo, J., Kim, M. Y., Goebel, R., Yoshioka, M., Kano, Y., & Satoh, K. (2020). A summary of the coliee 2019 competition. In M. Sakamoto, N. Okazaki, K. Mineshima, & K. Satoh (Eds.), New Frontiers in Artificial Intelligence (pp. 34\u201349). Cham: Springer International Publishing."},{"key":"104_CR13","unstructured":"Shao, H. L., Chen, Y. C., & Huang, S. C. (2020). BERT-based ensemble model for the statute law retrieval and legal information entailment. In: The Proceedings of the 14th International Workshop on Juris-Informatics (JURISIN2020), pp. 223\u2013234. The Japanese Society of Artificial Intelligence."},{"key":"104_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-019-0197-0","volume":"6","author":"C Shorten","year":"2019","unstructured":"Shorten, C., & Khoshgoftaar, T. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6, 1\u201348. https:\/\/doi.org\/10.1186\/s40537-019-0197-0","journal-title":"Journal of Big Data"},{"key":"104_CR15","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1007\/978-3-030-31605-1_15","volume-title":"New Frontiers in Artificial Intelligence","author":"R Taniguchi","year":"2019","unstructured":"Taniguchi, R., Hoshino, R., & Kano, Y. (2019). Legal question answering system using framenet. In K. Kojima, M. Sakamoto, K. Mineshima, & K. Satoh (Eds.), New Frontiers in Artificial Intelligence (pp. 193\u2013206). Cham: Springer International Publishing."},{"key":"104_CR16","unstructured":"Yoshioka, M., Kano, Y., Kiyota, N., & Satoh, K. (2018). Overview of japanese statute law retrieval and entailment task at coliee-2018. In: The Proceedings of the 12th International Workshop on Juris-Informatics (JURISIN2018), pp. 117\u2013128. The Japanese Society of Artificial Intelligence."}],"container-title":["The Review of Socionetwork Strategies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12626-022-00104-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12626-022-00104-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12626-022-00104-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,24]],"date-time":"2022-04-24T04:04:42Z","timestamp":1650773082000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12626-022-00104-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,28]]},"references-count":16,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,4]]}},"alternative-id":["104"],"URL":"https:\/\/doi.org\/10.1007\/s12626-022-00104-0","relation":{},"ISSN":["2523-3173","1867-3236"],"issn-type":[{"value":"2523-3173","type":"print"},{"value":"1867-3236","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,28]]},"assertion":[{"value":"10 September 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 January 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 February 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all authors, the corresponding author states that there is no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}]}}