{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,21]],"date-time":"2025-06-21T04:40:08Z","timestamp":1750480808883,"version":"3.41.0"},"reference-count":13,"publisher":"Association for Natural Language Processing","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Journal of Natural Language Processing"],"published-print":{"date-parts":[[2025]]},"DOI":"10.5715\/jnlp.32.691","type":"journal-article","created":{"date-parts":[[2025,6,14]],"date-time":"2025-06-14T22:09:43Z","timestamp":1749938983000},"page":"691-698","source":"Crossref","is-referenced-by-count":0,"title":["Investigation of the Inference Capabilities and Memorization of Pre-trained Language Models","\u300c\u672a\u77e5\u306e\u77e5\u8b58\u306b\u5bfe\u3059\u308b\u4e8b\u524d\u5b66\u7fd2\u6e08\u307f\u8a00\u8a9e\u30e2\u30c7\u30eb\u304c\u6301\u3064\u63a8\u8ad6\u80fd\u529b\u306e\u8abf\u67fb\u300d\u306e\u7814\u7a76\u904e\u7a0b"],"prefix":"10.5715","volume":"32","author":[{"given":"Yusuke","family":"Sakai","sequence":"first","affiliation":[{"name":"Nara Institute of Science and Technology"}]}],"member":"3685","reference":[{"key":"1","unstructured":"Bordes, A., Usunier, N., Garcia-Dur\u00e1n, A., Weston, J., and Yakhnenko, O. (2013). \u201cTranslating Embeddings for Modeling Multi-Relational Data.\u201d In <i>Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2<\/i>, NIPS\u201913, pp. 2787\u20132795, Red Hook, NY, USA. Curran Associates Inc."},{"key":"2","unstructured":"Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). \u201cBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.\u201d In Burstein, J., Doran, C., and Solorio, T. (Eds.), <i>Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)<\/i>, pp. 4171\u20134186, Minneapolis, Minnesota. Association for Computational Linguistics."},{"key":"3","doi-asserted-by":"crossref","unstructured":"\u5927\u7fbd\u672a\u60a0\uff0c\u6817\u6797\u6a39\u751f\uff0c\u5927\u5185\u5553\u6a39\uff0c\u6e21\u8fba\u592a\u90ce (2024). \u8a00\u8a9e\u30e2\u30c7\u30eb\u306e\u7b2c\u4e8c\u8a00\u8a9e\u7372\u5f97. \u81ea\u7136\u8a00\u8a9e\u51e6\u7406, 31 (2), pp. 433\u2013455. [M. Oba et al. (2024). Second Language Acquisition of Neural Language Models. Natural Language Processing, 31(2), pp. 433\u2013455.].","DOI":"10.5715\/jnlp.31.433"},{"key":"4","unstructured":"Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. (2020). \u201cExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.\u201d <i>Journal of Machine Learning Research<\/i>, 21 (140), pp. 1\u201367."},{"key":"5","doi-asserted-by":"crossref","unstructured":"\u5742\u4e95\u512a\u4ecb (2024). \u201cDoes Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion?\u201d \u81ea\u7136\u8a00\u8a9e\u51e6\u7406, 31 (4), pp. 1767\u20131773. [Y. Sakai (2024). Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion? Natural Language Processing, 31(4), pp. 1767\u20131773.].","DOI":"10.5715\/jnlp.31.1767"},{"key":"6","doi-asserted-by":"crossref","unstructured":"\u5742\u4e95\u512a\u4ecb\uff0c\u4e0a\u57a3\u5916\u82f1\u525b\uff0c\u6797\u514b\u5f66\uff0c\u6e21\u8fba\u592a\u90ce (2024). \u672a\u77e5\u306e\u77e5\u8b58\u306b\u5bfe\u3059\u308b\u4e8b\u524d\u5b66\u7fd2\u6e08\u307f\u8a00\u8a9e\u30e2\u30c7\u30eb\u304c\u6301\u3064\u63a8\u8ad6\u80fd\u529b\u306e\u8abf\u67fb. \u81ea\u7136\u8a00\u8a9e\u51e6\u7406, 31 (4), pp. 1427\u20131457. [Y. Sakai et al. (2024). Investigation of the Inference Capabilities and Memorization of Pre-trained Language Models. Natural Language Processing, 31(4), pp. 1427\u20131457.].","DOI":"10.5715\/jnlp.31.1427"},{"key":"7","doi-asserted-by":"crossref","unstructured":"\u5742\u4e95\u512a\u4ecb (2025). \u201cSimultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Pair.\u201d \u81ea\u7136\u8a00\u8a9e\u51e6\u7406, 32 (1), pp. 348\u2013354. [Y. Sakai (2025). Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Pair. Natural Language Processing, 32(1), pp. 348\u2013354.].","DOI":"10.5715\/jnlp.32.348"},{"key":"8","doi-asserted-by":"crossref","unstructured":"Sakai, Y., Kamigaito, H., Hayashi, K., and Watanabe, T. (2024a). \u201cDoes Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion?\u201d In Duh, K., Gomez, H., and Bethard, S. (Eds.), <i>Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)<\/i>, pp. 8091\u20138106, Mexico City, Mexico. Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.naacl-long.447"},{"key":"9","doi-asserted-by":"crossref","unstructured":"Sakai, Y., Kamigaito, H., and Watanabe, T. (2024b). \u201cmCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humans.\u201d In Ku, L.-W., Martins, A., and Srikumar, V. (Eds.), <i>Findings of the Association for Computational Linguistics: ACL 2024<\/i>, pp. 14182\u201314214, Bangkok, Thailand. Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.findings-acl.844"},{"key":"10","doi-asserted-by":"crossref","unstructured":"Sakai, Y., Makinae, M., Kamigaito, H., and Watanabe, T. (2024c). \u201cSimultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Pair.\u201d In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (Eds.), <i>Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing<\/i>, pp. 22375\u201322398, Miami, Florida, USA. Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.emnlp-main.1248"},{"key":"11","doi-asserted-by":"crossref","unstructured":"Saxena, A., Kochsiek, A., and Gemulla, R. (2022). \u201cSequence-to-Sequence Knowledge Graph Completion and Question Answering.\u201d In Muresan, S., Nakov, P., and Villavicencio, A. (Eds.), <i>Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)<\/i>, pp. 2814\u20132828, Dublin, Ireland. Association for Computational Linguistics.","DOI":"10.18653\/v1\/2022.acl-long.201"},{"key":"12","doi-asserted-by":"crossref","unstructured":"\u6749\u672c\u667a\u7d00\uff0c\u5c3e\u4e0a\u5eb7\u96c5\uff0c\u8c37\u4e2d\u77b3 (2024). \u30a2\u30b9\u30da\u30af\u30c8\u3092\u8003\u616e\u3057\u305f\u65e5\u672c\u8a9e\u6642\u9593\u63a8\u8ad6\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u69cb\u7bc9. \u81ea\u7136\u8a00\u8a9e\u51e6\u7406, 31 (2), pp. 637\u2013679. [T. Sugimoto et al. (2024). Jamp_sp : A Controlled Japanese Temporal Inference Dataset Considering Aspect. Journal of Natural Language Processing, 31(2), pp. 637\u2013679.].","DOI":"10.5715\/jnlp.31.637"},{"key":"13","doi-asserted-by":"crossref","unstructured":"Wang, L., Zhao, W., Wei, Z., and Liu, J. (2022). \u201cSimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models.\u201d In Muresan, S., Nakov, P., and Villavicencio, A. (Eds.), <i>Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)<\/i>, pp. 4281\u20134294, Dublin, Ireland. Association for Computational Linguistics.","DOI":"10.18653\/v1\/2022.acl-long.295"}],"container-title":["Journal of Natural Language Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jstage.jst.go.jp\/article\/jnlp\/32\/2\/32_691\/_pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,21]],"date-time":"2025-06-21T04:13:23Z","timestamp":1750479203000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jstage.jst.go.jp\/article\/jnlp\/32\/2\/32_691\/_article\/-char\/ja\/"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025]]},"references-count":13,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025]]}},"URL":"https:\/\/doi.org\/10.5715\/jnlp.32.691","relation":{},"ISSN":["1340-7619","2185-8314"],"issn-type":[{"type":"print","value":"1340-7619"},{"type":"electronic","value":"2185-8314"}],"subject":[],"published":{"date-parts":[[2025]]}}}