{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,3]],"date-time":"2026-05-03T09:56:34Z","timestamp":1777802194893,"version":"3.51.4"},"reference-count":61,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2025,1,16]],"date-time":"2025-01-16T00:00:00Z","timestamp":1736985600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,16]],"date-time":"2025-01-16T00:00:00Z","timestamp":1736985600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002241","name":"Japan Science and Technology Agency","doi-asserted-by":"publisher","award":["JPMJSP2124"],"award-info":[{"award-number":["JPMJSP2124"]}],"id":[{"id":"10.13039\/501100002241","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002241","name":"Japan Science and Technology Agency","doi-asserted-by":"publisher","award":["JPMJSP2124"],"award-info":[{"award-number":["JPMJSP2124"]}],"id":[{"id":"10.13039\/501100002241","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009757","name":"National Institute of Advanced Industrial Science and Technology","doi-asserted-by":"publisher","award":["AHD02075"],"award-info":[{"award-number":["AHD02075"]}],"id":[{"id":"10.13039\/100009757","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009757","name":"National Institute of Advanced Industrial Science and Technology","doi-asserted-by":"publisher","award":["AHD02075"],"award-info":[{"award-number":["AHD02075"]}],"id":[{"id":"10.13039\/100009757","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009757","name":"National Institute of Advanced Industrial Science and Technology","doi-asserted-by":"publisher","award":["AHD02075"],"award-info":[{"award-number":["AHD02075"]}],"id":[{"id":"10.13039\/100009757","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009757","name":"National Institute of Advanced Industrial Science and Technology","doi-asserted-by":"publisher","award":["AHD02075"],"award-info":[{"award-number":["AHD02075"]}],"id":[{"id":"10.13039\/100009757","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009757","name":"National Institute of Advanced Industrial Science and Technology","doi-asserted-by":"publisher","award":["AHD02075"],"award-info":[{"award-number":["AHD02075"]}],"id":[{"id":"10.13039\/100009757","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009757","name":"National Institute of Advanced Industrial Science and Technology","doi-asserted-by":"publisher","award":["AHD02075"],"award-info":[{"award-number":["AHD02075"]}],"id":[{"id":"10.13039\/100009757","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Knowl Inf Syst"],"published-print":{"date-parts":[[2025,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Commonsense explanation generation refers to reasoning and explaining why a commonsense statement contradicts commonsense knowledge, such as why the statement \u201cMy dad grew volleyballs in his garden\u201d is nonsensical. While such reasoning is trivial for humans, it remains a challenge for AI systems. Despite their notable performance in tasks like text generation and reasoning, large language models (LLMs) often fall short of consistently generating coherent and accurate commonsense explanations. To bridge this gap, we propose a novel Two-stage Identification and Prompting (TIP) framework for enhancing LLMs\u2019 ability to handle the task of commonsense explanation generation. Specifically, in the first stage, TIP identifies the nonsensical concept in the given statement, pinpointing the specific element that contradicts commonsense knowledge. In the second stage, TIP generates implicit knowledge based on the identified nonsensical concept and then leverages this implicit knowledge to guide the adopted LLMs in generating explanations. In order to demonstrate the effectiveness of the proposed TIP framework for commonsense explanation generation, we conducted extensive experiments based on the ComVE dataset and a newly constructed CSE dataset, where a variety of LLMs are evaluated. The experimental results show that TIP consistently outperforms all baseline methods across multiple metrics, demonstrating its effectiveness in improving LLMs\u2019 commonsense reasoning and explanation generation capabilities.<\/jats:p>","DOI":"10.1007\/s10115-024-02326-w","type":"journal-article","created":{"date-parts":[[2025,1,16]],"date-time":"2025-01-16T04:38:04Z","timestamp":1737002284000},"page":"3663-3698","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Implicit knowledge-augmented prompting for commonsense explanation generation"],"prefix":"10.1007","volume":"67","author":[{"given":"Yan","family":"Ge","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hai-Tao","family":"Yu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chao","family":"Lei","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xin","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adam","family":"Jatowt","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kyoung-sook","family":"Kim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Steven","family":"Lynden","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Akiyoshi","family":"Matono","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,1,16]]},"reference":[{"key":"2326_CR1","unstructured":"Gunning D (2018) Machine common sense concept paper. arXiv preprint arXiv:1810.07528"},{"issue":"4","key":"2326_CR2","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1145\/3186549.3186562","volume":"46","author":"N Tandon","year":"2018","unstructured":"Tandon N, Varde AS, Melo G (2018) Commonsense knowledge in machine intelligence. ACM SIGMOD Rec 46(4):49\u201352","journal-title":"ACM SIGMOD Rec"},{"issue":"9","key":"2326_CR3","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1145\/2701413","volume":"58","author":"E Davis","year":"2015","unstructured":"Davis E, Marcus G (2015) Commonsense reasoning and commonsense knowledge in artificial intelligence. Commun ACM 58(9):92\u2013103","journal-title":"Commun ACM"},{"key":"2326_CR4","doi-asserted-by":"crossref","unstructured":"Sap M, Shwartz V, Bosselut A, Choi Y, Roth D (2020) Commonsense reasoning for natural language processing. In: Proceedings of the 58th annual meeting of the association for computational linguistics: tutorial abstracts, pp 27\u201333","DOI":"10.18653\/v1\/2020.acl-tutorials.7"},{"key":"2326_CR5","doi-asserted-by":"publisher","first-page":"17309","DOI":"10.1007\/s00521-020-05102-3","volume":"32","author":"RA Potamias","year":"2020","unstructured":"Potamias RA, Siolas G, Stafylopatis A-G (2020) A transformer-based approach to irony and sarcasm detection. Neural Comput Appl 32:17309\u201317320","journal-title":"Neural Comput Appl"},{"key":"2326_CR6","doi-asserted-by":"crossref","unstructured":"Chakrabarty T, Ghosh D, Muresan S, Peng N (2020) $$R^3$$: Reverse, retrieve, and rank for sarcasm generation with commonsense knowledge. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 7976\u20137986","DOI":"10.18653\/v1\/2020.acl-main.711"},{"key":"2326_CR7","doi-asserted-by":"crossref","unstructured":"Hasan MK, Lee S, Rahman W, Zadeh A, Mihalcea R, Morency L-P, Hoque E (2021) Humor knowledge enriched transformer for understanding multimodal humor. In: Proceedings of the AAAI conference on artificial intelligence, vol 35. pp 12972\u201312980","DOI":"10.1609\/aaai.v35i14.17534"},{"key":"2326_CR8","doi-asserted-by":"crossref","unstructured":"Sap M, Rashkin H, Chen D, LeBras R, Choi Y (2019) Socialiqa: commonsense reasoning about social interactions. In: Conference on empirical methods in natural language processing","DOI":"10.18653\/v1\/D19-1454"},{"key":"2326_CR9","unstructured":"Talmor A, Herzig J, Lourie N, Berant J (2019) Commonsenseqa: a question answering challenge targeting commonsense knowledge. In: Proceedings of the 2019 conference of the north American chapter of the association for computational linguistics: human language technologies, vol 1 (Long and Short Papers). pp 4149\u20134158"},{"key":"2326_CR10","doi-asserted-by":"crossref","unstructured":"Mihaylov T, Clark P, Khot T, Sabharwal A (2018) Can a suit of armor conduct electricity? a new dataset for open book question answering. In: Proceedings of the 2018 conference on empirical methods in natural language processing. pp 2381\u20132391","DOI":"10.18653\/v1\/D18-1260"},{"key":"2326_CR11","doi-asserted-by":"crossref","unstructured":"Wang C, Liang S, Jin Y, Wang Y, Zhu X, Zhang Y (2020) Semeval-2020 task 4: commonsense validation and explanation. arXiv preprint arXiv:2007.00236","DOI":"10.18653\/v1\/2020.semeval-1.39"},{"key":"2326_CR12","doi-asserted-by":"crossref","unstructured":"Aggarwal S, Mandowara D, Agrawal V, Khandelwal D, Singla P, Garg D (2021) Explanations for commonsenseqa: New dataset and models. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (vol 1: Long Papers). pp 3050\u20133065","DOI":"10.18653\/v1\/2021.acl-long.238"},{"key":"2326_CR13","doi-asserted-by":"crossref","unstructured":"Wan J, Huang X (2020) Kalm at semeval-2020 task 4: knowledge-aware language models for comprehension and generation. arXiv preprint arXiv:2005.11768","DOI":"10.18653\/v1\/2020.semeval-1.67"},{"key":"2326_CR14","doi-asserted-by":"crossref","unstructured":"Yu W, Zhu C, Qin L, Zhang Z, Zhao T, Jiang M (2022) Diversifying content generation for commonsense reasoning with mixture of knowledge graph experts. Findings of the Association for Computational Linguistics: ACL 2022","DOI":"10.18653\/v1\/2022.findings-acl.149"},{"issue":"8","key":"2326_CR15","first-page":"9","volume":"1","author":"A Radford","year":"2019","unstructured":"Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I et al (2019) Language models are unsupervised multitask learners. OpenAI blog 1(8):9","journal-title":"OpenAI blog"},{"key":"2326_CR16","doi-asserted-by":"crossref","unstructured":"Jon J, Faj\u010d\u00edk M, Do\u010dekal M, Smr\u017e P (2020) But-fit at semeval-2020 task 4: Multilingual commonsense. arXiv preprint arXiv:2008.07259","DOI":"10.18653\/v1\/2020.semeval-1.46"},{"key":"2326_CR17","doi-asserted-by":"crossref","unstructured":"Fang Y, Zhang Y (2022) Data-efficient concept extraction from pre-trained language models for commonsense explanation generation. In: Findings of the association for computational linguistics: EMNLP 2022. pp 5883\u20135893","DOI":"10.18653\/v1\/2022.findings-emnlp.433"},{"key":"2326_CR18","doi-asserted-by":"crossref","unstructured":"Cheng S, Wu Z, Chen J, Li Z, Liu Y, Kong L (2023) Unsupervised explanation generation via correct instantiations. In: Proceedings of the AAAI conference on artificial intelligence, vol 37. pp 12700\u201312708","DOI":"10.1609\/aaai.v37i11.26494"},{"key":"2326_CR19","unstructured":"Zhao WX, Zhou K, Li J, Tang T, Wang X, Hou Y, Min Y, Zhang B, Zhang J, Dong Z, et al (2023) A survey of large language models. arXiv preprint arXiv:2303.18223"},{"key":"2326_CR20","unstructured":"Wei J, Tay Y, Bommasani R, Raffel C, Zoph B, Borgeaud S, Yogatama D, Bosma M, Zhou D, Metzler D, et al (2022) Emergent abilities of large language models. arXiv preprint arXiv:2206.07682"},{"key":"2326_CR21","unstructured":"Kaplan J, McCandlish S, Henighan T, Brown TB, Chess B, Child R, Gray S, Radford A, Wu J, Amodei D (2020) Scaling laws for neural language models. arXiv preprint arXiv:2001.08361"},{"key":"2326_CR22","doi-asserted-by":"crossref","unstructured":"Huang J, Chang KC-C (2023) Towards reasoning in large language models: a survey. In: 61st annual meeting of the association for computational linguistics, ACL 2023. Association for Computational Linguistics (ACL), pp. 1049\u20131065","DOI":"10.18653\/v1\/2023.findings-acl.67"},{"key":"2326_CR23","unstructured":"Zhang Z, Zhang A, Li M, Smola A. Automatic chain of thought prompting in large language models. In: The eleventh international conference on learning representations"},{"issue":"S1","key":"2326_CR24","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1121\/1.2016299","volume":"62","author":"F Jelinek","year":"1977","unstructured":"Jelinek F, Mercer RL, Bahl LR, Baker JK (1977) Perplexity\u2013a measure of the difficulty of speech recognition tasks. J Acoust Soc Am 62(S1):63\u201363","journal-title":"J Acoust Soc Am"},{"key":"2326_CR25","doi-asserted-by":"crossref","unstructured":"Speer R, Chin J, Havasi C (2017) Conceptnet 5.5: an open multilingual graph of general knowledge. In: Proceedings of the AAAI conference on artificial intelligence, vol 31","DOI":"10.1609\/aaai.v31i1.11164"},{"key":"2326_CR26","doi-asserted-by":"crossref","unstructured":"Sap M, Le\u00a0Bras R, Allaway E, Bhagavatula C, Lourie N, Rashkin H, Roof B, Smith NA, Choi Y (2019) Atomic: an atlas of machine commonsense for if-then reasoning. In: Proceedings of the AAAI conference on artificial intelligence, vol 33. pp 3027\u20133035","DOI":"10.1609\/aaai.v33i01.33013027"},{"key":"2326_CR27","doi-asserted-by":"crossref","unstructured":"Bosselut A, Rashkin H, Sap M, Malaviya C, Celikyilmaz A, Choi Y (2019) Comet: commonsense transformers for knowledge graph construction. In: Association for computational linguistics (ACL)","DOI":"10.18653\/v1\/P19-1470"},{"key":"2326_CR28","unstructured":"Zhang S, Roller S, Goyal N, Artetxe M, Chen M, Chen S, Dewan C, Diab M, Li X, Lin XV, et al (2022) Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068"},{"key":"2326_CR29","unstructured":"Storks S, Gao Q, Chai JY (2019) Commonsense reasoning for natural language understanding: A survey of benchmarks, resources, and approaches. arXiv preprint arXiv:1904.01172, 1\u201360"},{"key":"2326_CR30","doi-asserted-by":"crossref","unstructured":"Bhargava P, Ng V (2022) Commonsense knowledge reasoning and generation with pre-trained language models: a survey. In: Proceedings of the AAAI conference on artificial intelligence, vol 36. pp 12317\u201312325","DOI":"10.1609\/aaai.v36i11.21496"},{"issue":"11","key":"2326_CR31","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1145\/219717.219745","volume":"38","author":"DB Lenat","year":"1995","unstructured":"Lenat DB (1995) Cyc: a large-scale investment in knowledge infrastructure. Commun ACM 38(11):33\u201338","journal-title":"Commun ACM"},{"key":"2326_CR32","doi-asserted-by":"crossref","unstructured":"Singh P, Lin T, Mueller ET, Lim G, Perkins T, Li\u00a0Zhu W (2002) Open mind common sense: Knowledge acquisition from the general public. In: On the move to meaningful internet systems 2002: CoopIS, DOA, and ODBASE: confederated international conferences CoopIS, DOA, and ODBASE 2002 Proceedings. Springer. pp. 1223\u20131237","DOI":"10.1007\/3-540-36124-3_77"},{"key":"2326_CR33","doi-asserted-by":"crossref","unstructured":"Zellers R, Holtzman A, Bisk Y, Farhadi A, Choi Y (2019) Hellaswag: Can a machine really finish your sentence? In: Proceedings of the 57th annual meeting of the association for computational linguistics. pp. 4791\u20134800","DOI":"10.18653\/v1\/P19-1472"},{"key":"2326_CR34","doi-asserted-by":"crossref","unstructured":"Huang L, Le\u00a0Bras R, Bhagavatula C, Choi Y (2019) Cosmos qa: machine reading comprehension with contextual commonsense reasoning. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). pp 2391\u20132401","DOI":"10.18653\/v1\/D19-1243"},{"key":"2326_CR35","doi-asserted-by":"crossref","unstructured":"Ji H, Ke P, Huang S, Wei F, Huang M (2020) Generating commonsense explanation by extracting bridge concepts from reasoning paths. In: Proceedings of the 1st conference of the asia-pacific chapter of the association for computational linguistics and the 10th international joint conference on natural language processing. pp 248\u2013257","DOI":"10.18653\/v1\/2020.aacl-main.28"},{"issue":"11s","key":"2326_CR36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3512467","volume":"54","author":"W Yu","year":"2022","unstructured":"Yu W, Zhu C, Li Z, Hu Z, Wang Q, Ji H, Jiang M (2022) A survey of knowledge-enhanced text generation. ACM Comput Surv 54(11s):1\u201338","journal-title":"ACM Comput Surv"},{"key":"2326_CR37","doi-asserted-by":"crossref","unstructured":"Wu S, Wang M, Li Y, Zhang D, Wu Z (2022) Improving the applicability of knowledge-enhanced dialogue generation systems by using heterogeneous knowledge from multiple sources. In: Proceedings of the fifteenth ACM international conference on WEB search and data mining. pp 1149\u20131157","DOI":"10.1145\/3488560.3498393"},{"key":"2326_CR38","doi-asserted-by":"crossref","unstructured":"Hu L, Liu Z, Zhao Z, Hou L, Nie L, Li J (2023) A survey of knowledge enhanced pre-trained language models. IEEE Trans Knowl Data Eng","DOI":"10.1109\/TKDE.2023.3310002"},{"key":"2326_CR39","doi-asserted-by":"crossref","unstructured":"Fan A, Gardent C, Braud C, Bordes A (2019) Using local knowledge graph construction to scale seq2seq models to multi-document inputs. In: 2019 conference on empirical methods in natural language processing and 9th international joint conference on natural language processing","DOI":"10.18653\/v1\/D19-1428"},{"key":"2326_CR40","doi-asserted-by":"crossref","unstructured":"Wang J, Wang C, Qiu M, Shi Q, Wang H, Huang J, Gao M (2022) Kecp: knowledge enhanced contrastive prompting for few-shot extractive question answering. In: Proceedings of the 2022 conference on empirical methods in natural language processing. pp 3152\u20133163","DOI":"10.18653\/v1\/2022.emnlp-main.206"},{"key":"2326_CR41","doi-asserted-by":"crossref","unstructured":"Huang L, Wu L, Wang L (2020) Knowledge graph-augmented abstractive summarization with semantic-driven cloze reward. In: Proceedings of the 58th annual meeting of the association for computational linguistics. pp 5094\u20135107","DOI":"10.18653\/v1\/2020.acl-main.457"},{"key":"2326_CR42","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2022.109460","volume":"252","author":"Q Xie","year":"2022","unstructured":"Xie Q, Bishop JA, Tiwari P, Ananiadou S (2022) Pre-trained language models with domain knowledge for biomedical extractive summarization. Knowl-Based Syst 252:109460","journal-title":"Knowl-Based Syst"},{"key":"2326_CR43","doi-asserted-by":"crossref","unstructured":"Zhou H, Huang M, Liu Y, Chen W, Zhu X (2021) Earl: informative knowledge-grounded conversation generation with entity-agnostic representation learning. In: Proceedings of the 2021 conference on empirical methods in natural language processing. pp 2383\u20132395","DOI":"10.18653\/v1\/2021.emnlp-main.184"},{"key":"2326_CR44","unstructured":"Kenton JDM-WC, Toutanova LK (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, 1, 2. Minneapolis, Minnesota"},{"key":"2326_CR45","unstructured":"Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692"},{"key":"2326_CR46","doi-asserted-by":"crossref","unstructured":"Lewis M, Liu Y, Goyal N, Ghazvininejad M, Mohamed A, Levy O, Stoyanov V, Zettlemoyer L (2019) Bart: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"2326_CR47","unstructured":"Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov RR, Le QV (2019) Xlnet: generalized autoregressive pretraining for language understanding. Adv Neural Inf Process Syst 32"},{"key":"2326_CR48","unstructured":"Clark K, Luong M-T, Le QV, Manning CD (2020) Electra: pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555"},{"key":"2326_CR49","unstructured":"Sun Y, Wang S, Li Y, Feng S, Chen X, Zhang H, Tian X, Zhu D, Tian H, Wu H (2019) Ernie: enhanced representation through knowledge integration. arXiv preprint arXiv:1904.09223"},{"key":"2326_CR50","unstructured":"Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2019) Albert: a lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942"},{"key":"2326_CR51","first-page":"1877","volume":"33","author":"T Brown","year":"2020","unstructured":"Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877\u20131901","journal-title":"Adv Neural Inf Process Syst"},{"key":"2326_CR52","unstructured":"Touvron H, Martin L, Stone K, Albert P, Almahairi A, Babaei Y, Bashlykov N, Batra S, Bhargava P, Bhosale S, et al (2023) Llama 2: open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288"},{"issue":"240","key":"2326_CR53","first-page":"1","volume":"24","author":"A Chowdhery","year":"2023","unstructured":"Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S et al (2023) Palm: scaling language modeling with pathways. J Mach Learn Res 24(240):1\u2013113","journal-title":"J Mach Learn Res"},{"key":"2326_CR54","unstructured":"Le\u00a0Scao T, Fan A, Akiki C, Pavlick E, Ili\u0107 S, Hesslow D, Castagn\u00e9 R, Luccioni AS, Yvon F, Gall\u00e9 M, et al (2023) Bloom: a 176b-parameter open-access multilingual language model"},{"key":"2326_CR55","unstructured":"Chiang W-L, Li Z, Lin Z, Sheng Y, Wu Z, Zhang H, Zheng L, Zhuang S, Zhuang Y, Gonzalez JE, et al (2023) Vicuna: an open-source chatbot impressing gpt-4 with 90%* chatgpt quality, march 2023. https:\/\/lmsys.org\/blog\/2023-03-30-vicuna 3(5)"},{"key":"2326_CR56","unstructured":"Zhou Y, Muresanu AI, Han Z, Paster K, Pitis S, Chan H, Ba J. Large language models are human-level prompt engineers. In: The eleventh international conference on learning representations"},{"key":"2326_CR57","doi-asserted-by":"crossref","unstructured":"Papineni K, Roukos S, Ward T, Zhu W-J (2002) Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the association for computational linguistics. pp 311\u2013318","DOI":"10.3115\/1073083.1073135"},{"key":"2326_CR58","unstructured":"Lin C-Y (2004) Rouge: a package for automatic evaluation of summaries. In: Text summarization branches out. pp 74\u201381"},{"key":"2326_CR59","unstructured":"Banerjee S, Lavie A (2005) Meteor: an automatic metric for mt evaluation with improved correlation with human judgments. Intrinsic Extrinsic Eval Meas Mach Transl Summ 65"},{"key":"2326_CR60","doi-asserted-by":"crossref","unstructured":"Reimers N, Gurevych I (2019) Sentence-bert: sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084","DOI":"10.18653\/v1\/D19-1410"},{"key":"2326_CR61","unstructured":"Zhang T, Kishore V, Wu F, Weinberger KQ, Artzi Y. Bertscore: evaluating text generation with bert. In: International conference on learning representations"}],"container-title":["Knowledge and Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10115-024-02326-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10115-024-02326-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10115-024-02326-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,12]],"date-time":"2025-09-12T12:25:40Z","timestamp":1757679940000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10115-024-02326-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,16]]},"references-count":61,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,4]]}},"alternative-id":["2326"],"URL":"https:\/\/doi.org\/10.1007\/s10115-024-02326-w","relation":{},"ISSN":["0219-1377","0219-3116"],"issn-type":[{"value":"0219-1377","type":"print"},{"value":"0219-3116","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,16]]},"assertion":[{"value":"15 October 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 December 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 December 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 January 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflict of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}