{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T17:47:42Z","timestamp":1769104062894,"version":"3.49.0"},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:p>Recently, neural techniques have been used to generate source code automatically. While promising for declarative languages, these approaches achieve much poorer performance on datasets for imperative languages. Since a declarative language is typically embedded in an imperative language (i.e., the turducken-style programming) in real-world software development, the promising results on declarative languages can hardly lead to significant reduction of manual software development efforts.\n\n\n\nIn this paper, we define a new code generation task: given a natural language comment, this task aims to generate a program in a base imperative language with an embedded declarative language. To our knowledge, this is the first turducken-style code generation task. For this task, we present Lyra: a dataset in Python with embedded SQL. This dataset contains 2,000 carefully annotated database manipulation programs from real usage projects. Each program is paired with both a Chinese comment and an English comment. In our experiment, we adopted Transformer, BERT-style, and GPT-style models as baselines. In the best setting, GPT-style model can achieve 24% and 25.5% AST exact matching accuracy using Chinese and English comments, respectively. Therefore, we believe that Lyra provides a new challenge for code generation. Yet, overcoming this challenge may significantly boost the applicability of code generation techniques for real-world software development.<\/jats:p>","DOI":"10.24963\/ijcai.2022\/588","type":"proceedings-article","created":{"date-parts":[[2022,7,15]],"date-time":"2022-07-15T22:55:56Z","timestamp":1657925756000},"page":"4238-4244","source":"Crossref","is-referenced-by-count":4,"title":["Lyra: A Benchmark for Turducken-Style Code Generation"],"prefix":"10.24963","author":[{"given":"Qingyuan","family":"Liang","sequence":"first","affiliation":[{"name":"Peking University"}]},{"given":"Zeyu","family":"Sun","sequence":"additional","affiliation":[{"name":"Peking University"}]},{"given":"Qihao","family":"Zhu","sequence":"additional","affiliation":[{"name":"Peking University"}]},{"given":"Wenjie","family":"Zhang","sequence":"additional","affiliation":[{"name":"Peking University"}]},{"given":"Lian","family":"Yu","sequence":"additional","affiliation":[{"name":"Peking University"}]},{"given":"Yingfei","family":"Xiong","sequence":"additional","affiliation":[{"name":"Peking University"}]},{"given":"Lu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Peking University"}]}],"member":"10584","event":{"name":"Thirty-First International Joint Conference on Artificial Intelligence {IJCAI-22}","theme":"Artificial Intelligence","location":"Vienna, Austria","acronym":"IJCAI-2022","number":"31","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"start":{"date-parts":[[2022,7,23]]},"end":{"date-parts":[[2022,7,29]]}},"container-title":["Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2022,7,18]],"date-time":"2022-07-18T07:10:32Z","timestamp":1658128232000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2022\/588"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2022\/588","relation":{},"subject":[],"published":{"date-parts":[[2022,7]]}}}