{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T20:16:45Z","timestamp":1773951405675,"version":"3.50.1"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"name":"Indo-French Centre for the Promotion of Advanced Research (IFCPAR\/CEFIPRA) through CSRP","award":["Project No. 6702-2"],"award-info":[{"award-number":["Project No. 6702-2"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2026,4,30]]},"abstract":"<jats:p>\n                    Data-to-Text Generation (D2T) aims to transform semi-structured data (such as tables and graphs) into natural language text. With the exceptional capability of Large Language Models (LLMs), they have become ubiquitous as foundational models for D2T. This article presents a comprehensive evaluation of LLMs for D2T, focusing on three key qualities:\n                    <jats:italic toggle=\"yes\">readability<\/jats:italic>\n                    (fluency and coherence),\n                    <jats:italic toggle=\"yes\">informativeness<\/jats:italic>\n                    (content preservation), and\n                    <jats:italic toggle=\"yes\">faithfulness<\/jats:italic>\n                    (factual accuracy). We evaluate 12 LLMs from five prominent open source families (BART, T5, BLOOM, OPT, and Llama 2) across five widely used D2T datasets using six established automatic metrics, complemented by human evaluation for deeper insight. Our findings reveal that larger model sizes generally improve readability and informativeness, with Llama 2 showing superior overall performance. However, increased model size does not consistently enhance faithfulness and may sometimes degrade it. Human evaluations indicate that larger models are generally preferred for their readability, informativeness, and faithfulness from the human readers\u2019 perspective, as their minor faithfulness errors are assessed more selectively by automatic evaluation metrics. Through robustness analyses, we confirm that these trends remain stable across different fine-tuning (QLoRA vs. Prefix-Tuning) and decoding (Beam Search vs. Nucleus Sampling) strategies. Furthermore, our experiments show that performance consistently declines as source-reference divergence increases, regardless of model size. To mitigate this, we propose a\n                    <jats:italic toggle=\"yes\">source-reference divergence-weighted training<\/jats:italic>\n                    that adaptively reweights training instances based on their source-reference divergence, achieving consistent improvements across all three key evaluation qualities. This comprehensive study provides practical insights into LLM behavior in D2T and introduces an effective training paradigm for improving performance in D2T.\n                  <\/jats:p>","DOI":"10.1145\/3795137","type":"journal-article","created":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T14:38:20Z","timestamp":1769783900000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["A Comprehensive Performance Evaluation of LLMs for Data-to-Text Generation and Divergence-Weighted Training"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-8301-293X","authenticated-orcid":false,"given":"Joy","family":"Mahapatra","sequence":"first","affiliation":[{"name":"Indian Statistical Institute, Kolkata, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7207-5018","authenticated-orcid":false,"given":"Utpal","family":"Garain","sequence":"additional","affiliation":[{"name":"Indian Statistical Institute, Kolkata, India"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,3,19]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"65","volume-title":"Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization","author":"Banerjee Satanjeev","year":"2005","unstructured":"Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization, 65\u201372. Retrieved from https:\/\/aclanthology.org\/W05-0909\/"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11944"},{"key":"e_1_3_1_4_2","first-page":"643","volume-title":"Proceedings of the ACL","author":"Chaganty Arun Tejasvi","year":"2018","unstructured":"Arun Tejasvi Chaganty, Stephen Mussmann, and Percy Liang. 2018. The price of debiasing automatic metrics in natural language evaluation. In Proceedings of the ACL, 643\u2013653. Retrieved from https:\/\/aclanthology.org\/P18-1060\/"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.52202\/075280-0441"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/p19-1483"},{"key":"e_1_3_1_7_2","first-page":"1383","volume-title":"Proceedings of the ACL","author":"Dror Rotem","year":"2018","unstructured":"Rotem Dror, Gili Baumer, Segev Shlomov, and Roi Reichart. 2018. The hitchhiker\u2019s guide to testing statistical significance in natural language processing. In Proceedings of the ACL, 1383\u20131392. Retrieved from https:\/\/aclanthology.org\/P18-1128\/"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/w18-6539"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.12918"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/w17-3518"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.5477"},{"key":"e_1_3_1_12_2","volume-title":"Proceedings of the ICLR","author":"Holtzman Ari","year":"2020","unstructured":"Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In Proceedings of the ICLR. Retrieved from https:\/\/openreview.net\/forum?id=rygGQyrFvH"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-8656"},{"key":"e_1_3_1_14_2","unstructured":"Lei Huang Weijiang Yu Weitao Ma Weihong Zhong Zhangyin Feng Haotian Wang Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin and Ting Liu. 2023. A survey on hallucination in large language models: Principles taxonomy challenges and open questions. arXiv:2311.05232. Retrieved from https:\/\/arxiv.org\/abs\/2311.05232"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571730"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3603374"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-8623"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-demo.42"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.419"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.750"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.397"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00641"},{"key":"e_1_3_1_24_2","unstructured":"Wei Li Wenhao Wu Moye Chen Jiachen Liu Xinyan Xiao and Hua Wu. 2022. Faithfulness in natural language generation: A systematic survey of analysis evaluation and optimization methods. arXiv:2203.05227. Retrieved from https:\/\/arxiv.org\/abs\/2203.05227"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.353"},{"key":"e_1_3_1_26_2","unstructured":"Percy Liang Rishi Bommasani Tony Lee Dimitris Tsipras Dilara Soylu Michihiro Yasunaga Yian Zhang Deepak Narayanan Yuhuai Wu Ananya Kumar et al. 2022. Holistic evaluation of language models. arXiv:2211.09110. Retrieved from https:\/\/arxiv.org\/abs\/2211.09110"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.229"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2023.3304385"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-eacl.98"},{"key":"e_1_3_1_30_2","volume-title":"Proceedings of the ICLR","author":"Loshchilov Ilya","year":"2019","unstructured":"Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In Proceedings of the ICLR. Retrieved from https:\/\/openreview.net\/forum?id=Bkg6RiCqY7"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3616901.3616995"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i17.29839"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.173"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2010-343"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.37"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/w17-5525"},{"key":"e_1_3_1_37_2","first-page":"311","volume-title":"Proceedings of the ACL","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the ACL, 311\u2013318. Retrieved from https:\/\/aclanthology.org\/P02-1040\/"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/w18-6319"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00381"},{"key":"e_1_3_1_40_2","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21, Article 140 (2020), 1\u201367. Retrieved from http:\/\/jmlr.org\/papers\/v21\/20-074.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00322"},{"key":"e_1_3_1_42_2","unstructured":"Teven Le Scao Angela Fan Christopher Akiki Ellie Pavlick Suzana Ilic Daniel Hesslow Roman Castagn\u00e9 Alexandra Sasha Luccioni Fran\u00e7ois Yvon Matthias Gall\u00e9 et al. 2022. BLOOM: A 176b-parameter open-access multilingual language model. arXiv:2211.05100. Retrieved from https:\/\/arxiv.org\/abs\/2211.05100"},{"key":"e_1_3_1_43_2","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288. Retrieved from https:\/\/arxiv.org\/abs\/2307.09288"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.inlg-main.30"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.101"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"e_1_3_1_47_2","first-page":"27263","volume-title":"Proceedings of the NeurIPS","author":"Yuan Weizhe","year":"2021","unstructured":"Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. Bartscore: Evaluating generated text as text generation. In Proceedings of the NeurIPS, 27263\u201327277. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2021\/hash\/e4d2b6e6fdeca3e60e0f1a62fee3d9dd-Abstract.html"},{"key":"e_1_3_1_48_2","unstructured":"Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen Shuohui Chen Christopher Dewan Mona T. Diab Xian Li Xi Victoria Lin et al. 2022. OPT: Open pre-trained transformer language models. arXiv:2205.01068. Retrieved from https:\/\/arxiv.org\/abs\/2205.01068"},{"key":"e_1_3_1_49_2","volume-title":"Proceedings of the ICLR","author":"Zhang Tianyi","year":"2020","unstructured":"Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with BERT. In Proceedings of the ICLR. Retrieved from https:\/\/openreview.net\/forum?id=SkeHuCVFDr"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1053"},{"key":"e_1_3_1_51_2","volume-title":"Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023 (NeurIPS \u201923)","author":"Zheng Lianmin","year":"2023","unstructured":"Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, et al. 2023. Judging LLM-as-a-judge with MT-bench and Chatbot Arena. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023 (NeurIPS \u201923). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/91f18a1287b398d378ef22505bf41832-Abstract-Datasets_and_Benchmarks.html"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3795137","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T16:30:47Z","timestamp":1773937847000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3795137"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,19]]},"references-count":50,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,4,30]]}},"alternative-id":["10.1145\/3795137"],"URL":"https:\/\/doi.org\/10.1145\/3795137","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,19]]},"assertion":[{"value":"2024-11-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}