{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T18:17:31Z","timestamp":1778091451309,"version":"3.51.4"},"reference-count":52,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T00:00:00Z","timestamp":1778025600000},"content-version":"vor","delay-in-days":125,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,5,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Evaluating natural language generation systems is challenging due to the diversity of valid outputs. While human evaluation is the gold standard, it suffers from inconsistencies, lack of standardization, and demographic biases, limiting reproducibility. LLM-based evaluators offer a scalable alternative but are highly sensitive to prompt design, where small variations can lead to significant discrepancies. In this work, we propose an inversion learning method that learns effective reverse mappings from model outputs back to their input instructions, enabling the automatic generation of highly effective, model-specific evaluation prompts. Our method requires only a single evaluation sample and eliminates the need for time-consuming manual prompt engineering, thereby improving both efficiency and robustness. Our work contributes toward a new direction for more robust and efficient LLM-based evaluation.<\/jats:p>","DOI":"10.1162\/tacl.a.617","type":"journal-article","created":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T18:01:04Z","timestamp":1778090464000},"page":"689-710","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":0,"title":["<i>Beyond One-Size-Fits-All<\/i>\n                    : Inversion Learning for Highly Effective NLG Evaluation Prompts"],"prefix":"10.1162","volume":"14","author":[{"given":"Hanhua","family":"Hong","sequence":"first","affiliation":[{"name":"The University of Manchester, UK. hanhua.hong@postgrad.manchester.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chenghao","family":"Xiao","sequence":"additional","affiliation":[{"name":"Durham University, UK. chenghao.xiao@durham.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yang","family":"Wang","sequence":"additional","affiliation":[{"name":"The University of Manchester, UK. yang.yang.wang-27@postgrad.manchester.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yiqi","family":"Liu","sequence":"additional","affiliation":[{"name":"The University of Manchester, UK. yiqi.liu@manchester.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenge","family":"Rong","sequence":"additional","affiliation":[{"name":"Beihang University, China. w.rong@buaa.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chenghua","family":"Lin","sequence":"additional","affiliation":[{"name":"The University of Manchester, UK. chenghua.lin@manchester.ac.uk"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2026,5,4]]},"reference":[{"key":"2026050614005377400_bib1","first-page":"337","article-title":"Using large language models to simulate multiple humans and replicate human subject studies","volume-title":"International Conference on Machine Learning","author":"Aher","year":"2023"},{"key":"2026050614005377400_bib2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18653\/v1\/2024.inlg-tutorials.1","article-title":"The INLG 2024 tutorial on human evaluation of NLP system quality: Background, overall aims, and summaries of taught units","volume-title":"Proceedings of the 17th International Natural Language Generation Conference: Tutorial Abstract","author":"Belz","year":"2024"},{"key":"2026050614005377400_bib3","doi-asserted-by":"crossref","DOI":"10.52202\/079017-2205","article-title":"On the worst prompt performance of large language models","author":"Cao","year":"2024"},{"key":"2026050614005377400_bib4","first-page":"2633","article-title":"Extracting training data from large language models","volume-title":"30th USENIX Security Symposium (USENIX Security 21)","author":"Carlini","year":"2021"},{"key":"2026050614005377400_bib5","doi-asserted-by":"publisher","first-page":"1137","DOI":"10.18653\/v1\/2024.acl-long.63","article-title":"ConSiDERS-the-human evaluation framework: Rethinking human evaluation for generative large language models","volume-title":"Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Elangovan","year":"2024"},{"key":"2026050614005377400_bib6","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2025.naacl-long.73","article-title":"What did I do wrong? Quantifying LLMs\u2019 sensitivity and consistency to prompt engineering","author":"Errica","year":"2024","journal-title":"arXiv preprint arXiv:2406.12334"},{"key":"2026050614005377400_bib7","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1162\/tacl_a_00373","article-title":"SummEval: Re-evaluating summarization evaluation","volume":"9","author":"Fabbri","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2026050614005377400_bib8","doi-asserted-by":"publisher","first-page":"46","DOI":"10.18653\/v1\/2022.wmt-1.2","article-title":"Results of WMT22 metrics shared task: Stop using BLEU \u2013 neural metrics are better and more robust","volume-title":"Proceedings of the Seventh Conference on Machine Translation (WMT)","author":"Freitag","year":"2022"},{"key":"2026050614005377400_bib9","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2025.naacl-long.111","article-title":"Analyzing and evaluating correlation measures in NLG meta-evaluation","author":"Gao","year":"2025"},{"key":"2026050614005377400_bib10","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2019-3079","article-title":"Topical-chat: Towards knowledge-grounded open-domain conversations","author":"Gopalakrishnan","year":"2019","journal-title":"ArXiv"},{"key":"2026050614005377400_bib11","article-title":"The Llama 3 herd of models","author":"Grattafiori","year":"2024"},{"key":"2026050614005377400_bib12","article-title":"The curious case of neural text degeneration","author":"Holtzman","year":"2020"},{"key":"2026050614005377400_bib13","doi-asserted-by":"publisher","first-page":"169","DOI":"10.18653\/v1\/2020.inlg-1.23","article-title":"Twenty years of confusion in human evaluation: NLG needs evaluation sheets and standardised definitions","volume-title":"Proceedings of the 13th International Conference on Natural Language Generation","author":"Howcroft","year":"2020"},{"key":"2026050614005377400_bib14","article-title":"LoRA: Low-rank adaptation of large language models","volume-title":"International Conference on Learning Representations","author":"Edward","year":"2022"},{"key":"2026050614005377400_bib15","doi-asserted-by":"publisher","DOI":"10.31234\/osf.io\/munc9","article-title":"Large language models show human behavior","author":"Huijzer","year":"2023"},{"key":"2026050614005377400_bib16","first-page":"193","article-title":"Large language models are state-of-the-art evaluators of translation quality","volume-title":"Proceedings of the 24th Annual Conference of the European Association for Machine Translation","author":"Kocmi","year":"2023"},{"key":"2026050614005377400_bib17","doi-asserted-by":"publisher","first-page":"8397","DOI":"10.18653\/v1\/2025.findings-naacl.469","article-title":"Do LLMs have distinct and consistent personality? TRAIT: Personality testset designed for LLMs with psychometrics","volume-title":"Findings of the Association for Computational Linguistics: NAACL 2025","author":"Lee","year":"2025"},{"key":"2026050614005377400_bib18","doi-asserted-by":"publisher","first-page":"11481","DOI":"10.18653\/v1\/2024.emnlp-main.641","article-title":"PrExMe! Large scale prompt exploration of open source LLMs for machine translation and summarization evaluation","volume-title":"Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing","author":"Leiter","year":"2024"},{"key":"2026050614005377400_bib19","article-title":"LLMs-as-judges: A comprehensive survey on LLM-based evaluation methods","author":"Li","year":"2024"},{"key":"2026050614005377400_bib20","article-title":"Infinity instruct: Scaling instruction selection and synthesis to enhance language models","author":"Li","year":"2025"},{"key":"2026050614005377400_bib21","doi-asserted-by":"publisher","first-page":"2511","DOI":"10.18653\/v1\/2023.emnlp-main.153","article-title":"G-eval: NLG evaluation using GPT-4 with better human alignment","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Liu","year":"2023"},{"key":"2026050614005377400_bib22","doi-asserted-by":"publisher","first-page":"12688","DOI":"10.18653\/v1\/2024.findings-acl.753","article-title":"LLMs as narcissistic evaluators: When ego inflates evaluation scores","volume-title":"Findings of the Association for Computational Linguistics: ACL 2024","author":"Liu","year":"2024"},{"key":"2026050614005377400_bib23","doi-asserted-by":"publisher","first-page":"6676","DOI":"10.18653\/v1\/2023.findings-emnlp.444","article-title":"The iron(ic) melting pot: Reviewing human evaluation in humour, irony and sarcasm generation","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2023","author":"Loakman","year":"2023"},{"key":"2026050614005377400_bib24","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.765","article-title":"Text embeddings reveal (almost) as much as text","author":"Morris","year":"2023","journal-title":"arXiv preprint arXiv:2310.06816"},{"key":"2026050614005377400_bib25","article-title":"Language model inversion","author":"Morris","year":"2023"},{"issue":"10","key":"2026050614005377400_bib26","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1007\/s10462-024-10903-2","article-title":"Contrasting linguistic patterns in human and LLM-generated news text","volume":"57","author":"Mu\u00f1oz-Ortiz","year":"2024","journal-title":"Artificial Intelligence Review"},{"key":"2026050614005377400_bib27","article-title":"Dager: Exact gradient inversion for large language models","author":"Petrov","year":"2024"},{"key":"2026050614005377400_bib28","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2405.17202","article-title":"Efficient multi-prompt evaluation of LLMs","author":"Polo","year":"2024","journal-title":"ArXiv"},{"key":"2026050614005377400_bib29","doi-asserted-by":"publisher","first-page":"7957","DOI":"10.18653\/v1\/2023.emnlp-main.494","article-title":"Automatic prompt optimization with \u201cgradient descent\u201d and beam search","volume-title":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","author":"Pryzant","year":"2023"},{"key":"2026050614005377400_bib30","doi-asserted-by":"publisher","first-page":"3660","DOI":"10.18653\/v1\/2024.emnlp-main.214","article-title":"What do large language models need for machine translation evaluation?","volume-title":"Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing","author":"Qian","year":"2024"},{"key":"2026050614005377400_bib31","article-title":"Qwen2.5: A party of foundation models","author":"Team","year":"2024"},{"key":"2026050614005377400_bib32","article-title":"Direct preference optimization: Your language model is secretly a reward model","volume-title":"Thirty-seventh Conference on Neural Information Processing Systems","author":"Rafailov","year":"2023"},{"key":"2026050614005377400_bib33","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2025.emnlp-main.1681","article-title":"A systematic survey of automatic prompt optimization techniques","author":"Ramnath","year":"2025","journal-title":"arXiv preprint arXiv:2502.16923"},{"key":"2026050614005377400_bib34","article-title":"Quantifying language models\u2019 sensitivity to spurious features in prompt design or: How I learned to start worrying about prompt formatting","author":"Sclar","year":"2024"},{"key":"2026050614005377400_bib35","doi-asserted-by":"publisher","first-page":"4222","DOI":"10.18653\/v1\/2020.emnlp-main.346","article-title":"AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Shin","year":"2020"},{"key":"2026050614005377400_bib36","doi-asserted-by":"publisher","DOI":"10.1145\/3372297.3417270","article-title":"Information leakage in embedding models","author":"Song","year":"2020"},{"key":"2026050614005377400_bib37","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.07415","article-title":"Autohint: Automatic prompt optimization with hint generation","author":"Sun","year":"2023","journal-title":"arXiv preprint arXiv:2307.07415"},{"key":"2026050614005377400_bib38","article-title":"Idiosyncrasies in large language models","volume-title":"Forty-second International Conference on Machine Learning","author":"Sun","year":"2025"},{"key":"2026050614005377400_bib39","doi-asserted-by":"publisher","first-page":"5008","DOI":"10.18653\/v1\/2020.acl-main.450","article-title":"Asking and answering questions to evaluate the factual consistency of summaries","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Wang","year":"2020"},{"key":"2026050614005377400_bib40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18653\/v1\/2023.newsum-1.1","article-title":"Is ChatGPT a good NLG evaluator? A preliminary study","volume-title":"Proceedings of the 4th New Frontiers in Summarization Workshop","author":"Wang","year":"2023"},{"key":"2026050614005377400_bib41","doi-asserted-by":"publisher","first-page":"24974","DOI":"10.18653\/v1\/2025.findings-acl.1282","article-title":"HPSS: Heuristic prompting strategy search for LLM evaluators","volume-title":"Findings of the Association for Computational Linguistics: ACL 2025","author":"Wen","year":"2025"},{"key":"2026050614005377400_bib42","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2502.11573","article-title":"InfiR: Crafting effective small language models and multimodal small language models in reasoning","author":"Xie","year":"2025","journal-title":"arXiv preprint arXiv:2502.11573"},{"key":"2026050614005377400_bib43","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2309.03409","article-title":"Large language models as optimizers","author":"Yang","year":"2023","journal-title":"arXiv preprint arXiv:2309.03409"},{"key":"2026050614005377400_bib44","first-page":"27263","article-title":"BARTScore: Evaluating generated text as text generation","volume-title":"Advances in Neural Information Processing Systems","author":"Yuan","year":"2021"},{"key":"2026050614005377400_bib45","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.emnlp-main.819","article-title":"Extracting prompts by inverting LLM outputs","author":"Zhang","year":"2024"},{"key":"2026050614005377400_bib46","article-title":"BERTScore: Evaluating text generation with BERT","volume-title":"International Conference on Learning Representations","author":"Zhang","year":"2020"},{"key":"2026050614005377400_bib47","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.33","article-title":"Evaluating open-domain dialogues in latent space with next sentence prediction and mutual information","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Zhao","year":"2023"},{"key":"2026050614005377400_bib48","doi-asserted-by":"publisher","DOI":"10.1162\/99608f92.4d0c37ad","article-title":"SLIDE: A framework integrating small and large language models for open-domain dialogues evaluation","volume-title":"Findings of the Association for Computational Linguistics: ACL 2024","author":"Zhao","year":"2024"},{"key":"2026050614005377400_bib49","article-title":"A survey of large language models","author":"Zhao","year":"2025"},{"key":"2026050614005377400_bib50","article-title":"Judging LLM-as-a-judge with MT-bench and chatbot arena","volume-title":"Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track","author":"Zheng","year":"2023"},{"key":"2026050614005377400_bib51","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-demos.38","article-title":"Llamafactory: Unified efficient fine-tuning of 100+ language models","volume-title":"Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)","author":"Zheng","year":"2024"},{"key":"2026050614005377400_bib52","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.131","article-title":"Towards a unified multi-dimensional evaluator for text generation","author":"Zhong","year":"2022"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/TACL.a.617\/2598657\/tacl.a.617.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/TACL.a.617\/2598657\/tacl.a.617.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T18:01:08Z","timestamp":1778090468000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/TACL.a.617\/136551\/Beyond-One-Size-Fits-All-Inversion-Learning-for"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":52,"URL":"https:\/\/doi.org\/10.1162\/tacl.a.617","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026]]},"published":{"date-parts":[[2026]]}}}