{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T04:08:23Z","timestamp":1783656503743,"version":"3.55.0"},"reference-count":160,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T00:00:00Z","timestamp":1703030400000},"content-version":"vor","delay-in-days":353,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,12,19]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Natural language generation has witnessed significant advancements due to the training of large language models on vast internet-scale datasets. Despite these advancements, there exists a critical challenge: These models can inadvertently generate content that is toxic, inaccurate, and unhelpful, and existing automatic evaluation metrics often fall short of identifying these shortcomings. As models become more capable, human feedback is an invaluable signal for evaluating and improving models. This survey aims to provide an overview of recent research that has leveraged human feedback to improve natural language generation. First, we introduce a taxonomy distilled from existing research to categorize and organize the varied forms of feedback. Next, we discuss how feedback can be described by its format and objective, and cover the two approaches proposed to use feedback (either for training or decoding): directly using feedback or training feedback models. We also discuss existing datasets for human-feedback data collection, and concerns surrounding feedback collection. Finally, we provide an overview of the nascent field of AI feedback, which uses large language models to make judgments based on a set of principles and minimize the need for human intervention. We also release a website of this survey at feedback-gap-survey.info.<\/jats:p>","DOI":"10.1162\/tacl_a_00626","type":"journal-article","created":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T18:28:29Z","timestamp":1703096909000},"page":"1643-1668","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":30,"title":["Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation"],"prefix":"10.1162","volume":"11","author":[{"given":"Patrick","family":"Fernandes","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, USA. pfernand@cs.cmu.edu"},{"name":"Instituto Superior T\u00e9cnico (Lisbon ELLIS Unit), Portugal"},{"name":"Instituto de Telecomunica\u00e7\u00f5es, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aman","family":"Madaan","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emmy","family":"Liu","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ant\u00f3nio","family":"Farinhas","sequence":"additional","affiliation":[{"name":"Instituto Superior T\u00e9cnico (Lisbon ELLIS Unit), Portugal"},{"name":"Instituto de Telecomunica\u00e7\u00f5es, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pedro Henrique","family":"Martins","sequence":"additional","affiliation":[{"name":"Unbabel, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amanda","family":"Bertsch","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jos\u00e9 G. C.","family":"de Souza","sequence":"additional","affiliation":[{"name":"Unbabel, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuyan","family":"Zhou","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tongshuang","family":"Wu","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Graham","family":"Neubig","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, USA"},{"name":"Inspired Cognition, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andr\u00e9 F. T.","family":"Martins","sequence":"additional","affiliation":[{"name":"Instituto Superior T\u00e9cnico (Lisbon ELLIS Unit), Portugal"},{"name":"Instituto de Telecomunica\u00e7\u00f5es, Portugal"},{"name":"Unbabel, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2023,12,19]]},"reference":[{"key":"2023122018281809400_bib1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.427","article-title":"Rl4f: Generating natural language feedback with reinforcement learning for repairing model outputs","author":"Aky\u00fcrek","year":"2023"},{"issue":"4","key":"2023122018281809400_bib2","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1609\/aimag.v35i4.2513","article-title":"Power to the people: The role of humans in interactive machine learning","volume":"35","author":"Amershi","year":"2014","journal-title":"AI Magazine"},{"key":"2023122018281809400_bib3","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1606.06565","article-title":"Concrete problems in AI safety","author":"Amodei","year":"2016","journal-title":"CoRR"},{"key":"2023122018281809400_bib4","first-page":"1125","article-title":"Identifying weaknesses in machine translation metrics through minimum Bayes risk decoding: A case study for COMET","volume-title":"Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Amrhein","year":"2022"},{"key":"2023122018281809400_bib5","article-title":"Director: Generator-classifiers for supervised language modeling","volume-title":"Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing","author":"Arora","year":"2022"},{"key":"2023122018281809400_bib6","article-title":"A general language assistant as a laboratory for alignment","author":"Askell","year":"2021","journal-title":"arXiv preprint arXiv:2112.00861"},{"key":"2023122018281809400_bib7","volume-title":"Feedback systems: An introduction for scientists and engineers","author":"\u00c5str\u00f6m","year":"2021"},{"key":"2023122018281809400_bib8","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2204.05862","article-title":"Training a helpful and harmless assistant with reinforcement learning from human feedback","author":"Bai","year":"2022"},{"key":"2023122018281809400_bib9","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2212.08073","article-title":"Constitutional AI: Harmlessness from AI feedback","author":"Bai","year":"2022"},{"key":"2023122018281809400_bib10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3411764.3445717","article-title":"Does the whole exceed its parts? The effect of AI explanations on complementary team performance","volume-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems","author":"Bansal","year":"2021"},{"key":"2023122018281809400_bib11","doi-asserted-by":"publisher","first-page":"610","DOI":"10.1145\/3442188.3445922","article-title":"On the dangers of stochastic parrots: Can language models be too big?","volume-title":"Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency","author":"Bender","year":"2021"},{"key":"2023122018281809400_bib12","first-page":"109","article-title":"Findings of the WMT 2022 shared task on automatic post-editing","volume-title":"Proceedings of the Seventh Conference on Machine Translation (WMT)","author":"Bhattacharyya","year":"2022"},{"key":"2023122018281809400_bib13","doi-asserted-by":"publisher","first-page":"3110","DOI":"10.18653\/v1\/D19-1307","article-title":"Better rewards yield better summaries: Learning to summarise without references","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"B\u00f6hm","year":"2019"},{"key":"2023122018281809400_bib14","article-title":"Man is to computer programmer as woman is to homemaker? Debiasing word embeddings","volume":"29","author":"Bolukbasi","year":"2016","journal-title":"Advances in neural information processing systems"},{"key":"2023122018281809400_bib15","article-title":"On the opportunities and risks of foundation models","author":"Bommasani","year":"2021","journal-title":"ArXiv"},{"key":"2023122018281809400_bib16","volume-title":"Superintelligence: Paths, Dangers, Strategies","author":"Bostrom","year":"2014","edition":"1st"},{"key":"2023122018281809400_bib17","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2211.03540","article-title":"Measuring progress on scalable oversight for large language models","author":"Bowman","year":"2022"},{"key":"2023122018281809400_bib18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.15217","article-title":"Open problems and fundamental limitations of reinforcement learning from human feedback","author":"Casper","year":"2023"},{"key":"2023122018281809400_bib19","doi-asserted-by":"publisher","first-page":"643","DOI":"10.18653\/v1\/P18-1060","article-title":"The price of debiasing automatic metrics in natural language evaluation","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Chaganty","year":"2018"},{"key":"2023122018281809400_bib20","article-title":"Teaching large language models to self-debug","author":"Chen","year":"2023","journal-title":"arXiv preprint arXiv:2304.05128"},{"key":"2023122018281809400_bib21","doi-asserted-by":"publisher","first-page":"7282","DOI":"10.18653\/v1\/2021.acl-long.565","article-title":"All that\u2019s \u2018human\u2019 is not gold: Evaluating human evaluation of generated text","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Clark","year":"2021"},{"key":"2023122018281809400_bib22","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales","volume":"20","author":"Cohen","year":"1960","journal-title":"Educational and Psychological Measurement"},{"key":"2023122018281809400_bib23","doi-asserted-by":"publisher","first-page":"3050","DOI":"10.18653\/v1\/P19-1292","article-title":"A simple and effective approach to automatic post-editing with transfer learning","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Correia","year":"2019"},{"key":"2023122018281809400_bib24","doi-asserted-by":"publisher","first-page":"395","DOI":"10.3115\/v1\/E14-1042","article-title":"Learning from post-editing: Online model adaptation for statistical machine translation","volume-title":"Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics","author":"Denkowski","year":"2014"},{"issue":"CSCW2","key":"2023122018281809400_bib25","doi-asserted-by":"publisher","DOI":"10.1145\/3555632","article-title":"Impact of annotator demographics on sentiment dataset labeling","volume":"6","author":"Yi","year":"2022","journal-title":"Proceedings of the ACM on Human Computer Interaction"},{"key":"2023122018281809400_bib26","doi-asserted-by":"publisher","first-page":"1286","DOI":"10.18653\/v1\/2021.emnlp-main.98","article-title":"Documenting large webtext corpora: A case study on the colossal clean crawled corpus","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Dodge","year":"2021"},{"key":"2023122018281809400_bib27","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2207.08583","article-title":"Mad for robust reinforcement learning in machine translation","author":"Donato","year":"2022"},{"key":"2023122018281809400_bib28","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.444","article-title":"Nl-edit: Correcting semantic parse errors through natural language interaction","author":"Elgohary","year":"2021","journal-title":"arXiv preprint arXiv: 2103.14540"},{"key":"2023122018281809400_bib29","article-title":"Stanford human preferences dataset","author":"Ethayarajh","year":"2023"},{"key":"2023122018281809400_bib30","doi-asserted-by":"publisher","first-page":"1396","DOI":"10.18653\/v1\/2022.naacl-main.100","article-title":"Quality-aware decoding for neural machine translation","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Fernandes","year":"2022"},{"key":"2023122018281809400_bib31","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1037\/h0031619","article-title":"Measuring nominal scale agreement among many raters","volume":"76","author":"Fleiss","year":"1971","journal-title":"Psychological Bulletin"},{"key":"2023122018281809400_bib32","doi-asserted-by":"publisher","first-page":"1460","DOI":"10.1162\/tacl_a_00437","article-title":"Experts, errors, and context: A large-scale study of human evaluation for machine translation","volume":"9","author":"Freitag","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023122018281809400_bib33","doi-asserted-by":"publisher","first-page":"811","DOI":"10.1162\/tacl_a_00491","article-title":"High quality rather than high model probability: Minimum Bayes risk decoding with neural metrics","volume":"10","author":"Freitag","year":"2022","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023122018281809400_bib34","first-page":"46","article-title":"Results of WMT22 metrics shared task: Stop using BLEU \u2013 neural metrics are better and more robust","volume-title":"Proceedings of the Seventh Conference on Machine Translation (WMT)","author":"Freitag","year":"2022"},{"key":"2023122018281809400_bib35","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2210.1076","article-title":"Scaling laws for reward model overoptimization","author":"Gao","year":"2022"},{"key":"2023122018281809400_bib36","doi-asserted-by":"publisher","first-page":"4120","DOI":"10.18653\/v1\/D18-1445","article-title":"APRIL: Interactively learning to summarise by combining active preference learning and reinforcement learning","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Gao","year":"2018"},{"key":"2023122018281809400_bib37","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-demos.27","article-title":"Gemv2: Multilingual nlg benchmarking in a single line of code","volume-title":"Conference on Empirical Methods in Natural Language Processing","author":"Gehrmann","year":"2022"},{"key":"2023122018281809400_bib38","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1613\/jair.1.13715","article-title":"Repairing the cracked foundation: A survey of obstacles in evaluation practices for generated text","volume":"77","author":"Gehrmann","year":"2023","journal-title":"Journal of Artificial Intelligence Research"},{"issue":"CSCW3","key":"2023122018281809400_bib39","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3432934","article-title":"Explainable active learning (xal) toward AI explanations as interfaces for machine teachers","volume":"4","author":"Ghai","year":"2021","journal-title":"Proceedings of the ACM on Human-Computer Interaction"},{"key":"2023122018281809400_bib40","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i5.25740","article-title":"The effect of modeling human rationality level on learning rewards from multiple feedback types","author":"Ghosal","year":"2023"},{"key":"2023122018281809400_bib41","first-page":"148","article-title":"Non-expert evaluation of summarization systems is risky","volume-title":"Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon\u2019s Mechanical Turk","author":"Gillick","year":"2010"},{"key":"2023122018281809400_bib42","article-title":"Improving alignment of dialogue agents via targeted human judgements","author":"Glaese","year":"2022","journal-title":"arXiv preprint arXiv:2209.14375"},{"key":"2023122018281809400_bib43","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-emnlp.330","article-title":"Uncertainty-aware machine translation evaluation","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021","author":"Glushkova","year":"2021"},{"key":"2023122018281809400_bib44","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-349-17295-5_4","volume-title":"Problems of Monetary Management: The UK Experience","author":"Goodhart","year":"1984"},{"key":"2023122018281809400_bib45","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3502004","article-title":"Jury learning: Integrating dissenting voices into machine learning models","volume-title":"CHI Conference on Human Factors in Computing Systems","author":"Gordon","year":"2022"},{"key":"2023122018281809400_bib46","first-page":"33","article-title":"Continuous measurement scales in human evaluation of machine translation","volume-title":"Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse","author":"Graham","year":"2013"},{"key":"2023122018281809400_bib47","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.06074","article-title":"Susceptibility to influence of large language models","author":"Griffin","year":"2023"},{"key":"2023122018281809400_bib48","doi-asserted-by":"publisher","first-page":"3667","DOI":"10.18653\/v1\/P19-1358","article-title":"Learning from dialogue after deployment: Feed yourself, chatbot!","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Hancock","year":"2019"},{"key":"2023122018281809400_bib49","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.4316084","article-title":"The political ideology of conversational AI: Converging evidence on chatgpt\u2019s pro-environmental, left-libertarian orientation","author":"Hartmann","year":"2023"},{"issue":"1","key":"2023122018281809400_bib50","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/S0022-5371(77)80012-1","article-title":"Frequency and the conference of referential validity","volume":"16","author":"Hasher","year":"1977","journal-title":"Journal of Verbal Learning and Verbal Behavior"},{"key":"2023122018281809400_bib51","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1080\/19312450709336664","article-title":"Answering the call for a standard reliability measure for coding data","volume":"1","author":"Hayes","year":"2007","journal-title":"Communication Methods and Measures"},{"key":"2023122018281809400_bib52","article-title":"Aligning AI with shared human values","author":"Hendrycks","year":"2020","journal-title":"arXiv preprint arXiv:2008.02275"},{"issue":"5","key":"2023122018281809400_bib53","doi-asserted-by":"publisher","first-page":"553","DOI":"10.1007\/s12599-022-00769-5","article-title":"Hourly wages in crowdworking: A meta-analysis","volume":"64","author":"Hornuf","year":"2022","journal-title":"Business & Information Systems Engineering"},{"key":"2023122018281809400_bib54","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581196","article-title":"Co-writing with opinionated language models affects users\u2019 views","author":"Jakesch","year":"2023","journal-title":"ArXiv"},{"key":"2023122018281809400_bib55","article-title":"Way off-policy batch deep reinforcement learning of implicit human preferences in dialog","author":"Jaques","year":"2019","journal-title":"CoRR"},{"key":"2023122018281809400_bib56","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2110.07574","article-title":"Can machines learn morality? The delphi experiment","author":"Jiang","year":"2022"},{"key":"2023122018281809400_bib57","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2103.14659","article-title":"Alignment of language agents","author":"Kenton","year":"2021","journal-title":"CoRR"},{"key":"2023122018281809400_bib58","doi-asserted-by":"publisher","first-page":"191","DOI":"10.1145\/2449396.2449422","article-title":"Learning non-myopically from human-generated reward","volume-title":"Proceedings of the 2013 International Conference on Intelligent User Interfaces","author":"Bradley Knox","year":"2013"},{"key":"2023122018281809400_bib59","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2302.08582","article-title":"Pretraining language models with human preferences","author":"Korbak","year":"2023"},{"key":"2023122018281809400_bib60","doi-asserted-by":"publisher","first-page":"92","DOI":"10.18653\/v1\/N18-3012","article-title":"Can neural machine translation be improved with user feedback?","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers)","author":"Kreutzer","year":"2018"},{"key":"2023122018281809400_bib61","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1609\/hcomp.v2i1.13166","article-title":"Wish: Amplifying creative ability with expert crowds","volume-title":"Proceedings of the AAAI Conference on Human Computation and Crowdsourcing","author":"Kulkarni","year":"2014"},{"key":"2023122018281809400_bib62","doi-asserted-by":"publisher","first-page":"140","DOI":"10.3115\/1118693.1118712","article-title":"Minimum bayes-risk word alignments of bilingual texts","volume-title":"Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - Volume 10","author":"Kumar","year":"2002"},{"key":"2023122018281809400_bib63","article-title":"HuggingFace h4 stack exchange preference dataset","author":"Lambert","year":"2023"},{"key":"2023122018281809400_bib64","doi-asserted-by":"publisher","DOI":"10.1145\/3543758.3543947","article-title":"Suggestion lists vs. continuous generation: Interaction design for writing with generative models on mobile devices affect text length, wording and perceived authorship","author":"Lehmann","year":"2022","journal-title":"Proceedings of Mensch und Computer 2022"},{"key":"2023122018281809400_bib65","article-title":"Scalable agent alignment via reward modeling: A research direction","author":"Leike","year":"2018","journal-title":"ArXiv"},{"key":"2023122018281809400_bib66","doi-asserted-by":"publisher","first-page":"1508","DOI":"10.1162\/tacl_a_00440","article-title":"Explanation-based human debugging of nlp models: A survey","volume":"9","author":"Lertvittayakumjorn","year":"2021","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2023122018281809400_bib67","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1611.09823","article-title":"Dialogue learning with human-in-the-loop","volume-title":"International Conference on Learning Representations","author":"Li","year":"2017"},{"key":"2023122018281809400_bib68","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.findings-acl.75","article-title":"Using interactive feedback to improve the accuracy and explainability of question answering systems post-deployment","volume-title":"Findings of the Association for Computational Linguistics: ACL 2022","author":"Li","year":"2022"},{"key":"2023122018281809400_bib69","article-title":"Human-centered explainable AI (xAI): From algorithms to user experiences","author":"Vera Liao","year":"2021","journal-title":"arXiv preprint arXiv:2110.10790"},{"key":"2023122018281809400_bib70","doi-asserted-by":"publisher","first-page":"2060","DOI":"10.18653\/v1\/N18-1187","article-title":"Dialogue learning with human teaching and feedback in end-to-end trainable task-oriented dialogue systems","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Liu","year":"2018"},{"key":"2023122018281809400_bib71","article-title":"Languages are rewards: Hindsight finetuning using human feedback","author":"Liu","year":"2023","journal-title":"arXiv preprint arXiv:2302.02676"},{"key":"2023122018281809400_bib72","doi-asserted-by":"publisher","first-page":"2890","DOI":"10.18653\/v1\/2022.acl-long.207","article-title":"BRIO: Bringing order to abstractive summarization","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Liu","year":"2022"},{"key":"2023122018281809400_bib73","article-title":"Multidimensional quality metrics (MQM): A framework for declaring and describing translation quality metrics","author":"Lommel","year":"2014","journal-title":"Revista Tradum\u00e0tica: Tecnologies de la traducci\u00f3"},{"key":"2023122018281809400_bib74","doi-asserted-by":"publisher","first-page":"2833","DOI":"10.18653\/v1\/2022.emnlp-main.183","article-title":"Memory-assisted prompt editing to improve GPT-3 after deployment","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Madaan","year":"2022"},{"key":"2023122018281809400_bib75","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.17651","article-title":"Self-refine: Iterative refinement with self-feedback","author":"Madaan","year":"2023"},{"key":"2023122018281809400_bib76","doi-asserted-by":"publisher","first-page":"6291","DOI":"10.18653\/v1\/2021.emnlp-main.508","article-title":"Think about it! Improving defeasible reasoning by first modeling the question scenario.","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Madaan","year":"2021"},{"key":"2023122018281809400_bib77","doi-asserted-by":"publisher","first-page":"4984","DOI":"10.18653\/v1\/2020.acl-main.448","article-title":"Tangled up in BLEU: Reevaluating the evaluation of automatic machine translation evaluation metrics","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Mathur","year":"2020"},{"key":"2023122018281809400_bib78","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1195","article-title":"Improving natural language interaction with robots using advice","author":"Mehta","year":"2019","journal-title":"arXiv preprint arXiv:1905.04655"},{"key":"2023122018281809400_bib79","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1145\/3386527.3405940","article-title":"Towards crowdsourcing the identification of knowledge components","volume-title":"Proceedings of the Seventh ACM Conference on Learning@ Scale","author":"Moore","year":"2020"},{"key":"2023122018281809400_bib80","article-title":"The alignment problem from a deep learning perspective","author":"Ngo","year":"2022","journal-title":"arXiv preprint arXiv:2209.00626"},{"key":"2023122018281809400_bib81","doi-asserted-by":"publisher","first-page":"1919","DOI":"10.18653\/v1\/2022.findings-naacl.147","article-title":"Make the most of prior data: A solution for interactive text summarization with preference feedback","volume-title":"Findings of the Association for Computational Linguistics: NAACL 2022","author":"Nguyen","year":"2022"},{"key":"2023122018281809400_bib82","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.734","article-title":"What can we learn from collective human opinions on natural language inference data?","author":"Nie","year":"2020","journal-title":"arXiv preprint arXiv:2010.03532"},{"key":"2023122018281809400_bib83","first-page":"arXiv","article-title":"A conversational paradigm for program synthesis","author":"Nijkamp","year":"2022","journal-title":"arXiv e-prints"},{"key":"2023122018281809400_bib84","unstructured":"OpenAI. 2023a. Gpt-4 technical report. 10.48550\/arXiv.2303.08774"},{"key":"2023122018281809400_bib85","unstructured":"OpenAI. 2023b. Model index for researchers. Accessed: 2023-05-01."},{"key":"2023122018281809400_bib86","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2203.02155","article-title":"Training language models to follow instructions with human feedback","author":"Ouyang","year":"2022"},{"key":"2023122018281809400_bib87","doi-asserted-by":"publisher","first-page":"281","DOI":"10.18653\/v1\/P16-2046","article-title":"A neural network based approach to automatic post-editing","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Pal","year":"2016"},{"key":"2023122018281809400_bib88","doi-asserted-by":"publisher","first-page":"1779","DOI":"10.18653\/v1\/2023.eacl-main.130","article-title":"Don\u2019t blame the annotator: Bias already starts in the annotation instructions","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics","author":"Parmar","year":"2023"},{"key":"2023122018281809400_bib89","first-page":"21","article-title":"Towards human-centered summarization: A case study on financial news","volume-title":"Proceedings of the First Workshop on Bridging Human\u2013Computer Interaction and Natural Language Processing","author":"Passali","year":"2021"},{"key":"2023122018281809400_bib90","article-title":"Refiner: Reasoning feedback on intermediate representations","author":"Paul","year":"2023","journal-title":"arXiv preprint arXiv:2304.01904"},{"key":"2023122018281809400_bib91","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1705.04304","article-title":"A deep reinforced model for abstractive summarization","author":"Paulus","year":"2017","journal-title":"CoRR"},{"key":"2023122018281809400_bib92","article-title":"Check your facts and try again: Improving large language models with external knowledge and automated feedback","author":"Peng","year":"2023","journal-title":"ArXiv"},{"key":"2023122018281809400_bib93","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.225","article-title":"Red teaming language models with language models","author":"Perez","year":"2022"},{"key":"2023122018281809400_bib94","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-acl.847","article-title":"Discovering language model behaviors with model-written evaluations","author":"Perez","year":"2022"},{"key":"2023122018281809400_bib95","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2211.03622","article-title":"Do users write more insecure code with AI assistants?","author":"Perry","year":"2022"},{"key":"2023122018281809400_bib96","doi-asserted-by":"publisher","first-page":"74","DOI":"10.18653\/v1\/W17-4510","article-title":"Learning to score system summaries for better content selection evaluation.","volume-title":"Proceedings of the Workshop on New Frontiers in Summarization","author":"Peyrard","year":"2017"},{"key":"2023122018281809400_bib97","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.731","article-title":"The \u2018problem\u2019 of human label variation: On ground truth in data, modeling and evaluation","author":"Plank","year":"2022"},{"key":"2023122018281809400_bib98","doi-asserted-by":"publisher","first-page":"133","DOI":"10.18653\/v1\/2021.law-1.14","article-title":"On releasing annotator-level labels and information in datasets","volume-title":"Proceedings of the Joint 15th Linguistic Annotation Workshop (LAW) and 3rd Designing Meaning Representations (DMR) Workshop","author":"Prabhakaran","year":"2021"},{"key":"2023122018281809400_bib99","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2212.05726","article-title":"T5score: Discriminative fine-tuning of generative evaluation metrics","author":"Qin","year":"2022"},{"key":"2023122018281809400_bib100","article-title":"Direct preference optimization: Your language model is secretly a reward model","author":"Rafailov","year":"2023"},{"key":"2023122018281809400_bib101","doi-asserted-by":"publisher","first-page":"2685","DOI":"10.18653\/v1\/2020.emnlp-main.213","article-title":"COMET: A neural framework for MT evaluation","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Rei","year":"2020"},{"key":"2023122018281809400_bib102","doi-asserted-by":"publisher","first-page":"911","DOI":"10.18653\/v1\/2020.emnlp-main.213","article-title":"Unbabel\u2019s participation in the WMT(20 metrics shared task)","volume-title":"Proceedings of the Fifth Conference on Machine Translation","author":"Rei","year":"2020"},{"key":"2023122018281809400_bib103","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.findings-emnlp.280","article-title":"Learning to model editing processes","author":"Reid","year":"2022","journal-title":"arXiv preprint arXiv:2205.12374"},{"key":"2023122018281809400_bib104","first-page":"115","article-title":"Failure-driven reminding for incremental learning.","volume-title":"IJCAI","author":"Riesbeck","year":"1981"},{"issue":"1","key":"2023122018281809400_bib105","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1086\/286788","article-title":"Behavior, purpose and teleology","volume":"10","author":"Rosenblueth","year":"1943","journal-title":"Philosophy of Science"},{"key":"2023122018281809400_bib106","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3491101.3516384","article-title":"Towards fair and pro-social employment of digital pieceworkers for sourcing machine learning training data","volume-title":"CHI Conference on Human Factors in Computing Systems Extended Abstracts","author":"Rothschild","year":"2022"},{"issue":"2","key":"2023122018281809400_bib107","doi-asserted-by":"publisher","DOI":"10.1145\/3485766","article-title":"A survey of evaluation metrics used for nlg systems","volume":"55","author":"Sai","year":"2022","journal-title":"ACM Computing Surveys"},{"key":"2023122018281809400_bib108","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.17548","article-title":"Whose opinions do language models reflect?","author":"Santurkar","year":"2023"},{"key":"2023122018281809400_bib109","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.naacl-main.431","article-title":"Annotators with attitudes: How annotator beliefs and identities bias toxic language detection","author":"Sap","year":"2022"},{"key":"2023122018281809400_bib110","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2206.05802","article-title":"Self-critiquing models for assisting human evaluators","author":"Saunders","year":"2022"},{"key":"2023122018281809400_bib111","volume-title":"Dynamic Memory: A Theory of Reminding and Learning in Computers and People","author":"Schank","year":"1983"},{"key":"2023122018281809400_bib112","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2204.14146","article-title":"Training language models with language feedback","author":"Scheurer","year":"2022"},{"key":"2023122018281809400_bib113","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.16755","article-title":"Training language models with language feedback at scale","author":"Scheurer","year":"2023"},{"key":"2023122018281809400_bib114","article-title":"Peer: A collaborative language model","author":"Schick","year":"2022","journal-title":"ArXiv"},{"key":"2023122018281809400_bib115","doi-asserted-by":"publisher","first-page":"41","DOI":"10.18653\/v1\/E17-2007","article-title":"The limits of automatic summarisation according to ROUGE","volume-title":"Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers","author":"Schluter","year":"2017"},{"key":"2023122018281809400_bib116","article-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017","journal-title":"arXiv preprint arXiv:1707.06347"},{"key":"2023122018281809400_bib117","doi-asserted-by":"publisher","first-page":"7881","DOI":"10.18653\/v1\/2020.acl-main.704","article-title":"BLEURT: Learning robust metrics for text generation","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Sellam","year":"2020"},{"key":"2023122018281809400_bib118","doi-asserted-by":"publisher","first-page":"1683","DOI":"10.18653\/v1\/P16-1159","article-title":"Minimum risk training for neural machine translation","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Shen","year":"2016"},{"key":"2023122018281809400_bib119","doi-asserted-by":"publisher","first-page":"3407","DOI":"10.18653\/v1\/D19-1339","article-title":"The woman worked as a babysitter: On biases in language generation","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Sheng","year":"2019"},{"key":"2023122018281809400_bib120","doi-asserted-by":"publisher","first-page":"614","DOI":"10.1145\/1401890.1401965","article-title":"Get another label? Improving data quality and data mining using multiple, noisy labelers","volume-title":"Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Sheng","year":"2008"},{"key":"2023122018281809400_bib121","doi-asserted-by":"publisher","first-page":"3478","DOI":"10.18653\/v1\/2021.findings-emnlp.295","article-title":"Refine and imitate: Reducing repetition and inconsistency in persuasion dialogues via reinforcement learning and human demonstration","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021","author":"Shi","year":"2021"},{"key":"2023122018281809400_bib122","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.11366","article-title":"Reflexion: An autonomous agent with dynamic memory and self-reflection","author":"Shinn","year":"2023"},{"key":"2023122018281809400_bib123","doi-asserted-by":"publisher","first-page":"3758","DOI":"10.18653\/v1\/2021.naacl-main.295","article-title":"Beyond fair pay: Ethical implications of NLP crowdsourcing","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Shmueli","year":"2021"},{"key":"2023122018281809400_bib124","doi-asserted-by":"publisher","first-page":"203","DOI":"10.3115\/1626355.1626383","article-title":"Rule-based translation with statistical phrase-based post-editing","volume-title":"Proceedings of the Second Workshop on Statistical Machine Translation","author":"Simard","year":"2007"},{"key":"2023122018281809400_bib125","doi-asserted-by":"publisher","first-page":"254","DOI":"10.3115\/1613715.1613751","article-title":"Cheap and fast \u2013 but is it good? Evaluating non-expert annotations for natural language tasks","volume-title":"Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing","author":"Snow","year":"2008"},{"key":"2023122018281809400_bib126","doi-asserted-by":"publisher","first-page":"2220","DOI":"10.1177\/1071181322661532","article-title":"Effects of payment rate and country\u2019s income level on attitude toward a crowdsourcing task","volume-title":"Proceedings of the Human Factors and Ergonomics Society Annual Meeting","author":"Soratana","year":"2022"},{"key":"2023122018281809400_bib127","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445092","article-title":"The psychological well-being of content moderators: The emotional labor of commercial moderation and avenues for improving support","volume-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems","author":"Steiger","year":"2021"},{"key":"2023122018281809400_bib128","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2009.01325","article-title":"Learning to summarize from human feedback","author":"Stiennon","year":"2020"},{"key":"2023122018281809400_bib129","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1145\/1216295.1216316","article-title":"Toward harnessing user feedback for machine learning","volume-title":"Proceedings of the 12th International Conference on Intelligent User Interfaces","author":"Stumpf","year":"2007"},{"key":"2023122018281809400_bib130","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18653\/v1\/2021.emnlp-tutorials.1","article-title":"Crowdsourcing beyond annotation: Case studies in benchmark data collection","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts","author":"Suhr","year":"2021"},{"key":"2023122018281809400_bib131","doi-asserted-by":"publisher","first-page":"769","DOI":"10.18653\/v1\/2022.naacl-main.56","article-title":"On the machine learning of ethical judgments from natural language","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Talat","year":"2022"},{"issue":"6","key":"2023122018281809400_bib132","article-title":"Teaching pre-trained models to systematically reason over implicit knowledge","volume":"4","author":"Talmor","year":"2020","journal-title":"arXiv preprint arXiv:2006.06609"},{"key":"2023122018281809400_bib133","doi-asserted-by":"publisher","first-page":"339","DOI":"10.18653\/v1\/2022.findings-naacl.26","article-title":"Learning to repair: Repairing model output errors after deployment using a dynamic memory of feedback","volume-title":"Findings of the Association for Computational Linguistics: NAACL 2022","author":"Tandon","year":"2022"},{"issue":"6","key":"2023122018281809400_bib134","doi-asserted-by":"publisher","first-page":"716","DOI":"10.1016\/j.artint.2007.09.009","article-title":"Teachable robots: Understanding human teaching behavior to build more effective robot learners","volume":"172","author":"Thomaz","year":"2008","journal-title":"Artificial Intelligence"},{"key":"2023122018281809400_bib135","doi-asserted-by":"crossref","first-page":"240","DOI":"10.18653\/v1\/2021.inlg-1.23","article-title":"Generation challenges: Results of the accuracy evaluation shared task","volume-title":"Proceedings of the 14th International Conference on Natural Language Generation","author":"Thomson","year":"2021"},{"key":"2023122018281809400_bib136","article-title":"Lamda: Language models for dialog applications","author":"Thoppilan","year":"2022","journal-title":"CoRR"},{"key":"2023122018281809400_bib137","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.09288","article-title":"Llama 2: Open foundation and fine-tuned chat models","author":"Touvron","year":"2023"},{"issue":"CSCW2","key":"2023122018281809400_bib138","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3476060","article-title":"Quantifying the invisible labor in crowd work","volume":"5","author":"Toxtli","year":"2021","journal-title":"Proceedings of the ACM on Human-Computer Interaction"},{"key":"2023122018281809400_bib139","article-title":"Formalizing the problem of side effect regularization","volume-title":"NeurIPS ML Safety Workshop","author":"Turner","year":"2022"},{"key":"2023122018281809400_bib140","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1007\/s10676-017-9440-6","article-title":"Human-aligned artificial intelligence is a multiobjective problem","volume":"20","author":"Vamplew","year":"2018","journal-title":"Ethics and Information Technology"},{"key":"2023122018281809400_bib141","article-title":"Putting humans in the natural language processing loop: A survey","author":"Wang","year":"2021","journal-title":"arXiv preprint arXiv:2103.04044"},{"key":"2023122018281809400_bib142","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1023\/A:1022676722315","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Machine Learning"},{"key":"2023122018281809400_bib143","article-title":"Generating sequences by learning to self-correct","author":"Welleck","year":"2022","journal-title":"arXiv preprint arXiv: 2211.00053"},{"key":"2023122018281809400_bib144","article-title":"Memory networks","author":"Weston","year":"2014","journal-title":"arXiv preprint arXiv:1410.3916"},{"key":"2023122018281809400_bib145","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1604.06045","article-title":"Dialog-based language learning","author":"Weston","year":"2016","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2023122018281809400_bib146","article-title":"Cybernetics; or control and communication in the animal and the machine.","author":"Wiener","year":"1948"},{"issue":"3\u20134","key":"2023122018281809400_bib147","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","article-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","volume":"8","author":"Williams","year":"1992","journal-title":"Machine Learning"},{"key":"2023122018281809400_bib148","doi-asserted-by":"publisher","first-page":"1563","DOI":"10.1145\/3178876.3186068","article-title":"Query suggestion with feedback memory network","volume-title":"Proceedings of the 2018 World Wide Web Conference","author":"Bin","year":"2018"},{"key":"2023122018281809400_bib149","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2109.10862","article-title":"Recursively summarizing books with human feedback","author":"Jeff","year":"2021"},{"key":"2023122018281809400_bib150","article-title":"Learning new skills after deployment: Improving open-domain internet-driven dialogue with human feedback","author":"Jing","year":"2022"},{"key":"2023122018281809400_bib151","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2111.08596","article-title":"Reinforcement learning with feedback from multiple humans with diverse skills","author":"Yamagata","year":"2021"},{"key":"2023122018281809400_bib152","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.296","article-title":"Re3: Generating longer stories with recursive reprompting and revision","volume-title":"Conference on Empirical Methods in Natural Language Processing","author":"Yang","year":"2022"},{"key":"2023122018281809400_bib153","doi-asserted-by":"publisher","first-page":"10730","DOI":"10.48550\/arXiv.2005.10636","article-title":"Graph-based, self-supervised program repair from diagnostic feedback","volume-title":"37th International Conference on Machine Learning ICML 2020","author":"Yasunaga","year":"2020"},{"key":"2023122018281809400_bib154","article-title":"Rrhf: Rank responses to align language models with human feedback without tears","author":"Yuan","year":"2023","journal-title":"arXiv preprint arXiv:2304.05302"},{"key":"2023122018281809400_bib155","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.591","article-title":"Disentangling uncertainty in machine translation evaluation","author":"Zerva","year":"2022"},{"key":"2023122018281809400_bib156","article-title":"Slic-hf: Sequence likelihood calibration with human feedback","author":"Zhao","year":"2023","journal-title":"arXiv preprint arXiv:2305.10425"},{"key":"2023122018281809400_bib157","article-title":"Secrets of rlhf in large language models part i: Ppo","author":"Zheng","year":"2023"},{"key":"2023122018281809400_bib158","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.11206","article-title":"Lima: Less is more for alignment","author":"Zhou","year":"2023"},{"key":"2023122018281809400_bib159","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1909.08593","article-title":"Fine-tuning language models from human preferences","author":"Ziegler","year":"2019","journal-title":"CoRR"},{"key":"2023122018281809400_bib160","doi-asserted-by":"publisher","first-page":"1687","DOI":"10.18653\/v1\/N18-1152","article-title":"Estimating summary quality with pairwise preferences","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Zopf","year":"2018"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00626\/2199585\/tacl_a_00626.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00626\/2199585\/tacl_a_00626.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T18:28:51Z","timestamp":1703096931000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00626\/118795\/Bridging-the-Gap-A-Survey-on-Integrating-Human"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":160,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00626","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}