{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T01:57:54Z","timestamp":1776391074550,"version":"3.51.2"},"reference-count":63,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2022,3,23]],"date-time":"2022-03-23T00:00:00Z","timestamp":1647993600000},"content-version":"vor","delay-in-days":81,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,3,25]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Persuasion games are fundamental in economics and AI research and serve as the basis for important applications. However, work on this setup assumes communication with stylized messages that do not consist of rich human language. In this paper we consider a repeated sender (expert) \u2013 receiver (decision maker) game, where the sender is fully informed about the state of the world and aims to persuade the receiver to accept a deal by sending one of several possible natural language reviews. We design an automatic expert that plays this repeated game, aiming to achieve the maximal payoff. Our expert is implemented within the Monte Carlo Tree Search (MCTS) algorithm, with deep learning models that exploit behavioral and linguistic signals in order to predict the next action of the decision maker, and the future payoff of the expert given the state of the game and a candidate review. We demonstrate the superiority of our expert over strong baselines and its adaptability to different decision makers and potential proposed deals.1<\/jats:p>","DOI":"10.1162\/tacl_a_00462","type":"journal-article","created":{"date-parts":[[2022,3,23]],"date-time":"2022-03-23T19:20:09Z","timestamp":1648063209000},"page":"307-324","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":3,"title":["Designing an Automatic Agent for Repeated Language\u2013based Persuasion Games"],"prefix":"10.1162","volume":"10","author":[{"given":"Maya","family":"Raifer","sequence":"first","affiliation":[{"name":"Technion\u2014Israel Institute of Technology, Israel. mayatarno@campus.technion.ac.il"}]},{"given":"Guy","family":"Rotman","sequence":"additional","affiliation":[{"name":"Technion\u2014Israel Institute of Technology, Israel. grotman@campus.technion.ac.il"}]},{"given":"Reut","family":"Apel","sequence":"additional","affiliation":[{"name":"Technion\u2014Israel Institute of Technology, Israel. reutapel@campus.technion.ac.il"}]},{"given":"Moshe","family":"Tennenholtz","sequence":"additional","affiliation":[{"name":"Technion\u2014Israel Institute of Technology, Israel. moshet@technion.ac.il"}]},{"given":"Roi","family":"Reichart","sequence":"additional","affiliation":[{"name":"Technion\u2014Israel Institute of Technology, Israel. roiri@technion.ac.il"}]}],"member":"281","published-online":{"date-parts":[[2022,3,25]]},"reference":[{"key":"2022033118513582700_bib1","doi-asserted-by":"crossref","first-page":"164","DOI":"10.3115\/1622064.1622097","article-title":"User simulation as testing for spoken dialog systems","volume-title":"Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue","author":"Ai","year":"2008"},{"key":"2022033118513582700_bib2","doi-asserted-by":"crossref","first-page":"e93","DOI":"10.7717\/peerj-cs.93","article-title":"Predicting judicial decisions of the European Court of Human Rights: A natural language processing perspective","volume":"2","author":"Aletras","year":"2016","journal-title":"PeerJ Computer Science"},{"key":"2022033118513582700_bib3","first-page":"6","article-title":"Learning in one-shot strategic form games","volume-title":"European Conference on Machine Learning","author":"Altman","year":"2006"},{"key":"2022033118513582700_bib4","first-page":"1545","article-title":"Learning to compose neural networks for question answering","volume-title":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Andreas","year":"2016"},{"key":"2022033118513582700_bib5","article-title":"Predicting decisions in language based persuasion games","author":"Apel","year":"2020","journal-title":"arXiv preprint arXiv:2012.09966"},{"key":"2022033118513582700_bib6","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.jet.2019.04.008","article-title":"Private Bayesian persuasion","volume":"182","author":"Arieli","year":"2019","journal-title":"Journal of Economic Theory"},{"key":"2022033118513582700_bib7","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.mathsocsci.2020.01.006","article-title":"Multi-issue social learning","volume":"104","author":"Bahar","year":"2020","journal-title":"Mathematical Social Sciences"},{"key":"2022033118513582700_bib8","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1145\/2940716.2940719","article-title":"Economic recommendation systems: One page abstract","volume-title":"Proceedings of the 2016 ACM Conference on Economics and Computation","author":"Bahar","year":"2016"},{"key":"2022033118513582700_bib9","article-title":"Neural machine translation by jointly learning to align and translate","volume-title":"International Conference on Learning Representations","author":"Bahdanau","year":"2015"},{"key":"2022033118513582700_bib10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/CIG.2018.8490403","article-title":"Evolutionary MCTS for multi-action adversarial games","volume-title":"2018 IEEE Conference on Computational Intelligence and Games (CIG)","author":"Baier","year":"2018"},{"key":"2022033118513582700_bib11","doi-asserted-by":"crossref","first-page":"956","DOI":"10.18653\/v1\/D18-1115","article-title":"Conversational decision-making model for predicting the king\u2019s decision in the Annals of the Joseon Dynasty","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Bak","year":"2018"},{"key":"2022033118513582700_bib12","first-page":"40","article-title":"UCT for tactical assault planning in real-time strategy games","volume-title":"Proceedings of the 21st International Jont Conference on Artifical Intelligence","author":"Balla","year":"2009"},{"key":"2022033118513582700_bib13","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1613\/jair.1.11849","article-title":"Predicting strategic behavior from free text","volume":"68","author":"Ben-Porat","year":"2020","journal-title":"Journal of Artificial Intelligence Research"},{"issue":"1","key":"2022033118513582700_bib14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TCIAIG.2012.2186810","article-title":"A survey of Monte Carlo tree search methods","volume":"4","author":"Browne","year":"2012","journal-title":"IEEE Transactions on Computational Intelligence and AI in Games"},{"key":"2022033118513582700_bib15","article-title":"Weakly-supervised hierarchical models for predicting persuasive strategies in good-faith textual requests","author":"Chen","year":"2021","journal-title":"arXiv preprint arXiv:2101.06351"},{"issue":"3","key":"2022033118513582700_bib16","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support vector machine","volume":"20","author":"Cortes","year":"1995","journal-title":"Machine Learning"},{"key":"2022033118513582700_bib17","first-page":"72","article-title":"Efficient selectivity and backup operators in Monte-Carlo tree search","volume-title":"International Conference on Computers and Games","author":"Coulom","year":"2006"},{"key":"2022033118513582700_bib18","first-page":"4171","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2022033118513582700_bib19","doi-asserted-by":"crossref","first-page":"1383","DOI":"10.18653\/v1\/P18-1128","article-title":"The hitchhiker\u2019s guide to testing statistical significance in natural language processing","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Dror","year":"2018"},{"key":"2022033118513582700_bib20","first-page":"155","article-title":"Support vector regression machines","volume":"9","author":"Drucker","year":"1997","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"7","key":"2022033118513582700_bib21","article-title":"Adaptive subgradient methods for online learning and stochastic optimization.","volume":"12","author":"Duchi","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"2022033118513582700_bib22","doi-asserted-by":"crossref","first-page":"60","DOI":"10.3115\/v1\/W14-4308","article-title":"Learning non-cooperative dialogue behaviours","volume-title":"Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL)","author":"Efstathiou","year":"2014"},{"issue":"2","key":"2022033118513582700_bib23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2594564","article-title":"Signaling schemes for revenue maximization","volume":"2","author":"Emek","year":"2014","journal-title":"ACM Transactions on Economics and Computation (TEAC)"},{"key":"2022033118513582700_bib24","first-page":"259","article-title":"Simulation-based approach to general game playing.","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Finnsson","year":"2008"},{"issue":"6084","key":"2022033118513582700_bib25","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1126\/science.1218633","article-title":"Predicting pragmatic reasoning in language games","volume":"336","author":"Frank","year":"2012","journal-title":"Science"},{"key":"2022033118513582700_bib26","volume-title":"Game theory","author":"Fudenberg","year":"1991"},{"key":"2022033118513582700_bib27","article-title":"Mogo: Improvements in Monte-Carlo computer-go using UCT and sequence-like simulations","author":"Gelly","year":"2006","journal-title":"Presentation given at the University of Alberta"},{"key":"2022033118513582700_bib28","first-page":"410","article-title":"A game-theoretic approach to generating spatial descriptions","volume-title":"Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing","author":"Golland","year":"2010"},{"key":"2022033118513582700_bib29","first-page":"217","article-title":"Cooperative user models in statistical dialog simulators","volume-title":"Proceedings of the SIGDIAL 2010 Conference","author":"Gonz\u00e1lez","year":"2010"},{"key":"2022033118513582700_bib30","article-title":"Deep learning for predicting human strategic behavior","volume-title":"Proceedings of the 30th International Conference on Neural Information Processing Systems","author":"Hartford","year":"2016"},{"key":"2022033118513582700_bib31","first-page":"2146","article-title":"Emergence of language with multi-agent games: Learning to communicate with sequences of symbols","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems","author":"Havrylov","year":"2017"},{"key":"2022033118513582700_bib32","article-title":"Convention-formation in iterated reference games","volume-title":"CogSci","author":"Hawkins","year":"2017"},{"key":"2022033118513582700_bib33","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v32i1.12003","article-title":"Persuasive influence detection: The role of argument sequencing","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Hidey","year":"2018"},{"key":"2022033118513582700_bib34","doi-asserted-by":"crossref","first-page":"11","DOI":"10.18653\/v1\/W17-5102","article-title":"Analyzing the semantic types of claims and premises in an online persuasive forum","volume-title":"Proceedings of the 4th Workshop on Argument Mining","author":"Hidey","year":"2017"},{"key":"2022033118513582700_bib35","first-page":"1706","article-title":"Reinforcement learning of cooperative persuasive dialogue policies using framing","volume-title":"Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers","author":"Hiraoka","year":"2014"},{"issue":"8","key":"2022033118513582700_bib36","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Computation"},{"key":"2022033118513582700_bib37","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1162\/tacl_a_00065","article-title":"Google\u2019s multilingual neural machine translation system: Enabling zero-shot translation","volume":"5","author":"Johnson","year":"2017","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2022033118513582700_bib38","first-page":"9","article-title":"An integrated dialog simulation technique for evaluating spoken dialog systems","volume-title":"COLING 2008: Proceedings of the Workshop on Speech Processing for Safety Critical Translation and Pervasive Applications","author":"Jung","year":"2008"},{"key":"2022033118513582700_bib39","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1109\/CIG.2017.8080432","article-title":"Opponent modeling based on action table for MCTS-based fighting Game AI","volume-title":"2017 IEEE Conference on Computational Intelligence and Games (CIG)","author":"Kim","year":"2017"},{"key":"2022033118513582700_bib40","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1162\/tacl_a_00276","article-title":"Natural questions: A benchmark for question answering research","volume":"7","author":"Kwiatkowski","year":"2019","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2022033118513582700_bib41","article-title":"Multi-agent cooperation and the emergence of (natural) language","volume-title":"International Conference on Learning Representations","author":"Lazaridou","year":"2017"},{"key":"2022033118513582700_bib42","first-page":"2443","article-title":"Deal or no deal? End-to-end learning of negotiation dialogues","volume-title":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","author":"Lewis","year":"2017"},{"key":"2022033118513582700_bib43","first-page":"293","article-title":"End-to-end trainable non-collaborative dialog system","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Li","year":"2020"},{"key":"2022033118513582700_bib44","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1145\/2764468.2764508","article-title":"Bayesian incentive-compatible bandit exploration","volume-title":"Proceedings of the Sixteenth ACM Conference on Economics and Computation","author":"Mansour","year":"2015"},{"issue":"2","key":"2022033118513582700_bib45","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1007\/s10506-019-09255-y","article-title":"Using machine learning to predict decisions of the European court of human rights","volume":"28","author":"Medvedeva","year":"2020","journal-title":"Artificial Intelligence and Law"},{"key":"2022033118513582700_bib46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18653\/v1\/D15-1001","article-title":"Language understanding for text-based games using deep reinforcement learning","volume-title":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","author":"Narasimhan","year":"2015"},{"key":"2022033118513582700_bib47","first-page":"1650","article-title":"Linguistic harbingers of betrayal: A case study on an online strategy game","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Niculae","year":"2015"},{"key":"2022033118513582700_bib48","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/CIG.2016.7860394","article-title":"Informed Monte Carlo tree search for real-time strategy games","volume-title":"2016 IEEE Conference on Computational Intelligence and Games (CIG)","author":"Ontan\u00f3n","year":"2016"},{"issue":"3","key":"2022033118513582700_bib49","doi-asserted-by":"crossref","first-page":"667","DOI":"10.1162\/coli_a_00383","article-title":"Predicting in-game actions from interviews of NBA players","volume":"46","author":"Oved","year":"2020","journal-title":"Computational Linguistics"},{"key":"2022033118513582700_bib50","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v31i1.10613","article-title":"Psychological forest: Predicting human behavior","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Plonsky","year":"2017"},{"key":"2022033118513582700_bib51","volume-title":"Information rules: A strategic guide to the network economy","author":"Shapiro","year":"1998"},{"key":"2022033118513582700_bib52","doi-asserted-by":"crossref","first-page":"1990","DOI":"10.18653\/v1\/D19-1206","article-title":"How to build user simulators to train RL-based dialog systems","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference onNatural Language Processing (EMNLP-IJCNLP)","author":"Shi","year":"2019"},{"issue":"6419","key":"2022033118513582700_bib53","doi-asserted-by":"crossref","first-page":"1140","DOI":"10.1126\/science.aar6404","article-title":"A general reinforcement learning algorithm that masters chess, shogi, and go through self-play","volume":"362","author":"Silver","year":"2018","journal-title":"Science"},{"key":"2022033118513582700_bib54","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1007\/978-3-319-77538-8_25","article-title":"Self-adaptive mcts for general video game playing","volume-title":"International Conference on the Applications of Evolutionary Computation","author":"Sironi","year":"2018"},{"key":"2022033118513582700_bib55","first-page":"885","article-title":"Supervised open information extraction","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Stanovsky","year":"2018"},{"key":"2022033118513582700_bib56","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1145\/2872427.2883081","article-title":"Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions","volume-title":"Proceedings of the 25th International Conference on World Wide Web","author":"Tan","year":"2016"},{"key":"2022033118513582700_bib57","doi-asserted-by":"crossref","first-page":"2368","DOI":"10.18653\/v1\/P16-1224","article-title":"Learning language games through interaction","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Wang","year":"2016"},{"key":"2022033118513582700_bib58","doi-asserted-by":"crossref","first-page":"5635","DOI":"10.18653\/v1\/P19-1566","article-title":"Persuasion for good: Towards a personalized persuasive dialogue system for social good","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Wang","year":"2019"},{"key":"2022033118513582700_bib59","volume-title":"Philosophical investigations. philosophische untersuchungen","author":"Wittgenstein","year":"1953"},{"key":"2022033118513582700_bib60","first-page":"3620","article-title":"Let\u2019s make your request more persuasive: Modelingpersuasive strategies via semi-supervised neural nets on crowdfunding platforms","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Yang","year":"2019"},{"key":"2022033118513582700_bib61","first-page":"253","volume-title":"A recurrent attention network for judgment prediction","author":"Ze","year":"2019"},{"key":"2022033118513582700_bib62","doi-asserted-by":"crossref","first-page":"1512","DOI":"10.1145\/3394486.3403202","article-title":"Evaluating conversational recommender systems via user simulation","volume-title":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Zhang","year":"2020"},{"key":"2022033118513582700_bib63","doi-asserted-by":"crossref","first-page":"3540","DOI":"10.18653\/v1\/D18-1390","article-title":"Legal judgment prediction via topological learning","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Zhong","year":"2018"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00462\/2004063\/tacl_a_00462.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00462\/2004063\/tacl_a_00462.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,29]],"date-time":"2023-01-29T22:19:20Z","timestamp":1675030760000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00462\/110050\/Designing-an-Automatic-Agent-for-Repeated-Language"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022]]},"references-count":63,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00462","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022]]},"published":{"date-parts":[[2022]]}}}