{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T07:01:06Z","timestamp":1761807666074},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2011,8,31]],"date-time":"2011-08-31T00:00:00Z","timestamp":1314748800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/2.0"},{"start":{"date-parts":[[2011,8,31]],"date-time":"2011-08-31T00:00:00Z","timestamp":1314748800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/2.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Braz Comput Soc"],"published-print":{"date-parts":[[2011,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The Iterated Prisoner\u2019s Dilemma (IPD) has been used as a paradigm for studying the emergence of cooperation among individual agents. Many computer experiments show that cooperation does arise under certain conditions. In particular, the spatial version of the IPD has been used and analyzed to understand the role of local interactions in the emergence and maintenance of cooperation. It is known that individual learning leads players to the Nash equilibrium of the game, which means that cooperation is not selected. Therefore, in this paper we propose that when players have social attachment, learning may lead to a certain rate of cooperation. We perform experiments where agents play the spatial IPD considering social relationships such as belonging to a hierarchy or to coalition. Results show that learners end up cooperating, especially when coalitions emerge.<\/jats:p>","DOI":"10.1007\/s13173-011-0038-2","type":"journal-article","created":{"date-parts":[[2011,8,30]],"date-time":"2011-08-30T11:02:38Z","timestamp":1314702158000},"page":"163-174","update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Learning to cooperate in the Iterated Prisoner\u2019s Dilemma by means of social attachments"],"prefix":"10.1007","volume":"17","author":[{"given":"Ana L. C.","family":"Bazzan","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ana","family":"Peleteiro","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Juan C.","family":"Burguillo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2011,8,31]]},"reference":[{"key":"38_CR1","doi-asserted-by":"crossref","unstructured":"Abramson G, Kuperman M (2001) Social games in a social network. Phys Rev E 63","DOI":"10.1103\/PhysRevE.63.030901"},{"key":"38_CR2","volume-title":"The evolution of cooperation","author":"R Axelrod","year":"1984","unstructured":"Axelrod R (1984) The evolution of cooperation. Basic Books, New York"},{"key":"38_CR3","first-page":"1389","volume-title":"Proc. of the 7th int. joint conf. on aut. agents and multiagent systems","author":"M Babes","year":"2008","unstructured":"Babes M, Cote EMD, Littman ML (2008) Social reward shaping in the prisoner\u2019s dilemma. In: Padgham L, Parkes D, M\u00fcller J, Parsons S (eds) Proc. of the 7th int. joint conf. on aut. agents and multiagent systems, IFAAMAS, May 2008, pp 1389\u20131392"},{"key":"38_CR4","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1145\/375735.376313","volume-title":"Proceedings of the fifth international conference on autonomous agents","author":"ALC Bazzan","year":"2001","unstructured":"Bazzan ALC, Bordini RH (2001) A framework for the simulation of agents with emotions: Report on experiments with the iterated prisoners dilemma. In: M\u00fcller JP, Andre E, Sen S, Frasson C (eds) Proceedings of the fifth international conference on autonomous agents, Montreal, Canada, May 2001. ACM, New York, pp 292\u2013299"},{"key":"38_CR5","series-title":"Lecture notes in artificial intelligence","first-page":"113","volume-title":"Intelligent agents V","author":"ALC Bazzan","year":"1999","unstructured":"Bazzan ALC, Bordini RH, Campbell JA (1999) Moral sentiments in multi-agent systems. In: Intelligent agents V. Lecture notes in artificial intelligence, vol 1555. Springer, Berlin, pp 113\u2013131. Also appeared as Proc. of the workshop on agent theories, architecture and languages (ATAL98), Paris, July 1998"},{"key":"38_CR6","doi-asserted-by":"publisher","first-page":"560","DOI":"10.1016\/j.engappai.2009.11.009","volume":"23","author":"ALC Bazzan","year":"2010","unstructured":"Bazzan ALC, de Oliveira D, da Silva BC (2010) Learning in groups of traffic signals. Eng Appl Artif Intell 23:560\u2013568","journal-title":"Eng Appl Artif Intell"},{"key":"38_CR7","first-page":"1603","volume-title":"NIPS","author":"RI Brafman","year":"2002","unstructured":"Brafman RI, Tennenholtz M (2002) Efficient learning equilibrium. In: NIPS, pp 1603\u20131610"},{"key":"38_CR8","first-page":"746","volume-title":"Proceedings of the fifteenth national conference on artificial intelligence","author":"C Claus","year":"1998","unstructured":"Claus C, Boutilier C (1998) The dynamics of reinforcement learning in cooperative multiagent systems. In: Proceedings of the fifteenth national conference on artificial intelligence, pp 746\u2013752"},{"key":"38_CR9","series-title":"Studies in computational intelligence","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1007\/978-3-540-73177-1_2","volume-title":"Computational intelligence for agent-based systems","author":"E Costa-Montenegro","year":"2007","unstructured":"Costa-Montenegro E, Burguillo-Rial JC, Gonz\u00e1lez-Casta\u00f1o FJ, Vales-Alonso J (2007) Agent-controlled sharing of distributed resources in user networks. In: Lee RST, Loia V (eds) Computational intelligence for agent-based systems. Studies in computational intelligence, vol 72. Springer, Berlin, pp 29\u201360"},{"key":"38_CR10","doi-asserted-by":"publisher","first-page":"368","DOI":"10.1016\/j.jnca.2010.06.010","volume":"34","author":"E Costa-Montenegro","year":"2011","unstructured":"Costa-Montenegro E, Burguillo-Rial JC, Gil-Casti\u00f1eira F, Gonz\u00e1lez-Casta\u00f1o FJ (2011) Implementation and analysis of the bittorrent protocol with a multi-agent model. J Netw Comput Appl 34:368\u2013383","journal-title":"J Netw Comput Appl"},{"key":"38_CR11","first-page":"780","volume-title":"Proceedings of the 20th international joint conference on artificial intelligence (IJCAI)","author":"N Fulda","year":"2007","unstructured":"Fulda N, Ventura D (2007) Predicting and preventing coordination problems in cooperative Q-learning systems. In: Proceedings of the 20th international joint conference on artificial intelligence (IJCAI), pp 780\u2013785"},{"key":"38_CR12","first-page":"274","volume-title":"UAI","author":"G Hines","year":"2008","unstructured":"Hines G, Larson K (2008) Learning when to take advice: A statistical test for achieving a correlated equilibrium. In: McAllester DA, Myllym\u00e4ki P (eds) UAI. AUAI Press, Menlo Park, pp 274\u2013281"},{"key":"38_CR13","first-page":"242","volume-title":"Proc. 15th international conf. on machine learning","author":"J Hu","year":"1998","unstructured":"Hu J, Wellman MP (1998) Multiagent reinforcement learning: Theoretical framework and an algorithm. In: Proc. 15th international conf. on machine learning. Kaufmann, Los Altos, pp\u00a0242\u2013250"},{"key":"38_CR14","doi-asserted-by":"publisher","first-page":"7716","DOI":"10.1073\/pnas.90.16.7716","volume":"90","author":"BA Huberman","year":"1993","unstructured":"Huberman BA, Glance NS (1993) Evolutionary games and computer simulations. Proc Natl Acad Sci USA 90:7716\u20137718","journal-title":"Proc Natl Acad Sci USA"},{"key":"38_CR15","doi-asserted-by":"crossref","unstructured":"Humphrys M (1997) Action selection methods using reinforcement learning. PhD thesis, Cambridge","DOI":"10.7551\/mitpress\/3118.003.0018"},{"key":"38_CR16","doi-asserted-by":"crossref","unstructured":"Kim BJ, Trusina A, Holme P, Minnhagen P, Chung JS, Choi MY (2002) Dynamic instabilities induced by asymmetric influence: Prisoner\u2019s dilemma game in small-world networks. Phys Rev E 66","DOI":"10.1103\/PhysRevE.66.021907"},{"key":"38_CR17","first-page":"438","volume-title":"Proceeding of the ECAI","author":"D Kuminov","year":"2008","unstructured":"Kuminov D, Tennenholtz M (2008) As safe as it gets: Near-optimal learning in multi-stage games with imperfect monitoring. In: Proceeding of the ECAI. IOS Press, Amsterdam, pp 438\u2013442"},{"key":"38_CR18","first-page":"327","volume-title":"Proceedings of the 6th international joint conference on autonomous agents and multiagent systems (AAMAS 2007)","author":"R Lin","year":"2007","unstructured":"Lin R, Kraus S, Shavitt Y (2007) On the benefits of cheating by self-interested agents in vehicular networks. In: Proceedings of the 6th international joint conference on autonomous agents and multiagent systems (AAMAS 2007). ACM, New York, pp 327\u2013334"},{"key":"38_CR19","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1016\/0167-2789(94)90289-5","volume":"75","author":"K Lindgren","year":"1994","unstructured":"Lindgren K, Nordahl M (1994) Evolutionary dynamics of spatial games. Physica D 75:292\u2013309","journal-title":"Physica D"},{"key":"38_CR20","first-page":"157","volume-title":"Proceedings of the 11th international conference on machine learning, ML","author":"ML Littman","year":"1994","unstructured":"Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the 11th international conference on machine learning, ML, New Brunswick, NJ. Kaufmann, Los Altos, pp 157\u2013163"},{"key":"38_CR21","first-page":"322","volume-title":"Proceedings of the eighteenth international conference on machine learning (ICML01)","author":"ML Littman","year":"2001","unstructured":"Littman ML (2001) Friend-or-Foe Q-learning in general-sum games. In: Proceedings of the eighteenth international conference on machine learning (ICML01), San Francisco, CA, USA. Kaufmann, Los Altos, pp 322\u2013328"},{"key":"38_CR22","unstructured":"Mailath G, Samuelson L, Shaked A (1993) Correlated equilibria as network equilibria. Discussion paper, University of Bonn"},{"key":"38_CR23","volume-title":"Learning automata: an introduction","author":"KS Narendra","year":"1989","unstructured":"Narendra KS, Thathachar MAL (1989) Learning automata: an introduction. Prentice-Hall, Upper Saddle River"},{"key":"38_CR24","doi-asserted-by":"publisher","first-page":"826","DOI":"10.1038\/359826a0","volume":"359","author":"MA Nowak","year":"1992","unstructured":"Nowak MA, May RM (1992) Evolutionary games and spatial chaos. Nature 359:826\u2013829","journal-title":"Nature"},{"key":"38_CR25","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511571299","volume-title":"The cognitive structure of emotions","author":"A Ortony","year":"1988","unstructured":"Ortony A, Clore GL, Collins A (1988) The cognitive structure of emotions. Cambridge University Press, Cambridge"},{"issue":"3","key":"38_CR26","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1007\/s10458-005-2631-2","volume":"11","author":"L Panait","year":"2005","unstructured":"Panait L, Luke S (2005) Cooperative multi-agent learning: The state of the art. Auton Agents Multi-Agent Syst 11(3):387\u2013434","journal-title":"Auton Agents Multi-Agent Syst"},{"key":"38_CR27","volume-title":"Proc. of the 2nd Brazilian workshop on social simulation","author":"A Peleteiro","year":"2010","unstructured":"Peleteiro A, Burguillo JC, Bazzan ALC (2010) Enhancing cooperation in the ipd with learning and coalitions. In: Proc. of the 2nd Brazilian workshop on social simulation, S. Bernardo do Campo. SBC, Porto Alegre"},{"issue":"7","key":"38_CR28","doi-asserted-by":"publisher","first-page":"382","DOI":"10.1016\/j.artint.2007.02.004","volume":"171","author":"T Sandholm","year":"2007","unstructured":"Sandholm T (2007) Perspectives on multiagent learning. Artif Intell 171(7):382\u2013391","journal-title":"Artif Intell"},{"key":"38_CR29","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1016\/0303-2647(95)01551-5","volume":"37","author":"TW Sandholm","year":"1995","unstructured":"Sandholm TW, Crites RH (1995) Multiagent reinforcement learning in the iterated prisoner\u2019s dilemma. Biosystems 37:147\u2013166","journal-title":"Biosystems"},{"issue":"1\u20132","key":"38_CR30","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1016\/S0004-3702(99)00036-3","volume":"111","author":"T Sandholm","year":"1999","unstructured":"Sandholm T, Larson K, Andersson M, Shehory O, Tohm\u00e9 F (1999) Coalition structure generation with worst case guarantees. Artif Intell 111(1\u20132):209\u2013238","journal-title":"Artif Intell"},{"issue":"7","key":"38_CR31","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1016\/j.artint.2006.02.006","volume":"171","author":"Y Shoham","year":"2007","unstructured":"Shoham Y, Powers R, Grenager T (2007) If multi-agent learning is the answer, what is the question? Artif Intell 171(7):365\u2013377","journal-title":"Artif Intell"},{"issue":"7","key":"38_CR32","doi-asserted-by":"publisher","first-page":"402","DOI":"10.1016\/j.artint.2006.12.005","volume":"171","author":"P Stone","year":"2007","unstructured":"Stone P (2007) Multiagent learning is not the answer. It is the question. Artif Intell 171(7):402\u2013405","journal-title":"Artif Intell"},{"issue":"3","key":"38_CR33","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1023\/A:1008942012299","volume":"8","author":"P Stone","year":"2000","unstructured":"Stone P, Veloso M (2000) Multiagent systems: A survey from a machine learning perspective. Auton Robots 8(3):345\u2013383","journal-title":"Auton Robots"},{"key":"38_CR34","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1093\/comjnl\/bxq018","volume":"54","author":"M Vinyals","year":"2011","unstructured":"Vinyals M, Rodr\u00edguez-Aguilar JA, Cerquides J (2011) A survey on sensor networks from a multiagent perspective. Comput J 54:455\u2013470","journal-title":"Comput J"},{"key":"38_CR35","first-page":"307","volume-title":"Proceedings of the 7th international joint conference on autonomous agents and multiagent systems","author":"P Vrancx","year":"2008","unstructured":"Vrancx P, Tuyls K, Westra RL (2008) Switching dynamics of multi-agent learning. In: Padgham L, Parkes D, M\u00fcller J, Parsons S (eds) Proceedings of the 7th international joint conference on autonomous agents and multiagent systems, Estoril, vol 1. pp 307\u2013313"},{"key":"38_CR36","volume-title":"Advances in neural information processing systems (NIPS-2002)","author":"X Wang","year":"2002","unstructured":"Wang X, Sandholm T (2002) Reinforcement learning to play an optimal Nash equilibrium in team Markov games. In: Advances in neural information processing systems (NIPS-2002), vol 15"},{"issue":"3","key":"38_CR37","first-page":"279","volume":"8","author":"CJCH Watkins","year":"1992","unstructured":"Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8(3):279\u2013292","journal-title":"Mach Learn"},{"key":"38_CR38","first-page":"1365","volume-title":"Proceedings of the 7th international joint conference on autonomous agents and multiagent systems","author":"C Zhang","year":"2008","unstructured":"Zhang C, Abdallah S, Lesser VR (2008) Efficient multi-agent reinforcement learning through automated supervision (extended abstract). In: Padgham L, Parkes D, M\u00fcller J, Parsons S (eds) Proceedings of the 7th international joint conference on autonomous agents and multiagent systems, Estoril, vol 3. pp 1365\u20131368"},{"key":"38_CR39","volume-title":"Proceedings of the 8th international conference on autonomous agents and multiagent systems (AAMAS)","author":"C Zhang","year":"2009","unstructured":"Zhang C, Abdallah S, Lesser V (2009) Integrating organizational control into multi-agent learning. In: Sichman JS, Decker KS, Sierra C, Castelfranchi C (eds) Proceedings of the 8th international conference on autonomous agents and multiagent systems (AAMAS), Budapest, Hungary"}],"container-title":["Journal of the Brazilian Computer Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13173-011-0038-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s13173-011-0038-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s13173-011-0038-2","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13173-011-0038-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,10]],"date-time":"2024-04-10T18:23:57Z","timestamp":1712773437000},"score":1,"resource":{"primary":{"URL":"https:\/\/journal-bcs.springeropen.com\/articles\/10.1007\/s13173-011-0038-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,8,31]]},"references-count":39,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2011,10]]}},"alternative-id":["38"],"URL":"https:\/\/doi.org\/10.1007\/s13173-011-0038-2","relation":{},"ISSN":["0104-6500","1678-4804"],"issn-type":[{"value":"0104-6500","type":"print"},{"value":"1678-4804","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,8,31]]},"assertion":[{"value":"16 February 2011","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 July 2011","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 August 2011","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}