{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T20:24:23Z","timestamp":1771705463493,"version":"3.50.1"},"reference-count":48,"publisher":"Maximum Academic Press","license":[{"start":{"date-parts":[[2020,6,30]],"date-time":"2020-06-30T00:00:00Z","timestamp":1593475200000},"content-version":"unspecified","delay-in-days":181,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["The Knowledge Engineering Review"],"published-print":{"date-parts":[[2020]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>In multi-objective multi-agent systems (MOMASs), agents explicitly consider the possible trade-offs between conflicting objective functions. We argue that compromises between competing objectives in MOMAS should be analyzed on the basis of the utility that these compromises have for the users of a system, where an agent\u2019s utility function maps their payoff vectors to scalar utility values. This utility-based approach naturally leads to two different optimization criteria for agents in a MOMAS: expected scalarized returns (ESRs) and scalarized expected returns (SERs). In this article, we explore the differences between these two criteria using the framework of multi-objective normal-form games (MONFGs). We demonstrate that the choice of optimization criterion (ESR or SER) can radically alter the set of equilibria in a MONFG when nonlinear utility functions are used.<\/jats:p>","DOI":"10.1017\/s0269888920000351","type":"journal-article","created":{"date-parts":[[2020,6,30]],"date-time":"2020-06-30T04:08:42Z","timestamp":1593490122000},"source":"Crossref","is-referenced-by-count":15,"title":["A utility-based analysis of equilibria in multi-objective normal-form games"],"prefix":"10.48130","volume":"35","author":[{"given":"Roxana","family":"R\u0103dulescu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7951-878X","authenticated-orcid":false,"given":"Patrick","family":"Mannion","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yijie","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Diederik M.","family":"Roijers","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ann","family":"Now\u00e9","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"27968","published-online":{"date-parts":[[2020,6,30]]},"reference":[{"key":"S0269888920000351_ref48","unstructured":"Zintgraf, L. M. , Roijers, D. M. , Linders, S. , Jonker, C. M. & Now\u00e9, A. 2018. Ordered preference elicitation strategies for supporting multi-objective decision making. In Proceedings of the 17th International Conference on Autonomous Agents and Multi-Agent Systems, 1477\u20131485. International Foundation for Autonomous Agents and Multiagent Systems."},{"key":"S0269888920000351_ref47","unstructured":"Zintgraf, L. M. , Kanters, T. V. , Roijers, D. M. , Oliehoek, F. A. & Beau, P. 2015. Quality assessment of MORL algorithms: a utility-based approach. In Benelearn 2015: Proceedings of the Twenty-Fourth Belgian-Dutch Conference on Machine Learning."},{"key":"S0269888920000351_ref20","unstructured":"Mannion, P. , Mason, K. , Devlin, S. , Duggan, J. & Howley, E. 2016b. Multi-objective dynamic dispatch optimisation using multi-agent reinforcement learning. In Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), May 2016b."},{"key":"S0269888920000351_ref23","doi-asserted-by":"publisher","DOI":"10.2307\/1969529"},{"key":"S0269888920000351_ref39","unstructured":"Walraven, E. & Spaan, M. T. J. 2016. Planning under uncertainty for aggregated electric vehicle charging with renewable energy supply. In Proceedings of the European Conference on Artificial Intelligence, 904\u2013912."},{"key":"S0269888920000351_ref19","unstructured":"Mannion, P. , Duggan, J. & Howley, E. 2017b. A theoretical and empirical analysis of reward transformations in multi-objective stochastic games. In Proceedings of the 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), May 2017b."},{"key":"S0269888920000351_ref17","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2017.05.090"},{"key":"S0269888920000351_ref13","first-page":"175","article-title":"Sur les fonctions convexes et les in\u00e9galit\u00e9s entre les valeurs moyennes","volume":"30","author":"Jensen","year":"1906","journal-title":"Actamathematica"},{"key":"S0269888920000351_ref12","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-67504-6_14"},{"key":"S0269888920000351_ref11","doi-asserted-by":"publisher","DOI":"10.1287\/moor.14.1.18"},{"key":"S0269888920000351_ref30","unstructured":"Roijers, D. M. , Steckelmacher, D. & Now\u00e9, A. 2018. Multi-objective reinforcement learning for the expected utility of the return. In Proceedings of the Adaptive and Learning Agents Workshop at FAIM 2018."},{"key":"S0269888920000351_ref2","doi-asserted-by":"publisher","DOI":"10.1016\/0304-4068(74)90037-8"},{"key":"S0269888920000351_ref3","doi-asserted-by":"publisher","DOI":"10.2307\/1911154"},{"key":"S0269888920000351_ref4","doi-asserted-by":"publisher","DOI":"10.1007\/BF00133085"},{"key":"S0269888920000351_ref21","unstructured":"Mossalam, H. , Assael, Y. M. , Roijers, D. M. & Whiteson, S. 2016. Multi-objective deep reinforcement learning. In NIPS Workshop on Deep Reinforcement Learning."},{"key":"S0269888920000351_ref10","doi-asserted-by":"publisher","DOI":"10.1006\/game.1993.1021"},{"key":"S0269888920000351_ref18","author":"Mannion","year":"2016"},{"key":"S0269888920000351_ref16","doi-asserted-by":"publisher","DOI":"10.1017\/S0269888918000292"},{"key":"S0269888920000351_ref5","doi-asserted-by":"publisher","DOI":"10.2140\/pjm.1956.6.1"},{"key":"S0269888920000351_ref15","unstructured":"Lozovanu, D. , Solomon, D. & Zelikovsky, A. 2005. Multiobjective games and determining pareto-nashequilibria. Buletinul Academiei de \u015etiin\u0163e a Republicii Moldova. Matematica, (3), 115\u2013122."},{"key":"S0269888920000351_ref1","unstructured":"Arifovic, J. , Boitnott, J. F. & Duffy, J. 2016. Learning correlated equilibria: an evolutionary approach. Journal of Economic Behavior & Organization 157, 171\u2013190."},{"key":"S0269888920000351_ref6","first-page":"303","article-title":"Pareto equilibria in multi-objective games","volume":"60","author":"Borm","year":"1990","journal-title":"Methods of Operations Research"},{"key":"S0269888920000351_ref14","unstructured":"Lozan, V. & Ungureanu, V. 2013. Computing the pareto-nash equilibrium set in finite multi-objective mixed-strategy games. Computer Science Journal of Moldova, 21 (2)."},{"key":"S0269888920000351_ref41","article-title":"Multiple criteria games \u2013 theory and applications","volume":"60","author":"Wierzbicki","year":"1995","journal-title":"Journal of Systems Engineering and Electronics"},{"key":"S0269888920000351_ref9","doi-asserted-by":"publisher","DOI":"10.1006\/game.1999.0740"},{"key":"S0269888920000351_ref44","article-title":"Multi-objective multiagent credit assignment in reinforcement learning and nsga-ii","volume":"200","author":"Yliniemi","year":"2016","journal-title":"Soft Computing"},{"key":"S0269888920000351_ref33","doi-asserted-by":"publisher","DOI":"10.1002\/nav.3800060107"},{"key":"S0269888920000351_ref8","unstructured":"Devlin, S. & Kudenko, D. 2011. Theoretical considerations of potential-based reward shaping for multi-agent systems. In Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 225\u2013232."},{"key":"S0269888920000351_ref24","doi-asserted-by":"publisher","DOI":"10.1145\/1379759.1379762"},{"key":"S0269888920000351_ref40","unstructured":"Watkins, C. J. C. H. Learning from Delayed Rewards. PhD thesis, King\u2019s College, Cambridge, UK, 1989."},{"key":"S0269888920000351_ref35","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-010-5232-5"},{"key":"S0269888920000351_ref46","first-page":"1641","article-title":"Cyclic equilibria in markov games","author":"Zinkevich","year":"2006","journal-title":"In Advances in Neural Information Processing Systems"},{"key":"S0269888920000351_ref27","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-019-09433-x"},{"key":"S0269888920000351_ref25","unstructured":"R\u0103dulescu, R. , Legrand, M. , Efthymiadis, K. , Roijers, D. M. & Now\u00e9, A. 2018. Deep multi-agent reinforcement learning in a homogeneous open population. In Proceedings of the 30th Benelux Conference on Artificial Intelligence (BNAIC 2018), 177\u2013191."},{"key":"S0269888920000351_ref26","unstructured":"R\u0103dulescu, R. , Mannion, P. , Roijers, D. & Now\u00e9, A. 2019. Equilibria in multi-objective games: a utility-based perspective. In Adaptive and Learning Agents Workshop (at AAMAS 2019), May 2019."},{"key":"S0269888920000351_ref36","first-page":"3483","article-title":"Multi-objective reinforcement learning using sets of pareto dominating policies","volume":"15","author":"Van Moffaert","year":"2014","journal-title":"The Journal of Machine Learning Research"},{"key":"S0269888920000351_ref29","doi-asserted-by":"publisher","DOI":"10.1145\/3008665.3008670"},{"key":"S0269888920000351_ref37","unstructured":"Virtanen, P. , Gommers, R. , Oliphant, T. E. , Haberland, M. , Reddy, T. , Cournapeau, D. , Burovski, E. , Peterson, P. , Weckesser, W. , Bright, J. , van der Walt, S. J. , Brett, M. , Wilson, J. , Jarrod Millman, K. , Mayorov, N. , Nelson, A. R. J. , Jones, E. , Kern, R. , Larson, E. , Carey, C. , Polat, \u0130. , Feng, Y. , Moore, E. W. , Vand erPlas, J., Laxalde, D. , Perktold, J. , Cimrman, R. , Henriksen, I. , Quintero, E. A. , Harris, C. R. , Archibald, A. M. , Ribeiro, A. H. , Pedregosa, F. , van Mulbregt, P. & Contributors, S. 2019. SciPy 1.0\u2013Fundamental Algorithms for Scientific Computing in Python. arXiv e-prints, art. arXiv:1907.10121, July 2019."},{"key":"S0269888920000351_ref45","unstructured":"Zhang, Y. , R\u0103dulescu, R. , Mannion, P. , Roijers, D. M. & Now\u00e9, A. 2020. Opponent modelling for reinforcement learning in multi-objective normal form games. In Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2020), May 2020."},{"key":"S0269888920000351_ref22","unstructured":"Nash, J. 1950. Equilibrium points in n-person games. Proceedings of the National Academy of Sciences 36(1), 48\u201349. ISSN 0027-8424."},{"key":"S0269888920000351_ref28","unstructured":"Reymond, M. , Patyn, C. , R\u0103dulescu, R. , Deconinck, G. & Now\u00e9, A. 2018. Reinforcement learning for demand response of domestic household appliances. In Proceedings of the Adaptive and Learning Agents Workshop at FAIM 2018."},{"key":"S0269888920000351_ref32","doi-asserted-by":"publisher","DOI":"10.2200\/S00765ED1V01Y201704AIM034"},{"key":"S0269888920000351_ref7","doi-asserted-by":"publisher","DOI":"10.1145\/2739480.2754770"},{"key":"S0269888920000351_ref34","unstructured":"Talpert, V. , Sobh, I. , Kiran, B. R. , Mannion, P. , Yogamani, S. , El-Sallab, A. & Perez, P. 2019. Exploring applications of deep reinforcement learning for real-world autonomous driving systems. In International Conference on Computer Vision Theory and Applications (VISAPP), February 2019."},{"key":"S0269888920000351_ref31","doi-asserted-by":"publisher","DOI":"10.1613\/jair.3987"},{"key":"S0269888920000351_ref38","article-title":"Axiomatizations of paretoequilibria in multicriteria games","volume":"280","author":"Voorneveld","year":"1999","journal-title":"Games and Economic Behavior"},{"key":"S0269888920000351_ref42","unstructured":"Wiggers, A. J. , Oliehoek, F. A. & Roijers, D. M. 2016. Structure in the value function of two-player zero-sum games of incomplete information. In Proceedings of the Twenty-second European Conference on Artificial Intelligence, 1628\u20131629. IOS Press."},{"key":"S0269888920000351_ref43","article-title":"Simulation of the introduction of new technologies in air traffic management","volume":"270","author":"Yliniemi","year":"2015","journal-title":"Connection Science"}],"container-title":["The Knowledge Engineering Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0269888920000351","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T14:42:19Z","timestamp":1767624139000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0269888920000351\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020]]},"references-count":48,"alternative-id":["S0269888920000351"],"URL":"https:\/\/doi.org\/10.1017\/s0269888920000351","relation":{},"ISSN":["0269-8889","1469-8005"],"issn-type":[{"value":"0269-8889","type":"print"},{"value":"1469-8005","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020]]},"article-number":"e32"}}