{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T06:12:24Z","timestamp":1778911944925,"version":"3.51.4"},"reference-count":37,"publisher":"SAGE Publications","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["AIC"],"published-print":{"date-parts":[[2021,2,15]]},"abstract":"<jats:p>Navigation apps have become more and more popular, as they give information about the current traffic state to drivers who then adapt their route choice. In commuting scenarios, where people repeatedly travel between a particular origin and destination, people tend to learn and adapt to different situations. What if the experience gained from such a learning task is shared via an app? In this paper, we analyse the effects that adaptive driver agents cause on the overall network, when those agents share their aggregated experience about route choice in a reinforcement learning setup. In particular, in this investigation, Q-learning is used and drivers share what they have learnt about the system, not just information about their current travel times. Using a classical commuting scenario, we show that experience sharing can improve convergence times that underlie a typical learning task. Further, we analyse individual learning dynamics to get an impression how aggregate and individual dynamics are related to each other. Based on that interesting pattern of individual learning dynamics can be observed that would otherwise be hidden in an only aggregate analysis.<\/jats:p>","DOI":"10.3233\/aic-201582","type":"journal-article","created":{"date-parts":[[2020,12,29]],"date-time":"2020-12-29T10:58:45Z","timestamp":1609239525000},"page":"105-119","source":"Crossref","is-referenced-by-count":3,"title":["Accelerating route choice learning with experience sharing in a commuting scenario: An agent-based approach"],"prefix":"10.1177","volume":"34","author":[{"given":"Franziska","family":"Kl\u00fcgl","sequence":"first","affiliation":[{"name":"School of Science and Technology, \u00d6rebro University, Sweden. E-mail:\u00a0franziska.klugl@oru.se"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ana\u00a0Lucia\u00a0C.","family":"Bazzan","sequence":"additional","affiliation":[{"name":"Instituto de Informatica, Universidade Federal do Rio Grando do Sul (UFRGS), Brazil. E-mail:\u00a0bazzan@inf.ufrgs.br"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"key":"10.3233\/AIC-201582_ref1","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.engappai.2018.12.003","article-title":"Aligning individual and collective welfare in complex socio-technical systems by combining metaheuristics and reinforcement learning","volume":"79","author":"Bazzan","year":"2019","journal-title":"Eng. Appl. of AI"},{"key":"10.3233\/AIC-201582_ref2","unstructured":"A.L.C.\u00a0Bazzan and C.\u00a0Chira, Hybrid evolutionary and reinforcement learning approach to accelerate traffic assignment (extended abstract), in: Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015), R.\u00a0Bordini, E.\u00a0Elkind, G.\u00a0Weiss and P.\u00a0Yolum, eds, IFAAMAS, 2015, pp.\u00a01723\u20131724, http:\/\/www.aamas2015.com\/en\/AAMAS_2015_USB\/aamas\/p1723.pdf."},{"key":"10.3233\/AIC-201582_ref3","doi-asserted-by":"crossref","unstructured":"A.L.C.\u00a0Bazzan, M.\u00a0Fehler and F.\u00a0Kl\u00fcgl, Learning to coordinate in a network of social drivers: The role of information, in: Proceedings of the International Workshop on Learning and Adaptation in MAS (LAMAS 2005), K.\u00a0Tuyls, P.J.\u00a0Hoen, K.\u00a0Verbeeck and S.\u00a0Sen, eds, Lecture Notes in Artificial Intelligence, 2006, pp.\u00a0115\u2013128, www.inf.ufrgs.br\/~bazzan\/downloads\/lamas38980115.pdf.gz.","DOI":"10.1007\/11691839_6"},{"key":"10.3233\/AIC-201582_ref4","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2016.7727899"},{"key":"10.3233\/AIC-201582_ref5","unstructured":"A.L.C.\u00a0Bazzan and F.\u00a0Kl\u00fcgl, Experience sharing in a traffic scenario, in: Proc. of the 11th Int. Workshop on Agents in Traffic and Transportation, Santiago de Compostella, Spain, 4\u00a0Sept. 2020, 2020."},{"key":"10.3233\/AIC-201582_ref6","doi-asserted-by":"publisher","first-page":"619","DOI":"10.1007\/s11590-010-0226-6","article-title":"A\u00a0biased random-key genetic algorithm for road congestion minimization","volume":"4","author":"Buriol","year":"2010","journal-title":"Optimization Letters"},{"key":"10.3233\/AIC-201582_ref7","doi-asserted-by":"publisher","DOI":"10.1109\/VNC.2015.7385574"},{"key":"10.3233\/AIC-201582_ref8","unstructured":"C.\u00a0Claus and C.\u00a0Boutilier, The dynamics of reinforcement learning in cooperative multiagent systems, in: Proceedings of the Fifteenth National Conference on Artificial Intelligence, AAAI\u201998\/IAAI\u201998, American Association for Artificial Intelligence, Menlo Park, CA, USA, 1998, pp.\u00a0746\u2013752."},{"key":"10.3233\/AIC-201582_ref9","doi-asserted-by":"publisher","DOI":"10.1007\/11839088_3"},{"key":"10.3233\/AIC-201582_ref10","unstructured":"H.\u00a0Dia and S.\u00a0Panwai, Intelligent Transport Systems: Neural Agent (Neugent) Models of Driver Behaviour, LAP Lambert Academic Publishing, 2014, http:\/\/books.google.com.br\/books?id=fPXpoAEACAAJ. ISBN 9783659528682."},{"issue":"0","key":"10.3233\/AIC-201582_ref11","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1016\/j.engappai.2014.07.005","article-title":"An inverted ant colony optimization approach to traffic","volume":"36","author":"Dias","year":"2014","journal-title":"Engineering Applications of Artificial Intelligence"},{"issue":"1","key":"10.3233\/AIC-201582_ref12","doi-asserted-by":"publisher","first-page":"21","DOI":"10.3390\/make1010002","article-title":"Learning to teach reinforcement learning agents","volume":"1","author":"Fachantidis","year":"2019","journal-title":"Machine Learning and Knowledge Extraction"},{"key":"10.3233\/AIC-201582_ref13","doi-asserted-by":"publisher","DOI":"10.1145\/2001858.2002043"},{"key":"10.3233\/AIC-201582_ref14","unstructured":"R.\u00a0Grunitzki and A.L.C.\u00a0Bazzan, Combining car-to-infrastructure communication and multi-agent reinforcement learning in route choice, in: Proceedings of the Ninth Workshop on Agents in Traffic and Transportation (ATT-2016), A.L.C.\u00a0Bazzan, F.\u00a0Kl\u00fcgl, S.\u00a0Ossowski and G.\u00a0Vizzari, eds, CEUR-WS.org, New York, 2016, http:\/\/ceur-ws.org\/Vol-1678\/paper12.pdf. ISSN 1613-0073."},{"issue":"4","key":"10.3233\/AIC-201582_ref15","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1080\/15472450490523874","article-title":"Simulation studies on adaptative route decision and the influence of information on commuter scenarios","volume":"8","author":"Kl\u00fcgl","year":"2004","journal-title":"Journal of Intelligent Transportation Systems: Technology, Planning, and Operations"},{"key":"10.3233\/AIC-201582_ref16","doi-asserted-by":"publisher","DOI":"10.1145\/1015330.1015410"},{"key":"10.3233\/AIC-201582_ref17","doi-asserted-by":"publisher","DOI":"10.1109\/ITSC.2013.6728579"},{"issue":"5","key":"10.3233\/AIC-201582_ref18","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1016\/0041-1647(75)90030-1","article-title":"An efficient approach to solving the road network equilibrium traffic assignment problem","volume":"9","author":"LeBlanc","year":"1975","journal-title":"Transportation Research"},{"key":"10.3233\/AIC-201582_ref19","doi-asserted-by":"publisher","DOI":"10.1007\/11523468_41"},{"key":"10.3233\/AIC-201582_ref20","unstructured":"K.S.\u00a0Narendra and M.A.L.\u00a0Thathachar, Learning Automata: An Introduction, Prentice-Hall, Upper Saddle River, NJ, USA, 1989. ISBN 0-13-485558-2."},{"key":"10.3233\/AIC-201582_ref21","doi-asserted-by":"publisher","DOI":"10.4018\/978-1-60566-226-8.ch015"},{"key":"10.3233\/AIC-201582_ref22","unstructured":"J.\u00a0Ort\u00fazar and L.G.\u00a0Willumsen, Modelling Transport, 3rd edn, John Wiley & Sons, 2001."},{"key":"10.3233\/AIC-201582_ref23","doi-asserted-by":"publisher","DOI":"10.1017\/S0269888920000119"},{"key":"10.3233\/AIC-201582_ref24","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1016\/j.trc.2017.11.011","article-title":"Analysing the impact of travel information for minimising the regret of route choice","volume":"88","author":"Ramos","year":"2018","journal-title":"Transportation Research Part\u00a0C: Emerging Technologies"},{"key":"10.3233\/AIC-201582_ref25","unstructured":"G.d.O.\u00a0Ramos, B.C.\u00a0da Silva and A.L.C.\u00a0Bazzan, Learning to minimise regret in route choice, in: Proc. of the 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2017), S.\u00a0Das, E.\u00a0Durfee, K.\u00a0Larson and M.\u00a0Winikoff, eds, IFAAMAS, S\u00e3o Paulo, 2017, pp.\u00a0846\u2013855, http:\/\/ifaamas.org\/Proceedings\/aamas2017\/pdfs\/p846.pdf."},{"key":"10.3233\/AIC-201582_ref26","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-46241-6_6"},{"key":"10.3233\/AIC-201582_ref27","doi-asserted-by":"crossref","unstructured":"S.\u00a0Seele, R.\u00a0Herpers and C.\u00a0Bauckhage, Cognitive agents for microscopic traffic simulations in virtual environments, in: Proc. of ICEC 2012, M.\u00a0Herrlich, R.\u00a0Malaka and M.\u00a0Masuch, eds, LNCS, Vol.\u00a07522, 2012, pp.\u00a0318\u2013325.","DOI":"10.1007\/978-3-642-33542-6_27"},{"key":"10.3233\/AIC-201582_ref28","unstructured":"G.\u00a0Sharon, J.P.\u00a0Hanna, T.\u00a0Rambha, M.W.\u00a0Levin, M.\u00a0Albert, S.D.\u00a0Boyles and P.\u00a0Stone, Real-time adaptive tolling scheme for optimized social welfare in traffic networks, in: Proc. of the 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2017), S.\u00a0Das, E.\u00a0Durfee, K.\u00a0Larson and M.\u00a0Winikoff, eds, IFAAMAS, S\u00e3o Paulo, 2017, pp.\u00a0828\u2013836."},{"key":"10.3233\/AIC-201582_ref29","doi-asserted-by":"publisher","DOI":"10.1016\/B978-1-55860-307-3.50049-6"},{"key":"10.3233\/AIC-201582_ref30","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2014.6889438"},{"key":"10.3233\/AIC-201582_ref31","unstructured":"L.\u00a0Torrey and M.E.\u00a0Taylor, Teaching on a budget: Agents advising agents in reinforcement learning, in: Proceedings of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems, IFAAMAS, St. Paul, MN, USA, 2013, https:\/\/dl.acm.org\/doi\/10.5555\/2484920.2485086."},{"issue":"3","key":"10.3233\/AIC-201582_ref32","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1680\/ipeds.1952.11259","article-title":"Some theoretical aspects of road traffic research","volume":"1","author":"Wardrop","year":"1952","journal-title":"Proceedings of the Institution of Civil Engineers, Part\u00a0II"},{"issue":"3","key":"10.3233\/AIC-201582_ref33","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Machine Learning"},{"key":"10.3233\/AIC-201582_ref34","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1016\/j.trf.2019.05.007","article-title":"Driver profiling\u00a0\u2013 data-based identification of driver behavior dimensions and affecting driver characteristics for multi-agent traffic simulation","volume":"64","author":"Witt","year":"2019","journal-title":"Transportation Research Part\u00a0F: Traffic Psychology and Behaviour"},{"key":"10.3233\/AIC-201582_ref35","doi-asserted-by":"crossref","unstructured":"T.\u00a0Yamashita, K.\u00a0Izumi and K.\u00a0Kurumatani, Analysis of the effect of route information sharing on reduction of traffic congestion, in: Application of Agent Technology in Traffic and Transportation, F.\u00a0Kl\u00fcgl, A.L.C.\u00a0Bazzan and S.\u00a0Ossowski, eds, Birkh\u00e4user, 2004, pp.\u00a099\u2013112.","DOI":"10.1007\/3-7643-7363-6_7"},{"issue":"11","key":"10.3233\/AIC-201582_ref36","doi-asserted-by":"publisher","first-page":"712","DOI":"10.1287\/mnsc.17.11.712","article-title":"Finding the K shortest loopless paths in a network","volume":"17","author":"Yen","year":"1971","journal-title":"Management Science"},{"key":"10.3233\/AIC-201582_ref37","unstructured":"M.\u00a0Zimmer, P.\u00a0Viappiani and P.\u00a0Weng, Teacher\u2013student framework: A reinforcement learning approach, in: AAMAS Workshop Autonomous Robots and Multirobot Systems, Paris, France, 2014, https:\/\/hal.archives-ouvertes.fr\/hal-01215273."}],"container-title":["AI Communications"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/AIC-201582","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T18:27:58Z","timestamp":1777400878000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.medra.org\/servlet\/aliasResolver?alias=iospress&doi=10.3233\/AIC-201582"}},"subtitle":[],"editor":[{"given":"Marin","family":"Lujak","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]},{"given":"Ivana","family":"Dusparic","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]},{"given":"Franziska","family":"Kl\u00fcgl","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]},{"given":"Giuseppe","family":"Vizzari","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2021,2,15]]},"references-count":37,"journal-issue":{"issue":"1"},"URL":"https:\/\/doi.org\/10.3233\/aic-201582","relation":{},"ISSN":["1875-8452","0921-7126"],"issn-type":[{"value":"1875-8452","type":"electronic"},{"value":"0921-7126","type":"print"}],"subject":[],"published":{"date-parts":[[2021,2,15]]}}}