{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T16:56:59Z","timestamp":1782320219638,"version":"3.54.5"},"reference-count":47,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,2,17]],"date-time":"2024-02-17T00:00:00Z","timestamp":1708128000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the advent of IoT, cities will soon be populated by autonomous vehicles and managed by intelligent systems capable of actively interacting with city infrastructures and vehicles. In this work, we propose a model based on reinforcement learning that teaches to autonomous connected vehicles how to save resources while navigating in such an environment. In particular, we focus on budget savings in the context of auction-based intersection management systems. We trained several models with Deep Q-learning by varying traffic conditions to find the most performance-effective variant in terms of the trade-off between saved currency and trip times. Afterward, we compared the performance of our model with previously proposed and random strategies, even under adverse traffic conditions. Our model appears to be robust and manages to save a considerable amount of currency without significantly increasing the waiting time in traffic. For example, the learner bidder saves at least 20% of its budget with heavy traffic conditions and up to 74% in lighter traffic with respect to a standard bidder, and around three times the saving of a random bidder. The results and discussion suggest practical adoption of the proposal in a foreseen future real-life scenario.<\/jats:p>","DOI":"10.3390\/s24041288","type":"journal-article","created":{"date-parts":[[2024,2,19]],"date-time":"2024-02-19T03:18:38Z","timestamp":1708312718000},"page":"1288","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Learn to Bet: Using Reinforcement Learning to Improve Vehicle Bids in Auction-Based Smart Intersections"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4942-2453","authenticated-orcid":false,"given":"Giacomo","family":"Cabri","sequence":"first","affiliation":[{"name":"Department of Physics, Informatics and Mathematics, University of Modena e Reggio Emilia, 41125 Modena, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matteo","family":"Lugli","sequence":"additional","affiliation":[{"name":"Department of Physics, Informatics and Mathematics, University of Modena e Reggio Emilia, 41125 Modena, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8827-5495","authenticated-orcid":false,"given":"Manuela","family":"Montangero","sequence":"additional","affiliation":[{"name":"Department of Physics, Informatics and Mathematics, University of Modena e Reggio Emilia, 41125 Modena, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6523-0366","authenticated-orcid":false,"given":"Filippo","family":"Muzzini","sequence":"additional","affiliation":[{"name":"Department of Physics, Informatics and Mathematics, University of Modena e Reggio Emilia, 41125 Modena, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"429","DOI":"10.3390\/smartcities4020024","article-title":"IoT in smart cities: A survey of technologies, practices and challenges","volume":"4","author":"Syed","year":"2021","journal-title":"Smart Cities"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bertogna, M., Burgio, P., Cabri, G., and Capodieci, N. (2017, January 21\u201323). Adaptive coordination in autonomous driving: Motivations and perspectives. Proceedings of the 2017 IEEE 26th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE), Poznan, Poland.","DOI":"10.1109\/WETICE.2017.45"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"19954","DOI":"10.1109\/TITS.2022.3182410","article-title":"A review of vision-based traffic semantic understanding in ITSs","volume":"23","author":"Chen","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_4","unstructured":"(2018). Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. Standard No. J3016_202104."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Carlino, D., Boyles, S.D., and Stone, P. (2013, January 6\u20139). Auction-based autonomous intersection management. Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013), The Hague, The Netherlands.","DOI":"10.1109\/ITSC.2013.6728285"},{"key":"ref_6","first-page":"1","article-title":"Coordination of autonomous vehicles: Taxonomy and survey","volume":"54","author":"Mariani","year":"2021","journal-title":"ACM Comput. Surv. CSUR"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.image.2018.07.007","article-title":"Suitability of recent hardware accelerators (DSPs, FPGAs, and GPUs) for computer vision and image processing algorithms","volume":"68","author":"HajiRassouliha","year":"2018","journal-title":"Signal Process. Image Commun."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"14410","DOI":"10.1109\/ACCESS.2019.2893486","article-title":"Task Scheduling for Smart City Applications Based on Multi-Server Mobile Edge Computing","volume":"7","author":"Deng","year":"2019","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/MC.2016.376","article-title":"Swarm-Oriented Programming of Distributed Robot Networks","volume":"49","author":"Pinciroli","year":"2016","journal-title":"Computer"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Murthy, D.K., and Masrur, A. (September, January 31). Braking in Close Following Platoons: The Law of the Weakest. Proceedings of the 2016 Euromicro Conference on Digital System Design (DSD), Limassol, Cyprus.","DOI":"10.1109\/DSD.2016.78"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Diaz Og\u00e1s, M.G., Fabregat, R., and Aciar, S. (2020). Survey of smart parking systems. Appl. Sci., 10.","DOI":"10.3390\/app10113872"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/MITS.2017.2666586","article-title":"Smart parking guidance, monitoring and reservations: A review","volume":"9","author":"Kotb","year":"2017","journal-title":"IEEE Intell. Transp. Syst. Mag."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Tandon, R., and Gupta, P. (2019, January 12\u201313). Optimizing smart parking system by using fog computing. Proceedings of the Advances in Computing and Data Sciences: Third International Conference, ICACDS 2019, Ghaziabad, India. Revised Selected Papers, Part II 3.","DOI":"10.1007\/978-981-13-9942-8_67"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Khanna, A., and Anand, R. (2016, January 22\u201324). IoT based smart parking system. Proceedings of the 2016 International Conference on Internet of Things and Applications (IOTA), Pune, India.","DOI":"10.1109\/IOTA.2016.7562735"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2637","DOI":"10.1109\/TITS.2016.2531636","article-title":"iParker\u2014A new smart car-parking system based on dynamic resource allocation and pricing","volume":"17","author":"Kotb","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Sadhukhan, P. (2017, January 13\u201316). An IoT-based E-parking system for smart cities. Proceedings of the 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Udupi, India.","DOI":"10.1109\/ICACCI.2017.8125982"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1581","DOI":"10.1109\/ACCESS.2015.2477299","article-title":"A Cloud-Based Smart-Parking System Based on Internet-of-Things Technologies","volume":"3","author":"Pham","year":"2015","journal-title":"IEEE Access"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"522","DOI":"10.1080\/17445760.2023.2246166","article-title":"Improving urban viability through smart parking","volume":"38","author":"Muzzini","year":"2023","journal-title":"Int. J. Parallel Emergent Distrib. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1109\/MITS.2020.2970185","article-title":"Limited sensing and deep data mining: A new exploration of developing city-wide parking guidance systems","volume":"14","author":"Zou","year":"2020","journal-title":"IEEE Intell. Transp. Syst. Mag."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1016\/j.icte.2017.11.003","article-title":"A zone-based traffic assignment algorithm for scalable congestion reduction","volume":"3","author":"Cox","year":"2017","journal-title":"ICT Express"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1016\/j.icte.2021.03.005","article-title":"Improving emergency response in the era of ADAS vehicles in the Smart City","volume":"7","author":"Capodieci","year":"2021","journal-title":"ICT Express"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.procs.2020.03.012","article-title":"A multi-dimensional rescheduling model in disrupted transport network using rule-based decision making","volume":"170","author":"Li","year":"2020","journal-title":"Procedia Comput. Sci."},{"key":"ref_23","unstructured":"Schepperle, H., and B\u00f6hm, K. (2007, January 19\u201321). Agent-based traffic control using auctions. Proceedings of the International Workshop on Cooperative Information Agents, Delft, The Netherlands."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1111\/j.1540-6261.1961.tb02789.x","article-title":"Counterspeculation, auctions, and competitive sealed tenders","volume":"16","author":"Vickrey","year":"1961","journal-title":"J. Financ."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1613\/jair.3560","article-title":"A market-inspired approach for intersection management in urban road traffic networks","volume":"43","author":"Vasirani","year":"2012","journal-title":"J. Artif. Intell. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4909","DOI":"10.1109\/TITS.2021.3054625","article-title":"Deep reinforcement learning for autonomous driving: A survey","volume":"23","author":"Kiran","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"21163","DOI":"10.1109\/TITS.2022.3187016","article-title":"Coordination control strategy for human-machine cooperative steering of intelligent vehicles: A reinforcement learning approach","volume":"23","author":"Xie","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1145\/3447556.3447565","article-title":"Recent advances in reinforcement learning for traffic signal control: A survey of models and evaluation","volume":"22","author":"Wei","year":"2021","journal-title":"ACM SIGKDD Explor. Newsl."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Glorio, N., Mariani, S., Cabri, G., and Zambonelli, F. (2021, January 27\u201329). An Adaptive Approach for the Coordination of Autonomous Vehicles at Intersections. Proceedings of the 2021 IEEE 30th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE), Bayonne, France.","DOI":"10.1109\/WETICE53228.2021.00012"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","article-title":"Reinforcement learning: A survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"J. Artif. Intell. Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1016\/j.comcom.2020.03.005","article-title":"Traffic signal control for smart cities using reinforcement learning","volume":"154","author":"Joo","year":"2020","journal-title":"Comput. Commun."},{"key":"ref_32","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Karthikeyan, P., Chen, W.L., and Hsiung, P.A. (2022). Autonomous Intersection Management by Using Reinforcement Learning. Algorithms, 15.","DOI":"10.3390\/a15090326"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Antonio, G.P., and Maria-Dolores, C. (2022). AIM5la: A latency-aware deep reinforcement learning-based autonomous intersection management system for 5G communication networks. Sensors, 22.","DOI":"10.3390\/s22062217"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Mushtaq, A., Haq, I.U., Sarwar, M.A., Khan, A., Khalil, W., and Mughal, M.A. (2023). Multi-Agent Reinforcement Learning for Traffic Flow Management of Autonomous Vehicles. Sensors, 23.","DOI":"10.3390\/s23052373"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Shi, Y., Liu, Y., Qi, Y., and Han, Q. (2022). A control method with reinforcement learning for urban un-signalized intersection in hybrid traffic environment. Sensors, 22.","DOI":"10.3390\/s22030779"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Guti\u00e9rrez-Moreno, R., Barea, R., L\u00f3pez-Guill\u00e9n, E., Araluce, J., and Bergasa, L.M. (2022). Reinforcement learning-based autonomous driving at intersections in CARLA simulator. Sensors, 22.","DOI":"10.3390\/s22218373"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"15921","DOI":"10.1007\/s11042-020-10222-y","article-title":"About auction strategies for intersection management when human-driven and autonomous vehicles coexist","volume":"80","author":"Cabri","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_39","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_41","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Lopez, P.A., Behrisch, M., Bieker-Walz, L., Erdmann, J., Fl\u00f6tter\u00f6d, Y.P., Hilbrich, R., L\u00fccken, L., Rummel, J., Wagner, P., and Wie\u00dfner, E. (2018, January 4\u20137). Microscopic Traffic Simulation using SUMO. Proceedings of the 21st IEEE International Conference on Intelligent Transportation Systems, Maui, HI, USA.","DOI":"10.1109\/ITSC.2018.8569938"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Ejercito, P.M., Nebrija, K.G.E., Feria, R.P., and Lara-Figueroa, L.L. (2017, January 27\u201330). Traffic simulation software review. Proceedings of the 2017 8th International Conference on Information, Intelligence, Systems & Applications (IISA), Larnaca, Cyprus.","DOI":"10.1109\/IISA.2017.8316415"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Axhausen, K.W., Horni, A., and Nagel, K. (2016). The Multi-Agent Transport Simulation MATSim, Ubiquity Press.","DOI":"10.5334\/baw"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"05002","DOI":"10.1051\/matecconf\/20168105002","article-title":"A comparative study of urban road traffic simulators","volume":"81","author":"Saidallah","year":"2016","journal-title":"MATEC Web Conf."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Diallo, A.O., Lozenguez, G., Doniec, A., and Mandiau, R. (2021, January 4\u20136). Comparative evaluation of road traffic simulators based on modeler\u2019s specifications: An application to intermodal mobility behaviors. Proceedings of the 13th International Conference on Agents and Artificial Intelligence, Virtual.","DOI":"10.5220\/0010238302650272"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Gherardini, L., Cabri, G., and Montangero, M. (2022). Decentralized approaches for autonomous vehicles coordination. Internet Technol. Lett., e398.","DOI":"10.1002\/itl2.398"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/4\/1288\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:01:11Z","timestamp":1760104871000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/4\/1288"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,17]]},"references-count":47,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,2]]}},"alternative-id":["s24041288"],"URL":"https:\/\/doi.org\/10.3390\/s24041288","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,17]]}}}