{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T20:18:34Z","timestamp":1773951514077,"version":"3.50.1"},"reference-count":47,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2020,7,31]],"date-time":"2020-07-31T00:00:00Z","timestamp":1596153600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With smart city infrastructures growing, the Internet of Things (IoT) has been widely used in the intelligent transportation systems (ITS). The traditional adaptive traffic signal control method based on reinforcement learning (RL) has expanded from one intersection to multiple intersections. In this paper, we propose a multi-agent auto communication (MAAC) algorithm, which is an innovative adaptive global traffic light control method based on multi-agent reinforcement learning (MARL) and an auto communication protocol in edge computing architecture. The MAAC algorithm combines multi-agent auto communication protocol with MARL, allowing an agent to communicate the learned strategies with others for achieving global optimization in traffic signal control. In addition, we present a practicable edge computing architecture for industrial deployment on IoT, considering the limitations of the capabilities of network transmission bandwidth. We demonstrate that our algorithm outperforms other methods over 17% in experiments in a real traffic simulation environment.<\/jats:p>","DOI":"10.3390\/s20154291","type":"journal-article","created":{"date-parts":[[2020,8,3]],"date-time":"2020-08-03T06:16:47Z","timestamp":1596435407000},"page":"4291","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["An Edge Based Multi-Agent Auto Communication Method for Traffic Light Control"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0655-0479","authenticated-orcid":false,"given":"Qiang","family":"Wu","sequence":"first","affiliation":[{"name":"School of Information &amp; Engineering, Lanzhou University, Lanzhou 730000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7198-4199","authenticated-orcid":false,"given":"Jianqing","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Computing and Information Technology, University of Wollongong, Wollongong 2522, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9403-7140","authenticated-orcid":false,"given":"Jun","family":"Shen","sequence":"additional","affiliation":[{"name":"School of Computing and Information Technology, University of Wollongong, Wollongong 2522, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Binbin","family":"Yong","sequence":"additional","affiliation":[{"name":"School of Information &amp; Engineering, Lanzhou University, Lanzhou 730000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qingguo","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Information &amp; Engineering, Lanzhou University, Lanzhou 730000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,7,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1061\/(ASCE)0733-947X(2003)129:3(278)","article-title":"Reinforcement learning for true adaptive traffic signal control","volume":"129","author":"Abdulhai","year":"2003","journal-title":"J. Transp. Eng."},{"key":"ref_2","first-page":"285","article-title":"Reinforcement Learning: An Introduction","volume":"16","author":"Richard","year":"2005","journal-title":"MIT Press"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ghazal, B., ElKhatib, K., Chahine, K., and Kherfan, M. (2016, January 21\u201323). Smart traffic light control system. Proceedings of the 2016 3rd International Conference on Electrical, Electronics, Computer Engineering and their Applications (EECEA), Beirut, Lebanon.","DOI":"10.1109\/EECEA.2016.7470780"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wei, H., Zheng, G., Yao, H., and Li, Z. (2018). IntelliLight: A Reinforcement Learning Approach for Intelligent Traffic Light Control, Association for Computing Machinery.","DOI":"10.1145\/3219819.3220096"},{"key":"ref_5","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_6","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (July, January 26). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_7","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_8","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","article-title":"Deep neural networks for acoustic modeling in speech recognition","volume":"29","author":"Hinton","year":"2012","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_10","unstructured":"Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv."},{"key":"ref_11","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_12","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., and Gomez, A.N. (2017, January 4\u20139). Attention is All you Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_13","first-page":"506","article-title":"Arrhythmia recognition and classification through deep learning-based approach","volume":"19","author":"Zhou","year":"2019","journal-title":"Int. J. Comput. Sci. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"825","DOI":"10.1016\/j.future.2019.02.058","article-title":"Smart fog based workflow for traffic control networks","volume":"97","author":"Wu","year":"2019","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"522","DOI":"10.1504\/IJES.2020.107631","article-title":"A novel Monte Carlo-based neural network model for electricity load forecasting","volume":"12","author":"Yong","year":"2020","journal-title":"Int. J. Embed. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1007\/s10458-008-9062-9","article-title":"Opportunities for multi-agent systems and multi-agent reinforcement learning in traffic control","volume":"18","author":"Bazzan","year":"2009","journal-title":"Auton. Agents Multi-Agent Syst."},{"key":"ref_18","unstructured":"Sukhbaatar, S., and Fergus, R. (2016). Learning multiagent communication with backpropagation. NIPS, 2244\u20132252."},{"key":"ref_19","unstructured":"Hoshen, Y. (2017). Attentional multi-agent predictive modeling. Neural Inf. Process. Syst. (NIPS), 2701\u20132711."},{"key":"ref_20","first-page":"45","article-title":"Traffic signal settings","volume":"39","author":"Webster","year":"1958","journal-title":"H.M. Station. Off."},{"key":"ref_21","unstructured":"Thorpe, T.L. (1997). Vehicle Traffic Light Control Using SARSA, Colorado State University."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3390","DOI":"10.1016\/j.apm.2010.02.028","article-title":"An efficient algorithm for computing traffic equilibria using TRANSYT model","volume":"34","author":"Chiou","year":"2010","journal-title":"Appl. Math. Model."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1007\/BF00155579","article-title":"Trends in distributed artificial intelligence","volume":"6","author":"Moulin","year":"1992","journal-title":"Artif. Intell. Rev."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1016\/B978-1-55860-092-8.50014-9","article-title":"Negotiating task decomposition and allocation using partial global planning","volume":"2","author":"Durfee","year":"1989","journal-title":"Distrib. Artif. Intell."},{"key":"ref_25","unstructured":"Minsky, M. (2007). The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind, Simon & Schuster."},{"key":"ref_26","unstructured":"Peng, P., Yuan, Q., Wen, Y., Yang, Y., Tang, Z., Long, H., and Wang, J. (2017). Multiagent Bidirectionally-Coordinated Nets for Learning to Play StarCraft Combat Games. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3414","DOI":"10.1609\/aaai.v34i04.5744","article-title":"Toward a Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control","volume":"34","author":"Chen","year":"2020","journal-title":"AAAI"},{"key":"ref_28","first-page":"412","article-title":"Reinforcement learning with function approximation for traffic signal control","volume":"12","author":"Prashanth","year":"2010","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1049\/iet-its.2017.0153","article-title":"Traffic light control using deep policy-gradient and value-function-based reinforcement learning","volume":"11","author":"Mousavi","year":"2017","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"89","DOI":"10.3328\/TL.2010.02.02.89-110","article-title":"Towards multi-agent reinforcement learning for integrated network of optimal traffic controllers (MARLIN-OTC)","volume":"2","author":"El","year":"2010","journal-title":"Transp. Lett."},{"key":"ref_31","unstructured":"Weinberg, M., and Rosenschein, J.S. (2004, January 19\u201323). Best-response multiagent learning in non-stationary environments. Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multiagent Systems, New York, NY, USA."},{"key":"ref_32","first-page":"2048","article-title":"Show, Attend and Tell: Neural Image Caption Generation with Visual Attention","volume":"37","author":"Xu","year":"2015","journal-title":"Comput. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Luong, M.T., Pham, H., and Manning, C.D. (2015). Effective Approaches to Attention-based Neural Machine Translation. arXiv.","DOI":"10.18653\/v1\/D15-1166"},{"key":"ref_34","unstructured":"Pottie, G.J. (1998, January 22\u201326). Wireless sensor networks. Proceedings of the 1998 Information Theory Workshop (Cat. No.98EX131), Killarney, Ireland."},{"key":"ref_35","first-page":"36","article-title":"5th-Generation Mobile Communication: Data Highway for Surgery 4.0","volume":"35","author":"Jell","year":"2019","journal-title":"Surg. Technol. Int."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s10796-014-9492-7","article-title":"The internet of things: A survey","volume":"17","author":"Atzori","year":"2015","journal-title":"Inf. Syst. Front."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1855","DOI":"10.3390\/s20071855","article-title":"Multi-Cell LTE-U\/Wi-Fi Coexistence Evaluation Using a Reinforcement Learning Framework","volume":"20","author":"Neto","year":"2020","journal-title":"Sensors"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Faheem, M., Butt, R.A., Raza, B., Alquhayz, H., Ashraf, M.W., Shah, S.B., Ngadi, M.A., and Gungor, V.C. (2019). A Cross-Layer QoS Channel-Aware Routing Protocol for the Internet of Underwater Acoustic Sensor Networks. Sensors, 19.","DOI":"10.3390\/s19214762"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Zikria, Y.B., Afzal, M.K., and Kim, S.W. (2020). Internet of Multimedia Things (IoMT): Opportunities, Challenges and Solution. Sensors, 20.","DOI":"10.3390\/s20082334"},{"key":"ref_40","first-page":"279","article-title":"IoT-based intelligent fitness system","volume":"8","author":"Yong","year":"2017","journal-title":"J. Parallel Distrib. Comput."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A Mathematical Theory of Communication","volume":"27","author":"Shannon","year":"1963","journal-title":"Bell Syst. Tech. J."},{"key":"ref_42","unstructured":"Foerster, A. (2016). Learning to communicate with deep multi-agent reinforcement learning. NIPS, 2145\u20132153."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1147\/rd.224.0393","article-title":"General Technique for Communications Protocol Validation","volume":"22","author":"West","year":"1978","journal-title":"IBM J. Res. Dev."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Wei, H., Chen, C., Zheng, G., Wu, K., Xu, K., Gayah, V., and Li, Z. (2019). Presslight: Learning max pressure control for signalized intersections in arterial network. Int. Conf. Knowl. Discov. Data Min. (KDD), 1290\u20131298.","DOI":"10.1145\/3292500.3330949"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1145\/2723872.2723882","article-title":"An introduction to Docker for reproducible research","volume":"49","author":"Boettiger","year":"2015","journal-title":"ACM SIGOPS Oper. Syst. Rev."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_47","first-page":"1039","article-title":"Nash q-learning for general-sum stochastic games","volume":"4","author":"Hu","year":"2004","journal-title":"J. Mach. Learn. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/15\/4291\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:53:26Z","timestamp":1760176406000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/15\/4291"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,31]]},"references-count":47,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2020,8]]}},"alternative-id":["s20154291"],"URL":"https:\/\/doi.org\/10.3390\/s20154291","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,31]]}}}