{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,14]],"date-time":"2026-02-14T10:26:26Z","timestamp":1771064786857,"version":"3.50.1"},"reference-count":30,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,2,12]],"date-time":"2020-02-12T00:00:00Z","timestamp":1581465600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the development of global urbanization, the Internet of Things (IoT) and smart cities are becoming hot research topics. As an emerging model, edge computing can play an important role in smart cities because of its low latency and good performance. IoT devices can reduce time consumption with the help of a mobile edge computing (MEC) server. However, if too many IoT devices simultaneously choose to offload the computation tasks to the MEC server via the limited wireless channel, it may lead to the channel congestion, thus increasing time overhead. Facing a large number of IoT devices in smart cities, the centralized resource allocation algorithm needs a lot of signaling exchange, resulting in low efficiency. To solve the problem, this paper studies the joint policy of communication and computing of IoT devices in edge computing through game theory, and proposes distributed Q-learning algorithms with two learning policies. Simulation results show that the algorithm can converge quickly with a balanced solution.<\/jats:p>","DOI":"10.3390\/s20040973","type":"journal-article","created":{"date-parts":[[2020,2,12]],"date-time":"2020-02-12T09:51:56Z","timestamp":1581501116000},"page":"973","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Distributed Learning Based Joint Communication and Computation Strategy of IoT Devices in Smart Cities"],"prefix":"10.3390","volume":"20","author":[{"given":"Tianyi","family":"Liu","sequence":"first","affiliation":[{"name":"International School, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ruyu","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Information and Telecommunication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fangmin","family":"Xu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Universal Wireless Communications, Ministry of Education, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chaoqiong","family":"Fan","sequence":"additional","affiliation":[{"name":"Key Laboratory of Universal Wireless Communications, Ministry of Education, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chenglin","family":"Zhao","sequence":"additional","affiliation":[{"name":"Key Laboratory of Universal Wireless Communications, Ministry of Education, Beijing University of Posts and Telecommunications, Beijing 100876, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,2,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Founoun, A., and Hayar, A. (2018, January 16\u201319). Evaluation of the concept of the smart city through local regulation and the importance of local initiative. Proceedings of the 2018 IEEE International Smart Cities Conference (ISC2), Kansas City, MO, USA.","DOI":"10.1109\/ISC2.2018.8656933"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Mora, O.B., Rivera, R., Larios, V.M., Beltran-Ramirez, J.R., Maciel, R., and Ochoa, A. (2018, January 16\u201319). A Use Case in Cybersecurity based in Blockchain to deal with the security and privacy of citizens and Smart Cities Cyberinfrastructures. Proceedings of the 2018 IEEE International Smart Cities Conference (ISC2), Kansas City, MO, USA.","DOI":"10.1109\/ISC2.2018.8656694"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.iot.2018.08.008","article-title":"Internet of things reference architectures, security and interoperability: A survey","volume":"1","author":"Martino","year":"2018","journal-title":"Internet Things"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.future.2019.04.008","article-title":"SWITCH workbench: A novel approach for the development and deployment of time-critical microservice-based cloud-native applications","volume":"99","author":"Cigale","year":"2019","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_5","first-page":"4","article-title":"Smart cities and cloud computing: Lessons from the STORM CLOUDS experiment","volume":"1","author":"Kakderi","year":"2016","journal-title":"J. Smart Cities"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"104405","DOI":"10.1109\/ACCESS.2019.2929683","article-title":"Data and Task Offloading in Collaborative Mobile Fog-Based Networks","volume":"7","author":"Ciobanu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_7","unstructured":"Patel, M. (2014). Mobile-Edge Computing-Introductory Technical White Paper, ETSI MEC White Paper, ETSI. Technical Report."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/MCOM.2017.1600249CM","article-title":"Mobile Edge Computing Potential in Making Cities Smarter","volume":"55","author":"Taleb","year":"2017","journal-title":"IEEE Commun. Mag."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Pradhan, M.A., Patankar, S., Shinde, A., Shivarkar, V., and Phadatare, P. (2017, January 1\u20132). IoT for smart city: Improvising smart environment. Proceedings of the 2017 International Conference on Energy, Communication, Data Analytics and Soft Computing (ICECDS 2017), Chennai, India.","DOI":"10.1109\/ICECDS.2017.8389800"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zeng, Z., Veeravalli, B., Khan, S.U., and Teo, S.G. (2017, January 11\u201313). Cloud-of-clouds based resource provisioning strategy for continuous write applications. Proceedings of the 2017 23rd Asia-Pacific Conference on Communications (APCC), Perth, Australia.","DOI":"10.23919\/APCC.2017.8304029"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"17741","DOI":"10.1109\/ACCESS.2018.2820679","article-title":"V2V Data Offloading for Cellular Network Based on the Software Defined Network (SDN) Inside Mobile Edge Computing (MEC) Architecture","volume":"6","author":"Huang","year":"2018","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Berno, M., Alcaraz, J.J., and Rossi, M. (2019, January 10\u201313). On the Allocation of Computing Tasks under QoS Constraints in Hierarchical MEC Architectures. Proceedings of the 2019 Fourth International Conference on Fog and Mobile Edge Computing (FMEC), Rome, Italy.","DOI":"10.1109\/FMEC.2019.8795345"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"6353","DOI":"10.1109\/TCOMM.2018.2866572","article-title":"Learning for Computation Offloading in Mobile Edge Computing","volume":"66","author":"Dinh","year":"2018","journal-title":"IEEE Trans. Commun."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2795","DOI":"10.1109\/TNET.2015.2487344","article-title":"Efficient Multi-User Computation Offloading for Mobile-Edge Cloud Computing","volume":"24","author":"Chen","year":"2016","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_15","unstructured":"Ranadheera, S., Maghsudi, S., and Hossain, E. (2017). Mobile Edge Computation Offloading Using Game Theory and Reinforcement Learning. arXiv."},{"key":"ref_16","first-page":"31","article-title":"Software-Defined Networks with Mobile Edge Computing and Caching for Smart Cities: A Big Data Deep Reinforcement Learning Approach","volume":"55","author":"He","year":"2017","journal-title":"IEEE Trans. Commun."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/MNET.2019.1800149","article-title":"HybridIoT: Integration of Hierarchical Multiple Access and Computation Offloading for IoT-Based Smart Cities","volume":"33","author":"Qian","year":"2019","journal-title":"IEEE Netw."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1109\/MCOM.2017.1600263","article-title":"Vehicular Social Networks: Enabling Smart Mobility","volume":"55","author":"Ning","year":"2017","journal-title":"IEEE Commun. Mag."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1016\/j.jpdc.2017.12.010","article-title":"Event-based sensor data exchange and fusion in the Internet of Things environments","volume":"118","author":"Esposito","year":"2018","journal-title":"J. Parallel Distrib. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Sapienza, M., Guardo, E., Cavallo, M., Torre, G.L., Leombruno, G., and Tomarchio, O. (2016, January 18\u201320). Solving Critical Events through Mobile Edge Computing: An Approach for Smart Cities. Proceedings of the 2016 IEEE International Conference on Smart Computing (SMARTCOMP), St. Louis, MO, USA.","DOI":"10.1109\/SMARTCOMP.2016.7501719"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/MNET.2019.1800221","article-title":"Optimal Edge Resource Allocation in IoT-Based Smart Cities","volume":"33","author":"Zhao","year":"2019","journal-title":"IEEE Network"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"14410","DOI":"10.1109\/ACCESS.2019.2893486","article-title":"Task Scheduling for Smart City Applications Based on Multi-Server Mobile Edge Computing","volume":"7","author":"Deng","year":"2019","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Yeoh, P.L., Pan, C., Wang, K., Elkashlan, M., Wang, Z., Vucetic, B., and Li, Y. (2019). Offloading Optimization for Low-Latency Secure Mobile Edge Computing Systems. IEEE Wirel. Commun. Lett., 1.","DOI":"10.1109\/TCOMM.2019.2947921"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4954","DOI":"10.1109\/TVT.2017.2750801","article-title":"Learning-Based Spectrum Sharing and Spatial Reuse in mm-Wave Ultradense Networks","volume":"67","author":"Fan","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_25","unstructured":"Han, Z., Niyato, D., Saad, W., Basar, T., and Hjorungnes, A. (2009). Game Theory in Wireless and Communication Networks, Cambridge University Press."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1006\/game.1996.0044","article-title":"Potential Games","volume":"14","author":"Monderer","year":"1996","journal-title":"Games Econ. Behav."},{"key":"ref_27","unstructured":"Claus, C., and Boutilier, C. (1998, January 26\u201330). The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems. Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-98), Madison, WI, USA."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Apt, K.R., and Simon, S. (2015). A Classification of Weakly Acyclic Games. arXiv.","DOI":"10.1007\/s11238-014-9436-1"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3154","DOI":"10.1137\/120893501","article-title":"Convergent Learning Algorithms for Unknown Reward Games","volume":"51","author":"Chapman","year":"2013","journal-title":"SIAM J. Control Optim."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"9272","DOI":"10.1109\/TVT.2016.2517700","article-title":"Power-Efficient Resource Allocation in a Heterogeneous Network With Cellular and D2D Capabilities","volume":"65","author":"Agusti","year":"2016","journal-title":"IEEE Trans. Veh. Technol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/4\/973\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T08:56:59Z","timestamp":1760173019000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/4\/973"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,12]]},"references-count":30,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,2]]}},"alternative-id":["s20040973"],"URL":"https:\/\/doi.org\/10.3390\/s20040973","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,12]]}}}