{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:01:42Z","timestamp":1760144502492,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2024,4,28]],"date-time":"2024-04-28T00:00:00Z","timestamp":1714262400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Key Research and Development Project in the Hubei Province of China","award":["2022BCA035"],"award-info":[{"award-number":["2022BCA035"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, we consider a low-latency Mobile Edge Computing (MEC) network where multiple User Equipment (UE) wirelessly reports to a decision-making edge server. At the same time, the transmissions are operated with Finite Blocklength (FBL) codes to achieve low-latency transmission. We introduce the task of Age upon Decision (AuD) aimed at the timeliness of tasks used for decision-making, which highlights the timeliness of the information at decision-making moments. For the case in which dynamic task generation and random fading channels are considered, we provide a task AuD minimization design by jointly selecting UE and allocating blocklength. In particular, to solve the task AuD minimization problem, we transform the optimization problem to a Markov Decision Process problem and propose an Error Probability-Controlled Action-Masked Proximal Policy Optimization (EMPPO) algorithm. Via simulation, we show that the proposed design achieves a lower AuD than baseline methods across various network conditions, especially in scenarios with significant channel Signal-to-Noise Ratio (SNR) differences and low average SNR, which shows the robustness of EMPPO and its potential for real-time applications.<\/jats:p>","DOI":"10.3390\/s24092812","type":"journal-article","created":{"date-parts":[[2024,4,29]],"date-time":"2024-04-29T08:49:24Z","timestamp":1714380564000},"page":"2812","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Minimizing Task Age upon Decision for Low-Latency MEC Networks Task Offloading with Action-Masked Deep Reinforcement Learning"],"prefix":"10.3390","volume":"24","author":[{"given":"Zhouxi","family":"Jiang","sequence":"first","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2002-9073","authenticated-orcid":false,"given":"Jianfeng","family":"Yang","sequence":"additional","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3344-1860","authenticated-orcid":false,"given":"Xun","family":"Gao","sequence":"additional","affiliation":[{"name":"Electronic Information School, Wuhan University, Wuhan 430072, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11432-020-2955-6","article-title":"Towards 6G wireless communication networks: Vision, enabling technologies, and new paradigm shifts","volume":"64","author":"You","year":"2021","journal-title":"Sci. China Inf. Sci."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Al-Ansi, A., Al-Ansi, A.M., Muthanna, A., Elgendy, I.A., and Koucheryavy, A. (2021). Survey on intelligence edge computing in 6G: Characteristics, challenges, potential use cases, and market drivers. Future Internet, 13.","DOI":"10.3390\/fi13050118"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Javed, F., Khan, Z.A., Rizwan, S., Shahzadi, S., Chaudhry, N.R., and Iqbal, M. (2023). A Novel Energy-Efficient Reservation System for Edge Computing in 6G Vehicular Ad Hoc Network. Sensors, 23.","DOI":"10.3390\/s23135817"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1186\/s13638-022-02181-9","article-title":"Dynamic visual SLAM and MEC technologies for B5G: A comprehensive review","volume":"2022","author":"Peng","year":"2022","journal-title":"Eurasip J. Wirel. Commun. Netw."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3008","DOI":"10.1109\/JSAC.2023.3310087","article-title":"Distributed Communication and Computation Resource Management for Digital Twin-Aided Edge Computing With Short-Packet Communications","volume":"41","author":"Nguyen","year":"2023","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1019","DOI":"10.1109\/TCCN.2022.3149089","article-title":"Delay optimization in mobile edge computing: Cognitive UAV-assisted eMBB and mMTC services","volume":"8","author":"Sabuj","year":"2022","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"624","DOI":"10.1109\/TCCN.2020.3018159","article-title":"Collaborative cloud-edge-end task offloading in mobile-edge computing networks with limited communication capability","volume":"7","author":"Kai","year":"2020","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1109\/TWC.2020.3024538","article-title":"Mobility-aware joint task scheduling and resource allocation for cooperative mobile edge computing","volume":"20","author":"Saleem","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3341","DOI":"10.1109\/TVT.2020.2966500","article-title":"Mobility-aware multi-user offloading optimization for mobile edge computing","volume":"69","author":"Zhan","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"6058","DOI":"10.1109\/TVT.2021.3073791","article-title":"Reliability-Optimal Offloading in Low-Latency Edge Computing Networks: Analytical and Reinforcement Learning Based Designs","volume":"70","author":"Zhu","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3252","DOI":"10.1109\/JSAC.2021.3088657","article-title":"Let\u2019s trade in the future! A futures-enabled fast resource trading mechanism in edge computing-assisted UAV networks","volume":"39","author":"Liwang","year":"2021","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"5467","DOI":"10.1109\/TWC.2022.3141094","article-title":"Unifying futures and spot market: Overbooking-enabled resource trading in mobile edge networks","volume":"21","author":"Liwang","year":"2022","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2307","DOI":"10.1109\/TIT.2010.2043769","article-title":"Channel coding rate in the finite blocklength regime","volume":"56","author":"Polyanskiy","year":"2010","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1109\/TSIPN.2020.2981266","article-title":"Dynamic computation offloading in multi-access edge computing via ultra-reliable and low-latency communications","volume":"6","author":"Merluzzi","year":"2020","journal-title":"IEEE Trans. Signal Inf. Process. Over Netw."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3298","DOI":"10.1109\/TWC.2023.3307154","article-title":"Latency Minimization for UAV-Enabled URLLC-Based Mobile Edge Computing Systems","volume":"23","author":"Wu","year":"2024","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_16","first-page":"1","article-title":"Mobile Edge Computing aided Integrated Sensing and Communication with Short-Packet Transmissions","volume":"99","author":"Huang","year":"2023","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1186\/s13638-021-01905-7","article-title":"Energy-efficient offloading and resource allocation for mobile edge computing enabled mission-critical internet-of-things systems","volume":"2021","author":"Fu","year":"2021","journal-title":"Eurasip J. Wirel. Commun. Netw."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3006","DOI":"10.1109\/TWC.2022.3215450","article-title":"Low-Latency Hybrid NOMA-TDMA: QoS-Driven Design Framework","volume":"22","author":"Zhu","year":"2023","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"9261","DOI":"10.1109\/TWC.2022.3175386","article-title":"Latency-Critical Downlink Multiple Access: A Hybrid Approach and Reliability Maximization","volume":"21","author":"Yuan","year":"2023","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"7105","DOI":"10.1109\/TWC.2022.3154670","article-title":"Energy minimization of mobile edge computing networks with HARQ in the finite blocklength regime","volume":"21","author":"Zhu","year":"2022","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Yang, T., Hu, Y., Gursoy, M.C., Schmeink, A., and Mathar, R. (2018, January 28\u201331). Deep reinforcement learning based resource allocation in low latency edge computing networks. Proceedings of the 2018 15th International Symposium on Wireless Communication Systems (ISWCS), Lisbon, Portugal.","DOI":"10.1109\/ISWCS.2018.8491089"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Kaul, S., Yates, R., and Gruteser, M. (2012, January 25\u201330). Real-time status: How often should one update?. Proceedings of the 2012 IEEE INFOCOM, Orlando, FL, USA.","DOI":"10.1109\/INFCOM.2012.6195689"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1109\/LWC.2021.3139346","article-title":"Age of information analysis for finite blocklength regime in downlink cellular networks","volume":"11","author":"Sung","year":"2021","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"7903","DOI":"10.1109\/TWC.2021.3088719","article-title":"Fairness for freshness: Optimal age of information based OFDMA scheduling with minimal knowledge","volume":"20","author":"Han","year":"2021","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_25","first-page":"1911","article-title":"Optimized age of information tail for ultra-reliable low-latency communications in vehicular networks","volume":"68","author":"Samarakoon","year":"2019","journal-title":"IEEE Trans. Commun."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"7236","DOI":"10.1109\/TCOMM.2021.3102624","article-title":"Information age-delay correlation and optimization with finite block length","volume":"69","author":"Cao","year":"2021","journal-title":"IEEE Trans. Commun."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1585","DOI":"10.1109\/LWC.2022.3166562","article-title":"Average age of information in wireless powered mobile edge computing system","volume":"11","author":"Liu","year":"2022","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"9444","DOI":"10.1109\/JIOT.2023.3323291","article-title":"Minimizing Age-of-Information with Joint Transmission and Computing Scheduling in Mobile Edge Computing","volume":"11","author":"Zhu","year":"2024","journal-title":"IEEE Internet Things J."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"19782","DOI":"10.1109\/JIOT.2023.3283287","article-title":"Age of Information-Based Computation Offloading and Transmission Scheduling in Mobile Edge Computing-Enabled IoT Networks","volume":"10","author":"Jiang","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1600","DOI":"10.1109\/OJCOMS.2023.3294942","article-title":"Age of Information of Multi-User Mobile-Edge Computing Systems","volume":"4","author":"Tang","year":"2023","journal-title":"IEEE Open J. Commun. Soc."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Dong, Y., Chen, Z., Liu, S., and Fan, P. (2018, January 24\u201325). Age of information upon decisions. Proceedings of the 2018 IEEE 39th Sarnoff Symposium, Nejark, NJ, USA.","DOI":"10.1109\/SARNOF.2018.8720507"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1081","DOI":"10.1109\/JIOT.2019.2950054","article-title":"Age-Upon-Decisions Minimizing Scheduling in Internet of Things: To Be Random or To Be Deterministic?","volume":"7","author":"Dong","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2826","DOI":"10.1109\/JIOT.2020.3020875","article-title":"Age-Optimal Service and Decision Processes in Internet of Things","volume":"8","author":"Bao","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1016\/j.comcom.2023.07.010","article-title":"Average age upon decisions with truncated HARQ and optimization in the finite blocklength regime","volume":"209","author":"Bao","year":"2023","journal-title":"Comput. Commun."},{"key":"ref_35","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_36","unstructured":"Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015, January 6\u201311). Trust region policy optimization. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Huang, S., and Onta\u00f1\u00f3n, S. (2022, January 15\u201318). A Closer Look at Invalid Action Masking in Policy Gradient Algorithms. Proceedings of the The International FLAIRS Conference, Jensen Beach, FL, USA.","DOI":"10.32473\/flairs.v35i.130584"},{"key":"ref_38","first-page":"1","article-title":"Stable-Baselines3: Reliable Reinforcement Learning Implementations","volume":"22","author":"Raffin","year":"2021","journal-title":"J. Mach. Learn. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/9\/2812\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:35:08Z","timestamp":1760106908000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/9\/2812"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,28]]},"references-count":38,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2024,5]]}},"alternative-id":["s24092812"],"URL":"https:\/\/doi.org\/10.3390\/s24092812","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2024,4,28]]}}}