{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T04:29:42Z","timestamp":1772166582738,"version":"3.50.1"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T00:00:00Z","timestamp":1638316800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,12,11]],"date-time":"2021-12-11T00:00:00Z","timestamp":1639180800000},"content-version":"vor","delay-in-days":10,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Basic Scientific Research Project of Heilongjiang Province","award":["2020-KYYWF-1003"],"award-info":[{"award-number":["2020-KYYWF-1003"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["EURASIP J. Adv. Signal Process."],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>To meet the demands of massive connections in the Internet-of-vehicle communications, non-orthogonal multiple access (NOMA) is utilized in the local wireless networks. In NOMA technique, various optimization methods have been proposed to provide optimal resource allocation, but they are limited by computational complexity. Recently, the deep reinforcement learning network is utilized for resource optimization in NOMA system, where a uniform sampled experience replay algorithm is used to reduce the correlation between samples. However, the uniform sampling ignores the importance of sample. To this point, this paper proposes a joint prioritized DQN user grouping and DDPG power allocation algorithm to maximize the system sum rate. At the user grouping stage, a prioritized sampling method based on TD-error (temporal-difference error) is proposed. At the power allocation stage, to deal with the problem that DQN cannot process continuous tasks and needs to quantify power into discrete form, a DDPG network is utilized. Simulation results show that the proposed algorithm with prioritized sampling can increase the learning rate and perform a more stable training process. Compared with the previous DQN algorithm, the proposed method improves the sum rate of the system by 2% and reaches 94% and 93% of the exhaustive search algorithm and optimal iterative power optimization algorithm, respectively. Although the sum rate is improved by only 2%, the computational complexity is reduced by 43% and 64% compared to the exhaustive search algorithm and the optimal iterative power optimization algorithm, respectively.<\/jats:p>","DOI":"10.1186\/s13634-021-00828-1","type":"journal-article","created":{"date-parts":[[2021,12,11]],"date-time":"2021-12-11T15:02:28Z","timestamp":1639234948000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["NOMA resource allocation method in IoV based on prioritized DQN-DDPG network"],"prefix":"10.1186","volume":"2021","author":[{"given":"Mengli","family":"He","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8880-9773","authenticated-orcid":false,"given":"Yue","family":"Li","sequence":"additional","affiliation":[]},{"given":"Xiaofei","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Zelong","family":"Liu","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,12,11]]},"reference":[{"key":"828_CR1","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2021.3110942","author":"WU Khan","year":"2021","unstructured":"W.U. Khan, M.A. Javed, T.N. Nguyen et al., Energy-efficient resource allocation for 6G backscatter-enabled NOMA IoV networks. IEEE Trans. Intell. Transp. Syst. (2021). https:\/\/doi.org\/10.1109\/TITS.2021.3110942","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"issue":"4","key":"828_CR2","doi-asserted-by":"publisher","first-page":"5962","DOI":"10.1109\/JIOT.2018.2847731","volume":"6","author":"X Liu","year":"2018","unstructured":"X. Liu, M. Jia, X. Zhang et al., A novel multichannel Internet of things based on dynamic spectrum sharing in 5G communication. IEEE Internet Things J. 6(4), 5962\u20135970 (2018)","journal-title":"IEEE Internet Things J."},{"issue":"4","key":"828_CR3","doi-asserted-by":"publisher","first-page":"5971","DOI":"10.1109\/JIOT.2018.2863267","volume":"6","author":"X Liu","year":"2018","unstructured":"X. Liu, X. Zhang, Rate and energy efficiency improvements for 5G-based IoT with simultaneous transfer. IEEE Internet Things J. 6(4), 5971\u20135980 (2018)","journal-title":"IEEE Internet Things J."},{"key":"828_CR4","doi-asserted-by":"publisher","first-page":"130","DOI":"10.1016\/j.phycom.2018.03.015","volume":"28","author":"X Liu","year":"2018","unstructured":"X. Liu, X. Zhang, M. Jia et al., 5G-based green broadband communication system design with simultaneous wireless information and power transfer. Phys. Commun. 28, 130\u2013137 (2018)","journal-title":"Phys. Commun."},{"issue":"2","key":"828_CR5","doi-asserted-by":"publisher","first-page":"1480","DOI":"10.1109\/TCOMM.2018.2877671","volume":"67","author":"S Han","year":"2018","unstructured":"S. Han, Y. Huang, W. Meng et al., Optimal power allocation for SCMA downlink systems based on maximum capacity. IEEE Trans. Commun. 67(2), 1480\u20131489 (2018)","journal-title":"IEEE Trans. Commun."},{"issue":"2","key":"828_CR6","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1109\/MCOM.2018.1800179","volume":"57","author":"K Yang","year":"2018","unstructured":"K. Yang, N. Yang, N. Ye et al., Non-orthogonal multiple access: achieving sustainable future radio access. IEEE Commun. Mag. 57(2), 116\u2013121 (2018)","journal-title":"IEEE Commun. Mag."},{"issue":"2","key":"828_CR7","doi-asserted-by":"publisher","first-page":"721","DOI":"10.1109\/COMST.2016.2621116","volume":"19","author":"SR Islam","year":"2016","unstructured":"S.R. Islam, N. Avazov, O.A. Dobre et al., Power-domain non-orthogonal multiple access (NOMA) in 5G systems: potentials and challenges. IEEE Commun. Surv. Tutor. 19(2), 721\u2013742 (2016)","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"828_CR8","doi-asserted-by":"publisher","DOI":"10.1109\/TVT.2021.3121217","author":"Q Le","year":"2021","unstructured":"Q. Le, V.-D. Nguyen, O.A. Dobre et al., Learning-assisted user clustering in cell-free massive MIMO-NOMA networks. IEEE Trans. Veh. Technol. (2021). https:\/\/doi.org\/10.1109\/TVT.2021.3121217","journal-title":"IEEE Trans. Veh. Technol."},{"issue":"3","key":"828_CR9","doi-asserted-by":"publisher","first-page":"2052","DOI":"10.1109\/TII.2019.2951728","volume":"17","author":"X Liu","year":"2019","unstructured":"X. Liu, X.B. Zhai, W. Lu et al., QoS-guarantee resource allocation for multibeam satellite industrial Internet of Things with NOMA. IEEE Trans. Ind. Inform. 17(3), 2052\u20132061 (2019)","journal-title":"IEEE Trans. Ind. Inform."},{"issue":"8","key":"828_CR10","doi-asserted-by":"publisher","first-page":"5379","DOI":"10.1109\/TII.2019.2947435","volume":"16","author":"X Liu","year":"2019","unstructured":"X. Liu, X. Zhang, NOMA-based resource allocation for cluster-based cognitive industrial internet of things. IEEE Trans. Ind. Inform. 16(8), 5379\u20135388 (2019)","journal-title":"IEEE Trans. Ind. Inform."},{"issue":"2","key":"828_CR11","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1109\/MWC.2018.1700099","volume":"25","author":"SR Islam","year":"2018","unstructured":"S.R. Islam, M. Zeng, O.A. Dobre et al., Resource allocation for downlink NOMA systems: key techniques and open issues. IEEE Wirel. Commun. 25(2), 40\u201347 (2018)","journal-title":"IEEE Wirel. Commun."},{"key":"828_CR12","doi-asserted-by":"publisher","unstructured":"A. Benjebbovu, A. Li, Y. Saito et al. System-level performance of downlink NOMA for future LTE enhancements. IEEE Globecom. 66\u201370 (2013). https:\/\/doi.org\/10.1109\/GLOCOMW.2013.6824963","DOI":"10.1109\/GLOCOMW.2013.6824963"},{"key":"828_CR13","doi-asserted-by":"crossref","unstructured":"H. Zhang, D.-K. Zhang, W.-X. Meng et al. User pairing algorithm with SIC in non-orthogonal multiple access system, in Proceedings of International Conference on Communications (2016), p. 1\u20136","DOI":"10.1109\/ICC.2016.7511620"},{"key":"828_CR14","doi-asserted-by":"publisher","first-page":"2215","DOI":"10.1109\/TSP.2020.2982786","volume":"68","author":"L Sala\u00fcn","year":"2020","unstructured":"L. Sala\u00fcn, M. Coupechoux, C.S.J.I.T.O.S.P. Chen, Joint subcarrier and power allocation in NOMA: optimal and approximate algorithms. IEEE Trans. Signal Process. 68, 2215\u20132230 (2020)","journal-title":"IEEE Trans. Signal Process."},{"issue":"9","key":"828_CR15","doi-asserted-by":"publisher","first-page":"8440","DOI":"10.1109\/TVT.2018.2848294","volume":"67","author":"G Gui","year":"2018","unstructured":"G. Gui, H. Huang, Y. Song et al., Deep learning for an effective nonorthogonal multiple access scheme. IEEE Trans. Veh. Technol. 67(9), 8440\u20138450 (2018)","journal-title":"IEEE Trans. Veh. Technol."},{"key":"828_CR16","doi-asserted-by":"crossref","unstructured":"M. Liu, T. Song, L. Zhang et al. Resource allocation for NOMA based heterogeneous IoT with imperfect SIC: a deep learning method, in Proceedings of IEEE Annual International Symposium on Personal, Indoor and Mobile Radio Communications (2018) p. 1440\u20131446","DOI":"10.1109\/PIMRC.2018.8581013"},{"key":"828_CR17","doi-asserted-by":"crossref","unstructured":"W. Saetan, S. Thipchaksurat. Power allocation for sum rate maximization in 5G NOMA system with imperfect SIC: a deep learning approach, in Proceedings of the 4th International, Conference on Information Technology (2019), p. 195\u2013198","DOI":"10.1109\/INCIT.2019.8911864"},{"issue":"8","key":"828_CR18","doi-asserted-by":"publisher","first-page":"5373","DOI":"10.1109\/TWC.2020.2992786","volume":"19","author":"H Huang","year":"2020","unstructured":"H. Huang, Y. Yang, Z. Ding et al., Deep learning-based sum data rate and energy efficiency optimization for MIMO-NOMA systems. IEEE Trans. Wirel. Commun. 19(8), 5373\u20135388 (2020)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"1","key":"828_CR19","first-page":"1","volume":"41","author":"Q Liu","year":"2018","unstructured":"Q. Liu, J.W. Zhai, Z.-Z. Zhang et al., A survey on deep reinforcement learning. Chin. J. Comput. 41(1), 1\u201327 (2018)","journal-title":"Chin. J. Comput."},{"issue":"8","key":"828_CR20","doi-asserted-by":"publisher","first-page":"5083","DOI":"10.1109\/TWC.2021.3065523","volume":"20","author":"W Ahsan","year":"2021","unstructured":"W. Ahsan, W. Yi, Z. Qin et al., Resource allocation in uplink NOMA-IoT networks: a reinforcement-learning approach. IEEE Trans. Wirel. Commun. 20(8), 5083\u20135098 (2021)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"7540","key":"828_CR21","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"V. Mnih, K. Kavukcuoglu, D. Silver et al., Human-level control through deep reinforcement learning. Nature 518(7540), 529\u2013533 (2015)","journal-title":"Nature"},{"issue":"10","key":"828_CR22","doi-asserted-by":"publisher","first-page":"2200","DOI":"10.1109\/JSAC.2019.2933762","volume":"37","author":"C He","year":"2019","unstructured":"C. He, Y. Hu, Y. Chen et al., Joint power allocation and channel assignment for NOMA with deep reinforcement learning. IEEE J. Sel. Areas Commun. 37(10), 2200\u20132210 (2019)","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"828_CR23","unstructured":"T. Schaul, J. Quan, I. Antonoglou et al., Prioritized experience replay, in Proceedings of International Conference Learning, Representations (2015)"},{"key":"828_CR24","unstructured":"T.P. Lillicrap, J.J. Hunt, A. Pritzel et al., Continuous control with deep reinforcement learning, in ICLR (2015)"},{"issue":"10","key":"828_CR25","doi-asserted-by":"publisher","first-page":"6255","DOI":"10.1109\/TWC.2020.3001736","volume":"19","author":"F Meng","year":"2020","unstructured":"F. Meng, P. Chen, L. Wu et al., Power allocation in multi-user cellular networks: deep reinforcement learning approaches. IEEE Trans. Wirel. Commun. 19(10), 6255\u20136267 (2020)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"11","key":"828_CR26","doi-asserted-by":"publisher","first-page":"1924","DOI":"10.1109\/LWC.2020.3008174","volume":"9","author":"I-H Lee","year":"2020","unstructured":"I.-H. Lee, H.J.I.W.C.L. Jung, User selection and power allocation for downlink NOMA systems with quality-based feedback in rayleigh fading channels. IEEE Wirel. Commun. Lett. 9(11), 1924\u20131927 (2020)","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"828_CR27","doi-asserted-by":"crossref","unstructured":"J. Zhai, Q. Liu, Z. Zhang et al. Deep q-learning with prioritized sampling, in Proceedings of International Conference on Neural Information Processing (2016), p. 13\u201322","DOI":"10.1007\/978-3-319-46687-3_2"},{"key":"828_CR28","unstructured":"A. R. Mahmood, H. Van Hasselt, R. S. Sutton. Weighted importance sampling for off-policy learning with linear function approximation, in Proceedings of the NIPS (2014), p. 3014\u20133022"},{"issue":"8","key":"828_CR29","doi-asserted-by":"publisher","first-page":"7279","DOI":"10.1109\/JIOT.2020.2982699","volume":"7","author":"X Wang","year":"2020","unstructured":"X. Wang, Y. Zhang, R. Shen et al., DRL-based energy-efficient resource allocation frameworks for uplink NOMA systems. IEEE Internet Things J. 7(8), 7279\u20137294 (2020)","journal-title":"IEEE Internet Things J."},{"key":"828_CR30","doi-asserted-by":"crossref","unstructured":"Y. Zhang, X. Wang, Y. Xu. Energy-efficient resource allocation in uplink NOMA systems with deep reinforcement learning, in Proceedings of International Conference on Wireless Communications and Signal Processing (WCSP) (2019), p. 1\u20136","DOI":"10.1109\/WCSP.2019.8927898"},{"key":"828_CR31","doi-asserted-by":"publisher","first-page":"94238","DOI":"10.1109\/ACCESS.2019.2926757","volume":"7","author":"X Wang","year":"2019","unstructured":"X. Wang, R. Chen, Y. Xu et al., Low-complexity power allocation in NOMA systems with imperfect SIC for maximizing weighted sum-rate. IEEE Access 7, 94238\u201394253 (2019)","journal-title":"IEEE Access"}],"container-title":["EURASIP Journal on Advances in Signal Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-021-00828-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13634-021-00828-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-021-00828-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,11]],"date-time":"2021-12-11T15:16:20Z","timestamp":1639235780000},"score":1,"resource":{"primary":{"URL":"https:\/\/asp-eurasipjournals.springeropen.com\/articles\/10.1186\/s13634-021-00828-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12]]},"references-count":31,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["828"],"URL":"https:\/\/doi.org\/10.1186\/s13634-021-00828-1","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-1023427\/v1","asserted-by":"object"}]},"ISSN":["1687-6180"],"issn-type":[{"value":"1687-6180","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12]]},"assertion":[{"value":"29 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 November 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 December 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The picture materials quoted in this article have no copyright requirements, and the source has been indicated.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"120"}}