{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T18:57:00Z","timestamp":1772823420673,"version":"3.50.1"},"reference-count":39,"publisher":"SAGE Publications","issue":"7","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61976243"],"award-info":[{"award-number":["61976243"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Luoyang Major Scientic and Technological Innovation Projects","award":["2101017A"],"award-info":[{"award-number":["2101017A"]}]},{"name":"Scientific and Technological Innovation Team of Colleges and Universities in Henan Province","award":["20IRTSTHN018"],"award-info":[{"award-number":["20IRTSTHN018"]}]},{"name":"Leading talents of science and technology in the Central Plain of China","award":["224200510004"],"award-info":[{"award-number":["224200510004"]}]},{"name":"Key Technologies R & D Program of Henan Province","award":["212102210083"],"award-info":[{"award-number":["212102210083"]}]},{"name":"Key Technologies R & D Program of Henan Province","award":["222102210049"],"award-info":[{"award-number":["222102210049"]}]},{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"publisher","award":["2021M690914"],"award-info":[{"award-number":["2021M690914"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["International Journal of Distributed Sensor Networks"],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:p> Wireless sensor network has been widely used in different fields, such as structural health monitoring and artificial intelligence technology. The routing planning, an important part of wireless sensor network, can be formalized as an optimization problem needing to be solved. In this article, a reinforcement learning algorithm is proposed to solve the problem of optimal routing in wireless sensor networks, namely, adaptive TD([Formula: see text]) learning algorithm referred to as ADTD([Formula: see text]) under Markovian noise, which is more practical than i.i.d. (identically and independently distributed) noise in reinforcement learning. Moreover, we also present non-asymptotic analysis of ADTD([Formula: see text]) with both constant and diminishing step-sizes. Specifically, when the step-size is constant, the convergence rate of [Formula: see text] is achieved, where [Formula: see text] is the number of iterations; when the step-size is diminishing, the convergence rate of [Formula: see text] is also obtained. In addition, the performance of the algorithm is verified by simulation. <\/jats:p>","DOI":"10.1177\/15501329221114546","type":"journal-article","created":{"date-parts":[[2022,7,30]],"date-time":"2022-07-30T05:26:06Z","timestamp":1659158766000},"page":"155013292211145","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":1,"title":["A non-asymptotic analysis of adaptive TD(\u03bb) learning in wireless sensor networks"],"prefix":"10.1177","volume":"18","author":[{"given":"Bing","family":"Li","sequence":"first","affiliation":[{"name":"School of Information Engineering, Henan University of Science and Technology, Luoyang, China"}]},{"given":"Tao","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Information Technology Management, CITIC Heavy Industries Co., Ltd, Luoyang, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4707-1354","authenticated-orcid":false,"given":"Muhua","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Henan University of Science and Technology, Luoyang, China"}]},{"given":"Junlong","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Henan University of Science and Technology, Luoyang, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2523-1089","authenticated-orcid":false,"given":"Mingchuan","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Henan University of Science and Technology, Luoyang, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1572-5293","authenticated-orcid":false,"given":"Qingtao","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Henan University of Science and Technology, Luoyang, China"}]}],"member":"179","published-online":{"date-parts":[[2022,7,29]]},"reference":[{"key":"bibr1-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2019.2950109"},{"key":"bibr2-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1016\/j.cogsys.2018.03.003"},{"key":"bibr3-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2020.3029766"},{"key":"bibr4-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1016\/j.comnet.2021.108515"},{"key":"bibr5-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1016\/j.comcom.2021.11.001"},{"key":"bibr6-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2018.06.002"},{"key":"bibr7-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1177\/1550147717707896"},{"key":"bibr8-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1177\/1550147719833541"},{"key":"bibr9-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1016\/j.adhoc.2020.102243"},{"key":"bibr10-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1007\/BF00115009"},{"key":"bibr11-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1007\/BF00993978"},{"key":"bibr12-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1007\/BF00114725"},{"key":"bibr13-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1109\/9.580874"},{"key":"bibr14-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1016\/S0005-1098(99)00099-0"},{"key":"bibr15-15501329221114546","first-page":"8477","volume-title":"Proceedings of the 32nd annual conference Neural Information Processing Systems","author":"Hu B"},{"key":"bibr16-15501329221114546","first-page":"1075","volume-title":"Proceedings of the 9th conference Neural Information Processing Systems","author":"Tsitsiklis JN"},{"key":"bibr17-15501329221114546","first-page":"2235","volume-title":"Proceedings of the 30th conference Neural Information Processing Systems","author":"Devraj AM"},{"key":"bibr18-15501329221114546","first-page":"626","volume":"37","author":"Korda N","year":"2015","journal-title":"PMLR"},{"key":"bibr19-15501329221114546","first-page":"6144","volume-title":"Proceedings of the 32nd AAAI conference on artificial intelligence","author":"Dalal G"},{"key":"bibr20-15501329221114546","first-page":"10460","volume-title":"Proceedings of the 34th AAAI conference on artificial intelligence","author":"Xiong H"},{"key":"bibr21-15501329221114546","volume-title":"Proceedings of the 34th international conference on Neural Information Processing Systems","author":"Wang G"},{"key":"bibr22-15501329221114546","volume-title":"Proceedings of the international joint conference on neural networks (IJCNN)","author":"Altahhan A"},{"key":"bibr23-15501329221114546","unstructured":"Chen Z, Maguluri ST, Shakkottai S, et al. A Lyapunov theory for finite-sample guarantees of asynchronous Q-learning and TD-learning variants. CoRR 2021; abs\/2102.01567, https:\/\/arxiv.org\/abs\/2102.01567"},{"key":"bibr24-15501329221114546","first-page":"2803","volume":"99","author":"Srikant R","year":"2019","journal-title":"PMLR"},{"key":"bibr25-15501329221114546","first-page":"1691","volume":"75","author":"Bhandari J","year":"2018","journal-title":"PMLR"},{"key":"bibr26-15501329221114546","first-page":"4706","volume-title":"Proceedings of the 33rd annual conference on Neural Information Processing Systems","author":"Gupta H"},{"key":"bibr27-15501329221114546","unstructured":"Stooke A, Abbeel P. Accelerated methods for deep reinforcement learning. CoRR 2018; abs\/1803.02811, https:\/\/arxiv.org\/abs\/1803.02811"},{"key":"bibr28-15501329221114546","first-page":"3591","volume-title":"Proceedings of the 30th international conference on Neural Information Processing Systems","author":"Papini M"},{"key":"bibr29-15501329221114546","unstructured":"Sun T, Shen H, Chen T, et al. Adaptive temporal difference learning with linear function approximation. CoRR 2020; abs\/2002.08537, https:\/\/arxiv.org\/abs\/2002.08537"},{"key":"bibr30-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992701"},{"key":"bibr31-15501329221114546","first-page":"91","volume-title":"Proceedings of the 3rd conference on Artificial General Intelligence","volume":"1","author":"Maei HR"},{"key":"bibr32-15501329221114546","first-page":"494","volume-title":"Proceedings of the 15th international conference on Autonomous Agents and Multiagent Systems","author":"White AM"},{"key":"bibr33-15501329221114546","first-page":"1","volume":"17","author":"Sutton RS","year":"2016","journal-title":"J Mach Learn Res"},{"key":"bibr34-15501329221114546","volume-title":"Proceedings of the 3rd International Conference on Learning Representations","author":"Kingma DP"},{"key":"bibr35-15501329221114546","unstructured":"Reddi SJ, Kale S, Kumar S. On the convergence of Adam and beyond. CoRR 2019; abs\/1904.09237, https:\/\/arxiv.org\/abs\/1904.09237"},{"key":"bibr36-15501329221114546","author":"Zou F","year":"2018","journal-title":"CoRR"},{"key":"bibr37-15501329221114546","first-page":"983","volume-title":"Proceedings of the 22nd international conference on Artificial Intelligence and Statistics","volume":"89","author":"Li X"},{"key":"bibr38-15501329221114546","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2021.3071962"},{"key":"bibr39-15501329221114546","first-page":"6379","volume-title":"Advances in Neural Information Processing Systems 30: annual conference on Neural Information Processing Systems","author":"Lowe R"}],"container-title":["International Journal of Distributed Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/15501329221114546","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/15501329221114546","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/15501329221114546","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,30]],"date-time":"2022-07-30T05:26:44Z","timestamp":1659158804000},"score":1,"resource":{"primary":{"URL":"http:\/\/journals.sagepub.com\/doi\/10.1177\/15501329221114546"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":39,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10.1177\/15501329221114546"],"URL":"https:\/\/doi.org\/10.1177\/15501329221114546","relation":{},"ISSN":["1550-1329","1550-1477"],"issn-type":[{"value":"1550-1329","type":"print"},{"value":"1550-1477","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7]]}}}