{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T12:30:02Z","timestamp":1770813002089,"version":"3.50.1"},"reference-count":23,"publisher":"Association for Computing Machinery (ACM)","issue":"5s","license":[{"start":{"date-parts":[[2017,9,27]],"date-time":"2017-09-27T00:00:00Z","timestamp":1506470400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"JSPS KAKENHI","award":["16K12405"],"award-info":[{"award-number":["16K12405"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2017,10,31]]},"abstract":"<jats:p>In this paper, we present an adaptive power manager for solar energy harvesting sensor nodes. We use a simplified model consisting of a solar panel, an ideal battery and a general sensor node with variable duty cycle. Our power manager uses Reinforcement Learning (RL), specifically SARSA(\u03bb) learning, to train itself from historical data. Once trained, we show that our power manager is capable of adapting to changes in weather, climate, device parameters and battery degradation while ensuring near-optimal performance without depleting or overcharging its battery. Our approach uses a simple but novel general reward function and leverages the use of weather forecast data to enhance performance. We show that our method achieves near perfect energy neutral operation (ENO) with less than 6% root mean square deviation from ENO as compared to more than 23% deviation that occur when using other approaches.<\/jats:p>","DOI":"10.1145\/3126495","type":"journal-article","created":{"date-parts":[[2017,9,27]],"date-time":"2017-09-27T12:33:53Z","timestamp":1506515633000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":57,"title":["Adaptive Power Management in Solar Energy Harvesting Sensor Node Using Reinforcement Learning"],"prefix":"10.1145","volume":"16","author":[{"given":"Shaswot","family":"Shresthamali","sequence":"first","affiliation":[{"name":"The University of Tokyo"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Masaaki","family":"Kondo","sequence":"additional","affiliation":[{"name":"The University of Tokyo"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hiroshi","family":"Nakamura","sequence":"additional","affiliation":[{"name":"The University of Tokyo"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,9,27]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/54.914621"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TWC.2013.030413.121120"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2015.2478717"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2014.6710085"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1165573.1165616"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TETC.2014.2316518"},{"key":"e_1_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Roy Chaoming Hsu etal 2015. Dynamic energy management of energy harvesting wireless sensor nodes using fuzzy inference system with reinforcement learning. In IEEE 13th INDIN. 116--120. Roy Chaoming Hsu et al. 2015. Dynamic energy management of energy harvesting wireless sensor nodes using fuzzy inference system with reinforcement learning. In IEEE 13th INDIN. 116--120.","DOI":"10.1109\/INDIN.2015.7281720"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/1147685.1147765"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1274858.1274870"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1012888.1005714"},{"key":"e_1_2_1_11_1","doi-asserted-by":"crossref","unstructured":"Junaid Ahmed Khan etal 2015. Energy management in wireless sensor networks: a survey. Computers 8 Electrical Engineering 41 (2015) 159--176. Junaid Ahmed Khan et al. 2015. Energy management in wireless sensor networks: a survey. Computers 8 Electrical Engineering 41 (2015) 159--176.","DOI":"10.1016\/j.compeleceng.2014.06.009"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/IECON.2011.6119679"},{"key":"e_1_2_1_13_1","doi-asserted-by":"crossref","unstructured":"S. Danish Maqbool et al. 2011. Analysis of adaptability of Reinforcement Learning approach. In IEEE 14th INMIC. 45--49. S. Danish Maqbool et al. 2011. Analysis of adaptability of Reinforcement Learning approach. In IEEE 14th INMIC. 45--49.","DOI":"10.1109\/INMIC.2011.6151508"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCOMM.2013.111113.130022"},{"key":"e_1_2_1_15_1","volume-title":"2016 IEEE International Conference on Communications. 1--6.","author":"Andrea","unstructured":"Andrea Ortiz et al. 2016. Reinforcement learning for energy harvesting point-to-point communications . In 2016 IEEE International Conference on Communications. 1--6. Andrea Ortiz et al. 2016. Reinforcement learning for energy harvesting point-to-point communications. In 2016 IEEE International Conference on Communications. 1--6."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/1147685.1147764"},{"key":"e_1_2_1_17_1","doi-asserted-by":"crossref","unstructured":"Luigi Rucco et al. 2013. A bird\u2019s eye view on reinforcement learning approaches for power management in WSNs. In 6th WMNC. 1--8. Luigi Rucco et al. 2013. A bird\u2019s eye view on reinforcement learning approaches for power management in WSNs. In 6th WMNC. 1--8.","DOI":"10.1109\/WMNC.2013.6548988"},{"key":"e_1_2_1_18_1","doi-asserted-by":"crossref","unstructured":"Navin Sharma et al. 2010. Cloudy computing: Leveraging weather forecasts in energy harvesting sensor systems. In 7th IEEE SECON. 1--9. Navin Sharma et al. 2010. Cloudy computing: Leveraging weather forecasts in energy harvesting sensor systems. In 7th IEEE SECON. 1--9.","DOI":"10.1109\/SECON.2010.5508260"},{"key":"e_1_2_1_19_1","volume-title":"Energy harvesting sensor nodes: Survey and implications","author":"Sudevalayam Sujesha","year":"2011","unstructured":"Sujesha Sudevalayam and Purushottam Kulkarni . 2011. Energy harvesting sensor nodes: Survey and implications . IEEE Communications Surveys 8 Tutorials 13, 3 ( 2011 ), 443--461. Sujesha Sudevalayam and Purushottam Kulkarni. 2011. Energy harvesting sensor nodes: Survey and implications. IEEE Communications Surveys 8 Tutorials 13, 3 (2011), 443--461."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJSNET.2009.029017"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/37.126844"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/SIU.2016.7495924"},{"key":"e_1_2_1_23_1","volume-title":"4th IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks. 21--30","author":"Christopher","unstructured":"Christopher M. Vigorito et al. 2007. Adaptive control of duty cycling in energy-harvesting wireless sensor networks . In 4th IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks. 21--30 . Christopher M. Vigorito et al. 2007. Adaptive control of duty cycling in energy-harvesting wireless sensor networks. In 4th IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks. 21--30."}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3126495","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3126495","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T22:34:36Z","timestamp":1750890876000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3126495"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,9,27]]},"references-count":23,"journal-issue":{"issue":"5s","published-print":{"date-parts":[[2017,10,31]]}},"alternative-id":["10.1145\/3126495"],"URL":"https:\/\/doi.org\/10.1145\/3126495","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"value":"1539-9087","type":"print"},{"value":"1558-3465","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,9,27]]},"assertion":[{"value":"2017-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-09-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}