{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T16:11:27Z","timestamp":1780503087066,"version":"3.54.1"},"reference-count":34,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,5,17]],"date-time":"2022-05-17T00:00:00Z","timestamp":1652745600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSF of China","award":["62073300"],"award-info":[{"award-number":["62073300"]}]},{"name":"NSF of China","award":["U1911205"],"award-info":[{"award-number":["U1911205"]}]},{"name":"NSF of China","award":["62076225"],"award-info":[{"award-number":["62076225"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Reducing pollutant detection time based on a reasonable sensor combination is desirable. Clean drinking water is essential to life. However, the water supply network (WSN) is a vulnerable target for accidental or intentional contamination due to its extensive geographic coverage, multiple points of access, backflow, infrastructure aging, and designed sabotage. Contaminants entering WSN are one of the most dangerous events that may cause sickness or even death among people. Using sensors to monitor the water quality in real time is one of the most effective ways to minimize negative consequences on public health. However, it is a challenge to deploy a limited number of sensors in a large-scale WSN. In this study, the sensor placement problem (SPP) is modeled as a sequential decision optimization problem, then an evolutionary reinforcement learning (ERL) algorithm based on domain knowledge is proposed to solve SPP. Extensive experiments have been conducted and the results show that our proposed algorithm outperforms meta-heuristic algorithms and deep reinforcement learning (DRL).<\/jats:p>","DOI":"10.3390\/s22103799","type":"journal-article","created":{"date-parts":[[2022,5,17]],"date-time":"2022-05-17T08:34:29Z","timestamp":1652776469000},"page":"3799","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Domain Knowledge-Based Evolutionary Reinforcement Learning for Sensor Placement"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6319-4290","authenticated-orcid":false,"given":"Mingxuan","family":"Song","sequence":"first","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chengyu","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1610-6865","authenticated-orcid":false,"given":"Wenyin","family":"Gong","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuesong","family":"Yan","sequence":"additional","affiliation":[{"name":"School of Computer Science, China University of Geosciences, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2005","DOI":"10.1007\/s13524-019-00831-0","article-title":"The impact of the Flint water crisis on fertility","volume":"56","author":"Grossman","year":"2019","journal-title":"Demography"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1186\/s12302-020-00305-w","article-title":"Risk assessment of heavy metals in the surface sediment at the drinking water source of the Xiangjiang River in South China","volume":"32","author":"Huang","year":"2020","journal-title":"Environ. Sci. Eur."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"488","DOI":"10.1016\/j.ins.2018.06.055","article-title":"Modified NSGA-III for sensor placement in water distribution system","volume":"509","author":"Hu","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.envsoft.2015.10.030","article-title":"New formulation and optimization methods for water sensor placement","volume":"76","author":"Zhao","year":"2016","journal-title":"Environ. Model. Softw."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1061\/(ASCE)0733-9496(2006)132:4(218)","article-title":"Sensor placement in municipal water networks with temporal integer programming models","volume":"132","author":"Berry","year":"2006","journal-title":"J. Water Resour. Plan. Manag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1061\/(ASCE)0733-9496(2006)132:4(225)","article-title":"Contamination warning in water networks: General mixed-integer linear models for sensor location design","volume":"132","author":"Propato","year":"2006","journal-title":"J. Water Resour. Plan. Manag."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"937","DOI":"10.1007\/s10726-012-9321-7","article-title":"A sorting model for group decision making: A case study of water losses in Brazil","volume":"23","author":"Morais","year":"2014","journal-title":"Group Decis. Negot."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1016\/j.engappai.2014.02.010","article-title":"Uncertainty analysis of water supply networks using the fuzzy set theory and NSGA-II","volume":"32","author":"Haghighi","year":"2014","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1016\/j.proeng.2012.04.245","article-title":"Non-dominated sorting genetic algorithms-iibased on multi-objective optimization model in the water distribution system","volume":"37","author":"Li","year":"2012","journal-title":"Procedia Eng."},{"key":"ref_10","unstructured":"Packer, C., Gao, K., Kos, J., Kr\u00e4henb\u00fchl, P., Koltun, V., and Song, D. (2018). Assessing generalization in deep reinforcement learning. arXiv."},{"key":"ref_11","unstructured":"Lee, K., Lee, K., Shin, J., and Lee, H. (2019). Network randomization: A simple technique for generalization in deep reinforcement learning. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Ouyang, W., Wang, Y., Han, S., Jin, Z., and Weng, P. (2021, January 5\u20137). Improving Generalization of Deep Reinforcement Learning-based TSP Solvers. Proceedings of the 2021 IEEE Symposium Series on Computational Intelligence (SSCI), Orlando, FL, USA.","DOI":"10.1109\/SSCI50451.2021.9659970"},{"key":"ref_13","unstructured":"Lu, H., Zhang, X., and Yang, S. (2019, January 6\u20139). A learning-based iterative method for solving vehicle routing problems. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_14","unstructured":"Yolcu, E., and P\u00f3czos, B. (2019, January 8\u201314). Learning local search heuristics for boolean satisfiability. Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, BC, Canada."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3806","DOI":"10.1109\/TITS.2019.2909109","article-title":"Online vehicle routing with neural combinatorial optimization and deep reinforcement learning","volume":"20","author":"James","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_16","unstructured":"Khadka, S., and Tumer, K. (2018). Evolutionary reinforcement learning. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1016\/j.swevo.2018.03.011","article-title":"Reinforcement learning versus evolutionary computation: A survey on hybrid algorithms","volume":"44","author":"Drugan","year":"2019","journal-title":"Swarm Evol. Comput."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2861","DOI":"10.1109\/TCYB.2019.2901897","article-title":"Reinforcement learning-based optimal sensor placement for spatiotemporal modeling","volume":"50","author":"Wang","year":"2019","journal-title":"IEEE Trans. Cybernet."},{"key":"ref_19","first-page":"119","article-title":"Simplification of water supply network models through linearisation","volume":"1","author":"Maschler","year":"1999","journal-title":"Cent. Water Syst. Rep."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"767","DOI":"10.2166\/ws.2017.124","article-title":"Complex network and fractal theory for the assessment of water distribution network resilience to pipe failures","volume":"18","author":"Giudicianni","year":"2018","journal-title":"Water Sci. Technol. Water Supply"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wolsey, L.A. (2020). Integer Programming, John Wiley & Sons.","DOI":"10.1002\/9781119606475"},{"key":"ref_22","unstructured":"Brusco, M.J., and Stahl, S. (2005). Branch-and-Bound Applications in Combinatorial Data Analysis, Springer."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1089","DOI":"10.1007\/s10586-017-0838-z","article-title":"A Spark-based genetic algorithm for sensor placement in large scale drinking water distribution systems","volume":"20","author":"Hu","year":"2017","journal-title":"Clust. Comput."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1007\/s11276-016-1358-0","article-title":"A survey on sensor placement for contamination detection in water distribution systems","volume":"24","author":"Hu","year":"2018","journal-title":"Wirel. Netw."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"ref_26","unstructured":"Vinyals, O., Fortunato, M., and Jaitly, N. (2015). Pointer networks. arXiv."},{"key":"ref_27","unstructured":"Bello, I., Pham, H., Le, Q.V., Norouzi, M., and Bengio, S. (2016). Neural combinatorial optimization with reinforcement learning. arXiv."},{"key":"ref_28","unstructured":"Nazari, M., Oroojlooy, A., Snyder, L.V., and Tak\u00e1\u010d, M. (2018). Reinforcement learning for solving the vehicle routing problem. arXiv."},{"key":"ref_29","unstructured":"Chen, X., and Tian, Y. (2019, January 8\u201314). Learning to perform local rewriting for combinatorial optimization. Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, BC, Canada."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"04020079","DOI":"10.1061\/(ASCE)WR.1943-5452.0001287","article-title":"Deep Reinforcement Learning for Real-Time Optimization of Pumps in Water Distribution Systems","volume":"146","year":"2020","journal-title":"J. Water Resour. Plan. Manag."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Mambretti, S., and Orsi, E. (2016). Optimization of Pumping Stations in Complex Water Supply Networks through Evolutionary Computation Methods. J. Am. Water Works Assoc., E119\u2013E125.","DOI":"10.5942\/jawwa.2016.108.0025"},{"key":"ref_32","unstructured":"Khadka, S., Majumdar, S., Nassar, T., Dwiel, Z., Tumer, E., Miret, S., Liu, Y., and Tumer, K. (2019, January 9\u201315). Collaborative evolutionary reinforcement learning. Proceedings of the International Conference on Machine Learning (PMLR), Long Beach, CA, USA."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1034","DOI":"10.1109\/TCSVT.2021.3072412","article-title":"Real-time video emotion recognition based on reinforcement learning and domain knowledge","volume":"32","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zheng, Y., Chen, H., Duan, Q., Lin, L., Shao, Y., Wang, W., Wang, X., and Xu, Y. (2021, January 10\u201313). Leveraging Domain Knowledge for Robust Deep Reinforcement Learning in Networking. Proceedings of the IEEE INFOCOM 2021\u2014IEEE Conference on Computer Communications, Vancouver, BC, Canada.","DOI":"10.1109\/INFOCOM42981.2021.9488863"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/10\/3799\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:11:42Z","timestamp":1760137902000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/10\/3799"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,17]]},"references-count":34,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["s22103799"],"URL":"https:\/\/doi.org\/10.3390\/s22103799","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,17]]}}}