{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T14:03:44Z","timestamp":1781100224501,"version":"3.54.1"},"reference-count":30,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2020,9,29]],"date-time":"2020-09-29T00:00:00Z","timestamp":1601337600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>We study the foot plantar sensor placement by a deep reinforcement learning algorithm without using any prior knowledge of the foot anatomical area. To apply a reinforcement learning algorithm, we propose a sensor placement environment and reward system that aims to optimize fitting the center of pressure (COP) trajectory during the self-selected speed running task. In this environment, the agent considers placing eight sensors within a 7 \u00d7 20 grid coordinate system, and then the final pattern becomes the result of sensor placement. Our results show that this method (1) can generate a sensor placement, which has a low mean square error in fitting ground truth COP trajectory, and (2) robustly discovers the optimal sensor placement in a large number of combinations, which is more than 116 quadrillion. This method is also feasible for solving different tasks, regardless of the self-selected speed running task.<\/jats:p>","DOI":"10.3390\/s20195588","type":"journal-article","created":{"date-parts":[[2020,9,29]],"date-time":"2020-09-29T08:43:27Z","timestamp":1601369007000},"page":"5588","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Optimizing the Sensor Placement for Foot Plantar Center of Pressure without Prior Knowledge Using Deep Reinforcement Learning"],"prefix":"10.3390","volume":"20","author":[{"given":"Cheng-Wu","family":"Lin","sequence":"first","affiliation":[{"name":"Department of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei 106, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shanq-Jang","family":"Ruan","sequence":"additional","affiliation":[{"name":"Department of Electronic and Computer Engineering, National Taiwan University of Science and Technology, Taipei 106, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei-Chun","family":"Hsu","sequence":"additional","affiliation":[{"name":"Graduate Institute of Biomedical Engineering, National Taiwan University of Science and Technology, Taipei 106, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ya-Wen","family":"Tu","sequence":"additional","affiliation":[{"name":"Sijhih Cathay General Hospital, New Taipei 221, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shao-Li","family":"Han","sequence":"additional","affiliation":[{"name":"Sijhih Cathay General Hospital, New Taipei 221, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/MEMB.2003.1213622","article-title":"Wearable sensors\/systems and their impact on biomedical engineering","volume":"22","author":"Bonato","year":"2003","journal-title":"IEEE Eng. Med. Biol. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Nilpanapan, T., and Kerdcharoen, T. (2016, January 7\u20139). Social data shoes for gait monitoring of elderly people in smart home. Proceedings of the 2016 9th Biomedical Engineering International Conference (BMEiCON), Luang Prabang, Laos.","DOI":"10.1109\/BMEiCON.2016.7859611"},{"key":"ref_3","unstructured":"Yamakawa, T., Taniguchi, K., Asari, K., Kobashi, S., and Hata, Y. (2010, January 9\u201311). Biometric personal identification based on gait pattern using both feet pressure change. Proceedings of the 2010 World Automation Congress, Xiamen, China."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"9884","DOI":"10.3390\/s120709884","article-title":"Foot plantar pressure measurement system: A review","volume":"12","author":"Razak","year":"2012","journal-title":"Sensors"},{"key":"ref_5","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_7","unstructured":"Russell, S., and Norvig, P. (2002). Artificial Intelligence: A Modern Approach, Prentice Hall."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_9","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, CA, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Gatys, L.A., Ecker, A.S., and Bethge, M. (2016, January 27\u201330). Image style transfer using convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.265"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Graves, A., Mohamed, A.r., and Hinton, G. (2013, January 26\u201331). Speech recognition with deep recurrent neural networks. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638947"},{"key":"ref_14","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Tsitsiklis, J.N., and Van Roy, B. (1997, January 5). Analysis of temporal-diffference learning with function approximation. Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA.","DOI":"10.1109\/9.580874"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"ref_17","unstructured":"Berner, C., Brockman, G., Chan, B., Cheung, V., D\u0119biak, P., Dennison, C., Farhi, D., Fischer, Q., Hashme, S., and Hesse, C. (2019). Dota 2 with large scale deep reinforcement learning. arXiv."},{"key":"ref_18","unstructured":"Christodoulou, P. (2019). Soft actor\u2013critic for discrete action settings. arXiv."},{"key":"ref_19","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018). Soft actor\u2013critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. arXiv."},{"key":"ref_20","unstructured":"Jaderberg, M., Dalibard, V., Osindero, S., Czarnecki, W.M., Donahue, J., Razavi, A., Vinyals, O., Green, T., Dunning, I., and Simonyan, K. (2017). Population based training of neural networks. arXiv."},{"key":"ref_21","first-page":"186","article-title":"Validation of F-Scan pressure sensor system: A technical note","volume":"35","author":"Luo","year":"1998","journal-title":"J. Rehabil. Res. Dev."},{"key":"ref_22","unstructured":"Putnam, W., and Knapp, R.B. (1996). Input\/data acquisition system design for human computer interfacing. Online Course Notes, Stanford University."},{"key":"ref_23","unstructured":"Arjona-Medina, J.A., Gillhofer, M., Widrich, M., Unterthiner, T., Brandstetter, J., and Hochreiter, S. (2019, January 8\u201314). Rudder: Return decomposition for delayed rewards. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_24","unstructured":"Sutskever, I., Vinyals, O., and Le, Q.V. (2014, January 8\u201313). Sequence to sequence learning with neural networks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_25","unstructured":"Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016, January 19\u201324). Asynchronous methods for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_26","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_27","unstructured":"Sutton, R.S., McAllester, D.A., Singh, S.P., and Mansour, Y. (2000). Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_28","unstructured":"Fujimoto, S., van Hoof, H., and Meger, D. (2018). Addressing Function Approximation Error in Actor\u2013Critic Methods, PMLR."},{"key":"ref_29","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.foot.2011.11.001","article-title":"Repeatability of WalkinSense\u00ae in shoe pressure measurement system: A preliminary study","volume":"22","author":"Healy","year":"2012","journal-title":"Foot"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5588\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:14:54Z","timestamp":1760177694000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5588"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,29]]},"references-count":30,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2020,10]]}},"alternative-id":["s20195588"],"URL":"https:\/\/doi.org\/10.3390\/s20195588","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,29]]}}}