{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T16:00:22Z","timestamp":1774022422045,"version":"3.50.1"},"reference-count":34,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,5,20]],"date-time":"2022-05-20T00:00:00Z","timestamp":1653004800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Keio Leading-Edge Laboratory of Science and Technology","award":["KEIO-KLL-000045"],"award-info":[{"award-number":["KEIO-KLL-000045"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, we propose an activity detection system using a 24 \u00d7 32 resolution infrared array sensor placed on the ceiling. We first collect the data at different resolutions (i.e., 24 \u00d7 32, 12 \u00d7 16, and 6 \u00d7 8) and apply the advanced deep learning (DL) techniques of Super-Resolution (SR) and denoising to enhance the quality of the images. We then classify the images\/sequences of images depending on the activities the subject is performing using a hybrid deep learning model combining a Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM). We use data augmentation to improve the training of the neural networks by incorporating a wider variety of samples. The process of data augmentation is performed by a Conditional Generative Adversarial Network (CGAN). By enhancing the images using SR, removing the noise, and adding more training samples via data augmentation, our target is to improve the classification accuracy of the neural network. Through experiments, we show that employing these deep learning techniques to low-resolution noisy infrared images leads to a noticeable improvement in performance. The classification accuracy improved from 78.32% to 84.43% (for images with 6 \u00d7 8 resolution), and from 90.11% to 94.54% (for images with 12 \u00d7 16 resolution) when we used the CNN and CNN + LSTM networks, respectively.<\/jats:p>","DOI":"10.3390\/s22103898","type":"journal-article","created":{"date-parts":[[2022,5,21]],"date-time":"2022-05-21T09:18:08Z","timestamp":1653124688000},"page":"3898","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["An Infrared Array Sensor-Based Approach for Activity Detection, Combining Low-Cost Technology with Advanced Deep Learning Techniques"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7923-3995","authenticated-orcid":false,"given":"Krishnan Arumugasamy","family":"Muthukumar","sequence":"first","affiliation":[{"name":"Graduate School of Science and Technology, Keio University, Yokohama 223-8522, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7055-9318","authenticated-orcid":false,"given":"Mondher","family":"Bouazizi","sequence":"additional","affiliation":[{"name":"Faculty of Science and Technology, Keio University, Yokohama 223-8522, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3961-1426","authenticated-orcid":false,"given":"Tomoaki","family":"Ohtsuki","sequence":"additional","affiliation":[{"name":"Faculty of Science and Technology, Keio University, Yokohama 223-8522, Japan"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Statistics Japan (2021). Statistical Handbook of Japan 2021.","DOI":"10.1787\/9f9abbd5-en"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"E11","DOI":"10.5888\/pcd16.180170","article-title":"Patterns of Co-Occurrence of Chronic Disease Among Older Adults in Tokyo","volume":"16","author":"Mitsutake","year":"2019","journal-title":"Prev. Chronic Dis."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Mashiyama, S., Hong, J., and Ohtsuki, T. (2015, January 8\u201312). Activity recognition using low resolution infrared array sensor. Proceedings of the 2015 IEEE International Conference on Communications (ICC), London, UK.","DOI":"10.1109\/ICC.2015.7248370"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Mashiyama, S., Hong, J., and Ohtsuki, T. (2014, January 2\u20135). A fall detection system using low resolution infrared array sensor. Proceedings of the IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Helsinki, Finland.","DOI":"10.1109\/PIMRC.2014.7136520"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Hino, Y., Hong, J., and Ohtsuki, T. (2015, January 8\u201312). Activity recognition using array antenna. Proceedings of the 2015 IEEE International Conference on Communications (ICC), London, UK.","DOI":"10.1109\/ICC.2015.7248372"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Bouazizi, M., Ye, C., and Ohtsuki, T. (2021). 2D LIDAR-Based Approach for Activity Identification and Fall Detection. IEEE Internet Things J.","DOI":"10.1109\/JIOT.2021.3127186"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Nakamura, T., Bouazizi, M., Yamamoto, K., and Ohtsuki, T. (2020, January 7\u201311). Wi-fi-CSI-based fall detection by spectrogram analysis with CNN. Proceedings of the IEEE Global Communications Conference, Taipei, Taiwan.","DOI":"10.1109\/GLOBECOM42002.2020.9322323"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1311","DOI":"10.1109\/TCE.2006.273150","article-title":"A pyroelectric infrared sensor-based indoor location-aware system for the smart home","volume":"52","author":"Lee","year":"2006","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Liang, Q., Yu, L., Zhai, X., Wan, Z., and Nie, H. (2018, January 3\u20135). Activity recognition based on thermopile imaging array sensor. Proceedings of the 2018 IEEE International Conference on Electro\/Information Technology (EIT), Rochester, MI, USA.","DOI":"10.1109\/EIT.2018.8500177"},{"key":"ref_10","unstructured":"Muthukumar, K.A., Bouazizi, M., and Ohtsuki, T. (2020, January 15\u201318). Activity Detection Using Wide Angle Low-Resolution Infrared Array Sensors. Proceedings of the Institute of Electronics, Information and Communication Engineers (IEICE) Conference Archives, Virtual."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"6075","DOI":"10.1109\/JSEN.2020.2974633","article-title":"Intruder localization and tracking using two pyroelectric infrared sensors","volume":"20","author":"Aldalahmeh","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Bouazizi, M., and Ohtsuki, T. (2020, January 20\u201324). An infrared array sensor-based method for localizing and counting people for health care and monitoring. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada.","DOI":"10.1109\/EMBC44109.2020.9176199"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Bouazizi, M., Ye, C., and Ohtsuki, T. (2022). Low-Resolution Infrared Array Sensor for Counting and Localizing People Indoors: When Low End Technology Meets Cutting Edge Deep Learning Techniques. Information, 13.","DOI":"10.3390\/info13030132"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4874","DOI":"10.1109\/JSEN.2020.3029810","article-title":"Enhancing PIR-Based Multi-Person Localization Through Combining Deep Learning With Domain Knowledge","volume":"21","author":"Yang","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_15","first-page":"2739","article-title":"Locate, Size, and Count: Accurately Resolving People in Dense Crowds via Detection","volume":"43","author":"Sam","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Wu, C., Zhang, F., Wang, B., and Ray Liu, K.J. (2020, January 6\u20139). mmTrack: Passive Multi-Person Localization Using Commodity Millimeter Wave Radio. Proceedings of the IEEE INFOCOM 2020\u2014IEEE Conference on Computer Communications, Virtual.","DOI":"10.1109\/INFOCOM41043.2020.9155293"},{"key":"ref_17","first-page":"509","article-title":"Human activity recognition by infrared sensor arrays considering positional relation between user and sensors","volume":"116","author":"Kobayashi","year":"2017","journal-title":"IEICE Tech. Rep."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Quero, J.M., Burns, M., Razzaq, M.A., Nugent, C., and Espinilla, M. (2018). Detection of Falls from Non-Invasive Thermal Vision Sensors Using Convolutional Neural Networks. Proceedings, 2.","DOI":"10.3390\/proceedings2191236"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Burns, M., Cruciani, F., Morrow, P., Nugent, C., and McClean, S. (2020). Using Convolutional Neural Networks with Multiple Thermal Sensors for Unobtrusive Pose Recognition. Sensors, 20.","DOI":"10.3390\/s20236932"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Li, T., Yang, B., and Zhang, T. (2021, January 1\u20134). Human Action Recognition Based on State Detection in Low-resolution Infrared Video. Proceedings of the 2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA), Chengdu, China.","DOI":"10.1109\/ICIEA51954.2021.9516410"},{"key":"ref_21","first-page":"1550147720920485","article-title":"Evaluation of convolutional neural networks for the classification of falls from heterogeneous thermal vision sensors","volume":"16","author":"Espinilla","year":"2020","journal-title":"Int. J. Distrib. Sens. Netw."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Tateno, S., Meng, F., Qian, R., and Hachiya, Y. (2020). Privacy-Preserved Fall Detection Method with Three-Dimensional Convolutional Neural Network Using Low-Resolution Infrared Array Sensor. Sensors, 20.","DOI":"10.3390\/s20205957"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"82563","DOI":"10.1109\/ACCESS.2021.3084926","article-title":"A novel hybrid deep learning model for activity detection using wide-angle low-resolution infrared array sensor","volume":"9","author":"Muthukumar","year":"2021","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Dong, C., Loy, C.C., and Tang, X. (2016, January 11\u201314). Accelerating the super-resolution convolutional neural network. Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46475-6_25"},{"key":"ref_25","unstructured":"Ulyanov, D., Vedaldi, A., and Lempitsky, V. (2018, January 18\u201323). Deep image prior. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA."},{"key":"ref_26","unstructured":"Mirza, M., and Osindero, S. (2014). Conditional generative adversarial nets. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2179","DOI":"10.1007\/s40747-021-00428-4","article-title":"Methods for image denoising using convolutional neural network: A review","volume":"7","author":"Ilesanmi","year":"2021","journal-title":"Complex Intell. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1186\/s42492-019-0016-7","article-title":"Brief review of image denoising techniques","volume":"2","author":"Fan","year":"2019","journal-title":"Vis. Comput. Ind. Biomed. Art"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_31","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"705","DOI":"10.3390\/ai2040042","article-title":"Memory-Efficient AI Algorithm for Infant Sleeping Death Syndrome Detection in Smart Buildings","volume":"2","author":"Huang","year":"2021","journal-title":"AI"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"180","DOI":"10.3390\/ai3010011","article-title":"Weight-Quantized SqueezeNet for Resource-Constrained Robot Vacuums for Indoor Obstacle Classification","volume":"3","author":"Huang","year":"2022","journal-title":"AI"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. (2018, January 18\u201323). Quantization and training of neural networks for efficient integer-arithmetic-only inference. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00286"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/10\/3898\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:15:43Z","timestamp":1760138143000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/10\/3898"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,20]]},"references-count":34,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["s22103898"],"URL":"https:\/\/doi.org\/10.3390\/s22103898","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,20]]}}}