{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T10:53:58Z","timestamp":1768820038798,"version":"3.49.0"},"reference-count":26,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2022,5,31]],"date-time":"2022-05-31T00:00:00Z","timestamp":1653955200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and Technology","award":["MOST 109-2221-E-011-121-MY3"],"award-info":[{"award-number":["MOST 109-2221-E-011-121-MY3"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Falling is a major cause of personal injury and accidental death worldwide, in particular for the elderly. For aged care, a falling alarm system is highly demanded so that medical aid can be obtained immediately when the fall accidents happen. Previous studies on fall detection lacked practical considerations to deal with real-world situations, including the camera\u2019s mounting angle, lighting differences between day and night, and the privacy protection for users. In our experiments, IR-depth images and thermal images were used as the input source for fall detection; as a result, detailed facial information is not captured by the system for privacy reasons, and it is invariant to the lighting conditions. Due to the different occurrence rates between fall accidents and other normal activities, supervised learning approaches may suffer from the problem of data imbalance in the training phase. Accordingly, in this study, anomaly detection is performed using unsupervised learning approaches so that the models were trained only with the normal cases while the fall accident was defined as an anomaly event. The proposed system takes sequential frames as the inputs to predict future frames based on a GAN structure, and it provides (1) multi-subject detection, (2) real-time fall detection triggered by motion, (3) a solution to the situation that subjects were occluded after falling, and (4) a denoising scheme for depth images. The experimental results show that the proposed system achieves the state-of-the-art performance and copes with the real-world cases successfully.<\/jats:p>","DOI":"10.3390\/s22114194","type":"journal-article","created":{"date-parts":[[2022,5,31]],"date-time":"2022-05-31T09:24:29Z","timestamp":1653989069000},"page":"4194","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Fall Detection with the Spatial-Temporal Correlation Encoded by a Sequence-to-Sequence Denoised GAN"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8554-1265","authenticated-orcid":false,"given":"Wei-Wen","family":"Hsu","sequence":"first","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Taitung University, Tatung 950309, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8041-6326","authenticated-orcid":false,"given":"Jing-Ming","family":"Guo","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei 106335, Taiwan"},{"name":"Advanced Intelligent Image and Vision Technology Research Center, National Taiwan University of Science and Technology, Taipei 106335, Taiwan"}]},{"given":"Chien-Yu","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Taiwan University of Science and Technology, Taipei 106335, Taiwan"}]},{"given":"Yao-Chung","family":"Chang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Taitung University, Tatung 950309, Taiwan"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,31]]},"reference":[{"key":"ref_1","unstructured":"World Health Organization (2015). World Report on Ageing and Health, World Health Organization."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1001\/jama.1993.03500010075035","article-title":"Predictors and prognosis of inability to get up after falls among elderly persons","volume":"269","author":"Tinetti","year":"1993","journal-title":"JAMA"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1186\/1475-925X-12-66","article-title":"Challenges, issues and trends in fall detection systems","volume":"12","author":"Igual","year":"2013","journal-title":"Biomed. Eng. Online"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1109\/TNSRE.2007.916282","article-title":"Portable preimpact fall detector with inertial sensors","volume":"16","author":"Wu","year":"2008","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2994","DOI":"10.1109\/JSYST.2017.2780260","article-title":"A skeleton-free fall detection system from depth images using random decision forest","volume":"12","author":"Abobakr","year":"2017","journal-title":"IEEE Syst. J."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1080\/14639230410001684387","article-title":"Older adults\u2019 attitudes towards and perceptions of \u2018smart home\u2019technologies: A pilot study","volume":"29","author":"Demiris","year":"2004","journal-title":"Med. Inform. Internet Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1109\/JBHI.2014.2312180","article-title":"Fall detection in homes of older adults using the Microsoft Kinect","volume":"19","author":"Stone","year":"2014","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Abobakr, A., Hossny, M., Abdelkader, H., and Nahavandi, S. (2018, January 10\u201313). Rgb-d fall detection via deep residual convolutional lstm networks. Proceedings of the 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia.","DOI":"10.1109\/DICTA.2018.8615759"},{"key":"ref_9","first-page":"9474806","article-title":"Vision-based fall detection with convolutional neural networks","volume":"2017","author":"Azkune","year":"2017","journal-title":"Wirel. Commun. Mob. Comput."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"153049","DOI":"10.1109\/ACCESS.2019.2947518","article-title":"Implementation of Fall Detection System Based on 3D Skeleton for Deep Learning Technique","volume":"7","author":"Tsai","year":"2019","journal-title":"IEEE Access"},{"key":"ref_12","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, University of Toronto."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1007\/s41666-019-00061-4","article-title":"DeepFall: Non-Invasive Fall Detection with Deep Spatio-Temporal Convolutional Autoencoders","volume":"4","author":"Nogas","year":"2020","journal-title":"J. Healthc. Inform. Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the dimensionality of data with neural networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Dosovitskiy, A., Fischer, P., Ilg, E., H\u00e4usser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D., and Brox, T. (2015, January 7\u201313). Flownet: Learning optical flow with convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.316"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Vadivelu, S., Ganesan, S., Murthy, O.R., and Dhall, A. (2016, January 20\u201324). Thermal imaging based elderly fall detection. Proceedings of the Asian Conference on Computer Vision, Taipei, Taiwan.","DOI":"10.1007\/978-3-319-54526-4_40"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.cmpb.2014.09.005","article-title":"Human fall detection on embedded platform using depth maps and wireless accelerometer","volume":"117","author":"Kwolek","year":"2014","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. (2011, January 20\u201325). Real-time human pose recognition in parts from single depth images. Proceedings of the CVPR 2011, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995316"},{"key":"ref_19","unstructured":"Bertalmio, M., Bertozzi, A.L., and Sapiro, G. (2001, January 8\u201314). Navier-stokes, fluid dynamics, and image and video inpainting. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhu, J.-Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22\u201329). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Li, C., and Wand, M. (2016, January 11\u201314). Precomputed real-time texture synthesis with markovian generative adversarial networks. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46487-9_43"},{"key":"ref_24","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Goudelis, G., Tsatiris, G., Karpouzis, K., and Kollias, S. (2015, January 16\u201318). Fall detection using history triple features. Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments, Athens, Greece.","DOI":"10.1145\/2769493.2769562"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"6405","DOI":"10.1007\/s11042-017-4549-5","article-title":"Combined curvelets and hidden Markov models for human fall detection","volume":"77","author":"Zerrouki","year":"2018","journal-title":"Multimed. Tools Appl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/11\/4194\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:22:43Z","timestamp":1760138563000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/11\/4194"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,31]]},"references-count":26,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["s22114194"],"URL":"https:\/\/doi.org\/10.3390\/s22114194","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,31]]}}}