{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T17:10:26Z","timestamp":1772644226758,"version":"3.50.1"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,11,20]],"date-time":"2019-11-20T00:00:00Z","timestamp":1574208000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,11,20]],"date-time":"2019-11-20T00:00:00Z","timestamp":1574208000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["15K12037"],"award-info":[{"award-number":["15K12037"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["IPSJ T Comput Vis Appl"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Gait-based features provide the potential for a subject to be recognized even from a low-resolution image sequence, and they can be captured at a distance without the subject\u2019s cooperation. Person recognition using gait-based features (gait recognition) is a promising real-life application. However, several body parts of the subjects are often occluded because of beams, pillars, cars and trees, or another walking person. Therefore, gait-based features are not applicable to approaches that require an unoccluded gait image sequence. Occlusion handling is a challenging but important issue for gait recognition. In this paper, we propose silhouette sequence reconstruction from an occluded sequence (sVideo) based on a conditional deep generative adversarial network (GAN). From the reconstructed sequence, we estimate the gait cycle and extract the gait features from a one gait cycle image sequence. To regularize the training of the proposed generative network, we use adversarial loss based on triplet hinge loss incorporating Wasserstein GAN (WGAN-hinge). To the best of our knowledge, WGAN-hinge is the first adversarial loss that supervises the generator network during training by incorporating pairwise similarity ranking information. The proposed approach was evaluated on multiple challenging occlusion patterns. The experimental results demonstrate that the proposed approach outperforms the existing state-of-the-art benchmarks.<\/jats:p>","DOI":"10.1186\/s41074-019-0061-3","type":"journal-article","created":{"date-parts":[[2019,11,20]],"date-time":"2019-11-20T14:02:57Z","timestamp":1574258577000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["Spatio-temporal silhouette sequence reconstruction for gait recognition against occlusion"],"prefix":"10.1186","volume":"11","author":[{"given":"Md. Zasim","family":"Uddin","sequence":"first","affiliation":[]},{"given":"Daigo","family":"Muramatsu","sequence":"additional","affiliation":[]},{"given":"Noriko","family":"Takemura","sequence":"additional","affiliation":[]},{"given":"Md. Atiqur Rahman","family":"Ahad","sequence":"additional","affiliation":[]},{"given":"Yasushi","family":"Yagi","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,11,20]]},"reference":[{"key":"61_CR1","unstructured":"M Arjovsky, S Chintala, L Bottou, Wasserstein GAN. CoRR (2017). abs\/1701.07875, 1701.07875."},{"key":"61_CR2","unstructured":"LJ Ba, R Kiros, GE Hinton, Layer normalization. CoRR (2016). abs\/1607.06450."},{"key":"61_CR3","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1145\/344779.344972","volume-title":"Proc. of the 27th Annual Conf. on Computer Graphics and Interactive Techniques. SIGGRAPH \u201900","author":"M Bertalmio","year":"2000","unstructured":"M Bertalmio, G Sapiro, V Caselles, C Ballester, in Proc. of the 27th Annual Conf. on Computer Graphics and Interactive Techniques. SIGGRAPH \u201900. Image inpainting (ACM Press\/Addison-Wesley Publishing Co.New York, 2000), pp. 417\u2013424."},{"key":"61_CR4","doi-asserted-by":"publisher","unstructured":"H Cai, C Bai, Y Tai, C Tang, in Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14 2018, Proceedings, Part II. Deep video generation, prediction and completion of human action sequences, (2018), pp. 374\u2013390. https:\/\/doi.org\/10.1007\/978-3-030-01216-8_23.","DOI":"10.1007\/978-3-030-01216-8_23"},{"issue":"11","key":"61_CR5","doi-asserted-by":"publisher","first-page":"977","DOI":"10.1016\/j.patrec.2009.04.012","volume":"30","author":"C Chen","year":"2009","unstructured":"C Chen, J Liang, H Zhao, H Hu, J Tian, Frame difference energy image for gait recognition with incomplete silhouettes. Pattern Recogn Lett. 30(11), 977\u2013984 (2009).","journal-title":"Pattern Recogn Lett"},{"key":"61_CR6","first-page":"658","volume-title":"Proc. of the Int. Conf. on Neural Information Processing Systems","author":"A Dosovitskiy","year":"2016","unstructured":"A Dosovitskiy, T Brox, in Proc. of the Int. Conf. on Neural Information Processing Systems. Generating images with perceptual similarity metrics based on deep networks (Curran Associates Inc.USA, 2016), pp. 658\u2013666."},{"key":"61_CR7","doi-asserted-by":"publisher","unstructured":"AA Efros, TK Leung, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, vol 2. Texture synthesis by non-parametric sampling, (1999), pp. 1033\u20131038. https:\/\/doi.org\/10.1109\/iccv.1999.790383.","DOI":"10.1109\/iccv.1999.790383"},{"key":"61_CR8","first-page":"2672","volume-title":"Proc. of the Int. Conf. on Neural Information Processing Systems - Vol 2","author":"IJ Goodfellow","year":"2014","unstructured":"IJ Goodfellow, J Pouget-Abadie, M Mirza, B Xu, D Warde-Farley, S Ozair, A Courville, Y Bengio, in Proc. of the Int. Conf. on Neural Information Processing Systems - Vol 2. Generative adversarial nets (MIT PressCambridge, 2014), pp. 2672\u20132680."},{"key":"61_CR9","unstructured":"I Gulrajani, F Ahmed, M Arjovsky, V Dumoulin, AC Courville, in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017. Improved training of Wasserstein GANS (Long Beach, 2017), pp. 5769\u20135779."},{"key":"61_CR10","doi-asserted-by":"publisher","unstructured":"J Han, B Bhanu, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, vol 2. Statistical feature fusion for gait-based human recognition, (2004), pp. 842\u2013847. https:\/\/doi.org\/10.1109\/cvpr.2004.1315252.","DOI":"10.1109\/cvpr.2004.1315252"},{"key":"61_CR11","doi-asserted-by":"publisher","unstructured":"K He, X Zhang, S Ren, J Sun, in Proc. of the IEEE Int. Conf. on Computer Vision. Delving deep into rectifiers: surpassing human-level performance on imagenet classification (Washington, 2015), pp. 1026\u20131034. https:\/\/doi.org\/10.1109\/iccv.2015.123.","DOI":"10.1109\/iccv.2015.123"},{"key":"61_CR12","doi-asserted-by":"publisher","unstructured":"M Hofmann, D Wolf, G Rigoll, in Proc. of the Int. Conf. on Computer Vision Theory and Applications. Identification and reconstruction of complete gait cycles for person identification in crowded scenes (Vilamoura, 2011), pp. 594\u2013597. https:\/\/doi.org\/10.5220\/0003329305940597.","DOI":"10.5220\/0003329305940597"},{"issue":"6","key":"61_CR13","doi-asserted-by":"publisher","first-page":"2281","DOI":"10.1016\/j.patcog.2009.12.020","volume":"43","author":"MA Hossain","year":"2010","unstructured":"MA Hossain, Y Makihara, J Wang, Y Yagi, Clothing-invariant gait identification using part-based clothing categorization and adaptive weight control. Pattern Recogn. 43(6), 2281\u20132291 (2010).","journal-title":"Pattern Recogn"},{"key":"61_CR14","unstructured":"How biometrics could change security, BBC (online). available from http:\/\/news.bbc.co.uk\/2\/hi\/programmes\/click_online\/7702065.stm."},{"issue":"4","key":"61_CR15","doi-asserted-by":"publisher","first-page":"107:1","DOI":"10.1145\/3072959.3073659","volume":"36","author":"S Iizuka","year":"2017","unstructured":"S Iizuka, E Simo-Serra, H Ishikawa, Globally and locally consistent image completion. ACM Trans Graph. 36(4), 107:1\u2013107:14 (2017).","journal-title":"ACM Trans Graph"},{"key":"61_CR16","first-page":"448","volume-title":"Proc. of the Int. Conf. on International Conference on Machine Learning - Vol 37","author":"S Ioffe","year":"2015","unstructured":"S Ioffe, C Szegedy, in Proc. of the Int. Conf. on International Conference on Machine Learning - Vol 37. Batch normalization: accelerating deep network training by reducing internal covariate shift (PMLRLille, 2015), pp. 448\u2013456."},{"issue":"5","key":"61_CR17","doi-asserted-by":"publisher","first-page":"1511","DOI":"10.1109\/TIFS.2012.2204253","volume":"7","author":"H Iwama","year":"2012","unstructured":"H Iwama, M Okumura, Y Makihara, Y Yagi, The OU-ISIR Gait Database comprising the large population dataset and performance evaluation of gait recognition. IEEE Trans Inf Forensic Secur. 7(5), 1511\u20131521 (2012).","journal-title":"IEEE Trans Inf Forensic Secur"},{"issue":"1","key":"61_CR18","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1109\/TPAMI.2012.59","volume":"35","author":"S Ji","year":"2013","unstructured":"S Ji, W Xu, M Yang, K Yu, 3D convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell. 35(1), 221\u2013231 (2013).","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"61_CR19","unstructured":"DP Kingma, J Ba, Adam: a method for stochastic optimization. CoRR (2014). abs\/1412.6980."},{"key":"61_CR20","unstructured":"B Kratzwald, Z Huang, DP Paudel, LV Gool, Improving video generation for multi-functional applications. CoRR (2017). abs\/1711.11453, 1711.11453."},{"key":"61_CR21","doi-asserted-by":"publisher","unstructured":"Y Li, S Liu, J Yang, M Yang, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. Generative face completion (Honolulu, 2017), pp. 5892\u20135900. https:\/\/doi.org\/10.1109\/cvpr.2017.624.","DOI":"10.1109\/cvpr.2017.624"},{"issue":"2","key":"61_CR22","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1109\/TSMCB.2004.842251","volume":"35","author":"Z Liu","year":"2005","unstructured":"Z Liu, S Sarkar, Effect of silhouette quality on hard problems in gait recognition. IEEE Trans Syst Man Cybern Part B Cybern. 35(2), 170\u2013183 (2005).","journal-title":"IEEE Trans Syst Man Cybern Part B Cybern"},{"key":"61_CR23","doi-asserted-by":"publisher","unstructured":"C Lu, M Hirsch, B Sch\u00f6lkopf, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. Flexible spatio-temporal networks for video prediction (Honolulu, 2017), pp. 2137\u20132145. https:\/\/doi.org\/10.1109\/cvpr.2017.230.","DOI":"10.1109\/cvpr.2017.230"},{"issue":"12","key":"61_CR24","doi-asserted-by":"publisher","first-page":"3793","DOI":"10.1016\/j.patcog.2014.06.010","volume":"47","author":"R Mart\u00edn-F\u00e9lez","year":"2014","unstructured":"R Mart\u00edn-F\u00e9lez, T Xiang, Uncooperative gait recognition by learning to rank. Pattern Recogn. 47(12), 3793\u20133806 (2014).","journal-title":"Pattern Recogn"},{"key":"61_CR25","unstructured":"M Mirza, S Osindero, Conditional generative adversarial nets. CoRR (2014). abs\/1411.1784."},{"key":"61_CR26","doi-asserted-by":"publisher","unstructured":"D Muramatsu, Y Makihara, Y Yagi, in Int. Conf. on Biometrics (ICB). Gait regeneration for recognition, (2015a), pp. 169\u2013176. https:\/\/doi.org\/10.1109\/icb.2015.7139048.","DOI":"10.1109\/icb.2015.7139048"},{"key":"61_CR27","doi-asserted-by":"crossref","unstructured":"D Muramatsu, A Shiraishi, Y Makihara, M Uddin, Y Yagi, Gait-based person recognition using arbitrary view transformation model. IEEE Trans Image Process. 24(1), 140\u2013154 (2015b).","DOI":"10.1109\/TIP.2014.2371335"},{"key":"61_CR28","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/j.jvcir.2016.01.008","volume":"36","author":"P Nangtin","year":"2016","unstructured":"P Nangtin, P Kumhom, K Chamnongthai, Gait identification with partial occlusion using six modules and consideration of occluded module exclusion. J Vis Commun Image Represent. 36:, 107\u2013121 (2016).","journal-title":"J Vis Commun Image Represent"},{"issue":"1","key":"61_CR29","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1007\/s00138-016-0798-y","volume":"28","author":"J Ortells","year":"2017","unstructured":"J Ortells, RA Mollineda, B Mederos, R Mart\u00edn-F\u00e9lez, Gait recognition from corrupted silhouettes: a robust statistical approach. Mach Vis Appl. 28(1), 15\u201333 (2017).","journal-title":"Mach Vis Appl"},{"key":"61_CR30","doi-asserted-by":"publisher","unstructured":"D Pathak, P Kr\u00e4henb\u00fchl, J Donahue, T Darrell, AA Efros, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. Context encoders: feature learning by inpainting (Las Vegas, 2016), pp. 2536\u20132544. https:\/\/doi.org\/10.1109\/cvpr.2016.278.","DOI":"10.1109\/cvpr.2016.278"},{"key":"61_CR31","unstructured":"A Radford, L Metz, S Chintala, in Int. Conf. on Learning Representations. Unsupervised representation learning with deep convolutional generative adversarial networks (San Juan, 2016)."},{"key":"61_CR32","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1007\/978-3-319-24574-4_28","volume-title":"Medical Image Computing and Computer-Assisted Intervention \u2013 MICCAI 2015","author":"O Ronneberger","year":"2015","unstructured":"O Ronneberger, P Fischer, Thomas Be, J Hornegger, WM Wells, AF Frangi, in Medical Image Computing and Computer-Assisted Intervention \u2013 MICCAI 2015. U-net: Convolutional networks for biomedical image segmentation (Springer International PublishingCham, 2015), pp. 234\u2013241."},{"issue":"4","key":"61_CR33","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1007\/s11760-011-0245-5","volume":"5","author":"A Roy","year":"2011","unstructured":"A Roy, S Sural, J Mukherjee, G Rigoll, Occlusion detection and gait silhouette reconstruction from degraded scenes. Signal Image Video Proc. 5(4), 415 (2011).","journal-title":"Signal Image Video Proc"},{"issue":"1","key":"61_CR34","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1186\/s41074-018-0039-6","volume":"10","author":"N Takemura","year":"2018","unstructured":"N Takemura, Y Makihara, D Muramatsu, T Echigo, Y Yagi, Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition. IPSJ Trans Comput Vis Appl. 10(1), 4 (2018).","journal-title":"IPSJ Trans Comput Vis Appl"},{"key":"61_CR35","doi-asserted-by":"publisher","unstructured":"D Tran, L Bourdev, R Fergus, L Torresani, M Paluri, in Proc. of the IEEE Int. Conf. on Computer Vision. Learning spatiotemporal features with 3D convolutional networks (Washington, 2015), pp. 4489\u20134497. https:\/\/doi.org\/10.1109\/iccv.2015.510.","DOI":"10.1109\/iccv.2015.510"},{"issue":"1","key":"61_CR36","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1186\/s41074-017-0029-0","volume":"9","author":"M Uddin","year":"2017","unstructured":"M Uddin, D Muramatsu, T Kimura, Y Makihara, Y Yagi, MultiQ: single sensor-based multi-quality multi-modal large-scale biometric score database and its performance evaluation. IPSJ Trans Comput Vis Appl. 9(1), 18 (2017).","journal-title":"IPSJ Trans Comput Vis Appl"},{"issue":"1","key":"61_CR37","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1186\/s41074-018-0041-z","volume":"10","author":"M Uddin","year":"2018","unstructured":"M Uddin, TT Ngo, Y Makihara, N Takemura, X Li, D Muramatsu, Y Yagi, The OU-ISIR Large Population Gait Database with real-life carried object and its performance evaluation. IPSJ Trans Comput Vis Appl. 10(1), 5 (2018).","journal-title":"IPSJ Trans Comput Vis Appl"},{"key":"61_CR38","unstructured":"C Vondrick, H Pirsiavash, A Torralba, in Advances in Neural Information Processing Systems 29: Annual Conf. on Neural Information Processing Systems 2016. Generating videos with scene dynamics (Barcelona, 2016), pp. 613\u2013621."},{"key":"61_CR39","doi-asserted-by":"publisher","unstructured":"C Wang, H Huang, X Han, J Wang, Video inpainting by jointly learning temporal structure and spatial details. CoRR (2018). abs\/1806.08482, 1806.08482. https:\/\/doi.org\/10.1609\/aaai.v33i01.33015232.","DOI":"10.1609\/aaai.v33i01.33015232"},{"issue":"3","key":"61_CR40","doi-asserted-by":"publisher","first-page":"463","DOI":"10.1109\/TPAMI.2007.60","volume":"29","author":"Y Wexler","year":"2007","unstructured":"Y Wexler, E Shechtman, M Irani, Space-time completion of video. IEEE Trans Pattern Anal Mach Intell. 29(3), 463\u2013476 (2007).","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"61_CR41","unstructured":"B Xu, N Wang, T Chen, M Li, Empirical evaluation of rectified activations in convolutional network. CoRR (2015). abs\/1505.00853."},{"key":"61_CR42","doi-asserted-by":"publisher","unstructured":"RA Yeh, C Chen, T Lim, AG Schwing, M Hasegawa-Johnson, MN Do, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. Semantic image inpainting with deep generative models (Honolulu, 2017), pp. 6882\u20136890. https:\/\/doi.org\/10.1109\/cvpr.2017.728.","DOI":"10.1109\/cvpr.2017.728"},{"key":"61_CR43","unstructured":"F Yu, V Koltun, Multi-scale context aggregation by dilated convolutions. CoRR (2015). abs\/1511.07122."},{"key":"61_CR44","doi-asserted-by":"publisher","unstructured":"J Yu, Z Lin, J Yang, X Shen, X Lu, TS Huang, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. Generative image inpainting with contextual attention (Salt Lake City, 2018), pp. 5505\u20135514. https:\/\/doi.org\/10.1109\/cvpr.2018.00577.","DOI":"10.1109\/cvpr.2018.00577"},{"key":"61_CR45","doi-asserted-by":"publisher","unstructured":"S Yu, D Tan, T Tan, in Proc. of the 18th Int. Conf. on Pattern Recognition vol 4. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition (Hong Kong, 2006), pp. 441\u2013444. https:\/\/doi.org\/10.1109\/icpr.2006.67.","DOI":"10.1109\/icpr.2006.67"},{"key":"61_CR46","doi-asserted-by":"publisher","unstructured":"S Yu, H Chen, EBG Reyes, N Poh, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition Workshops. GaitGAN: invariant gait feature extraction using generative adversarial networks, (2017), pp. 532\u2013539. https:\/\/doi.org\/10.1109\/cvprw.2017.80.","DOI":"10.1109\/cvprw.2017.80"},{"key":"61_CR47","doi-asserted-by":"publisher","unstructured":"MD Zeiler, D Krishnan, GW Taylor, R Fergus, in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. Deconvolutional networks, (2010), pp. 2528\u20132535. https:\/\/doi.org\/10.1109\/iccv.2011.6126474.","DOI":"10.1109\/iccv.2011.6126474"},{"issue":"3","key":"61_CR48","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1007\/s10044-007-0064-z","volume":"10","author":"G Zhao","year":"2007","unstructured":"G Zhao, L Cui, H Li, Gait recognition using fractal scale. Pattern Anal Appl. 10(3), 235\u2013246 (2007).","journal-title":"Pattern Anal Appl"}],"container-title":["IPSJ Transactions on Computer Vision and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s41074-019-0061-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s41074-019-0061-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s41074-019-0061-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,30]],"date-time":"2021-07-30T12:15:02Z","timestamp":1627647302000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1186\/s41074-019-0061-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,20]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["61"],"URL":"https:\/\/doi.org\/10.1186\/s41074-019-0061-3","relation":{},"ISSN":["1882-6695"],"issn-type":[{"value":"1882-6695","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,20]]},"assertion":[{"value":"8 March 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 October 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 November 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare that they have no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"9"}}