{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:51:06Z","timestamp":1760241066005,"version":"build-2065373602"},"reference-count":50,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2019,11,19]],"date-time":"2019-11-19T00:00:00Z","timestamp":1574121600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61640214, 454 U1804152 and 61331021"],"award-info":[{"award-number":["61640214, 454 U1804152 and 61331021"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Image set matching (ISM) has attracted increasing attention in the field of computer vision and pattern recognition. Some studies attempt to model query and gallery sets under a joint or collaborative representation framework, achieving impressive performance. However, existing models consider only the competition and collaboration among gallery sets, neglecting the inter-instance relationships within the query set which are also regarded as one important clue for ISM. In this paper, inter-instance relationships within the query set are explored for robust image set matching. Specifically, we propose to represent the query set instances jointly via a combined dictionary learned from the gallery sets. To explore the commonality and variations within the query set simultaneously to benefit the matching, both low rank and class-level sparsity constraints are imposed on the representation coefficients. Then, to deal with nonlinear data in real scenarios, the\u2018kernelized version is also proposed. Moreover, to tackle the gross corruptions mixed in the query set, the proposed model is extended for robust ISM. The optimization problems are solved efficiently by employing singular value thresholding and block soft thresholding operators in an alternating direction manner. Experiments on five public datasets demonstrate the effectiveness of the proposed method, comparing favorably with state-of-the-art methods.<\/jats:p>","DOI":"10.3390\/s19225051","type":"journal-article","created":{"date-parts":[[2019,11,19]],"date-time":"2019-11-19T11:30:17Z","timestamp":1574163017000},"page":"5051","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Exploring Inter-Instance Relationships within the Query Set for Robust Image Set Matching"],"prefix":"10.3390","volume":"19","author":[{"given":"Deyin","family":"Liu","sequence":"first","affiliation":[{"name":"School of Information Engineering, Zhengzhou University, Zhengzhou 450001, China"},{"name":"School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane 4072, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6291-5212","authenticated-orcid":false,"given":"Chengwu","family":"Liang","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Zhengzhou University, Zhengzhou 450001, China"},{"name":"School of Electrical and Control Engineering, Henan University of Urban Construction, Pingdingshan 467036, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhiming","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Control Science and Engineering, Shandong University, Jinan 250100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lin","family":"Qi","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Zhengzhou University, Zhengzhou 450001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Brian C.","family":"Lovell","sequence":"additional","affiliation":[{"name":"School of Information Technology and Electrical Engineering, The University of Queensland, Brisbane 4072, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,11,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.eswa.2019.05.025","article-title":"Simultaneous learning of reduced prototypes and local metric for image set classification","volume":"134","author":"Ren","year":"2019","journal-title":"Expert Syst. Appl."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1016\/j.patcog.2017.11.020","article-title":"Regularized constraint subspace based method for image set classification","volume":"76","author":"Tan","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1109\/TPAMI.2014.2353635","article-title":"Deep reconstruction models for image set classification","volume":"37","author":"Hayat","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1007\/s11263-017-1000-3","article-title":"Empowering simple binary classifiers for image set based face recognition","volume":"123","author":"Hayat","year":"2017","journal-title":"Int. J. Comput. Vis."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.neucom.2018.09.090","article-title":"A review of image set classification","volume":"335","author":"Zhao","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Harandi, M., Salzmann, M., and Baktashmotlagh, M. (2015, January 7\u201313). Beyond gauss: Image-set matching on the riemannian manifold of pdfs. Proceedings of the IEEE International Conference on Computer Vision, Tampa, FL, USA.","DOI":"10.1109\/ICCV.2015.468"},{"key":"ref_7","unstructured":"Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018). Dependency-Aware Attention Control for Unconstrained Face Recognition with Image Sets. Computer Vision\u2013ECCV 2018, Springer International Publishing."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1318","DOI":"10.1109\/LSP.2017.2723084","article-title":"Prototype discriminative learning for image set classification","volume":"24","author":"Wang","year":"2017","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1992","DOI":"10.1109\/TPAMI.2011.283","article-title":"Face recognition using sparse approximated nearest points between image sets","volume":"34","author":"Hu","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1016\/j.knosys.2018.10.043","article-title":"Multi-model fusion metric learning for image set classification","volume":"164","author":"Gao","year":"2019","journal-title":"Knowl. Based Syst."},{"key":"ref_11","unstructured":"Arandjelovic, O., Shakhnarovich, G., Fisher, J., Cipolla, R., and Darrell, T. (2005, January 20\u201325). Face recognition with image sets using manifold density divergence. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_12","unstructured":"Lee, K.C., Ho, J., Yang, M.H., and Kriegman, D. (2003, January 18\u201320). Video-based face recognition using probabilistic appearance manifolds. Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI, USA."},{"key":"ref_13","unstructured":"Yamaguchi, O., Fukui, K., and Maeda, K.I. (1998, January 14\u201316). Face recognition using temporal image sequence. Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1005","DOI":"10.1109\/TPAMI.2007.1037","article-title":"Discriminative learning and recognition of image set classes using canonical correlations","volume":"29","author":"Kim","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Lu, J., Wang, G., Deng, W., Moulin, P., and Zhou, J. (2015, January 7\u201312). Multi-manifold deep metric learning for image set classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298717"},{"key":"ref_16","first-page":"151","article-title":"Discriminant Analysis on Riemannian Manifold of Gaussian Distributions for Face Recognition With Image Sets","volume":"27","author":"Wang","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"36395","DOI":"10.1109\/ACCESS.2018.2841855","article-title":"Kernelized Fast Algorithm for Regularized Hull-Based Face Recognition With Image Sets","volume":"6","author":"Tan","year":"2018","journal-title":"IEEE Access"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Cevikalp, H., and Triggs, B. (2010, January 13\u201318). Face recognition based on image sets. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539965"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1120","DOI":"10.1109\/TIFS.2014.2324277","article-title":"Image set-based collaborative representation for face recognition","volume":"9","author":"Zhu","year":"2014","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.imavis.2016.07.008","article-title":"Joint regularized nearest points for image set based face recognition","volume":"58","author":"Yang","year":"2017","journal-title":"Image Vis. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Cevikalp, H., Yavuz, H.S., and Triggs, B. (2019). Face Recognition Based on Videos by Using Convex Hulls. IEEE Trans. Circ. Syst. Video Technol.","DOI":"10.1109\/TCSVT.2019.2926165"},{"key":"ref_22","unstructured":"Jawahar, C., Li, H., Mori, G., and Schindler, K. (2019). Nonlinear Subspace Feature Enhancement for Image Set Classification. Computer Vision\u2013ACCV 2018, Springer International Publishing."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, R., Wu, X., Chen, K., and Kittler, J. (2018, January 20\u201324). Multiple Manifolds Metric Learning with Application to Image Set Classification. Proceedings of the 24th International Conference on Pattern Recognition, ICPR, Beijing, China.","DOI":"10.1109\/ICPR.2018.8546030"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Sun, H., Zhen, X., Zheng, Y., Yang, G., Yin, Y., and Li, S. (2017, January 21\u201326). Learning Deep Match Kernels for Image-Set Classification. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.661"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sun, H., Zhen, X., and Yin, Y. (2019, January 22\u201325). Learning the Set Graphs: Image-Set Classification Using Sparse Graph Convolutional Networks. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.","DOI":"10.1109\/ICIP.2019.8803557"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Liu, X., Guo, Z., Li, S., Kong, L., Jia, P., You, J., and Kumar, B.V.K.V. (2019, January 15\u201318). Permutation-invariant Feature Restructuring for Correlation-aware Image Set-based Recognition. Proceedings of the IEEE International Conference on Computer Vision, Jeju Island, Korea.","DOI":"10.1109\/ICCV.2019.00509"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sogi, N., Nakayama, T., and Fukui, K. (2018, January 8\u201313). A Method Based on Convex Cone Model for Image-Set Classification With CNN Features. Proceedings of the 2018 International Joint Conference on Neural Networks, IJCNN 2018, Rio de Janeiro, Brazil.","DOI":"10.1109\/IJCNN.2018.8489151"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Silva, C., Bouwmans, T., and Fr\u00e9licot, C. (2015, January 11\u201314). An eXtended Center-Symmetric Local Binary Pattern for Background Modeling and Subtraction in Videos. Proceedings of the 10th International Conference on Computer Vision Theory and Applications: VISIGRAPP 2015, Berlin, Germany.","DOI":"10.5220\/0005266303950402"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1109\/TPAMI.2008.79","article-title":"Robust Face Recognition via Sparse Representation","volume":"31","author":"Wright","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","unstructured":"Zhang, L., Yang, M., and Feng, X. (2011, January 6\u201313). Sparse representation or collaborative representation: Which helps face recognition?. Proceedings of the 2011 International conference on computer vision, Barcelona, Spain."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.sigpro.2016.10.022","article-title":"Joint sparse model-based discriminative K-SVD for hyperspectral image classification","volume":"133","author":"Wang","year":"2017","journal-title":"Signal Process."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1109\/TIP.2007.911828","article-title":"Sparse Representation for Color Image Restoration","volume":"17","author":"Mairal","year":"2008","journal-title":"IEEE Trans. Image Process."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"7086","DOI":"10.1109\/TGRS.2014.2307354","article-title":"Recovering Quantitative Remote Sensing Products Contaminated by Thick Clouds and Shadows Using Multitemporal Dictionary Learning","volume":"52","author":"Li","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.isprsjprs.2015.03.009","article-title":"Sparse-based reconstruction of missing information in remote sensing images from spectral\/temporal complementary information","volume":"106","author":"Li","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3629","DOI":"10.1109\/JSTARS.2016.2533547","article-title":"Patch Matching-Based Multitemporal Group Sparse Representation for the Missing Information Reconstruction of Remote-Sensing Images","volume":"9","author":"Li","year":"2016","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ortiz, E.G., Wright, A., and Shah, M. (2013, January 23\u201328). Face Recognition in Movie Trailers via Mean Sequence Sparse Representation- Based Classification. Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.453"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Fathy, M.E., and Chellappa, R. (2017, January 24\u201331). Image Set Classification Using Sparse Bayesian Regression. Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA.","DOI":"10.1109\/WACV.2017.137"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.ins.2018.02.062","article-title":"A set-level joint sparse representation for image set classification","volume":"448","author":"Zheng","year":"2018","journal-title":"Inf. Sci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1007\/s11263-018-1088-0","article-title":"Group Collaborative Representation for Image Set Classification","volume":"127","author":"Liu","year":"2019","journal-title":"Int. J. Comput. Vis."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1109\/TPAMI.2013.109","article-title":"Joint Sparse Representation for Robust Multimodal Biometrics Recognition","volume":"36","author":"Shekhar","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"4349","DOI":"10.1109\/TIP.2012.2205006","article-title":"Visual Classification With Multitask Joint Sparse Representation","volume":"21","author":"Yuan","year":"2012","journal-title":"IEEE Trans. Image Process."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1109\/TPAMI.2015.2462360","article-title":"Laplacian Regularized Low-Rank Representation and Its Applications","volume":"38","author":"Yin","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1956","DOI":"10.1137\/080738970","article-title":"A singular value thresholding algorithm for matrix completion","volume":"20","author":"Cai","year":"2010","journal-title":"SIAM J. Optim."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1561\/2400000003","article-title":"Proximal Algorithms","volume":"1","author":"Parikh","year":"2014","journal-title":"Found. Trends Optim."},{"key":"ref_45","unstructured":"Kim, M., Kumar, S., Pavlovic, V., and Rowley, H. (2008, January 23\u201328). Face tracking and recognition with visual constraints in real-world videos. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA."},{"key":"ref_46","unstructured":"Leibe, B., and Schiele, B. (2003, January 18\u201320). Analyzing appearance and contour based methods for object categorization. Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI, USA."},{"key":"ref_47","unstructured":"Chan, A.B., and Vasconcelos, N. (2005, January 20\u201325). Probabilistic kernels for the classification of auto-regressive visual processes. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_48","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Shroff, N., Turaga, P., and Chellappa, R. (2010, January 13\u201318). Moving vistas: Exploiting motion for describing scenes. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539864"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1715","DOI":"10.1109\/TCSVT.2018.2848543","article-title":"Fusing Object Semantics and Deep Appearance Features for Scene Recognition","volume":"29","author":"Sun","year":"2019","journal-title":"IEEE Trans. Circuits Syst. Video Technol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/22\/5051\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:35:47Z","timestamp":1760189747000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/22\/5051"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,19]]},"references-count":50,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2019,11]]}},"alternative-id":["s19225051"],"URL":"https:\/\/doi.org\/10.3390\/s19225051","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,11,19]]}}}