{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,2]],"date-time":"2026-02-02T19:46:53Z","timestamp":1770061613692,"version":"3.49.0"},"reference-count":46,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2024,9,30]],"date-time":"2024-09-30T00:00:00Z","timestamp":1727654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China (NSFC)\u2019s Key Project Number for International Cooperation","award":["61520106007"],"award-info":[{"award-number":["61520106007"]}]},{"name":"National Natural Science Foundation of China (NSFC)\u2019s Key Project Number for International Cooperation","award":["62027827"],"award-info":[{"award-number":["62027827"]}]},{"name":"Development of Heart-Sound Cardio-Ultrasonic Multimodal Auxiliary Diagnostic Equipment for Fetal Hearts","award":["61520106007"],"award-info":[{"award-number":["61520106007"]}]},{"name":"Development of Heart-Sound Cardio-Ultrasonic Multimodal Auxiliary Diagnostic Equipment for Fetal Hearts","award":["62027827"],"award-info":[{"award-number":["62027827"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Graph Neural Networks (GNNs) have gained popularity in image matching methods, proving useful for various computer vision tasks like Structure from Motion (SfM) and 3D reconstruction. A well-known example is SuperGlue. Lightweight variants, such as LightGlue, have been developed with a focus on stacking fewer GNN layers compared to SuperGlue. This paper proposes the h-GNN, a lightweight image matching model, with improvements in the two processing modules, the GNN and matching modules. After image features are detected and described as keypoint nodes of a base graph, the GNN module, which primarily aims at increasing the h-GNN\u2019s depth, creates successive hierarchies of compressed-size graphs from the base graph through a clustering technique termed SC+PCA. SC+PCA combines Principal Component Analysis (PCA) with Spectral Clustering (SC) to enrich nodes with local and global information during graph clustering. A dual non-contrastive clustering loss is used to optimize graph clustering. Additionally, four message-passing mechanisms have been proposed to only update node representations within a graph cluster at the same hierarchical level or to update node representations across graph clusters at different hierarchical levels. The matching module performs iterative pairwise matching on the enriched node representations to obtain a scoring matrix. This matrix comprises scores indicating potential correct matches between the image keypoint nodes. The score matrix is refined with a \u2018dustbin\u2019 to further suppress unmatched features. There is a reprojection loss used to optimize keypoint match positions. The Sinkhorn algorithm generates a final partial assignment from the refined score matrix. Experimental results demonstrate the performance of the proposed h-GNN against competing state-of-the-art (SOTA) GNN-based methods on several image matching tasks under homography, estimation, indoor and outdoor camera pose estimation, and 3D reconstruction on multiple datasets. Experiments also demonstrate improved computational memory and runtime, approximately 38.1% and 26.14% lower than SuperGlue, and an average of about 6.8% and 7.1% lower than LightGlue. Future research will explore the effects of integrating more recent simplicial message-passing mechanisms, which concurrently update both node and edge representations, into our proposed model.<\/jats:p>","DOI":"10.3390\/info15100602","type":"journal-article","created":{"date-parts":[[2024,9,30]],"date-time":"2024-09-30T12:06:32Z","timestamp":1727697992000},"page":"602","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Hierarchical Graph Neural Network: A Lightweight Image Matching Model with Enhanced Message Passing of Local and Global Information in Hierarchical Graph Neural Networks"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6920-5288","authenticated-orcid":false,"given":"Enoch","family":"Opanin Gyamfi","sequence":"first","affiliation":[{"name":"School of Information and Software Engineering (SISE), University of Electronic Science and Technology of China (UESTC), Chengdu 610054, China"},{"name":"Department of Cyber Security and Computer Engineering Technology (DCSCET), School of Computing and Information Sciences (SCIS), C.K. Tedam University of Technology and Applied Sciences (CKT-UTAS), Navrongo P.O. Box 24, Upper East Region, Ghana"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6745-6377","authenticated-orcid":false,"given":"Zhiguang","family":"Qin","sequence":"additional","affiliation":[{"name":"School of Information and Software Engineering (SISE), University of Electronic Science and Technology of China (UESTC), Chengdu 610054, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5955-0515","authenticated-orcid":false,"given":"Juliana","family":"Mantebea Danso","sequence":"additional","affiliation":[{"name":"School of Information and Software Engineering (SISE), University of Electronic Science and Technology of China (UESTC), Chengdu 610054, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3164-0409","authenticated-orcid":false,"given":"Daniel","family":"Adu-Gyamfi","sequence":"additional","affiliation":[{"name":"Department of Cyber Security and Computer Engineering Technology (DCSCET), School of Computing and Information Sciences (SCIS), C.K. Tedam University of Technology and Applied Sciences (CKT-UTAS), Navrongo P.O. Box 24, Upper East Region, Ghana"}]}],"member":"1968","published-online":{"date-parts":[[2024,9,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"DeTone, D., Malisiewicz, T., and Rabinovich, A. (2018, January 18\u201323). SuperPoint: Self-Supervised Interest Point Detection and Description. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00060"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sarlin, P.-E., DeTone, D., Malisiewicz, T., and Rabinovich, A. (2020, January 13\u201319). SuperGlue: Learning Feature Matching with Graph Neural Networks. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00499"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Lindenberger, P., Sarlin, P.-E., and Pollefeys, M. (2023, January 1\u20136). LightGlue: Local Feature Matching at Light Speed. Proceedings of the 2023 IEEE\/CVF International Conference on Computer Vision, Paris, France.","DOI":"10.1109\/ICCV51070.2023.01616"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"116613","DOI":"10.1016\/j.eswa.2022.116613","article-title":"LifelongGlue: Keypoint matching for 3D reconstruction with continual neural networks","volume":"195","author":"Zaman","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"696","DOI":"10.1109\/TIP.2023.3346682","article-title":"Image Matching by Bare Homography","volume":"33","author":"Bellavia","year":"2024","journal-title":"IEEE Trans. Image Process."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"127125","DOI":"10.1016\/j.neucom.2023.127125","article-title":"A critical analysis of image-based camera pose estimation techniques","volume":"570","author":"Xu","year":"2024","journal-title":"Neurocomputing"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Cao, M., Jia, W., Lv, Z., Zheng, L., and Liu, X. (2019). SuperPixel-Based Feature Tracking for Structure from Motion. Appl. Sci., 9.","DOI":"10.3390\/app9152961"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Liu, Y., Huang, K., Li, J., Li, X., Zeng, Z., Chang, L., and Zhou, J. (2022). AdaSG: A Lightweight Feature Point Matching Method Using Adaptive Descriptor with GNN for VSLAM. Sensors, 22.","DOI":"10.3390\/s22165992"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Salimpour, S., Queralta, J.P., and Westerlund, T. (2022, January 5\u20137). Self-calibrating anomaly and change detection for autonomous inspection robots. Proceedings of the 6th IEEE International Conference on Robotic Computing, Naples, Italy.","DOI":"10.1109\/IRC55401.2022.00042"},{"key":"ref_10","unstructured":"Le, V.P., and De Tran, C. (2015, January 25\u201328). Key-point matching with post-filter using sift and brief in logo spotting. Proceedings of the 2015 IEEE International Conference on Computing & Communication Technologies-Research, Innovation, and Vision for Future, Can Tho, Vietnam."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Xu, Y., Li, Y.J., Weng, X., and Kitani, K. (2021, January 20\u201325). Wide-baseline multi-camera calibration using person re-identification. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01293"},{"key":"ref_12","unstructured":"Zhou, Y., Guo, Y., Lin, K.P., Yang, F., and Li, L. (2023). USuperGlue: An unsupervised UAV image matching network based on local self-attention. Soft Comput., 1\u201321."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s11263-020-01359-2","article-title":"Image matching from handcrafted to deep features: A survey","volume":"129","author":"Ma","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1186\/s40537-023-00876-4","article-title":"A review of graph neural networks: Concepts, architectures, techniques, challenges, datasets, applications, and future directions","volume":"11","author":"Khemani","year":"2024","journal-title":"J. Big Data"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.aiopen.2021.05.002","article-title":"Robustness of deep learning models on graphs: A survey","volume":"2","author":"Xu","year":"2021","journal-title":"AI Open"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/TNNLS.2020.2978386","article-title":"A comprehensive survey on graph neural networks","volume":"32","author":"Wu","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1007\/978-3-030-40245-7_10","article-title":"Message Passing Neural Networks","volume":"Volume 968","author":"Chmiela","year":"2020","journal-title":"Machine Learning Meets Quantum Physics"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ahmed, A., Shervashidze, N., Narayanamurthy, S.M., Josifovski, V., and Smola, A.J. (2013, January 13\u201317). Distributed large-scale natural graph factorization. Proceedings of the 22nd International World Wide Web Conference, Janeiro, Brazil.","DOI":"10.1145\/2488388.2488393"},{"key":"ref_19","unstructured":"Alon, U., and Yahav, E. (2021, January 3\u20137). On the bottleneck of graph neural networks and its practical implications. Proceedings of the 9th International Conference on Learning Representations, Vienna, Austria. Available online: https:\/\/openreview.net\/pdf?id=i80OPhOCVH2."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1007\/s10618-022-00890-9","article-title":"Hierarchical message-passing graph neural networks","volume":"37","author":"Zhong","year":"2023","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_21","unstructured":"Oono, K., and Suzuki, T. (2020, January 26\u201330). Graph neural networks exponentially lose expressive power for node classification. Proceedings of the 8th International Conference on Learning Representations, Addis Ababa, Ethiopia. Available online: https:\/\/openreview.net\/forum?id=S1ldO2EFPr."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1016\/j.neunet.2021.11.001","article-title":"Multi-level attention pooling for graph neural networks: Unifying graph representations with multiple localities","volume":"145","author":"Itoh","year":"2022","journal-title":"Neural Netw."},{"key":"ref_23","unstructured":"Xu, K., Hu, W., Leskovec, J., and Jegelka, S. (2019, January 6\u20139). How powerful are graph neural networks?. Proceedings of the 7th International Conference on Learning Representations, New Orleans, LA, USA. Available online: https:\/\/openreview.net\/pdf?id=ryGs6iA5Km."},{"key":"ref_24","unstructured":"Xu, K., Li, C., Tian, Y., Sonobe, T., Kawarabayashi, K.I., and Jegelka, S. (2018, January 10\u201315). Representation learning on graphs with jumping knowledge networks. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden. Available online: https:\/\/proceedings.mlr.press\/v80\/xu18c.html."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Li, Q., Han, Z., and Wu, X. (2018, January 2\u20137). Deeper insights into graph convolutional networks for semi-supervised learning. Proceedings of the 2018 AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11604"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhou, S., Liu, X., Zhu, C., Liu, Q., and Yin, J. (2014, January 6\u201311). Spectral clustering-based local and global structure preservation for feature selection. Proceedings of the 2014 International Joint Conference on Neural Networks, Beijing, China.","DOI":"10.1109\/IJCNN.2014.6889641"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1665","DOI":"10.1109\/TKDE.2009.40","article-title":"Clustering with local and global regularization","volume":"21","author":"Wang","year":"2009","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.cosrev.2007.05.001","article-title":"Graph clustering","volume":"1","author":"Elisa","year":"2007","journal-title":"Comput. Sci. Rev."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Shun, J., and Blelloch, G.E. (2013, January 23\u201327). Ligra: A lightweight graph processing framework for shared memory. Proceedings of the 18th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Shenzhen, China.","DOI":"10.1145\/2442516.2442530"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Li, G., Rao, W., and Jin, Z. (2017, January 7\u20139). Efficient compression on real world directed graphs. Proceedings of the Web and Big Data 1st International Joint Conference, APWeb-WAIM, Beijing, China.","DOI":"10.1007\/978-3-319-63579-8_10"},{"key":"ref_31","unstructured":"Ma, E.J. (2024, January 02). Computational Representations of Message Passing\u2014Essays on Data Science. Available online: https:\/\/ericmjl.github.io\/essays-on-data-science\/machine-learning\/message-passing."},{"key":"ref_32","first-page":"1952","article-title":"Propagation enhanced neural message passing for graph representation learning","volume":"35","author":"Fan","year":"2021","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Tu, W., Guan, R., Zhou, S., Ma, C., Peng, X., Cai, Z., Liu, Z., Cheng, J., and Liu, X. (2024, January 20\u201327). Attribute-Missing Graph Clustering Network. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i14.29464"},{"key":"ref_34","unstructured":"Revaud, J., De Souza, C., Humenberger, M., and Weinzaepfel, P. (2019). R2D2: Repeatable and reliable detector and descriptor. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Balntas, V., Lenc, K., Vedaldi, A., and Mikolajczyk, K. (2017, January 21\u201326). KHPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.410"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Radenovi\u0107, F., Iscen, A., Tolias, G., Avrithis, Y., and Chum, O. (2018, January 18\u201323). Revisiting Oxford and Paris: Large-scale image retrieval benchmarking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00598"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Yi, K.M., Trulls, E., Ono, Y., Lepetit, V., Salzmann, M., and Fua, P. (2018, January 18\u201323). Learning to find good correspondences. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00282"},{"key":"ref_39","unstructured":"Zhang, J., Sun, D., Luo, Z., Yao, A., Zhou, L., Shen, T., Chen, Y., Liao, H., and Quan, L. (November, January 27). Learning two-view correspondences and geometry using order-aware network. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Chen, H., Luo, Z., Zhang, J., Zhou, L., Bai, X., Hu, Z., Tai, C.-L., and Quan, L. (2021, January 11\u201317). Learning to Match Features with Seeded Graph Matching Network. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00624"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Li, Z., and Snavely, N. (2018, January 18\u201323). MegaDepth: Learning single-view depth prediction from internet photos. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00218"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Sun, J., Shen, Z., Wang, Y., Bao, H., and Zhou, X. (2021, January 11\u201317). LoFTR: Detector-Free Local Feature Matching with Transformers. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, QC, Canada.","DOI":"10.1109\/CVPR46437.2021.00881"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Xiao, J., Owens, A., and Torralba, A. (2013, January 1\u20138). SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.458"},{"key":"ref_44","first-page":"023039","article-title":"Image matching via the local neighborhood for low inlier ratio","volume":"31","author":"Wang","year":"2022","journal-title":"J. Electron. Imaging"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.isprsjprs.2022.06.009","article-title":"Learning for mismatch removal via graph attention networks","volume":"190","author":"Jiang","year":"2022","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Truong, Q., and Chin, P. (2024, January 20\u201327). Weisfeiler and Lehman Go Paths: Learning Topological Features via Path Complexes. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i14.29463"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/10\/602\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:08:07Z","timestamp":1760112487000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/10\/602"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,30]]},"references-count":46,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2024,10]]}},"alternative-id":["info15100602"],"URL":"https:\/\/doi.org\/10.3390\/info15100602","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,30]]}}}