{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:13:21Z","timestamp":1760058801516,"version":"build-2065373602"},"reference-count":56,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,4,26]],"date-time":"2025-04-26T00:00:00Z","timestamp":1745625600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Knee osteoarthritis (KOA) is a highly prevalent muscoloskeletal joint disorder affecting a significant portion of the population worldwide. Accurate predictions of KOA progression can assist clinicians in drawing preventive strategies for patients. In this paper, we present an integrated approach based on hypergraph convolutional networks (HGCNs) for longitudinal predictions of KOA grades and progressions from MRI images. We propose two novel models, namely, the C_Shape.Net and the predictor network. The C_Shape.Net operates on a hypergraph of volumetric nodes, especially designed to represent the surface and volumetric features of the cartilage. It encompasses deep HGCN convolutions, graph pooling, and readout operations in a hierarchy of layers, providing, at the output, expressive 3D shape descriptors of the cartilage volume. The predictor is a spatio-temporal HGCN network (ST_HGCN), following the sequence-to-sequence learning scheme. Concretely, it transforms sequences of knee representations at the historical stage into sequences of KOA predictions at the prediction stage. The predictor includes spatial HGCN convolutions, attention-based temporal fusion of feature embeddings at multiple layers, and a transformer module that generates longitudinal predictions at follow-up times. We present comprehensive experiments on the Osteoarthritis Initiative (OAI) cohort to evaluate the performance of our methodology for various tasks, including node classification, longitudinal KL grading, and progression. The basic finding of the experiments is that the larger the depth of the historical stage, the higher the accuracy of the obtained predictions in all tasks. For the maximum historic depth of four years, our method yielded an average balanced accuracy (BA) of 85.94% in KOA grading, and accuracies of 91.89% (+1), 88.11% (+2), 84.35% (+3), and 79.41% (+4) for the four consecutive follow-up visits. Under the same setting, we also achieved an average value of Area Under Curve (AUC) of 0.94 for the prediction of progression incidence, and follow-up AUC values of 0.81 (+1), 0.77 (+2), 0.73 (+3), and 0.68 (+4), respectively.<\/jats:p>","DOI":"10.3390\/make7020040","type":"journal-article","created":{"date-parts":[[2025,4,28]],"date-time":"2025-04-28T11:48:33Z","timestamp":1745840913000},"page":"40","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A Novel Approach Based on Hypergraph Convolutional Neural Networks for Cartilage Shape Description and Longitudinal Prediction of Knee Osteoarthritis Progression"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3814-8652","authenticated-orcid":false,"given":"John B.","family":"Theocharis","sequence":"first","affiliation":[{"name":"Department of Electrical & Computer Engineering, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7724-3308","authenticated-orcid":false,"given":"Christos G.","family":"Chadoulos","sequence":"additional","affiliation":[{"name":"Department of Electrical & Computer Engineering, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0235-6046","authenticated-orcid":false,"given":"Andreas L.","family":"Symeonidis","sequence":"additional","affiliation":[{"name":"Department of Electrical & Computer Engineering, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Wang, Z., Xiao, Z., Sun, C., Xu, G., and He, J. (2024). Global, regional and national burden of osteoarthritis in 1990\u20132021: A systematic analysis of the global burden of disease study 2021. BMC Musculoskelet. Disord., 25.","DOI":"10.1186\/s12891-024-08122-5"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1186\/s13075-015-0670-x","article-title":"Prognostic factors for progression of clinical osteoarthritis of the knee: A systematic review of observational studies","volume":"17","author":"Bastick","year":"2015","journal-title":"Arthritis Res. Ther."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"494","DOI":"10.1136\/ard.16.4.494","article-title":"Radiological Assessment of Osteo-Arthrosis","volume":"16","author":"Kellgren","year":"1957","journal-title":"Ann. Rheum. Dis."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1007\/s00330-024-10928-9","article-title":"The value of deep learning-based X-ray techniques in detecting and classifying K-L grades of knee osteoarthritis: A systematic review and meta-analysis","volume":"35","author":"Zhao","year":"2024","journal-title":"Eur. Radiol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4015","DOI":"10.1007\/s00167-021-06768-5","article-title":"Defining and predicting radiographic knee osteoarthritis progression: A systematic review of findings from the osteoarthritis initiative","volume":"30","author":"Joo","year":"2022","journal-title":"Knee Surgery Sport. Traumatol. Arthrosc."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1186\/s42836-022-00118-7","article-title":"Artifcial intelligence in diagnosis of knee osteoarthritis and prediction of arthroplasty outcomes: A review","volume":"4","author":"Lee","year":"2022","journal-title":"Arhtroplasty"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Chadoulos, C., Tsaopoulos, D., Symeonidis, A., Moustakidis, S., and Theocharis, J. (2024). Dense Multi-Scale Graph Convolutional Network for Knee Joint Cartilage Segmentation. Bioengineering, 11.","DOI":"10.3390\/bioengineering11030278"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1433","DOI":"10.1016\/j.joca.2008.06.016","article-title":"The osteoarthritis initiative: Report on the design rationale for the magnetic resonance imaging protocol for the knee","volume":"16","author":"Peterfy","year":"2008","journal-title":"Osteoarthr. Cartil."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"5327","DOI":"10.1109\/TIP.2021.3082765","article-title":"Multi-Scale Representation Learning on Hypergraph for 3D Shape Retrieval and Recognition","volume":"30","author":"Bai","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Su, H., Maji, S., Kalogerakis, E., and Learned-Miller, E. (2015). Multi-view Convolutional Neural Networks for 3D Shape Recognition. arXiv.","DOI":"10.1109\/ICCV.2015.114"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Feng, Y., Zhang, Z., Zhao, X., Ji, R., and Gao, Y. (2018, January 18\u201322). GVCNN: Group-View Convolutional Neural Networks for 3D Shape Recognition. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00035"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"He, X., Huang, T., Bai, S., and Bai, X. (November, January 27). View N-Gram Network for 3D Object Retrieval. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.","DOI":"10.1109\/ICCV.2019.00761"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yu, T., Meng, J., and Yuan, J. (2018, January 18\u201322). Multi-view Harmonized Bilinear Network for 3D Object Recognition. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00027"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Charles, R.Q., Su, H., Kaichun, M., and Guibas, L.J. (2017, January 21\u201326). PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.16"},{"key":"ref_15","unstructured":"Qi, C.R., Yi, L., Su, H., and Guibas, L.J. (2017, January 4\u20139). PointNet++: Deep hierarchical feature learning on point sets in a metric space. Proceedings of the 31st International Conference on Neural Information Processing Systems, Red Hook, NY, USA. NIPS\u201917."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhao, H., Jiang, L., Fu, C.W., and Jia, J. (2019, January 15\u201320). PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00571"},{"key":"ref_17","unstructured":"Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., and Xiao, J. (2015, January 7\u201312). 3D ShapeNets: A deep representation for volumetric shapes. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Maturana, D., and Scherer, S. (October, January 28). VoxNet: A 3D Convolutional Neural Network for real-time object recognition. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353481"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Qi, C.R., Su, H., NieBner, M., Dai, A., Yan, M., and Guibas, L.J. (2016, January 27\u201330). Volumetric and Multi-view CNNs for Object Classification on 3D Data. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.609"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"8279","DOI":"10.1609\/aaai.v33i01.33018279","article-title":"MeshNet: Mesh Neural Network for 3D Shape Representation","volume":"33","author":"Feng","year":"2019","journal-title":"AAAI"},{"key":"ref_21","first-page":"1","article-title":"MeshCNN: A Network with an Edge","volume":"38","author":"Hanocka","year":"2019","journal-title":"ACM Trans. Graph."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"7525","DOI":"10.1109\/TPAMI.2022.3221785","article-title":"Learning View-Based Graph Convolutional Network for Multi-View 3D Shape Analysis","volume":"45","author":"Wei","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"104265","DOI":"10.1016\/j.imavis.2021.104265","article-title":"Dense graph convolutional neural networks on 3D meshes for 3D object segmentation and classification","volume":"114","author":"Tang","year":"2021","journal-title":"Image Vis. Comput."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Tiulpin, A., Thevenot, J., Rahtu, E., Lehenkari, P., and Saarakkala, S. (2018). Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach. Sci. Rep., 8.","DOI":"10.1038\/s41598-018-20132-7"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, B., Tan, J., Cho, K., Chang, G., and Deniz, C.M. (2020, January 3\u20137). Attention-based CNN for KL Grade Classification: Data from the Osteoarthritis Initiative. Proceedings of the 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), Iowa City, IA, USA.","DOI":"10.1109\/ISBI45749.2020.9098456"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J., and Kweon, I.S. (2018). CBAM: Convolutional Block Attention Module. arXiv.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"41497","DOI":"10.1007\/s11042-021-10557-0","article-title":"Knee osteoarthritis severity classification with ordinal regression module","volume":"81","author":"Yong","year":"2022","journal-title":"Multimed. Tools Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.compmedimag.2019.06.002","article-title":"Fully automatic knee osteoarthritis severity grading using deep neural networks with a novel ordinal loss","volume":"75","author":"Chen","year":"2019","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"100266","DOI":"10.1016\/j.prime.2023.100266","article-title":"Evaluating the efficacy of deep learning models for knee osteoarthritis prediction based on Kellgren-Lawrence grading system","volume":"5","author":"V","year":"2023","journal-title":"E-Prime Adv. Electr. Eng. Electron. Energy"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"584","DOI":"10.1148\/radiol.2020192091","article-title":"Prediction of Total Knee Replacement and Diagnosis of Osteoarthritis by Using Deep Learning on Knee Radiographs: Data from the Osteoarthritis Initiative","volume":"296","author":"Leung","year":"2020","journal-title":"Radiology"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"100112","DOI":"10.1016\/j.ostima.2023.100112","article-title":"Early detection of knee osteoarthritis using deep learning on knee MRI","volume":"3","author":"Alexopoulos","year":"2023","journal-title":"Osteoarthr. Imaging"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1016\/j.joca.2020.01.010","article-title":"Deep learning risk assessment models for predicting progression of radiographic medial joint space loss over a 48-MONTH follow-up period","volume":"28","author":"Guan","year":"2020","journal-title":"Osteoarthr. Cartil."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1186\/s13075-022-02743-8","article-title":"Prediction of knee osteoarthritis progression using radiological descriptors obtained from bone texture analysis and Siamese neural networks: Data from OAI and MOST cohorts","volume":"24","author":"Nguyen","year":"2022","journal-title":"Arthritis Res. Ther."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1109\/TNB.2018.2840082","article-title":"A Novel Method to Predict Knee Osteoarthritis Progression on MRI Using Machine Learning Methods","volume":"17","author":"Du","year":"2018","journal-title":"IEEE Trans. Nanobiosci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1643","DOI":"10.1016\/j.joca.2018.08.003","article-title":"Modeling and predicting osteoarthritis progression: Data from the osteoarthritis initiative","volume":"26","author":"Halilaj","year":"2018","journal-title":"Osteoarthr. Cartil."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3207","DOI":"10.1109\/TMI.2022.3181060","article-title":"Adversarial Evolving Neural Network for Longitudinal Knee Osteoarthritis Prediction","volume":"41","author":"Hu","year":"2022","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_37","unstructured":"Panfilov, E., Saarakkala, S., Nieminen, M.T., and Tiulpin, A. (2023). End-To-End Prediction of Knee Osteoarthritis Progression with Multi-Modal Transformers. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Yu, B., Yin, H., and Zhu, Z. (2018, January 13\u201319). Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/505"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"922","DOI":"10.1609\/aaai.v33i01.3301922","article-title":"Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting","volume":"33","author":"Guo","year":"2019","journal-title":"AAAI"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"3848","DOI":"10.1109\/TITS.2019.2935152","article-title":"T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Prediction","volume":"21","author":"Zhao","year":"2020","journal-title":"IEEE Trans. Intell. Transport. Syst."},{"key":"ref_41","unstructured":"Defferrard, M., Bresson, X., and Vandergheynst, P. (2016). Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. arXiv."},{"key":"ref_42","unstructured":"Kipf, T.N., and Welling, M. (2017). Semi-Supervised Classification with Graph Convolutional Networks. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"5517314","DOI":"10.1109\/TGRS.2022.3225438","article-title":"Hyperspectral Image Classification Using Feature Fusion Hypergraph Convolution Neural Network","volume":"60","author":"Ma","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"107637","DOI":"10.1016\/j.patcog.2020.107637","article-title":"Hypergraph convolution and hypergraph attention","volume":"110","author":"Bai","year":"2021","journal-title":"Pattern Recognit."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.media.2018.11.009","article-title":"Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: Data from the Osteoarthritis Initiative","volume":"52","author":"Ambellan","year":"2019","journal-title":"Med. Image Anal."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"12562","DOI":"10.1109\/ACCESS.2024.3354709","article-title":"A Comprehensive Survey on Delaunay Triangulation: Applications, Algorithms, and Implementations Over CPUs, GPUs, and FPGAs","volume":"12","author":"Elshakhs","year":"2024","journal-title":"IEEE Access"},{"key":"ref_47","unstructured":"Lee, J., Lee, I., and Kang, J. (2019). Self-Attention Graph Pooling. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"914","DOI":"10.1609\/aaai.v34i01.5438","article-title":"Spatial-Temporal Synchronous Graph Convolutional Networks: A New Framework for Spatial-Temporal Network Data Forecasting","volume":"34","author":"Song","year":"2020","journal-title":"AAAI"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1080\/01621459.1937.10503522","article-title":"The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance","volume":"32","author":"Friedman","year":"1937","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_50","unstructured":"Nemenyi, P. (1963). Distribution-Free Multiple Comparisons. [Ph.D. Thesis, Princeton University]."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Tiulpin, A., and Saarakkala, S. (2020). Automatic Grading of Individual Knee Osteoarthritis Features in Plain Radiographs Using Deep Convolutional Neural Networks. Diagnostics, 10.","DOI":"10.3390\/diagnostics10110932"},{"key":"ref_52","first-page":"197","article-title":"Assessing Knee OA Severity with CNN attention-based end-to-end architectures","volume":"102","author":"Gorriz","year":"2019","journal-title":"Proc. Mach. Learn. Res."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1002","DOI":"10.1016\/j.joca.2019.02.800","article-title":"Diagnosing osteoarthritis from T2 maps using deep learning: An analysis of the entire Osteoarthritis Initiative baseline cohort","volume":"27","author":"Pedoia","year":"2019","journal-title":"Osteoarthr. Cartil."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Alexopoulos, A., Hirvasniemi, J., and Tumer, N. (2022). Early detection of knee osteoarthritis using deep learning on knee magnetic resonance images. arXiv.","DOI":"10.1016\/j.ostima.2023.100112"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1186\/s13075-021-02634-4","article-title":"A deep learning method for predicting knee osteoarthritis radiographic progression from MRI","volume":"23","author":"Schiratti","year":"2021","journal-title":"Arthritis Res. Ther."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"2274","DOI":"10.1109\/TMI.2023.3247543","article-title":"SDMT: Spatial Dependence Multi-Task Transformer Network for 3D Knee MRI Segmentation and Landmark Localization","volume":"42","author":"Li","year":"2023","journal-title":"IEEE Trans. Med Imaging"}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/40\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:22:17Z","timestamp":1760030537000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/40"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,26]]},"references-count":56,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["make7020040"],"URL":"https:\/\/doi.org\/10.3390\/make7020040","relation":{},"ISSN":["2504-4990"],"issn-type":[{"type":"electronic","value":"2504-4990"}],"subject":[],"published":{"date-parts":[[2025,4,26]]}}}