{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:48:14Z","timestamp":1782841694198,"version":"3.54.5"},"reference-count":39,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Rheumatic diseases are chronic, immune-mediated conditions characterized by significant heterogeneity in presentation and disease course. However, current clinical approaches often rely on snapshot-based assessments that fail to capture the complex longitudinal evolution of these conditions. To address these limitations and support the implementation of precision medicine, we present the design for the Rheumatic Digital Twin, a novel, modular conceptual framework intended to integrate heterogeneous multimodal data, ranging from electronic health records and clinical notes to imaging and omics, into a dynamic, computational representation of the patient journey. Our theoretical architecture addresses challenges related to data silos and variable availability of data modalities through a multistage approach that envisions the use of domain-specific foundation models to independently process distinct data modalities. To effectively model the temporal progression inherent in chronic diseases, the proposed design utilizes Transformer architectures, leveraging self-attention mechanisms to treat patient events, such as lab results or medication changes, as sequential data tokens. We describe how these unimodal representations would subsequently be fused via joint embedding techniques to construct a shared, multimodal representational space. Envisioned to function analogously to a recommender system, the Rheumatic Digital Twin framework is modeled to map patients into a latent space where proximity reflects clinical and biological similarity. By identifying \u201cnearest neighbors,\u201d historical patients with comparable trajectories, the system aims to enable in silico cohorting, theoretically allowing clinicians to forecast key clinical events, predict treatment responses, and identify likely disease courses based on the outcomes of similar peers.<\/jats:p>","DOI":"10.2196\/86763","type":"journal-article","created":{"date-parts":[[2026,2,14]],"date-time":"2026-02-14T13:55:08Z","timestamp":1771077308000},"page":"e86763-e86763","source":"Crossref","is-referenced-by-count":0,"title":["Rheumatic Digital Twin: Proposed Machine Learning\u2013Based Multimodal Framework to Inform Clinical Decision-Making"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-5954-3685","authenticated-orcid":false,"given":"Daniyal","family":"Selani","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7494-3023","authenticated-orcid":false,"given":"Rachel","family":"Knevel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1148-1562","authenticated-orcid":false,"given":"Marcel","family":"Reinders","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7693-0728","authenticated-orcid":false,"given":"Erik B","family":"van den Akker","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,5,4]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01073-0","article-title":"Digital twins for health: a scoping review","volume":"7","author":"Katsoulakis","journal-title":"NPJ Digit Med"},{"issue":"2","key":"R2","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbab569","article-title":"Multimodal deep learning for biomedical data fusion: a review","volume":"23","author":"Stahlschmidt","journal-title":"Brief Bioinformatics"},{"issue":"9","key":"R3","doi-asserted-by":"publisher","first-page":"2455","DOI":"10.1016\/j.celrep.2019.07.091","article-title":"Molecular portraits of early rheumatoid arthritis identify clinical and treatment response phenotypes","volume":"28","author":"Lewis","journal-title":"Cell Rep"},{"issue":"545","key":"R4","doi-asserted-by":"publisher","DOI":"10.1126\/scitranslmed.aay1548","article-title":"Using genetics to prioritize diagnoses for rheumatology outpatients with inflammatory arthritis","volume":"12","author":"Knevel","journal-title":"Sci Transl Med"},{"key":"R5","doi-asserted-by":"publisher","DOI":"10.1109\/RBME.2024.3496744","article-title":"Foundation model for advancing healthcare: challenges, opportunities and future directions","volume":"18","author":"He","journal-title":"IEEE Rev Biomed Eng"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2025.1624877","article-title":"From data-driven cities to data-driven tumors: dynamic digital twins for adaptive oncology","volume":"8","author":"Karaman","journal-title":"Front Artif Intell"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.12701\/jyms.2024.01053","article-title":"Advances and utility of digital twins in critical care and acute care medicine: a narrative review","volume":"42","author":"Halpern","journal-title":"J Yeungnam Med Sci"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.1136\/rmdopen-2024-004309","article-title":"Artificial intelligence in rheumatology research: what is it good for?","volume":"11","author":"Sequ\u00ed-Sabater","journal-title":"RMD Open"},{"key":"R9","doi-asserted-by":"publisher","DOI":"10.2196\/69544","article-title":"Advancing health care with digital twins: meta-review of applications and implementation challenges","volume":"27","author":"Ringeval","journal-title":"J Med Internet Res"},{"key":"R10","doi-asserted-by":"crossref","unstructured":"National Academies of Sciences, Engineering, and Medicine; National Academy of Engineering; Division on Earth and Life Studies; Division on Engineering and Physical Sciences; Board on Atmospheric Sciences and Climate; Board on Life Sciences; Computer Science and Telecommunications Board; Committee on Applied and Theoretical Statistics; Board on Mathematical Sciences and Analytics; Committee on Foundational Research Gaps and Future Directions for Digital Twins. Foundational Research Gaps and Future Directions for Digital Twins. National Academies Press; 2024. [doi: 10.17226\/26894]","DOI":"10.17226\/26894"},{"issue":"1","key":"R11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3285029","article-title":"Deep learning based recommender system: a survey and new perspectives","volume":"52","author":"Zhang","journal-title":"ACM Comput Surv"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-021-01553-3","article-title":"Medical recommender systems based on continuous-valued logic and multi-criteria decision operators, using interpretable neural networks","volume":"21","author":"Ochoa","journal-title":"BMC Med Inform Decis Mak"},{"issue":"1","key":"R13","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01997-1","article-title":"Location and amount of joint involvement differentiates rheumatoid arthritis into different clinical subsets","volume":"8","author":"Maarseveen","journal-title":"NPJ Digit Med"},{"issue":"8","key":"R14","doi-asserted-by":"publisher","first-page":"1301","DOI":"10.1016\/j.ard.2025.04.011","article-title":"Time-independent disease state identification defines distinct trajectories determined by localised vs systemic inflammation in patients with early rheumatoid arthritis","volume":"84","author":"Steinz","journal-title":"Ann Rheum Dis"},{"key":"R15","unstructured":"Bommasani R Hudson DA Adeli E et al. On the opportunities and risks of foundation models. arXiv. Preprint posted online on Aug 16, 2021. [doi: 10.48550\/arXiv.2108.07258]"},{"issue":"4","key":"R16","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1007\/s11633-022-1406-4","article-title":"A review of predictive and contrastive self-supervised learning for medical images","volume":"20","author":"Wang","journal-title":"Mach Intell Res"},{"issue":"Pt C","key":"R17","doi-asserted-by":"publisher","first-page":"110843","DOI":"10.1016\/j.compbiomed.2025.110843","article-title":"MARIA: a multimodal transformer model for incomplete healthcare data","volume":"196","author":"Caruso","journal-title":"Comput Biol Med"},{"issue":"6","key":"R18","doi-asserted-by":"publisher","first-page":"743","DOI":"10.1038\/s41551-023-01045-x","article-title":"A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics","volume":"7","author":"Zhou","journal-title":"Nat Biomed Eng"},{"key":"R19","unstructured":"Bai S Kolter JZ Koltun V . An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv. Preprint posted online on Mar 4, 2018. [doi: 10.48550\/arXiv.1803.01271]"},{"issue":"1","key":"R20","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-24271-9","article-title":"Recurrent neural networks for multivariate time series with missing values","volume":"8","author":"Che","journal-title":"Sci Rep"},{"key":"R21","unstructured":"Vaswani A Shazeer N Parmar N et al. Attention is all you need. Presented at: Advances in Neural Information Processing Systems 30 (NIPS 2017); Dec 4-9, 2017; Red Hook, NY, USA. URL: https:\/\/papers.nips.cc\/paper_files\/paper\/2017\/hash\/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html [Accessed 04-04-2026]"},{"issue":"2","key":"R22","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1109\/72.279181","article-title":"Learning long-term dependencies with gradient descent is difficult","volume":"5","author":"Bengio","journal-title":"IEEE Trans Neural Netw"},{"key":"R23","unstructured":"Pascanu R Mikolov T Bengio Y . On the difficulty of training recurrent neural networks. Presented at: ICML\u201913: Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28; Jun 16-21, 2013:1310-1318; Atlanta GA, USA. URL: https:\/\/dl.acm.org\/doi\/10.5555\/3042817.3043083 [Accessed 04-04-2026]"},{"key":"R24","unstructured":"Beltagy I Peters ME Cohan A . Longformer: the long-document transformer. arXiv. Preprint posted online on Apr 10, 2020. URL: https:\/\/arxiv.org\/abs\/2004.05150 [Accessed 04-04-2026]"},{"key":"R25","unstructured":"Zaheer M Guruganesh G Dubey A et al. Big bird: transformers for longer sequences. Presented at: NIPS\u201920: Proceedings of the 34th International Conference on Neural Information Processing Systems; Dec 6-12, 2020:17283-17297; Vancouver BC, Canada. URL: https:\/\/dl.acm.org\/doi\/abs\/10.5555\/3495724.3497174 [Accessed 04-04-2026]"},{"key":"R26","unstructured":"Chen T Kornblith S Norouzi M Hinton G . A simple framework for contrastive learning of visual representations. Presented at: ICML\u201920: Proceedings of the 37th International Conference on Machine Learning; Jul 13-18, 2020:1597-1607; Virtual (online). URL: https:\/\/dl.acm.org\/doi\/10.5555\/3524938.3525087 [Accessed 04-04-2026]"},{"key":"R27","unstructured":"Radford A Kim JW Hallacy C et al. Learning transferable visual models from natural language supervision. Presented at: Proceedings of the 38th International Conference on Machine Learning; Jul 18-24, 2021:8748-8763; Virtual Event. URL: https:\/\/proceedings.mlr.press\/v139\/radford21a.html [Accessed 04-04-2026]"},{"key":"R28","unstructured":"Zhang Y Jiang H Miura Y Manning CD Langlotz CP . Contrastive learning of medical visual representations from paired images and text. Presented at: Machine Learning for Healthcare Conference; Aug 5-6, 2022:2-25; Durham, NC, USA. URL: https:\/\/proceedings.mlr.press\/v182\/zhang22a\/zhang22a.pdf [Accessed 04-04-2026]"},{"key":"R29","doi-asserted-by":"crossref","unstructured":"He K Chen X Xie S Li Y Dollar P Girshick R . Masked autoencoders are scalable vision learners. Presented at: 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR); Jun 18-24, 2022:16000-16009; New Orleans, LA, USA. [doi: 10.1109\/CVPR52688.2022.01553]","DOI":"10.1109\/CVPR52688.2022.01553"},{"issue":"7956","key":"R30","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1038\/s41586-023-05881-4","article-title":"Foundation models for generalist medical artificial intelligence","volume":"616","author":"Moor","journal-title":"Nature"},{"issue":"2","key":"R31","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1109\/TPAMI.2018.2798607","article-title":"Multimodal machine learning: a survey and taxonomy","volume":"41","author":"Baltrusaitis","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"1","key":"R32","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1023\/A:1007379606734","article-title":"Multitask learning","volume":"28","author":"Caruana","journal-title":"Mach Learn"},{"issue":"11","key":"R33","doi-asserted-by":"publisher","first-page":"1716","DOI":"10.1109\/JPROC.2024.3525147","article-title":"Cross-Modal Retrieval: A Systematic Review of Methods and Future Directions","volume":"112","author":"Wang","journal-title":"Proc IEEE"},{"issue":"1","key":"R34","doi-asserted-by":"publisher","DOI":"10.2196\/medinform.6730","article-title":"Patient similarity in prediction models based on health data: a scoping review","volume":"5","author":"Sharafoddini","journal-title":"JMIR Med Inform"},{"issue":"1150","key":"R35","doi-asserted-by":"publisher","DOI":"10.1259\/bjr.20220878","article-title":"Data drift in medical machine learning: implications and potential remedies","volume":"96","author":"Sahiner","journal-title":"Br J Radiol"},{"issue":"4","key":"R36","doi-asserted-by":"publisher","first-page":"1328","DOI":"10.1109\/TKDE.2019.2946162","article-title":"A survey on data collection for machine learning: a big data - AI integration perspective","volume":"33","author":"Roh","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"9","key":"R37","doi-asserted-by":"publisher","first-page":"1322","DOI":"10.1016\/j.cjca.2020.11.010","article-title":"Missing data in clinical research: a tutorial on multiple imputation","volume":"37","author":"Austin","journal-title":"Can J Cardiol"},{"key":"R38","author":"Oladele","journal-title":"Health Inform Rev"},{"issue":"10-12","key":"R39","first-page":"24","volume":"17","author":"Howe","journal-title":"Innov Clin Neurosci"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2026\/1\/e86763","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:14:29Z","timestamp":1782839669000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e86763"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,4]]},"references-count":39,"URL":"https:\/\/doi.org\/10.2196\/86763","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,4]]},"article-number":"v28i1e86763"}}