{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,18]],"date-time":"2026-10-18T03:57:37Z","timestamp":1792295857855,"version":"4.4.0"},"reference-count":29,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,4,25]],"date-time":"2022-04-25T00:00:00Z","timestamp":1650844800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Digital diagnostic decision support tools promise to accelerate diagnosis and increase health care efficiency in rheumatology. <jats:italic>Rheumatic?<\/jats:italic> is an online tool developed by specialists in rheumatology and general medicine together with patients and patient organizations. It calculates a risk score for several rheumatic diseases. We ran a pilot study retrospectively testing <jats:italic>Rheumatic?<\/jats:italic> for its ability to differentiate symptoms from existing or emerging immune-mediated rheumatic diseases from other rheumatic and musculoskeletal complaints and disorders in patients visiting rheumatology clinics.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>The performance of <jats:italic>Rheumatic?<\/jats:italic> was tested using in three university rheumatology centers: (A) patients at Risk for RA (Karolinska Institutet, <jats:italic>n<\/jats:italic> = 50 individuals with musculoskeletal complaints and anti-citrullinated protein antibody positivity) (B) patients with early joint swelling [dataset B (Erlangen) <jats:italic>n<\/jats:italic> = 52]. (C) Patients with early arthritis where the clinician considered it likely to be of auto-immune origin [dataset C (Leiden) <jats:italic>n<\/jats:italic> = 73]. In dataset A we tested whether <jats:italic>Rheumatic?<\/jats:italic> could predict the development of arthritis. In dataset B and C we tested whether <jats:italic>Rheumatic?<\/jats:italic> could predict the development of an immune-mediated rheumatic diseases. We examined the discriminative power of the total score with the Wilcoxon rank test and the area-under-the-receiver-operating-characteristic curve (AUC-ROC). Next, we calculated the test characteristics for these patients passing the first or second expert-based <jats:italic>Rheumatic?<\/jats:italic> scoring threshold.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The total test scores differentiated between: (A) Individuals developing arthritis or not, median 245 vs. 163, <jats:italic>P<\/jats:italic> &amp;lt; 0.0001, AUC-ROC = 75.3; (B) patients with an immune-mediated arthritic disease or not median 191 vs. 107, <jats:italic>P<\/jats:italic> &amp;lt; 0.0001, AUC-ROC = 79.0; but less patients with an immune-mediated arthritic disease or not amongst those where the clinician already considered an immune mediated disease most likely (median 262 vs. 212, <jats:italic>P<\/jats:italic> &amp;lt; 0.0001, AUC-ROC = 53.6). Threshold-1 (advising to visit primary care doctor) was highly specific in dataset A and B (0.72, 0.87, and 0.23, respectively) and sensitive (0.67, 0.61, and 0.67). Threshold-2 (advising to visit rheumatologic care) was very specific in all three centers but not very sensitive: specificity of 1.0, 0.96, and 0.91, sensitivity 0.05, 0.07, 0.14 in dataset A, B, and C, respectively.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p><jats:italic>Rheumatic?<\/jats:italic> is a web-based patient-centered multilingual diagnostic tool capable of differentiating immune-mediated rheumatic conditions from other musculoskeletal problems. The current scoring system needs to be further optimized.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fmed.2022.774945","type":"journal-article","created":{"date-parts":[[2022,4,25]],"date-time":"2022-04-25T13:00:42Z","timestamp":1650891642000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Rheumatic?\u2014A Digital Diagnostic Decision Support Tool for Individuals Suspecting Rheumatic Diseases: A Multicenter Pilot Validation Study"],"prefix":"10.3389","volume":"9","author":[{"given":"Rachel","family":"Knevel","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Johannes","family":"Knitza","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aase","family":"Hensvold","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexandra","family":"Circiumaru","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tor","family":"Bruce","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"Evans","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tjardo","family":"Maarseveen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marc","family":"Maurits","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liesbeth","family":"Beaart-van de Voorde","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Simon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Arnd","family":"Kleyer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martina","family":"Johannesson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Georg","family":"Schett","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tom","family":"Huizinga","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sofia","family":"Svanteson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexandra","family":"Lindfors","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lars","family":"Klareskog","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anca","family":"Catrina","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2022,4,25]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"e28164","DOI":"10.2196\/28164","article-title":"A real-world rheumatology registry and research consortium: the German RheumaDatenRhePort (RHADAR) registry.","volume":"23","author":"Kleinert","year":"2021","journal-title":"J Med Internet Res."},{"key":"B2","doi-asserted-by":"publisher","first-page":"e19661","DOI":"10.2196\/19661","article-title":"Mobile health usage, preferences, barriers, and eHealth literacy in rheumatology: patient survey study.","volume":"8","author":"Knitza","year":"2020","journal-title":"JMIR Mhealth Uhealth."},{"key":"B3","doi-asserted-by":"publisher","first-page":"839","DOI":"10.1007\/s00393-018-0578-3","article-title":"Arbeitskreis junge rheumatologen. nutzung von medizin-apps und online-plattformen unter deutschen rheumatologen.","volume":"78","author":"Knitza","year":"2019","journal-title":"Z Rheumatol."},{"key":"B4","doi-asserted-by":"publisher","first-page":"e001548","DOI":"10.1136\/rmdopen-2020-001548","article-title":"Digital rheumatology in the era of COVID-19: results of a national patient and physician survey.","volume":"7","author":"Kernder","year":"2021","journal-title":"RMD Open."},{"key":"B5","doi-asserted-by":"publisher","first-page":"722","DOI":"10.1007\/s00393-019-0683-y","article-title":"[Early recognition and screening consultation: a necessary way to improve early detection and treatment in rheumatology?: overview of the early recognition and screening consultation models for rheumatic and musculoskeletal diseases in Germany].","volume":"78","author":"Benesova","year":"2019","journal-title":"Z Rheumatol."},{"key":"B6","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1136\/annrheumdis-2011-201063","article-title":"A systematic literature review of strategies promoting early referral and reducing delays in the diagnosis and management of inflammatory arthritis.","volume":"72","author":"Villeneuve","year":"2013","journal-title":"Ann Rheum Dis."},{"key":"B7","doi-asserted-by":"publisher","first-page":"e024361","DOI":"10.1136\/bmjopen-2018-024361","article-title":"Delays between the onset of symptoms and first rheumatology consultation in patients with rheumatoid arthritis in the UK: an observational study.","volume":"9","author":"Stack","year":"2019","journal-title":"BMJ Open."},{"key":"B8","doi-asserted-by":"publisher","first-page":"1822","DOI":"10.1136\/ard.2011.151902","article-title":"Delays in assessment of patients with rheumatoid arthritis: variations across Europe.","volume":"70","author":"Raza","year":"2011","journal-title":"Ann Rheumat Dis."},{"key":"B9","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1186\/s13075-021-02498-8","article-title":"Accuracy, patient-perceived usability, and acceptance of two symptom checkers (ada and rheport) in rheumatology: interim results from a randomized controlled crossover trial.","volume":"23","author":"Knitza","year":"2021","journal-title":"Arthritis Res Ther."},{"key":"B10","doi-asserted-by":"publisher","first-page":"181","DOI":"10.4137\/CMAMD.S40361","article-title":"Rate of proven rheumatic diseases in a large collective of referrals to an outpatient rheumatology clinic under routine conditions.","volume":"9","author":"Feuchtenberger","year":"2016","journal-title":"Clin Med Insights Arthritis Musculoskelet Disord."},{"key":"B11","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1007\/s00063-015-0069-0","article-title":"[Triage in emergency departments. Comparative evaluation of 4 international triage systems].","volume":"111","author":"Krey","year":"2016","journal-title":"Med Klin Intensivmed Notfmed."},{"key":"B12","doi-asserted-by":"publisher","first-page":"672714","DOI":"10.1155\/2014\/672714","article-title":"Computer-based diagnostic expert systems in rheumatology: where do we stand in 2014?","volume":"2014","author":"Alder","year":"2014","journal-title":"Int J Rheumatol."},{"key":"B13","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1016\/0049-0172(91)90004-j","article-title":"Computer-assisted diagnosis of rheumatic disorders.","volume":"21","author":"Moens","year":"1991","journal-title":"Semin Arthritis Rheum."},{"key":"B14","doi-asserted-by":"publisher","first-page":"1015","DOI":"10.1016\/j.semarthrit.2020.07.018","article-title":"Comparison of an online self-referral tool with a physician-based referral strategy for early recognition of patients with a high probability of axial spa.","volume":"50","author":"Proft","year":"2020","journal-title":"Semin Arthritis Rheum."},{"key":"B15","doi-asserted-by":"publisher","first-page":"e17507","DOI":"10.2196\/17507","article-title":"Toward earlier diagnosis using combined eHealth tools in rheumatology: the joint pain assessment scoring tool (JPAST) project.","volume":"8","author":"Knitza","year":"2020","journal-title":"JMIR Mhealth Uhealth."},{"key":"B16","doi-asserted-by":"publisher","first-page":"rkaa005","DOI":"10.1093\/rap\/rkaa005","article-title":"Applied machine learning and artificial intelligence in rheumatology.","volume":"4","author":"H\u00fcgle","year":"2020","journal-title":"Rheumatol Adv Pract."},{"key":"B17","doi-asserted-by":"publisher","first-page":"e040269","DOI":"10.1136\/bmjopen-2020-040269","article-title":"How accurate are digital symptom assessment apps for suggesting conditions and urgency advice? A clinical vignettes comparison to GPs.","volume":"10","author":"Gilbert","year":"2020","journal-title":"BMJ Open."},{"key":"B18","doi-asserted-by":"publisher","first-page":"514","DOI":"10.5694\/mja2.50600","article-title":"The quality of diagnosis and triage advice provided by free online symptom checkers and apps in Australia.","volume":"212","author":"Hill","year":"2020","journal-title":"Med J Aust."},{"key":"B19","doi-asserted-by":"publisher","first-page":"h3480","DOI":"10.1136\/bmj.h3480","article-title":"Evaluation of symptom checkers for self diagnosis and triage: audit study.","volume":"351","author":"Semigran","year":"2015","journal-title":"BMJ."},{"key":"B20","doi-asserted-by":"publisher","first-page":"e001014","DOI":"10.1136\/rmdopen-2019-001014","article-title":"EULAR points to consider for the development, evaluation and implementation of mobile health applications aiding self-management in people living with rheumatic and musculoskeletal diseases.","volume":"5","author":"Najm","year":"2019","journal-title":"RMD Open."},{"key":"B21","doi-asserted-by":"publisher","first-page":"e14991","DOI":"10.2196\/14991","article-title":"German mobile apps in rheumatology: review and analysis using the mobile application rating scale (MARS).","volume":"7","author":"Knitza","year":"2019","journal-title":"JMIR Mhealth Uhealth."},{"key":"B22","doi-asserted-by":"publisher","first-page":"1251","DOI":"10.1136\/annrheumdis-2020-eular.5578","article-title":"Sat0584 specific Acpa reactivities and inflammatory biomarkers along with ultrasound tenosynovitis are associated with arthritis onset in a population at risk for rheumatoid arthritis.","volume":"79","author":"Circiumaru","year":"2020","journal-title":"Ann Rheumat Dis."},{"key":"B23","year":"2010","journal-title":"2010 Rheumatoid Arthritis Classification Criteria: An American College of Rheumatology\/European League Against Rheumatism Collaborative Initiative Aletaha. Arthritis &amp;Rheumatism."},{"key":"B24","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1093\/rheumatology\/keq230","article-title":"Predicting arthritis outcomes\u2013what can be learned from the Leiden early arthritis clinic?","volume":"50","author":"de Rooy","year":"2011","journal-title":"Rheumatology (Oxford)."},{"key":"B25","doi-asserted-by":"publisher","first-page":"2636","DOI":"10.3390\/ijerph18052636","article-title":"Digital health transition in rheumatology: a qualitative study.","volume":"18","author":"M\u00fchlensiepen","year":"2021","journal-title":"Int J Environ Res Public Health."},{"key":"B26","doi-asserted-by":"publisher","first-page":"100279","DOI":"10.1016\/j.imu.2019.100279","article-title":"Decision support system for diagnosing rheumatic-musculoskeletal disease using fuzzy cognitive map technique.","volume":"18","author":"Akinnuwesi","year":"2020","journal-title":"Inform Med Unlocked."},{"key":"B27","doi-asserted-by":"publisher","first-page":"362","DOI":"10.1186\/s12891-016-1189-2","article-title":"Are online symptoms checkers useful for patients with inflammatory arthritis?","volume":"17","author":"Powley","year":"2016","journal-title":"BMC Musculoskelet Disord."},{"key":"B28","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1136\/ard.2003.011247","article-title":"How to diagnose axial spondyloarthritis early.","volume":"63","author":"Rudwaleit","year":"2004","journal-title":"Ann Rheumat Dis."},{"key":"B29","doi-asserted-by":"publisher","first-page":"e13863","DOI":"10.2196\/13863","article-title":"Accuracy of a chatbot (Ada) in the diagnosis of mental disorders: comparative case study with lay and expert users.","volume":"3","author":"Jungmann","year":"2019","journal-title":"JMIR Form Res."}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2022.774945\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,25]],"date-time":"2022-04-25T13:00:45Z","timestamp":1650891645000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2022.774945\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,25]]},"references-count":29,"alternative-id":["10.3389\/fmed.2022.774945"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2022.774945","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,25]]},"article-number":"774945"}}