{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T09:01:50Z","timestamp":1793091710313,"version":"4.5.0"},"reference-count":30,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Visits to medical subspecialists are common, with follow-up timing often based on heuristics rather than evidence. Unnecessary visits contribute to long wait times for new patients. Specialists could enhance visit timing and reduce frequency by systematically monitoring patients\u2019 symptoms between visits, especially for symptom-driven conditions like rheumatoid arthritis (RA). We previously designed an intervention using a mobile health (mHealth) app to collect patient-reported outcomes (PRO). One of several aims of the app was to assist rheumatologists in determining visit timing for patients with RA. The intervention did not reduce visit frequency.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n To explore possible reasons for the lack of association between the intervention and visit frequency, we describe app usage, assess usability, and identify barriers and facilitators for using between-visit PRO data to reduce visits when patients\u2019 symptoms are stable.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We analyzed patients\u2019 use of the app by reporting adherence (percent of PRO questionnaires completed during the 12-month study) and retention (use in the last month of the study). To examine rheumatologists\u2019 experiences, we summarized views of the electronic health record (EHR)\u2013embedded PRO dashboard and EHR inbox messages suggesting early or deferred visits. We assessed app usability using the interactive mHealth App Usability Questionnaire for Ease of Use and Usefulness for patients and the System Usability Scale for rheumatologists. We assessed rheumatologist-level effects of intervention usage using Kruskal-Wallis rank sum and equality of proportion tests. We identified barriers and facilitators through interviews and surveys.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n The analysis included 150 patients with RA and their 11 rheumatologists. Patients answered a median of 53.3% (IQR 34.1%-69.2%) of PRO questionnaires; this proportion varied by rheumatologist (range 40.7%\u201067%). Over half of the patients used the app during the final month of the study (56%, range 51%\u201065%, by rheumatologists); the median number of months of use was 12 (IQR 9-12). Rheumatologists viewed the dashboard 78 times (17.6% of 443 visits) with significant differences in viewing rates by rheumatologist (range 10%\u201066%;\n P<\/jats:italic>\n <.01). There were 108 generated messages sent to rheumatologists suggesting a deferred visit (24.4% of 443 visits) with significant differences in message counts received per visit by rheumatologist (range 10.8%\u201022.6%;\n P<\/jats:italic>\n =.03). Rheumatologists\u2019 reported barriers to offering visit deferrals included already scheduling as far out as they were comfortable and rescheduling complexities for staff. Based on 39 patient interviews and 44 surveys, patients reported 2 main barriers to app usage: questionnaire frequency not being tailored to them and reduced motivation after not discussing PRO data with their rheumatologist. A total of 5 interviewed patients received the option to defer their visits, of which 3 elected to defer the appointment and 2 chose to keep it.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n While an mHealth app for reporting RA PROs was used frequently by patients, using these data to reduce the frequency of unneeded visits was not straightforward. Better engagement of clinicians may improve the use of PRO data.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/60854","type":"journal-article","created":{"date-parts":[[2025,4,21]],"date-time":"2025-04-21T17:15:56Z","timestamp":1745255756000},"page":"e60854-e60854","source":"Crossref","is-referenced-by-count":3,"title":["Tailoring Rheumatoid Arthritis Visit Timing Based on mHealth App Data: Mixed Methods Assessment of Implementation and Usability"],"prefix":"10.2196","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9172-5506","authenticated-orcid":false,"given":"Robert S","family":"Rudin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8951-2296","authenticated-orcid":false,"given":"Leah M","family":"Santacroce","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3793-0995","authenticated-orcid":false,"given":"Ishani","family":"Ganguli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8202-5428","authenticated-orcid":false,"given":"Daniel H","family":"Solomon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,4,21]]},"reference":[{"key":"R1","doi-asserted-by":"crossref","unstructured":"Santo L Kang K . National hospital ambulatory medical care survey: 2019 national summary tables. Centers for Disease Control and Prevention; Jan20, 2023. [doi: 10.15620\/cdc:123251]","DOI":"10.15620\/cdc:117687"},{"issue":"1","key":"R2","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1377\/hlthaff.2020.02022","article-title":"National health care spending in 2019: steady growth for the fourth consecutive year","volume":"40","author":"Martin","journal-title":"Health Aff (Millwood)"},{"issue":"1","key":"R3","first-page":"17","volume":"20","author":"Javorsky","journal-title":"Am J Manag Care"},{"issue":"19","key":"R4","doi-asserted-by":"publisher","first-page":"1905","DOI":"10.1001\/jama.2015.3356","article-title":"What is the right number of clinic appointments? Visit frequency and the accountable care organization","volume":"313","author":"Ganguli","journal-title":"JAMA"},{"issue":"10","key":"R5","doi-asserted-by":"publisher","first-page":"2054","DOI":"10.1002\/acr.25106","article-title":"Clinical visit frequencies in rheumatology: a systematic literature review","volume":"75","author":"Jiang","journal-title":"Arthritis Care Res (Hoboken)"},{"issue":"2","key":"R6","doi-asserted-by":"publisher","first-page":"125","DOI":"10.7326\/M23-2331","article-title":"Health care contact days among older adults in traditional Medicare: a cross-sectional study","volume":"177","author":"Ganguli","journal-title":"Ann Intern Med"},{"issue":"23","key":"R7","doi-asserted-by":"publisher","first-page":"2258","DOI":"10.1056\/NEJMe1103390","article-title":"Ensuring equal access to specialty care","volume":"364","author":"Sequist","journal-title":"N Engl J Med"},{"issue":"11","key":"R8","doi-asserted-by":"publisher","first-page":"964","DOI":"10.1002\/acr2.11498","article-title":"Development and testing of an electronic health record-integrated patient-reported outcome application and intervention to improve efficiency of rheumatoid arthritis care","volume":"4","author":"Solomon","journal-title":"ACR Open Rheumatol"},{"issue":"5","key":"R9","doi-asserted-by":"publisher","first-page":"677","DOI":"10.1002\/art.42774","article-title":"A mobile health application integrated in the electronic health record for rheumatoid arthritis patient\u2013reported outcomes: a controlled interrupted time\u2010series analysis of impact on visit efficiency","volume":"76","author":"Solomon","journal-title":"Arthritis Rheumatol"},{"issue":"4","key":"R10","doi-asserted-by":"publisher","first-page":"617","DOI":"10.1002\/acr.23518","article-title":"2015 American College of Rheumatology Workforce Study: supply and demand projections of adult rheumatology workforce, 2015\u20132030","volume":"70","author":"Battafarano","journal-title":"Arthritis Care Res (Hoboken)"},{"issue":"7484","key":"R11","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1136\/bmj.38265.493773.8F","article-title":"Patient initiated outpatient follow up in rheumatoid arthritis: six year randomised controlled trial","volume":"330","author":"Hewlett","journal-title":"BMJ"},{"issue":"11","key":"R12","doi-asserted-by":"publisher","first-page":"1737","DOI":"10.1002\/art.42292","article-title":"Smartphone-assisted patient-initiated care versus usual care in patients with rheumatoid arthritis and low disease activity: a randomized controlled trial","volume":"74","author":"Seppen","journal-title":"Arthritis Rheumatol"},{"issue":"Suppl 10","key":"R13","volume":"68","author":"Butt","journal-title":"Arthritis Rheumatol"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1093\/rheumatology\/keaa202","article-title":"Patient adherence with a smartphone app for patient-reported outcomes in rheumatoid arthritis","volume":"60","author":"Colls","journal-title":"Rheumatology (Sunnyvale)"},{"issue":"8","key":"R15","doi-asserted-by":"publisher","first-page":"1421","DOI":"10.1002\/art.41686","article-title":"Outcomes of a mobile app to monitor patient-reported outcomes in rheumatoid arthritis: a randomized controlled trial","volume":"73","author":"Lee","journal-title":"Arthritis Rheumatol"},{"issue":"4","key":"R16","doi-asserted-by":"publisher","DOI":"10.2196\/11500","article-title":"The mHealth App Usability Questionnaire (MAUQ): development and validation study","volume":"7","author":"Zhou","journal-title":"JMIR Mhealth Uhealth"},{"issue":"6","key":"R17","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1080\/10447310802205776","article-title":"An empirical evaluation of the System Usability Scale","volume":"24","author":"Bangor","journal-title":"Int J Hum Comput Interact"},{"issue":"7","key":"R18","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1080\/10447318.2018.1455307","article-title":"The System Usability Scale: past, present, and future","volume":"34","author":"Lewis","journal-title":"Int J Hum Comput Interact"},{"issue":"9","key":"R19","doi-asserted-by":"publisher","first-page":"1277","DOI":"10.1177\/1049732305276687","article-title":"Three approaches to qualitative content analysis","volume":"15","author":"Hsieh","journal-title":"Qual Health Res"},{"issue":"1","key":"R20","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2288-13-117","article-title":"Using the framework method for the analysis of qualitative data in multi-disciplinary health research","volume":"13","author":"Gale","journal-title":"BMC Med Res Methodol"},{"issue":"8","key":"R21","doi-asserted-by":"publisher","DOI":"10.2196\/37290","article-title":"System Usability Scale benchmarking for digital health apps: meta-analysis","volume":"10","author":"Hyzy","journal-title":"JMIR Mhealth Uhealth"},{"issue":"5","key":"R22","doi-asserted-by":"publisher","first-page":"783","DOI":"10.1055\/s-0039-1697597","article-title":"A clinically integrated mHealth app and practice model for collecting patient-reported outcomes between visits for asthma patients: implementation and feasibility","volume":"10","author":"Rudin","journal-title":"Appl Clin Inform"},{"issue":"4","key":"R23","doi-asserted-by":"publisher","first-page":"1031","DOI":"10.4338\/ACI-2017-06-RA-0096","article-title":"Core components for a clinically integrated mHealth app for asthma symptom monitoring","volume":"8","author":"Rudin","journal-title":"Appl Clin Inform"},{"issue":"4","key":"R24","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1055\/s-0044-1788978","article-title":"Evaluation of a primary care-integrated mobile health intervention to monitor between-visit asthma symptoms","volume":"15","author":"Sulca Flores","journal-title":"Appl Clin Inform"},{"issue":"1","key":"R25","doi-asserted-by":"publisher","DOI":"10.2196\/15588","article-title":"Benefits and disadvantages of electronic patient-reported outcome measures: systematic review","volume":"3","author":"Meirte","journal-title":"JMIR Perioper Med"},{"issue":"7","key":"R26","doi-asserted-by":"publisher","first-page":"1662","DOI":"10.1093\/rheumatology\/kez506","article-title":"Patient-reported outcomes in RA care improve patient communication, decision-making, satisfaction and confidence: qualitative results","volume":"59","author":"Bartlett","journal-title":"Rheumatology (Oxford)"},{"issue":"4","key":"R27","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1016\/j.jcjq.2022.01.005","article-title":"Addressing the drivers of medical test overuse and cascades: user-centered design to improve patient-doctor communication","volume":"48","author":"Rudin","journal-title":"Jt Comm J Qual Patient Saf"},{"issue":"11","key":"R28","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.42464","article-title":"Effect of a peer comparison and educational intervention on medical test conversation quality: a randomized clinical trial","volume":"6","author":"Ganguli","journal-title":"JAMA Netw Open"},{"issue":"1","key":"R29","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-018-1050-6","article-title":"Analysing the role of complexity in explaining the fortunes of technology programmes: empirical application of the NASSS framework","volume":"16","author":"Greenhalgh","journal-title":"BMC Med"},{"issue":"5","key":"R30","doi-asserted-by":"publisher","first-page":"847","DOI":"10.1093\/poq\/nfn063","article-title":"Social desirability bias in CATI, IVR, and web surveys: the effects of mode and question sensitivity","volume":"72","author":"Kreuter","journal-title":"Public Opin Q"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/formative.jmir.org\/2025\/1\/e60854","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:31:38Z","timestamp":1785537098000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2025\/1\/e60854"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,21]]},"references-count":30,"URL":"https:\/\/doi.org\/10.2196\/60854","relation":{},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,21]]},"article-number":"v9i1e60854"}}