{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T06:15:22Z","timestamp":1778220922257,"version":"3.51.4"},"reference-count":44,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T00:00:00Z","timestamp":1778198400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Online symptom checkers are often developed and validated on data subject to self-selection and selective attrition, potentially introducing biases in prediction models.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>To assess recruitment, selection, and attrition patterns in a large Dutch online symptom checker for musculoskeletal complaints and to evaluate potential biases by comparing participant characteristics across recruitment sources and with external target populations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      Using data from the online Dutch\n                      <jats:italic>Rheumatic<\/jats:italic>\n                      ? Questionnaire on musculoskeletal complaints, we compared baseline characteristics and key self-reported symptoms between responders to the follow-up survey and nonresponders. The survey responders were furthermore compared according to source of recruitment to the questionnaire, i.e., via primary care clinics, secondary care clinics, or via different online sources. Sex, age and BMI distributions from the total study group were compared to external data of potential target populations of primary and secondary care patients within the Netherlands.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      The total study group of answers to the questionnaire comprised 31,457 responders, of which 50% (\n                      <jats:italic>n<\/jats:italic>\n                      \u202f=\u202f15,591) responded to the follow-up survey. Study participants were predominantly female (76%), middle-aged (one-third 50\u201360\u202fyears), never-smokers (66%), and overweight. While participants recruited through healthcare settings resembled target populations, follow-up survey responders were older, had more rheumatic diagnoses (49% vs. 32%), and reported more symptoms than non-responders. Participant characteristics varied by recruitment source, with social media attracting younger females while healthcare routes reached more diverse populations with varying symptom presentations.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Patterns of recruitment and attrition produced differences in participant characteristics. Healthcare-based recruitment yielded participants resembling intended target populations, and follow-up survey responders differed on some points from nonresponders. Awareness of these selection processes is essential when using real-world symptom checker data for model development.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/frai.2026.1815241","type":"journal-article","created":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T05:50:47Z","timestamp":1778219447000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Attrition and representativeness in development and validation of online symptom checkers\u2014a case study on the Rheumatic? Questionnaire"],"prefix":"10.3389","volume":"9","author":[{"given":"Floor Dijkstra","family":"Zegers","sequence":"first","affiliation":[{"name":"Department of Biomedical Data Sciences, Leiden University Medical Center","place":["Leiden, Netherlands"]},{"name":"Department of Rheumatology, Leiden University Medical Center","place":["Leiden, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ling","family":"Qin","sequence":"additional","affiliation":[{"name":"Department of Rheumatology, Leiden University Medical Center","place":["Leiden, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniyal","family":"Selani","sequence":"additional","affiliation":[{"name":"Department of Intelligent Systems, Pattern Recognition and Bioinformatics, Delft University of Technology","place":["Delft, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Georgy","family":"Gomon","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Sciences, Leiden University Medical Center","place":["Leiden, Netherlands"]},{"name":"Department of Rheumatology, Leiden University Medical Center","place":["Leiden, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tjardo","family":"Maarseveen","sequence":"additional","affiliation":[{"name":"Department of Rheumatology, Leiden University Medical Center","place":["Leiden, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kasper","family":"Glas","sequence":"additional","affiliation":[{"name":"Reumazorg Zuid-West Nederland","place":["Roosendaal, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Astrid","family":"van Tubergen","sequence":"additional","affiliation":[{"name":"Department of Rheumatology, Maastricht University Medical Centre+","place":["Maastricht, Netherlands"]},{"name":"Care and Public Health Research Institute (CAPHRI), Maastricht University","place":["Maastricht, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yvonne Goekoop","family":"Ruiterman","sequence":"additional","affiliation":[{"name":"Department of Clinical Epidemiology, Haga Hospital","place":["The Hague, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marcel","family":"Reinders","sequence":"additional","affiliation":[{"name":"Department of Intelligent Systems, Pattern Recognition and Bioinformatics, Delft University of Technology","place":["Delft, Netherlands"]},{"name":"Department of Clinical Epidemiology, Leiden University Medical Center","place":["Leiden, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Erik","family":"van den Akker","sequence":"additional","affiliation":[{"name":"Department of Intelligent Systems, Pattern Recognition and Bioinformatics, Delft University of Technology","place":["Delft, Netherlands"]},{"name":"Department of Clinical Epidemiology, Leiden University Medical Center","place":["Leiden, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Corne Baatenburg","family":"de Jong","sequence":"additional","affiliation":[{"name":"Research, Innovation and Impact Funding, ReumaNederland (Dutch Arthritis Society)","place":["Amsterdam, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lars","family":"Klareskog","sequence":"additional","affiliation":[{"name":"Division of Rheumatology, Department of Medicine, Karolinska Institutet","place":["Stockholm, Sweden"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Barbara","family":"Axn\u00e4s","sequence":"additional","affiliation":[{"name":"Elsa Science","place":["Stockholm, Sweden"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Reinhard","family":"Bos","sequence":"additional","affiliation":[{"name":"Department of Rheumatology, Medical Center Leeuwarden","place":["Leeuwarden, Netherlands"]},{"name":"Department of Rheumatology, Nij Smellinghe Hospital","place":["Drachten, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Saskia","family":"le Cessie","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Sciences, Leiden University Medical Center","place":["Leiden, Netherlands"]},{"name":"Department of Clinical Epidemiology, Leiden University Medical Center","place":["Leiden, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rachel","family":"Knevel","sequence":"additional","affiliation":[{"name":"Department of Rheumatology, Leiden University Medical Center","place":["Leiden, Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2026,5,8]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"170","DOI":"10.1038\/s41746-023-00913-9","article-title":"Considerations for addressing bias in artificial intelligence for health equity","volume":"6","author":"Abramoff","year":"2023","journal-title":"npj Digit. Med."},{"key":"ref2","doi-asserted-by":"publisher","first-page":"104769","DOI":"10.1016\/j.healthpol.2023.104769","article-title":"The introduction of a data-driven population health management approach in the Netherlands since 2019: the extramural LUMC academic network data infrastructure","volume":"132","author":"Ardesch","year":"2023","journal-title":"Health Policy"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"e36322","DOI":"10.2196\/36322","article-title":"Health information seeking from an intelligent web-based symptom checker: cross-sectional questionnaire study","volume":"24","author":"Arellano Carmona","year":"2022","journal-title":"J. Med. Internet Res."},{"key":"ref4","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1093\/jbcr\/irz186","article-title":"Factors associated with attrition of adult participants in a longitudinal database: a national institute on disability, independent living, and rehabilitation research burn model system study","volume":"41","author":"Bamer","year":"2020","journal-title":"J. Burn Care Res."},{"key":"ref5","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1111\/j.1751-5823.2010.00112.x","article-title":"Selection bias in web surveys","volume":"78","author":"Bethlehem","year":"2010","journal-title":"Int. Stat. Rev."},{"key":"ref6","doi-asserted-by":"publisher","first-page":"91","DOI":"10.2147\/CLEP.S72247","article-title":"Using multiple imputation to deal with missing data and attrition in longitudinal studies with repeated measures of patient-reported outcomes","volume":"7","author":"Biering","year":"2015","journal-title":"Clin. Epidemiol."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"1996","DOI":"10.1177\/1461444820924631","article-title":"Vulnerability in a tracked society: combining tracking and survey data to understand who gets targeted with what content","volume":"22","author":"Bol","year":"2020","journal-title":"New Media Soc."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1177\/1403494820919544","article-title":"Assessment of selection bias due to dropouts in the follow-up of the Scania public health cohort","volume":"49","author":"Canivet","year":"2021","journal-title":"Scand. J. Public Health"},{"key":"ref9","first-page":"1","volume-title":"Wiley StatsRef: Statistics Reference Online","author":"de Leeuw","year":"2015"},{"key":"ref10","volume-title":"Tweede Nationale Studie naar ziekten en verrichtingen in de huisartspraktijk. Klachten en aandoeningen in de bevolking en in de huisartspraktijk","author":"der van Linden","year":"2004"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"e053566","DOI":"10.1136\/bmjopen-2021-053566","article-title":"What is the suitability of clinical vignettes in benchmarking the performance of online symptom checkers? An audit study","volume":"12","author":"El-Osta","year":"2022","journal-title":"BMJ Open"},{"key":"ref12","year":"2024"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1097\/BOR.0b013e3283235ac4","article-title":"Early inflammatory arthritis versus rheumatoid arthritis","volume":"21","author":"Finckh","year":"2009","journal-title":"Curr. Opin. Rheumatol."},{"key":"ref14","doi-asserted-by":"publisher","first-page":"2263","DOI":"10.1016\/S0140-6736(18)32819-8","article-title":"Safety of patient-facing digital symptom checkers","volume":"392","author":"Fraser","year":"2018","journal-title":"Lancet"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1002\/acr.22125","article-title":"Response rates of three modes of survey administration and survey preferences of rheumatoid arthritis patients","volume":"66","author":"Garcia","year":"2014","journal-title":"Arthritis Care Res."},{"key":"ref16","year":"2024"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"918","DOI":"10.1186\/1471-2458-12-918","article-title":"Attrition and generalizability in longitudinal studies: findings from a 15-year population-based study and a Monte Carlo simulation study","volume":"12","author":"Gustavson","year":"2012","journal-title":"BMC Public Health"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1186\/s12875-023-01976-z","article-title":"Prevalence and characteristics of musculoskeletal complaints in primary care: an analysis from the population level and analysis reporting (POLAR) database","volume":"24","author":"Haas","year":"2023","journal-title":"BMC Prim. Care"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1093\/fampra\/13.3.294","article-title":"The international classification of primary care (ICPC): new applications in research and computer-based patient records in family practice","volume":"13","author":"Hofmans-Okkes","year":"1996","journal-title":"Fam. Pract."},{"key":"ref20","article-title":"gt: Easily create presentation-ready display tables R package version 1.3.0.9000","author":"Iannone","year":"2024"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"503","DOI":"10.1007\/s00296-022-05187-0","article-title":"Delay in diagnosis of rheumatoid arthritis: reasons and trends over a decade","volume":"43","author":"Jain","year":"2023","journal-title":"Rheumatol. Int."},{"key":"ref22","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1111\/rssa.12136","article-title":"Perils and potentials of self-selected entry to epidemiological studies and surveys","volume":"179","author":"Keiding","year":"2016","journal-title":"J. R. Stat. Soc. A"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"774945","DOI":"10.3389\/fmed.2022.774945","article-title":"Rheumatic?\u2014a digital diagnostic decision support tool for individuals suspecting rheumatic diseases: a Multicenter pilot validation study","volume":"9","author":"Knevel","year":"2022","journal-title":"Front. Med."},{"key":"ref24","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1186\/s13075-021-02498-8","article-title":"Accuracy, patient-perceived usability, and acceptance of two symptom checkers (Ada and Rheport) in rheumatology: interim results from a randomized controlled crossover trial","volume":"23","author":"Knitza","year":"2021","journal-title":"Arthritis Res. Ther."},{"key":"ref25","doi-asserted-by":"publisher","first-page":"e46231","DOI":"10.2196\/46231","article-title":"Characteristics of users and nonusers of symptom checkers in Germany: cross-sectional survey study","volume":"25","author":"Kopka","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref26","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1038\/s41746-024-01037-4","article-title":"Why do probabilistic clinical models fail to transport between sites?","volume":"7","author":"Lasko","year":"2024","journal-title":"npj Digit Med."},{"key":"ref27","doi-asserted-by":"publisher","first-page":"e002974","DOI":"10.1136\/rmdopen-2022-002974","article-title":"Population-based user-perceived experience of Rheumatic?: a novel digital symptom-checker in rheumatology","volume":"9","author":"Lundberg","year":"2023","journal-title":"RMD Open"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"e20549","DOI":"10.2196\/20549","article-title":"Use characteristics and triage acuity of a digital symptom checker in a large integrated health system: population-based descriptive study","volume":"22","author":"Morse","year":"2020","journal-title":"J. Med. Internet Res."},{"key":"ref29","doi-asserted-by":"publisher","first-page":"226","DOI":"10.1093\/ije\/dyx206","article-title":"Collider scope: when selection bias can substantially influence observed associations","volume":"47","author":"Munafo","year":"2018","journal-title":"Int. J. Epidemiol."},{"key":"ref30","doi-asserted-by":"publisher","first-page":"e066594","DOI":"10.1136\/bmjopen-2022-066594","article-title":"Exploring participant attrition in a longitudinal follow-up of older adults: the Global Longitudinal Study of Osteoporosis in Women (GLOW) Hamilton cohort","volume":"13","author":"Okpara","year":"2023","journal-title":"BMJ Open"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"dyae097","DOI":"10.1093\/ije\/dyae097","article-title":"Inverse probability weighting for self-selection bias correction in the investigation of social inequality in mortality","volume":"53","author":"Petersen","year":"2024","journal-title":"Int. J. Epidemiol."},{"key":"ref32","volume-title":"R: A Language and Environment for Statistical Computing","year":"2023"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"169","DOI":"10.5114\/reum.2017.69777","article-title":"Diagnostic delays in rheumatic diseases with associated arthritis","volume":"55","author":"Raciborski","year":"2017","journal-title":"Reumatologia"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"570","DOI":"10.32614\/RJ-2021-053","article-title":"Reproducible summary tables with the gtsummary package","volume":"31","author":"Sjoberg","year":"2021","journal-title":"R J."},{"key":"ref35","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1186\/s41512-022-00136-8","article-title":"Targeted validation: validating clinical prediction models in their intended population and setting","volume":"6","author":"Sperrin","year":"2022","journal-title":"Diagn. Progn. Res."},{"key":"ref36","doi-asserted-by":"publisher","first-page":"b2393","DOI":"10.1136\/bmj.b2393","article-title":"Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls","volume":"338","author":"Sterne","year":"2009","journal-title":"BMJ"},{"key":"ref37","doi-asserted-by":"crossref","DOI":"10.1201\/9780429492259","volume-title":"Flexible Imputation of Missing Data","author":"van Buuren","year":"2018"},{"key":"ref38","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1038\/s41746-022-00667-w","article-title":"The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review","volume":"5","author":"Wallace","year":"2022","journal-title":"npj Digit Med."},{"key":"ref39","volume-title":"StatPearls","author":"Weir","year":"2024"},{"key":"ref40","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-24277-4","volume-title":"ggplot2: Elegant Graphics for Data Analysis","author":"Wickham","year":"2016"},{"key":"ref41","doi-asserted-by":"publisher","first-page":"e1918561","DOI":"10.1001\/jamanetworkopen.2019.18561","article-title":"Association of use of online symptom checkers with patients' plans for seeking care","volume":"2","author":"Winn","year":"2019","journal-title":"JAMA Netw. Open"},{"key":"ref42","doi-asserted-by":"publisher","first-page":"51","DOI":"10.7326\/m18-1376","article-title":"PROBAST: a tool to assess the risk of bias and applicability of prediction model studies","volume":"170","author":"Wolff","year":"2019","journal-title":"Ann. Intern. Med."},{"key":"ref43","doi-asserted-by":"publisher","first-page":"100206","DOI":"10.1016\/j.chbr.2022.100206","article-title":"Response rates of online surveys in published research: a meta-analysis","volume":"7","author":"Wu","year":"2022","journal-title":"Comput. Hum. Behav. Rep."},{"key":"ref44","doi-asserted-by":"publisher","first-page":"1025","DOI":"10.1093\/aje\/kwu018","article-title":"Multilevel regression and poststratification for small-area estimation of population health outcomes: a case study of chronic obstructive pulmonary disease prevalence using the behavioral risk factor surveillance system","volume":"179","author":"Zhang","year":"2014","journal-title":"Am. J. Epidemiol."}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1815241\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T05:50:49Z","timestamp":1778219449000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1815241\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,8]]},"references-count":44,"alternative-id":["10.3389\/frai.2026.1815241"],"URL":"https:\/\/doi.org\/10.3389\/frai.2026.1815241","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,8]]},"article-number":"1815241"}}