{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,13]],"date-time":"2025-09-13T16:11:21Z","timestamp":1757779881494,"version":"3.37.3"},"reference-count":36,"publisher":"Oxford University Press (OUP)","issue":"9","license":[{"start":{"date-parts":[[2021,6,22]],"date-time":"2021-06-22T00:00:00Z","timestamp":1624320000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["K24CA212294"],"award-info":[{"award-number":["K24CA212294"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,8,13]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We evaluated the usability of mobile COVID-19 contact tracing apps, especially for individuals with barriers to communication and limited digital literacy skills. We searched the Apple App Store, Google Play, peer-reviewed literature, and lay press to find contact tracing apps in the United States. We evaluated apps with a framework focused on user characteristics and user interface. Of the final 26 apps, 77% were on both iPhone and Android. 69% exceeded 9th grade readability, and 65% were available only in English. Only 12% had inclusive illustrations (different genders, skin tones, physical abilities). 92% alerted users of an exposure, 42% linked to a testing site, and 62% linked to a public health website within 3 clicks. Most apps alert users of COVID-19 exposure but require high English reading levels and are not fully inclusive of the U.S. population, which may limit their reach as public health tools.<\/jats:p>","DOI":"10.1093\/jamia\/ocab093","type":"journal-article","created":{"date-parts":[[2021,5,21]],"date-time":"2021-05-21T11:10:11Z","timestamp":1621595411000},"page":"1982-1989","source":"Crossref","is-referenced-by-count":27,"title":["Usability, inclusivity, and content evaluation of COVID-19 contact tracing apps in the United States"],"prefix":"10.1093","volume":"28","author":[{"given":"Serena O","family":"Blacklow","sequence":"first","affiliation":[{"name":"San Francisco School of Medicine, University of California, San Francisco, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sarah","family":"Lisker","sequence":"additional","affiliation":[{"name":"Center for Vulnerable Populations, Department of Medicine, San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA"},{"name":"Division of General Internal Medicine, Department of Medicine, San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Madelena Y","family":"Ng","sequence":"additional","affiliation":[{"name":"School of Public Health, University of California, Berkeley, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4213-4405","authenticated-orcid":false,"given":"Urmimala","family":"Sarkar","sequence":"additional","affiliation":[{"name":"Center for Vulnerable Populations, Department of Medicine, San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA"},{"name":"Division of General Internal Medicine, Department of Medicine, San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1111-8595","authenticated-orcid":false,"given":"Courtney","family":"Lyles","sequence":"additional","affiliation":[{"name":"Center for Vulnerable Populations, Department of Medicine, San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA"},{"name":"Division of General Internal Medicine, Department of Medicine, San Francisco General Hospital, University of California San Francisco, San Francisco, California, USA"},{"name":"Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2021,6,22]]},"reference":[{"year":"2020","key":"2021081407010982700_ocab093-B1"},{"year":"2020","author":"Eskenazi","key":"2021081407010982700_ocab093-B2"},{"year":"2020","author":"Bollag","key":"2021081407010982700_ocab093-B3"},{"key":"2021081407010982700_ocab093-B4","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1186\/s41256-020-00164-1","article-title":"Digital technology applications for contact tracing: the new promise for COVID-19 and beyond?","volume":"5","author":"Owusu","year":"2020","journal-title":"Global Health Res Pol"},{"issue":"10","key":"2021081407010982700_ocab093-B5","doi-asserted-by":"crossref","first-page":"1064","DOI":"10.1016\/j.eng.2020.09.001","article-title":"COVID-19 Fight enlists digital technology: contact tracing apps","volume":"6","author":"Leslie","year":"2020","journal-title":"Engineering"},{"issue":"6","key":"2021081407010982700_ocab093-B6","first-page":"1631","article-title":"Integrating emerging technologies into COVID-19 contact tracing: Opportunities, challenges, and pitfalls. Diabetes & Metabolic Syndrome","volume":"14","author":"Mbunge","year":"2020","journal-title":"Clin Res Rev"},{"key":"2021081407010982700_ocab093-B7","article-title":"Effective configurations of a digital contact tracing app: a report to NHSX","author":"Hinch","year":"2020","journal-title":". MedSci"},{"issue":"8","key":"2021081407010982700_ocab093-B8","doi-asserted-by":"crossref","first-page":"e19857","DOI":"10.2196\/19857","article-title":"Acceptability of app-based contact tracing for COVID-19: Cross-Country Survey Study","volume":"8","author":"Altmann","year":"2020","journal-title":"JMIR Mhealth Uhealth"},{"issue":"9","key":"2021081407010982700_ocab093-B9","doi-asserted-by":"crossref","first-page":"e19796","DOI":"10.2196\/19796","article-title":"Health apps on COVID-19 launched in the early days of the pandemic: content analysis and review","volume":"8","author":"Ming","year":"2020","journal-title":"JMIR Mhealth Uhealth"},{"issue":"4","key":"2021081407010982700_ocab093-B10","doi-asserted-by":"crossref","first-page":"e18936","DOI":"10.2196\/18936","article-title":"Peer-to-peer contact tracing: development of a privacy-preserving smartphone app","volume":"8","author":"Yasaka","year":"2020","journal-title":"JMIR Mhealth Uhealth"},{"issue":"1","key":"2021081407010982700_ocab093-B11","doi-asserted-by":"crossref","DOI":"10.1186\/s40249-020-00711-5","article-title":"Personalizing symptom monitoring and contact tracing efforts through a COVID-19 web-app","volume":"9","author":"Yap","year":"2020","journal-title":"Infect Dis Poverty"},{"year":"2020","key":"2021081407010982700_ocab093-B12"},{"issue":"12","key":"2021081407010982700_ocab093-B13","doi-asserted-by":"crossref","first-page":"e24693","DOI":"10.2196\/24693","article-title":"Attitudes toward using COVID-19 mHealth tools among adults with chronic health conditions: secondary data analysis of the COVID-19 impact survey","volume":"8","author":"Camacho-Rivera","year":"2020","journal-title":"JMIR Mhealth Uhealth"},{"issue":"1","key":"2021081407010982700_ocab093-B14","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1038\/s41746-020-00357-5","article-title":"Belief of having had unconfirmed Covid-19 infection reduces willingness to participate in app-based contact tracing","volume":"3","author":"Bachtiger","year":"2020","journal-title":"NPJ Digit Med"},{"issue":"10","key":"2021081407010982700_ocab093-B15","doi-asserted-by":"crossref","first-page":"e0241473","DOI":"10.1371\/journal.pone.0241473","article-title":"Facilitators and barriers to engagement with contact tracing during infectious disease outbreaks: a rapid review of the evidence","volume":"15","author":"Megnin-Viggars","year":"2020","journal-title":"PLoS One"},{"issue":"3","key":"2021081407010982700_ocab093-B16","doi-asserted-by":"crossref","first-page":"e20572","DOI":"10.2196\/20572","article-title":"Adoption of a contact tracing app for containing COVID-19: a health belief model approach","volume":"6","author":"Walrave","year":"2020","journal-title":"JMIR Public Health Surveill"},{"year":"12, 2019.","key":"2021081407010982700_ocab093-B17"},{"key":"2021081407010982700_ocab093-B18","first-page":"08","article-title":"Accessible health information technology for populations with limited literacy: a guide for developers and purchasers of health IT. (Prepared by the National Opinion Research Center for the National Resource Center for Health IT.)","author":"Eichner","year":"2007","journal-title":"Agency Healthcare Res Qual"},{"key":"2021081407010982700_ocab093-B19","article-title":"Developing a framework for evaluating the patient engagement, quality, and safety of mobile health applications","author":"Singh","year":"2016","journal-title":"The Commonwealth Fund"},{"issue":"9","key":"2021081407010982700_ocab093-B20","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1007\/s10916-020-01633-3","article-title":"A systematic review of smartphone applications available for Corona Virus Disease 2019 (COVID19) and the assessment of their quality using the Mobile Application Rating Scale (MARS)","volume":"44","author":"Davalbhakta","year":"2020","journal-title":"J Med Syst"},{"issue":"8","key":"2021081407010982700_ocab093-B21","first-page":"e20334","article-title":"Features and functionalities of smartphone apps related to COVID-19: systematic search in app stores and content analysis","volume":"22","author":"Collado-Borrell","year":"2020","journal-title":"JMIR"},{"issue":"11","key":"2021081407010982700_ocab093-B22","doi-asserted-by":"crossref","first-page":"e22594","DOI":"10.2196\/22594","article-title":"Citizen-centered mobile health apps collecting individual-level spatial data for infectious disease management: scoping review","volume":"8","author":"Wirth","year":"2020","journal-title":"JMIR Mhealth Uhealth"},{"issue":"12","key":"2021081407010982700_ocab093-B23","doi-asserted-by":"crossref","first-page":"e21572","DOI":"10.2196\/21572","article-title":"COVID-19 contact-tracing apps: analysis of the readability of privacy policies","volume":"22","author":"Zhang","year":"2020","journal-title":"J Med Internet Res"},{"year":"2020","key":"2021081407010982700_ocab093-B24"},{"key":"2021081407010982700_ocab093-B25","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1145\/1882992.1883084","article-title":"Heuristic Evaluation of Persuasive Health Technologies","author":"Kientz","year":"2010","journal-title":"ACM Int Health Inform Symp"},{"key":"2021081407010982700_ocab093-B26"},{"key":"2021081407010982700_ocab093-B27","first-page":"8","article-title":"Derivation of new readability formulas (automated readability index, fog count, and Flesch reading ease formula) for Navy enlisted personnel","author":"Kincaid","year":"1975","journal-title":"Res Branch Rep"},{"year":"2018","key":"2021081407010982700_ocab093-B28"},{"author":"Kirsh","key":"2021081407010982700_ocab093-B29","article-title":"Why illustrations aid understanding. Department of Cognitive Science"},{"issue":"1","key":"2021081407010982700_ocab093-B30","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/0738-3991(90)90010-I","article-title":"Effective use of patient education illustrations","volume":"15","author":"Rohret","year":"1990","journal-title":"Patient Educ Couns"},{"issue":"2","key":"2021081407010982700_ocab093-B31","first-page":"29","article-title":"SUS: a retrospective","volume":"8","author":"Brooke","year":"2013","journal-title":"J Usab Stat"},{"issue":"1","key":"2021081407010982700_ocab093-B32","doi-asserted-by":"crossref","first-page":"e27","DOI":"10.2196\/mhealth.3422","article-title":"Mobile app rating scale: a new tool for assessing the quality of health mobile apps","volume":"3","author":"Stoyanov","year":"2015","journal-title":"JMIR Mhealth Uhealth"},{"key":"2021081407010982700_ocab093-B33","doi-asserted-by":"crossref","first-page":"w20282","DOI":"10.4414\/smw.2020.20282","article-title":"Digital health and the COVID-19 epidemic: an assessment framework for apps from an epidemiological and legal perspective","volume":"150","author":"Vokinger","year":"2020","journal-title":"Swiss Medical Weekly"},{"key":"2021081407010982700_ocab093-B34"},{"key":"2021081407010982700_ocab093-B35"},{"issue":"7","key":"2021081407010982700_ocab093-B36","doi-asserted-by":"crossref","DOI":"10.31478\/201707f","article-title":"Health literacy insights for health crises","volume":"7","author":"Parson","year":"2017","journal-title":"NAM Perspect"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/advance-article-pdf\/doi\/10.1093\/jamia\/ocab093\/38105983\/ocab093.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/9\/1982\/39731702\/ocab093.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/9\/1982\/39731702\/ocab093.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T06:14:37Z","timestamp":1672208077000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/9\/1982\/6281223"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,22]]},"references-count":36,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,6,22]]},"published-print":{"date-parts":[[2021,8,13]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab093","relation":{},"ISSN":["1527-974X"],"issn-type":[{"type":"electronic","value":"1527-974X"}],"subject":[],"published-other":{"date-parts":[[2021,9,1]]},"published":{"date-parts":[[2021,6,22]]}}}