{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T00:49:29Z","timestamp":1791420569513,"version":"4.3.3"},"reference-count":29,"publisher":"Georg Thieme Verlag KG","issue":"04","funder":[{"name":"Agency for Healthcare Research and Quality","award":["R01HS029513"],"award-info":[{"award-number":["R01HS029513"]}],"id":[{"id":"https:\/\/ror.org\/03jmfdf59","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2026,8]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Background Diagnostic errors are common in medical care and can significantly affect patient outcomes. Symptom checker apps promise to help patients understand their symptoms, possible diagnoses, and actions to take, but lack evidence on accuracy, safety, and usability.<\/jats:p>\n                  <jats:p>Objectives This study aimed to evaluate the usability and acceptability of the Ada symptom checker app for patients in urgent primary care (PC) settings, and extended it with more diverse data from an earlier emergency department (ED) study.<\/jats:p>\n                  <jats:p>Methods Patients entered their symptoms into Ada before assessment by a physician. Both the PC study and the ED study compared Ada\u2019s diagnoses to the treating physicians\u2019 diagnoses. Usability and acceptability were evaluated using a questionnaire based on the Technology Acceptance Model (TAM). Factors influencing usability were analyzed using linear and logistic regression. Quality of clinical history was also evaluated.<\/jats:p>\n                  <jats:p>Results The study included 214 urgent PC patients and 40 ED patients. Usability scores were generally high; 73% of PC patients agreed or strongly agreed with the seven TAM usability questions, and ED patients scored higher at 84% (with more direct assistance). Predictors of lower usability scores were age over 50 and limited technology experience. Sex, race, ethnicity, socioeconomic status, and educational level were not significant. In the PC study, Ada\u2019s top three diagnoses matched at least one treating physician diagnosis in 109\/172 (63.3%) of cases with clear final diagnoses (sensitivity); three independent physicians had median matching of 124\/172 (72.1%) on Ada summaries. Clinical history was rated complete in two-thirds of cases.<\/jats:p>\n                  <jats:p>Conclusion The usability and acceptance of Ada were rated highly except by people older than 50 or with lower technology experience. Extensive research is needed to validate the usability of different diagnostic tools, including symptom checkers and large language models (used widely for symptom checking), across diverse populations, and their effects on patient decision-making and care-seeking behavior.<\/jats:p>","DOI":"10.1055\/a-2962-9261","type":"journal-article","created":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T23:16:50Z","timestamp":1790637410000},"page":"863-872","source":"Crossref","is-referenced-by-count":0,"title":["Usability and Acceptability of a Symptom Checker App by Patients Seeking Urgent Care"],"prefix":"10.1055","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4383-2854","authenticated-orcid":false,"given":"Hamish S.F.","family":"Fraser","sequence":"additional","affiliation":[{"name":"Brown Center for Biomedical Informatics, Brown University Warren Alpert Medical School, Rhode Island, United States","place":["Providence"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ian","family":"Bacher","sequence":"additional","affiliation":[{"name":"Brown Center for Biomedical Informatics, Brown University Warren Alpert Medical School, Rhode Island, United States","place":["Providence"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ishaani","family":"Khatri","sequence":"additional","affiliation":[{"name":"New York University Langone Hospital, New York, United States","place":["New York"],"department":["Department of Internal Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sasha","family":"Raman","sequence":"additional","affiliation":[{"name":"Baylor College of Medicine, Texas, United States","place":["Houston"],"department":["Department of Internal Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Drew","family":"Nagle","sequence":"additional","affiliation":[{"name":"Brown University Health, Rhode Island, United States","place":["Providence"],"department":["Department of Internal Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daven","family":"Crossland","sequence":"additional","affiliation":[{"name":"Health Equity, Policy and Planning, Alameda County Public Health Department, California, United States","place":["San Leandro"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tracy","family":"Madsen","sequence":"additional","affiliation":[{"name":"Larner College of Medicine, University of Vermont, Vermont, United States","place":["Burlington"],"department":["Department of Emergency Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ross","family":"Hilliard","sequence":"additional","affiliation":[{"name":"Maine Medical Center, Maine, United States","place":["Portland"],"department":["Department of Internal Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"194","published-online":{"date-parts":[[2026,10,7]]},"reference":[{"issue":"09","key":"ref1","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1136\/bmjqs-2013-002627","article-title":"The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations","volume":"23","author":"H Singh","year":"2014","journal-title":"BMJ Qual Saf"},{"issue":"02","key":"ref2","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1136\/bmjqs-2021-014130","article-title":"Burden of serious harms from diagnostic error in the USA","volume":"33","author":"D E Newman-Toker","year":"2024","journal-title":"BMJ Qual Saf"},{"issue":"08","key":"ref3","doi-asserted-by":"crossref","first-page":"2723","DOI":"10.1007\/s00415-020-09803-6","article-title":"Factors delaying intravenous thrombolytic therapy in acute ischaemic stroke: a systematic review of the literature","volume":"268","author":"A Sharobeam","year":"2021","journal-title":"J Neurol"},{"issue":"10","key":"ref4","doi-asserted-by":"crossref","first-page":"1223","DOI":"10.1161\/01.CIR.0000121424.76486.20","article-title":"Time delay to treatment and mortality in primary angioplasty for acute myocardial infarction: every minute of delay counts","volume":"109","author":"G De Luca","year":"2004","journal-title":"Circulation"},{"issue":"01","key":"ref5","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1186\/s13075-021-02498-8","article-title":"Accuracy, patient-perceived usability, and acceptance of two symptom checkers (Ada and Rheport) in rheumatology: interim results from a randomized controlled crossover trial","volume":"23","author":"J Knitza","year":"2021","journal-title":"Arthritis Res Ther"},{"issue":"05","key":"ref6","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1038\/nrclinonc.2012.63","article-title":"Diagnosis: cancer diagnosis in UK primary care","volume":"9","author":"W Hamilton","year":"2012","journal-title":"Nat Rev Clin Oncol"},{"issue":"03","key":"ref7","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1515\/dx-2018-0009","article-title":"Google: the evidence on consumer-facing digital tools for diagnosis","volume":"5","author":"M L Millenson","year":"2018","journal-title":"Diagnosis (Berl)"},{"issue":"03","key":"ref8","doi-asserted-by":"crossref","first-page":"e213287","DOI":"10.1001\/jamanetworkopen.2021.3287","article-title":"Assessment of diagnosis and triage in validated case vignettes among nonphysicians before and after internet search","volume":"4","author":"D M Levine","year":"2021","journal-title":"JAMA Netw Open"},{"key":"ref9","doi-asserted-by":"crossref","first-page":"h3480","DOI":"10.1136\/bmj.h3480","article-title":"Evaluation of symptom checkers for self diagnosis and triage: audit study","volume":"351","author":"H L Semigran","year":"2015","journal-title":"BMJ"},{"issue":"05","key":"ref10","doi-asserted-by":"crossref","first-page":"e31810","DOI":"10.2196\/31810","article-title":"Triage accuracy of symptom checker apps: 5-year follow-up evaluation","volume":"24","author":"M L Schmieding","year":"2022","journal-title":"J Med Internet Res"},{"issue":"12","key":"ref11","doi-asserted-by":"crossref","first-page":"e040269","DOI":"10.1136\/bmjopen-2020-040269","article-title":"How accurate are digital symptom assessment apps for suggesting conditions and urgency advice? A clinical vignettes comparison to GPs","volume":"10","author":"S Gilbert","year":"2020","journal-title":"BMJ Open"},{"key":"ref12","doi-asserted-by":"crossref","first-page":"2263","DOI":"10.1016\/S0140-6736(18)32819-8","article-title":"Safety of patient-facing digital symptom checkers","volume":"392","author":"H Fraser","year":"2018","journal-title":"Lancet"},{"issue":"08","key":"ref13","doi-asserted-by":"crossref","first-page":"e027743","DOI":"10.1136\/bmjopen-2018-027743","article-title":"Digital and online symptom checkers and health assessment\/triage services for urgent health problems: systematic review","volume":"9","author":"D Chambers","year":"2019","journal-title":"BMJ Open"},{"issue":"12","key":"ref14","doi-asserted-by":"crossref","first-page":"e1918561","DOI":"10.1001\/jamanetworkopen.2019.18561","article-title":"Association of use of online symptom checkers with patients\u2019 plans for seeking care","volume":"2","author":"A N Winn","year":"2019","journal-title":"JAMA Netw Open"},{"issue":"11","key":"ref15","doi-asserted-by":"crossref","first-page":"e0000373","DOI":"10.1371\/journal.pdig.0000373","article-title":"Digital determinants of health: Editorial","volume":"2","author":"H S Fraser","year":"2023","journal-title":"PLOS Digit Health"},{"issue":"09","key":"ref17","doi-asserted-by":"crossref","first-page":"e38364","DOI":"10.2196\/38364","article-title":"Evaluation of diagnostic and triage accuracy and usability of a symptom checker in an emergency department: observational study","volume":"10","author":"H SF Fraser","year":"2022","journal-title":"JMIR Mhealth Uhealth"},{"issue":"03","key":"ref18","doi-asserted-by":"crossref","first-page":"e32340","DOI":"10.2196\/32340","article-title":"Safety of triage self-assessment using a symptom assessment app for walk-in patients in the emergency care setting: observational prospective cross-sectional study","volume":"10","author":"F Cotte","year":"2022","journal-title":"JMIR Mhealth Uhealth"},{"issue":"07","key":"ref19","doi-asserted-by":"crossref","first-page":"e0254088","DOI":"10.1371\/journal.pone.0254088","article-title":"Accuracy of online symptom checkers and the potential impact on service utilisation","volume":"16","author":"A Ceney","year":"2021","journal-title":"PLoS One"},{"issue":"09","key":"ref20","doi-asserted-by":"crossref","first-page":"1141","DOI":"10.1089\/tmj.2019.0144","article-title":"Assessment of the intention to use mobile health applications using a technology acceptance model in an Israeli adult population","volume":"26","author":"T Shemesh","year":"2020","journal-title":"Telemed J e-Health"},{"issue":"05","key":"ref22","doi-asserted-by":"crossref","first-page":"e13117","DOI":"10.2196\/13117","article-title":"Theories predicting end-user acceptance of telemedicine use: systematic review","volume":"21","author":"L Harst","year":"2019","journal-title":"J Med Internet Res"},{"issue":"04","key":"ref23","doi-asserted-by":"crossref","first-page":"e11604","DOI":"10.2196\/11604","article-title":"Using the technology acceptance model to explore user experience, intent to use, and use behavior of a patient portal among older adults with multiple chronic conditions: descriptive qualitative study","volume":"21","author":"J D Portz","year":"2019","journal-title":"J Med Internet Res"},{"key":"ref24","doi-asserted-by":"crossref","first-page":"105535","DOI":"10.1016\/j.childyouth.2020.105535","article-title":"Perceived usability evaluation of Microsoft Teams as an online learning platform during COVID-19 using system usability scale and technology acceptance model in India","volume":"119","author":"D Pal","year":"2020","journal-title":"Child Youth Serv Rev"},{"issue":"02","key":"ref25","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1016\/j.jbi.2008.08.010","article-title":"Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support","volume":"42","author":"P A Harris","year":"2009","journal-title":"J Biomed Inform"},{"issue":"01","key":"ref26","doi-asserted-by":"crossref","first-page":"e32832","DOI":"10.2196\/32832","article-title":"Diagnostic performance of an app-based symptom checker in mental disorders: comparative study in psychotherapy outpatients","volume":"9","author":"S Hennemann","year":"2022","journal-title":"JMIR Ment Health"},{"issue":"02","key":"ref27","doi-asserted-by":"crossref","first-page":"e28199","DOI":"10.2196\/28199","article-title":"Improving emergency department patient-physician conversation through an artificial intelligence symptom-taking tool: mixed methods pilot observational study","volume":"6","author":"J Scheder-Bieschin","year":"2022","journal-title":"JMIR Form Res"},{"key":"ref28","doi-asserted-by":"crossref","first-page":"e56628","DOI":"10.2196\/56628","article-title":"Transforming health care through chatbots for medical history-taking and future directions: comprehensive systematic review","volume":"12","author":"M Hindelang","year":"2024","journal-title":"JMIR Med Inform"},{"issue":"01","key":"ref29","doi-asserted-by":"crossref","first-page":"e14679","DOI":"10.2196\/14679","article-title":"Patient perspectives on the usefulness of an artificial intelligence-assisted symptom checker: cross-sectional survey study","volume":"22","author":"A ND Meyer","year":"2020","journal-title":"J Med Internet Res"},{"key":"ref30","doi-asserted-by":"crossref","first-page":"e49995","DOI":"10.2196\/49995","article-title":"Comparison of diagnostic and triage accuracy of Ada Health and WebMD Symptom Checkers, ChatGPT, and physicians for patients in an emergency department: clinical data analysis study","volume":"11","author":"H Fraser","year":"2023","journal-title":"JMIR Mhealth Uhealth"},{"issue":"04","key":"ref31","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1001\/jama.2024.21700","article-title":"Testing and evaluation of health care applications of large language models: a systematic review","volume":"333","author":"S Bedi","year":"2025","journal-title":"JAMA"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/a-2962-9261.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T23:51:17Z","timestamp":1791417077000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/a-2962-9261"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8]]},"references-count":29,"journal-issue":{"issue":"04","published-online":{"date-parts":[[2026,8,14]]},"published-print":{"date-parts":[[2026,8]]}},"URL":"https:\/\/doi.org\/10.1055\/a-2962-9261","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,9,28]]},"published":{"date-parts":[[2026,8]]}}}