{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,20]],"date-time":"2026-10-20T09:23:59Z","timestamp":1792488239129,"version":"4.5.0"},"reference-count":50,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Hum Factors"],"abstract":"\n Background<\/jats:title>\n Previous studies have evaluated the accuracy of the diagnostics of electronic symptom checkers (ESCs) and triage using clinical case vignettes. National Omaolo digital services (Omaolo) in Finland consist of an ESC for various symptoms. Omaolo is a medical device with a Conformit\u00e9 Europ\u00e9enne marking (risk class: IIa), based on Duodecim Clinical Decision Support, EBMEDS.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study investigates how well triage performed by the ESC nurse triage within the chief symptom list available in Omaolo (anal region symptoms, cough, diarrhea, discharge from the eye or watery or reddish eye, headache, heartburn, knee symptom or injury, lower back pain or injury, oral health, painful or blocked ear, respiratory tract infection, sexually transmitted disease, shoulder pain or stiffness or injury, sore throat or throat symptom, and urinary tract infection). In addition, the accuracy, specificity, sensitivity, and safety of the Omaolo ESC were assessed.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n This is a clinical validation study in a real-life setting performed at multiple primary health care (PHC) centers across Finland. The included units were of the walk-in model of primary care, where no previous phone call or contact was required. Upon arriving at the PHC center, users (patients) answered the ESC questions and received a triage recommendation; a nurse then assessed their triage. Findings on 877 patients were analyzed by matching the ESC recommendations with triage by the triage nurse.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Safe assessments by the ESC accounted for 97.6% (856\/877; 95% CI 95.6%-98.0%) of all assessments made. The mean of the exact match for all symptom assessments was 53.7% (471\/877; 95% CI 49.2%-55.9%). The mean value of the exact match or overly conservative but suitable for all (ESC\u2019s assessment was 1 triage level higher than the nurse\u2019s triage) symptom assessments was 66.6% (584\/877; 95% CI 63.4%-69.7%). When the nurse concluded that urgent treatment was needed, the ESC\u2019s exactly matched accuracy was 70.9% (244\/344; 95% CI 65.8%-75.7%). Sensitivity for the Omaolo ESC was 62.6% and specificity of 69.2%. A total of 21 critical assessments were identified for further analysis: there was no indication of compromised patient safety.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n The primary objectives of this study were to evaluate the safety and to explore the accuracy, specificity, and sensitivity of the Omaolo ESC. The results indicate that the ESC is safe in a real-life setting when appraised with assessments conducted by triage nurses. Furthermore, the Omaolo ESC exhibits the potential to guide patients to appropriate triage destinations effectively, helping them to receive timely and suitable care.<\/jats:p>\n <\/jats:sec>\n \n International Registered Report Identifier (IRRID)<\/jats:title>\n RR2-10.2196\/41423<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/55099","type":"journal-article","created":{"date-parts":[[2024,7,16]],"date-time":"2024-07-16T09:56:11Z","timestamp":1721123771000},"page":"e55099","source":"Crossref","is-referenced-by-count":9,"title":["Triage Accuracy and the Safety of User-Initiated Symptom Assessment With an Electronic Symptom Checker in a Real-Life Setting: Instrument Validation Study"],"prefix":"10.2196","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8441-3480","authenticated-orcid":false,"given":"Ville","family":"Liu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9645-4925","authenticated-orcid":false,"given":"Minna","family":"Kaila","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1518-4944","authenticated-orcid":false,"given":"Tuomas","family":"Koskela","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2024,9,26]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.2196\/15065"},{"key":"ref2","unstructured":"SebastianCAhmedMGuidoZBevanKSearch engines vs. symptom checkers: a comparison of their effectiveness for online health advice2021Proceedings of the Web Conference 2021June 3, 2021Ljubljana, Slovenia206216"},{"key":"ref3","unstructured":"EPatient surveyHealth&Care Management20202021-03-06https:\/\/www.hcm-magazin.de\/epatient-survey-2020-digital-health-studies\/150\/10992\/407743"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.2196\/22637"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.3961\/jpmph.20.094"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2022-065748"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-019-0206-x"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.2196\/31810"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.3390\/jpm10020021"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.2196\/20549"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.5811\/westjem.2019.6.41911"},{"key":"ref12","author":"Rodgers, M","year":"2019","journal-title":"Southampton (UK): NIHR Journals Library"},{"key":"ref13","unstructured":"Prepared by ipsos MORI and york health economics consortium with prof. chris salisbury for NHS hammersmith and fulham CCG and NHS EnglandEvaluation of Babylon GP at hand Final evaluation report20192024-08-17http:\/\/allcatsrgrey.org.uk\/wp\/download\/informatics\/Evaluation-of-Babylon-GP-at-Hand-Final-Report.pdf"},{"key":"ref14","unstructured":"NHS 111 minimum data set, England, March 2021NHS England2024-08-17https:\/\/www.england.nhs.uk\/statistics\/wp-content\/uploads\/sites\/2\/2021\/04\/NHS-111-MDS-Apr-2021-Statistical-Note.pdf"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1136\/bmjinnov-2020-000498"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocaa051"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1177\/14604582211052259"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.2196\/33505"},{"key":"ref19","author":"Thompson, C","year":"2009","journal-title":"My library My History Essential Decision Making and Clinical Judgement for Nurses"},{"key":"ref20","doi-asserted-by":"crossref","DOI":"10.17226\/21794","author":"Balogh, EP","year":"2015","journal-title":"Improving Diagnosis in Health Care"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0254088"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.h3480"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2020-040269"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1001\/jamainternmed.2016.6001"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1001\/jamaophthalmol.2019.0571"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.2196\/21299"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-022-00667-w"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.2186\/jpr.JPOR_2019_499"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.1177\/1024907919842486"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2021-053566"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.2196\/14679"},{"issue":"2","key":"ref32","first-page":"100","volume":"13","author":"Liu, VD","year":"2021","journal-title":"Finnish Journal of eHealth and eWelfare"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2020.543405"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.5694\/mja2.50600"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1080\/026404198366867"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.2196\/36322"},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.2196\/37408"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.2196\/24475"},{"key":"ref39","unstructured":"DigiFinland Oy2024-08-17https:\/\/digifinland.fi\/wp-content\/uploads\/2023\/07\/FI22_871876_Iss4_DigiFinland_Oy_EU-QMS_2023-05-31.pdf"},{"key":"ref40","doi-asserted-by":"publisher","DOI":"10.2196\/41423"},{"key":"ref41","unstructured":"Omaolo instructions for useOmaolo2024-08-17https:\/\/www.omaolo.fi\/kayttoohjeet\/omaolo-instructions.pdf"},{"key":"ref42","unstructured":"Omaolo symptom checker triage classificationsDigiFinland2024-08-17https:\/\/digifinland.fi\/omaolohelp\/verkkokoulutus\/2-oirearviot\/2-1-oirearvio\/prioriteetti-eli-kiireellisyys\/"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1136\/bmjqs-2013-002627"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.2196\/32340"},{"key":"ref45","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-021-01291-0"},{"key":"ref46","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0253664"},{"key":"ref47","unstructured":"Omaolo-palvelu laajenee t\u00e4m\u00e4n vuoden aikana kattamaan l\u00e4hes koko Suomen2024-08-17https:\/\/digifinland.fi\/omaolo-palvelu-laajenee-taman-vuoden-aikana-kattamaan-lahes-koko-suomen\/"},{"key":"ref48","doi-asserted-by":"publisher","DOI":"10.1177\/20552076231194929"},{"key":"ref49","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2022.1040926"},{"key":"ref50","doi-asserted-by":"publisher","DOI":"10.2196\/43803"}],"container-title":["JMIR Human Factors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/humanfactors.jmir.org\/2024\/1\/e55099","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:42:32Z","timestamp":1782841352000},"score":1,"resource":{"primary":{"URL":"https:\/\/humanfactors.jmir.org\/2024\/1\/e55099"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,26]]},"references-count":50,"URL":"https:\/\/doi.org\/10.2196\/55099","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.55099","asserted-by":"object"}]},"ISSN":["2292-9495"],"issn-type":[{"value":"2292-9495","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,26]]}}}