{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T01:33:49Z","timestamp":1780623229828,"version":"3.54.1"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,2,3]],"date-time":"2025-02-03T00:00:00Z","timestamp":1738540800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,2,3]],"date-time":"2025-02-03T00:00:00Z","timestamp":1738540800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Background<\/jats:title>\n            <jats:p>England\u2019s urgent care telephone triage system comprises non-clinician-led primary triage (NHS111) assessment followed, for approximately 50% patients, by clinician-led secondary triage. Digital decision support is utilised by both. We explore the system\u2019s safety and accuracy relative to patients\u2019 use of emergency departments (EDs) and in-patient care in the subsequent 24\u00a0h.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>Descriptive analyses were used to investigate outcomes of 98,946 calls that underwent primary and secondary triage. We investigated sensitivity (safety) and specificity (efficiency\/accuracy) in relation to subsequent ED attendance and in-patient hospital admission. Mixed effects regression models were used to explore potential under-estimation of clinical risk (under-triage).<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>Sensitivity was greater in primary triage, whilst specificity was greater in secondary triage. The positive predictive value for attending ED after being assigned a triage urgency level of within 2\u00a0h was 46.0% for secondary triage compared to 20.7% for primary triage; for inpatient admission it was 18.0% and 9.2% respectively. 1.5% (<jats:italic>n<\/jats:italic>\u2009=\u20091468) patients triaged to same-day or less urgent care at secondary triage were subsequently admitted for in-patient care. In relation to in-patient admission within 24\u00a0h, there were greater odds of potential under-triage for calls made between midnight and 6am, and for shorter duration calls, respectively OR\u2009=\u20091.71; CI:1.32\u20132.21 and OR: 1.66, CI: 1.30\u20132.11. The service provider (e.g., service provider 2, OR\u2009=\u20095.61; CI:3.36\u20139.36) and individual clinician (OR covering the 95% midrange\u2009=\u200916.15) conducting triage were the characteristics most greatly associated with this potential under-triage; <jats:italic>p<\/jats:italic>\u2009&lt;\u20090.001 for all.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusions<\/jats:title>\n            <jats:p>Clinician-led urgent care triage is more accurate in identifying the likelihood of a need for ED or in-patient care than non-clinician triage. Non-clinician primary triage is risk averse, reflected in its high sensitivity but low specificity. Service and clinician characteristics associated with potential under-triage need further investigation to inform ways of improving the safety and effectiveness of urgent care telephone triage.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Clinical trial number<\/jats:title>\n            <jats:p>Not applicable.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1186\/s12911-025-02888-x","type":"journal-article","created":{"date-parts":[[2025,2,3]],"date-time":"2025-02-03T13:15:56Z","timestamp":1738588556000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Safety and accuracy of digitally supported primary and secondary urgent care telephone triage in England: an observational study using routine data"],"prefix":"10.1186","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6935-016X","authenticated-orcid":false,"given":"Vanashree","family":"Sexton","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Catherine","family":"Grimley","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9256-3553","authenticated-orcid":false,"given":"Jeremy","family":"Dale","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7072-1925","authenticated-orcid":false,"given":"Helen","family":"Atherton","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2231-5161","authenticated-orcid":false,"given":"Gary","family":"Abel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,2,3]]},"reference":[{"key":"2888_CR1","doi-asserted-by":"crossref","unstructured":"Bunn F, Byrne G, Kendall S. Telephone consultation and triage: effects on health care use and patient satisfaction. Cochrane Database Syst Reviews. 2004(3).","DOI":"10.1002\/14651858.CD004180.pub2"},{"issue":"1","key":"2888_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.healthpol.2018.11.001","volume":"123","author":"N Baier","year":"2019","unstructured":"Baier N, Geissler A, Bech M, Bernstein D, Cowling TE, Jackson T, et al. Emergency and urgent care systems in Australia, Denmark, England, France, Germany and the Netherlands \u2013 analyzing organization, payment and reforms. Health Policy. 2019;123(1):1\u201310.","journal-title":"Health Policy"},{"key":"2888_CR3","first-page":"313","volume":"45","author":"R McKenzie","year":"2016","unstructured":"McKenzie R, Williamson M, Roberts R. Who uses the \u2018after hours GP Helpline\u2019? A profile of users of an after-hours primary care helpline. Aus Fam Physician. 2016;45:313\u20138.","journal-title":"Aus Fam Physician"},{"issue":"1","key":"2888_CR4","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1089\/tmj.2010.0120","volume":"17","author":"F North","year":"2011","unstructured":"North F, Varkey P, Laing B, Cha SS, Tulledge-Scheitel S. Are e-health web users looking for different symptom information than callers to triage centers? Telemedicine J E Health: Official J Am Telemedicine Association. 2011;17(1):19\u201324.","journal-title":"Telemedicine J E Health: Official J Am Telemedicine Association"},{"issue":"1","key":"2888_CR5","doi-asserted-by":"publisher","first-page":"e051569","DOI":"10.1136\/bmjopen-2021-051569","volume":"12","author":"V Sexton","year":"2022","unstructured":"Sexton V, Dale J, Bryce C, Barry J, Sellers E, Atherton H. Service use, clinical outcomes and user experience associated with urgent care services that use telephone-based digital triage: a systematic review. BMJ Open. 2022;12(1):e051569.","journal-title":"BMJ Open"},{"issue":"3\u20134","key":"2888_CR6","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1080\/13698575.2017.1324946","volume":"19","author":"J Turnbull","year":"2017","unstructured":"Turnbull J, Prichard J, Pope C, Brook S, Rowsell A. Risk work in NHS 111: the everyday work of managing risk in telephone assessment using a computer decision support system. Health Risk Soc. 2017;19(3\u20134):189\u2013208.","journal-title":"Health Risk Soc"},{"issue":"5","key":"2888_CR7","doi-asserted-by":"publisher","first-page":"e0251362","DOI":"10.1371\/journal.pone.0251362","volume":"16","author":"J Lewis","year":"2021","unstructured":"Lewis J, Stone T, Simpson R, Jacques R, O\u2019Keeffe C, Croft S, Mason S. Patient compliance with NHS 111 advice: analysis of adult call and ED attendance data 2013\u20132017. PLoS ONE. 2021;16(5):e0251362\u2013e.","journal-title":"PLoS ONE"},{"issue":"11","key":"2888_CR8","doi-asserted-by":"publisher","first-page":"e009444","DOI":"10.1136\/bmjopen-2015-009444","volume":"5","author":"A Anderson","year":"2015","unstructured":"Anderson A, Roland M. Potential for advice from doctors to reduce the number of patients referred to emergency departments by NHS 111 call handlers: observational study. BMJ Open. 2015;5(11):e009444.","journal-title":"BMJ Open"},{"issue":"6","key":"2888_CR9","doi-asserted-by":"publisher","first-page":"227","DOI":"10.12968\/jpar.2020.12.6.227","volume":"12","author":"JS Phillips","year":"2020","unstructured":"Phillips JS. Paramedics\u2019 perceptions and experiences of NHS 111 in the south west of England. J Paramedic Pract. 2020;12(6):227\u201334.","journal-title":"J Paramedic Pract"},{"issue":"11","key":"2888_CR10","doi-asserted-by":"publisher","first-page":"e003451","DOI":"10.1136\/bmjopen-2013-003451","volume":"3","author":"J Turner","year":"2013","unstructured":"Turner J, O\u2019Cathain A, Knowles E, Nicholl J. Impact of the urgent care telephone service NHS 111 pilot sites: a controlled before and after study. BMJ Open. 2013;3(11):e003451.","journal-title":"BMJ Open"},{"key":"2888_CR11","unstructured":"England NHS. NHS 111 Minimum Data Set 2018\u201319."},{"key":"2888_CR12","unstructured":"NHS-Digital. NHS Pathways [Available from: https:\/\/digital.nhs.uk\/services\/nhs-pathways"},{"issue":"731","key":"2888_CR13","doi-asserted-by":"publisher","first-page":"e427","DOI":"10.3399\/BJGP.2022.0374","volume":"73","author":"V Sexton","year":"2023","unstructured":"Sexton V, Atherton H, Dale J, Abel G. Clinician-led secondary triage in England\u2019s urgent care delivery: a cross-sectional study. Br J Gen Pract. 2023;73(731):e427.","journal-title":"Br J Gen Pract"},{"issue":"2","key":"2888_CR14","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1016\/S2155-8256(17)30097-2","volume":"8","author":"PI Buerhaus","year":"2017","unstructured":"Buerhaus PI, Skinner LE, Auerbach DI, Staiger DO. Four challenges facing the nursing workforce in the United States. J Nurs Regul. 2017;8(2):40\u20136.","journal-title":"J Nurs Regul"},{"issue":"4","key":"2888_CR15","doi-asserted-by":"publisher","first-page":"703","DOI":"10.1111\/j.1748-720X.2008.00325.x","volume":"36","author":"F Mullan","year":"2008","unstructured":"Mullan F, Frehywot S, Jolley LJ, Aging. Primary care, and Self-Sufficiency: Health Care Workforce challenges ahead. J Law Med Ethics. 2008;36(4):703\u20138.","journal-title":"J Law Med Ethics"},{"key":"2888_CR16","unstructured":"NHS-Digital. Emergency Care Data Set (ECDS) [Available from: https:\/\/digital.nhs.uk\/data-and-information\/data-collections-and-data-sets\/data-sets\/emergency-care-data-set-ecds"},{"key":"2888_CR17","doi-asserted-by":"crossref","unstructured":"Abel G, Elliott MN. Identifying and quantifying variation between healthcare organisations and geographical regions: using mixed-effects models. BMJ Quality & Safety. 2019;28(12):1032.","DOI":"10.1136\/bmjqs-2018-009165"},{"issue":"2","key":"2888_CR18","doi-asserted-by":"publisher","first-page":"e0170811","DOI":"10.1371\/journal.pone.0170811","volume":"12","author":"JM Zachariasse","year":"2017","unstructured":"Zachariasse JM, Seiger N, Rood PP, Alves CF, Freitas P, Smit FJ, et al. Validity of the Manchester Triage System in emergency care: a prospective observational study. PLoS ONE [Electronic Resource]. 2017;12(2):e0170811.","journal-title":"PLoS ONE [Electronic Resource]"},{"issue":"5","key":"2888_CR19","doi-asserted-by":"publisher","first-page":"e026471","DOI":"10.1136\/bmjopen-2018-026471","volume":"9","author":"JM Zachariasse","year":"2019","unstructured":"Zachariasse JM, van der Hagen V, Seiger N, Mackway-Jones K, van Veen M, Moll HA. Performance of triage systems in emergency care: a systematic review and meta-analysis. BMJ Open. 2019;9(5):e026471.","journal-title":"BMJ Open"},{"issue":"3","key":"2888_CR20","doi-asserted-by":"publisher","first-page":"189","DOI":"10.7326\/0003-4819-140-3-200402030-00010","volume":"140","author":"P Whiting","year":"2004","unstructured":"Whiting P, Rutjes AW, Reitsma JB, Glas AS, Bossuyt PM, Kleijnen J. Sources of variation and bias in studies of diagnostic accuracy: a systematic review. Ann Intern Med. 2004;140(3):189\u2013202.","journal-title":"Ann Intern Med"},{"issue":"1","key":"2888_CR21","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1186\/s12873-018-0215-0","volume":"18","author":"HL Tam","year":"2018","unstructured":"Tam HL, Chung SF, Lou CK. A review of triage accuracy and future direction. BMC Emerg Med. 2018;18(1):58.","journal-title":"BMC Emerg Med"},{"issue":"2","key":"2888_CR22","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1080\/17843286.2020.1839719","volume":"77","author":"S Morreel","year":"2022","unstructured":"Morreel S, Colliers A, Remmen R, Verhoeven V, Philips H. How accurate is telephone triage in out-of-hours care? An observational trial in real patients. Acta Clin Belg. 2022;77(2):301\u20136.","journal-title":"Acta Clin Belg"},{"issue":"1","key":"2888_CR23","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1186\/s12875-020-01122-z","volume":"21","author":"DS Graversen","year":"2020","unstructured":"Graversen DS, Christensen MB, Pedersen AF, Carlsen AH, Bro F, Christensen HC, et al. Safety, efficiency and health-related quality of telephone triage conducted by general practitioners, nurses, or physicians in out-of-hours primary care: a quasi-experimental study using the Assessment of Quality in Telephone Triage (AQTT) to assess audio-recorded telephone calls. BMC Fam Pract. 2020;21(1):84.","journal-title":"BMC Fam Pract"},{"key":"2888_CR24","doi-asserted-by":"crossref","unstructured":"Dickson SJ, Dewar C, Richardson A, Hunter A, Searle S, Hodgson LE. Agreement and validity of electronic patient self-triage (eTriage) with nurse triage in two UK emergency departments: a retrospective study. Eur J Emerg Med. 2022;29(1).","DOI":"10.1097\/MEJ.0000000000000863"},{"issue":"1","key":"2888_CR25","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1186\/s12875-020-01334-3","volume":"21","author":"DC Erkelens","year":"2020","unstructured":"Erkelens DC, Rutten FH, Wouters LT, Dolmans LS, de Groot E, Damoiseaux RA, Zwart DL. Accuracy of telephone triage in patients suspected of transient ischaemic attack or stroke: a cross-sectional study. BMC Fam Pract. 2020;21(1):256.","journal-title":"BMC Fam Pract"},{"issue":"4","key":"2888_CR26","doi-asserted-by":"publisher","first-page":"747","DOI":"10.1093\/fampra\/cmab115","volume":"39","author":"S Matenge","year":"2022","unstructured":"Matenge S, Sturgiss E, Desborough J, Hall Dykgraaf S, Dut G, Kidd M. Ensuring the continuation of routine primary care during the COVID-19 pandemic: a review of the international literature. Fam Pract. 2022;39(4):747\u201361.","journal-title":"Fam Pract"},{"issue":"724","key":"2888_CR27","doi-asserted-by":"publisher","first-page":"e799","DOI":"10.3399\/BJGP.2021.0703","volume":"72","author":"L Huibers","year":"2022","unstructured":"Huibers L, Bech BH, Kirk UB, Kallestrup P, Vestergaard CH, Christensen MB. Contacts in general practice during the COVID-19 pandemic: a register-based study. Br J Gen Pract. 2022;72(724):e799\u2013808.","journal-title":"Br J Gen Pract"},{"key":"2888_CR28","unstructured":"Use NHS. 111 online service to protect busy A&Es, says health chief: The Guardian; [Available from: https:\/\/www.theguardian.com\/world\/2021\/nov\/01\/use-nhs-111-online-service-to-protect-busy-aes-says-health-chief"},{"issue":"7\u20138","key":"2888_CR29","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1177\/08982643211002084","volume":"33","author":"P Lu","year":"2021","unstructured":"Lu P, Kong D, Shelley M, Risk, Perception. Preventive behavior, and Medical Care Avoidance among American older adults during the COVID-19 pandemic. J Aging Health. 2021;33(7\u20138):577\u201384.","journal-title":"J Aging Health"},{"key":"2888_CR30","doi-asserted-by":"crossref","unstructured":"Nicholson E, McDonnell T, Conlon C, Barrett M, Cummins F, Hensey C, McAuliffe E. Parental hesitancy and concerns around accessing paediatric unscheduled Healthcare during COVID-19: a cross-sectional survey. Int J Environ Res Public Health. 2020;17(24).","DOI":"10.3390\/ijerph17249264"},{"issue":"678","key":"2888_CR31","doi-asserted-by":"publisher","first-page":"e70","DOI":"10.3399\/bjgp18X700193","volume":"69","author":"D Jones","year":"2019","unstructured":"Jones D, Dunn L, Watt I, Macleod U. Safety netting for primary care: evidence from a literature review. Br J Gen Pract. 2019;69(678):e70.","journal-title":"Br J Gen Pract"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-025-02888-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-025-02888-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-025-02888-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,3]],"date-time":"2025-02-03T13:16:07Z","timestamp":1738588567000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-025-02888-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,3]]},"references-count":31,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["2888"],"URL":"https:\/\/doi.org\/10.1186\/s12911-025-02888-x","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,3]]},"assertion":[{"value":"23 August 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 January 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 February 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study was approved by the London - Camden & Kings Cross Research Ethics Committee (REC reference:21\/LO\/0184) and NHS confidentiality advisory group (CAG) for data linkage involving the use of identifiable data without consent.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"In addition to his university role, Jeremy Dale is Director of the Clinical Knowledge Unit at Advanced Health and Care. Jeremy Dale played no part in the retrieval or analysis of the data. The other authors have declared no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"52"}}