{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T05:57:52Z","timestamp":1772776672099,"version":"3.50.1"},"reference-count":54,"publisher":"BMJ","issue":"11","license":[{"start":{"date-parts":[[2020,11,12]],"date-time":"2020-11-12T00:00:00Z","timestamp":1605139200000},"content-version":"unspecified","delay-in-days":11,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Open"],"accepted":{"date-parts":[[2020,11,4]]},"published-print":{"date-parts":[[2020,11]]},"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>To develop items for an early warning score (RECAP: REmote COVID-19 Assessment in Primary Care) for patients with suspected COVID-19 who need escalation to next level of care.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>The study was based in UK primary healthcare. The mixed-methods design included rapid review, Delphi panel, interviews, focus groups and software development. Participants were 112 primary care clinicians and 50 patients recovered from COVID-19, recruited through social media, patient groups and snowballing. Using rapid literature review, we identified signs and symptoms which are commoner in severe COVID-19. Building a preliminary set of items from these, we ran four rounds of an online Delphi panel with 72 clinicians, the last incorporating fictional vignettes, collating data on R software. We refined the items iteratively in response to quantitative and qualitative feedback. Items in the penultimate round were checked against narrative interviews with 50 COVID-19 patients. We required, for each item, at least 80% clinician agreement on relevance, wording and cut-off values, and that the item addressed issues and concerns raised by patients. In focus groups, 40 clinicians suggested further refinements and discussed workability of the instrument in relation to local resources and care pathways. This informed design of an electronic template for primary care systems.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The prevalidation RECAP-V0 comprises a red flag alert box and 10 assessment items: pulse, shortness of breath or respiratory rate, trajectory of breathlessness, pulse oximeter reading (with brief exercise test if appropriate) or symptoms suggestive of hypoxia, temperature or fever symptoms, duration of symptoms, muscle aches, new confusion, shielded list and known risk factors for poor outcome. It is not yet known how sensitive or specific it is.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Items on RECAP-V0 align strongly with published evidence, clinical judgement and patient experience. The validation phase of this study is ongoing.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Trial registration number<\/jats:title>\n                    <jats:p>\n                      <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"clintrialgov\" xlink:href=\"NCT04435041\">NCT04435041<\/jats:ext-link>\n                      .\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.1136\/bmjopen-2020-042626","type":"journal-article","created":{"date-parts":[[2020,11,12]],"date-time":"2020-11-12T16:12:33Z","timestamp":1605197553000},"page":"e042626","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":43,"title":["What items should be included in an early warning score for remote assessment of suspected COVID-19? qualitative and Delphi study"],"prefix":"10.1136","volume":"10","clinical-trial-number":[{"clinical-trial-number":"nct04435041","registry":"10.18810\/clinical-trials-gov"}],"author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2369-8088","authenticated-orcid":false,"given":"Trisha","family":"Greenhalgh","sequence":"first","affiliation":[{"name":"Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK"}]},{"given":"Paul","family":"Thompson","sequence":"additional","affiliation":[{"name":"Department of Experimental Psychology, University of Oxford, Oxford, UK"}]},{"given":"Sietse","family":"Weiringa","sequence":"additional","affiliation":[{"name":"Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7107-7211","authenticated-orcid":false,"given":"Ana Luisa","family":"Neves","sequence":"additional","affiliation":[{"name":"Institute of Global Health Innovation, Imperial College London, London, UK"}]},{"given":"Laiba","family":"Husain","sequence":"additional","affiliation":[{"name":"Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK"}]},{"given":"Merlin","family":"Dunlop","sequence":"additional","affiliation":[{"name":"Ardens Ltd, Oxford, UK"}]},{"given":"Alexander","family":"Rushforth","sequence":"additional","affiliation":[{"name":"Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK"}]},{"given":"David","family":"Nunan","sequence":"additional","affiliation":[{"name":"Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5613-6810","authenticated-orcid":false,"given":"Simon","family":"de Lusignan","sequence":"additional","affiliation":[{"name":"Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK"}]},{"given":"Brendan","family":"Delaney","sequence":"additional","affiliation":[{"name":"Institute of Global Health Innovation, Imperial College London, London, UK"}]}],"member":"239","published-online":{"date-parts":[[2020,11,12]]},"reference":[{"key":"2025121513034043000_10.11.e042626.1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2020.2648"},{"key":"2025121513034043000_10.11.e042626.2","unstructured":"Cohen P , Blau J . Coronavirus disease 2019 (COVID-19): outpatient management in adults. up to date: Wolters Kluwer 2020, 2020. Available: https:\/\/www.uptodate.com\/contents\/coronavirus-disease-2019-covid-19-outpatient-management-in-adults"},{"key":"2025121513034043000_10.11.e042626.3","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m1464"},{"key":"2025121513034043000_10.11.e042626.4","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m1328"},{"key":"2025121513034043000_10.11.e042626.5","unstructured":"Nunan D , Brassey J , Mahtani K , et al . COVID-19 signs and symptoms Tracker. Oxford: centre for evidence-based medicine 2020, 2020. Available: https:\/\/www.cebm.net\/covid-19\/covid-19-signs-and-symptoms-tracker\/"},{"key":"2025121513034043000_10.11.e042626.6","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m1182"},{"key":"2025121513034043000_10.11.e042626.7","unstructured":"Royal College of General Practitioners . Using the National early warning score (NEWS2) in primary care. London: RCGP 2020, 2020. Available: https:\/\/elearning.rcgp.org.uk\/mod\/page\/view.php?id=10568 [Accessed 20th April 2020]."},{"key":"2025121513034043000_10.11.e042626.8","unstructured":"Royal College of Physicians (UK) . National early warning score (news) 2. London: RCP 2017, 2017. Available: https:\/\/www.rcplondon.ac.uk\/projects\/outputs\/national-early-warning-score-news-2"},{"key":"2025121513034043000_10.11.e042626.9","unstructured":"National Institute for Health and Clincial Excellence . COVID-19 rapid guideline: managing suspected or confirmed pneumonia in adults in the community. London: NICE 2020, 2020. Available: https:\/\/www.nice.org.uk\/guidance\/ng165"},{"key":"2025121513034043000_10.11.e042626.10","doi-asserted-by":"crossref","unstructured":"Carr E , Bendayan R , Bean D , et al . Supplementing the National early warning score (NEWS2) for anticipating early deterioration among patients with COVID-19 infection. medRxiv 2020.doi:https:\/\/doi.org\/10.1101\/2020.04.24.20078006","DOI":"10.1101\/2020.04.24.20078006"},{"key":"2025121513034043000_10.11.e042626.11","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.j4008"},{"key":"2025121513034043000_10.11.e042626.12","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0020476","article-title":"Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review","volume":"6","author":"Boulkedid","year":"2011","journal-title":"PLoS One"},{"key":"2025121513034043000_10.11.e042626.13","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1016\/j.jclinepi.2020.04.029","article-title":"Response changes in delphi processes: why is it important to provide high-quality feedback to Delphi participants?","volume":"125","author":"Khodyakov","year":"2020","journal-title":"J Clin Epidemiol"},{"key":"2025121513034043000_10.11.e042626.14","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.techfore.2019.07.002","article-title":"Improving the practical application of the Delphi method in group-based judgment: a six-step prescription for a well-founded and defensible process","volume":"147","author":"Belton","year":"2019","journal-title":"Technol Forecast Soc Change"},{"key":"2025121513034043000_10.11.e042626.15","doi-asserted-by":"publisher","DOI":"10.1111\/1467-9566.ep11347023"},{"key":"2025121513034043000_10.11.e042626.16","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1080\/13645579.1998.10846874","article-title":"Focus group methodology: a review","volume":"1","author":"Wilkinson","year":"1998","journal-title":"Int J Soc Res Methodol"},{"key":"2025121513034043000_10.11.e042626.17","doi-asserted-by":"crossref","unstructured":"Abrams KM , Gaiser TJ , Online focus groups . The SAGE handbook of online research methods. Thousand Oaks, California: Sage, 2017: 435\u201350.","DOI":"10.4135\/9781473957992.n25"},{"key":"2025121513034043000_10.11.e042626.18","doi-asserted-by":"crossref","unstructured":"Ritchie J , Spencer L . Qualitative data analysis for applied policy research. Analyzing qualitative data: Routledge, 2002: 187\u2013208.","DOI":"10.4324\/9780203413081-14"},{"key":"2025121513034043000_10.11.e042626.19","unstructured":"Greenhalgh T , Kotze K , Van Der Westhuizen H-M . Are there any evidence-based ways of assessing dyspnoea (breathlessness) by telephone or video? : Oxford COVID-19 Evidence Service Rapid Review., 2020. Available: https:\/\/www.cebm.net\/covid-19\/are-there-any-evidence-based-ways-of-assessing-dyspnoea-breathlessness-by-telephone-or-video\/"},{"key":"2025121513034043000_10.11.e042626.20","unstructured":"Greenhalgh T . Should the Roth score be used in the remote assessment of breathlessness? Oxford: Oxford COVID-19 evidence service, 2020. Available: https:\/\/www.cebm.net\/covid-19\/roth-score-not-recommended-to-assess-breathlessness-over-the-phone\/"},{"key":"2025121513034043000_10.11.e042626.21","unstructured":"Greenhalgh T , Javid B , Knight M , et al . What is the efficacy and safety of rapid exercise tests for exertional desaturation in covid-19? Oxford: Oxford COVID-19 evidence service, 2020. Available: https:\/\/www.cebm.net\/covid-19\/what-is-the-efficacy-and-safety-of-rapid-exercise-tests-for-exertional-desaturation-in-covid-19\/"},{"key":"2025121513034043000_10.11.e042626.22","unstructured":"Tarassenko L , Greenhalgh T . Should smartphone apps be used as oximeters? Oxford: Oxford COVID-19 evidence service, 2020. Available: https:\/\/www.cebm.net\/covid-19\/question-should-smartphone-apps-be-used-as-oximeters-answer-no\/\/"},{"key":"2025121513034043000_10.11.e042626.23","unstructured":"Tarassenko L , McManus R , Greenhalgh T . Is the \u2018Lifelight\u2019 app adequately validated for blood pressure measurement? Oxford: Oxford COVID-19 evidence service, 2020. Available: https:\/\/www.cebm.net\/covid-19\/question-is-the-lifelight-app-adequately-validated-for-blood-pressure-measurement-answer-no\/"},{"key":"2025121513034043000_10.11.e042626.24","doi-asserted-by":"crossref","first-page":"636","DOI":"10.1002\/clc.22586","article-title":"Assessment of respiratory distress by the Roth score","volume":"39","author":"Chorin","year":"2016","journal-title":"Clin Cardiol"},{"key":"2025121513034043000_10.11.e042626.25","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa2002032"},{"key":"2025121513034043000_10.11.e042626.26","doi-asserted-by":"crossref","first-page":"1037","DOI":"10.1038\/s41591-020-0916-2","article-title":"Real-Time tracking of self-reported symptoms to predict potential COVID-19","volume":"26","author":"Menni","year":"2020","journal-title":"Nat Med"},{"key":"2025121513034043000_10.11.e042626.27","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m1501"},{"key":"2025121513034043000_10.11.e042626.28","doi-asserted-by":"crossref","unstructured":"Inada-Kim M . NEWS 2: An opportunity to standardise the management of deterioration and sepsis BMJ (blog) 2018, 2018. Available: http:\/\/blogs.bmj.com\/bmj\/2018\/02\/08\/news-2-an-opportunity-to-standardise-the-management-of-deterioration-and-sepsis\/","DOI":"10.1136\/bmj.k1260"},{"key":"2025121513034043000_10.11.e042626.29","unstructured":"National Institute for Health and Clincial Excellence . National early warning score systems that alert to deteriorating adult patients in hospital (Medtech innovation briefing 205). London: NICE, 2020."},{"key":"2025121513034043000_10.11.e042626.30","unstructured":"National Institute for Health and Clincial Excellence . COVID-19 rapid guideline: critical care in adults. London: NICE, 2020. Available: https:\/\/www.nice.org.uk\/guidance\/ng159\/resources\/covid19-rapid-guideline-critical-care-pdf-66141848681413"},{"key":"2025121513034043000_10.11.e042626.31","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.resuscitation.2018.08.028","article-title":"Can early warning scores identify deteriorating patients in pre-hospital settings? A systematic review","volume":"132","author":"Patel","year":"2018","journal-title":"Resuscitation"},{"key":"2025121513034043000_10.11.e042626.32","doi-asserted-by":"crossref","first-page":"705","DOI":"10.12968\/bjon.2018.27.12.705","article-title":"Limitations of track and trigger systems and the National early warning score. Part 2: sensitivity versus specificity","volume":"27","author":"Grant","year":"2018","journal-title":"Br J Nurs"},{"key":"2025121513034043000_10.11.e042626.33","doi-asserted-by":"crossref","first-page":"234","DOI":"10.12968\/bjon.2019.28.4.234","article-title":"Limitations of track and trigger systems and the National early warning score. Part 3: cultural and behavioural factors","volume":"28","author":"Grant","year":"2019","journal-title":"Br J Nurs"},{"key":"2025121513034043000_10.11.e042626.34","doi-asserted-by":"crossref","first-page":"624","DOI":"10.12968\/bjon.2018.27.11.624","article-title":"Limitations of track and trigger systems and the National early warning score. Part 1: areas of contention","volume":"27","author":"Grant","year":"2018","journal-title":"Br J Nurs"},{"key":"2025121513034043000_10.11.e042626.35","doi-asserted-by":"publisher","DOI":"10.3399\/bjgp20X709349"},{"key":"2025121513034043000_10.11.e042626.36","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.l6226"},{"key":"2025121513034043000_10.11.e042626.37","doi-asserted-by":"publisher","DOI":"10.1136\/emermed-2018-208140"},{"key":"2025121513034043000_10.11.e042626.38","doi-asserted-by":"crossref","unstructured":"Stow D , Barker RO , Matthews FE , et al . National early warning scores (NEWS\/NEWS2) and COVID-19 deaths in care homes: a longitudinal ecological study. medRxiv 2020.doi:https:\/\/doi.org\/10.1101\/2020.06.15.20131516","DOI":"10.1101\/2020.06.15.20131516"},{"key":"2025121513034043000_10.11.e042626.39","doi-asserted-by":"crossref","DOI":"10.1111\/acem.14013","article-title":"Defining high-risk emergency chief complaints: data-driven triage for low- and middle-income countries","author":"Rice","year":"2020","journal-title":"Acad Emerg Med"},{"key":"2025121513034043000_10.11.e042626.40","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m3339"},{"key":"2025121513034043000_10.11.e042626.41","doi-asserted-by":"crossref","unstructured":"Gupta RK , Harrison EM , Ho A , et al . Development and validation of the 4C deterioration model for adults hospitalised with COVID-19. medRxiv 2020.doi:https:\/\/doi.org\/10.1101\/2020.10.09.20209957","DOI":"10.1101\/2020.10.09.20209957"},{"key":"2025121513034043000_10.11.e042626.42","doi-asserted-by":"publisher","DOI":"10.1002\/bjs.9736"},{"key":"2025121513034043000_10.11.e042626.43","unstructured":"Delaney B . Recap project. London: Imperial College London, 2020. Available: https:\/\/imperialbrc.nihr.ac.uk\/research\/covid-19\/covid-19-ongoing-studies\/recap\/"},{"key":"2025121513034043000_10.11.e042626.44","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m1091"},{"key":"2025121513034043000_10.11.e042626.45","doi-asserted-by":"crossref","first-page":"1099","DOI":"10.1007\/s00134-020-06033-2","article-title":"COVID-19 pneumonia: different respiratory treatments for different phenotypes?","volume":"46","author":"Gattinoni","year":"2020","journal-title":"Intensive Care Med"},{"key":"2025121513034043000_10.11.e042626.46","article-title":"COVID-19 with silent hypoxemia","volume":"140","author":"Ottestad","year":"2020","journal-title":"Tidsskr Nor Laegeforen"},{"key":"2025121513034043000_10.11.e042626.47","doi-asserted-by":"crossref","unstructured":"Wilkerson RG , Adler JD , Shah NG , et al . Silent hypoxia: a harbinger of clinical deterioration in patients with COVID-19. Am J Emerg Med 2020.doi:10.1016\/j.ajem.2020.05.044","DOI":"10.1016\/j.ajem.2020.05.044"},{"key":"2025121513034043000_10.11.e042626.48","doi-asserted-by":"crossref","DOI":"10.4103\/jgid.jgid_96_20","article-title":"State of the globe: The trials and tribulations of the COVID-19 pandemic: Separated but together, telemedicine revolution, frontline struggle against \u201cSilent Hypoxia,\u201d the relentless search for novel therapeutics and vaccines, and the daunting prospect of \u201cCOVIFLU\u201d","volume":"12","author":"Chauhan","year":"2020","journal-title":"J Glob Infect Dis"},{"key":"2025121513034043000_10.11.e042626.49","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1007\/s00134-020-05979-7","article-title":"Critical care crisis and some recommendations during the COVID-19 epidemic in China","volume":"46","author":"Xie","year":"2020","journal-title":"Intensive Care Med"},{"key":"2025121513034043000_10.11.e042626.50","doi-asserted-by":"publisher","DOI":"10.1007\/s10072-020-04486-3"},{"key":"2025121513034043000_10.11.e042626.51","doi-asserted-by":"crossref","first-page":"1049","DOI":"10.1016\/S0140-6736(88)90066-9","article-title":"Severe exercise hypoxaemia with normal or near normal X-rays: a feature of Pneumocystis carinii infection","volume":"332","author":"Smith","year":"1988","journal-title":"The Lancet"},{"key":"2025121513034043000_10.11.e042626.52","doi-asserted-by":"publisher","DOI":"10.1016\/j.jinf.2020.04.021"},{"key":"2025121513034043000_10.11.e042626.53","unstructured":"Public Health England . Disparities in the risk and outcomes of COVID-19. London: PHE publications. Gateway number: GW-1311, 2020. Available: https:\/\/assets.publishing.service.gov.uk\/government\/uploads\/system\/uploads\/attachment_data\/file\/890258\/disparities_review.pdf 2020:354-360"},{"key":"2025121513034043000_10.11.e042626.54","doi-asserted-by":"crossref","first-page":"1034","DOI":"10.1016\/S1473-3099(20)30371-6","article-title":"Risk factors for SARS-CoV-2 among patients in the Oxford Royal College of general practitioners research and surveillance centre primary care network: a cross-sectional study","volume":"20","author":"de Lusignan","year":"2020","journal-title":"Lancet Infect Dis"}],"container-title":["BMJ Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjopen-2020-042626","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,15]],"date-time":"2025-12-15T21:03:53Z","timestamp":1765832633000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmjopen.bmj.com\/lookup\/doi\/10.1136\/bmjopen-2020-042626"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11]]},"references-count":54,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2020,11,12]]},"published-print":{"date-parts":[[2020,11]]}},"alternative-id":["10.1136\/bmjopen-2020-042626"],"URL":"https:\/\/doi.org\/10.1136\/bmjopen-2020-042626","relation":{},"ISSN":["2044-6055","2044-6055"],"issn-type":[{"value":"2044-6055","type":"print"},{"value":"2044-6055","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11]]}}}