{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,27]],"date-time":"2026-01-27T10:12:46Z","timestamp":1769508766254,"version":"3.49.0"},"reference-count":26,"publisher":"BMJ","issue":"12","license":[{"start":{"date-parts":[[2021,12,5]],"date-time":"2021-12-05T00:00:00Z","timestamp":1638662400000},"content-version":"unspecified","delay-in-days":4,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Open"],"accepted":{"date-parts":[[2021,11,5]]},"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>High-quality data are crucial for guiding decision-making and practising evidence-based healthcare, especially if previous knowledge is lacking. Nevertheless, data quality frailties have been exposed worldwide during the current COVID-19 pandemic. Focusing on a major Portuguese epidemiological surveillance dataset, our study aims to assess COVID-19 data quality issues and suggest possible solutions.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Settings<\/jats:title>\n                  <jats:p>On 27 April 2020, the Portuguese Directorate-General of Health (DGS) made available a dataset (DGSApril) for researchers, upon request. On 4 August, an updated dataset (DGSAugust) was also obtained.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Participants<\/jats:title>\n                  <jats:p>All COVID-19-confirmed cases notified through the medical component of National System for Epidemiological Surveillance until end of June.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Primary and secondary outcome measures<\/jats:title>\n                  <jats:p>Data completeness and consistency.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>DGSAugust has not followed the data format and variables as DGSApril and a significant number of missing data and inconsistencies were found (eg, 4075 cases from the DGSApril were apparently not included in DGSAugust). Several variables also showed a low degree of completeness and\/or changed their values from one dataset to another (eg, the variable \u2018underlying conditions\u2019 had more than half of cases showing different information between datasets). There were also significant inconsistencies between the number of cases and deaths due to COVID-19 shown in DGSAugust and by the DGS reports publicly provided daily.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Important quality issues of the Portuguese COVID-19 surveillance datasets were described. These issues can limit surveillance data usability to inform good decisions and perform useful research. Major improvements in surveillance datasets are therefore urgently needed\u2014for example, simplification of data entry processes, constant monitoring of data, and increased training and awareness of healthcare providers\u2014as low data quality may lead to a deficient pandemic control.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1136\/bmjopen-2020-047623","type":"journal-article","created":{"date-parts":[[2022,10,25]],"date-time":"2022-10-25T15:11:03Z","timestamp":1666710663000},"page":"e047623","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":25,"title":["COVID-19 surveillance data quality issues: a national consecutive case series"],"prefix":"10.1136","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7109-1101","authenticated-orcid":false,"given":"Cristina","family":"Costa-Santos","sequence":"first","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7107-7211","authenticated-orcid":false,"given":"Ana Luisa","family":"Neves","sequence":"additional","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Patient Safety Translational Research Centre, Institute of Global Health Innovation, Imperial College London, London, UK"}]},{"given":"Ricardo","family":"Correia","sequence":"additional","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2362-5527","authenticated-orcid":false,"given":"Paulo","family":"Santos","sequence":"additional","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4586-2910","authenticated-orcid":false,"given":"Matilde","family":"Monteiro-Soares","sequence":"additional","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Escola Superior de sa\u00fade da Cruz Vermelha Portuguesa, Lisbon, Portugal"}]},{"given":"Alberto","family":"Freitas","sequence":"additional","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"}]},{"given":"Ines","family":"Ribeiro-Vaz","sequence":"additional","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Porto Pharmacovigilance Centre, Faculty of Medicine, University of Porto, 4200-450 Porto, Portugal"}]},{"given":"Teresa S","family":"Henriques","sequence":"additional","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"}]},{"given":"Pedro","family":"Pereira Rodrigues","sequence":"additional","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"}]},{"given":"Altamiro","family":"Costa-Pereira","sequence":"additional","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"}]},{"given":"Ana Margarida","family":"Pereira","sequence":"additional","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"}]},{"given":"Joao A","family":"Fonseca","sequence":"additional","affiliation":[{"name":"Department of Community Medicine, Information and Health Decision Sciences (MEDCIDS), Faculty of Medicine, University of Porto, Porto, Portugal"},{"name":"Centre for Health Technology and Services Research (CINTESIS), Faculty of Medicine, University of Porto, Porto, Portugal"}]}],"member":"239","published-online":{"date-parts":[[2021,12,6]]},"reference":[{"key":"2025101811415951000_11.12.e047623.1","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2018.0365"},{"key":"2025101811415951000_11.12.e047623.2","doi-asserted-by":"publisher","DOI":"10.1016\/S1473-3099(20)30119-5"},{"key":"2025101811415951000_11.12.e047623.3","doi-asserted-by":"publisher","DOI":"10.1038\/518477a"},{"key":"2025101811415951000_11.12.e047623.4","first-page":"1-35; quiz CE1-7","article-title":"Updated guidelines for evaluating public health surveillance systems: recommendations from the guidelines Working group","volume":"50","author":"German","year":"2001","journal-title":"MMWR Recomm Rep"},{"key":"2025101811415951000_11.12.e047623.5","first-page":"28-37","article-title":"Health records as the basis of clinical coding: is the quality adequate? A qualitative study of medical coders' perceptions","volume":"49","author":"Alonso","year":"2020","journal-title":"Health Inf Manag"},{"key":"2025101811415951000_11.12.e047623.6","doi-asserted-by":"publisher","DOI":"10.3390\/ijerph110505170"},{"key":"2025101811415951000_11.12.e047623.7","doi-asserted-by":"crossref","first-page":"291","DOI":"10.3233\/SJI-200674","article-title":"A study on the quality of novel coronavirus (COVID-19) official datasets","volume":"36","author":"Ashofteh","year":"2020","journal-title":"Stat J IAOS"},{"key":"2025101811415951000_11.12.e047623.8","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/j.puhe.2019.11.012","article-title":"Evaluation of the German surveillance system for hepatitis B regarding timeliness, data quality, and simplicity, from 2005 to 2014","volume":"180","author":"Boes","year":"2020","journal-title":"Public Health"},{"key":"2025101811415951000_11.12.e047623.9","doi-asserted-by":"publisher","DOI":"10.1016\/j.puhe.2015.02.015"},{"key":"2025101811415951000_11.12.e047623.10","doi-asserted-by":"crossref","DOI":"10.1186\/s12889-019-7035-3","article-title":"Evaluation of the influenza-like illness surveillance system in Tunisia, 2012-2015","volume":"19","author":"Yazidi","year":"2019","journal-title":"BMC Public Health"},{"key":"2025101811415951000_11.12.e047623.11","doi-asserted-by":"crossref","first-page":"15.","DOI":"10.1186\/s40249-020-0629-2","article-title":"Evaluation of malaria surveillance system in Kano state, Nigeria, 2013-2016","volume":"9","author":"Visa","year":"2020","journal-title":"Infect Dis Poverty"},{"key":"2025101811415951000_11.12.e047623.12","doi-asserted-by":"crossref","DOI":"10.1186\/s12874-020-00972-6","article-title":"Methodological challenges of analysing COVID-19 data during the pandemic","volume":"20","author":"Wolkewitz","year":"2020","journal-title":"BMC Med Res Methodol"},{"key":"2025101811415951000_11.12.e047623.13","unstructured":"Dire\u00e7\u00e3o Geral da Sa\u00fade . Comunicado: Casos de infe\u00e7\u00e3o POR novo Coronav\u00edrus (COVID-19), 2020. Available: https:\/\/covid19.min-saude.pt\/wp-content\/uploads\/2020\/03\/Atualiza%C3%A7%C3%A3o-de-02032020-1728.pdf [Accessed 17 Aug 2020]."},{"key":"2025101811415951000_11.12.e047623.14","unstructured":"Carta aberta AO Conselho Nacional de Sa\u00fade P\u00fablica: Um contributo pessoal acerca dA epidemia de Covid-19, em Portugal, 2020. Available: https:\/\/sigarra.up.pt\/fmup\/pt\/noticias_geral.noticias_cont?p_id=F307210300\/CartaAberta_COVID19_11.03.2020_.pdf [Accessed 17 Aug 2020]."},{"key":"2025101811415951000_11.12.e047623.15","unstructured":"Dire\u00e7\u00e3o Geral da Sa\u00fade . COVID-19: Disponibiliza\u00e7\u00e3o de Dados, 2020. Available: https:\/\/covid19.min-saude.pt\/disponibilizacao-de-dados\/ [Accessed 11 Aug 2020]."},{"key":"2025101811415951000_11.12.e047623.16","unstructured":"Dire\u00e7\u00e3o Geral da Sa\u00fade . COVID metadata, 2020. Available: https:\/\/covid19.min-saude.pt\/wp-content\/uploads\/2020\/04\/PT_COVID19_metadata-1.pdf [Accessed 11 Aug 2020]."},{"key":"2025101811415951000_11.12.e047623.17","unstructured":"Dire\u00e7\u00e3o Geral da Sa\u00fade . Relat\u00f3rio de Situa\u00e7\u00e3o - Informa\u00e7\u00e3o publicada diariamente, 2020. Available: https:\/\/covid19.min-saude.pt\/relatorio-de-situacao\/ [Accessed 11 Aug 2020]."},{"key":"2025101811415951000_11.12.e047623.18","doi-asserted-by":"crossref","DOI":"10.3390\/jcm9113442","article-title":"The hidden factor-low quality of data is a major peril in the identification of risk factors for COVID-19 deaths: a comment on Nogueira, P.J., et al. \"The role of health preconditions on COVID-19 deaths in Portugal: evidence from surveillance data of the first 20293 infection cases\". J. Clin. Med. 2020, 9, 2368","volume":"9","author":"Costa-Santos","year":"2020","journal-title":"J Clin Med"},{"key":"2025101811415951000_11.12.e047623.19","doi-asserted-by":"crossref","first-page":"2368","DOI":"10.3390\/jcm9082368","article-title":"The role of health preconditions on COVID-19 deaths in Portugal: evidence from surveillance data of the first 20293 infection cases","volume":"9","author":"Nogueira","year":"2020","journal-title":"J Clin Med"},{"key":"2025101811415951000_11.12.e047623.20","doi-asserted-by":"publisher","DOI":"10.1186\/2045-7022-2-15"},{"key":"2025101811415951000_11.12.e047623.21","doi-asserted-by":"crossref","first-page":"30","DOI":"10.20344\/amp.11227","article-title":"[Multimorbidity in Portugal: Results from The First National Health Examination Survey]","volume":"32","author":"Quinaz Romana","year":"2019","journal-title":"Acta Med Port"},{"key":"2025101811415951000_11.12.e047623.22","doi-asserted-by":"crossref","unstructured":"Peixoto R , Viera V; , Aguar A; . COVID-19: determinants of hospitalization, ICU and death among 20,293 reported cases in Portugal. medRxiv 2020.doi:10.1101\/2020.05.29.20115824","DOI":"10.1101\/2020.05.29.20115824"},{"key":"2025101811415951000_11.12.e047623.23","doi-asserted-by":"crossref","unstructured":"Froes MT , Neves BD , Martins B . Comparison of multimorbidity in COVID-19 infected and general population in Portugal. medRxiv 2020.doi:10.1101\/2020.07.02.20144378","DOI":"10.1101\/2020.07.02.20144378"},{"key":"2025101811415951000_11.12.e047623.24","first-page":"358","article-title":"Interoperability progress and remaining data quality barriers of certified health information technologies","volume":"2018","author":"D'Amore","year":"2018","journal-title":"AMIA Annu Symp Proc"},{"key":"2025101811415951000_11.12.e047623.25","doi-asserted-by":"publisher","DOI":"10.3390\/ijerph110505170"},{"key":"2025101811415951000_11.12.e047623.26","unstructured":"IOM Roundtable on Value & Science-Driven Care, Institute of Medicine . Integrating Research and Practice: Health System Leaders Working Toward High-Value Care: Workshop Summary. In: Continuously learning health care: the value proposition. Washington (DC): National Academies Press (US), 2015. https:\/\/www.ncbi.nlm.nih.gov\/books\/NBK284656\/"}],"container-title":["BMJ Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjopen-2020-047623","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,18]],"date-time":"2025-10-18T18:42:09Z","timestamp":1760812929000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmjopen.bmj.com\/lookup\/doi\/10.1136\/bmjopen-2020-047623"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12]]},"references-count":26,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,12,6]]},"published-print":{"date-parts":[[2021,12]]}},"alternative-id":["10.1136\/bmjopen-2020-047623"],"URL":"https:\/\/doi.org\/10.1136\/bmjopen-2020-047623","relation":{},"ISSN":["2044-6055","2044-6055"],"issn-type":[{"value":"2044-6055","type":"print"},{"value":"2044-6055","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12]]}}}