{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T16:41:07Z","timestamp":1781109667480,"version":"3.54.1"},"reference-count":30,"publisher":"Oxford University Press (OUP)","issue":"7","license":[{"start":{"date-parts":[[2021,1,26]],"date-time":"2021-01-26T00:00:00Z","timestamp":1611619200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"Australian Institute of Health and Welfare"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,7,14]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Data quality (DQ) must be consistently defined in context. The attributes, metadata, and context of longitudinal real-world data (RWD) have not been formalized for quality improvement across the data production and curation life cycle. We sought to complete a literature review on DQ assessment frameworks, indicators and tools for research, public health, service, and quality improvement across the data life cycle.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>The review followed PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Databases from health, physical and social sciences were used: Cinahl, Embase, Scopus, ProQuest, Emcare, PsycINFO, Compendex, and Inspec. Embase was used instead of PubMed (an interface to search MEDLINE) because it includes all MeSH (Medical Subject Headings) terms used and journals in MEDLINE as well as additional unique journals and conference abstracts. A combined data life cycle and quality framework guided the search of published and gray literature for DQ frameworks, indicators, and tools. At least 2 authors independently identified articles for inclusion and extracted and categorized DQ concepts and constructs. All authors discussed findings iteratively until consensus was reached.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The 120 included articles yielded concepts related to contextual (data source, custodian, and user) and technical (interoperability) factors across the data life cycle. Contextual DQ subcategories included relevance, usability, accessibility, timeliness, and trust. Well-tested computable DQ indicators and assessment tools were also found.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>A DQ assessment framework that covers intrinsic, technical, and contextual categories across the data life cycle enables assessment and management of RWD repositories to ensure fitness for purpose. Balancing security, privacy, and FAIR principles requires trust and reciprocity, transparent governance, and organizational cultures that value good documentation.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocaa340","type":"journal-article","created":{"date-parts":[[2020,12,22]],"date-time":"2020-12-22T12:19:40Z","timestamp":1608639580000},"page":"1591-1599","source":"Crossref","is-referenced-by-count":82,"title":["Quality assessment of real-world data repositories across the data life cycle: A literature review"],"prefix":"10.1093","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5989-3614","authenticated-orcid":false,"given":"Siaw-Teng","family":"Liaw","sequence":"first","affiliation":[{"name":"WHO Collaborating Centre on eHealth, School of Population Health, Faculty of Medicine, UNSW Sydney, Sydney, New South Wales, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jason Guan Nan","family":"Guo","sequence":"additional","affiliation":[{"name":"WHO Collaborating Centre on eHealth, School of Population Health, Faculty of Medicine, UNSW Sydney, Sydney, New South Wales, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sameera","family":"Ansari","sequence":"additional","affiliation":[{"name":"WHO Collaborating Centre on eHealth, School of Population Health, Faculty of Medicine, UNSW Sydney, Sydney, New South Wales, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9912-2344","authenticated-orcid":false,"given":"Jitendra","family":"Jonnagaddala","sequence":"additional","affiliation":[{"name":"WHO Collaborating Centre on eHealth, School of Population Health, Faculty of Medicine, UNSW Sydney, Sydney, New South Wales, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0081-2506","authenticated-orcid":false,"given":"Myron Anthony","family":"Godinho","sequence":"additional","affiliation":[{"name":"WHO Collaborating Centre on eHealth, School of Population Health, Faculty of Medicine, UNSW Sydney, Sydney, New South Wales, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"suffix":"Jr","given":"Alder Jose","family":"Borelli","sequence":"additional","affiliation":[{"name":"WHO Collaborating Centre on eHealth, School of Population Health, Faculty of Medicine, UNSW Sydney, Sydney, New South Wales, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8553-2641","authenticated-orcid":false,"given":"Simon","family":"de Lusignan","sequence":"additional","affiliation":[{"name":"Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Capurro","sequence":"additional","affiliation":[{"name":"Faculty of Engineering and Information Technology, University of Melbourne, Melbourne, Victoria, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Harshana","family":"Liyanage","sequence":"additional","affiliation":[{"name":"Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Navreet","family":"Bhattal","sequence":"additional","affiliation":[{"name":"Australian Institute of Health and Welfare, Canberra, Australian Capital Territory, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vicki","family":"Bennett","sequence":"additional","affiliation":[{"name":"Australian Institute of Health and Welfare, Canberra, Australian Capital Territory, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jaclyn","family":"Chan","sequence":"additional","affiliation":[{"name":"Australian Institute of Health and Welfare, Canberra, Australian Capital Territory, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4786-6875","authenticated-orcid":false,"given":"Michael G","family":"Kahn","sequence":"additional","affiliation":[{"name":"Department of Pediatrics (Section of Informatics and Data Sciences), University of Colorado Anschutz Medical Campus, Denver, Colorado, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,1,26]]},"reference":[{"issue":"1","key":"2021092713440974700_ocaa340-B1","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1055\/s-0039-1677901","article-title":"Artificial intelligence in primary health care: perceptions, issues, and challenges","volume":"28","author":"Liyanage","year":"2019","journal-title":"Yearb Med Inform"},{"issue":"1","key":"2021092713440974700_ocaa340-B2","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1055\/s-0040-1701980","article-title":"Ethical use of electronic health record data and artificial intelligence: recommendations of the primary care informatics working group of the international medical informatics association","volume":"29","author":"Liaw","year":"2020","journal-title":"Yearb Med Inform"},{"issue":"1","key":"2021092713440974700_ocaa340-B3","doi-asserted-by":"crossref","first-page":"138","DOI":"10.15265\/IY-2016-035","article-title":"Building a privacy, ethics, and data access framework for real world computerised medical record system data: a Delphi study. Contribution of the Primary Health Care Informatics Working Group","volume":"25","author":"Liyanage","year":"2016","journal-title":"Yearb Med Inform"},{"issue":"1","key":"2021092713440974700_ocaa340-B4","doi-asserted-by":"crossref","first-page":"160018","DOI":"10.1038\/sdata.2016.18","article-title":"The FAIR Guiding Principles for scientific data management and stewardship","volume":"3","author":"Wilkinson","year":"2016","journal-title":"Sci Data"},{"issue":"1","key":"2021092713440974700_ocaa340-B5","first-page":"3","article-title":"Evaluating foundational data quality in the national patientcentered clinical research network (PCORnet","volume":"6","author":"Qualls","year":"2018","journal-title":"EGEMS (Wash DC)"},{"key":"2021092713440974700_ocaa340-B6","year":"2016"},{"issue":"4","key":"2021092713440974700_ocaa340-B7","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1080\/07421222.1996.11518099","article-title":"Beyond accuracy: what data quality means to data consumers","volume":"12","author":"Wang","year":"1996","journal-title":"J Manage Inf Syst"},{"issue":"1","key":"2021092713440974700_ocaa340-B8","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.ijmedinf.2012.10.001","article-title":"Towards an ontology for data quality in integrated chronic disease: a realist review of the literature","volume":"82","author":"Liaw","year":"2013","journal-title":"Int J Med Inform"},{"issue":"1","key":"2021092713440974700_ocaa340-B9","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1136\/amiajnl-2011-000681","article-title":"Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research","volume":"20","author":"Weiskopf","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2021092713440974700_ocaa340-B10","first-page":"721","article-title":"Organizing data quality assessment of shifting biomedical data","volume":"180","author":"Saez","year":"2012","journal-title":"Stud Health Technol Inform"},{"key":"2021092713440974700_ocaa340-B11","first-page":"628","article-title":"Methods for examining data quality in healthcare integrated data repositories","volume":"23","author":"Huser","year":"2018","journal-title":"Biocomputing"},{"key":"2021092713440974700_ocaa340-B12","year":"2017"},{"key":"2021092713440974700_ocaa340-B13","volume-title":"Improving Data Quality: A Guide for Developing Countries","year":"2003"},{"issue":"12","key":"2021092713440974700_ocaa340-B14","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1016\/j.ijmedinf.2015.09.008","article-title":"Structured data quality reports to improve EHR data quality","volume":"84","author":"Taggart","year":"2015","journal-title":"Int J Med Inform"},{"issue":"1","key":"2021092713440974700_ocaa340-B15","first-page":"18","article-title":"A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data","volume":"4","author":"Kahn","year":"2016","journal-title":"EGEMS (Wash DC)"},{"key":"2021092713440974700_ocaa340-B16","first-page":"1488","article-title":"Extending Achilles heel data quality tool with new rules informed by multi-site data quality comparison","volume":"264","author":"Huser","year":"2019","journal-title":"Stud Health Technol Inform"},{"issue":"6","key":"2021092713440974700_ocaa340-B17","doi-asserted-by":"crossref","first-page":"1072","DOI":"10.1093\/jamia\/ocx033","article-title":"A longitudinal analysis of data quality in a large pediatric data research network","volume":"24","author":"Khare","year":"2017","journal-title":"J Am Med Inform Assoc"},{"key":"2021092713440974700_ocaa340-B18","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.cmpb.2019.05.017","article-title":"Towards a content agnostic computable knowledge repository for data quality assessment","volume":"177","author":"Rajan","year":"2019","journal-title":"Comput Methods Progr Biomed"},{"issue":"1","key":"2021092713440974700_ocaa340-B19","first-page":"38","article-title":"Improving a secondary use health data warehouse: proposing a multi-level data quality framework","volume":"7","author":"Henley-Smith","year":"2019","journal-title":"EGEMS (Wash DC)"},{"key":"2021092713440974700_ocaa340-B20","doi-asserted-by":"crossref","first-page":"104954","DOI":"10.1016\/j.cmpb.2019.06.013","article-title":"Guest editorial: Special issue in biomedical data quality assessment methods","volume":"181","author":"S\u00e1ez","year":"2019","journal-title":"Comput Methods Programs Biomed"},{"key":"2021092713440974700_ocaa340-B21","year":"2015"},{"key":"2021092713440974700_ocaa340-B22","author":"Lee","year":"2001"},{"issue":"1","key":"2021092713440974700_ocaa340-B23","article-title":"Multisite evaluation of a data quality tool for patient-level clinical data sets","volume":"4","author":"Huser","year":"2016","journal-title":"EGEMS (Wash DC)"},{"issue":"2","key":"2021092713440974700_ocaa340-B24","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1985347.1985353","article-title":"Trust in a specific technology: An investigation of its components and measures","volume":"2","author":"McKnight","year":"2011","journal-title":"ACM Trans Manage Inf Syst"},{"key":"2021092713440974700_ocaa340-B25","first-page":"574","article-title":"Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers","volume":"216","author":"Hripcsak","year":"2015","journal-title":"Stud Health Technol Inform"},{"key":"2021092713440974700_ocaa340-B26"},{"key":"2021092713440974700_ocaa340-B27","doi-asserted-by":"crossref","first-page":"104824","DOI":"10.1016\/j.cmpb.2018.12.029","article-title":"TAQIH, a tool for tabular data quality assessment and improvement in the context of health data","volume":"181","author":"\u00c1lvarez S\u00e1nchez","year":"2019","journal-title":"Comput Methods Programs Biomed"},{"issue":"3","key":"2021092713440974700_ocaa340-B28","doi-asserted-by":"crossref","first-page":"547","DOI":"10.14236\/jhi.v23i3.826","article-title":"An \u2018integrated health neighbourhood\u2019 framework to optimise the use of EHR data","volume":"23","author":"Liaw","year":"2016","journal-title":"J Innov Health Inform"},{"issue":"1","key":"2021092713440974700_ocaa340-B29","first-page":"7","article-title":"Transparent reporting of data quality in distributed data networks","volume":"3","author":"Kahn","year":"2015","journal-title":"EGEMS (Wash DC)"},{"key":"2021092713440974700_ocaa340-B30","doi-asserted-by":"crossref","first-page":"S22","DOI":"10.1097\/MLR.0b013e31829b1e2c","article-title":"Data quality assessment for comparative effectiveness research in distributed data networks","volume":"51","author":"Brown","year":"2013","journal-title":"Med Care"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/7\/1591\/40444246\/ocaa340.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/7\/1591\/40444246\/ocaa340.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,27]],"date-time":"2021-09-27T18:20:22Z","timestamp":1632766822000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/7\/1591\/6120399"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,26]]},"references-count":30,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,1,26]]},"published-print":{"date-parts":[[2021,7,14]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa340","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,7,1]]},"published":{"date-parts":[[2021,1,26]]}}}