{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T22:49:48Z","timestamp":1790808588329,"version":"4.1.0"},"reference-count":44,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2016,4,23]],"date-time":"2016-04-23T00:00:00Z","timestamp":1461369600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"Spanish Ministry of Economy and Competitiveness","award":["RTC-2014-1530-1"],"award-info":[{"award-number":["RTC-2014-1530-1"]}]},{"name":"Spanish Ministry of Economy and Competitiveness","award":["TIN-2013-43457-R"],"award-info":[{"award-number":["TIN-2013-43457-R"]}]},{"DOI":"10.13039\/501100004233","name":"Universitat Polit\u00e8cnica de Val\u00e8ncia","doi-asserted-by":"publisher","award":["SP20141432"],"award-info":[{"award-number":["SP20141432"]}],"id":[{"id":"10.13039\/501100004233","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2016,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Objective To assess the variability in data distributions among data sources and over time through a case study of a large multisite repository as a systematic approach to data quality (DQ).<\/jats:p>\n                  <jats:p>Materials and Methods Novel probabilistic DQ control methods based on information theory and geometry are applied to the Public Health Mortality Registry of the Region of Valencia, Spain, with 512 143 entries from 2000 to 2012, disaggregated into 24 health departments. The methods provide DQ metrics and exploratory visualizations for (1) assessing the variability among multiple sources and (2) monitoring and exploring changes with time. The methods are suited to big data and multitype, multivariate, and multimodal data.<\/jats:p>\n                  <jats:p>Results The repository was partitioned into 2 probabilistically separated temporal subgroups following a change in the Spanish National Death Certificate in 2009. Punctual temporal anomalies were noticed due to a punctual increment in the missing data, along with outlying and clustered health departments due to differences in populations or in practices.<\/jats:p>\n                  <jats:p>Discussion Changes in protocols, differences in populations, biased practices, or other systematic DQ problems affected data variability. Even if semantic and integration aspects are addressed in data sharing infrastructures, probabilistic variability may still be present. Solutions include fixing or excluding data and analyzing different sites or time periods separately. A systematic approach to assessing temporal and multisite variability is proposed.<\/jats:p>\n                  <jats:p>Conclusion Multisite and temporal variability in data distributions affects DQ, hindering data reuse, and an assessment of such variability should be a part of systematic DQ procedures.<\/jats:p>","DOI":"10.1093\/jamia\/ocw010","type":"journal-article","created":{"date-parts":[[2016,4,23]],"date-time":"2016-04-23T20:18:19Z","timestamp":1461442699000},"page":"1085-1095","source":"Crossref","is-referenced-by-count":45,"title":["Applying probabilistic temporal and multisite data quality control methods to a public health mortality registry in Spain: a systematic approach to quality control of repositories"],"prefix":"10.1093","volume":"23","author":[{"given":"Carlos","family":"S\u00e1ez","sequence":"first","affiliation":[{"name":"Instituto Universitario de Aplicaciones de las Tecnolog\u00edas de la Informaci\u00f3n y de las Comunicaciones Avanzadas. Universitat Polit\u00e8cnica de Val\u00e8ncia. Camino de Vera s\/n. 46022 Valencia, Espa\u00f1a"},{"name":"Centre for Health Technologies and Services Research, University of Porto, Porto, Portugal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Oscar","family":"Zurriaga","sequence":"additional","affiliation":[{"name":"Direcci\u00f3n General de Salud P\u00fablica, Conselleria de Sanidad, Valencia, Spain"},{"name":"FISABIO \u2013 Salud P\u00fablica, Conseller\u00eda de Sanidad, Valencia, Spain"},{"name":"CIBERESP, Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jordi","family":"P\u00e9rez-Panad\u00e9s","sequence":"additional","affiliation":[{"name":"Direcci\u00f3n General de Salud P\u00fablica, Conselleria de Sanidad, Valencia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Inma","family":"Melchor","sequence":"additional","affiliation":[{"name":"Direcci\u00f3n General de Salud P\u00fablica, Conselleria de Sanidad, Valencia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Montserrat","family":"Robles","sequence":"additional","affiliation":[{"name":"Instituto Universitario de Aplicaciones de las Tecnolog\u00edas de la Informaci\u00f3n y de las Comunicaciones Avanzadas. Universitat Polit\u00e8cnica de Val\u00e8ncia. Camino de Vera s\/n. 46022 Valencia, Espa\u00f1a"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juan M","family":"Garc\u00eda-G\u00f3mez","sequence":"additional","affiliation":[{"name":"Instituto Universitario de Aplicaciones de las Tecnolog\u00edas de la Informaci\u00f3n y de las Comunicaciones Avanzadas. Universitat Polit\u00e8cnica de Val\u00e8ncia. Camino de Vera s\/n. 46022 Valencia, Espa\u00f1a"},{"name":"Unidad Mixta de Investigaci\u00f3n en TICs aplicadas a la Reingenier\u00eda de Procesos Sociosanitarios (eRPSS), Instituto de Investigaci\u00f3n Sanitaria del Hospital Universitario y Polit\u00e9cnico La Fe, Valencia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2016,4,23]]},"reference":[{"issue":"1","key":"2020110612354487000_ocw010-B1","first-page":"125","article-title":"Big data and smart health strategies: findings from the health information systems perspective","volume":"9","author":"Toubiana","year":"2014","journal-title":"IMIA Yearb."},{"issue":"5","key":"2020110612354487000_ocw010-B2","doi-asserted-by":"crossref","first-page":"624","DOI":"10.1197\/jamia.M3191","article-title":"The Shared Health Research Information Network (SHRINE): a prototype federated query tool for clinical data repositories","volume":"16","author":"Weber","year":"2009","journal-title":"J Am Med Inform Assoc."},{"issue":"3","key":"2020110612354487000_ocw010-B3","doi-asserted-by":"crossref","first-page":"e55811","DOI":"10.1371\/journal.pone.0055811","article-title":"SHRINE: enabling nationally scalable multisite disease studies. Carter KW, editor","volume":"8","author":"McMurry","year":"2013","journal-title":"PLoS ONE."},{"issue":"1","key":"2020110612354487000_ocw010-B4","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1136\/amiajnl-2012-001042","article-title":"An i2b2-based, generalizable, open source, self-scaling chronic disease registry","volume":"20","author":"Natter","year":"2013","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612354487000_ocw010-B5","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1136\/amiajnl-2011-000200","article-title":"Direct2Experts: a pilot national network to demonstrate interoperability among research-networking platforms","volume":"18","author":"Weber","year":"2011","journal-title":"J Am Med Inform Assoc."},{"issue":"6","key":"2020110612354487000_ocw010-B6","doi-asserted-by":"crossref","first-page":"1129","DOI":"10.1136\/amiajnl-2013-002629","article-title":"Using the CER Hub to ensure data quality in a multi-institution smoking cessation study","volume":"21","author":"Walker","year":"2014","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612354487000_ocw010-B7","first-page":"1","article-title":"Open access to and reuse of research data - The state of the art in Finland","volume":"7","author":"Kuula","year":"2008","journal-title":"Finnish Soc Sci Data Archive."},{"issue":"5","key":"2020110612354487000_ocw010-B8","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1016\/j.ejca.2008.11.032","article-title":"Evaluation of data quality in the cancer registry: Principles and methods. Part I: comparability, validity and timeliness","volume":"45","author":"Bray","year":"2009","journal-title":"Eur J Cancer."},{"key":"2020110612354487000_ocw010-B9","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1136\/amiajnl-2011-000508","article-title":"Practices and perspectives on building integrated data repositories: results from a 2010 CTSA survey","volume":"19","author":"MacKenzie","year":"2012","journal-title":"J Am Med Inform Assoc."},{"issue":"5","key":"2020110612354487000_ocw010-B10","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1136\/amiajnl-2011-000507","article-title":"An informatics agenda for public health: summarized recommendations from the 2011 AMIA PHI Conference","volume":"19","author":"Massoudi","year":"2012","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612354487000_ocw010-B11","doi-asserted-by":"crossref","first-page":"S21","DOI":"10.1097\/MLR.0b013e318257dd67","article-title":"A pragmatic framework for single-site and multisite data quality assessment in electronic health record-based clinical research","volume":"50","author":"Kahn","year":"2012","journal-title":"Med Care."},{"issue":"1","key":"2020110612354487000_ocw010-B12","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1136\/amiajnl-2011-000681","article-title":"Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research","volume":"20","author":"Weiskopf","year":"2013","journal-title":"J Am Med Inform Assoc."},{"issue":"1","key":"2020110612354487000_ocw010-B13","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.ijmedinf.2012.10.001","article-title":"Towards an ontology for data quality in integrated chronic disease management: a realist review of the literature","volume":"82","author":"Liaw","year":"2013","journal-title":"Int J Med Inf."},{"issue":"5","key":"2020110612354487000_ocw010-B14","doi-asserted-by":"crossref","first-page":"5170","DOI":"10.3390\/ijerph110505170","article-title":"A review of data quality assessment methods for public health information systems","volume":"11","author":"Chen","year":"2014","journal-title":"Int J Environ Res Public Health."},{"key":"2020110612354487000_ocw010-B15","first-page":"55","article-title":"Data quality and integration issues in electronic health records","author":"Cruz-Correia","year":"2009","journal-title":"In: Information Discovery on Electronic Health Records, Chapman and Hall\/CRC."},{"issue":"11","key":"2020110612354487000_ocw010-B16","doi-asserted-by":"crossref","first-page":"2417","DOI":"10.1016\/j.socscimed.2004.11.028","article-title":"A model of underlying socioeconomic vulnerability in human populations: evidence from variability in population health and implications for public health","volume":"60","author":"Galea","year":"2005","journal-title":"Soc Sci Med."},{"issue":"5","key":"2020110612354487000_ocw010-B17","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1016\/S0197-2456(98)00033-6","article-title":"Guidelines for quality assurance in multicenter trials: a position paper","volume":"19","author":"Knatterud","year":"1998","journal-title":"Control Clin Trials."},{"key":"2020110612354487000_ocw010-B18","article-title":"Stability metrics for multi-source biomedical data based on simplicial projections from probability distribution distances","author":"S\u00e1ez","year":"2014","journal-title":"Stat Methods Med Res."},{"issue":"4","key":"2020110612354487000_ocw010-B19","doi-asserted-by":"crossref","first-page":"950","DOI":"10.1007\/s10618-014-0378-6","article-title":"Probabilistic change detection and visualization methods for the assessment of temporal stability in biomedical data quality","volume":"29","author":"S\u00e1ez","year":"2015","journal-title":"Data Min Knowl Discov."},{"key":"2020110612354487000_ocw010-B20","volume-title":"Statistical Method from the Viewpoint of Quality Control","author":"Shewhart","year":"1986"},{"key":"2020110612354487000_ocw010-B21","volume-title":"Basic QC Practices: Training in Statistical Quality Control for Medical Laboratories","author":"Westgard","year":"2010"},{"key":"2020110612354487000_ocw010-B22","first-page":"3226","volume-title":"Comparative Study of Probability Distribution Distances to Define a Metric for the Stability of Multi-source Biomedical Research Data","author":"S\u00e1ez","year":"2013"},{"issue":"3","key":"2020110612354487000_ocw010-B23","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1038\/nmeth.3288","article-title":"The fickle P value generates irreproducible results","volume":"12","author":"Halsey","year":"2015","journal-title":"Nat Methods."},{"issue":"13","key":"2020110612354487000_ocw010-B24","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1038\/506150a","article-title":"Statistical errors","volume":"506","author":"Nuzzo","year":"2014","journal-title":"Nature."},{"issue":"4","key":"2020110612354487000_ocw010-B25","doi-asserted-by":"crossref","first-page":"906","DOI":"10.1287\/isre.2013.0480","article-title":"Too Big to Fail: large samples and the p-value problem","volume":"24","author":"Lin","year":"2013","journal-title":"Inf Syst Res."},{"key":"2020110612354487000_ocw010-B26","article-title":"UCI Machine Learning Repository","author":"Asunci\u00f3n A ND"},{"key":"2020110612354487000_ocw010-B27","article-title":"National Hospital Discharge Survey (NHDS) data","author":"National Center for Health Statistics"},{"issue":"1","key":"2020110612354487000_ocw010-B28","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1109\/18.61115","article-title":"Divergence measures based on the Shannon entropy","volume":"37","author":"Lin","year":"1991","journal-title":"IEEE Trans Inf Theory."},{"key":"2020110612354487000_ocw010-B29","volume-title":"Elements of Information Theory","author":"Cover","year":"2006","edition":"2nd ed"},{"key":"2020110612354487000_ocw010-B30","volume-title":"Modern Multidimensional Scaling: Theory and Applications","author":"Borg","year":"2005"},{"key":"2020110612354487000_ocw010-B31","volume-title":"Strengthening Civil Registration and Vital Statistics for Births, Deaths and Causes of Death: Resource Kit","author":"World Health Organization","year":"2012"},{"key":"2020110612354487000_ocw010-B32","volume-title":"International Statistical Classification of Diseases and Related Health Problems. - 10th revision, 2008 edition","author":"World Health Organization","year":"2009"},{"issue":"8","key":"2020110612354487000_ocw010-B33","doi-asserted-by":"crossref","first-page":"e1","DOI":"10.1016\/j.jclinepi.2010.03.004","article-title":"CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials","volume":"63","author":"Moher","year":"2010","journal-title":"J Clin Epidemiol."},{"issue":"1","key":"2020110612354487000_ocw010-B34","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1186\/1471-2407-8-35","article-title":"Spatio-temporal evolution of female lung cancer mortality in a region of Spain, is it worth taking migration into account?","volume":"8","author":"Zurriaga","year":"2008","journal-title":"BMC Cancer."},{"issue":"1","key":"2020110612354487000_ocw010-B35","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s10334-008-0146-y","article-title":"Multiproject\u2013multicenter evaluation of automatic brain tumor classification by magnetic resonance spectroscopy","volume":"22","author":"Garc\u00eda-G\u00f3mez","year":"2009","journal-title":"Magn Reson Mater Phys Biol Med."},{"key":"2020110612354487000_ocw010-B36","doi-asserted-by":"crossref","first-page":"2349","DOI":"10.1002\/(SICI)1097-0258(19971030)16:20<2349::AID-SIM667>3.0.CO;2-E","article-title":"Tutorial in biostatistics: using the general linear mixed model to analyse unbalanced repeated measures and longitudinal data","volume":"16","author":"Cnaan","year":"1997","journal-title":"Stat Med."},{"key":"2020110612354487000_ocw010-B37","doi-asserted-by":"crossref","DOI":"10.1007\/3-540-73679-4","volume-title":"Learning from Data Streams: Processing Techniques in Sensor Networks","author":"Gama","year":"2007"},{"issue":"4","key":"2020110612354487000_ocw010-B38","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1016\/j.jbi.2011.02.009","article-title":"Incremental Gaussian Discriminant Analysis based on Graybill and Deal weighted combination of estimators for brain tumour diagnosis","volume":"44","author":"Tortajada","year":"2011","journal-title":"J Biomed Inform."},{"issue":"6","key":"2020110612354487000_ocw010-B39","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1016\/S0197-2456(99)00029-X","article-title":"Statistical quality control in clinical trials","volume":"20","author":"Svolba","year":"1999","journal-title":"Control Clin Trials."},{"issue":"2","key":"2020110612354487000_ocw010-B40","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1016\/0197-2456(94)00095-K","article-title":"Data quality assurance, monitoring, and reporting","volume":"16","author":"Gassman","year":"1995","journal-title":"Control Clin Trials."},{"issue":"1","key":"2020110612354487000_ocw010-B41","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1093\/epirev\/24.1.12","article-title":"Management and conduct of randomized controlled trials","volume":"24","author":"Knatterud","year":"2002","journal-title":"Epidemiol Rev."},{"key":"2020110612354487000_ocw010-B42","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1111\/j.1399-0039.2006.760_6.x","article-title":"Pilot study: assessment of interlaboratory variability of sequencing-based typing DNA sequence data quality","volume":"69","author":"Sayer","year":"2007","journal-title":"Tissue Antigens."},{"issue":"6","key":"2020110612354487000_ocw010-B43","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/51.731318","article-title":"Using mutual information to measure coupling in the cardiorespiratory system","volume":"17","author":"Pompe","year":"1998","journal-title":"Eng Med Biol Mag IEEE."},{"key":"2020110612354487000_ocw010-B44","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1109\/JCN.2008.6388332","article-title":"Mutual information applied to anomaly detection","volume":"10","author":"Kopylova","year":"2008","journal-title":"J Commun Netw."}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/23\/6\/1085\/34147851\/ocw010.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/23\/6\/1085\/34147851\/ocw010.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T12:39:41Z","timestamp":1604666381000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/23\/6\/1085\/2399251"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,4,23]]},"references-count":44,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2016,4,23]]},"published-print":{"date-parts":[[2016,11,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocw010","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2016,11]]},"published":{"date-parts":[[2016,4,23]]}}}