{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T11:42:31Z","timestamp":1774957351272,"version":"3.50.1"},"reference-count":24,"publisher":"Oxford University Press (OUP)","issue":"7","license":[{"start":{"date-parts":[[2019,3,30]],"date-time":"2019-03-30T00:00:00Z","timestamp":1553904000000},"content-version":"vor","delay-in-days":1,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/100006093","name":"Patient-Centered Outcomes Research Institute","doi-asserted-by":"publisher","award":["CDRN-1306-04608"],"award-info":[{"award-number":["CDRN-1306-04608"]}],"id":[{"id":"10.13039\/100006093","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Patient-Centered Clinical Research Network"},{"DOI":"10.13039\/100000051","name":"National Human Genome Research Institute","doi-asserted-by":"publisher","award":["R01-HG009174"],"award-info":[{"award-number":["R01-HG009174"]}],"id":[{"id":"10.13039\/100000051","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000051","name":"National Human Genome Research Institute","doi-asserted-by":"publisher","award":["U01-HG008685"],"award-info":[{"award-number":["U01-HG008685"]}],"id":[{"id":"10.13039\/100000051","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006545","name":"National Institute on Minority Health and Health Disparities","doi-asserted-by":"publisher","award":["U54MD008149"],"award-info":[{"award-number":["U54MD008149"]}],"id":[{"id":"10.13039\/100006545","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006545","name":"National Institute on Minority Health and Health Disparities","doi-asserted-by":"publisher","award":["8U54MD007588"],"award-info":[{"award-number":["8U54MD007588"]}],"id":[{"id":"10.13039\/100006545","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006545","name":"National Institute on Minority Health and Health Disparities","doi-asserted-by":"publisher","award":["U54MD008173"],"award-info":[{"award-number":["U54MD008173"]}],"id":[{"id":"10.13039\/100006545","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["UL1 TR000371"],"award-info":[{"award-number":["UL1 TR000371"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["UL1TR002378"],"award-info":[{"award-number":["UL1TR002378"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["UL1 TR001105"],"award-info":[{"award-number":["UL1 TR001105"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["UL1 TR001857"],"award-info":[{"award-number":["UL1 TR001857"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["U01 TR002393"],"award-info":[{"award-number":["U01 TR002393"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["R01 LM011829"],"award-info":[{"award-number":["R01 LM011829"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Reynolds and Reynolds Professorship in Clinical Informatics"},{"DOI":"10.13039\/100004917","name":"Cancer Prevention Research Institute of Texas","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004917","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Data Science and Informatics Core for Cancer Research","award":["RP170668"],"award-info":[{"award-number":["RP170668"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,7,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>The study sought to design, pilot, and evaluate a federated data completeness tracking system (CTX) for assessing completeness in research data extracted from electronic health record data across the Accessible Research Commons for Health (ARCH) Clinical Data Research Network.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>The CTX applies a systems-based approach to design workflow and technology for assessing completeness across distributed electronic health record data repositories participating in a queryable, federated network. The CTX invokes 2 positive feedback loops that utilize open source tools (DQe-c and Vue) to integrate technology and human actors in a system geared for increasing capacity and taking action. A pilot implementation of the system involved 6 ARCH partner sites between January 2017 and May 2018.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The ARCH CTX has enabled the network to monitor and, if needed, adjust its data management processes to maintain complete datasets for secondary use. The system allows the network and its partner sites to profile data completeness both at the network and partner site levels. Interactive visualizations presenting the current state of completeness in the context of the entire network as well as changes in completeness across time were valued among the CTX user base.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Distributed clinical data networks are complex systems. Top-down approaches that solely rely on technology to report data completeness may be necessary but not sufficient for improving completeness (and quality) of data in large-scale clinical data networks. Improving and maintaining complete (high-quality) data in such complex environments entails sociotechnical systems that exploit technology and empower human actors to engage in the process of high-quality data curating.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>The CTX has increased the network\u2019s capacity to rapidly identify data completeness issues and empowered ARCH partner sites to get involved in improving the completeness of respective data in their repositories.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocz014","type":"journal-article","created":{"date-parts":[[2019,1,18]],"date-time":"2019-01-18T04:21:56Z","timestamp":1547785316000},"page":"637-645","source":"Crossref","is-referenced-by-count":19,"title":["A federated EHR network data completeness tracking system"],"prefix":"10.1093","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0204-8978","authenticated-orcid":false,"given":"Hossein","family":"Estiri","sequence":"first","affiliation":[{"name":"Laboratory of Computer Science, Massachusetts General Hospital, Boston, Massachusetts, USA"},{"name":"Research Information Science and Computing, Partners HealthCare, Charlestown, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"}]},{"given":"Jeffrey G","family":"Klann","sequence":"additional","affiliation":[{"name":"Laboratory of Computer Science, Massachusetts General Hospital, Boston, Massachusetts, USA"},{"name":"Research Information Science and Computing, Partners HealthCare, Charlestown, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"}]},{"given":"Sarah R","family":"Weiler","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, Massachusetts, USA"}]},{"given":"Ernest","family":"Alema-Mensah","sequence":"additional","affiliation":[{"name":"Morehouse School of Medicine, Atlanta, Georgia, USA"}]},{"given":"R","family":"Joseph Applegate","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Galina","family":"Lozinski","sequence":"additional","affiliation":[{"name":"Boston University School of Medicine\/Boston Medical Center, Boston, Massachusetts, USA"}]},{"given":"Nandan","family":"Patibandla","sequence":"additional","affiliation":[{"name":"Information Services Department, Boston Children\u2019s Hospital, Boston, Massachusetts, USA"}]},{"given":"Kun","family":"Wei","sequence":"additional","affiliation":[{"name":"Wake Forest School of Medicine, Winston-Salem, North Carolina, USA"}]},{"given":"William G","family":"Adams","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, Boston University School of Medicine\/Boston Medical Center, Boston, Massachusetts, USA"}]},{"given":"Marc D","family":"Natter","sequence":"additional","affiliation":[{"name":"Computational Health Informatics Program, Boston Children\u2019s Hospital, Boston, Massachusetts, USA"},{"name":"Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, USA"},{"name":"Program in Pediatric Rheumatology, Department of Pediatrics, Mass General Hospital for Children, Boston, Massachusetts, USA"}]},{"given":"Elizabeth O","family":"Ofili","sequence":"additional","affiliation":[{"name":"Morehouse School of Medicine, Atlanta, Georgia, USA"}]},{"given":"Brian","family":"Ostasiewski","sequence":"additional","affiliation":[{"name":"Wake Forest School of Medicine, Winston-Salem, North Carolina, USA"}]},{"given":"Alexander","family":"Quarshie","sequence":"additional","affiliation":[{"name":"Morehouse School of Medicine, Atlanta, Georgia, USA"}]},{"given":"Gary E","family":"Rosenthal","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Wake Forest School of Medicine, Winston-Salem, North Carolina, USA"}]},{"given":"Elmer V","family":"Bernstam","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, Texas, USA"},{"name":"Division of General Internal Medicine, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Kenneth D","family":"Mandl","sequence":"additional","affiliation":[{"name":"Computational Health Informatics Program, Boston Children\u2019s Hospital, Boston, Massachusetts, USA"},{"name":"Department of Pediatrics, Harvard Medical School, Boston, Massachusetts, USA"},{"name":"Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA"}]},{"given":"Shawn N","family":"Murphy","sequence":"additional","affiliation":[{"name":"Laboratory of Computer Science, Massachusetts General Hospital, Boston, Massachusetts, USA"},{"name":"Research Information Science and Computing, Partners HealthCare, Charlestown, Massachusetts, USA"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"},{"name":"Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, USA"},{"name":"Department of Neurology, Massachusetts General Hospital, Boston, Massachusetts, USA"}]}],"member":"286","published-online":{"date-parts":[[2019,3,29]]},"reference":[{"issue":"79","key":"2020110613061834100_ocz014-B1","first-page":"1","article-title":"Electronic health record systems and intent to apply for meaningful use incentives among office-based physician practices: United States, 2001-2011","author":"Hsiao","year":"2011","journal-title":"NCHS Data Brief"},{"issue":"13","key":"2020110613061834100_ocz014-B2","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1001\/jama.2013.393","article-title":"The inevitable application of big data to health care","volume":"309","author":"Murdoch","year":"2013","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110613061834100_ocz014-B3","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.ijmedinf.2012.10.001","article-title":"Towards an ontology for data quality in integrated chronic disease management: a realist review of the literature","volume":"82","author":"Liaw","year":"2013","journal-title":"Int J Med Inform"},{"key":"2020110613061834100_ocz014-B4","first-page":"97","article-title":"Big data\u201d and the electronic health record","volume":"9","author":"Ross","year":"2014","journal-title":"Yearb Med Inform"},{"issue":"4","key":"2020110613061834100_ocz014-B5","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1136\/amiajnl-2014-002747","article-title":"Launching PCORnet, a national patient-centered clinical research network","volume":"21","author":"Fleurence","year":"2014","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110613061834100_ocz014-B6","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1093\/jamia\/ocx109","article-title":"Exploring completeness in clinical data research networks with DQe-c","volume":"25","author":"Estiri","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110613061834100_ocz014-B7","article-title":"A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data","volume":"4","author":"Kahn","year":"2016","journal-title":"EGEMS (Wash DC)"},{"issue":"4","key":"2020110613061834100_ocz014-B8","doi-asserted-by":"crossref","first-page":"615","DOI":"10.1136\/amiajnl-2014-002727","article-title":"Scalable Collaborative Infrastructure for a Learning Healthcare System (SCILHS): architecture","volume":"21","author":"Mandl","year":"2014","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"2020110613061834100_ocz014-B9","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1002\/cpt.320","article-title":"The FDA\u2019s sentinel initiative\u2013a comprehensive approach to medical product surveillance","volume":"99","author":"Ball","year":"2016","journal-title":"Clin Pharmacol Ther"},{"issue":"1","key":"2020110613061834100_ocz014-B10","article-title":"Evaluating foundational data quality in the National Patient-Centered Clinical Research Network (PCORnet\u00ae)","volume":"6","author":"Qualls","year":"2018","journal-title":"EGEMS (Wash DC)"},{"issue":"6","key":"2020110613061834100_ocz014-B11","doi-asserted-by":"crossref","first-page":"1072","DOI":"10.1093\/jamia\/ocx033","article-title":"A longitudinal analysis of data quality in a large pediatric data research network","volume":"24","author":"Khare","year":"2017","journal-title":"J Am Med Informatics Assoc"},{"issue":"3","key":"2020110613061834100_ocz014-B12","doi-asserted-by":"crossref","first-page":"538","DOI":"10.2105\/AJPH.2005.066001","article-title":"Practical challenges of systems thinking and modelling in public health","volume":"96","author":"Trochim","year":"2006","journal-title":"Am J Public Health"},{"issue":"2","key":"2020110613061834100_ocz014-B13","doi-asserted-by":"crossref","first-page":"S196","DOI":"10.1016\/j.amepre.2008.05.014","article-title":"Systems thinking to improve the public\u2019s health","volume":"35","author":"Leischow","year":"2008","journal-title":"Am J Prev Med"},{"key":"2020110613061834100_ocz014-B14","doi-asserted-by":"crossref","first-page":"51.","DOI":"10.1186\/1478-4505-12-51","article-title":"The application of systems thinking in health: why use systems thinking?","volume":"12","author":"Peters","year":"2014","journal-title":"Health Res Policy Syst"},{"key":"2020110613061834100_ocz014-B15","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2015-009002","article-title":"Systems science and systems thinking for public health: a systematic review of the field","volume":"5","author":"Carey","year":"2015","journal-title":"BMJ Open"},{"issue":"2","key":"2020110613061834100_ocz014-B16","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1136\/jamia.2009.000893","article-title":"Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2)","volume":"17","author":"Murphy","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2020110613061834100_ocz014-B17","doi-asserted-by":"crossref","first-page":"909","DOI":"10.1093\/jamia\/ocv188","article-title":"Data interchange using i2b2","volume":"23","author":"Klann","year":"2016","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613061834100_ocz014-B18","first-page":"17.","article-title":"What it is, its importance and results so far","volume":"107 (Suppl 1)","author":"Observational Medical Outcomes Partnership (OMOP) Initiative","year":"2010","journal-title":"Basic Clin Pharmacol Toxicol"},{"key":"2020110613061834100_ocz014-B19","doi-asserted-by":"crossref","first-page":"S22","DOI":"10.1097\/MLR.0b013e31829b1e2c","article-title":"Data quality assessment for comparative effectiveness research in distributed data networks","volume":"51","author":"Brown","year":"2013","journal-title":"Med Care"},{"key":"2020110613061834100_ocz014-B20","doi-asserted-by":"crossref","first-page":"S60","DOI":"10.1097\/MLR.0b013e318259bff4","article-title":"Data model considerations for clinical effectiveness researchers","volume":"50 (Suppl)","author":"Kahn","year":"2012","journal-title":"Med Care"},{"issue":"5","key":"2020110613061834100_ocz014-B21","doi-asserted-by":"crossref","first-page":"830","DOI":"10.1016\/j.jbi.2013.06.010","article-title":"Defining and measuring completeness of electronic health records for secondary use","volume":"46","author":"Weiskopf","year":"2013","journal-title":"J Biomed Inform"},{"key":"2020110613061834100_ocz014-B22","doi-asserted-by":"crossref","first-page":"456","DOI":"10.1002\/9781119940012.ch23","volume-title":"Statistical Methods in Healthcare","author":"Gregori","year":"2012"},{"issue":"1","key":"2020110613061834100_ocz014-B23","first-page":"1052.","article-title":"Transparent reporting of data quality in distributed data networks","volume":"3","author":"Kahn","year":"2015","journal-title":"EGEMS (Wash DC)"},{"key":"2020110613061834100_ocz014-B24","first-page":"60","article-title":"Applying a participatory design approach to define objectives and properties of a \u201cdata profiling\u201d tool for electronic health data","volume":"2016","author":"Estiri","year":"2016","journal-title":"AMIA Summits Transl Sci Proc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/26\/7\/637\/34151624\/ocz014.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/26\/7\/637\/34151624\/ocz014.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T19:06:56Z","timestamp":1604689616000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/26\/7\/637\/5423491"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,3,29]]},"references-count":24,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2019,3,29]]},"published-print":{"date-parts":[[2019,7,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocz014","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,7]]},"published":{"date-parts":[[2019,3,29]]}}}