{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T03:42:40Z","timestamp":1772163760133,"version":"3.50.1"},"reference-count":14,"publisher":"BMJ","issue":"1","license":[{"start":{"date-parts":[[2020,3,23]],"date-time":"2020-03-23T00:00:00Z","timestamp":1584921600000},"content-version":"unspecified","delay-in-days":22,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","award":["UL1 TR001872"],"award-info":[{"award-number":["UL1 TR001872"]}],"id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000062","name":"National Institute of Diabetes and Digestive and Kidney Diseases","doi-asserted-by":"publisher","award":["T32 DK007007-42"],"award-info":[{"award-number":["T32 DK007007-42"]}],"id":[{"id":"10.13039\/100000062","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Open Qual"],"accepted":{"date-parts":[[2020,3,5]]},"published-print":{"date-parts":[[2020,3]]},"abstract":"<jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Medical billing data are an attractive source of secondary analysis because of their ease of use and potential to answer population-health questions with statistical power. Although these datasets have known susceptibilities to biases, the degree to which they can distort the assessment of quality measures such as colorectal cancer screening rates are not widely appreciated, nor are their causes and possible solutions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Using a billing code database derived from our institution\u2019s electronic health records, we estimated the colorectal cancer screening rate of average-risk patients aged 50\u201374 years seen in primary care or gastroenterology clinic in 2016\u20132017. 200 records (150 unscreened, 50 screened) were sampled to quantify the accuracy against manual review.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Out of 4611 patients, an analysis of billing data suggested a 61% screening rate, an estimate that matches the estimate by the Centers for Disease Control. Manual review revealed a positive predictive value of 96% (86%\u2013100%), negative predictive value of 21% (15%\u201329%) and a corrected screening rate of 85% (81%\u201390%). Most false negatives occurred due to examinations performed outside the scope of the database\u2014both within and outside of our institution\u2014but 21% of false negatives fell within the database\u2019s scope. False positives occurred due to incomplete examinations and inadequate bowel preparation. Reasons for screening failure include ordered but incomplete examinations (48%), lack of or incorrect documentation by primary care (29%) including incorrect screening intervals (13%) and patients declining screening (13%).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Billing databases are prone to substantial bias that may go undetected even in the presence of confirmatory external estimates. Caution is recommended when performing population-level inference from these data. We propose several solutions to improve the use of these data for the assessment of healthcare quality.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1136\/bmjoq-2019-000856","type":"journal-article","created":{"date-parts":[[2020,3,24]],"date-time":"2020-03-24T15:49:27Z","timestamp":1585064967000},"page":"e000856","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":14,"title":["Accuracy of medical billing data against the electronic health record in the measurement of colorectal cancer screening rates"],"prefix":"10.1136","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1789-3004","authenticated-orcid":false,"given":"Vivek A","family":"Rudrapatna","sequence":"first","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, United States"},{"name":"Division of Gastroenterology, Department of Medicine, University of California, San Francisco, CA, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4515-8090","authenticated-orcid":false,"given":"Benjamin S","family":"Glicksberg","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, United States"},{"name":"The Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY, United States"},{"name":"Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Patrick","family":"Avila","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology, Department of Medicine, University of California, San Francisco, CA, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Emily","family":"Harding-Theobald","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology and Hepatology, Department of Internal Medicine, University of Michigan, Ann Arbor, MI, United States"},{"name":"Department of Medicine, University of California, San Francisco, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6621-8112","authenticated-orcid":false,"given":"Connie","family":"Wang","sequence":"additional","affiliation":[{"name":"Division of Gastroenterology, Department of Medicine, University of California, San Francisco, CA, United States"},{"name":"Department of Medicine, University of California, San Francisco, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7433-2740","authenticated-orcid":false,"given":"Atul J","family":"Butte","sequence":"additional","affiliation":[{"name":"Bakar Computational Health Sciences Institute, University of California, San Francisco, CA, United States"},{"name":"Department of Pediatrics, University of California, San Francsico, CA, United States"},{"name":"Center for Data-Driven Insights and Innovation, University of California Health, Oakland, CA, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"239","published-online":{"date-parts":[[2020,3,24]]},"reference":[{"key":"2025083111201053000_9.1.e000856.1","doi-asserted-by":"publisher","DOI":"10.3322\/caac.21387"},{"key":"2025083111201053000_9.1.e000856.2","doi-asserted-by":"crossref","first-page":"481","DOI":"10.7326\/M17-0723","article-title":"Colonoscopy and colorectal cancer mortality in the Veterans Affairs health care system: a case-control study","volume":"168","author":"Kahi","year":"2018","journal-title":"Ann Intern Med"},{"key":"2025083111201053000_9.1.e000856.3","first-page":"253","article-title":"Use of colorectal cancer tests--United States, 2002, 2004, and 2006","volume":"57","year":"2008","journal-title":"MMWR Morb Mortal Wkly Rep"},{"key":"2025083111201053000_9.1.e000856.4","doi-asserted-by":"crossref","first-page":"201","DOI":"10.15585\/mmwr.mm6608a1","article-title":"Cancer Screening Test Use - United States, 2015","volume":"66","author":"White","year":"2017","journal-title":"MMWR Morb Mortal Wkly Rep"},{"key":"2025083111201053000_9.1.e000856.5","doi-asserted-by":"publisher","DOI":"10.1016\/S0749-3797(99)00069-0"},{"key":"2025083111201053000_9.1.e000856.6","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1172\/JCI129197","article-title":"Opportunities and challenges in using real-world data for health care","volume":"130","author":"Rudrapatna","year":"2020","journal-title":"J Clin Invest"},{"key":"2025083111201053000_9.1.e000856.7","doi-asserted-by":"publisher","DOI":"10.1200\/JOP.2014.001308"},{"key":"2025083111201053000_9.1.e000856.8","doi-asserted-by":"publisher","DOI":"10.1097\/MLR.0b013e31829b1dbd"},{"key":"2025083111201053000_9.1.e000856.9","unstructured":"EPIC . Electronic Health Record Software [Internet]. Available: www.epic.com [Accessed 2 Jan 2018]."},{"key":"2025083111201053000_9.1.e000856.10","unstructured":"Vienna, Austria: R Foundation for Statistical Computing . R: A language and environment for statistical computing, 2013. Available: http:\/\/www.r-project.org\/"},{"key":"2025083111201053000_9.1.e000856.11","unstructured":"Database Interface and \u201cMySQL\u201d Driver for R [R package RMySQL version 0.10.17]. Available: https:\/\/cran.r-project.org\/web\/packages\/RMySQL\/index.html [Accessed 30 Jun 2019]."},{"key":"2025083111201053000_9.1.e000856.12","unstructured":"Extension of \u201cdata.frame\u201d [R package data.table version 1.12.2]. Available: https:\/\/cran.r-project.org\/web\/packages\/data.table\/index.html [Accessed 30 Jun 2019]."},{"key":"2025083111201053000_9.1.e000856.13","article-title":"Approximate Is Better than \"Exact\" for Interval Estimation of Binomial Proportions","volume":"52","author":"Agresti","year":"1998","journal-title":"Am Stat"},{"key":"2025083111201053000_9.1.e000856.14","doi-asserted-by":"publisher","DOI":"10.1093\/fampra\/cmw065"}],"container-title":["BMJ Open Quality"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjoq-2019-000856","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,31]],"date-time":"2025-08-31T18:20:20Z","timestamp":1756664420000},"score":1,"resource":{"primary":{"URL":"https:\/\/qir.bmj.com\/lookup\/doi\/10.1136\/bmjoq-2019-000856"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3]]},"references-count":14,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,3,24]]},"published-print":{"date-parts":[[2020,3]]}},"alternative-id":["10.1136\/bmjoq-2019-000856"],"URL":"https:\/\/doi.org\/10.1136\/bmjoq-2019-000856","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/19004598","asserted-by":"object"}]},"ISSN":["2399-6641"],"issn-type":[{"value":"2399-6641","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3]]}}}