{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,20]],"date-time":"2026-10-20T05:57:39Z","timestamp":1792475859214,"version":"4.5.0"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"11","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2017,8]]},"abstract":"<jats:p>We introduce HoloClean, a framework for holistic data repairing driven by probabilistic inference. HoloClean unifies qualitative data repairing, which relies on integrity constraints or external data sources, with quantitative data repairing methods, which leverage statistical properties of the input data. Given an inconsistent dataset as input, HoloClean automatically generates a probabilistic program that performs data repairing. Inspired by recent theoretical advances in probabilistic inference, we introduce a series of optimizations which ensure that inference over HoloClean's probabilistic model scales to instances with millions of tuples. We show that HoloClean finds data repairs with an average precision of \u223c 90% and an average recall of above \u223c 76% across a diverse array of datasets exhibiting different types of errors. This yields an average F1 improvement of more than 2\u00d7 against state-of-the-art methods.<\/jats:p>","DOI":"10.14778\/3137628.3137631","type":"journal-article","created":{"date-parts":[[2017,9,7]],"date-time":"2017-09-07T09:35:53Z","timestamp":1504776953000},"page":"1190-1201","source":"Crossref","is-referenced-by-count":410,"title":["HoloClean"],"prefix":"10.14778","volume":"10","author":[{"given":"Theodoros","family":"Rekatsinas","sequence":"first","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xu","family":"Chu","sequence":"additional","affiliation":[{"name":"University of Waterloo"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ihab F.","family":"Ilyas","sequence":"additional","affiliation":[{"name":"University of Waterloo"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"R\u00e9","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2017,8]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"https:\/\/data.cityofchicago.org.  https:\/\/data.cityofchicago.org."},{"key":"e_1_2_1_2_1","unstructured":"https:\/\/alchemy.cs.washington.edu\/.  https:\/\/alchemy.cs.washington.edu\/."},{"key":"e_1_2_1_3_1","unstructured":"https:\/\/data.medicare.gov\/data\/physician-compare.  https:\/\/data.medicare.gov\/data\/physician-compare."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.14778\/2994509.2994518"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1514894.1514899"},{"key":"e_1_2_1_6_1","volume-title":"Hinge-loss markov random fields and probabilistic soft logic. CoRR, abs\/1505.04406","author":"Bach S. H.","year":"2015","unstructured":"S. H. Bach , M. Broecheler , B. Huang , and L. Getoor . Hinge-loss markov random fields and probabilistic soft logic. CoRR, abs\/1505.04406 , 2015 . S. H. Bach, M. Broecheler, B. Huang, and L. Getoor. Hinge-loss markov random fields and probabilistic soft logic. CoRR, abs\/1505.04406, 2015."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1938551.1938585"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920870"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2013.6544854"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1066157.1066175"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2007.367920"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1302293110"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ic.2004.04.007"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536258.2536262"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2013.6544847"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2749431"},{"key":"e_1_2_1_17_1","first-page":"315","volume-title":"PVLDB","author":"Cong G.","year":"2007","unstructured":"G. Cong , W. Fan , F. Geerts , X. Jia , and S. Ma . Improving data quality: Consistency and accuracy . In PVLDB , pages 315 -- 326 . VLDB Endowment , 2007 . G. Cong, W. Fan, F. Geerts, X. Jia, and S. Ma. Improving data quality: Consistency and accuracy. In PVLDB, pages 315--326. VLDB Endowment, 2007."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2465327"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1281192.1281219"},{"key":"e_1_2_1_20_1","first-page":"255","volume-title":"UAI","author":"Ermon S.","year":"2012","unstructured":"S. Ermon , C. P. Gomes , and B. Selman . Uniform solution sampling using a constraint solver as an oracle . In UAI , pages 255 -- 264 , 2012 . S. Ermon, C. P. Gomes, and B. Selman. Uniform solution sampling using a constraint solver as an oracle. In UAI, pages 255--264, 2012."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2745754.2745762"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376916.1376940"},{"key":"e_1_2_1_23_1","volume-title":"Foundations of Data Quality Management. Synthesis Lectures on Data Management","author":"Fan W.","year":"2012","unstructured":"W. Fan and F. Geerts . Foundations of Data Quality Management. Synthesis Lectures on Data Management . 2012 . W. Fan and F. Geerts. Foundations of Data Quality Management. Synthesis Lectures on Data Management. 2012."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687627.1687674"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920867"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536360.2536363"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.14778\/2367502.2367564"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1214\/105051607000000104"},{"key":"e_1_2_1_29_1","volume-title":"United Nations Economic Commission for Europe (UNECE)","author":"Hellerstein J. M.","year":"2008","unstructured":"J. M. Hellerstein . Quantitative data cleaning for large databases . United Nations Economic Commission for Europe (UNECE) , 2008 . J. M. Hellerstein. Quantitative data cleaning for large databases. United Nations Economic Commission for Europe (UNECE), 2008."},{"key":"e_1_2_1_30_1","first-page":"38","article-title":"Effective data cleaning with continuous evaluation","volume":"39","author":"Ilyas I. F.","year":"2016","unstructured":"I. F. Ilyas . Effective data cleaning with continuous evaluation . IEEE Data Eng. Bull. , 39 : 38 -- 46 , 2016 . I. F. Ilyas. Effective data cleaning with continuous evaluation. IEEE Data Eng. Bull., 39:38--46, 2016.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1561\/1900000045"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2015.7113269"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979444"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1514894.1514901"},{"key":"e_1_2_1_35_1","volume-title":"Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning","author":"Koller D.","year":"2009","unstructured":"D. Koller and N. Friedman . Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning . The MIT Press , 2009 . D. Koller and N. Friedman. Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning. The MIT Press, 2009."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2009.219"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1142473.1142599"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14778\/2535568.2448943"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807167.1807178"},{"key":"e_1_2_1_40_1","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-031-01835-0","volume-title":"An Introduction to Duplicate Detection. Synthesis Lectures on Data Management","author":"Naumann F.","year":"2010","unstructured":"F. Naumann and M. Herschel . An Introduction to Duplicate Detection. Synthesis Lectures on Data Management . 2010 . F. Naumann and M. Herschel. An Introduction to Duplicate Detection. Synthesis Lectures on Data Management. 2010."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.14778\/1978665.1978669"},{"issue":"4","key":"e_1_2_1_42_1","first-page":"3","article-title":"Problems and current approaches","volume":"23","author":"Rahm E.","year":"2000","unstructured":"E. Rahm and H. H. Do . Data cleaning : Problems and current approaches . IEEE Data Eng. Bull. , 23 ( 4 ): 3 -- 13 , 2000 . E. Rahm and H. H. Do. Data cleaning: Problems and current approaches. IEEE Data Eng. Bull., 23(4):3--13, 2000.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_2_1_43_1","first-page":"3567","volume-title":"NIPS","author":"Ratner A. J.","year":"2016","unstructured":"A. J. Ratner , C. D. Sa , S. Wu , D. Selsam , and C. R\u00e9 . Data programming: Creating large training sets, quickly . In NIPS , pages 3567 -- 3575 , 2016 . A. J. Ratner, C. D. Sa, S. Wu, D. Selsam, and C. R\u00e9. Data programming: Creating large training sets, quickly. In NIPS, pages 3567--3575, 2016."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3035951"},{"key":"e_1_2_1_45_1","first-page":"3097","volume-title":"NIPS","author":"Sa C. D.","year":"2015","unstructured":"C. D. Sa , C. Zhang , K. Olukotun , and C. R\u00e9 . Rapidly mixing gibbs sampling for a class of factor graphs using hierarchy width . In NIPS , pages 3097 -- 3105 , 2015 . C. D. Sa, C. Zhang, K. Olukotun, and C. R\u00e9. Rapidly mixing gibbs sampling for a class of factor graphs using hierarchy width. In NIPS, pages 3097--3105, 2015."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.14778\/2809974.2809991"},{"key":"e_1_2_1_47_1","volume-title":"CIDR","author":"Stonebraker M.","year":"2013","unstructured":"M. Stonebraker , D. Bruckner , I. F. Ilyas , G. Beskales , M. Cherniack , S. B. Zdonik , A. Pagan , and S. Xu . Data curation at scale: The data tamer system . In CIDR , 2013 . M. Stonebraker, D. Bruckner, I. F. Ilyas, G. Beskales, M. Cherniack, S. B. Zdonik, A. Pagan, and S. Xu. Data curation at scale: The data tamer system. In CIDR, 2013."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2610494"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463706"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732977.2733001"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3137628.3137631","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T04:56:35Z","timestamp":1672203395000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3137628.3137631"}},"subtitle":["holistic data repairs with probabilistic inference"],"short-title":[],"issued":{"date-parts":[[2017,8]]},"references-count":50,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2017,8]]}},"alternative-id":["10.14778\/3137628.3137631"],"URL":"https:\/\/doi.org\/10.14778\/3137628.3137631","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2017,8]]}}}