{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T06:33:52Z","timestamp":1790750032900,"version":"4.1.0"},"reference-count":45,"publisher":"Association for Computing Machinery (ACM)","issue":"11","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2015,7]]},"abstract":"<jats:p>Populating a database with unstructured information is a long-standing problem in industry and research that encompasses problems of extraction, cleaning, and integration. Recent names used for this problem include dealing with dark data and knowledge base construction (KBC). In this work, we describe DeepDive, a system that combines database and machine learning ideas to help develop KBC systems, and we present techniques to make the KBC process more efficient. We observe that the KBC process is iterative, and we develop techniques to incrementally produce inference results for KBC systems. We propose two methods for incremental inference, based respectively on sampling and variational techniques. We also study the tradeoff space of these methods and develop a simple rule-based optimizer. DeepDive includes all of these contributions, and we evaluate DeepDive on five KBC systems, showing that it can speed up KBC inference tasks by up to two orders of magnitude with negligible impact on quality.<\/jats:p>","DOI":"10.14778\/2809974.2809991","type":"journal-article","created":{"date-parts":[[2015,7,30]],"date-time":"2015-07-30T14:37:34Z","timestamp":1438267054000},"page":"1310-1321","source":"Crossref","is-referenced-by-count":135,"title":["Incremental knowledge base construction using DeepDive"],"prefix":"10.14778","volume":"8","author":[{"given":"Jaeho","family":"Shin","sequence":"first","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sen","family":"Wu","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feiran","family":"Wang","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"De Sa","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ce","family":"Zhang","sequence":"additional","affiliation":[{"name":"Stanford University and University of Wisconsin-Madison"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"R\u00e9","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2015,7]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"UAI","author":"Acar U. A.","year":"2008","unstructured":"U. A. Acar , A. Ihler , R. Mettu , and O. S\u00fcmer . Adaptive inference on general graphical models . In UAI , 2008 . U. A. Acar, A. Ihler, R. Mettu, and O. S\u00fcmer. Adaptive inference on general graphical models. In UAI, 2008."},{"key":"e_1_2_1_2_1","volume-title":"Machine Learning","author":"Andrieu C.","year":"2003","unstructured":"C. Andrieu , N. De Freitas , A. Doucet , and M. I. Jordan . An introduction to MCMC for machine learning . Machine Learning , 2003 . C. Andrieu, N. De Freitas, A. Doucet, and M. I. Jordan. An introduction to MCMC for machine learning. Machine Learning, 2003."},{"key":"e_1_2_1_3_1","volume-title":"TAC KBP","author":"Angeli G.","year":"2014","unstructured":"G. Angeli , S. Gupta , M. Jose , C. D. Manning , C. R\u00e9 , J. Tibshirani , J. Y. Wu , S. Wu , and C. Zhang . Stanford's 2014 slot filling systems . TAC KBP , 2014 . G. Angeli, S. Gupta, M. Jose, C. D. Manning, C. R\u00e9, J. Tibshirani, J. Y. Wu, S. Wu, and C. Zhang. Stanford's 2014 slot filling systems. TAC KBP, 2014."},{"key":"e_1_2_1_4_1","volume-title":"JMLR","author":"Banerjee O.","year":"2008","unstructured":"O. Banerjee , L. El Ghaoui , and A. d' Aspremont . Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data . JMLR , 2008 . O. Banerjee, L. El Ghaoui, and A. d'Aspremont. Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data. JMLR, 2008."},{"key":"e_1_2_1_5_1","volume-title":"AAAI Spring Symposium","author":"Betteridge J.","year":"2009","unstructured":"J. Betteridge , A. Carlson , S. A. Hong , E. R. Hruschka Jr , E. L. Law , T. M. Mitchell , and S. H. Wang . Toward never ending language learning . In AAAI Spring Symposium , 2009 . J. Betteridge, A. Carlson, S. A. Hong, E. R. Hruschka Jr, E. L. Law, T. M. Mitchell, and S. H. Wang. Toward never ending language learning. In AAAI Spring Symposium, 2009."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/646543.696220"},{"key":"e_1_2_1_7_1","volume-title":"IBM Research Report","author":"Brown E.","year":"2013","unstructured":"E. Brown , E. Epstein , J. W. Murdock , and T.-H. Fin . Tools and methods for building watson . IBM Research Report , 2013 . E. Brown, E. Epstein, J. W. Murdock, and T.-H. Fin. Tools and methods for building watson. IBM Research Report, 2013."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/2898607.2898816"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2008.4497503"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2012.60"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2610516"},{"key":"e_1_2_1_12_1","author":"Delcher A. L.","year":"1996","unstructured":"A. L. Delcher , A. Grove , S. Kasif , and J. Pearl . Logarithmic-time updates and queries in probabilistic networks. J. Artif. Intell. Res. , 1996 . A. L. Delcher, A. Grove, S. Kasif, and J. Pearl. Logarithmic-time updates and queries in probabilistic networks. J. Artif. Intell. Res., 1996.","journal-title":"Logarithmic-time updates and queries in probabilistic networks. J. Artif. Intell. Res."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/1855041"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732951.2732962"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/988672.988687"},{"key":"e_1_2_1_16_1","author":"Ferrucci D.","year":"2010","unstructured":"D. Ferrucci , E. Brown , J. Chu-Carroll , J. Fan , D. Gondek , A. A. Kalyanpur , A. Lally , J. W. Murdock , E. Nyberg , J. Prager , N. Schlaefer , and C. Welty . Building Watson: An overview of the DeepQA project. AI Magazine , 2010 . D. Ferrucci, E. Brown, J. Chu-Carroll, J. Fan, D. Gondek, A. A. Kalyanpur, A. Lally, J. W. Murdock, E. Nyberg, J. Prager, N. Schlaefer, and C. Welty. Building Watson: An overview of the DeepQA project. AI Magazine, 2010.","journal-title":"Building Watson: An overview of the DeepQA project. AI Magazine"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1055558.1055560"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/170036.170066"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.3115\/992133.992154"},{"key":"e_1_2_1_20_1","volume-title":"ACL","author":"Hoffmann R.","year":"2011","unstructured":"R. Hoffmann , C. Zhang , X. Ling , L. Zettlemoyer , and D. S. Weld . Knowledge-based weak supervision for information extraction of overlapping relations . In ACL , 2011 . R. Hoffmann, C. Zhang, X. Ling, L. Zettlemoyer, and D. S. Weld. Knowledge-based weak supervision for information extraction of overlapping relations. In ACL, 2011."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376686"},{"key":"e_1_2_1_22_1","volume-title":"The Logic of Science","author":"Jaynes E. T.","year":"2003","unstructured":"E. T. Jaynes . Probability Theory : The Logic of Science . Cambridge University Press , 2003 . E. T. Jaynes. Probability Theory: The Logic of Science. Cambridge University Press, 2003."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2012.156"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.14778\/1952376.1952380"},{"key":"e_1_2_1_25_1","volume-title":"HLT","author":"Li Y.","year":"2011","unstructured":"Y. Li , F. R. Reiss , and L. Chiticariu . SystemT: A declarative information extraction system . In HLT , 2011 . Y. Li, F. R. Reiss, and L. Chiticariu. SystemT: A declarative information extraction system. In HLT, 2011."},{"key":"e_1_2_1_26_1","volume-title":"CIDR","author":"Madhavan J.","year":"2007","unstructured":"J. Madhavan , S. Jeffery , S. Cohen , X. Dong , D. Ko , C. Yu , and A. Halevy . Web-scale data integration: You can only afford to pay as you go . In CIDR , 2007 . J. Madhavan, S. Jeffery, S. Cohen, X. Dong, D. Ko, C. Yu, and A. Halevy. Web-scale data integration: You can only afford to pay as you go. In CIDR, 2007."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/1690219.1690287"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935869"},{"key":"e_1_2_1_29_1","volume-title":"AAAI","author":"Nath A.","year":"2010","unstructured":"A. Nath and P. Domingos . Efficient belief propagation for utility maximization and repeated inference . In AAAI , 2010 . A. Nath and P. Domingos. Efficient belief propagation for utility maximization and repeated inference. In AAAI, 2010."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.14778\/1978665.1978669"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.4018\/jswis.2012070103"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0113523"},{"key":"e_1_2_1_33_1","volume-title":"NIPS","author":"Ravikumar P. D.","year":"2008","unstructured":"P. D. Ravikumar , G. Raskutti , M. J. Wainwright , and B. Yu . Model selection in gaussian graphical models: High-dimensional consistency of l1-regularized MLE . In NIPS , 2008 . P. D. Ravikumar, G. Raskutti, M. J. Wainwright, and B. Yu. Model selection in gaussian graphical models: High-dimensional consistency of l1-regularized MLE. In NIPS, 2008."},{"key":"e_1_2_1_34_1","volume-title":"Feature engineering for knowledge base construction","author":"R\u00e9 C.","year":"2014","unstructured":"C. R\u00e9 , A. A. Sadeghian , Z. Shan , J. Shin , F. Wang , S. Wu , and C. Zhang . Feature engineering for knowledge base construction . IEEE Data Eng. Bull ., 2014 . C. R\u00e9, A. A. Sadeghian, Z. Shan, J. Shin, F. Wang, S. Wu, and C. Zhang. Feature engineering for knowledge base construction. IEEE Data Eng. Bull., 2014."},{"key":"e_1_2_1_35_1","volume-title":"Monte Carlo Statistical Methods","author":"Robert C. P.","year":"2005","unstructured":"C. P. Robert and G. Casella . Monte Carlo Statistical Methods . Springer-Verlag New York, Inc. , Secaucus, NJ, USA , 2005 . C. P. Robert and G. Casella. Monte Carlo Statistical Methods. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2005."},{"key":"e_1_2_1_36_1","volume-title":"VLDB","author":"Shen W.","year":"2007","unstructured":"W. Shen , A. Doan , J. F. Naughton , and R. Ramakrishnan . Declarative information extraction using datalog with embedded extraction predicates . In VLDB , 2007 . W. Shen, A. Doan, J. F. Naughton, and R. Ramakrishnan. Declarative information extraction using datalog with embedded extraction predicates. In VLDB, 2007."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.5555\/2031527"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2006.874409"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1561\/2200000001"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807085.1807097"},{"key":"e_1_2_1_41_1","volume-title":"NIPS","author":"Wick M.","year":"2011","unstructured":"M. Wick and A. McCallum . Query-aware MCMC . In NIPS , 2011 . M. Wick and A. McCallum. Query-aware MCMC. In NIPS, 2011."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920942"},{"key":"e_1_2_1_43_1","volume-title":"EMNLP","author":"Yao L.","year":"2010","unstructured":"L. Yao , S. Riedel , and A. McCallum . Collective cross-document relation extraction without labelled data . In EMNLP , 2010 . L. Yao, S. Riedel, and A. McCallum. Collective cross-document relation extraction without labelled data. In EMNLP, 2010."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463702"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732977.2733001"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/2809974.2809991","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:48:36Z","timestamp":1672224516000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/2809974.2809991"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,7]]},"references-count":45,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2015,7]]}},"alternative-id":["10.14778\/2809974.2809991"],"URL":"https:\/\/doi.org\/10.14778\/2809974.2809991","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2015,7]]}}}