{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,31]],"date-time":"2026-10-31T19:46:57Z","timestamp":1793476017992,"version":"4.5.0"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2016,6,2]],"date-time":"2016-06-02T00:00:00Z","timestamp":1464825600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGMOD Rec."],"published-print":{"date-parts":[[2016,6,2]]},"abstract":"<jats:p>The dark data extraction or knowledge base construction (KBC) problem is to populate a SQL database with information from unstructured data sources including emails, webpages, and pdf reports. KBC is a long-standing problem in industry and research that encompasses problems of data extraction, cleaning, and integration. We describe DeepDive, a system that combines database and machine learning ideas to help develop KBC systems. The key idea in DeepDive is that statistical inference and machine learning are key tools to attack classical data problems in extraction, cleaning, and integration in a unified and more effective manner. DeepDive programs are declarative in that one cannot write probabilistic inference algorithms; instead, one interacts by defining features or rules about the domain. A key reason for this design choice is to enable domain experts to build their own KBC systems. We present the applications, abstractions, and techniques of DeepDive employed to accelerate construction of KBC systems.<\/jats:p>","DOI":"10.1145\/2949741.2949756","type":"journal-article","created":{"date-parts":[[2016,6,3]],"date-time":"2016-06-03T08:14:26Z","timestamp":1464941666000},"page":"60-67","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":66,"title":["DeepDive"],"prefix":"10.1145","volume":"45","author":[{"given":"Christopher","family":"De Sa","sequence":"first","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alex","family":"Ratner","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"R\u00e9","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jaeho","family":"Shin","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feiran","family":"Wang","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sen","family":"Wu","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ce","family":"Zhang","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2016,6,2]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"TAC KBP","author":"Angeli G.","year":"2014","unstructured":"G. Angeli 2014 slot filling systems . TAC KBP , 2014. G. Angeli et al. Stanford's 2014 slot filling systems. TAC KBP, 2014."},{"key":"e_1_2_1_2_1","volume-title":"IJCAI","author":"Banko M.","year":"2007","unstructured":"M. Banko Open information extraction from the Web . In IJCAI , 2007 . M. Banko et al. Open information extraction from the Web. In IJCAI, 2007."},{"key":"e_1_2_1_3_1","volume-title":"AAAI Spring Symposium","author":"Betteridge J.","year":"2009","unstructured":"J. Betteridge , A. Carlson , S. A. Hong , E. R. Hruschka Jr , E. L. Law , T. M. Mitchell , and S. H. Wang . Toward never ending language learning . In AAAI Spring Symposium , 2009 . J. Betteridge, A. Carlson, S. A. Hong, E. R. Hruschka Jr, E. L. Law, T. M. Mitchell, and S. H. Wang. Toward never ending language learning. In AAAI Spring Symposium, 2009."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.5555\/646543.696220"},{"key":"e_1_2_1_5_1","volume-title":"IBM Research Report","author":"Brown E.","year":"2013","unstructured":"E. Brown Tools and methods for building watson . IBM Research Report , 2013 . E. Brown et al. Tools and methods for building watson. IBM Research Report, 2013."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/2898607.2898816"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2008.4497503"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2012.60"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2610516"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/1855041"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732951.2732962"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/988672.988687"},{"key":"e_1_2_1_13_1","volume-title":"AI Magazine","author":"Ferrucci D.","year":"2010","unstructured":"D. Ferrucci : An overview of the DeepQA project . AI Magazine , 2010 . D. Ferrucci et al. Building Watson: An overview of the DeepQA project. AI Magazine, 2010."},{"key":"e_1_2_1_14_1","volume-title":"ACL","author":"Govindaraju V.","year":"2013","unstructured":"V. Govindaraju Understanding tables in context using standard NLP toolkits . In ACL , 2013 . V. Govindaraju et al. Understanding tables in context using standard NLP toolkits. In ACL, 2013."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/170036.170066"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.3115\/992133.992154"},{"key":"e_1_2_1_17_1","volume-title":"ACL","author":"Hoffmann R.","year":"2011","unstructured":"R. Hoffmann Knowledge-based weak supervision for information extraction of overlapping relations . In ACL , 2011 . R. Hoffmann et al. Knowledge-based weak supervision for information extraction of overlapping relations. In ACL, 2011."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376686"},{"key":"e_1_2_1_19_1","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511790423","volume-title":"Probability Theory: The Logic of Science","author":"Jaynes E. T.","year":"2003","unstructured":"E. T. Jaynes . Probability Theory: The Logic of Science . Cambridge University Press , 2003 . E. T. Jaynes. Probability Theory: The Logic of Science. Cambridge University Press, 2003."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2012.156"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1519103.1519110"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.14778\/1952376.1952380"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1519103.1519105"},{"key":"e_1_2_1_24_1","volume-title":"HLT","author":"Li Y.","year":"2011","unstructured":"Y. Li , F. R. Reiss , and L. Chiticariu . SystemT: A declarative information extraction system . In HLT , 2011 . Y. Li, F. R. Reiss, and L. Chiticariu. SystemT: A declarative information extraction system. In HLT, 2011."},{"key":"e_1_2_1_25_1","volume-title":"ICML","author":"Liu J.","year":"2014","unstructured":"J. Liu and et al. An asynchronous parallel stochastic coordinate descent algorithm . ICML , 2014 . J. Liu and et al. An asynchronous parallel stochastic coordinate descent algorithm. ICML, 2014."},{"key":"e_1_2_1_26_1","volume-title":"CIDR","author":"Madhavan J.","year":"2007","unstructured":"J. Madhavan Web-scale data integration: You can only afford to pay as you go . In CIDR , 2007 . J. Madhavan et al. Web-scale data integration: You can only afford to pay as you go. In CIDR, 2007."},{"key":"e_1_2_1_27_1","volume-title":"Bioinformatics","author":"Mallory E. K.","year":"2015","unstructured":"E. K. Mallory Large-scale extraction of gene interactions from full text literature using deepdive . Bioinformatics , 2015 . E. K. Mallory et al. Large-scale extraction of gene interactions from full text literature using deepdive. Bioinformatics, 2015."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/1690219.1690287"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935869"},{"key":"e_1_2_1_30_1","volume-title":"NIPS","author":"Niu F.","year":"2011","unstructured":"F. Niu : A lock-free approach to parallelizing stochastic gradient descent . In NIPS , 2011 . F. Niu et al. Hogwild!: A lock-free approach to parallelizing stochastic gradient descent. In NIPS, 2011."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.14778\/1978665.1978669"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.4018\/jswis.2012070103"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2012.96"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0113523"},{"key":"e_1_2_1_35_1","volume-title":"AAAI","author":"Poon H.","year":"2007","unstructured":"H. Poon and P. Domingos . Joint inference in information extraction . In AAAI , 2007 . H. Poon and P. Domingos. Joint inference in information extraction. In AAAI, 2007."},{"key":"e_1_2_1_36_1","volume-title":"Feature engineering for knowledge base construction","author":"R\u00e9 C.","year":"2014","unstructured":"C. R\u00e9 Feature engineering for knowledge base construction . IEEE Data Eng. Bull ., 2014 . C. R\u00e9 et al. Feature engineering for knowledge base construction. IEEE Data Eng. Bull., 2014."},{"key":"e_1_2_1_37_1","volume-title":"Monte Carlo Statistical Methods","author":"Robert C. P.","year":"2005","unstructured":"C. P. Robert and G. Casella . Monte Carlo Statistical Methods . Springer-Verlag New York, Inc. , Secaucus, NJ, USA , 2005 . C. P. Robert and G. Casella. Monte Carlo Statistical Methods. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2005."},{"key":"e_1_2_1_38_1","volume-title":"VLDB","author":"Shen W.","year":"2007","unstructured":"W. Shen Declarative information extraction using datalog with embedded extraction predicates . In VLDB , 2007 . W. Shen et al. Declarative information extraction using datalog with embedded extraction predicates. In VLDB, 2007."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.14778\/2809974.2809991"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1526709.1526794"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2006.874409"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1561\/2200000001"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807085.1807097"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920942"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.5555\/1614164.1614177"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463680"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463702"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732977.2733001"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1526709.1526724"},{"key":"e_1_2_1_50_1","first-page":"2595","volume-title":"NIPS","author":"Zinkevich M.","year":"2010","unstructured":"M. Zinkevich and et al. Parallelized stochastic gradient descent . In NIPS , pages 2595 -- 2603 , 2010 . M. Zinkevich and et al. Parallelized stochastic gradient descent. In NIPS, pages 2595--2603, 2010."}],"container-title":["ACM SIGMOD Record"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2949741.2949756","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2949741.2949756","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:56:00Z","timestamp":1750208160000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2949741.2949756"}},"subtitle":["Declarative Knowledge Base Construction"],"short-title":[],"issued":{"date-parts":[[2016,6,2]]},"references-count":50,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2016,6,2]]}},"alternative-id":["10.1145\/2949741.2949756"],"URL":"https:\/\/doi.org\/10.1145\/2949741.2949756","relation":{},"ISSN":["0163-5808"],"issn-type":[{"value":"0163-5808","type":"print"}],"subject":[],"published":{"date-parts":[[2016,6,2]]},"assertion":[{"value":"2016-06-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}