{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T16:07:33Z","timestamp":1781626053454,"version":"3.54.5"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2018,11,19]],"date-time":"2018-11-19T00:00:00Z","timestamp":1542585600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2019,9,30]]},"abstract":"<jats:p>\n            Relation extraction is a subtask of information extraction where\n            <jats:italic>semantic relationships<\/jats:italic>\n            are extracted from natural language text and then classified. In essence, it allows us to acquire structured knowledge from unstructured text. In this article, we present a survey of relation extraction methods that leverage pre-existing structured or semi-structured data to guide the extraction process. We introduce a taxonomy of existing methods and describe distant supervision approaches in detail. We describe, in addition, the evaluation methodologies and the datasets commonly used for quality assessment. Finally, we give a high-level outlook on the field, highlighting open problems as well as the most promising research directions.\n          <\/jats:p>","DOI":"10.1145\/3241741","type":"journal-article","created":{"date-parts":[[2018,11,20]],"date-time":"2018-11-20T14:19:23Z","timestamp":1542723563000},"page":"1-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":111,"title":["Relation Extraction Using Distant Supervision"],"prefix":"10.1145","volume":"51","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7108-9917","authenticated-orcid":false,"given":"Alisa","family":"Smirnova","sequence":"first","affiliation":[{"name":"eXascale Infolab, University of Fribourg\u2014Switzerland, Fribourg, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2588-4212","authenticated-orcid":false,"given":"Philippe","family":"Cudr\u00e9-Mauroux","sequence":"additional","affiliation":[{"name":"eXascale Infolab, University of Fribourg\u2014Switzerland, Fribourg, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,11,19]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/336597.336644"},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the 25th International Conference on Computational Linguistics (COLING\u201914)","author":"Akbik Alan","year":"2014","unstructured":"Alan Akbik, Thilo Michael, and Christoph Boden. 2014. Exploratory relation extraction in large text corpora. In Proceedings of the 25th International Conference on Computational Linguistics (COLING\u201914). 2087--2096."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.5555\/2390665.2390679"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201914)","author":"Angeli Gabor","unstructured":"Gabor Angeli, Julie Tibshirani, Jean Wu, and Christopher D. Manning. 2014. Combining distant and partial supervision for relation extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201914). 1556--1567."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.3233\/SW-150180"},{"key":"e_1_2_1_6_1","volume-title":"A survey on relation extraction","author":"Bach Nguyen","unstructured":"Nguyen Bach and Sameer Badaskar. 2007. A survey on relation extraction. Language Technologies Institute, Carnegie Mellon University."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/1625275.1625705"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/944919.944937"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/646543.696220"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1165"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/320434.320440"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/645634.663209"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187836.2187900"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1158\/2159-8290.CD-14-1118"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186000"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409360.1409378"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.5555\/1090483.1644538"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/2283396.2283398"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.5555\/2145432.2145596"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL\u201914)","author":"Fan Miao","unstructured":"Miao Fan, Deli Zhao, Qiang Zhou, Zhiyuan Liu, Thomas Fang Zheng, and Edward Y. Chang. 2014. Distant supervision for relation extraction with matrix completion. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL\u201914). 839--849."},{"key":"e_1_2_1_21_1","unstructured":"Evgeniy Gabrilovich Michael Ringgaard and Amarnag Subramanya. 2013. FACC1: Freebase annotation of ClueWeb corpora Version 1. Retrieved from http:\/\/lemurproject.org\/clueweb09\/FACC1\/Cited by."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/3016100.3016315"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2010019"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.5555\/2002472.2002541"},{"key":"e_1_2_1_26_1","volume-title":"Overview of the TAC2011 knowledge base population track. In Proceedings of the Text Analysis Conference.","author":"Ji Heng","year":"2011","unstructured":"Heng Ji, Ralph Grishman, and Hoa Dang. 2011. Overview of the TAC2011 knowledge base population track. In Proceedings of the Text Analysis Conference."},{"key":"e_1_2_1_27_1","volume-title":"Overview of the TAC 2010 knowledge base population track. In Proceedings of the Text Analysis Conference.","author":"Ji Heng","year":"2010","unstructured":"Heng Ji, Ralph Grishman, Hoa Trang Dang, Kira Griffitt, and Joe Ellis. 2010. Overview of the TAC 2010 knowledge base population track. In Proceedings of the Text Analysis Conference."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/1572340.1572342"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1181"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC\u201914)","author":"Kirschnick Johannes","year":"2014","unstructured":"Johannes Kirschnick, Alan Akbik, and Holmer Hemsen. 2014. Freepal: A large collection of deep lexico-syntactic patterns for relation extraction. In Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC\u201914). 2071--2075."},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201914). 1891","author":"Koch Mitchell","year":"1901","unstructured":"Mitchell Koch, John Gilmer, Stephen Soderland, and Daniel S. Weld. 2014. Type-aware distantly supervised relation extraction with linked arguments. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201914). 1891--1901."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00152"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.5555\/1005332.1005345"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1200"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-009-0306-5"},{"key":"e_1_2_1_36_1","unstructured":"Tomas Mikolov Kai Chen Greg Corrado and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv Preprint arXiv:1301.3781."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458150"},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the Human Language Technologies Conference of the North American Chapter of the Association of Computational Linguistics. 777--782","author":"Min Bonan","year":"2013","unstructured":"Bonan Min, Ralph Grishman, Li Wan, Chang Wang, and David Gondek. 2013. Distant supervision for relation extraction with an incomplete knowledge base. In Proceedings of the Human Language Technologies Conference of the North American Chapter of the Association of Computational Linguistics. 777--782."},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the 5th Text Analysis Conference (TAC\u201912)","author":"Min Bonan","year":"2012","unstructured":"Bonan Min, Xiang Li, Ralph Grishman, and Ang Sun. 2012. New york university 2012 system for KBP slot filling. In Proceedings of the 5th Text Analysis Conference (TAC\u201912)."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/1690219.1690287"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.5555\/2976248.2976270"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00179"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W15-1506"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.5555\/3104482.3104584"},{"key":"e_1_2_1_45_1","volume-title":"Inductive Dependency Parsing","author":"Nivre Joakim","unstructured":"Joakim Nivre. 2006. Inductive Dependency Parsing, Vol. 34. Springer."},{"key":"e_1_2_1_46_1","unstructured":"Nanyun Peng Hoifung Poon Chris Quirk Kristina Toutanova and Wen-tau Yih. 2017. Cross-sentence N-ary relation extraction with graph LSTMs. Trans. Assoc. Comput. Linguist. (2017). arXiv preprint arXiv:1708.03743."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-2119"},{"key":"e_1_2_1_48_1","volume-title":"Proceedings of the Pacific Symposium on Biocomputing. 120--131","author":"Poon Hoifung","year":"2015","unstructured":"Hoifung Poon, Kristina Toutanova, and Chris Quirk. 2015. Distant supervision for cancer pathway extraction from text. In Proceedings of the Pacific Symposium on Biocomputing. 120--131."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.3115\/1073083.1073092"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.5555\/1889788.1889799"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the Human Language Technologies Conference of the North American Chapter of the Association of Computational Linguistics. 74--84","author":"Riedel Sebastian","unstructured":"Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M. Marlin. 2013. Relation extraction with matrix factorization and universal schemas. In Proceedings of the Human Language Technologies Conference of the North American Chapter of the Association of Computational Linguistics. 74--84."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00234"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/N15-1118"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2509558.2509571"},{"key":"e_1_2_1_55_1","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201913)","author":"Roth Benjamin","year":"2013","unstructured":"Benjamin Roth and Dietrich Klakow. 2013. Combining generative and discriminative model scores for distant supervision. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201913). 24--29."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2505515.2507850"},{"key":"e_1_2_1_57_1","volume-title":"Proceedings of the Linguistic Data Consortium.","author":"Sandhaus Evan","year":"2008","unstructured":"Evan Sandhaus. 2008. The new york times annotated corpus. Proceedings of the Linguistic Data Consortium."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkn653"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2014.2327028"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.3115\/1220835.1220874"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.5555\/2390948.2391003"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.5555\/2390524.2390626"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-41335-3_40"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1174"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.5555\/2145432.2145583"},{"key":"e_1_2_1_66_1","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP\u201913)","author":"Weston Jason","year":"2013","unstructured":"Jason Weston, Antoine Bordes, Oksana Yakhnenko, and Nicolas Usunier. 2013. Connecting language and knowledge bases with embedding models for relation extraction. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP\u201913). 1366--1371."},{"key":"e_1_2_1_67_1","volume-title":"Proceedings of the Workshop on Advances in Ranking: Neural Information Processing Systems (NIPS\u201909)","author":"Wick Michael","year":"2009","unstructured":"Michael Wick, Khashayar Rohanimanesh, Aron Culotta, and Andrew McCallum. 2009. Samplerank: Learning preferences from atomic gradients. In Proceedings of the Workshop on Advances in Ranking: Neural Information Processing Systems (NIPS\u201909)."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/1321440.1321449"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.5555\/1858681.1858694"},{"key":"e_1_2_1_70_1","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL\u201913)","author":"Xu Wei","year":"2013","unstructured":"Wei Xu, Raphael Hoffmann, Le Zhao, and Ralph Grishman. 2013. Filling knowledge base gaps for distant supervision of relation extraction. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL\u201913). 665--670."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.5555\/2145432.2145587"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.5555\/944919.944964"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1203"},{"key":"e_1_2_1_74_1","volume-title":"Proceedings of the 25th International Conference on Computational Linguistics (COLING\u201914)","author":"Zeng Daojian","unstructured":"Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao et al. 2014. Relation classification via convolutional deep neural network. In Proceedings of the 25th International Conference on Computational Linguistics (COLING\u201914). 2335--2344."},{"key":"e_1_2_1_75_1","volume-title":"DeepDive: A data management system for automatic knowledge base construction","author":"Zhang Ce","unstructured":"Ce Zhang. 2015. DeepDive: A data management system for automatic knowledge base construction. University of Wisconsin-Madison, Madison, Wisconsin."}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3241741","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3241741","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:08:10Z","timestamp":1750212490000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3241741"}},"subtitle":["A Survey"],"short-title":[],"issued":{"date-parts":[[2018,11,19]]},"references-count":75,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2019,9,30]]}},"alternative-id":["10.1145\/3241741"],"URL":"https:\/\/doi.org\/10.1145\/3241741","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,11,19]]},"assertion":[{"value":"2018-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-11-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}