{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:13:30Z","timestamp":1760235210634,"version":"build-2065373602"},"reference-count":28,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,8,6]],"date-time":"2021-08-06T00:00:00Z","timestamp":1628208000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>In this paper, we propose a fully automated system to extend knowledge graphs using external information from web-scale corpora. The designed system leverages a deep-learning-based technology for relation extraction that can be trained by a distantly supervised approach. In addition, the system uses a deep learning approach for knowledge base completion by utilizing the global structure information of the induced KG to further refine the confidence of the newly discovered relations. The designed system does not require any effort for adaptation to new languages and domains as it does not use any hand-labeled data, NLP analytics, and inference rules. Our experiments, performed on a popular academic benchmark, demonstrate that the suggested system boosts the performance of relation extraction by a wide margin, reporting error reductions of 50%, resulting in relative improvement of up to 100%. Furthermore, a web-scale experiment conducted to extend DBPedia with knowledge from Common Crawl shows that our system is not only scalable but also does not require any adaptation cost, while yielding a substantial accuracy gain.<\/jats:p>","DOI":"10.3390\/info12080316","type":"journal-article","created":{"date-parts":[[2021,8,6]],"date-time":"2021-08-06T08:01:42Z","timestamp":1628236902000},"page":"316","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Populating Web-Scale Knowledge Graphs Using Distantly Supervised Relation Extraction and Validation"],"prefix":"10.3390","volume":"12","author":[{"given":"Sarthak","family":"Dash","sequence":"first","affiliation":[{"name":"IBM Research AI, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael R.","family":"Glass","sequence":"additional","affiliation":[{"name":"IBM Research AI, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alfio","family":"Gliozzo","sequence":"additional","affiliation":[{"name":"IBM Research AI, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mustafa","family":"Canim","sequence":"additional","affiliation":[{"name":"IBM Research AI, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1042-4782","authenticated-orcid":false,"given":"Gaetano","family":"Rossiello","sequence":"additional","affiliation":[{"name":"IBM Research AI, IBM Thomas J. Watson Research Center, Yorktown Heights, NY 10598, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,6]]},"reference":[{"key":"ref_1","first-page":"25","article-title":"DeepDive: Web-scale Knowledge-base Construction using Statistical Learning and Inference","volume":"Volume 884","author":"Brambilla","year":"2012","journal-title":"Proceedings of the Second International Workshop on Searching and Integrating New Web Data Sources"},{"key":"ref_2","unstructured":"Nakashole, N. (2013). Automatic Extraction of Facts, Relations, and Entities for Web-Scale Knowledge Base Population. [Ph.D. Thesis, Saarland University]."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Glass, M., Gliozzo, A., Hassanzadeh, O., Mihindukulasooriya, N., and Rossiello, G. (2018, January 8\u201312). Inducing Implicit Relations from Text using Distantly Supervised Deep Nets. Proceedings of the International Semantic Web Conference, Monterey, CA, USA.","DOI":"10.1007\/978-3-030-00671-6_3"},{"key":"ref_4","unstructured":"Zeng, D., Liu, K., Lai, S., Zhou, G., and Zhao, J. (2014, January 23\u201329). Relation classification via convolutional deep neural network. Proceedings of the COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, Dublin, Ireland."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Xu, Y., Mou, L., Li, G., Chen, Y., Peng, H., and Jin, Z. (2015, January 17\u201321). Classifying relations via long short term memory networks along shortest dependency paths. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1206"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zeng, W., Lin, Y., Liu, Z., and Sun, M. (2016). Incorporating Relation Paths in Neural Relation Extraction. arXiv.","DOI":"10.18653\/v1\/D17-1186"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zeng, D., Liu, K., Chen, Y., and Zhao, J. (2015, January 17\u201321). Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks. Proceedings of the EMNLP, Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1203"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Lin, Y., Shen, S., Liu, Z., Luan, H., and Sun, M. (2016, January 7\u201312). Neural relation extraction with selective attention over instances. Proceedings of the ACL, Berlin, Germany.","DOI":"10.18653\/v1\/P16-1200"},{"key":"ref_9","unstructured":"Riedel, S., Yao, L., McCallum, A., and Marlin, B.M. (2013, January 9\u201314). Relation extraction with matrix factorization and universal schemas. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, GA, USA."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/s10994-006-5833-1","article-title":"Markov Logic networks","volume":"62","author":"Richardson","year":"2006","journal-title":"Mach. Learn."},{"key":"ref_11","first-page":"3","article-title":"Toward an architecture for never-ending language learning","volume":"Volume 5","author":"Carlson","year":"2010","journal-title":"Twenty-Fourth AAAI Conference on Artificial Intelligence"},{"key":"ref_12","unstructured":"Pujara, J., Miao, H., Getoor, L., and Cohen, W. (2013, January 21\u201325). Knowledge graph identification. Proceedings of the International Semantic Web Conference, Sydney, Australia."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.websem.2015.08.001","article-title":"Defacto\u2014temporal and multilingual deep fact validation","volume":"35","author":"Gerber","year":"2015","journal-title":"J. Web Semant."},{"key":"ref_14","first-page":"2787","article-title":"Translating embeddings for modeling multi-relational data","volume":"26","author":"Bordes","year":"2013","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_15","unstructured":"Nickel, M., Tresp, V., and Kriegel, H.P. (2021, August 06). A Three-Way Model for Collective Learning on Multi-Relational Data. Available online: https:\/\/openreview.net\/forum?id=H14QEiZ_WS."},{"key":"ref_16","unstructured":"Socher, R., Chen, D., Manning, C.D., and Ng, A. (2021, August 06). Reasoning with Neural Tensor Networks for Knowledge Base Completion. Available online: http:\/\/papers.nips.cc\/paper\/5028-reasoning-with-neural-tenten-sor-networks-for-knowledge-base-completion.pdf."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Nickel, M., Rosasco, L., and Poggio, T.A. (2016). Holographic Embeddings of Knowledge Graphs, AAAI.","DOI":"10.1609\/aaai.v30i1.10314"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Dettmers, T., Minervini, P., Stenetorp, P., and Riedel, S. (2017). Convolutional 2d knowledge graph embeddings. arXiv.","DOI":"10.1609\/aaai.v32i1.11573"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Nguyen, D.Q., Nguyen, T.D., Nguyen, D.Q., and Phung, D. (2017). A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network. arXiv.","DOI":"10.18653\/v1\/N18-2053"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Cai, L., and Wang, W.Y. (2017). KBGAN: Adversarial Learning for Knowledge Graph Embeddings. arXiv.","DOI":"10.18653\/v1\/N18-1133"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Shi, B., and Weninger, T. (2017). ProjE: Embedding Projection for Knowledge Graph Completion, AAAI.","DOI":"10.1609\/aaai.v31i1.10677"},{"key":"ref_22","unstructured":"Gong, Y., Jia, Y., Leung, T., Toshev, A., and Ioffe, S. (2013). Deep convolutional ranking for multilabel image annotation. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Riedel, S., Yao, L., and McCallum, A. (2010). Modeling relations and their mentions without labeled text. Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer.","DOI":"10.1007\/978-3-642-15939-8_10"},{"key":"ref_24","unstructured":"Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., and Weld, D.S. (2011). Knowledge-based weak supervision for information extraction of overlapping relations. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, Association for Computational Linguistics."},{"key":"ref_25","unstructured":"Surdeanu, M., Tibshirani, J., Nallapati, R., and Manning, C.D. (2012). Multi-instance Multi-label Learning for Relation Extraction. Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Association for Computational Linguistics."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Glass, M., and Gliozzo, A. (2018, January 3\u20137). A Dataset for Web-scale Knowledge Base Population. Proceedings of the 15th Extended Semantic Web Conference, Heraklion, Greece.","DOI":"10.1007\/978-3-319-93417-4_17"},{"key":"ref_27","unstructured":"Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., and Ives, Z. (2017, January 11\u201315). DBpedia: A Nucleus for a Web of Open Data. Proceedings of the 6th Int\u2019l Semantic Web Conference, Busan, Korea."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jiang, S., Lowd, D., and Dou, D. (2012, January 10\u201313). Learning to Refine an Automatically Extracted Knowledge Base Using Markov Logic. Proceedings of the 2012 IEEE 12th International Conference on Data Mining, Brussels, Belgium.","DOI":"10.1109\/ICDM.2012.156"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/8\/316\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:41:45Z","timestamp":1760164905000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/8\/316"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,6]]},"references-count":28,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["info12080316"],"URL":"https:\/\/doi.org\/10.3390\/info12080316","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2021,8,6]]}}}