{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:00:07Z","timestamp":1760238007976,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2020,7,6]],"date-time":"2020-07-06T00:00:00Z","timestamp":1593993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Increasing concerns about the prevalence of false information and fake news has led to calls for automated fact-checking systems that are capable of verifying the truthfulness of statements, especially on the internet. Most previous automated fact-checking systems have focused on the use of grammar rules only for determining the properties of the language used in statements. Here, we demonstrate a novel approach to the fact-checking of natural language text, which uses a combination of all the following techniques: knowledge extraction to establish a knowledge base, logical inference for fact-checking of claims not explicitly mentioned in the text through the verification of the consistency of a set of beliefs with established trusted knowledge, and a re-querying approach that enables continuous learning. The approach that is presented here addresses the limitations of existing automated fact-checking systems via this novel procedure. This procedure is as follows: the approach investigates the consistency of presented facts or claims while using probabilistic soft logic and a Knowledge Base, which is continuously updated through continuous learning strategies. We demonstrate this approach by focusing on the task of checking facts about family-tree relationships against a corpus of web resources concerned with the UK Royal Family.<\/jats:p>","DOI":"10.3390\/make2030009","type":"journal-article","created":{"date-parts":[[2020,7,6]],"date-time":"2020-07-06T11:07:42Z","timestamp":1594033662000},"page":"147-171","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Claim Consistency Checking Using Soft Logic"],"prefix":"10.3390","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5205-9600","authenticated-orcid":false,"given":"Nouf","family":"Bindris","sequence":"first","affiliation":[{"name":"Intelligent Systems Laboratory, University of Bristol, Bristol BS8 1UB, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nello","family":"Cristianini","sequence":"additional","affiliation":[{"name":"Intelligent Systems Laboratory, University of Bristol, Bristol BS8 1UB, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan","family":"Lawry","sequence":"additional","affiliation":[{"name":"Intelligent Systems Laboratory, University of Bristol, Bristol BS8 1UB, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,7,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ba, M.L., Berti-Equille, L., Shah, K., and Hammady, H.M. (2016, January 11). Vera: A platform for veracity estimation over web data. Proceedings of the 25th International Conference Companion on World Wide Web. International World Wide Web Conferences Steering Committee, Montreal QC, Canada.","DOI":"10.1145\/2872518.2890536"},{"key":"ref_2","unstructured":"Hassan, N., Adair, B., Hamilton, J.T., Li, C., Tremayne, M., Yang, J., and Yu, C. (2015, January 2\u20133). The quest to automate fact-checking. Proceedings of the 2015 Computation+ Journalism Symposium, New York, NY, USA."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1945","DOI":"10.14778\/3137765.3137815","article-title":"ClaimBuster: The first-ever end-to-end fact-checking system","volume":"10","author":"Hassan","year":"2017","journal-title":"Proc. Vldb Endow."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wu, Y., Walenz, B., Li, P., Shim, A., Sonmez, E., Agarwal, P.K., Li, C., Yang, J., and Yu, C. (2014, January 22\u201327). iCheck: Computationally combating lies, d\u2013ned lies, and statistics. Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. ACM, Snowbird, UT, USA.","DOI":"10.1145\/2588555.2594522"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Rashkin, H., Choi, E., Jang, J.Y., Volkova, S., and Choi, Y. (2017, January 7\u201311). Truth of varying shades: Analyzing language in fake news and political fact-checking. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.","DOI":"10.18653\/v1\/D17-1317"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Nakashole, N., and Mitchell, T.M. (2014, January 23\u201325). Language-aware truth assessment of fact candidates. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Baltimore, MD, USA.","DOI":"10.3115\/v1\/P14-1095"},{"key":"ref_7","first-page":"3846","article-title":"Hinge-loss markov random fields and probabilistic soft logic","volume":"18","author":"Bach","year":"2017","journal-title":"J. Machine Learn. Res."},{"key":"ref_8","unstructured":"Duivesteijn, W., Siebes, A., and Ukkonen, A. (2018). Fact Checking from Natural Text with Probabilistic Soft Logic. Advances in Intelligent Data Analysis XVII, Springer."},{"key":"ref_9","unstructured":"Carlson, A., Betteridge, J., Kisiel, B., Settles, B., and Mitchell, T.M. (2010, January 11\u201315). Toward an Architecture for Never-Ending Language Learning. Proceedings of the Twenty-Fourth Conference on Artificial Intelligence (AAAI 2010), Atlanta, GA, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Soleimani, A., Monz, C., and Worring, M. (2020, January 14\u201317). Bert for evidence retrieval and claim verification. Proceedings of the European Conference on Information Retrieval, Lisbon, Portugal.","DOI":"10.1007\/978-3-030-45442-5_45"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ciampaglia, G.L., Shiralkar, P., Rocha, L.M., Bollen, J., Menczer, F., and Flammini, A. (2015). Computational fact checking from knowledge networks. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0141938"},{"key":"ref_12","unstructured":"Thorne, J., and Vlachos, A. (2018). Automated Fact Checking: Task formulations, methods and future directions. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Mihaylova, T., Nakov, P., Marquez, L., Barron-Cedeno, A., Mohtarami, M., Karadzhov, G., and Glass, J. (2018, January 2\u20137). Fact checking in community forums. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11983"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mohtarami, M., Baly, R., Glass, J., Nakov, P., M\u00e0rquez, L., and Moschitti, A. (2018). Automatic Stance Detection Using End-to-End Memory Networks. arXiv.","DOI":"10.18653\/v1\/N18-1070"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Vlachos, A., and Riedel, S. (2014, January 26). Fact Checking: Task definition and dataset construction. Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science, Baltimore, MD, USA.","DOI":"10.3115\/v1\/W14-2508"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Thorne, J., Chen, M., Myrianthous, G., Pu, J., Wang, X., and Vlachos, A. (2017, January 7). Fake news stance detection using stacked ensemble of classifiers. Proceedings of the 2017 EMNLP Workshop: Natural Language Processing meets Journalism, Copenhagen, Denmark.","DOI":"10.18653\/v1\/W17-4214"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Popat, K., Mukherjee, S., Str\u00f6tgen, J., and Weikum, G. (2018, January 23\u201327). CredEye: A credibility lens for analyzing and explaining misinformation. Proceedings of the The Web Conference 2018, Lyon, France.","DOI":"10.1145\/3184558.3186967"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Thorne, J., and Vlachos, A. (2017, January 3\u20137). An extensible framework for verification of numerical claims. Proceedings of the Software Demonstrations of the 15th Conference of the European Chapter of theAssociation for Computational Linguistics. Association for Computational Linguistics, Valencia, Spain.","DOI":"10.18653\/v1\/E17-3010"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Hassan, N., Arslan, F., Li, C., and Tremayne, M. (2017, January 13\u201317). Toward automated fact-checking: Detecting check-worthy factual claims by ClaimBuster. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, Halifax, NS, Canada.","DOI":"10.1145\/3097983.3098131"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Karadzhov, G., Nakov, P., M\u00e0rquez, L., Barron-Cedeno, A., and Koychev, I. (2017). Fully automated fact checking using external sources. arXiv.","DOI":"10.26615\/978-954-452-049-6_046"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Baly, R., Mohtarami, M., Glass, J., M\u00e0rquez, L., Moschitti, A., and Nakov, P. (2018). Integrating stance detection and fact checking in a unified corpus. arXiv.","DOI":"10.18653\/v1\/N18-2004"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Shi, B., and Weninger, T. (2016, January 11). Fact checking in heterogeneous information networks. Proceedings of the 25th International Conference Companion on World Wide Web, Montreal, QC, Canada.","DOI":"10.1145\/2872518.2889354"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Popat, K., Mukherjee, S., Str\u00f6tgen, J., and Weikum, G. (2016, January 24\u201328). Credibility assessment of textual claims on the web. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, Indianapolis, IN, USA.","DOI":"10.1145\/2983323.2983661"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Popat, K., Mukherjee, S., Str\u00f6tgen, J., and Weikum, G. (2017, January 29). Where the truth lies: Explaining the credibility of emerging claims on the web and social media. Proceedings of the 26th International Conference on World Wide Web Companion. International World Wide Web Conferences Steering Committee, Perth, Australia.","DOI":"10.1145\/3041021.3055133"},{"key":"ref_25","unstructured":"Kimmig, A., Bach, S., Broecheler, M., Huang, B., and Getoor, L. (2012, January 8). A short introduction to probabilistic soft logic. Proceedings of the NIPS Workshop on Probabilistic Programming: Foundations and Applications, Harrahs and Harveys, Lake Tahoe, CA, USA."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/s10994-006-5833-1","article-title":"Markov logic networks","volume":"62","author":"Richardson","year":"2006","journal-title":"Mach. Learn."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Boyd, S., Parikh, N., Chu, E., Peleato, B., and Eckstein, J. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends\u00ae in Machine Learning, Now Publishers Inc.","DOI":"10.1561\/2200000016"},{"key":"ref_28","unstructured":"Thakker, D., Osman, T., and Lakin, P. (2009). GATE JAPE Grammar Tutorial, Nottingham Trent University, UK."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1162\/089120101753342653","article-title":"A machine learning approach to coreference resolution of noun phrases","volume":"27","author":"Soon","year":"2001","journal-title":"Comput. Linguist."},{"key":"ref_30","unstructured":"Cunningham, H., Maynard, D., Bontcheva, K., Tablan, V., Aswani, N., Roberts, I., Gorrell, G., Funk, A., Roberts, A., and Damljanovic, D. (2011). Developing Language Processing Components With GATE Version 6 (A User Guide), University of Sheffield."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1007\/s00607-016-0490-0","article-title":"A systematic review and comparative analysis of cross-document coreference resolution methods and tools","volume":"99","author":"Benatallah","year":"2017","journal-title":"Computing"},{"key":"ref_32","unstructured":"Fleischman, M., and Hovy, E. (2004, January 21\u201326). Multi-document person name resolution. Proceedings of the Conference on Reference Resolution and Its Applications, Barcelona, Spain."},{"key":"ref_33","unstructured":"Mayfield, J., Alexander, D., Dorr, B.J., Eisner, J., Elsayed, T., Finin, T., Fink, C., Freedman, M., Garera, N., and McNamee, P. (2009, January 23\u201325). Cross-Document Coreference Resolution: A Key Technology for Learning by Reading. Proceedings of the AAAI Spring Symposium: Learning by Reading and Learning to Read, Palo Alto, CA, USA."},{"key":"ref_34","unstructured":"Ji, H., Grishman, R., Chen, Z., and Gupta, P. (2009, January 14\u201316). Cross-document event extraction and tracking: Task, evaluation, techniques and challenges. Proceedings of the International Conference RANLP-2009, Borovets, Bulgaria."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Dutta, S., and Weikum, G. (2015). Cross-document co-reference resolution using sample-based clustering with knowledge enrichment. Transactions of the Association for Computational Linguistics, MIT Press.","DOI":"10.1162\/tacl_a_00119"},{"key":"ref_36","unstructured":"Hoffart, J., Yosef, M.A., Bordino, I., F\u00fcrstenau, H., Pinkal, M., Spaniol, M., Taneva, B., Thater, S., and Weikum, G. (2011, January 27\u201329). Robust disambiguation of named entities in text. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Edinburgh, UK."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lewis, D.D. (1991, January 19\u201322). Evaluating text categorization. Proceedings of the Speech and Natural Language, Pacific Grove, CL, USA.","DOI":"10.3115\/112405.112471"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Lewis, D.D. (1990, January 24\u201327). Representation quality in text classification: An introduction and experiment. Proceedings of the Speech and Natural Language, Hidden Valley, PA, USA.","DOI":"10.3115\/116580.116681"}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/2\/3\/9\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:48:13Z","timestamp":1760176093000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/2\/3\/9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,6]]},"references-count":38,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["make2030009"],"URL":"https:\/\/doi.org\/10.3390\/make2030009","relation":{},"ISSN":["2504-4990"],"issn-type":[{"type":"electronic","value":"2504-4990"}],"subject":[],"published":{"date-parts":[[2020,7,6]]}}}