{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T11:51:58Z","timestamp":1770897118468,"version":"3.50.1"},"reference-count":25,"publisher":"Cambridge University Press (CUP)","issue":"5-6","license":[{"start":{"date-parts":[[2019,9,20]],"date-time":"2019-09-20T00:00:00Z","timestamp":1568937600000},"content-version":"unspecified","delay-in-days":19,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Theory and Practice of Logic Programming"],"published-print":{"date-parts":[[2019,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The inherent difficulty of knowledge specification and the lack of trained specialists are some of the key obstacles on the way to making intelligent systems based on the knowledge representation and reasoning (KRR) paradigm commonplace.<jats:italic>Knowledge and query authoring<\/jats:italic>using natural language, especially<jats:italic>controlled<\/jats:italic>natural language (CNL), is one of the promising approaches that could enable domain experts, who are not trained logicians, to both create formal knowledge and query it. In previous work, we introduced the<jats:italic>KALM<\/jats:italic>system (Knowledge Authoring Logic Machine) that supports knowledge authoring (and simple querying) with very high accuracy that at present is unachievable via machine learning approaches. The present paper expands on the question answering aspect of KALM and introduces<jats:italic>KALM-QA<\/jats:italic>(KALM for Question Answering) that is capable of answering much more complex English questions. We show that KALM-QA achieves 100% accuracy on an extensive suite of movie-related questions, called<jats:italic>MetaQA<\/jats:italic>, which contains almost 29,000 test questions and over 260,000 training questions. We contrast this with a published machine learning approach, which falls far short of this high mark.<\/jats:p>","DOI":"10.1017\/s1471068419000103","type":"journal-article","created":{"date-parts":[[2019,9,20]],"date-time":"2019-09-20T09:06:21Z","timestamp":1568970381000},"page":"636-653","source":"Crossref","is-referenced-by-count":6,"title":["Querying Knowledge via Multi-Hop English Questions"],"prefix":"10.1017","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2056-5784","authenticated-orcid":false,"given":"TIANTIAN","family":"GAO","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"PAUL","family":"FODOR","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"MICHAEL","family":"KIFER","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2019,9,20]]},"reference":[{"key":"S1471068419000103_ref18","unstructured":"Miller, A. H. , Fisch, A. , Dodge, J. , Karimi, A. , Bordes, A. , and Weston, J. 2016. Key-value memory networks for directly reading documents. In 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), J. Su, X. Carreras, and K. Duh, Eds. The Association for Computational Linguistics, Austin, TX, 1400\u20131409."},{"key":"S1471068419000103_ref5","unstructured":"Fuchs, N. E. , Kaljurand, K. , and Kuhn, T. 2008. Attempto controlled english for knowledge representation. In Reasoning Web. Springer, Venice, Italy, 104\u2013124."},{"key":"S1471068419000103_ref24","unstructured":"Zhang, Y. , Dai, H. , Kozareva, Z. , Smola, A. J. , and Song, L. 2018a. The MetaQA dataset. https:\/\/github.com\/yuyuz\/MetaQA."},{"key":"S1471068419000103_ref4","unstructured":"Fillmore, C. J. and Baker, C. F. 2001. Frame semantics for text understanding. In Proceedings of WordNet and Other Lexical Resources Workshop. NAACL, Pittsburgh, USA."},{"key":"S1471068419000103_ref21","unstructured":"Ringgaard, M. , Gupta, R. , and Pereira, F. C. N. 2017. SLING: A framework for frame semantic parsing. CoRR 1710.07032, 1\u20139."},{"key":"S1471068419000103_ref22","first-page":"1209","article-title":"ATHENA: an ontology-driven system for natural language querying over relational data stores","volume":"12","author":"Saha","year":"2016","journal-title":"PVLDB 9"},{"key":"S1471068419000103_ref12","unstructured":"Johnson, C. R. , Fillmore, C. J. , Petruck, M. R. , Baker, C. F. , Ellsworth, M. J. , Ruppenhofer, J. , and Wood, E. J. 2002. FrameNet: Theory and Practice."},{"key":"S1471068419000103_ref15","doi-asserted-by":"crossref","first-page":"249","DOI":"10.3233\/SW-2011-0030","article-title":"PowerAqua: Supporting users in querying and exploring the semantic web","volume":"3","author":"L\u00f3pez","year":"2012","journal-title":"Semantic Web 3"},{"key":"S1471068419000103_ref20","unstructured":"Object-Management-Group. 2017. Semantics of business vocabulary and business rules (SBVR), v. 1.4. OMG standards document. http:\/\/www.omg.org\/spec\/SBVR\/Current."},{"key":"S1471068419000103_ref7","unstructured":"Gao, T. , Fodor, P. , and Kifer, M. 2018a. High accuracy question answering via hybrid controlled natural language. In 2018 IEEE\/WIC\/ACM International Conference on Web Intelligence (WI2018). IEEE, Santiago, Chile, 17\u201324."},{"key":"S1471068419000103_ref1","first-page":"344","volume-title":"53rd Annual Meeting of the Association for Computational Linguistics","author":"Angeli","year":"2015"},{"key":"S1471068419000103_ref2","doi-asserted-by":"crossref","first-page":"615","DOI":"10.3115\/v1\/D14-1067","volume-title":"2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Bordes","year":"2014"},{"key":"S1471068419000103_ref17","unstructured":"Mausam , Schmitz, M. , Soderland, S. , Bart, R. , and Etzioni, O. 2012. Open language learning for information extraction. In The Joint Conf. on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL. Association for Computational Linguistics, Jeju Island, Korea, 523\u2013534."},{"key":"S1471068419000103_ref14","first-page":"73","article-title":"Constructing an interactive natural language interface for relational databases","volume":"1","author":"Li","year":"2014","journal-title":"PVLDB 8"},{"key":"S1471068419000103_ref11","unstructured":"Gomez, F. , Hull, R. D. , and Segami, C. 1994. Acquiring knowledge from encyclopedic texts. In 4th Applied Natural Language Processing Conference (ANLP). ACL, Stuttgart, Germany, 84\u201390."},{"key":"S1471068419000103_ref16","doi-asserted-by":"crossref","unstructured":"Manning, C. D. , Surdeanu, M. , Bauer, J. , Finkel, J. , Bethard, S. J. , and McClosky, D. 2014. The Stanford CoreNLP natural language processing toolkit.","DOI":"10.3115\/v1\/P14-5010"},{"key":"S1471068419000103_ref10","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-69858-6_6"},{"key":"S1471068419000103_ref8","unstructured":"Gao, T. , Fodor, P. , and Kifer, M. 2018b. Knowledge authoring for rule-based reasoning. In On the Move to Meaningful Internet Systems. OTM 2018 Conferences - Confederated International Conferences: CoopIS, C&TC, and ODBASE 2018, H. Panetto, C. Debruyne, H. A. Proper, C. A. Ardagna, D. Roman, and R. Meersman, Eds. Lecture Notes in Computer Science, vol. 11230. Springer, Valletta, Malta, 461\u2013480."},{"key":"S1471068419000103_ref23","unstructured":"Schwitter, R. 2010. Controlled natural languages for knowledge representation. In COLING 2010, 23rd Intl. Conf. on Computational Linguistics, Posters Volume, 23-27. ACL, Beijing, China, 1113\u20131121."},{"key":"S1471068419000103_ref19","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.artint.2012.07.001","article-title":"BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network","volume":"193","author":"Navigli","year":"2012","journal-title":"Artificial Intelligence"},{"key":"S1471068419000103_ref6","unstructured":"Gao, T. 2019. Development of KALM-QA. Stony Brook University. https:\/\/github.com\/tiantiangao7\/kalm-qa."},{"key":"S1471068419000103_ref25","unstructured":"Zhang, Y. , Dai, H. , Kozareva, Z. , Smola, A. J. , and Song, L. 2018b. Variational reasoning for question answering with knowledge graph. See McIlraith and Weinberger (2018), 6069\u20136076."},{"key":"S1471068419000103_ref9","unstructured":"Gao, T. , Fodor, P. , and Kifer, M. 2019. Querying Knowledge via Multi-Hop English Questions. ArXiv e-prints abs\/1907.08176, 1\u201319."},{"key":"S1471068419000103_ref3","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1162\/COLI_a_00163","article-title":"Frame-semantic parsing","volume":"1","author":"Das","year":"2014","journal-title":"Comp, Linguistics 40"},{"key":"S1471068419000103_ref13","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00168"}],"container-title":["Theory and Practice of Logic Programming"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S1471068419000103","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,12,7]],"date-time":"2019-12-07T12:21:54Z","timestamp":1575721314000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S1471068419000103\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9]]},"references-count":25,"journal-issue":{"issue":"5-6","published-print":{"date-parts":[[2019,9]]}},"alternative-id":["S1471068419000103"],"URL":"https:\/\/doi.org\/10.1017\/s1471068419000103","relation":{},"ISSN":["1471-0684","1475-3081"],"issn-type":[{"value":"1471-0684","type":"print"},{"value":"1475-3081","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,9]]}}}