{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T16:28:06Z","timestamp":1783009686120,"version":"3.54.5"},"reference-count":45,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2019,9,20]],"date-time":"2019-09-20T00:00:00Z","timestamp":1568937600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012659","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61772059, 61421003"],"award-info":[{"award-number":["61772059, 61421003"]}],"id":[{"id":"10.13039\/501100012659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012503","name":"Ministry of Science and Technology of the People's Republic of China","doi-asserted-by":"publisher","award":["No. 2015CB358700"],"award-info":[{"award-number":["No. 2015CB358700"]}],"id":[{"id":"10.13039\/501100012503","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011347","name":"State Key Laboratory of Software Development Environment","doi-asserted-by":"publisher","award":["SKLSDE-2018ZX-17"],"award-info":[{"award-number":["SKLSDE-2018ZX-17"]}],"id":[{"id":"10.13039\/501100011347","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2019,10,31]]},"abstract":"<jats:p>Question answering over knowledge bases aims to take full advantage of the information in knowledge bases with the ultimate purpose of returning answers to questions. To access the substantial knowledge within the KB, many model architectures are hindered by the bottleneck of accurately predicting relations that connect subject entities in questions to object entities in the knowledge base. To break the bottleneck, this article presents a novel model architecture, APVA, which includes a verification mechanism to check the correctness of predicted relations. Specifically, APVA takes advantage of KB-based information to improve relation prediction but verifies the correctness of the predicted relation by means of simple negative sampling in a logistic regression framework. The APVA architecture offers a natural way to integrate an iterative training procedure, which we call turbo training. Accordingly, we introduce APVA-TURBO to perform question answering over knowledge bases. We demonstrate extensive experiments to show that APVA-TURBO outperforms existing approaches on question answering.<\/jats:p>","DOI":"10.1145\/3345557","type":"journal-article","created":{"date-parts":[[2019,9,20]],"date-time":"2019-09-20T13:37:15Z","timestamp":1568986635000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Question Answering in Knowledge Bases"],"prefix":"10.1145","volume":"37","author":[{"given":"Richong","family":"Zhang","sequence":"first","affiliation":[{"name":"BDBC and SKLSDE, School of Computer Science and Engineering, Beihang University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yue","family":"Wang","sequence":"additional","affiliation":[{"name":"BDBC and SKLSDE, School of Computer Science and Engineering, Beihang University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongyi","family":"Mao","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinpeng","family":"Huai","sequence":"additional","affiliation":[{"name":"BDBC and SKLSDE, School of Computer Science and Engineering, Beihang University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,9,20]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Dbpedia: A nucleus for a web of open data. In The Semantic Web","author":"Auer S\u00f6ren","year":"2007","unstructured":"S\u00f6ren Auer , Christian Bizer , Georgi Kobilarov , Jens Lehmann , Richard Cyganiak , and Zachary Ives . 2007 . Dbpedia: A nucleus for a web of open data. In The Semantic Web . Springer , 722--735. S\u00f6ren Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. In The Semantic Web. Springer, 722--735."},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 1533--1544","author":"Berant Jonathan","year":"2013","unstructured":"Jonathan Berant , Andrew Chou , Roy Frostig , and Percy Liang . 2013 . Semantic parsing on freebase from question-answer pairs . In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 1533--1544 . Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 1533--1544."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376746"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1067"},{"key":"e_1_2_1_5_1","volume-title":"Large-scale simple question answering with memory networks. CoRR abs\/1506.02075","author":"Bordes Antoine","year":"2015","unstructured":"Antoine Bordes , Nicolas Usunier , Sumit Chopra , and Jason Weston . 2015. Large-scale simple question answering with memory networks. CoRR abs\/1506.02075 ( 2015 ). Antoine Bordes, Nicolas Usunier, Sumit Chopra, and Jason Weston. 2015. Large-scale simple question answering with memory networks. CoRR abs\/1506.02075 (2015)."},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the 27th Annual Conference on Neural Information Processing Systems","author":"Bordes Antoine","year":"2013","unstructured":"Antoine Bordes , Nicolas Usunier , Alberto Garc\u00eda-Dur\u00e1n , Jason Weston , and Oksana Yakhnenko . 2013 . Translating embeddings for modeling multi-relational data . In Proceedings of the 27th Annual Conference on Neural Information Processing Systems 2013. 2787--2795. Antoine Bordes, Nicolas Usunier, Alberto Garc\u00eda-Dur\u00e1n, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 27th Annual Conference on Neural Information Processing Systems 2013. 2787--2795."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-44848-9_11"},{"key":"e_1_2_1_8_1","first-page":"1","article-title":"Learning an executable neural semantic parser","volume":"45","author":"Cheng Jianpeng","year":"2017","unstructured":"Jianpeng Cheng , Siva Reddy , Vijay Saraswat , and Mirella Lapata . 2017 . Learning an executable neural semantic parser . Comput. Linguist. 45 , 1 (March 2019), 59--94. Jianpeng Cheng, Siva Reddy, Vijay Saraswat, and Mirella Lapata. 2017. Learning an executable neural semantic parser. Comput. Linguist. 45, 1 (March 2019), 59--94.","journal-title":"Comput. Linguist."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1076"},{"key":"e_1_2_1_10_1","volume-title":"Convolutional 2D knowledge graph embeddings. CoRR abs\/1707.01476","author":"Dettmers Tim","year":"2017","unstructured":"Tim Dettmers , Pasquale Minervini , Pontus Stenetorp , and Sebastian Riedel . 2017. Convolutional 2D knowledge graph embeddings. CoRR abs\/1707.01476 ( 2017 ). http:\/\/arxiv.org\/abs\/1707.01476. Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2017. Convolutional 2D knowledge graph embeddings. CoRR abs\/1707.01476 (2017). http:\/\/arxiv.org\/abs\/1707.01476."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1091"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1026"},{"key":"e_1_2_1_13_1","volume-title":"Capturing semantic similarity for entity linking with convolutional neural networks. arXiv preprint arXiv:1604.00734","author":"Francis-Landau Matthew","year":"2016","unstructured":"Matthew Francis-Landau , Greg Durrett , and Dan Klein . 2016. Capturing semantic similarity for entity linking with convolutional neural networks. arXiv preprint arXiv:1604.00734 ( 2016 ). Matthew Francis-Landau, Greg Durrett, and Dan Klein. 2016. Capturing semantic similarity for entity linking with convolutional neural networks. arXiv preprint arXiv:1604.00734 (2016)."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/3304222.3304339"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1284"},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS\u201910)","author":"Gutmann Michael","year":"2010","unstructured":"Michael Gutmann and Aapo Hyv\u00e4rinen . 2010 . Noise-contrastive estimation: A new estimation principle for unnormalized statistical models . In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS\u201910) . 297--304. http:\/\/jmlr.org\/proceedings\/papers\/v9\/gutmann10a.html. Michael Gutmann and Aapo Hyv\u00e4rinen. 2010. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS\u201910). 297--304. http:\/\/jmlr.org\/proceedings\/papers\/v9\/gutmann10a.html."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1021"},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","volume":"1","author":"Hsiao Wei-Chuan","year":"2017","unstructured":"Wei-Chuan Hsiao , Hen-Hsen Huang , and Hsin-Hsi Chen . 2017 . Integrating subject, type, and property identification for simple question answering over knowledge base . In Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , Vol. 1 . 976--985. Wei-Chuan Hsiao, Hen-Hsen Huang, and Hsin-Hsi Chen. 2017. Integrating subject, type, and property identification for simple question answering over knowledge base. In Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Vol. 1. 976--985."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-2016"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/511446.511524"},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the 18th International Conference on Machine Learning (ICML\u201901)","author":"Lafferty John D.","unstructured":"John D. Lafferty , Andrew McCallum , and Fernando C. N. Pereira . 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data . In Proceedings of the 18th International Conference on Machine Learning (ICML\u201901) . Morgan Kaufmann Publishers Inc., San Francisco, CA, 282--289. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning (ICML\u201901). Morgan Kaufmann Publishers Inc., San Francisco, CA, 282--289."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3086703"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052675"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-5010"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the 27th Annual Conference on Neural Information Processing Systems","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov , Ilya Sutskever , Kai Chen , Gregory S. Corrado , and Jeffrey Dean . 2013 . Distributed representations of words and phrases and their compositionality . In Proceedings of the 27th Annual Conference on Neural Information Processing Systems 2013. 3111--3119. http:\/\/papers.nips.cc\/paper\/5021-distributed-representations-of-words-and-phrases-and-their-compositionality. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 27th Annual Conference on Neural Information Processing Systems 2013. 3111--3119. http:\/\/papers.nips.cc\/paper\/5021-distributed-representations-of-words-and-phrases-and-their-compositionality."},{"key":"e_1_2_1_26_1","volume-title":"Advances in Neural Information Processing Systems 26","author":"Mikolov Tomas","unstructured":"Tomas Mikolov , Ilya Sutskever , Kai Chen , Greg S Corrado , and Jeff Dean . 2013. Distributed representations of words and phrases and their compositionality . In Advances in Neural Information Processing Systems 26 , C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.). Curran Associates, Inc. , 3111--3119. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 3111--3119."},{"key":"e_1_2_1_27_1","volume-title":"Versa: Path-based rdf query language. Retrieved from O\u2019Reilly XML.com.","author":"Ogbuji Chimezie","year":"2005","unstructured":"Chimezie Ogbuji . 2005 . Versa: Path-based rdf query language. Retrieved from O\u2019Reilly XML.com. Chimezie Ogbuji. 2005. Versa: Path-based rdf query language. Retrieved from O\u2019Reilly XML.com."},{"key":"e_1_2_1_28_1","unstructured":"Eric Prud'hommeaux and Andy Seaborne. 2006. SPARQL query language for RDF. (2006). https:\/\/www.w3.org\/TR\/rdf-sparql-query\/.  Eric Prud'hommeaux and Andy Seaborne. 2006. SPARQL query language for RDF. (2006). https:\/\/www.w3.org\/TR\/rdf-sparql-query\/."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1009"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-2068"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the 31st AAAI Conference on Artificial Intelligence. 1236--1242","author":"Shi Baoxu","year":"2017","unstructured":"Baoxu Shi and Tim Weninger . 2017 . ProjE: Embedding projection for knowledge graph completion . In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 1236--1242 . Baoxu Shi and Tim Weninger. 2017. ProjE: Embedding projection for knowledge graph completion. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 1236--1242."},{"key":"e_1_2_1_32_1","unstructured":"Behrouz Haji Soleimani and Stan Matwin. 2018. Spectral word embedding with negative sampling. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI\u201918) the 30th Innovative Applications of Artificial Intelligence (IAAI\u201918) and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI\u201918). 5481--5487.  Behrouz Haji Soleimani and Stan Matwin. 2018. Spectral word embedding with negative sampling. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI\u201918) the 30th Innovative Applications of Artificial Intelligence (IAAI\u201918) and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI\u201918). 5481--5487."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2008.06.001"},{"key":"e_1_2_1_34_1","volume-title":"Le","author":"Sutskever Ilya","year":"2014","unstructured":"Ilya Sutskever , Oriol Vinyals , and Quoc V . Le . 2014 . Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2. MIT Press , 3104--3112. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2. MIT Press, 3104--3112."},{"key":"e_1_2_1_35_1","doi-asserted-by":"crossref","unstructured":"Ferhan Ture and Oliver Jojic. 2017. No need to pay attention: Simple recurrent neural networks work! In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2866--2872.  Ferhan Ture and Oliver Jojic. 2017. No need to pay attention: Simple recurrent neural networks work! In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2866--2872.","DOI":"10.18653\/v1\/D17-1307"},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of the 28th AAAI Conference on Artificial Intelligence. 1112--1119","author":"Wang Zhen","year":"2014","unstructured":"Zhen Wang , Jianwen Zhang , Jianlin Feng , and Zheng Chen . 2014 . Knowledge graph embedding by translating on hyperplanes . In Proceedings of the 28th AAAI Conference on Artificial Intelligence. 1112--1119 . Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. 1112--1119."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1062"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1220"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1049"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1015"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1128"},{"key":"e_1_2_1_42_1","volume-title":"Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers (COLING\u201916)","author":"Yin Wenpeng","year":"2016","unstructured":"Wenpeng Yin , Mo Yu , Bing Xiang , Bowen Zhou , and Hinrich Sch\u00fctze . 2016 . Simple question answering by attentive convolutional neural network . In Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers (COLING\u201916) . 1746--1756. Wenpeng Yin, Mo Yu, Bing Xiang, Bowen Zhou, and Hinrich Sch\u00fctze. 2016. Simple question answering by attentive convolutional neural network. In Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers (COLING\u201916). 1746--1756."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1053"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3054240"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.5555\/2969239.2969312"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3345557","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3345557","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:13:19Z","timestamp":1750201999000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3345557"}},"subtitle":["A Verification Assisted Model with Iterative Training"],"short-title":[],"issued":{"date-parts":[[2019,9,20]]},"references-count":45,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,10,31]]}},"alternative-id":["10.1145\/3345557"],"URL":"https:\/\/doi.org\/10.1145\/3345557","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,9,20]]},"assertion":[{"value":"2018-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-09-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}