{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T19:07:04Z","timestamp":1776107224723,"version":"3.50.1"},"reference-count":15,"publisher":"Association for Computing Machinery (ACM)","issue":"9","license":[{"start":{"date-parts":[[2017,8,23]],"date-time":"2017-08-23T00:00:00Z","timestamp":1503446400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Commun. ACM"],"published-print":{"date-parts":[[2017,8,23]]},"abstract":"<jats:p>Answering questions correctly from standardized eighth-grade science tests is itself a test of machine intelligence.<\/jats:p>","DOI":"10.1145\/3122814","type":"journal-article","created":{"date-parts":[[2017,8,24]],"date-time":"2017-08-24T11:49:04Z","timestamp":1503575344000},"page":"60-64","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":27,"title":["Moving beyond the Turing Test with the Allen AI Science Challenge"],"prefix":"10.1145","volume":"60","author":[{"given":"Carissa","family":"Schoenick","sequence":"first","affiliation":[{"name":"Allen Institute for Artificial Intelligence in Seattle, WA"}]},{"given":"Peter","family":"Clark","sequence":"additional","affiliation":[{"name":"Allen Institute for Artificial Intelligence in Seattle, WA"}]},{"given":"Oyvind","family":"Tafjord","sequence":"additional","affiliation":[{"name":"Allen Institute for Artificial Intelligence in Seattle, WA"}]},{"given":"Peter","family":"Turney","sequence":"additional","affiliation":[{"name":"Allen Institute for Artificial Intelligence in Seattle, WA"}]},{"given":"Oren","family":"Etzioni","sequence":"additional","affiliation":[{"name":"Allen Institute for Artificial Intelligence in Seattle, WA and University of Washington in Seattle, WA"}]}],"member":"320","published-online":{"date-parts":[[2017,8,23]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Allen Institute for Artificial Intelligence. Datasets; http:\/\/allenai.org\/data  Allen Institute for Artificial Intelligence. Datasets; http:\/\/allenai.org\/data"},{"key":"e_1_2_1_2_1","volume-title":"Software tricks people into thinking it is human. New Scientist 2829 (Sept. 6","author":"Aron J.","year":"2011","unstructured":"Aron , J. Software tricks people into thinking it is human. New Scientist 2829 (Sept. 6 , 2011 ). Aron, J. Software tricks people into thinking it is human. New Scientist 2829 (Sept. 6, 2011)."},{"key":"e_1_2_1_3_1","author":"BBC","year":"2014","unstructured":"BBC News. Computer AI passes Turing Test in 'world first.' BBC News ( June 9, 2014 ); http:\/\/www.bbc.com\/news\/technology-27762088 BBC News. Computer AI passes Turing Test in 'world first.' BBC News (June 9, 2014); http:\/\/www.bbc.com\/news\/technology-27762088","journal-title":"News. Computer AI passes Turing Test in 'world first.' BBC News"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing","author":"Berant J.","year":"2013","unstructured":"Berant , J. , Chou , A. , Frostig , R. , and Liang , P . Semantic parsing on Freebase from question-answer pairs . In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing ( Seattle, WA, Oct. 18--21). Association for Computational Linguistics, Stroudsburg, PA , 2013 , 6. Berant, J., Chou, A., Frostig, R., and Liang, P. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (Seattle, WA, Oct. 18--21). Association for Computational Linguistics, Stroudsburg, PA, 2013, 6."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623677"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.3115\/992628.992709"},{"key":"e_1_2_1_7_1","unstructured":"Kaggle. The Allen AI Science Challenge; https:\/\/www.kaggle.com\/c\/the-allen-ai-science-challenge  Kaggle. The Allen AI Science Challenge ; https:\/\/www.kaggle.com\/c\/the-allen-ai-science-challenge"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 19th International Florida Artificial Intelligence Research Society Conference","author":"Katz B.","year":"2006","unstructured":"Katz , B. , Borchardt , G. , and Felshin , S . Natural language annotations for question answering . In Proceedings of the 19th International Florida Artificial Intelligence Research Society Conference ( Melbourne Beach, FL, May 11--13). AAAI Press, Menlo Park, CA , 2006 . Katz, B., Borchardt, G., and Felshin, S. Natural language annotations for question answering. In Proceedings of the 19th International Florida Artificial Intelligence Research Society Conference (Melbourne Beach, FL, May 11--13). AAAI Press, Menlo Park, CA, 2006."},{"key":"e_1_2_1_9_1","volume-title":"Beyond the Turing Test. AI Magazine (Special Edition) 37, 1 (Spring","author":"Marcus G.","year":"2016","unstructured":"Marcus , G. , Rossi , F. , and Veloso , M. , Eds . Beyond the Turing Test. AI Magazine (Special Edition) 37, 1 (Spring 2016 ). Marcus, G., Rossi, F., and Veloso, M., Eds. Beyond the Turing Test. AI Magazine (Special Edition) 37, 1 (Spring 2016)."},{"key":"e_1_2_1_10_1","volume-title":"Semantic Focus (Feb. 26","author":"Simmons J.","year":"2008","unstructured":"Simmons , J. True Knowledge : The natural language question answering Wikipedia for facts . Semantic Focus (Feb. 26 , 2008 ); http:\/\/www.semanticfocus.com\/blog\/entry\/title\/true-knowledge-the-natural-language-question-answering-wikipedia-for-facts\/ Simmons, J. True Knowledge: The natural language question answering Wikipedia for facts. Semantic Focus (Feb. 26, 2008); http:\/\/www.semanticfocus.com\/blog\/entry\/title\/true-knowledge-the-natural-language-question-answering-wikipedia-for-facts\/"},{"key":"e_1_2_1_11_1","volume-title":"Computing machinery and intelligence. Mind59, 236 (Oct","author":"Turing A.M.","year":"1950","unstructured":"Turing , A.M. Computing machinery and intelligence. Mind59, 236 (Oct . 1950 ), 433--460. Turing, A.M. Computing machinery and intelligence. Mind59, 236 (Oct. 1950), 433--460."},{"key":"e_1_2_1_12_1","volume-title":"The plan to replace the Turing Test with a 'Turing Olympics' Motherboard (Jan. 28","author":"Turk V.","year":"2015","unstructured":"Turk , V. The plan to replace the Turing Test with a 'Turing Olympics' Motherboard (Jan. 28 , 2015 ); https:\/\/motherboard.vice.com\/en_us\/article\/the-plan-to-replace-the-turing-test-with-a-turing-olympics Turk, V. The plan to replace the Turing Test with a 'Turing Olympics' Motherboard (Jan. 28, 2015); https:\/\/motherboard.vice.com\/en_us\/article\/the-plan-to-replace-the-turing-test-with-a-turing-olympics"},{"key":"e_1_2_1_13_1","volume-title":"Eds. In Proceedings of the 24th Text REtrieval Conference","author":"Voorhees E.","year":"2015","unstructured":"Voorhees , E. and Ellis , A ., Eds. In Proceedings of the 24th Text REtrieval Conference ( Gaithersburg, MD, Nov. 17--20). Publication SP 500--319, National Institute of Standards and Technology, Gaithersburg, MD , 2015 . Voorhees, E. and Ellis, A., Eds. In Proceedings of the 24th Text REtrieval Conference (Gaithersburg, MD, Nov. 17--20). Publication SP 500--319, National Institute of Standards and Technology, Gaithersburg, MD, 2015."},{"key":"e_1_2_1_14_1","volume-title":"Stephen Wolfram Blog (Sept. 24, 2010","author":"Wolfram S.","year":"2010","unstructured":"Wolfram , S. Making the world's data computable . Stephen Wolfram Blog (Sept. 24, 2010 ); http:\/\/blog.stephenwolfram.com\/ 2010 \/09\/making-the-worlds-data-computable\/ Wolfram, S. Making the world's data computable. Stephen Wolfram Blog (Sept. 24, 2010); http:\/\/blog.stephenwolfram.com\/2010\/09\/making-the-worlds-data-computable\/"},{"key":"e_1_2_1_15_1","first-page":"966","volume":"956","author":"Yao X.","year":"2014","unstructured":"Yao , X. and Van Durme , B. Information extraction over structured data: Question answering with Freebase. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Baltimore, MD, June 22--27). Association for Computational Linguistics, Stroudsburg , PA , 2014 , 956 -- 966 . Yao, X. and Van Durme, B. Information extraction over structured data: Question answering with Freebase. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Baltimore, MD, June 22--27). Association for Computational Linguistics, Stroudsburg, PA, 2014, 956--966.","journal-title":"PA"}],"container-title":["Communications of the ACM"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3122814","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3122814","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T19:05:08Z","timestamp":1750273508000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3122814"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,8,23]]},"references-count":15,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2017,8,23]]}},"alternative-id":["10.1145\/3122814"],"URL":"https:\/\/doi.org\/10.1145\/3122814","relation":{},"ISSN":["0001-0782","1557-7317"],"issn-type":[{"value":"0001-0782","type":"print"},{"value":"1557-7317","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,8,23]]},"assertion":[{"value":"2017-08-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}