{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T17:51:56Z","timestamp":1774720316378,"version":"3.50.1"},"reference-count":62,"publisher":"MIT Press - Journals","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Transactions of the Association for Computational Linguistics"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:p> Machine reading comprehension tasks require a machine reader to answer questions relevant to the given document. In this paper, we present the first free-form multiple-Choice Chinese machine reading Comprehension dataset (C<jats:sup>3<\/jats:sup>), containing 13,369 documents (dialogues or more formally written mixed-genre texts) and their associated 19,577 multiple-choice free-form questions collected from Chinese-as-a-second-language examinations. We present a comprehensive analysis of the prior knowledge (i.e., linguistic, domain-specific, and general world knowledge) needed for these real-world problems. We implement rule-based and popular neural methods and find that there is still a significant performance gap between the best performing model (68.5%) and human readers (96.0%), especiallyon problems that require prior knowledge. We further study the effects of distractor plausibility and data augmentation based on translated relevant datasets for English on model performance. We expect C<jats:sup>3<\/jats:sup> to present great challenges to existing systems as answering 86.8% of questions requires both knowledge within and beyond the accompanying document, and we hope that C<jats:sup>3<\/jats:sup> can serve as a platform to study how to leverage various kinds of prior knowledge to better understand a given written or orally oriented text. C<jats:sup>3<\/jats:sup> is available at https:\/\/dataset.org\/c3\/ . <\/jats:p>","DOI":"10.1162\/tacl_a_00305","type":"journal-article","created":{"date-parts":[[2020,3,31]],"date-time":"2020-03-31T14:54:52Z","timestamp":1585666492000},"page":"141-155","source":"Crossref","is-referenced-by-count":46,"title":["Investigating Prior Knowledge for Challenging Chinese Machine Reading Comprehension"],"prefix":"10.1162","volume":"8","author":[{"given":"Kai","family":"Sun","sequence":"first","affiliation":[{"name":"Cornell University, Ithaca, NY."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dian","family":"Yu","sequence":"additional","affiliation":[{"name":"Tencent AI Lab, Bellevue, WA."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dong","family":"Yu","sequence":"additional","affiliation":[{"name":"Tencent AI Lab, Bellevue, WA."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Claire","family":"Cardie","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"bib1","first-page":"2","volume":"13","author":"Adams Marilyn","year":"1982","journal-title":"Reader Meets Author: Bridging the Gap"},{"key":"bib2","author":"Bajgar Ondrej","year":"2016","journal-title":"arXiv preprint"},{"key":"bib3","first-page":"2479","volume-title":"Proceedings of the IJCAI","author":"Cheng Gong","year":"2016"},{"key":"bib4","first-page":"2174","volume-title":"Proceedings of the EMNLP","author":"Choi Eunsol","year":"2018"},{"key":"bib5","first-page":"2924","volume-title":"Proceedings of the NAACL-HLT","author":"Clark Christopher","year":"2019"},{"key":"bib6","first-page":"2580","volume-title":"Proceedings of the AAAI","author":"Clark Peter","year":"2016"},{"key":"bib7","author":"Cui Yiming","year":"2019","journal-title":"arXiv preprint"},{"key":"bib8","first-page":"2721","volume-title":"Proceedings of the LREC","author":"Cui Yiming","year":"2018"},{"key":"bib9","first-page":"1777","volume-title":"Proceedings of the COLING","author":"Cui Yiming","year":"2016"},{"key":"bib10","first-page":"5882","volume-title":"Proceedings of the EMNLP","author":"Cui Yiming","year":"2018"},{"key":"bib11","first-page":"1595","volume-title":"Proceedings of the NAACL-HLT","author":"Dalvi Bhavana","year":"2018"},{"key":"bib12","first-page":"4171","volume-title":"Proeedings of the NAACL-HLT","author":"Devlin Jacob","year":"2019"},{"key":"bib13","doi-asserted-by":"publisher","DOI":"10.1162\/089120102760173625"},{"key":"bib14","first-page":"1156","volume-title":"Proceedings of the SIGKDD","author":"Fader Anthony","year":"2014"},{"key":"bib15","first-page":"1774","volume-title":"Proceedings of the ACL","author":"Feng Song","year":"2013"},{"key":"bib16","first-page":"46","volume-title":"Proceedings of the ANLP","author":"Grishman Ralph","year":"1983"},{"key":"bib17","first-page":"34","volume-title":"Proceedings of the IJCNLP 2017, Shared Tasks","author":"Guo Shangmin","year":"2017"},{"key":"bib18","first-page":"111","volume-title":"Proceedings of the EACL","author":"Guo Shangmin","year":"2017"},{"key":"bib19","first-page":"111","volume-title":"Proceedings of the NODALIDA","author":"Hansen Steffen Leo","year":"1994"},{"key":"bib20","first-page":"938","volume-title":"Proceedings of the AAAI","author":"Hao Yu","year":"2019"},{"key":"bib21","first-page":"37","volume-title":"Proceedings of the MRQA","author":"He Wei","year":"2017"},{"key":"bib22","first-page":"1693","volume-title":"Proceedings of the NIPS","author":"Hermann Karl Moritz","year":"2015"},{"key":"bib23","volume-title":"Proceedings of the ICLR","author":"Hill Felix","year":"2016"},{"key":"bib24","first-page":"5865","volume-title":"Proceedings of the EMNLP-IJCNLP","author":"Huang Zixian","year":"2019"},{"key":"bib25","doi-asserted-by":"crossref","first-page":"12","DOI":"10.3115\/v1\/W14-2903","volume-title":"Proceedings of the Second Workshop on EVENTS: Definition, Detection, Coreference, and Representation","author":"Ikuta Rei","year":"2014"},{"key":"bib26","author":"Joshi Mandar","year":"2017","journal-title":"arXiv preprint"},{"key":"bib27","first-page":"252","volume-title":"Proceedings of the NAACL-HLT","author":"Khashabi Daniel","year":"2018"},{"key":"bib28","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00023"},{"key":"bib29","first-page":"785","volume-title":"Proceedings of the EMNLP","author":"Lai Guokun","year":"2017"},{"issue":"2","key":"bib30","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1609\/aimag.v38i2.2715","volume":"38","author":"Lally Adam","year":"2017","journal-title":"AI Magazine"},{"issue":"4","key":"bib31","first-page":"65","volume":"6","author":"Lenat Douglas B.","year":"1985","journal-title":"AI Magazine"},{"key":"bib32","author":"Li Peng","year":"2016","journal-title":"arXiv preprint"},{"key":"bib33","first-page":"138","volume-title":"Proceedings of the ACL","author":"Li Shen","year":"2018"},{"key":"bib34","first-page":"329","volume-title":"Proceedings of the ACL","author":"LoBue Peter","year":"2011"},{"key":"bib35","volume-title":"WordNet: An Electronic Lexical database","author":"Miller George","year":"1998"},{"key":"bib36","first-page":"839","volume-title":"Proceedings of the NAACL-HLT","author":"Mostafazadeh Nasrin","year":"2016"},{"issue":"3","key":"bib37","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1111\/j.1540-4781.2006.00431.x","volume":"90","author":"Nassaji Hossein","year":"2006","journal-title":"The Modern Language Journal"},{"key":"bib38","author":"Nguyen Tri","year":"2016","journal-title":"arXiv preprint"},{"key":"bib39","first-page":"47","volume-title":"Proceedings of the CNS","author":"O\u2019Gorman Tim","year":"2016"},{"key":"bib40","first-page":"747","volume-title":"Proceedings of the SemEval","author":"Ostermann Simon","year":"2018"},{"key":"bib41","first-page":"87","volume-title":"Proceedings of the NAACL-HLT FAM-LbR","author":"Poon Hoifung","year":"2010"},{"key":"bib42","unstructured":"Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. https:\/\/openai.com\/blog\/language-unsupervised\/."},{"key":"bib43","first-page":"784","volume-title":"Proceedings of the ACL","author":"Rajpurkar Pranav","year":"2018"},{"key":"bib44","first-page":"2383","volume-title":"Proceedings of the EMNLP","author":"Rajpurkar Pranav","year":"2016"},{"key":"bib45","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00266"},{"key":"bib46","first-page":"193","volume-title":"Proceedings of the EMNLP","author":"Richardson Matthew","year":"2013"},{"key":"bib47","first-page":"94","volume-title":"Proceedings of the HLT","author":"Schubert Lenhart","year":"2002"},{"key":"bib48","author":"Shao Chih Chieh","year":"2018","journal-title":"arXiv preprint"},{"key":"bib49","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00264"},{"key":"bib50","author":"Sun Yu","year":"2019","journal-title":"arXiv preprint"},{"key":"bib51","first-page":"pages 191\u2013pages","volume-title":"Proceedings of the RepL4NLP","author":"Trischler Adam","year":"2017"},{"issue":"4","key":"bib52","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1162\/coli_a_00337","volume":"44","author":"Hee Cynthia Van","year":"2018","journal-title":"Computational Linguistics"},{"key":"bib53","first-page":"1","volume-title":"Proceedings of the ACL","author":"Wang Shuohang","year":"2018"},{"key":"bib54","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00021"},{"issue":"4","key":"bib55","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1207\/s15516709cog1104_2","volume":"11","author":"Winston Morton E.","year":"1987","journal-title":"Cognitive Science"},{"key":"bib56","unstructured":"Fei Xia. 2000. The part-of-speech tagging guidelines for the Penn Chinese Treebank (3.0). IRCS Technical Reports Series, 1\u201343."},{"key":"bib57","first-page":"234","volume-title":"Proceedings of the EMNLP","author":"Xie Qizhe","year":"2018"},{"key":"bib58","first-page":"2369","volume-title":"Proceedings of the EMNLP","author":"Yang Zhilin","year":"2018"},{"key":"bib59","volume-title":"arXiv preprint","volume":"1810","author":"Zhang Sheng","year":"2018"},{"key":"bib60","first-page":"5706","volume-title":"Proceedings of the AAAI","author":"Zhang Xiao","year":"2018"},{"key":"bib61","first-page":"449","volume-title":"Proceedings of the COLING","author":"Zhang Zhuosheng","year":"2018"},{"key":"bib62","first-page":"778","volume-title":"Proceedings of the ACL","author":"Zheng Chujie","year":"2019"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/tacl_a_00305","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:39:34Z","timestamp":1615585174000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/43546"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12]]},"references-count":62,"alternative-id":["10.1162\/tacl_a_00305"],"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00305","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12]]}}}