{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T12:41:23Z","timestamp":1770813683209,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T00:00:00Z","timestamp":1770249600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"DeepInfo","award":["PID2021-127777OB-C22"],"award-info":[{"award-number":["PID2021-127777OB-C22"]}]},{"name":"DeepSocial","award":["PID2024-159202OB-C22"],"award-info":[{"award-number":["PID2024-159202OB-C22"]}]},{"DOI":"10.13039\/501100008530","name":"FEDER","doi-asserted-by":"crossref","award":["MCIN\/AEI\/10.13039\/501100011033"],"award-info":[{"award-number":["MCIN\/AEI\/10.13039\/501100011033"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Benchmarking Spanish reading comprehension remains challenging due to the scarcity of proficiency-calibrated resources grounded in authentic human assessments. We introduce IC-UNED-RC-ES, a benchmark comprising more than 6000 items derived from Instituto Cervantes examinations, converted to a machine-readable format while preserving exam structure, proficiency levels, and scoring criteria. Unlike many existing resources, IC-UNED-RC-ES includes a diverse set of exercise formats, combining common multiple-choice questions with new formats such as matching and fill-in-the-gap, which support a broader assessment of reading skills. The benchmark supports evaluation at both the item and exam levels and includes an exercise taxonomy with category-specific metrics. Baseline results with current AI systems reveal a strong difficulty effect (a 15-point drop from lower to advanced levels) and substantial variation across exercise types, with inference- and discourse-heavy categories reaching only 41%. IC-UNED-RC-ES provides a human-aligned, interpretable testbed for diagnosing strengths and weaknesses in Spanish reading comprehension and for tracking progress across model generations.<\/jats:p>","DOI":"10.3390\/info17020159","type":"journal-article","created":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T10:35:37Z","timestamp":1770287737000},"page":"159","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Spanish Language Proficiency Dataset for AI Evaluation"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7867-0149","authenticated-orcid":false,"given":"Anselmo","family":"Pe\u00f1as","sequence":"first","affiliation":[{"name":"NLP&IR, Departamento de Lenguajes y Sistemas Inform\u00e1ticos, Universidad Nacional de Educaci\u00f3n a Distancia, 28040 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6331-4117","authenticated-orcid":false,"given":"\u00c1lvaro","family":"Rodrigo","sequence":"additional","affiliation":[{"name":"NLP&IR, Departamento de Lenguajes y Sistemas Inform\u00e1ticos, Universidad Nacional de Educaci\u00f3n a Distancia, 28040 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Javier","family":"Fruns-Jim\u00e9nez","sequence":"additional","affiliation":[{"name":"Instituto Cervantes, 28014 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"In\u00e9s","family":"Soria-Pastor","sequence":"additional","affiliation":[{"name":"Instituto Cervantes, 28014 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1858-9920","authenticated-orcid":false,"given":"Sergio","family":"Moreno-\u00c1lvarez","sequence":"additional","affiliation":[{"name":"NLP&IR, Departamento de Lenguajes y Sistemas Inform\u00e1ticos, Universidad Nacional de Educaci\u00f3n a Distancia, 28040 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2710-9319","authenticated-orcid":false,"given":"Alberto","family":"P\u00e9rez","sequence":"additional","affiliation":[{"name":"NLP&IR, Departamento de Lenguajes y Sistemas Inform\u00e1ticos, Universidad Nacional de Educaci\u00f3n a Distancia, 28040 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-5980-5214","authenticated-orcid":false,"given":"Julio","family":"Reyes-Montesinos","sequence":"additional","affiliation":[{"name":"NLP&IR, Departamento de Lenguajes y Sistemas Inform\u00e1ticos, Universidad Nacional de Educaci\u00f3n a Distancia, 28040 Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,5]]},"reference":[{"key":"ref_1","unstructured":"European-Council (2025, December 09). Relating Language Examinations to the CEFR. Highlights from the Manual. Available online: https:\/\/www.ecml.at\/Portals\/1\/documents\/ECML-resources\/2011_10_10_relex._E_web.pdf?ver=2018-03-21-100940-823."},{"key":"ref_2","first-page":"112502","article-title":"The MERIT Dataset: Modelling and Efficiently Rendering Interpretable Transcripts","volume":"172","author":"Boal","year":"2025","journal-title":"Pattern Recognit."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"111332","DOI":"10.1016\/j.patcog.2024.111332","article-title":"Enhancing textual textbook question answering with large language models and retrieval augmented generation","volume":"162","author":"Alawwad","year":"2025","journal-title":"Pattern Recognit."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wang, L., Zheng, K., Qian, L., and Li, S. (2022, January 10\u201311). A Survey of Extractive Question Answering. Proceedings of the 2022 International Conference on High Performance Big Data and Intelligent Systems (HDIS), Tianjin, China.","DOI":"10.1109\/HDIS56859.2022.9991478"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Aydin, B.I., Yilmaz, Y.S., Li, Y., Li, Q., Gao, J., and Demirbas, M. (2014). Crowdsourcing for multiple-choice question answering. Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, Qu\u00e9bec City, QC, Canada, 27\u201331 July 2014, AAAI Press.","DOI":"10.1609\/aaai.v28i2.19016"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, A., Stanovsky, G., Singh, S., and Gardner, M. (2019, January 4). Evaluating Question Answering Evaluation. Proceedings of the 2nd Workshop on Machine Reading for Question Answering, Hong Kong, China.","DOI":"10.18653\/v1\/D19-5817"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Voorhees, E.M., and Tice, D.M. (2000, January 24\u201328). Building a Question Answering Test Collection. Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, NY, USA.","DOI":"10.1145\/345508.345577"},{"key":"ref_8","unstructured":"Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., and Deng, L. (2016, January 9). MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. Proceedings of the Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches 2016 Co-Located with the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain."},{"key":"ref_9","unstructured":"Joshi, M., Choi, E., Weld, D.S., and Zettlemoyer, L. (August, January 30). TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, BC, Canada."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., and Suleman, K. (2017, January 3). NewsQA: A Machine Comprehension Dataset. Proceedings of the 2nd Workshop on Representation Learning for NLP, Vancouver, BC, Canada.","DOI":"10.18653\/v1\/W17-2623"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Rajpurkar, P., Jia, R., and Liang, P. (2018, January 15\u201320). Know What You Don\u2019t Know: Unanswerable Questions for SQuAD. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), Melbourne, Australia.","DOI":"10.18653\/v1\/P18-2124"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Gururangan, S., Swayamdipta, S., Levy, O., Schwartz, R., Bowman, S., and Smith, N.A. (2018, January 1\u20136). Annotation Artifacts in Natural Language Inference Data. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), New Orleans, LA, USA.","DOI":"10.18653\/v1\/N18-2017"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1162\/tacl_a_00023","article-title":"The NarrativeQA Reading Comprehension Challenge","volume":"6","author":"Schwarz","year":"2018","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Rogers, A., Kovaleva, O., Downey, M., and Rumshisky, A. (2020, January 11\u201315). Getting Closer to AI Complete Question Answering: A Set of Prerequisite Real Tasks. Proceedings of the AAAI Conference on Artificial Intelligence, Atlanta, GA, USA.","DOI":"10.1609\/aaai.v34i05.6398"},{"key":"ref_15","unstructured":"Dua, D., Wang, Y., Dasigi, P., Stanovsky, G., Singh, S., and Gardner, M. (2019, January 4). DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Lai, G., Xie, Q., Liu, H., Yang, Y., and Hovy, E. (2017, January 9\u201311). RACE: Large-scale ReAding Comprehension Dataset From Examinations. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.","DOI":"10.18653\/v1\/D17-1082"},{"key":"ref_17","first-page":"441","article-title":"Results and Lessons of the Question Answering Track at CLEF","volume":"Volume 41","author":"Rodrigo","year":"2019","journal-title":"Information Retrieval Evaluation in a Changing World-Lessons Learned from 20 Years of CLEF"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1016\/j.ipm.2018.03.002","article-title":"Do systems pass university entrance exams?","volume":"54","author":"Rodrigo","year":"2018","journal-title":"Inf. Process. Manag."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3440755","article-title":"Evolution of Semantic Similarity\u2014A Survey","volume":"54","author":"Chandrasekaran","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1221","DOI":"10.1007\/s11145-021-10230-w","article-title":"Clozing in on reading comprehension: A deep cloze test of global inference making","volume":"35","author":"Jensen","year":"2022","journal-title":"Read. Writ. Interdiscip. J."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1016\/j.procs.2025.03.352","article-title":"Retrieval-Augmented Generation for Multiple-Choice Questions and Answers Generation","volume":"259","author":"Pradeesh","year":"2025","journal-title":"Procedia Comput. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"114627","DOI":"10.1016\/j.knosys.2025.114627","article-title":"Coordinated LLM multi-agent systems for collaborative question-answer generation","volume":"330","author":"Saadaoui","year":"2025","journal-title":"Knowl.-Based Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Lewis, P., Oguz, B., Rinott, R., Riedel, S., and Schwenk, H. (2020, January 5\u201310). MLQA: Evaluating Cross-lingual Extractive Question Answering. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.653"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Artetxe, M., Ruder, S., and Yogatama, D. (2020, January 5\u201310). On the Cross-lingual Transferability of Monolingual Representations. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.421"},{"key":"ref_25","unstructured":"Carrino, C.P., Costa-juss\u00e0, M.R., and Fonollosa, J.A.R. (2020, January 11\u201316). Automatic Spanish Translation of SQuAD Dataset for Multi-lingual Question Answering. Proceedings of the Twelfth Language Resources and Evaluation Conference, Marseille, France."},{"key":"ref_26","first-page":"273","article-title":"Overview of QuALES at IberLEF 2022: Question Answering Learning from Examples in Spanish","volume":"69","author":"Chiruzzo","year":"2022","journal-title":"Proces. Leng. Nat."},{"key":"ref_27","first-page":"39","article-title":"MarIA: Spanish Language Models","volume":"68","author":"Carrino","year":"2022","journal-title":"Proces. Leng. Nat."},{"key":"ref_28","unstructured":"Taul\u00e9, M., Mart\u00ed, M.A., and Recasens, M. (June, January 26). AnCora: Multilevel Annotated Corpora for Catalan and Spanish. Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC\u201908), Marrakech, Morocco."},{"key":"ref_29","unstructured":"Cervantes, I. (2025, December 09). Plan Curricular del Instituto Cervantes. Niveles de Referencia Para el Espa\u00f1ol. Available online: https:\/\/cvc.cervantes.es\/ensenanza\/biblioteca_ele\/plan_curricular\/."},{"key":"ref_30","unstructured":"European-Council (2025, December 09). Marco Com\u00fan Europeo de Referencia Para las Lenguas: Ense\u00f1anza, Aprendizaje y Evaluaci\u00f3n. Available online: https:\/\/cvc.cervantes.es\/ensenanza\/biblioteca_ele\/marco\/cvc_mer.pdf."},{"key":"ref_31","first-page":"127","article-title":"The Text Complexity Library","volume":"65","year":"2020","journal-title":"Proces. Leng. Nat."},{"key":"ref_32","unstructured":"Law, G. (2025, December 09). Error in the Fernandez Huerta Readability Formula. Available online: https:\/\/linguistlist.org\/issues\/22\/2332\/."},{"key":"ref_33","first-page":"162","article-title":"Lecturas adaptadas a la ense\u00f1anza del espa\u00f1ol como L2: Variables ling\u00fc\u00edsticas para la determinaci\u00f3n del nivel de legibilidad","volume":"2","author":"Anula","year":"2008","journal-title":"Dialnet"},{"key":"ref_34","first-page":"29","article-title":"Medidas sencillas de lecturabilidad","volume":"214","year":"1959","journal-title":"Consigna"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1037\/h0057532","article-title":"A new readability yardstick","volume":"32","author":"Flesch","year":"1948","journal-title":"J. Appl. Psychol."},{"key":"ref_36","unstructured":"Pazos, F.S. (1993). Sistemas Predictivos de Legilibilidad del Mensaje Escrito: F\u00f3rmula de Perspicuidad. [Ph.D. Thesis, Universidad Complutense de Madrid]."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2738046","article-title":"Making it simplext: Implementation and evaluation of a text simplification system for spanish","volume":"6","author":"Saggion","year":"2015","journal-title":"ACM Trans. Access. Comput. TACCESS"},{"key":"ref_38","first-page":"49","article-title":"Legibilidad de los folletos informativos","volume":"1","year":"2001","journal-title":"Pharm. Care Esp."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Akyol, P., Key, J., and Krishna, K. (2016). Hit or Miss? Test Taking Behavior in Multiple Choice Exams, National Bureau of Economic Research, Inc.. NBER Working Papers 22401.","DOI":"10.3386\/w22401"},{"key":"ref_40","first-page":"381","article-title":"Correcting scores of tests taking into account the guessing factor: Yes or no","volume":"2","author":"Valadares","year":"2011","journal-title":"Int. J. Cross-Discip. Subj. Educ."},{"key":"ref_41","unstructured":"Liang, Y., Li, J., and Yin, J. (2019, January 17\u201319). A New Multi-choice Reading Comprehension Dataset for Curriculum Learning. Proceedings of the Eleventh Asian Conference on Machine Learning, PMLR, Nagoya, Japan."},{"key":"ref_42","unstructured":"Pal, A., Umapathi, L.K., and Sankarasubbu, M. (2022, January 7\u20138). MedMCQA: A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answering. Proceedings of the Conference on Health, Inference, and Learning, PMLR, Virtually."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/2\/159\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T05:17:23Z","timestamp":1770441443000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/2\/159"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,5]]},"references-count":42,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["info17020159"],"URL":"https:\/\/doi.org\/10.3390\/info17020159","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,5]]}}}