{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T21:58:42Z","timestamp":1790805522583,"version":"4.1.0"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T00:00:00Z","timestamp":1755820800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"},{"start":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T00:00:00Z","timestamp":1755820800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J. King Saud Univ. Comput. Inf. Sci."],"published-print":{"date-parts":[[2025,9]]},"DOI":"10.1007\/s44443-025-00122-2","type":"journal-article","created":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T14:59:17Z","timestamp":1755874757000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Elimination-based reasoning with LLM for multiple-choice educational question answering"],"prefix":"10.1007","volume":"37","author":[{"given":"Qianli","family":"Zhao","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-4407-7838","authenticated-orcid":false,"given":"Mei","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,8,22]]},"reference":[{"key":"122_CR1","unstructured":"Abdin M, Aneja J, Behl H, Bubeck S, Eldan R, Gunasekar S, Harrison M, Hewett RJ, Javaheripi M, Kauffmann P et al (2024) Phi-4 technical report. arXiv:2412.08905"},{"key":"122_CR2","unstructured":"Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman FL, Almeida D, Altenschmidt J, Altman S, Anadkat S et al (2023) Gpt-4 technical report. arXiv:2303.08774"},{"key":"122_CR3","unstructured":"Alkhaldi TYS (2023) Studies on question answering in open-book and closed-book settings"},{"key":"122_CR4","unstructured":"Anil R, Dai AM, Firat O, Johnson M, Lepikhin D, Passos A, Shakeri S, Taropa E, Bailey P, Chen Z et al (2023) Palm 2 technical report. arXiv:2305.10403"},{"key":"122_CR5","unstructured":"Bai J, Bai S, Chu Y, Cui Z, Dang K, Deng X, Fan Y, Ge W, Han Y, Huang F et al (2023) Qwen technical report. arXiv:2309.16609"},{"key":"122_CR6","doi-asserted-by":"crossref","unstructured":"Balepur N, Palta S, Rudinger R (2023) It\u2019s not easy being wrong: large language models struggle with process of elimination reasoning. arXiv:2311.07532","DOI":"10.18653\/v1\/2024.findings-acl.604"},{"key":"122_CR7","first-page":"1877","volume":"33","author":"T Brown","year":"2020","unstructured":"Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877\u20131901","journal-title":"Adv Neural Inf Process Syst"},{"key":"122_CR8","unstructured":"Cai Z, Cao M, Chen H, Chen K, Chen K, Chen X, Chen X, Chen Z, Chen Z, Chu P et al (2024) Internlm2 technical report. arXiv:2403.17297"},{"key":"122_CR9","unstructured":"Choi C, Kim J, Lee S, Kwon J, Gu S, Kim Y, Cho M, Sohn J-y (2024) Linq-embed-mistral technical report. arXiv:2412.03223"},{"issue":"240","key":"122_CR10","first-page":"1","volume":"24","author":"A Chowdhery","year":"2023","unstructured":"Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S et al (2023) Palm: scaling language modeling with pathways. J Mach Learn Res 24(240):1\u2013113","journal-title":"J Mach Learn Res"},{"issue":"70","key":"122_CR11","first-page":"1","volume":"25","author":"HW Chung","year":"2024","unstructured":"Chung HW, Hou L, Longpre S, Zoph B, Tay Y, Fedus W, Li Y, Wang X, Dehghani M, Brahma S et al (2024) Scaling instruction-finetuned language models. J Mach Learn Res 25(70):1\u201353","journal-title":"J Mach Learn Res"},{"key":"122_CR12","unstructured":"Clark P, Cowhey I, Etzioni O, Khot T, Sabharwal A, Schoenick C, Tafjord O (2018) Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv:1803.05457"},{"key":"122_CR13","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, Volume 1 (long and Short Papers), pp 4171\u20134186"},{"key":"122_CR14","unstructured":"GLM T, Zeng A, Xu B, Wang B, Zhang C, Yin D, Zhang D, Rojas D, Feng G, Zhao H et al (2024) Chatglm: a family of large language models from glm-130b to glm-4 all tools. arXiv:2406.12793"},{"key":"122_CR15","unstructured":"Hendrycks D, Burns C, Basart S, Zou A, Mazeika M, Song D, Steinhardt J (2020) Measuring massive multitask language understanding. arXiv:2009.03300"},{"key":"122_CR16","unstructured":"Jain S, Wallace BC (2019) Attention is not explanation. arXiv:1902.10186"},{"key":"122_CR17","doi-asserted-by":"crossref","unstructured":"Khashabi D, Min S, Khot T, Sabharwal A, Tafjord O, Clark P, Hajishirzi H (2020) Unifiedqa: crossing format boundaries with a single qa system. arXiv:2005.00700","DOI":"10.18653\/v1\/2020.findings-emnlp.171"},{"key":"122_CR18","first-page":"22199","volume":"35","author":"T Kojima","year":"2022","unstructured":"Kojima T, Gu SS, Reid M, Matsuo Y, Iwasawa Y (2022) Large language models are zero-shot reasoners. Adv Neural Inf Process Syst 35:22199\u201322213","journal-title":"Adv Neural Inf Process Syst"},{"key":"122_CR19","unstructured":"Kraaij W, Hain T, Lincoln M, Post W (2005) The ami meeting corpus. In: Proceedings on international conference on methods and techniques in behavioral research, pp 1\u20134"},{"key":"122_CR20","doi-asserted-by":"crossref","unstructured":"Lai G, Xie Q, Liu H, Yang Y, Hovy E (2017) Race: large-scale reading comprehension dataset from examinations. arXiv:1704.04683","DOI":"10.18653\/v1\/D17-1082"},{"key":"122_CR21","doi-asserted-by":"crossref","unstructured":"Laskar MTR, Fu X-Y, Chen C, Tn SB (2023) Building real-world meeting summarization systems using large language models: a practical perspective. arXiv:2310.19233","DOI":"10.18653\/v1\/2023.emnlp-industry.33"},{"key":"122_CR22","first-page":"36407","volume":"36","author":"Z Ling","year":"2023","unstructured":"Ling Z, Fang Y, Li X, Huang Z, Lee M, Memisevic R, Su H (2023) Deductive verification of chain-of-thought reasoning. Adv Neural Inf Process Syst 36:36407\u201336433","journal-title":"Adv Neural Inf Process Syst"},{"key":"122_CR23","unstructured":"Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized bert pretraining approach. arXiv:1907.11692"},{"key":"122_CR24","doi-asserted-by":"crossref","unstructured":"Ma C, Du X (2023) Poe: process of elimination for multiple choice reasoning. arXiv:2310.15575","DOI":"10.18653\/v1\/2023.emnlp-main.273"},{"key":"122_CR25","unstructured":"Nye M, Andreassen AJ, Gur-Ari G, Michalewski H, Austin J, Bieber D, Dohan D, Lewkowycz A, Bosma M, Luan D et al (2021) Show your work: scratchpads for intermediate computation with language models"},{"key":"122_CR26","unstructured":"Pal A, Umapathi LK, Sankarasubbu M (2022) Medmcqa: a large-scale multi-subject multi-choice dataset for medical domain question answering. In: Conference on health, inference, and learning. PMLR, pp 248\u2013260"},{"key":"122_CR27","doi-asserted-by":"crossref","unstructured":"Rajpurkar P, Zhang J, Lopyrev K, Liang P (2016) Squad: 100,000+ questions for machine comprehension of text. arXiv:1606.05250","DOI":"10.18653\/v1\/D16-1264"},{"key":"122_CR28","first-page":"68539","volume":"36","author":"T Schick","year":"2023","unstructured":"Schick T, Dwivedi-Yu J, Dess\u00ec R, Raileanu R, Lomeli M, Hambro E, Zettlemoyer L, Cancedda N, Scialom T (2023) Toolformer: language models can teach themselves to use tools. Adv Neural Inf Process Syst 36:68539\u201368551","journal-title":"Adv Neural Inf Process Syst"},{"key":"122_CR29","first-page":"8634","volume":"36","author":"N Shinn","year":"2023","unstructured":"Shinn N, Cassano F, Gopinath A, Narasimhan K, Yao S (2023) Reflexion: language agents with verbal reinforcement learning. Adv Neural Inf Process Syst 36:8634\u20138652","journal-title":"Adv Neural Inf Process Syst"},{"key":"122_CR30","unstructured":"Srivastava A, Rastogi A, Rao A, Shoeb AAM, Abid A, Fisch A, Brown AR, Santoro A, Gupta A, Garriga-Alonso A et al (2022) Beyond the imitation game: quantifying and extrapolating the capabilities of language models. arXiv:2206.04615"},{"key":"122_CR31","unstructured":"Taylor R, Kardas M, Cucurull G, Scialom T, Hartshorn A, Saravia E, Poulton A, Kerkez V, Stojnic R (2022) Galactica: a large language model for science. arXiv:2211.09085"},{"key":"122_CR32","unstructured":"Team G, Kamath A, Ferret J, Pathak S, Vieillard N, Merhej R, Perrin S, Matejovicova T, Ram\u00e9 A, Rivi\u00e8re M et al (2025) Gemma 3 technical report. arXiv:2503.19786"},{"key":"122_CR33","unstructured":"Touvron H, Lavril T, Izacard G, Martinet X, Lachaux M-A, Lacroix T, Rozi\u00e8re B, Goyal N, Hambro E, Azhar F et al (2023a) Llama: open and efficient foundation language models. arXiv:2302.13971"},{"key":"122_CR34","unstructured":"Touvron H, Martin L, Stone K, Albert P, Almahairi A, Babaei Y, Bashlykov N, Batra S, Bhargava P, Bhosale S et al (2023b) Llama 2: open foundation and fine-tuned chat models. arXiv:2307.09288"},{"key":"122_CR35","unstructured":"Tunstall L, Beeching E, Lambert N, Rajani N, Rasul K, Belkada Y, Huang S, Von\u00a0Werra L, Fourrier C, Habib N et al (2023) Zephyr: direct distillation of lm alignment. arXiv:2310.16944"},{"key":"122_CR36","unstructured":"Vacareanu R, Pratik A, Spiliopoulou E, Qi Z, Paolini G, John NA, Ma J, Benajiba Y, Ballesteros M (2024) General purpose verification for chain of thought prompting. arXiv:2405.00204"},{"key":"122_CR37","first-page":"95095","volume":"37","author":"K Wang","year":"2024","unstructured":"Wang K, Pan J, Shi W, Lu Z, Ren H, Zhou A, Zhan M, Li H (2024) Measuring multimodal mathematical reasoning with math-vision dataset. Adv Neural Inf Process Syst 37:95095\u201395169","journal-title":"Adv Neural Inf Process Syst"},{"key":"122_CR38","unstructured":"Wang X, Wei J, Schuurmans D, Le Q, Chi E, Narang S, Chowdhery A, Zhou D (2022) Self-consistency improves chain of thought reasoning in language models. arXiv:2203.11171"},{"key":"122_CR39","first-page":"24824","volume":"35","author":"J Wei","year":"2022","unstructured":"Wei J, Wang X, Schuurmans D, Bosma M, Xia F, Chi E, Le QV, Zhou D et al (2022) Chain-of-thought prompting elicits reasoning in large language models. Adv Neural Inf Process Syst 35:24824\u201324837","journal-title":"Adv Neural Inf Process Syst"},{"key":"122_CR40","doi-asserted-by":"crossref","unstructured":"Weng Y, Zhu M, Xia F, Li B, He S, Liu S, Sun B, Liu K, Zhao J (2022) Large language models are better reasoners with self-verification. arXiv:2212.09561","DOI":"10.18653\/v1\/2023.findings-emnlp.167"},{"key":"122_CR41","unstructured":"Yang C, Wang X, Lu Y, Liu H, Le QV, Zhou D, Chen X (2023a) Large language models as optimizers. arXiv:2309.03409"},{"key":"122_CR42","unstructured":"Yang A, Xiao B, Wang B, Zhang B, Bian C, Yin C, Lv C, Pan D, Wang D, Yan D et al (2023b) Baichuan 2: open large-scale language models. arXiv:2309.10305"},{"key":"122_CR43","unstructured":"Yao L (2024) Large language models are contrastive reasoners. arXiv:2403.08211"},{"key":"122_CR44","first-page":"11809","volume":"36","author":"S Yao","year":"2023","unstructured":"Yao S, Yu D, Zhao J, Shafran I, Griffiths T, Cao Y, Narasimhan K (2023) Tree of thoughts: deliberate problem solving with large language models. Adv Neural Inf Process Syst 36:11809\u201311822","journal-title":"Adv Neural Inf Process Syst"},{"key":"122_CR45","unstructured":"Young A, Chen B, Li C, Huang C, Zhang G, Zhang G, Wang G, Li H, Zhu J, Chen J et al (2024) Yi: open foundation models by 01. ai. arXiv:2403.04652"},{"key":"122_CR46","first-page":"15476","volume":"35","author":"E Zelikman","year":"2022","unstructured":"Zelikman E, Wu Y, Mu J, Goodman N (2022) Star: bootstrapping reasoning with reasoning. Adv Neural Inf Process Syst 35:15476\u201315488","journal-title":"Adv Neural Inf Process Syst"},{"key":"122_CR47","unstructured":"Zhou D, Sch\u00e4rli N, Hou L, Wei J, Scales N, Wang X, Schuurmans D, Cui C, Bousquet O, Le Q et al (2022) Least-to-most prompting enables complex reasoning in large language models. arXiv:2205.10625"},{"key":"122_CR48","unstructured":"Zhu Z, Liu B, Ai Q, Liu Y (2025) Option-id based elimination for multiple choice questions. arXiv:2501.15175"}],"container-title":["Journal of King Saud University Computer and Information Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44443-025-00122-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s44443-025-00122-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44443-025-00122-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,17]],"date-time":"2025-09-17T12:44:53Z","timestamp":1758113093000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s44443-025-00122-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,22]]},"references-count":48,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,9]]}},"alternative-id":["122"],"URL":"https:\/\/doi.org\/10.1007\/s44443-025-00122-2","relation":{},"ISSN":["1319-1578","2213-1248"],"issn-type":[{"value":"1319-1578","type":"print"},{"value":"2213-1248","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,22]]},"assertion":[{"value":"12 May 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 June 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 August 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"204"}}