{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T22:57:55Z","timestamp":1781650675591,"version":"3.54.5"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T00:00:00Z","timestamp":1776729600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T00:00:00Z","timestamp":1776729600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004909","name":"Universidade Federal Do Rio Grande","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004909","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2026,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Chain of Thought (CoT) prompting has been shown to improve the performance of large language models (LLMs) in a wide range of tasks, including arithmetic, common-sense, and symbolic reasoning. However, this improvement requires the development of effective CoT prompts. On the other hand, more recent work has shown that CoT reasoning paths are often inherently present in top-k alternative decoding sequences, even in the absence of any specific prompting technique. In this study, we propose a new fine-tuning method that exploits this property by targeting only two specific tokens of these pre-existing CoT responses. We demonstrate that fine-tuning only two tokens using the model\u2019s own implicitly generated CoT paths leads to a significant efficiency gain, reducing training time while still achieving meaningful performance improvements. When evaluated on arithmetic datasets, we achieved a 22.7% improvement on MultiArith, 9.0% on GSM8K, and 2.3% on SVAMP when validated on the Phi-2 model from a greedy decoding perspective, reducing the processing time by over 90% compared to the LoRA fine-tuning method.\u00a0Code is publicly available at:\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/github.com\/paulosantosneto\/2tft\" ext-link-type=\"uri\">https:\/\/github.com\/paulosantosneto\/2tft<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1007\/s00521-026-12052-9","type":"journal-article","created":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T05:20:47Z","timestamp":1776748847000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["2T-FT: Two-Token Fine-tuning improves zero-shot performance with minimal training"],"prefix":"10.1007","volume":"38","author":[{"given":"Paulo S.","family":"Neto","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jardel D. S.","family":"Dyonisio","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jo\u00e3o F. S. S.","family":"Lemos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felipe","family":"K\u00fchne","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4011-0901","authenticated-orcid":false,"given":"Rodrigo S.","family":"Guerra","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"suffix":"Jr.","given":"Paulo L. J.","family":"Drews","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,4,21]]},"reference":[{"key":"12052_CR1","unstructured":"Anil R, Dai AM, Firat O et al (2023) Palm 2 technical report. arXiv:2305.10403"},{"key":"12052_CR2","doi-asserted-by":"crossref","unstructured":"Bisk Y, Zellers R, Bras RL et al (2019) Piqa: Reasoning about physical commonsense in natural language. arXiv:1911.11641","DOI":"10.1609\/aaai.v34i05.6239"},{"key":"12052_CR3","unstructured":"Brown T, Mann B, Ryder N et al (2020) Language models are few-shot learners. In: Larochelle H, Ranzato M, Hadsell R et al (eds) Advances in Neural Information Processing Systems, vol\u00a033. Curran Associates, Inc., pp 1877\u20131901, https:\/\/papers.nips.cc\/paper\/2020\/hash\/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html"},{"key":"12052_CR4","unstructured":"Chung HW, Hou L, Longpre S et al (2024) Scaling instruction-finetuned language models. Journal of Machine Learning Research 25(70):1\u201353. http:\/\/jmlr.org\/papers\/v25\/23-0870.html"},{"key":"12052_CR5","unstructured":"Cobbe K, Kosaraju V, Bavarian M et al (2021) Training verifiers to solve math word problems. arXiv:2110.14168"},{"key":"12052_CR6","doi-asserted-by":"publisher","unstructured":"Cui L, Wu Y, Liu S et al (2020) MuTual: A dataset for multi-turn dialogue reasoning. In: Jurafsky D, Chai J, Schluter N et al (eds) Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, pp 1406\u20131416, https:\/\/doi.org\/10.18653\/v1\/2020.acl-main.130. https:\/\/aclanthology.org\/2020.acl-main.130","DOI":"10.18653\/v1\/2020.acl-main.130"},{"key":"12052_CR7","unstructured":"Deng Y, Prasad K, Fernandez R et al (2024) Implicit chain of thought reasoning via knowledge distillation. https:\/\/openreview.net\/forum?id=9cumTvvlHG"},{"key":"12052_CR8","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.10256836","author":"L Gao","year":"2023","unstructured":"Gao L, Tow J, Abbasi B et al (2023). A framework for few-shot language model evaluation. https:\/\/doi.org\/10.5281\/zenodo.10256836 (https:\/\/zenodo.org\/records\/10256836)","journal-title":"A framework for few-shot language model evaluation"},{"key":"12052_CR9","unstructured":"Gunasekar S, Zhang Y, Aneja J et al (2023) Textbooks are all you need. CoRR arXiv:2306.11644"},{"key":"12052_CR10","doi-asserted-by":"crossref","unstructured":"Hayati SA, Jung T, Bodding-Long T et al (2024) Chain-of-instructions: Compositional instruction tuning on large language models. CoRR arXiv:2402.11532","DOI":"10.1609\/aaai.v39i22.34574"},{"key":"12052_CR11","unstructured":"Hendrycks D, Burns C, Basart S et al (2023) Aligning ai with shared human values. arXiv:2008.02275"},{"key":"12052_CR12","doi-asserted-by":"publisher","unstructured":"Ho N, Schmid L, Yun SY (2023) Large language models are reasoning teachers. In: Rogers A, Boyd-Graber J, Okazaki N (eds) Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Toronto, Canada, pp 14852\u201314882. https:\/\/doi.org\/10.18653\/v1\/2023.acl-long.830","DOI":"10.18653\/v1\/2023.acl-long.830"},{"key":"12052_CR13","doi-asserted-by":"publisher","unstructured":"Ji K, Xu J, Liang T et al (2025) The first few tokens are all you need: An efficient and effective unsupervised prefix fine-tuning method for reasoning models. https:\/\/doi.org\/10.13140\/RG.2.2.33772.07043, https:\/\/www.researchgate.net\/doi\/10.13140\/RG.2.2.33772.07043, unpublished, ResearchGate preprint","DOI":"10.13140\/RG.2.2.33772.07043"},{"key":"12052_CR14","unstructured":"Jiang AQ, Sablayrolles A, Mensch A et al (2023) Mistral 7b. https:\/\/arxiv.org\/abs\/2310.06825, arXiv:2310.06825"},{"key":"12052_CR15","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1162\/tacl_a_00300","volume":"8","author":"M Joshi","year":"2020","unstructured":"Joshi M, Chen D, Liu Y et al (2020) SpanBERT: improving pre-training by representing and predicting spans. Trans Assoc Comput Linguist 8:64\u201377. https:\/\/doi.org\/10.1162\/tacl_a_00300","journal-title":"Trans Assoc Comput Linguist"},{"key":"12052_CR16","unstructured":"Kojima T, Gu SS, Reid M et al (2022) Large language models are zero-shot reasoners. In: Oh AH, Agarwal A, Belgrave D et al (eds) Advances in Neural Information Processing Systems, https:\/\/openreview.net\/forum?id=e2TBb5y0yFf"},{"key":"12052_CR17","unstructured":"Lin Z, Gou Z, Gong Y et al (2024) Rho-1: Not all tokens are what you need. https:\/\/arxiv.org\/abs\/2404.07965, arXiv:2404.07965"},{"key":"12052_CR18","doi-asserted-by":"publisher","unstructured":"Liu J, Cui L, Liu H et al (2020) Logiqa: A challenge dataset for machine reading comprehension with logical reasoning. In: Bessiere C (ed) Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20. International Joint Conferences on Artificial Intelligence Organization, pp 3622\u20133628, https:\/\/doi.org\/10.24963\/ijcai.2020\/501, main track","DOI":"10.24963\/ijcai.2020\/501"},{"key":"12052_CR19","doi-asserted-by":"publisher","unstructured":"Mihaylov T, Clark P, Khot T et al (2018) Can a suit of armor conduct electricity? a new dataset for open book question answering. In: Riloff E, Chiang D, Hockenmaier J et al (eds) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, pp 2381\u20132391, https:\/\/doi.org\/10.18653\/v1\/D18-1260","DOI":"10.18653\/v1\/D18-1260"},{"key":"12052_CR20","doi-asserted-by":"publisher","unstructured":"Mosbach M, Pimentel T, Ravfogel S (2023) Few-shot fine-tuning vs. in-context learning: A fair comparison and evaluation. In: Rogers A, Boyd-Graber J, Okazaki N (eds) Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics, Toronto, Canada, pp 12284\u201312314. https:\/\/doi.org\/10.18653\/v1\/2023.findings-acl.779","DOI":"10.18653\/v1\/2023.findings-acl.779"},{"key":"12052_CR21","unstructured":"OpenAI (2024) Gpt-4 technical report. arXiv:2303.08774"},{"key":"12052_CR22","doi-asserted-by":"publisher","unstructured":"Patel A, Bhattamishra S, Goyal N (2021) Are NLP models really able to solve simple math word problems? In: Toutanova K, Rumshisky A, Zettlemoyer L et al (eds) Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Online, pp 2080\u20132094, https:\/\/doi.org\/10.18653\/v1\/2021.naacl-main.168","DOI":"10.18653\/v1\/2021.naacl-main.168"},{"key":"12052_CR23","unstructured":"Puerto H, Chubakov T, Zhu X et al (2024) Fine-tuning with divergent chains of thought boosts reasoning through self-correction in language models. arXiv:2407.03181"},{"key":"12052_CR24","unstructured":"Rein D, Hou BL, Stickland AC et al (2024) Gpqa: A graduate-level google-proof q&a benchmark. In: First Conference on Language Modeling, https:\/\/openreview.net\/forum?id=Ti67584b98"},{"key":"12052_CR25","doi-asserted-by":"publisher","unstructured":"Roy S, Roth D (2015) Solving general arithmetic word problems. In: M\u00e0rquez L, Callison-Burch C, Su J (eds) Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pp 1743\u20131752, https:\/\/doi.org\/10.18653\/v1\/D15-1202","DOI":"10.18653\/v1\/D15-1202"},{"key":"12052_CR26","doi-asserted-by":"publisher","unstructured":"Segal E, Efrat A, Shoham M et al (2020) A simple and effective model for answering multi-span questions. In: Webber B, Cohn T, He Y et al (eds) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, pp 3074\u20133080, https:\/\/doi.org\/10.18653\/v1\/2020.emnlp-main.248","DOI":"10.18653\/v1\/2020.emnlp-main.248"},{"key":"12052_CR27","unstructured":"Shalev Y, Feder A, Goldstein A (2024) Distributional reasoning in llms: parallel reasoning processes in multi-hop reasoning. arXiv:2406.13858"},{"key":"12052_CR28","doi-asserted-by":"publisher","unstructured":"Shridhar K, Stolfo A, Sachan M (2023) Distilling reasoning capabilities into smaller language models. In: Rogers A, Boyd-Graber J, Okazaki N (eds) Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics, Toronto, Canada, pp 7059\u20137073, https:\/\/doi.org\/10.18653\/v1\/2023.findings-acl.441","DOI":"10.18653\/v1\/2023.findings-acl.441"},{"key":"12052_CR29","doi-asserted-by":"publisher","unstructured":"Suzgun M, Scales N, Sch\u00e4rli N et al (2023) Challenging BIG-bench tasks and whether chain-of-thought can solve them. In: Rogers A, Boyd-Graber J, Okazaki N (eds) Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics, Toronto, Canada, pp 13003\u201313051, https:\/\/doi.org\/10.18653\/v1\/2023.findings-acl.824","DOI":"10.18653\/v1\/2023.findings-acl.824"},{"key":"12052_CR30","unstructured":"Team G (2024) Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv:2403.05530"},{"key":"12052_CR31","unstructured":"Team G, Mesnard T, Hardin C et al (2024) Gemma: Open models based on gemini research and technology. arXiv:2403.08295"},{"key":"12052_CR32","doi-asserted-by":"crossref","unstructured":"Trung L, Zhang X, Jie Z et al (2024) ReFT: Reasoning with reinforced fine-tuning. In: Ku LW, Martins A, Srikumar V (eds) Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Bangkok, Thailand, pp 7601\u20137614, https:\/\/aclanthology.org\/2024.acl-long.410","DOI":"10.18653\/v1\/2024.acl-long.410"},{"key":"12052_CR33","doi-asserted-by":"crossref","unstructured":"Wang X, Zhou D (2024) Chain-of-thought reasoning without prompting. arXiv:2402.10200","DOI":"10.52202\/079017-2123"},{"key":"12052_CR34","unstructured":"Wang Y, Hu S, Zhang Y et al (2024) How large language models implement chain-of-thought? https:\/\/openreview.net\/forum?id=b2XfOm3RJa"},{"key":"12052_CR35","unstructured":"Wei J, Bosma M, Zhao V et al (2022) Finetuned language models are zero-shot learners. In: International Conference on Learning Representations, https:\/\/openreview.net\/forum?id=gEZrGCozdqR"},{"key":"12052_CR36","doi-asserted-by":"publisher","unstructured":"Yadav V, Bethard S, Surdeanu M (2019) Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, https:\/\/doi.org\/10.18653\/v1\/d19-1260","DOI":"10.18653\/v1\/d19-1260"},{"key":"12052_CR37","unstructured":"Yang A, Yang B, Hui B et al (2024) Qwen2 technical report. arXiv:2407.10671"},{"key":"12052_CR38","unstructured":"YAO Y, Wu H, Guo Z et al (2024) Learning from correctness without prompting makes LLM efficient reasoner. In: First Conference on Language Modeling, https:\/\/openreview.net\/forum?id=dcbNzhVVQj"},{"key":"12052_CR39","unstructured":"Zhou D, Sch\u00e4rli N, Hou L et al (2023) Least-to-most prompting enables complex reasoning in large language models. In: The Eleventh International Conference on Learning Representations, https:\/\/openreview.net\/forum?id=WZH7099tgfM"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-026-12052-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-026-12052-9","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-026-12052-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T22:27:40Z","timestamp":1781648860000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-026-12052-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,21]]},"references-count":39,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2026,5]]}},"alternative-id":["12052"],"URL":"https:\/\/doi.org\/10.1007\/s00521-026-12052-9","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,21]]},"assertion":[{"value":"28 September 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 February 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 April 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Materials availability"}},{"value":"The code used to generate the results of this study is publicly available at:\n                      \n                      .","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code availability"}}],"article-number":"321"}}