{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T04:26:10Z","timestamp":1787286370505,"version":"build-2736575974"},"reference-count":51,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2026,8,19]],"date-time":"2026-08-19T00:00:00Z","timestamp":1787097600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62377003"],"award-info":[{"award-number":["62377003"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"award":["62377003"],"award-info":[{"award-number":["62377003"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"publisher"}]},{"name":"STI 2030\u2014Major Projects","award":["2021ZD0200500"],"award-info":[{"award-number":["2021ZD0200500"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Intell."],"abstract":"<jats:p>In an era of information explosion and overload, individuals face a constant influx of mixed-quality, contradictory data. Navigating this complex landscape requires critical thinking, which enables individuals to dynamically adjust their reasoning and refine judgments. Consequently, accurately measuring this dynamic ability is essential. However, existing assessments predominantly focus on single-round argumentation and fail to capture the iterative process, potentially yielding unrepresentative evaluations of real-world performance. To address this gap, this study develops the Iterative Argumentation Task (IAT), a novel assessment tool that presents participants with a sequence of interrelated passages simulating real-world information flow. By capturing the process of evaluation, position changes, and argument revisions, the IAT evaluates three ability dimensions: single-round argument analysis, inter-argument relationship analysis, and argument integration. We recruited 355 undergraduates to validate the IAT. Confirmatory factor analysis provided evidence for the proposed measurement structure. Furthermore, correlations with established critical thinking and reasoning tests demonstrated convergent and discriminant validity, respectively. Moreover, performance differences between the undergraduates and 33 debaters established known-groups validity. Finally, latent profile analysis identified four distinct profiles across the IAT dimensions. Together, these findings establish the IAT as a theoretically sound and practically effective tool for assessing critical thinking.<\/jats:p>","DOI":"10.3390\/jintelligence14080190","type":"journal-article","created":{"date-parts":[[2026,8,19]],"date-time":"2026-08-19T13:14:01Z","timestamp":1787145241000},"page":"190","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Beyond Static Snapshots: Assessing Critical Thinking Ability Through Iterative Argumentation Processes"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6464-2326","authenticated-orcid":false,"given":"Liming","family":"Jiang","sequence":"first","affiliation":[{"name":"Faculty of Psychology, Beijing Normal University, Beijing 100875, China"},{"name":"Microsoft Research Asia, Beijing 100080, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fang","family":"Luo","sequence":"additional","affiliation":[{"name":"Faculty of Psychology, Beijing Normal University, Beijing 100875, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8407-8718","authenticated-orcid":false,"given":"Xuetao","family":"Tian","sequence":"additional","affiliation":[{"name":"Faculty of Psychology, Beijing Normal University, Beijing 100875, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1102","DOI":"10.3102\/0034654308326084","article-title":"Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis","volume":"78","author":"Abrami","year":"2008","journal-title":"Review of Educational Research"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Allagui, B. (2024). A scaffolding intervention to improve self-efficacy in source-based argumentative writing. Frontiers in Psychology, 15.","DOI":"10.3389\/fpsyg.2024.1454104"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1080\/23752696.2018.1449128","article-title":"Threats to the validity of the Collegiate Learning Assessment (CLA+) as a measure of critical thinking skills and implications for learning gain","volume":"3","author":"Aloisi","year":"2018","journal-title":"Higher Education Pedagogies"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1080\/002202799183133","article-title":"Conceptualizing critical thinking","volume":"31","author":"Bailin","year":"1999","journal-title":"Journal of Curriculum Studies"},{"key":"ref_5","unstructured":"Black, B., Chislett, J., Thompson, A., Thwaites, G., and Thwaites, J. (2008). Critical thinking\u2014A definition and taxonomy for Cambridge Assessment, Cambridge Assessment."},{"key":"ref_6","unstructured":"Brookfield, S. D. (2011). Teaching for critical thinking: Tools and techniques to help students question their assumptions, John Wiley & Sons."},{"key":"ref_7","first-page":"197","article-title":"The impact of recurrent on-line synchronous scientific argumentation on students\u2019 argumentation and conceptual change","volume":"15","author":"Chen","year":"2012","journal-title":"Journal of Educational Technology & Society"},{"key":"ref_8","unstructured":"College of the Mainland (2015). College-level examination program\u00ae 2015\u20132016, College of the Mainland."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.intell.2014.01.004","article-title":"The international cognitive ability resource: Development and initial validation of a public-domain measure","volume":"43","author":"Condon","year":"2014","journal-title":"Intelligence"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"187","DOI":"10.3262\/ZP2502187","article-title":"Assessing critical thinking skills in data literacy: A digital performance assessment","volume":"71","author":"Cui","year":"2025","journal-title":"Zeitschrift f\u00fcr P\u00e4dagogik"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"i-60","DOI":"10.1002\/j.2333-8504.2011.tb02250.x","article-title":"Writing assessment and cognition","volume":"2011","author":"Deane","year":"2011","journal-title":"ETS Research Report Series"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Dwyer, C. P. (2023). An evaluative review of barriers to critical thinking in educational and real-world settings. Journal of Intelligence, 11, (6).","DOI":"10.3390\/jintelligence11060105"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.tsc.2013.12.004","article-title":"An integrated critical thinking framework for the 21st century","volume":"12","author":"Dwyer","year":"2014","journal-title":"Thinking Skills and Creativity"},{"key":"ref_14","unstructured":"Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. Teaching thinking skills: Theory and practice, W. H. Freeman."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1080\/00405849309543594","article-title":"Critical thinking assessment","volume":"32","author":"Ennis","year":"1993","journal-title":"Theory into Practice"},{"key":"ref_16","unstructured":"Ennis, R. H., and Weir, E. E. (1985). The Ennis-Weir critical thinking essay test: An instrument for teaching and testing, Midwest Publications."},{"key":"ref_17","unstructured":"Facione, P. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction (The Delphi Report), The California Academic Press."},{"key":"ref_18","first-page":"17","article-title":"Mapping decisions and arguments","volume":"30","author":"Facione","year":"2015","journal-title":"Inquiry: Critical Thinking Across the Disciplines"},{"key":"ref_19","unstructured":"Floridi, L. (2014). The fourth revolution: How the infosphere is reshaping human reality, OUP Oxford."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"436","DOI":"10.2307\/798843","article-title":"The constant comparative method of qualitative analysis","volume":"12","author":"Glaser","year":"1965","journal-title":"Social Problems"},{"key":"ref_21","unstructured":"Halpern, D. F. (1998). Teaching critical thinking for transfer across domains, American Psychologist."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Halpern, D. F. (2003). Thought and knowledge: An introduction to critical thinking, Erlbaum.","DOI":"10.4324\/9781410606433"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1177\/0741088312451260","article-title":"Modeling and remodeling writing","volume":"29","author":"Hayes","year":"2012","journal-title":"Written Communication"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1207\/s15326985ep3901_2","article-title":"Reflective judgment: Theory and research on the development of epistemic assumptions through adulthood","volume":"39","author":"King","year":"2004","journal-title":"Educational Psychologist"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1177\/0193841X07303318","article-title":"The collegiate learning assessment: Facts and fantasies","volume":"31","author":"Klein","year":"2007","journal-title":"Evaluation Review"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.tsc.2009.02.001","article-title":"Assessing students\u2019 critical thinking performance: Urging for measurements using multi-response format","volume":"4","author":"Ku","year":"2009","journal-title":"Thinking Skills and Creativity"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Kuhn, D. (1991). The skills of argument, Cambridge University Press.","DOI":"10.1017\/CBO9780511571350"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1007\/s11245-016-9373-4","article-title":"A role for reasoning in a dialogic approach to critical thinking","volume":"37","author":"Kuhn","year":"2018","journal-title":"Topoi"},{"key":"ref_29","first-page":"66","article-title":"Argumentation as core curriculum","volume":"1","author":"Kuhn","year":"2015","journal-title":"Learning: Research and Practice"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"15","DOI":"10.3102\/0013189X020008015","article-title":"Complex, performance-based assessment: Expectations and validation criteria","volume":"20","author":"Linn","year":"1991","journal-title":"Educational Researcher"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"352","DOI":"10.3102\/0013189X12459679","article-title":"Measuring learning outcomes in higher education: Motivation matters","volume":"41","author":"Liu","year":"2012","journal-title":"Education Researcher"},{"key":"ref_32","first-page":"1","article-title":"Assessing critical thinking in higher education: Current state and directions for next-generation assessment","volume":"2014","author":"Liu","year":"2014","journal-title":"ETS Research Report Series"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1017\/S0140525X10000968","article-title":"Why do humans reason? Arguments for an argumentative theory","volume":"34","author":"Mercier","year":"2011","journal-title":"Behavioral and Brain Sciences"},{"key":"ref_34","unstructured":"Miao, F., and Shiohira, K. (2024). AI competency framework for students, UNESCO Publishing."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1037\/h0049330","article-title":"Recency and primacy in persuasion as a function of the timing of speeches and measurements","volume":"59","author":"Miller","year":"1959","journal-title":"The Journal of Abnormal and Social Psychology"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/j.edurev.2012.12.002","article-title":"Do instructional interventions influence college students\u2019 critical thinking skills? A meta-analysis","volume":"9","author":"Niu","year":"2013","journal-title":"Educational Research Review"},{"key":"ref_37","unstructured":"Paul, R., and Elder, L. (2014). Critical thinking: Tools for taking charge of your learning and your life, Pearson Education."},{"key":"ref_38","first-page":"284","article-title":"Critical thinking as a self-regulatory process component in teaching and learning","volume":"22","author":"Phan","year":"2010","journal-title":"Psicothema"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.lindif.2013.06.004","article-title":"Assessment of argumentative writing and critical thinking in higher education: Educational correlates and gender differences","volume":"28","author":"Preiss","year":"2013","journal-title":"Learning and Individual Differences"},{"key":"ref_40","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI Blog"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"63","DOI":"10.5817\/SP2019-4-3","article-title":"Measuring production and comprehension of written arguments in upper-elementary grades","volume":"24","author":"Reznitskaya","year":"2019","journal-title":"Studia Paedagogica"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"4787","DOI":"10.1038\/s41467-018-06930-7","article-title":"The spread of low-credibility content by social bots","volume":"9","author":"Shao","year":"2018","journal-title":"Nature Communications"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1111\/j.1745-3984.1993.tb00424.x","article-title":"Sampling variability of performance assessments","volume":"30","author":"Shavelson","year":"1993","journal-title":"Journal of Educational Measurement"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1177\/0741088390007002004","article-title":"Transforming texts: Constructive processes in reading and writing","volume":"7","author":"Spivey","year":"1990","journal-title":"Written Communication"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1080\/10705511.2013.824781","article-title":"Statistical power to detect the correct number of classes in latent profile analysis","volume":"20","author":"Tein","year":"2013","journal-title":"Structural Equation Modeling: A Multidisciplinary Journal"},{"key":"ref_46","unstructured":"Toulmin, S. (1958). The uses of argument, Cambridge University Press."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1080\/00220272.2012.668938","article-title":"A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies","volume":"44","author":"Voogt","year":"2012","journal-title":"Journal of Curriculum Studies"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1016\/0959-4752(91)90013-X","article-title":"Learning to reason via instruction in argumentation","volume":"1","author":"Voss","year":"1991","journal-title":"Learning and Instruction"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Walton, D. N. (1998). The new dialectic: Conversational contexts of argument, University of Toronto Press.","DOI":"10.3138\/9781442681859"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"24824","DOI":"10.52202\/068431-1800","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., and Liu, Q. (2019). ERNIE: Enhanced language representation with informative entities. Proceedings of the 57th annual meeting of the association for computational linguistics, Association for Computational Linguistics (ACL).","DOI":"10.18653\/v1\/P19-1139"}],"container-title":["Journal of Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-3200\/14\/8\/190\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T04:15:11Z","timestamp":1787285711000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-3200\/14\/8\/190"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,19]]},"references-count":51,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2026,8]]}},"alternative-id":["jintelligence14080190"],"URL":"https:\/\/doi.org\/10.3390\/jintelligence14080190","relation":{},"ISSN":["2079-3200"],"issn-type":[{"value":"2079-3200","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,19]]}}}