{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T15:04:26Z","timestamp":1789571066587,"version":"build-2803163510"},"reference-count":39,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T00:00:00Z","timestamp":1789516800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Intell."],"abstract":"<jats:p>Artificial intelligence (AI) is increasingly embedded in organizational decision-making, requiring employees not only to use AI-generated recommendations but also to evaluate their quality and determine when reliance is appropriate. Although established research examines behavioral reliance on algorithmic and AI advice, fewer studies have approached performance under AI advice as an individual-differences assessment problem integrating psychometric structure, cognitive correlates, process indicators, and criterion-related evidence. This study developed and initially validated a CHC-informed, performance-based assessment of cognitive performance under AI advice using 24 organizational decision scenarios. The assessment was designed around three closely related content\/performance dimensions\u2014AI error detection, evidence integration, and cognitive control and adaptive reliance\u2014while also capturing confidence, response time, and reliance behavior. The validation sample comprised 780 employed adults in T\u00fcrkiye. Psychometric analyses included confirmatory factor analysis, multidimensional item response theory, response-time analyses, scenario-level logistic regression, measurement invariance, differential item functioning, and internal cross-validation. Results indicated a dominant general cognitive-performance component together with additional structure corresponding to the three theoretically specified dimensions. Assessment performance was positively associated with established cognitive measures, including ICAR-16 reasoning performance, working memory, processing speed, and attentional control, whereas associations with AI-related self-reports were generally weaker. Dimension-aligned analyses supported the expected associations of ICAR-16 with AI error detection and working memory with evidence integration. Performance was also moderately associated with concurrently assessed organizational decision quality (r = 0.408) and explained additional variance in this criterion beyond demographic and work characteristics, AI experience, conventional cognitive-performance measures, and AI-related self-reports (\u0394R2 = 0.103, p &lt; .001). In contrast, several hypothesized scenario-specific associations involving AI confidence, time pressure, interruptions, and resistance to confidently inaccurate advice were not supported. Overall, the findings provide initial evidence for a performance-based approach to assessing how employees evaluate and respond to AI advice, while indicating that the proposed scenario-specific mechanisms and group-comparability findings require further replication before consequential applications are considered.<\/jats:p>","DOI":"10.3390\/jintelligence14090223","type":"journal-article","created":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T13:50:27Z","timestamp":1789566627000},"page":"223","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Cognitive Performance Under AI Advice: Development and Initial Validation of a CHC-Informed Assessment for Organizational Decision-Making"],"prefix":"10.3390","volume":"14","author":[{"given":"Filiz","family":"Mizrak","sequence":"first","affiliation":[{"name":"Management Information Systems, Atlas University, Istanbul 34408, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"funding-acquisition","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0818-8522","authenticated-orcid":false,"given":"Turhan","family":"Karakaya","sequence":"additional","affiliation":[{"name":"Industrial Engineering, Dogus University, Istanbul 34680, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8940-0761","authenticated-orcid":false,"given":"Burcak Vatansever","family":"Durmaz","sequence":"additional","affiliation":[{"name":"Business Administration, Bahcesehir University, Istanbul 34350, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2","article-title":"Verification of forecasts expressed in terms of probability","volume":"78","author":"Brier","year":"1950","journal-title":"Monthly Weather Review"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"100014","DOI":"10.1016\/j.chbah.2023.100014","article-title":"MAILS\u2014Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies","volume":"1","author":"Carolus","year":"2023","journal-title":"Computers in Human Behavior: Artificial Humans"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies, Cambridge University Press.","DOI":"10.1017\/CBO9780511571312"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"103286","DOI":"10.1016\/j.techsoc.2026.103286","article-title":"The trust crisis in artificial intelligence: AI hallucinations and human\u2013AI collaboration","volume":"86","author":"Cheng","year":"2026","journal-title":"Technology in Society"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.intell.2014.01.004","article-title":"The International Cognitive Ability Resource: Development and initial validation of a public-domain measure","volume":"43","author":"Condon","year":"2014","journal-title":"Intelligence"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"769","DOI":"10.3758\/BF03196772","article-title":"Working memory span tasks: A methodological review and user\u2019s guide","volume":"12","author":"Conway","year":"2005","journal-title":"Psychonomic Bulletin & Review"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"29154","DOI":"10.1038\/s41598-025-12205-1","article-title":"Human-alignment influences the utility of AI-assisted decision making","volume":"15","year":"2025","journal-title":"Scientific Reports"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1037\/xge0000033","article-title":"Algorithm aversion: People erroneously avoid algorithms after seeing them err","volume":"144","author":"Dietvorst","year":"2015","journal-title":"Journal of Experimental Psychology: General"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"103932","DOI":"10.1016\/j.ergon.2026.103932","article-title":"Predicting human trust decisions during human\u2013AI collaborative decision-making in binary decision tasks","volume":"113","author":"Ding","year":"2026","journal-title":"International Journal of Industrial Ergonomics"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2593251","DOI":"10.1080\/12460125.2025.2593251","article-title":"Investigating appropriate reliance on AI-based decision support systems: The role of expertise, trust, and self-confidence","volume":"34","author":"Dogru","year":"2025","journal-title":"Journal of Decision Systems"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"103064","DOI":"10.1016\/j.techsoc.2025.103064","article-title":"Towards an integrative model of organizational human\u2013AI collaboration: A semi-systematic review of the current state of the art","volume":"84","author":"Leyer","year":"2026","journal-title":"Technology in Society"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"109906","DOI":"10.1016\/j.paid.2020.109906","article-title":"Using the International Cognitive Ability Resource as an open source tool to explore individual differences in cognitive ability","volume":"169","author":"Dworak","year":"2021","journal-title":"Personality and Individual Differences"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"143","DOI":"10.3758\/BF03203267","article-title":"Effects of noise letters upon the identification of a target letter in a nonsearch task","volume":"16","author":"Eriksen","year":"1974","journal-title":"Perception & Psychophysics"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1162\/089892902317361886","article-title":"Testing the efficiency and independence of attentional networks","volume":"14","author":"Fan","year":"2002","journal-title":"Journal of Cognitive Neuroscience"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1257\/089533005775196732","article-title":"Cognitive reflection and decision making","volume":"19","author":"Frederick","year":"2005","journal-title":"Journal of Economic Perspectives"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"108985","DOI":"10.1016\/j.chb.2026.108985","article-title":"Self-regulation and overreliance on artificial intelligence: Unpacking a paradox through a mixed-methods study in higher education","volume":"181","author":"Delgado","year":"2026","journal-title":"Computers in Human Behavior"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Gerlich, M. (2025). AI tools in society: Impacts on cognitive offloading and the future of critical thinking. Societies, 15, (1).","DOI":"10.3390\/soc15010006"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"808","DOI":"10.1038\/s44159-025-00499-x","article-title":"A cognitive approach to human\u2013AI complementarity in dynamic decision-making","volume":"4","author":"Gonzalez","year":"2025","journal-title":"Nature Reviews Psychology"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Guo, X., Jiao, Y., Huang, Z., and Liu, T. (2022). Joint modeling of response accuracy and time in between-item multidimensional tests based on bi-factor model. Frontiers in Psychology, 13.","DOI":"10.3389\/fpsyg.2022.763959"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1038\/s41598-025-30899-1","article-title":"Explainable AI and echo state networks calibrate trust in human machine interaction","volume":"16","author":"Hao","year":"2026","journal-title":"Scientific Reports"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1207\/S15327566IJCE0401_04","article-title":"Foundations for an empirically determined scale of trust in automated systems","volume":"4","author":"Jian","year":"2000","journal-title":"International Journal of Cognitive Ergonomics"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e39686","DOI":"10.1016\/j.heliyon.2024.e39686","article-title":"Meta AI literacy scale: Further validation and development of a short version","volume":"10","author":"Koch","year":"2024","journal-title":"Heliyon"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1007\/s10726-026-10007-y","article-title":"Generative AI\u2019s impact on individuals and organizations: An analysis based on human\u2013AI collaboration","volume":"35","author":"Latto","year":"2026","journal-title":"Group Decision and Negotiation"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1518\/hfes.46.1.50.30392","article-title":"Trust in automation: Designing for appropriate reliance","volume":"46","author":"Lee","year":"2004","journal-title":"Human Factors"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1007\/s10726-026-09980-1","article-title":"Advancing decision-making through AI-human collaboration: A systematic review and conceptual framework","volume":"35","author":"Li","year":"2026","journal-title":"Group Decision and Negotiation"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1007\/s10009-026-00840-6","article-title":"Dynamic calibration of trust and trustworthiness in AI-enabled systems","volume":"28","author":"Liebherr","year":"2026","journal-title":"International Journal on Software Tools for Technology Transfer"},{"key":"ref_27","first-page":"18","article-title":"Human\u2013AI collaboration: Trade-offs between performance and preferences","volume":"11","author":"Mayer","year":"2026","journal-title":"Cognitive Research: Principles and Implications"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.intell.2008.08.004","article-title":"CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research","volume":"37","author":"McGrew","year":"2009","journal-title":"Intelligence"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"11868","DOI":"10.1038\/s41598-026-44258-1","article-title":"Advice quality and source disclosure shape trust in AI-generated ethical advice","volume":"16","author":"Meincke","year":"2026","journal-title":"Scientific Reports"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Morales-Garc\u00eda, W. C., Sairitupa-Sanchez, L. Z., Morales-Garc\u00eda, S. B., and Morales-Garc\u00eda, M. (2024). Adaptation and psychometric properties of a brief version of the general self-efficacy scale for use with artificial intelligence (GSE-6AI) among university students. Frontiers in Education, 9.","DOI":"10.3389\/feduc.2024.1293437"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1082","DOI":"10.1111\/bjet.13411","article-title":"Design and validation of the AI literacy questionnaire: The affective, behavioural, cognitive and ethical approach","volume":"55","author":"Ng","year":"2024","journal-title":"British Journal of Educational Technology"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"5345","DOI":"10.1038\/s41598-026-34983-y","article-title":"Examining human reliance on artificial intelligence in decision making","volume":"16","author":"Pearson","year":"2026","journal-title":"Scientific Reports"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1007\/s00146-025-02422-7","article-title":"Exploring automation bias in human\u2013AI collaboration: A review and implications for explainable AI","volume":"41","author":"Romeo","year":"2026","journal-title":"AI & Society"},{"key":"ref_34","unstructured":"Flanagan, D. P., and McDonough, E. M. (2018). The Cattell-Horn-Carroll theory of cognitive abilities. Contemporary intellectual assessment: Theories, tests, and issues, Guilford Press. [4th ed.]."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1017\/S1930297500007622","article-title":"Investigating an alternate form of the cognitive reflection test","volume":"11","author":"Thomson","year":"2016","journal-title":"Judgment and Decision Making"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"498","DOI":"10.3758\/BF03192720","article-title":"An automated version of the operation span task","volume":"37","author":"Unsworth","year":"2005","journal-title":"Behavior Research Methods"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"4785","DOI":"10.1007\/s10639-023-12015-w","article-title":"Artificial intelligence self-efficacy: Scale development and validation","volume":"29","author":"Wang","year":"2024","journal-title":"Education and Information Technologies"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1017\/psy.2026.10097","article-title":"A joint model for graded responses and response times","volume":"91","author":"Zhang","year":"2026","journal-title":"Psychometrika"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Zhu, Q., Li, X., Dong, Y., Chang, P., and Fan, M. (2026). Not all cognitive offloading is equal: Distinguishing dependent and autonomous offloading to generative AI. Frontiers in Psychology, 17.","DOI":"10.3389\/fpsyg.2026.1878629"}],"container-title":["Journal of Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-3200\/14\/9\/223\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T14:27:13Z","timestamp":1789568833000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-3200\/14\/9\/223"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,16]]},"references-count":39,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["jintelligence14090223"],"URL":"https:\/\/doi.org\/10.3390\/jintelligence14090223","relation":{},"ISSN":["2079-3200"],"issn-type":[{"value":"2079-3200","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,16]]}}}