{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,4]],"date-time":"2026-08-04T18:13:11Z","timestamp":1785867191233,"version":"3.56.0"},"reference-count":39,"publisher":"Public Library of Science (PLoS)","issue":"8","license":[{"start":{"date-parts":[[2023,8,25]],"date-time":"2023-08-25T00:00:00Z","timestamp":1692921600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["PID2020-112894GB-I00"],"award-info":[{"award-number":["PID2020-112894GB-I00"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>Exploratory factor analysis is widely used for item analysis in the earlier stages of scale development, usually with large pools of items. In this scenario, the presence of inappropriate or ineffective items can hamper the process of analysis, making it very difficult to correctly assess dimensionality and structure. To minimize, this (quite frequent) problem, we propose and implement a simple procedure designed to flag potentially problematic items before we specify any particular factorial solution. The procedure defines regions of item appropriateness and efficiency based on the combined impact of two prior item features: extremeness and consistency. The general proposal is related to the most widely used frameworks for item analysis. The limits of the appropriateness regions are obtained by extensive simulation in conditions that mimic those found in applications. An Item Response Theory index of prior item efficiency is then defined, and a combined approach for selecting the most effective and problem-free item sub-set is proposed. The proposal is useful to normal-range measures, such as questionnaire surveys that elicit reports about non-extreme attitudes, facts, beliefs or states, or personality questionnaires that measure normal-range constructs. The procedure is implemented in a freeware software.<\/jats:p>","DOI":"10.1371\/journal.pone.0290611","type":"journal-article","created":{"date-parts":[[2023,8,25]],"date-time":"2023-08-25T17:34:10Z","timestamp":1692984850000},"page":"e0290611","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":19,"title":["Gulliksen\u2019s pool: A quick tool for preliminary detection of problematic items in item factor analysis"],"prefix":"10.1371","volume":"18","author":[{"given":"Pere J.","family":"Ferrando","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5369-3099","authenticated-orcid":true,"given":"Urbano","family":"Lorenzo-Seva","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M. Teresa","family":"Bargall\u00f3-Escriv\u00e0","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2023,8,25]]},"reference":[{"issue":"3","key":"pone.0290611.ref001","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1037\/1040-3590.7.3.286","article-title":"Factor analysis in the development and refinement of clinical assessment instruments","volume":"7","author":"FJ Floyd","year":"1995","journal-title":"Psychol Assess"},{"key":"pone.0290611.ref002","author":"RL Johnson","year":"2016"},{"issue":"3","key":"pone.0290611.ref003","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1037\/1040-3590.12.3.287","article-title":"Factor analysis and scale revision.","volume":"12","author":"SP Reise","year":"2000","journal-title":"Psychol Assess"},{"issue":"1","key":"pone.0290611.ref004","first-page":"7","article-title":"Decalogue for the Factor Analysis of Test Items.","volume":"34","author":"PJ Ferrando","year":"2022","journal-title":"Psicothema"},{"key":"pone.0290611.ref005","first-page":"54","volume-title":"Functional Psychological Testing.","author":"RB Cattell","year":"1986"},{"key":"pone.0290611.ref006","volume-title":"Factor analysis and related methods","author":"RP McDonald","year":"1985"},{"issue":"1\u20132","key":"pone.0290611.ref007","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1108\/LM-08-2022-0076","article-title":"Library\u2019s quality impact on satisfaction and improvement priorities","volume":"44","author":"S. Thuannadee","year":"2023","journal-title":"Library Management"},{"key":"pone.0290611.ref008","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1146\/annurev.clinpsy.032408.153553","article-title":"Item response theory and clinical measurement","volume":"5","author":"SP Reise","year":"2009","journal-title":"Annual Review of Clinical Psychology"},{"key":"pone.0290611.ref009","doi-asserted-by":"crossref","DOI":"10.1037\/13240-000","volume-title":"Theory of mental tests","author":"H. Gulliksen","year":"1950"},{"key":"pone.0290611.ref010","volume-title":"Statistical Theories of Mental Test Scores, Reading, Mass.:","author":"FM Lord","year":"1968"},{"issue":"1","key":"pone.0290611.ref011","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1177\/014662169001400103","article-title":"Some contrasts between maximum likelihood factor analysis and alpha factor analysis.","volume":"14","author":"HF Kaiser","year":"1990","journal-title":"Appl Psychol Meas"},{"issue":"3","key":"pone.0290611.ref012","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1177\/001316447503500311","article-title":"The concept of efficiency in item analysis.","volume":"35","author":"RJ Hofmann","year":"1975","journal-title":"Educ Psychol Meas"},{"key":"pone.0290611.ref013","first-page":"153","volume-title":"Educational measurement.","author":"S Henryson","year":"1971"},{"key":"pone.0290611.ref014","author":"SA Livingston","year":"2004"},{"issue":"5","key":"pone.0290611.ref015","article-title":"Essentials of Visual Diagnosis of Test Items. Logical, Illogical, and Anomalous Patterns in Tests Items to be Detected.","volume":"27","author":"J. Mets\u00e4muuronen","year":"2022","journal-title":"Practical Assessment, Research & Evaluation."},{"key":"pone.0290611.ref016","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1207\/s15327906mbr2903_2","article-title":"A unidimensional latent trait model for continuous item responses.","volume":"29","author":"GJ Mellenbergh","year":"1994","journal-title":"Multivariate Behav Res."},{"issue":"2","key":"pone.0290611.ref017","first-page":"395","article-title":"The Psychometric Toolbox: an Excel package for use in measurement and psychometrics courses.","volume":"38","author":"PJ Ferrando","year":"2017","journal-title":"Psicol\u00f3gica"},{"issue":"4","key":"pone.0290611.ref018","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1007\/BF02290130","article-title":"Variation in test validity with variation in the distribution of item difficulties, number of items, and degree of their intercorrelation.","volume":"11","author":"HE Brogden","year":"1946","journal-title":"Psychometrika"},{"issue":"2","key":"pone.0290611.ref019","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1007\/BF02288781","article-title":"The relation of the reliability of multiple-choice test to the distribution of item difficulties.","volume":"17","author":"FM Lord","year":"1952","journal-title":"Psychometrika"},{"issue":"2","key":"pone.0290611.ref020","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1007\/BF02288778","article-title":"Efficiency of multiple-choice tests as a function of spread of item difficulties.","volume":"17","author":"LJ Cronbach","year":"1952","journal-title":"Psychometrika"},{"issue":"1","key":"pone.0290611.ref021","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1111\/j.1745-3984.1988.tb00288.x","article-title":"Item discrimination: When more is worse.","volume":"25","author":"GN Masters","year":"1988","journal-title":"J Educ Meas."},{"key":"pone.0290611.ref022","doi-asserted-by":"crossref","first-page":"2025","DOI":"10.1017\/S0033291716000520","article-title":"Item response theory and the measurement of psychiatric constructs: Some empirical and conceptual issues and challenges.","volume":"46","author":"SP Reise","year":"2016","journal-title":"Psychol Med"},{"issue":"3","key":"pone.0290611.ref023","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1016\/0191-8869(91)90115-R","article-title":"Does item homogeneity indicate internal consistency or item redundancy in psychometric scales?.","volume":"12","author":"GJ Boyle","year":"1991","journal-title":"Pers Individ Dif"},{"key":"pone.0290611.ref024","volume-title":"Test theory: A unified treatment","author":"RP McDonald","year":"1999"},{"key":"pone.0290611.ref025","first-page":"205","volume-title":"Testing structural equation models","author":"B. Muth\u00e9n","year":"1993"},{"issue":"2","key":"pone.0290611.ref026","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1177\/014662169101500201","article-title":"Three approaches to determining the dimensionality of binary items.","volume":"15","author":"M Roznowski","year":"1991","journal-title":"Appl Psychol Meas"},{"issue":"4","key":"pone.0290611.ref027","doi-asserted-by":"crossref","first-page":"296","DOI":"10.5964\/meth.7185","article-title":"MSA: The Forgotten Index for Identifying Inappropriate Items Before Computing Exploratory Item Factor Analysis.","volume":"17","author":"U Lorenzo-Seva","year":"2021","journal-title":"Methodology"},{"key":"pone.0290611.ref028","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1177\/001316447403400115","article-title":"Little Jiffy, Mark IV","volume":"34","author":"HF Kaiser","year":"1974","journal-title":"Educ Psychol Meas"},{"key":"pone.0290611.ref029","volume-title":"Psychometric Methods.","author":"JP Guilford","year":"1954"},{"key":"pone.0290611.ref030","volume-title":"Applications of item response theory to practical testing problems","author":"FM Lord","year":"1980"},{"key":"pone.0290611.ref031","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF02289137","article-title":"\u201cBest possible\u201d systematic estimates of communalities","volume":"21","author":"L. Guttman","year":"1956","journal-title":"Psychometrika"},{"key":"pone.0290611.ref032","first-page":"202","article-title":"Redundancy analysis: An alternative to canonical correlation analysis.","volume":"42","author":"AL Van den Wallenberg","year":"1977","journal-title":"Psychometrika"},{"issue":"6","key":"pone.0290611.ref033","doi-asserted-by":"crossref","first-page":"2665","DOI":"10.3758\/s13428-021-01750-y","article-title":"SOLOMON: a method for splitting a sample into equivalent subsamples in factor analysis","volume":"54","author":"U. Lorenzo-Seva","year":"2022","journal-title":"Behav Res Methods"},{"key":"pone.0290611.ref034","first-page":"1","article-title":"Should suicidal ideation be regarded as a dimension, a unipolar trait or a mixture? A model-based analysis at the score level","author":"F Morales-Vives","year":"2022","journal-title":"Curr Psychol."},{"issue":"7","key":"pone.0290611.ref035","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1016\/j.jclinepi.2012.12.015","article-title":"Item reduction based on rigorous methodological guidelines is necessary to maintain validity when shortening composite measurement scales.","volume":"66","author":"C Goetz","year":"2013","journal-title":"J Clin Epidemiol"},{"issue":"2","key":"pone.0290611.ref036","doi-asserted-by":"crossref","first-page":"i","DOI":"10.1002\/j.2333-8504.1951.tb00219.x","article-title":"A maximum likelihood approach to test scores.","volume":"1951","author":"FM Lord","year":"1951","journal-title":"ETS Research Bulletin Series"},{"key":"pone.0290611.ref037","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-89976-3","volume-title":"Multidimensional item response theory","author":"MD Reckase","year":"2009"},{"issue":"4","key":"pone.0290611.ref038","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1207\/S15327906MBR3704_05","article-title":"Theoretical and empirical comparisons between two models for continuous item response.","volume":"37","author":"PJ Ferrando","year":"2002","journal-title":"Multivariate Behav Res"},{"issue":"3","key":"pone.0290611.ref039","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1177\/014662168801200305","article-title":"Full-information item factor analysis.","volume":"12","author":"RD Bock","year":"1988","journal-title":"Appl Psychol Meas"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0290611","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,25]],"date-time":"2023-08-25T17:34:41Z","timestamp":1692984881000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0290611"}},"subtitle":[],"editor":[{"given":"Juan Antonio","family":"Garc\u00eda","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2023,8,25]]},"references-count":39,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2023,8,25]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0290611","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,25]]}}}