{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T12:07:42Z","timestamp":1774958862935,"version":"3.50.1"},"reference-count":146,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2021,9,28]],"date-time":"2021-09-28T00:00:00Z","timestamp":1632787200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100009238","name":"Swedish Armed Forces","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100009238","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Swedish Defense Materiel Administration"},{"DOI":"10.13039\/501100001858","name":"Swedish Governmental Agency for Innovation Systems","doi-asserted-by":"crossref","award":["2017-04874"],"award-info":[{"award-number":["2017-04874"]}],"id":[{"id":"10.13039\/501100001858","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100011898","name":"Marianne and Marcus Wallenberg Foundation","doi-asserted-by":"crossref","award":["2017.0071"],"award-info":[{"award-number":["2017.0071"]}],"id":[{"id":"10.13039\/501100011898","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Alexander von Humboldt (AvH) Foundation"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2022,1,31]]},"abstract":"<jats:p>A meaningful and deep understanding of the human aspects of software engineering (SE) requires psychological constructs to be considered. Psychology theory can facilitate the systematic and sound development as well as the adoption of instruments (e.g., psychological tests, questionnaires) to assess these constructs. In particular, to ensure high quality, the psychometric properties of instruments need evaluation. In this article, we provide an introduction to psychometric theory for the evaluation of measurement instruments for SE researchers. We present guidelines that enable using existing instruments and developing new ones adequately. We conducted a comprehensive review of the psychology literature framed by the Standards for Educational and Psychological Testing. We detail activities used when operationalizing new psychological constructs, such as item pooling, item review, pilot testing, item analysis, factor analysis, statistical property of items, reliability, validity, and fairness in testing and test bias. We provide an openly available example of a psychometric evaluation based on our guideline. We hope to encourage a culture change in SE research towards the adoption of established methods from psychology. To improve the quality of behavioral research in SE, studies focusing on introducing, validating, and then using psychometric instruments need to be more common.<\/jats:p>","DOI":"10.1145\/3469888","type":"journal-article","created":{"date-parts":[[2021,9,28]],"date-time":"2021-09-28T20:49:24Z","timestamp":1632862164000},"page":"1-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines"],"prefix":"10.1145","volume":"31","author":[{"given":"Daniel","family":"Graziotin","sequence":"first","affiliation":[{"name":"University of Stuttgart, Institute of Software Engineering, Stuttgart, Germany"}]},{"given":"Per","family":"Lenberg","sequence":"additional","affiliation":[{"name":"Chalmers and the University of Gothenburg, Dept. of Computer Science and Engineering, Gothenbug, Sweden"}]},{"given":"Robert","family":"Feldt","sequence":"additional","affiliation":[{"name":"Chalmers and the University of Gothenburg, Dept. of Computer Science and Engineering, Gothenbug, Sweden"}]},{"given":"Stefan","family":"Wagner","sequence":"additional","affiliation":[{"name":"University of Stuttgart, Institute of Software Engineering, Stuttgart, Germany"}]}],"member":"320","published-online":{"date-parts":[[2021,9,28]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Encyclopedia of Social Sciences Research Methods","author":"Abdi Herv\u00e9","unstructured":"Herv\u00e9 Abdi . 2003. Factor rotations in factor analyses . In Encyclopedia of Social Sciences Research Methods , A. Lewis-Beck M., Bryman and Futing T. (Eds.). SAGE , Thousand Oaks, CA , 792\u2013795. Herv\u00e9 Abdi. 2003. Factor rotations in factor analyses. In Encyclopedia of Social Sciences Research Methods, A. Lewis-Beck M., Bryman and Futing T. (Eds.). SAGE, Thousand Oaks, CA, 792\u2013795."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1046\/j.1365-2648.1994.20010196.x"},{"key":"e_1_2_2_3_1","volume-title":"American Psychological Association","author":"American Educational Research Association","unstructured":"American Educational Research Association , American Psychological Association , National Council on Measurement in Education, and Joint Committee on Standards for Educational and Psychological Testing (U.S.). 2014. Standards for Educational and Psychological Testing. American Educational Research Association, Washington, DC. American Educational Research Association, American Psychological Association, National Council on Measurement in Education, and Joint Committee on Standards for Educational and Psychological Testing (U.S.). 2014. Standards for Educational and Psychological Testing. American Educational Research Association, Washington, DC."},{"key":"e_1_2_2_4_1","unstructured":"Anne Anastasi and Susana Urbina. 1997. Psychological Testing. Prentice Hall Upper Saddle River NJ. Anne Anastasi and Susana Urbina. 1997. Psychological Testing. Prentice Hall Upper Saddle River NJ."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02294709"},{"key":"e_1_2_2_6_1","volume-title":"Encyclopedia of Autism Spectrum Disorders","author":"Berger Michael","unstructured":"Michael Berger . 2013. Criterion-referenced testing . In Encyclopedia of Autism Spectrum Disorders . Springer New York , 823\u2013823. DOI:https:\/\/doi.org\/10.1007\/978-1-4419-1698-3_146 Michael Berger. 2013. Criterion-referenced testing. In Encyclopedia of Autism Spectrum Disorders. Springer New York, 823\u2013823. DOI:https:\/\/doi.org\/10.1007\/978-1-4419-1698-3_146"},{"key":"e_1_2_2_7_1","volume-title":"Encyclopedia of Autism Spectrum Disorders","author":"Berger Michael","year":"2063","unstructured":"Michael Berger . 2013. Norm-referenced testing . In Encyclopedia of Autism Spectrum Disorders . Springer New York , 2063 \u20132064. DOI:https:\/\/doi.org\/10.1007\/978-1-4419-1698-3_451 Michael Berger. 2013. Norm-referenced testing. In Encyclopedia of Autism Spectrum Disorders. Springer New York, 2063\u20132064. DOI:https:\/\/doi.org\/10.1007\/978-1-4419-1698-3_451"},{"key":"e_1_2_2_8_1","doi-asserted-by":"crossref","unstructured":"R. Boulkedid H. Abdoul M. Loustau O. Sibony and C. Alberti. 2011. Using and reporting the delphi method for selecting healthcare quality indicators: A systematic review.PLoS One 6 (2011) e20476. DOI:https:\/\/doi.org\/10.1371\/journal.pone.0020476 R. Boulkedid H. Abdoul M. Loustau O. Sibony and C. Alberti. 2011. Using and reporting the delphi method for selecting healthcare quality indicators: A systematic review.PLoS One 6 (2011) e20476. DOI:https:\/\/doi.org\/10.1371\/journal.pone.0020476","DOI":"10.1371\/journal.pone.0020476"},{"key":"e_1_2_2_9_1","unstructured":"Norman M. Bradburn Seymour Sudman and Brian Wansink. 2004. Asking Questions. John Wiley & Sons Hoboken NJ. Norman M. Bradburn Seymour Sudman and Brian Wansink. 2004. Asking Questions. John Wiley & Sons Hoboken NJ."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1207\/S15327906MBR3601_05"},{"key":"e_1_2_2_11_1","first-page":"528","article-title":"The goal question metric approach","volume":"1","author":"Basili Victor R.","year":"1994","unstructured":"Victor R. Basili , Gianluigi Caldiera , and H. Dieter Rombach . 1994 . The goal question metric approach . Encyc. Softw. Eng. 1 , 1 (1994), 528 \u2013 532 . Victor R. Basili, Gianluigi Caldiera, and H. Dieter Rombach. 1994. The goal question metric approach. Encyc. Softw. Eng. 1, 1 (1994), 528\u2013532.","journal-title":"Encyc. Softw. Eng."},{"key":"e_1_2_2_12_1","first-page":"1","article-title":"Fairness","volume":"1","author":"Cambridge English Dictionary","year":"2018","unstructured":"English Dictionary Cambridge . 2018 . Fairness . Cambridge Eng. Dict. 1 , 1 (2018), 1 . Retrieved from https:\/\/dictionary.cambridge.org\/dictionary\/english\/fairness. English Dictionary Cambridge. 2018. Fairness. Cambridge Eng. Dict. 1, 1 (2018), 1. Retrieved from https:\/\/dictionary.cambridge.org\/dictionary\/english\/fairness.","journal-title":"Cambridge Eng. Dict."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0046016"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1071-5819(02)00137-4"},{"key":"e_1_2_2_15_1","volume-title":"Logical Foundations of Probability","author":"Carnap Rudolf","unstructured":"Rudolf Carnap . 1962. Logical Foundations of Probability . University of Chicago Press , Chicago. Rudolf Carnap. 1962. Logical Foundations of Probability. University of Chicago Press, Chicago."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2020.3042683"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15327906mbr0102_10"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380330"},{"key":"e_1_2_2_19_1","volume-title":"Empirical Methods and Studies in Software Engineering, Gerhard Goos, Juris Hartmanis, Jan van Leeuwen","author":"Ciolkowski Marcus","unstructured":"Marcus Ciolkowski , Oliver Laitenberger , Sira Vegas , and Stefan Biffl . 2003. Practical experiences in the design and conduct of surveys in empirical software engineering . In Empirical Methods and Studies in Software Engineering, Gerhard Goos, Juris Hartmanis, Jan van Leeuwen , Reidar Conradi, and Alf Inge Wang (Eds.). Springer Berlin , 104\u2013128. DOI:https:\/\/doi.org\/10.1007\/978-3-540-45143-3_7 Marcus Ciolkowski, Oliver Laitenberger, Sira Vegas, and Stefan Biffl. 2003. Practical experiences in the design and conduct of surveys in empirical software engineering. In Empirical Methods and Studies in Software Engineering, Gerhard Goos, Juris Hartmanis, Jan van Leeuwen, Reidar Conradi, and Alf Inge Wang (Eds.). Springer Berlin, 104\u2013128. DOI:https:\/\/doi.org\/10.1007\/978-3-540-45143-3_7"},{"key":"e_1_2_2_20_1","volume-title":"An Introduction to Psychological Assessment & Psychometrics","author":"Coaley Keith","unstructured":"Keith Coaley . 2014. An Introduction to Psychological Assessment & Psychometrics . SAGE , Los Angeles, CA . Keith Coaley. 2014. An Introduction to Psychological Assessment & Psychometrics. SAGE, Los Angeles, CA."},{"key":"e_1_2_2_21_1","volume-title":"Phillips","author":"Cohen Ronald Jay","year":"1995","unstructured":"Ronald Jay Cohen , Mark E. Swerdlik , and Suzanne M . Phillips . 1995 . Psychological Testing and Assessment: An Introduction to Tests and Measurement. Mayfield Pub Co., California City, CA. Ronald Jay Cohen, Mark E. Swerdlik, and Suzanne M. Phillips. 1995. Psychological Testing and Assessment: An Introduction to Tests and Measurement. Mayfield Pub Co., California City, CA."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1023254226592"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jeconom.2014.06.008"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jeconom.2014.06.008"},{"key":"e_1_2_2_25_1","volume-title":"Nebraska Symposium on Motivation: Psychology and Aging","author":"Paul","unstructured":"Paul T. Costa Jr and Robert R. McCrae. 1992. Comes of age . In Nebraska Symposium on Motivation: Psychology and Aging . University of Nebraska Press, Lincoln, NE. Paul T. Costa Jr and Robert R. McCrae. 1992. Comes of age. In Nebraska Symposium on Motivation: Psychology and Aging. University of Nebraska Press, Lincoln, NE."},{"key":"e_1_2_2_26_1","first-page":"1","article-title":"Determining the number of factors to retain in EFA: Using the SPSS r-menu v2.0 to make more judicious estimations","volume":"18","author":"Rau Courtney Matthew Gordon","year":"2013","unstructured":"Matthew Gordon Rau Courtney . 2013 . Determining the number of factors to retain in EFA: Using the SPSS r-menu v2.0 to make more judicious estimations . Practic. Assess. Res. Eval. 18 , 8 (2013), 1 \u2013 14 . Matthew Gordon Rau Courtney. 2013. Determining the number of factors to retain in EFA: Using the SPSS r-menu v2.0 to make more judicious estimations. Practic. Assess. Res. Eval. 18, 8 (2013), 1\u201314.","journal-title":"Practic. Assess. Res. Eval."},{"key":"e_1_2_2_27_1","volume-title":"Research Design","author":"Creswell John W.","unstructured":"John W. Creswell and J. David Creswell . 2018. Research Design . SAGE Publications, Inc orporated, Thousand Oaks, CA . John W. Creswell and J. David Creswell. 2018. Research Design. SAGE Publications, Incorporated, Thousand Oaks, CA."},{"key":"e_1_2_2_28_1","volume-title":"Introduction to Classical and Modern Test Theory. Cengage Learning","author":"Crocker Linda","unstructured":"Linda Crocker . 2008. Introduction to Classical and Modern Test Theory. Cengage Learning , Mason, OH . Linda Crocker. 2008. Introduction to Classical and Modern Test Theory. Cengage Learning, Mason, OH."},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02310555"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2014.12.008"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.9.3.458"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2005.261"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.2307\/2988040"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1046\/j.1365-2648.2003.02579.x"},{"key":"e_1_2_2_36_1","volume-title":"Reise","author":"Embretson Susan E.","year":"2013","unstructured":"Susan E. Embretson and Steven P . Reise . 2013 . Item Response Theory. Psychology Press , Hove, East Sussex, UK. Susan E. Embretson and Steven P. Reise. 2013. Item Response Theory. Psychology Press, Hove, East Sussex, UK."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1037\/1082-989X.4.3.272"},{"key":"e_1_2_2_39_1","volume-title":"Contemporary Empirical Methods in Software Engineering","author":"Fagerholm Fabian","unstructured":"Fabian Fagerholm and Thomas Fritz . 2020. Biometric measurement in software engineering . In Contemporary Empirical Methods in Software Engineering , Michael Felderer and Guilherme Horta Travassos (Eds.). Springer International Publishing , Cham, Switzerland , 151\u2013172. DOI:https:\/\/doi.org\/10.1007\/978-3-030-32489-6_6 Fabian Fagerholm and Thomas Fritz. 2020. Biometric measurement in software engineering. In Contemporary Empirical Methods in Software Engineering, Michael Felderer and Guilherme Horta Travassos (Eds.). Springer International Publishing, Cham, Switzerland, 151\u2013172. DOI:https:\/\/doi.org\/10.1007\/978-3-030-32489-6_6"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/2813544.2813559"},{"key":"e_1_2_2_41_1","volume-title":"22nd International Conference on Software Engineering & Knowledge Engineering (SEKE'2010)","author":"Feldt Robert","year":"2010","unstructured":"Robert Feldt and Ana Magazinius . 2010 . Validity threats in empirical software engineering research\u2014An initial survey . In 22nd International Conference on Software Engineering & Knowledge Engineering (SEKE'2010) \u2026 Retrieved from http:\/\/www.cse.chalmers.se\/ feldt\/publications\/feldt_ 2010_validity_threats_in_ese_initial_survey.pdf. Robert Feldt and Ana Magazinius. 2010. Validity threats in empirical software engineering research\u2014An initial survey. In 22nd International Conference on Software Engineering & Knowledge Engineering (SEKE'2010)\u2026 Retrieved from http:\/\/www.cse.chalmers.se\/ feldt\/publications\/feldt_2010_validity_threats_in_ese_initial_survey.pdf."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1370114.1370127"},{"key":"e_1_2_2_43_1","volume-title":"Contemporary Empirical Methods in Software Engineering","author":"Fern\u00e1ndez Daniel M\u00e9ndez","unstructured":"Daniel M\u00e9ndez Fern\u00e1ndez , Daniel Graziotin , Stefan Wagner , and Heidi Seibold . 2020. Open science in software engineering . In Contemporary Empirical Methods in Software Engineering , Michael Felderer and Guilherme Horta Travassos (Eds.). Springer International Publishing , Cham, Switzerland , 479\u2013504. DOI:https:\/\/doi.org\/10.1007\/978-3-030-32489-6_17 Daniel M\u00e9ndez Fern\u00e1ndez, Daniel Graziotin, Stefan Wagner, and Heidi Seibold. 2020. Open science in software engineering. In Contemporary Empirical Methods in Software Engineering, Michael Felderer and Guilherme Horta Travassos (Eds.). Springer International Publishing, Cham, Switzerland, 479\u2013504. DOI:https:\/\/doi.org\/10.1007\/978-3-030-32489-6_17"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1037\/1082-989X.9.4.466"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2842201"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2935974"},{"key":"e_1_2_2_48_1","volume-title":"Encyclopedia of Behavioral Medicine, Marc D","author":"Ginty Annie T.","unstructured":"Annie T. Ginty . 2013. Psychometric properties . In Encyclopedia of Behavioral Medicine, Marc D . Gellman and J. Rick Turner (Eds.). Springer New York , NY , 1563\u20131564. DOI:https:\/\/doi.org\/10.1007\/978-1-4419-1005-9_480 Annie T. Ginty. 2013. Psychometric properties. In Encyclopedia of Behavioral Medicine, Marc D. Gellman and J. Rick Turner (Eds.). Springer New York, NY, 1563\u20131564. DOI:https:\/\/doi.org\/10.1007\/978-1-4419-1005-9_480"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0049294"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3084226.3084242"},{"key":"e_1_2_2_51_1","unstructured":"Daniel Graziotin Per Lenberg Robert Feldt and Stefan Wagner. 2021. Behavioral software engineering\u2014Example of psychometric evaluation with R. DOI:https:\/\/doi.org\/10.5281\/zenodo.3799603 Daniel Graziotin Per Lenberg Robert Feldt and Stefan Wagner. 2021. Behavioral software engineering\u2014Example of psychometric evaluation with R. DOI:https:\/\/doi.org\/10.5281\/zenodo.3799603"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.5555\/2819321.2819350"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1002\/smr.1673"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2804381.2804386"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0013338"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3239235.3267437"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897586.2897588"},{"key":"e_1_2_2_58_1","volume-title":"Psychometric Methods","author":"Guilford Joy Paul","unstructured":"Joy Paul Guilford . 1954. Psychometric Methods ( 1 st ed.). McGraw-Hill Book Co. , New York , London. Joy Paul Guilford. 1954. Psychometric Methods (1st ed.). McGraw-Hill Book Co., New York, London.","edition":"1"},{"key":"e_1_2_2_59_1","volume-title":"Psychometric Methods","author":"Guilford Joy Paul","unstructured":"Joy Paul Guilford . 1954. Psychometric Methods ( 2 nd ed.). McGraw-Hill Book Co. , New York , London. Joy Paul Guilford. 1954. Psychometric Methods (2nd ed.). McGraw-Hill Book Co., New York, London.","edition":"2"},{"key":"e_1_2_2_60_1","volume-title":"Hambleton and Jane Rodgers","author":"Ronald","year":"1995","unstructured":"Ronald K. Hambleton and Jane Rodgers . 1995 . Item Bias Review. Technical Report. ERIC Clearinghouse on Assessment and Evaluation, the Catholic University of America, Department of Education. Ronald K. Hambleton and Jane Rodgers. 1995. Item Bias Review. Technical Report. ERIC Clearinghouse on Assessment and Evaluation, the Catholic University of America, Department of Education."},{"key":"e_1_2_2_61_1","volume-title":"Fundamentals of Item Response Theory","author":"Hambleton Ronald K.","unstructured":"Ronald K. Hambleton , Hariharan Swaminathan , and H. Jane Rogers . 1991. Fundamentals of Item Response Theory . Sage Publications , Newbury Park, CA . Ronald K. Hambleton, Hariharan Swaminathan, and H. Jane Rogers. 1991. Fundamentals of Item Response Theory. Sage Publications, Newbury Park, CA."},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1002\/1520-6696(198004)16:2<107::AID-JHBS2300160202>3.0.CO;2-Y"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1080\/13645579.2015.1091640"},{"key":"e_1_2_2_64_1","volume-title":"Personality and the Fate of Organizations","author":"Hogan Robert","unstructured":"Robert Hogan . 2007. Personality and the Fate of Organizations . Erlbaum, Mahwah , NJ. Robert Hogan. 2007. Personality and the Fate of Organizations. Erlbaum, Mahwah, NJ."},{"key":"e_1_2_2_65_1","volume-title":"A rationale and test for the number of factors in factor analysis.Psychometrika 30","author":"Horn John L.","year":"1965","unstructured":"John L. Horn . 1965. A rationale and test for the number of factors in factor analysis.Psychometrika 30 ( 1965 ), 179\u2013185. John L. Horn. 1965. A rationale and test for the number of factors in factor analysis.Psychometrika 30 (1965), 179\u2013185."},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/1414004.1414033"},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1177\/0013164409355692"},{"key":"e_1_2_2_68_1","volume-title":"Jones and David Thissen","author":"Lyle","year":"2006","unstructured":"Lyle V. Jones and David Thissen . 2006 . A history and overview of psychometrics. In Handbook of Statistics: Psychometrics. Elsevier , Amsterdam Boston, 1\u201327. DOI:https:\/\/doi.org\/10.1016\/s0169-7161(06)26001-2 Lyle V. Jones and David Thissen. 2006. A history and overview of psychometrics. In Handbook of Statistics: Psychometrics. Elsevier, Amsterdam Boston, 1\u201327. DOI:https:\/\/doi.org\/10.1016\/s0169-7161(06)26001-2"},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02289233"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000116"},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.2307\/2981421"},{"key":"e_1_2_2_72_1","volume-title":"Structural Equation Modeling: Foundations and Extensions","author":"Kaplan David","unstructured":"David Kaplan . 2008. Structural Equation Modeling: Foundations and Extensions . Vol. 10 . SAGE , Los Angeles, CA . David Kaplan. 2008. Structural Equation Modeling: Foundations and Extensions. Vol. 10. SAGE, Los Angeles, CA."},{"key":"e_1_2_2_74_1","volume-title":"Pfleeger","author":"Kitchenham Barbara A.","year":"2008","unstructured":"Barbara A. Kitchenham and Shari L . Pfleeger . 2008 . Personal opinion surveys. In Guide to Advanced Empirical Software Engineering, Forrest Shull, Janice Singer, and Dag I. K. Sj\u00f8berg (Eds.). Springer London , 63\u201392. DOI:https:\/\/doi.org\/10.1007\/978-1-84800-044-5_3 Barbara A. Kitchenham and Shari L. Pfleeger. 2008. Personal opinion surveys. In Guide to Advanced Empirical Software Engineering, Forrest Shull, Janice Singer, and Dag I. K. Sj\u00f8berg (Eds.). Springer London, 63\u201392. DOI:https:\/\/doi.org\/10.1007\/978-1-84800-044-5_3"},{"key":"e_1_2_2_75_1","volume-title":"A Handbook of Test Construction (Psychology Revivals): Introduction to Psychometric Design","author":"Kline Paul","unstructured":"Paul Kline . 2015. A Handbook of Test Construction (Psychology Revivals): Introduction to Psychometric Design . Routledge , London New York. Paul Kline. 2015. A Handbook of Test Construction (Psychology Revivals): Introduction to Psychometric Design. Routledge, London New York."},{"key":"e_1_2_2_76_1","doi-asserted-by":"publisher","DOI":"10.1037\/1089-2680.8.1.3"},{"key":"e_1_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02293896"},{"key":"e_1_2_2_79_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.04.084"},{"key":"e_1_2_2_80_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.04.084"},{"key":"e_1_2_2_81_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-016-9482-0"},{"key":"e_1_2_2_82_1","volume-title":"Mislevy","author":"Levy Roy","year":"2016","unstructured":"Roy Levy and Robert J . Mislevy . 2016 . Bayesian Psychometric Modeling. CRC Press , Boca Raton, FL. Roy Levy and Robert J. Mislevy. 2016. Bayesian Psychometric Modeling. CRC Press, Boca Raton, FL."},{"key":"e_1_2_2_83_1","volume-title":"31st Workshop of the Psychology of Programming Interest Group (PPIG'20)","author":"Lewis Clayton","year":"2020","unstructured":"Clayton Lewis . 2020 . On personality testing and software engineering . In 31st Workshop of the Psychology of Programming Interest Group (PPIG'20) . PPIG, 39\u201341. Clayton Lewis. 2020. On personality testing and software engineering. In 31st Workshop of the Psychology of Programming Interest Group (PPIG'20). PPIG, 39\u201341."},{"key":"e_1_2_2_84_1","first-page":"1","article-title":"A technique for the measurement of attitudes","volume":"22","author":"Likert Rensis","year":"1932","unstructured":"Rensis Likert . 1932 . A technique for the measurement of attitudes . Arch. Psychol. 22 , 40 (1932), 1 \u2013 55 . Rensis Likert. 1932. A technique for the measurement of attitudes.Arch. Psychol. 22, 40 (1932), 1\u201355.","journal-title":"Arch. Psychol."},{"key":"e_1_2_2_85_1","first-page":"635","article-title":"Objective tests as instruments of psychological theory","volume":"3","author":"Loevinger Jane","year":"1957","unstructured":"Jane Loevinger . 1957 . Objective tests as instruments of psychological theory . Psychol. Rep. 3 (1957), 635 \u2013 694 . DOI:https:\/\/doi.org\/10.2466\/pr0.1957.3.3.635 Jane Loevinger. 1957. Objective tests as instruments of psychological theory. Psychol. Rep. 3 (1957), 635\u2013694. DOI:https:\/\/doi.org\/10.2466\/pr0.1957.3.3.635","journal-title":"Psychol. Rep."},{"key":"e_1_2_2_86_1","doi-asserted-by":"publisher","DOI":"10.1080\/00273171.2016.1168279"},{"key":"e_1_2_2_87_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.psych.51.1.201"},{"key":"e_1_2_2_88_1","doi-asserted-by":"publisher","DOI":"10.1037\/1082-989X.4.1.84"},{"key":"e_1_2_2_89_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1988.tb00288.x"},{"key":"e_1_2_2_90_1","doi-asserted-by":"publisher","DOI":"10.1145\/1188913.1188919"},{"key":"e_1_2_2_91_1","doi-asserted-by":"publisher","DOI":"10.1177\/0959354300105004"},{"key":"e_1_2_2_92_1","volume-title":"Cognitive Interviewing Methodology","author":"Miller Kristen","unstructured":"Kristen Miller , Valerie Chepp , Stephanie Willson , and Jose-Luis Padilla . 2014. Cognitive Interviewing Methodology . John Wiley & Sons, Hoboken , NJ. Kristen Miller, Valerie Chepp, Stephanie Willson, and Jose-Luis Padilla. 2014. Cognitive Interviewing Methodology. John Wiley & Sons, Hoboken, NJ."},{"key":"e_1_2_2_93_1","doi-asserted-by":"publisher","DOI":"10.1145\/2961111.2962619"},{"key":"e_1_2_2_94_1","doi-asserted-by":"publisher","DOI":"10.1080\/02701367.1980.10609275"},{"key":"e_1_2_2_95_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0026802"},{"key":"e_1_2_2_96_1","volume-title":"Research in medical education: Three decades of progress. BMJ 324, 7353","author":"Norman Geoff","year":"2002","unstructured":"Geoff Norman . 2002. Research in medical education: Three decades of progress. BMJ 324, 7353 ( 2002 ), 1560\u20131562. DOI:https:\/\/doi.org\/10.1136\/bmj.324.7353.1560 Geoff Norman. 2002. Research in medical education: Three decades of progress. BMJ 324, 7353 (2002), 1560\u20131562. DOI:https:\/\/doi.org\/10.1136\/bmj.324.7353.1560"},{"key":"e_1_2_2_97_1","article-title":"Evaluating the use of exploratory factor analysis in developmental disability psychological research","volume":"40","author":"Norris Megan","year":"2010","unstructured":"Megan Norris and Luc Lecavalier . 2010 . Evaluating the use of exploratory factor analysis in developmental disability psychological research . J. Autis. Devel. Disord. 40 , 1 (01 Jan. 2010), 8\u201320. DOI:https:\/\/doi.org\/10.1007\/s10803-009-0816-2 Megan Norris and Luc Lecavalier. 2010. Evaluating the use of exploratory factor analysis in developmental disability psychological research. J. Autis. Devel. Disord. 40, 1 (01 Jan. 2010), 8\u201320. DOI:https:\/\/doi.org\/10.1007\/s10803-009-0816-2","journal-title":"J. Autis. Devel. Disord."},{"key":"e_1_2_2_98_1","volume-title":"Psychometric Theory","author":"Nunnally Jum","unstructured":"Jum Nunnally . 1994. Psychometric Theory . McGraw-Hill , New York . Jum Nunnally. 1994. Psychometric Theory. McGraw-Hill, New York."},{"key":"e_1_2_2_99_1","volume-title":"Clinical Diagnosis of Mental Disorders","author":"Nunnally Jum C.","unstructured":"Jum C. Nunnally . 1978. An overview of psychological measurement . In Clinical Diagnosis of Mental Disorders . Springer US , 97\u2013146. DOI:https:\/\/doi.org\/10.1007\/978-1-4684-2490-4_4 Jum C. Nunnally. 1978. An overview of psychological measurement. In Clinical Diagnosis of Mental Disorders. Springer US, 97\u2013146. DOI:https:\/\/doi.org\/10.1007\/978-1-4684-2490-4_4"},{"key":"e_1_2_2_100_1","volume-title":"Questionnaire Design, Interviewing and Attitude Measurement","author":"Oppenheim A. N.","unstructured":"A. N. Oppenheim . 1992. Questionnaire Design, Interviewing and Attitude Measurement . Pinter Pub Ltd, London , UK. A. N. Oppenheim. 1992. Questionnaire Design, Interviewing and Attitude Measurement. Pinter Pub Ltd, London, UK."},{"key":"e_1_2_2_101_1","doi-asserted-by":"publisher","DOI":"10.5555\/3105556.3105562"},{"key":"e_1_2_2_102_1","doi-asserted-by":"publisher","DOI":"10.1080\/14786440109462720"},{"key":"e_1_2_2_103_1","first-page":"1","article-title":"Differential item functioning and item bias: Critical considerations in test fairness","volume":"6","author":"Perrone Michael","year":"2006","unstructured":"Michael Perrone . 2006 . Differential item functioning and item bias: Critical considerations in test fairness . Teach. Coll., Columbia Univ. Work. Pap. TESOL Appl. Ling. 6 (2006), 1 \u2013 3 . Michael Perrone. 2006. Differential item functioning and item bias: Critical considerations in test fairness. Teach. Coll., Columbia Univ. Work. Pap. TESOL Appl. Ling. 6 (2006), 1\u20133.","journal-title":"Teach. Coll., Columbia Univ. Work. Pap. TESOL Appl. Ling."},{"key":"e_1_2_2_104_1","doi-asserted-by":"publisher","DOI":"10.1109\/IWSM-Mensura.2013.22"},{"key":"e_1_2_2_105_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jval.2014.10.005"},{"key":"e_1_2_2_106_1","first-page":"48","article-title":"Measuring the MBTI\u2026 and coming up short","volume":"54","author":"Pittenger David J.","year":"1993","unstructured":"David J. Pittenger . 1993 . Measuring the MBTI\u2026 and coming up short . J. Career Plan. Employ. 54 , 1 (1993), 48 \u2013 52 . David J. Pittenger. 1993. Measuring the MBTI\u2026 and coming up short. J. Career Plan. Employ. 54, 1 (1993), 48\u201352.","journal-title":"J. Career Plan. Employ."},{"key":"e_1_2_2_107_1","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1080\/00031305.1994.10476030","article-title":"The three sigma rule","volume":"48","author":"Pukelsheim Friedrich","year":"1994","unstructured":"Friedrich Pukelsheim . 1994 . The three sigma rule . Amer. Statist. 48 , 2 (1994), 88 \u2013 91 . Friedrich Pukelsheim. 1994. The three sigma rule. Amer. Statist. 48, 2 (1994), 88\u201391.","journal-title":"Amer. Statist."},{"key":"e_1_2_2_108_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09875-y"},{"key":"e_1_2_2_109_1","doi-asserted-by":"publisher","DOI":"10.1145\/3210459.3210461"},{"key":"e_1_2_2_110_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2702.2006.01573.x"},{"key":"e_1_2_2_111_1","volume-title":"Can we build a better mousetrap? Improving the measures of practice performance in the field practicum. J. Soc. Work Educ.ation 43, 2","author":"Regehr Glenn","year":"2007","unstructured":"Glenn Regehr , Marion Bogo , Cheryl Regehr , and Roxanne Power . 2007. Can we build a better mousetrap? Improving the measures of practice performance in the field practicum. J. Soc. Work Educ.ation 43, 2 ( 2007 ), 327\u2013344. DOI:https:\/\/doi.org\/10.5175\/jswe.2007.200600607 Glenn Regehr, Marion Bogo, Cheryl Regehr, and Roxanne Power. 2007. Can we build a better mousetrap? Improving the measures of practice performance in the field practicum. J. Soc. Work Educ.ation 43, 2 (2007), 327\u2013344. DOI:https:\/\/doi.org\/10.5175\/jswe.2007.200600607"},{"key":"e_1_2_2_112_1","unstructured":"William Revelle. 2009. An Introduction to Psychometric Theory with Applications in R. Retrieved from personality-project.org. William Revelle. 2009. An Introduction to Psychometric Theory with Applications in R. Retrieved from personality-project.org."},{"key":"e_1_2_2_115_1","volume-title":"psych: Procedures for Psychological, Psychometric, and Personality Research","author":"Revelle William","unstructured":"William Revelle . 2019. psych: Procedures for Psychological, Psychometric, and Personality Research . Northwestern University, Evanston , IL. Retrieved from https:\/\/cran.r-project.org\/package=psych. R package version 1.9.12. William Revelle. 2019. psych: Procedures for Psychological, Psychometric, and Personality Research. Northwestern University, Evanston, IL. Retrieved from https:\/\/cran.r-project.org\/package=psych. R package version 1.9.12."},{"key":"e_1_2_2_116_1","volume-title":"How To: Use the Psych Package for Factor Analysis and Data Reduction. Technical Report. The Comprehensive R Archive Network.","author":"Revelle William","year":"2020","unstructured":"William Revelle . 2020 . How To: Use the Psych Package for Factor Analysis and Data Reduction. Technical Report. The Comprehensive R Archive Network. William Revelle. 2020. How To: Use the Psych Package for Factor Analysis and Data Reduction. Technical Report. The Comprehensive R Archive Network."},{"key":"e_1_2_2_117_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15327906mbr1404_2"},{"key":"e_1_2_2_118_1","doi-asserted-by":"publisher","DOI":"10.1186\/2229-0443-3-9"},{"key":"e_1_2_2_119_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2002.1003450"},{"key":"e_1_2_2_120_1","doi-asserted-by":"publisher","DOI":"10.1177\/014616702237645"},{"key":"e_1_2_2_121_1","volume-title":"Modern Psychometrics : the Science of Psychological Assessment","author":"Rust John","unstructured":"John Rust . 2009. Modern Psychometrics : the Science of Psychological Assessment . Routledge , Hove, East Sussex New York. John Rust. 2009. Modern Psychometrics : the Science of Psychological Assessment. Routledge, Hove, East Sussex New York."},{"key":"e_1_2_2_122_1","doi-asserted-by":"publisher","DOI":"10.1037\/met0000275"},{"key":"e_1_2_2_123_1","doi-asserted-by":"publisher","DOI":"10.1037\/0003-066X.47.10.1173"},{"key":"e_1_2_2_124_1","first-page":"719","article-title":"Psychometrics and its discontents: An historical perspective on the discourse of the measurement tradition.Adv. Health Sci","volume":"21","author":"Schoenherr J. R.","year":"2016","unstructured":"J. R. Schoenherr and S. J. Hamstra . 2016 . Psychometrics and its discontents: An historical perspective on the discourse of the measurement tradition.Adv. Health Sci . Educ. Theor. Pract. 21 , 3 (2016), 719 \u2013 729 . DOI:https:\/\/doi.org\/10.1007\/s10459-015-9623-z J. R. Schoenherr and S. J. Hamstra. 2016. Psychometrics and its discontents: An historical perspective on the discourse of the measurement tradition.Adv. Health Sci. Educ. Theor. Pract. 21, 3 (2016), 719\u2013729. DOI:https:\/\/doi.org\/10.1007\/s10459-015-9623-z","journal-title":"Educ. Theor. Pract."},{"key":"e_1_2_2_125_1","doi-asserted-by":"publisher","DOI":"10.1177\/109821400102200202"},{"key":"e_1_2_2_126_1","doi-asserted-by":"publisher","DOI":"10.1186\/1748-5908-9-7"},{"key":"e_1_2_2_127_1","doi-asserted-by":"publisher","DOI":"10.5555\/2818754.2818759"},{"key":"e_1_2_2_128_1","doi-asserted-by":"publisher","DOI":"10.1086\/268650"},{"key":"e_1_2_2_129_1","volume-title":"Measures of Positive Psychology","author":"Singh Kamlesh","unstructured":"Kamlesh Singh , Mohita Junnarkar , Jasleen Kaur , Kamlesh Singh , Mohita Junnarkar , and Jasleen Kaur . 2016. Norms for test construction . In Measures of Positive Psychology . Springer India , New Delhi , 17\u201334. Kamlesh Singh, Mohita Junnarkar, Jasleen Kaur, Kamlesh Singh, Mohita Junnarkar, and Jasleen Kaur. 2016. Norms for test construction. In Measures of Positive Psychology. Springer India, New Delhi, 17\u201334."},{"key":"e_1_2_2_130_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1748-8583.2003.tb00091.x"},{"key":"e_1_2_2_131_1","volume-title":"Ullman","author":"Tabachnick Barbara G.","year":"2007","unstructured":"Barbara G. Tabachnick , Linda S. Fidell , and Jodie B . Ullman . 2007 . Using Multivariate Statistics. Vol. 5 . Pearson Boston, MA. Barbara G. Tabachnick, Linda S. Fidell, and Jodie B. Ullman. 2007. Using Multivariate Statistics. Vol. 5. Pearson Boston, MA."},{"key":"e_1_2_2_132_1","volume-title":"Goldsmith","author":"Thabane Lehana","year":"2010","unstructured":"Lehana Thabane , Jinhui Ma , Rong Chu , Ji Cheng , Afisi Ismaila , Lorena P. Rios , Reid Robson , Marroon Thabane , Lora Giangregorio , and Charles H . Goldsmith . 2010 . A tutorial on pilot studies: The what, why. and how. BMC Med. Res. Methodol . 10, 1 (2010). DOI:https:\/\/doi.org\/10.1186\/1471-2288-10-1 Lehana Thabane, Jinhui Ma, Rong Chu, Ji Cheng, Afisi Ismaila, Lorena P. Rios, Reid Robson, Marroon Thabane, Lora Giangregorio, and Charles H. Goldsmith. 2010. A tutorial on pilot studies: The what, why. and how. BMC Med. Res. Methodol. 10, 1 (2010). DOI:https:\/\/doi.org\/10.1186\/1471-2288-10-1"},{"key":"e_1_2_2_133_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.85.2201.227"},{"key":"e_1_2_2_134_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0167.34.4.414"},{"key":"e_1_2_2_135_1","first-page":"4","article-title":"Classical test theory in historical perspective","volume":"16","author":"Traub Ross E.","year":"2005","unstructured":"Ross E. Traub . 2005 . Classical test theory in historical perspective . Educ. Measur.: Iss. Pract. 16 , 4 (Oct. 2005), 8\u201314. DOI:https:\/\/doi.org\/10.1111\/j.1745-3992.1997.tb00603.x Ross E. Traub. 2005. Classical test theory in historical perspective. Educ. Measur.: Iss. Pract. 16, 4 (Oct. 2005), 8\u201314. DOI:https:\/\/doi.org\/10.1111\/j.1745-3992.1997.tb00603.x","journal-title":"Educ. Measur.: Iss. Pract."},{"key":"e_1_2_2_136_1","doi-asserted-by":"publisher","DOI":"10.17705\/1jais.00426"},{"key":"e_1_2_2_137_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2018.02599"},{"key":"e_1_2_2_138_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.paid.2020.110205"},{"key":"e_1_2_2_139_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13423-017-1343-3"},{"key":"e_1_2_2_140_1","volume-title":"Contemporary Empirical Methods in Software Engineering","author":"Wagner Stefan","unstructured":"Stefan Wagner , Daniel Mendez , Michael Felderer , Daniel Graziotin , and Marcos Kalinowski . 2020. Challenges in survey research . In Contemporary Empirical Methods in Software Engineering , Michael Felderer and Guilherme Horta Travassos (Eds.). Springer International Publishing , Cham , 93\u2013125. DOI:https:\/\/doi.org\/10.1007\/978-3-030-32489-6_4. Available https:\/\/arxiv.org\/abs\/1908.05899 Stefan Wagner, Daniel Mendez, Michael Felderer, Daniel Graziotin, and Marcos Kalinowski. 2020. Challenges in survey research. In Contemporary Empirical Methods in Software Engineering, Michael Felderer and Guilherme Horta Travassos (Eds.). Springer International Publishing, Cham, 93\u2013125. DOI:https:\/\/doi.org\/10.1007\/978-3-030-32489-6_4. Available https:\/\/arxiv.org\/abs\/1908.05899"},{"key":"e_1_2_2_141_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409762"},{"key":"e_1_2_2_142_1","first-page":"1","article-title":"Moving to a world beyond \u201cp\u00a0 <\u00a0 0.05","volume":"1","author":"Wasserstein Ronald L.","year":"2019","unstructured":"Ronald L. Wasserstein , Allen L. Schirm , and Nicole A. Lazar . 2019 . Moving to a world beyond \u201cp\u00a0 <\u00a0 0.05 .\u201d The American Statistician 73, sup 1 (2019), 1 \u2013 19 . DOI:https:\/\/doi.org\/10.1080\/00031305.2019.1583913 Ronald L. Wasserstein, Allen L. Schirm, and Nicole A. Lazar. 2019. Moving to a world beyond \u201cp\u00a0 <\u00a0 0.05.\u201dThe American Statistician 73, sup 1 (2019), 1\u201319. DOI:https:\/\/doi.org\/10.1080\/00031305.2019.1583913","journal-title":"The American Statistician 73, sup"},{"key":"e_1_2_2_143_1","volume-title":"The Psychology of Computer Programming","author":"Weinberg Gerald M.","unstructured":"Gerald M. Weinberg . 1971. The Psychology of Computer Programming . Dorset House Publishing , New York . Gerald M. Weinberg. 1971. The Psychology of Computer Programming. Dorset House Publishing, New York."},{"key":"e_1_2_2_144_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1053-4822(02)00045-1"},{"key":"e_1_2_2_145_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15327906mbr2803_1"},{"key":"e_1_2_2_146_1","volume-title":"Methods for Testing and Evaluating Survey Questionnaires: Wiley Series in Survey Methodology","author":"Willis Gordon B.","unstructured":"Gordon B. Willis . 2004. Cognitive interviewing revisited: A useful technique, in theory . In Methods for Testing and Evaluating Survey Questionnaires: Wiley Series in Survey Methodology . John Wiley & Sons, Inc. , Hoboken, NJ , 23\u201343. DOI:https:\/\/doi.org\/10.1002\/0471654728.ch2 Gordon B. Willis. 2004. Cognitive interviewing revisited: A useful technique, in theory. In Methods for Testing and Evaluating Survey Questionnaires: Wiley Series in Survey Methodology. John Wiley & Sons, Inc., Hoboken, NJ, 23\u201343. DOI:https:\/\/doi.org\/10.1002\/0471654728.ch2"},{"key":"e_1_2_2_147_1","doi-asserted-by":"publisher","DOI":"10.5555\/2349018"},{"key":"e_1_2_2_148_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0038423"},{"key":"e_1_2_2_149_1","first-page":"508","article-title":"Two quantitative approaches for estimating content validity.West","volume":"25","author":"Wynd C. A.","year":"2003","unstructured":"C. A. Wynd , B. Schmidt , and M. A. Schaefer . 2003 . Two quantitative approaches for estimating content validity.West . J. Nurs. Res. 25 , 5 (2003), 508 \u2013 518 . DOI:https:\/\/doi.org\/10.1177\/0193945903252998 C. A. Wynd, B. Schmidt, and M. A. Schaefer. 2003. Two quantitative approaches for estimating content validity.West. J. Nurs. Res. 25, 5 (2003), 508\u2013518. DOI:https:\/\/doi.org\/10.1177\/0193945903252998","journal-title":"J. Nurs. Res."},{"key":"e_1_2_2_150_1","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.173"},{"key":"e_1_2_2_151_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00055"},{"key":"e_1_2_2_152_1","doi-asserted-by":"publisher","DOI":"10.20982\/tqmp.09.2.p079"},{"key":"e_1_2_2_153_1","doi-asserted-by":"publisher","DOI":"10.1080\/15434300701375832"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3469888","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3469888","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:28:16Z","timestamp":1750195696000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3469888"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,28]]},"references-count":146,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1,31]]}},"alternative-id":["10.1145\/3469888"],"URL":"https:\/\/doi.org\/10.1145\/3469888","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,28]]},"assertion":[{"value":"2020-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-09-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}