{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T01:11:57Z","timestamp":1787793117659,"version":"build-2784847793"},"reference-count":47,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2021,4,2]],"date-time":"2021-04-02T00:00:00Z","timestamp":1617321600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Assessment"],"published-print":{"date-parts":[[2022,1]]},"abstract":"<jats:p>As part of a broader project to create a comprehensive self-report measure for the Hierarchical Taxonomy of Psychopathology consortium, we developed preliminary scales to assess internalizing symptoms. The item pool was created in four steps: (a) clarifying the range of content to be assessed, (b) identifying target constructs to guide item writing, (c) developing formal definitions for each construct, and (d) writing multiple items for each construct. This yielded 430 items assessing 57 target constructs. Responses from a heterogeneous scale development sample ( N = 1,870) were subjected to item-level factor analyses based on polychoric correlations. This resulted in 39 scales representing a total of 213 items. The psychometric properties of these scales replicated well across the development sample and an independent validation sample ( N = 496 adults). Internal consistency analyses established that most scales assess relatively narrow forms of psychopathology. Structural analyses demonstrated the presence of a strong general factor. Additional analyses of the 35 nonsexual dysfunction scales revealed a replicable four-factor structure with dimensions we labeled Distress, Fear, Body Dysmorphia, and Mania. A final set of analyses established that the internalizing scales varied widely\u2014and consistently\u2014in the strength of their associations with neuroticism and extraversion.<\/jats:p>","DOI":"10.1177\/10731911211003976","type":"journal-article","created":{"date-parts":[[2021,4,2]],"date-time":"2021-04-02T03:47:29Z","timestamp":1617335249000},"page":"17-33","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":47,"title":["The Development of Preliminary HiTOP Internalizing Spectrum Scales"],"prefix":"10.1177","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9605-0576","authenticated-orcid":false,"given":"David","family":"Watson","sequence":"first","affiliation":[{"name":"University of Notre Dame, Notre Dame, IN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Miriam K.","family":"Forbes","sequence":"additional","affiliation":[{"name":"Macquarie University, Sydney, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3980-8299","authenticated-orcid":false,"given":"Holly F.","family":"Levin-Aspenson","sequence":"additional","affiliation":[{"name":"University of Notre Dame, Notre Dame, IN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8482-6269","authenticated-orcid":false,"given":"Camilo J.","family":"Ruggero","sequence":"additional","affiliation":[{"name":"University of North Texas, Denton, TX, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuliya","family":"Kotelnikova","sequence":"additional","affiliation":[{"name":"University of New Orleans, New Orleans, LA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shereen","family":"Khoo","sequence":"additional","affiliation":[{"name":"University of Notre Dame, Notre Dame, IN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"R. Michael","family":"Bagby","sequence":"additional","affiliation":[{"name":"University of Toronto, Toronto, Ontario, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8452-364X","authenticated-orcid":false,"given":"Matthew","family":"Sunderland","sequence":"additional","affiliation":[{"name":"University of Sydney, Randwick, New South Wales, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Praveetha","family":"Patalay","sequence":"additional","affiliation":[{"name":"University College London, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Roman","family":"Kotov","sequence":"additional","affiliation":[{"name":"Stony Brook University, Stony Brook, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2021,4,2]]},"reference":[{"key":"bibr1-10731911211003976","doi-asserted-by":"crossref","unstructured":"Achenbach T. M. (1966). The classification of children\u2019s psychiatric symptoms: A factor-analytic study. Psychological Monographs: General and Applied, 80(7), 1-37. https:\/\/doi.org\/10.1037\/h0093906","DOI":"10.1037\/h0093906"},{"key":"bibr2-10731911211003976","doi-asserted-by":"publisher","DOI":"10.1176\/appi.books.9780890425596"},{"key":"bibr3-10731911211003976","doi-asserted-by":"crossref","unstructured":"Basson R. (2000). The female sexual response: A different model. Journal of Sex & Marital Therapy, 26(1), 51-65. https:\/\/doi.org\/10.1080\/009262300278641","DOI":"10.1080\/009262300278641"},{"key":"bibr4-10731911211003976","doi-asserted-by":"crossref","unstructured":"Clark L. A., Watson D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309-319. https:\/\/doi.org\/10.1037\/1040-3590.7.3.309","DOI":"10.1037\/\/1040-3590.7.3.309"},{"key":"bibr5-10731911211003976","doi-asserted-by":"crossref","unstructured":"Clark L. A., Watson D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412-1427. https:\/\/doi.org\/10.1037\/pas0000626","DOI":"10.1037\/pas0000626"},{"key":"bibr6-10731911211003976","doi-asserted-by":"crossref","unstructured":"Cutshall C., Watson D. (2004). The Phobic Stimuli Response Scales: A new self-report measure of fear. Behaviour Research and Therapy, 42(10), 1193-1201. https:\/\/doi.org\/10.1016\/j.brat.2003.08.003","DOI":"10.1016\/j.brat.2003.08.003"},{"key":"bibr7-10731911211003976","doi-asserted-by":"crossref","unstructured":"Dornbach-Bender A., Ruggero C. J., Waszczuk M. A., Gamez W., Watson D., Kotov R. (2017). Mapping emotional disorders at the finest level: Convergent validity and joint structure based on alternative measures. Comprehensive Psychiatry, 79(November), 31-39. https:\/\/doi.org\/10.1016\/j.comppsych.2017.06.011","DOI":"10.1016\/j.comppsych.2017.06.011"},{"key":"bibr8-10731911211003976","doi-asserted-by":"crossref","unstructured":"Everett J. E. (1983). Factor comparability as a means of determining the number of factors and their rotation. Multivariate Behavioral Research, 18(2), 197-218. https:\/\/doi.org\/10.1207\/s15327906mbr1802_5","DOI":"10.1207\/s15327906mbr1802_5"},{"key":"bibr9-10731911211003976","doi-asserted-by":"crossref","unstructured":"Everett J. E., Entrekin L. V. (1980). Factor comparability and the advantages of multiple group factor analysis. Multivariate Behavioral Research, 15(2), 165-180. https:\/\/doi.org\/10.1207\/s15327906mbr1502_4","DOI":"10.1207\/s15327906mbr1502_4"},{"key":"bibr10-10731911211003976","doi-asserted-by":"crossref","unstructured":"Finn S. E. (1986). Structural stability of the MMPI in adult males. Journal of Consulting and Clinical Psychology, 54(5), 703-707. https:\/\/doi.org\/10.1037\/0022-006X.54.5.703","DOI":"10.1037\/\/0022-006X.54.5.703"},{"key":"bibr11-10731911211003976","doi-asserted-by":"crossref","unstructured":"Foa E. B., Huppert J. D., Leiberg S., Langner R., Kichic R., Hajcak G., Salkovskis P. M. (2002). The Obsessive-Compulsive Inventory: Development and validation of a short version. Psychological Assessment, 14(4), 485-496. https:\/\/doi.org\/10.1037\/1040-3590.14.4.485","DOI":"10.1037\/1040-3590.14.4.485"},{"key":"bibr12-10731911211003976","doi-asserted-by":"crossref","unstructured":"Forbes M. K., Baillie A. J., Schniering C. A. (2016a). A structural equation modeling analysis of the relationships between depression, anxiety, and sexual problems over time. Journal of Sex Research, 53(8), 942-954. https:\/\/doi.org\/10.1080\/00224499.2015.1063576","DOI":"10.1080\/00224499.2015.1063576"},{"key":"bibr13-10731911211003976","doi-asserted-by":"crossref","unstructured":"Forbes M. K., Baillie A. J., Schniering C. A. (2016b). Should sexual problems be included in the internalising spectrum? A comparison of dimensional and categorical models. Journal of Sex and Marital Therapy, 42(1), 70-90. https:\/\/doi.org\/10.1080\/0092623X.2014.996928","DOI":"10.1080\/0092623X.2014.996928"},{"key":"bibr14-10731911211003976","doi-asserted-by":"crossref","unstructured":"Forbush K. T., South S. C., Krueger R. F., Iacono W. G., Clark L. A., Keel P. K, Legrand L. N., Watson D. (2010). Locating eating pathology within an empirical diagnostic taxonomy: Evidence from a community-based sample. Journal of Abnormal Psychology, 119(2), 282-292. https:\/\/doi.org\/10.1037\/a0019189","DOI":"10.1037\/a0019189"},{"key":"bibr15-10731911211003976","doi-asserted-by":"crossref","unstructured":"Forbush K. T., Watson D. (2013). The structure of common and uncommon mental disorders. Psychological Medicine, 43(1), 97-108. https:\/\/doi.org\/10.1017\/S0033291712001092","DOI":"10.1017\/S0033291712001092"},{"key":"bibr16-10731911211003976","doi-asserted-by":"crossref","unstructured":"Goldberg D. P., Krueger R. F., Andrews G., Hobbs M. J. (2009). Emotional disorders: Cluster 4 of the proposed meta-structure for\n                      DSM-V\n                      and\n                      ICD-11\n                      . Psychological Medicine, 39(12), 2043-2059. https:\/\/doi.org\/10.1017\/S0033291709990298","DOI":"10.1017\/S0033291709990298"},{"key":"bibr17-10731911211003976","doi-asserted-by":"crossref","unstructured":"Gootzeit J., Markon K., Watson D. (2015). Measuring dimensions of PTSD: The Iowa Traumatic Response Inventory. Assessment, 22(2), 152-166. https:\/\/doi.org\/10.1177\/1073191114540747","DOI":"10.1177\/1073191114540747"},{"key":"bibr18-10731911211003976","volume-title":"Factor analysis","author":"Gorsuch R. L.","year":"1983","edition":"2"},{"key":"bibr19-10731911211003976","first-page":"55","volume-title":"1982 Nebraska Symposium on Motivation","author":"Hogan R. T.","year":"1983"},{"key":"bibr20-10731911211003976","doi-asserted-by":"crossref","unstructured":"Horn J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179-185. https:\/\/doi.org\/10.1007\/BF02289447","DOI":"10.1007\/BF02289447"},{"key":"bibr21-10731911211003976","doi-asserted-by":"crossref","unstructured":"Johnson R. E., Rosen C. C., Djurdjevic E. (2011). Assessing the impact of common method variance on higher order multidimensional constructs. Journal of Applied Psychology, 96(4), 744-761. https:\/\/doi.org\/10.1037\/a0021504","DOI":"10.1037\/a0021504"},{"key":"bibr22-10731911211003976","doi-asserted-by":"crossref","unstructured":"Koffel E. (2011). Further validation of the Iowa Sleep Disturbances Inventory. Psychological Assessment, 23(3), 587-598. https:\/\/doi.org\/10.1037\/a0022818","DOI":"10.1037\/a0022818"},{"key":"bibr23-10731911211003976","doi-asserted-by":"crossref","unstructured":"Koffel E., Khawaja I. S., Germain A. (2016). Sleep disturbances in posttraumatic stress disorder: Updated review and implications for treatment. Psychiatric Annals, 46(3), 173-176. https:\/\/doi.org\/10.3928\/00485713-20160125-01","DOI":"10.3928\/00485713-20160125-01"},{"key":"bibr24-10731911211003976","doi-asserted-by":"crossref","unstructured":"Koffel E., Krebs E. E., Arbisi P. A., Erbes C. R., Polusny M. A. (2015). The unhappy triad: Pain, sleep complaints, and internalizing symptoms. Clinical Psychological Science, 4(1), 96-106. https:\/\/doi.org\/10.1177\/2167702615579342","DOI":"10.1177\/2167702615579342"},{"key":"bibr25-10731911211003976","doi-asserted-by":"crossref","unstructured":"Koffel E., Watson D. (2009). The two-factor structure of sleep complaints and its relation to depression and anxiety. Journal of Abnormal Psychology, 118(1), 183-194. https:\/\/doi.org\/10.1037\/a0013945","DOI":"10.1037\/a0013945"},{"key":"bibr26-10731911211003976","doi-asserted-by":"crossref","unstructured":"Kotov R., Gamez W., Schmidt F. L., Watson D. (2010). Linking \u201cbig\u201d personality traits to anxiety, depressive, and substance use disorders: A meta-analysis. Psychological Bulletin, 136(5), 768-821. https:\/\/doi.org\/10.1037\/a0020327","DOI":"10.1037\/a0020327"},{"key":"bibr27-10731911211003976","doi-asserted-by":"crossref","unstructured":"Kotov R., Krueger R. F., Watson D., Achenbach T. M., Althoff R. R., Bagby R. M., Brown T. A., Carpenter W. T., Caspi A., Clark L. A., Eaton N. R., Forbes M. K., Forbush K. T., Goldberg D., Hasin D., Hyman S. E., Ivanova M. Y., Lynam D. R., Markon K.\u2004.\u2004. Zimmerman M. (2017). The hierarchical taxonomy of psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126(4), 454-477. https:\/\/doi.org\/10.1037\/abn0000258","DOI":"10.1037\/abn0000258"},{"key":"bibr28-10731911211003976","doi-asserted-by":"crossref","unstructured":"Levin R., Nielsen T. A. (2007). Disturbed dreaming, posttraumatic stress disorder, and affect distress: A review and neurocognitive model. Psychological Bulletin, 133(3), 482-528. https:\/\/doi.org\/10.1037\/0033-2909.133.3.482","DOI":"10.1037\/0033-2909.133.3.482"},{"key":"bibr29-10731911211003976","doi-asserted-by":"crossref","unstructured":"Lorenzo-Seva U., ten-Berge J. M. F. (2006). Tucker\u2019s congruence coefficient as a meaningful index of factor similarity. Methodology, 2(2), 57-64. https:\/\/doi.org\/10.1027\/1614-2241.2.2.57","DOI":"10.1027\/1614-2241.2.2.57"},{"key":"bibr30-10731911211003976","doi-asserted-by":"crossref","unstructured":"Mahoney A. E. J., Hobbs M. J., Newby J. M., Williams A. D., Sunderland M., Andrews G. (2016). The Worry Behaviors Inventory: Assessing the behavioral avoidance associated with generalized anxiety disorder. Journal of Affective Disorders, 203(October), 256-264. https:\/\/doi.org\/10.1016\/j.jad.2016.06.020","DOI":"10.1016\/j.jad.2016.06.020"},{"key":"bibr31-10731911211003976","doi-asserted-by":"crossref","unstructured":"Meade A. W., Craig S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437-455. https:\/\/doi.org\/10.1037\/a0028085","DOI":"10.1037\/a0028085"},{"key":"bibr32-10731911211003976","doi-asserted-by":"crossref","unstructured":"Reio T. G.Jr. (2010). The threat of common method variance bias to theory building. Human Resource Development Review, 9(4), 405-411. https:\/\/doi.org\/10.1177\/1534484310380331","DOI":"10.1177\/1534484310380331"},{"key":"bibr33-10731911211003976","doi-asserted-by":"crossref","unstructured":"Ruggero C. J., Kotov R., Hopwood C. J., First M., Clark L. A., Skodol A. E., Mullins-Sweatt S. N., Patrick C. J., Bach B., Cicero D. C., Docherty A., Simms L. J., Bagby R. M., Krueger R. F., Callahan J. L., Chmielewski M., Conway C. C., De Clercq B., Dornbach-Bender A.\u2004.\u2004. Zimmermann J. (2019). Integrating the Hierarchical Taxonomy of Psychopathology (HiTOP) into clinical practice. Journal of Consulting and Clinical Psychology, 87(12), 1069-1084. https:\/\/doi.org\/10.1037\/ccp0000452","DOI":"10.1037\/ccp0000452"},{"key":"bibr34-10731911211003976","doi-asserted-by":"crossref","unstructured":"Ruggero C. J., Kotov K., Watson D., Kilmer J. N., Perlman G., Liu K. (2014). Beyond a single index of mania symptoms: Structure and validity of subdimensions. Journal of Affective Disorders, 161(June), 8-15. https:\/\/doi.org\/10.1016\/j.jad.2014.02.044","DOI":"10.1016\/j.jad.2014.02.044"},{"key":"bibr35-10731911211003976","doi-asserted-by":"crossref","unstructured":"Soto C. J., John O. P. (2017). Short and extra-short forms of the Big Five Inventory-2: The BFI-2-S and BFI-2-XS. Journal of Research in Personality, 68(June), 69-81. https:\/\/doi.org\/10.1016\/j.jrp.2017.02.004","DOI":"10.1016\/j.jrp.2017.02.004"},{"key":"bibr36-10731911211003976","doi-asserted-by":"publisher","DOI":"10.21236\/AD0047524"},{"key":"bibr37-10731911211003976","doi-asserted-by":"crossref","unstructured":"Waszczuk M. A., Kotov R., Ruggero C., Gamez W., Watson D. (2017). Hierarchical structure of emotional disorders: From individual symptoms to the spectrum. Journal of Abnormal Psychology, 126(5), 613-634. https:\/\/doi.org\/10.1037\/abn0000264","DOI":"10.1037\/abn0000264"},{"key":"bibr38-10731911211003976","doi-asserted-by":"crossref","unstructured":"Watson D. (2012). Objective tests as instruments of psychological theory and research. In Cooper H. (Ed.), Handbook of research methods in psychology. Vol. 1: Foundations, planning, measures, and psychometrics (pp. 349-369). American Psychological Association. https:\/\/doi.org\/10.1037\/13619-019","DOI":"10.1037\/13619-019"},{"key":"bibr39-10731911211003976","author":"Watson D.","journal-title":"World Psychiatry"},{"key":"bibr40-10731911211003976","doi-asserted-by":"crossref","unstructured":"Watson D., Naragon-Gainey K. (2014). Personality, emotions, and the emotional disorders. Clinical Psychological Science, 2(4), 422-442. https:\/\/doi.org\/10.1177\/2167702614536162","DOI":"10.1177\/2167702614536162"},{"key":"bibr41-10731911211003976","doi-asserted-by":"crossref","unstructured":"Watson D., O\u2019Hara M. W., Naragon-Gainey K., Koffel E., Chmielewski M., Kotov R., Stasik S. M., Ruggero C. J. (2012). Development and validation of new anxiety and bipolar symptom scales for an expanded version of the IDAS (the IDAS-II). Assessment, 19(4), 399-420. https:\/\/doi.org\/10.1177\/1073191112449857","DOI":"10.1177\/1073191112449857"},{"key":"bibr42-10731911211003976","doi-asserted-by":"crossref","unstructured":"Watson D., Stanton K., Khoo S., Ellickson-Larew S., Stasik-O\u2019Brien S. M. (2019). Extraversion and psychopathology: A multilevel hierarchical review. Journal of Research in Personality, 81(August), 1-10. https:\/\/doi.org\/10.1016\/j.jrp.2019.04.009","DOI":"10.1016\/j.jrp.2019.04.009"},{"key":"bibr43-10731911211003976","doi-asserted-by":"crossref","unstructured":"Watson D., Stasik S. M., Ellickson-Larew S., Stanton K. (2015). Explicating the psychopathological correlates of anomalous sleep experiences. Psychology of Consciousness: Theory, Research, and Practice, 2(1), 57-78. https:\/\/doi.org\/10.1037\/cns0000038","DOI":"10.1037\/cns0000038"},{"key":"bibr44-10731911211003976","doi-asserted-by":"crossref","unstructured":"Watson D., Stasik-O\u2019Brien S. M., Ellickson-Larew S., Stanton K. (2018). Explicating the dispositional basis of the OCRDs: A hierarchical perspective. Journal of Psychopathology and Behavioral Assessment, 40(3), 497-513. https:\/\/doi.org\/10.1007\/s10862-018-9667-5","DOI":"10.1007\/s10862-018-9667-5"},{"key":"bibr45-10731911211003976","doi-asserted-by":"crossref","unstructured":"Watson D., Wu K. D. (2005). Development and validation of the Schedule of Compulsions, Obsessions, and Pathological Impulses (SCOPI). Assessment, 12(1), 50-65. https:\/\/doi.org\/10.1177\/1073191104271483","DOI":"10.1177\/1073191104271483"},{"key":"bibr46-10731911211003976","doi-asserted-by":"crossref","unstructured":"Widiger T. A., Sellbom M., Chmielewski M., Clark L. A., DeYoung C. G., Kotov R., Kruger R. F., Lynam D. R., Miller J. D., Mullins-Sweatt S., Samuel D. B., South S. C., Tackett J. L., Thomas K. M., Watson D., Wright A. G. C. (2019). Personality in a hierarchical model of psychopathology. Clinical Psychological Science, 7(1), 77-92. https:\/\/doi.org\/10.1177\/2167702618797105","DOI":"10.1177\/2167702618797105"},{"key":"bibr47-10731911211003976","doi-asserted-by":"crossref","unstructured":"Yufik T., Simms L. J. (2010). A meta-analytic investigation of the structure of posttraumatic stress disorder symptoms. Journal of Abnormal Psychology, 119(4), 764-776. https:\/\/doi.org\/10.1037\/a0020981","DOI":"10.1037\/a0020981"}],"container-title":["Assessment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10731911211003976","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/10731911211003976","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10731911211003976","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T12:11:11Z","timestamp":1777378271000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/10731911211003976"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,2]]},"references-count":47,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1]]}},"alternative-id":["10.1177\/10731911211003976"],"URL":"https:\/\/doi.org\/10.1177\/10731911211003976","relation":{},"ISSN":["1073-1911","1552-3489"],"issn-type":[{"value":"1073-1911","type":"print"},{"value":"1552-3489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,2]]}}}