{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T22:40:32Z","timestamp":1792622432343,"version":"4.5.0"},"reference-count":44,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Hum Factors"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Privacy concerns represent a primary barrier to the adoption of large language model (LLM) applications, particularly in sensitive domains such as health care. Although a growing body of survey-based research has examined user privacy concerns with LLMs, no standardized instrument exists for measuring this construct.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to develop and validate a concise, standardized questionnaire for assessing user privacy concerns with LLM-based systems, suitable for use by researchers and practitioners such as AI application developers and health care professionals.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n A literature review was conducted to identify privacy-related domains relevant to LLM use. A draft questionnaire was developed and subjected to content validity review by 8 experts. The finalized instrument was administered via Qualtrics to adults aged 18 years or older who were aware of LLM-based applications, recruited through the University of Pittsburgh, LinkedIn, and the Pitt+Me research participant registry between April 2025 and July 2026. Internal consistency reliability and factor structure were assessed, with factor structure examined in a split-sample design, in which exploratory factor analysis (EFA) was conducted on a random one-third of the sample and confirmatory factor analysis (CFA) on the holdout two-thirds.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n All questionnaire items achieved acceptable content validity (item-level content validity index >0.80; average scale-level content validity index=0.964). A total of 603 responses were collected. After preprocessing, the final sample comprised 361 participants. Most participants were younger than 35 years of age (198\/361, 54.8%), female (230\/361, 63.7%), and White (285\/361, 78.9%); 88.1% (318\/361) held at least a bachelor\u2019s degree. The 6-item questionnaire showed high internal consistency (Cronbach \u03b1=0.90; McDonald \u03c9=0.90). EFA (n=121) supported a single dominant factor with all standardized loadings \u22650.70. CFA (n=240) showed excellent fit (comparative fit index=0.996; Tucker-Lewis index=0.993; root mean square error of approximation=0.046; standardized root mean square residual=0.057). A 2-factor model separating the transparency- and control-related items fit significantly better than the 1-factor model (\u0394\n \u03c7<\/jats:italic>\n 2<\/jats:sup>\n 1<\/jats:sub>\n =8.9;\n P<\/jats:italic>\n =.003), but the 2 factors were correlated at\n r<\/jats:italic>\n =0.83, indicating closely related facets. Because the Fornell-Larcker criterion was met for the transparency and control subscale but not for data use and risk salience, the 6-item total score remains the recommended primary metric.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n The LLM privacy concern assessment is a concise, psychometrically sound 6-item instrument that measures privacy concern with LLM-based systems via 2 correlated subdimensions (transparency and control; data use and risk salience) summarized by a single total score. It enables standardized measurement across studies and practical privacy assessment in real-world LLM applications.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/102910","type":"journal-article","created":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T22:08:41Z","timestamp":1792620521000},"page":"e102910-e102910","source":"Crossref","is-referenced-by-count":0,"title":["Development and Validation of the Large Language Model Privacy Concern Assessment Questionnaire: Psychometric Evaluation Study on a Brief Instrument for Assessing User Privacy Concerns With Large Language Model Applications"],"prefix":"10.2196","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4398-0267","authenticated-orcid":false,"given":"Leming","family":"Zhou","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-4333-6440","authenticated-orcid":false,"given":"Hongtao","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-9034-7461","authenticated-orcid":false,"given":"Yi","family":"Xu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8383-8471","authenticated-orcid":false,"given":"I Made Agus","family":"Setiawan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9200-062X","authenticated-orcid":false,"given":"Haomin","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0933-8150","authenticated-orcid":false,"given":"Andi","family":"Saptono","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7882-4358","authenticated-orcid":false,"given":"Yong K","family":"Choi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4907-8402","authenticated-orcid":false,"given":"Bambang","family":"Parmanto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,21]]},"reference":[{"key":"R1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chbr.2025.100841","article-title":"Privacy disclosure to large language models: a large-scale study on awareness, benefits, and concerns in health contexts across three countries","volume":"20","author":"Choi","journal-title":"Comput Hum Behav Rep"},{"key":"R2","doi-asserted-by":"publisher","DOI":"10.1016\/j.pec.2025.109271","article-title":"Exploring the influence of privacy concerns, AI literacy, and perceived health stigma on AI chatbot use in healthcare: an uncertainty reduction approach","volume":"140","author":"Liu","journal-title":"Patient Educ Couns"},{"key":"R3","doi-asserted-by":"crossref","unstructured":"Liu Z Hu L Zhou T Tang Y Cai Z . Prevalence overshadows concerns? Understanding Chinese users\u2019 privacy awareness and expectations towards LLM-based healthcare consultation. Presented at: 2025 IEEE Symposium on Security and Privacy (SP); May 12-15, 2025; San Francisco, CA, USA. 2025.[doi: 10.1109\/SP61157.2025.00092]","DOI":"10.1109\/SP61157.2025.00092"},{"key":"R4","doi-asserted-by":"crossref","unstructured":"Tran S Lu H Slaughter I et al. Understanding privacy norms around LLM-based chatbots: a contextual integrity perspective. Presented at: Proceedings of the Eighth AAAI\/ACM Conference on AI, Ethics, and Society (AIES-25); Oct 20-22, 2025; Madrid, Spain. [doi: 10.1609\/aies.v8i3.36735]","DOI":"10.1609\/aies.v8i3.36735"},{"issue":"4","key":"R5","doi-asserted-by":"publisher","first-page":"989","DOI":"10.2307\/41409970","article-title":"Information privacy research: an interdisciplinary review1","volume":"35","author":"Smith","journal-title":"MIS Q"},{"issue":"2","key":"R6","doi-asserted-by":"publisher","first-page":"167","DOI":"10.2307\/249477","article-title":"Information privacy: measuring individuals\u2019 concerns about organizational practices1","volume":"20","author":"Smith","journal-title":"MIS Q"},{"issue":"4","key":"R7","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1287\/isre.1040.0032","article-title":"Internet Users\u2019 Information Privacy Concerns (IUIPC): the construct, the scale, and a causal model","volume":"15","author":"Malhotra","journal-title":"Inf Syst Res"},{"key":"R8","unstructured":"Xu H Gupta S Rosson MB Carroll JM . Measuring mobile users\u2019 concerns for information privacy. Presented at: 33rd International Conference on Information Systems (ICIS 2012); Dec 16-19, 2012; Orlando, FL, USA. URL: https:\/\/aisel.aisnet.org\/cgi\/viewcontent.cgi?article=1108&context=icis2012 [Accessed 11-10-2026]"},{"issue":"1","key":"R9","doi-asserted-by":"publisher","first-page":"275","DOI":"10.25300\/MISQ\/2013\/37.1.12","article-title":"Internet privacy concerns: an integrated conceptualization and four empirical studies1","volume":"37","author":"Hong","journal-title":"MIS Q"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1509\/jppm.19.1.62.16949","article-title":"Dimensions of privacy concern among online consumers","volume":"19","author":"Sheehan","journal-title":"J Public Policy Mark"},{"key":"R11","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2022.954964","article-title":"Privacy concerns toward short-form video platforms: scale development and validation","volume":"13","author":"Wang","journal-title":"Front Psychol"},{"key":"R12","unstructured":"Colnago J Cranor LF Acquisti A Stanton KH . Is it a concern or a preference? An investigation into the ability of privacy scales to capture and distinguish granular privacy constructs. Presented at: Proceedings of the Eighteenth Symposium on Usable Privacy and Security (SOUPS 2022); Aug 8-9, 2022; Boston, MA, USA. URL: https:\/\/www.usenix.org\/system\/files\/soups2022-colnago.pdf [Accessed 11-10-2026]"},{"issue":"4","key":"R13","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1108\/ICS-11-2024-0289","article-title":"Revisiting privacy scales: an investigation into the ability of privacy scales to capture and distinguish granular privacy constructs","volume":"34","author":"Farzand","journal-title":"Inf Comput Secur"},{"issue":"2","key":"R14","doi-asserted-by":"publisher","first-page":"235","DOI":"10.2478\/popets-2021-0026","article-title":"Validity and reliability of the scale Internet Users\u2019 Information Privacy Concerns (IUIPC)","volume":"2021","author":"Gro\u00df","journal-title":"Proc Priv Enh Technol"},{"issue":"1","key":"R15","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1287\/isre.13.1.36.97","article-title":"An empirical examination of the concern for information privacy instrument","volume":"13","author":"Stewart","journal-title":"Inf Syst Res"},{"key":"R16","doi-asserted-by":"crossref","unstructured":"Zhang Z Jia M Lee H et al. \u201cIt\u2019s a fair game\u201d, or is it? Examining how users navigate disclosure risks and benefits when using LLM-based conversational agents. Presented at: CHI \u201924: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems; May 11-16, 2024; Honolulu, HI, USA. [doi: 10.1145\/3613904.3642385]","DOI":"10.1145\/3613904.3642385"},{"key":"R17","unstructured":"Carlini N Tramer F Wallace E et al. Extracting training data from large language models. Presented at: 30th USENIX Security Symposium (USENIX Security 21); Aug 11-13, 2021; Vancouver, BC. URL: https:\/\/www.usenix.org\/system\/files\/sec21-carlini-extracting.pdf [Accessed 11-10-2026]"},{"key":"R18","unstructured":"Staab R Vero M Balunovi\u0107 M Vechev M . Beyond memorization: violating privacy via inference with large language models. Presented at: Proceedings of ICLR 2024 (The Twelfth International Conference on Learning Representations); May 7-11, 2024; Vienna, Austria. URL: https:\/\/proceedings.iclr.cc\/paper_files\/paper\/2024\/file\/9028b8a3ca98f58e373f0c1497a17448-Paper-Conference.pdf [Accessed 11-10-2026]"},{"issue":"2","key":"R19","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1111\/j.1468-2958.1975.tb00258.x","article-title":"Some explorations in initial interaction and beyond: toward a developmental theory of interpersonal communication","volume":"1","author":"Berger","journal-title":"Human Comm Res"},{"issue":"1","key":"R20","first-page":"119","volume":"79","author":"Nissenbaum","journal-title":"Wash L Rev"},{"key":"R21","doi-asserted-by":"publisher","DOI":"10.3389\/fpubh.2018.00149","article-title":"Best practices for developing and validating scales for health, social, and behavioral research: a primer","volume":"6","author":"Boateng","journal-title":"Front Public Health"},{"issue":"2","key":"R22","doi-asserted-by":"publisher","first-page":"252","DOI":"10.2307\/258705","article-title":"Toward a unified conception of business ethics: integrative social contracts theory","volume":"19","author":"Donaldson","journal-title":"Acad Manag Rev"},{"issue":"3","key":"R23","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1007\/s11747-011-0300-3","article-title":"Guidelines for choosing between multi-item and single-item scales for construct measurement: a predictive validity perspective","volume":"40","author":"Diamantopoulos","journal-title":"J Acad Mark Sci"},{"issue":"2","key":"R24","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1509\/jmkr.44.2.175","article-title":"The predictive validity of multiple-item versus single-item measures of the same constructs","volume":"44","author":"Bergkvist","journal-title":"J Mark Res"},{"issue":"2","key":"R25","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1037\/0021-9010.82.2.247","article-title":"Overall job satisfaction: how good are single-item measures?","volume":"82","author":"Wanous","journal-title":"J Appl Psychol"},{"issue":"2","key":"R26","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1086\/376806","article-title":"A critical review of construct indicators and measurement model misspecification in marketing and consumer research","volume":"30","author":"Jarvis","journal-title":"J Consum Res"},{"issue":"4","key":"R27","doi-asserted-by":"publisher","first-page":"689","DOI":"10.1177\/1094428106294734","article-title":"The usefulness of unit weights in creating composite scores: a literature review, application to content validity, and meta-analysis","volume":"10","author":"Bobko","journal-title":"Organ Res Methods"},{"issue":"1","key":"R28","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1186\/s41155-016-0057-1","article-title":"Scale development: ten main limitations and recommendations to improve future research practices","volume":"30","author":"Morgado","journal-title":"Psicol Refl Cr\u00edt"},{"issue":"6","key":"R29","first-page":"382","volume":"35","author":"Lynn","journal-title":"Nurs Res"},{"issue":"5","key":"R30","doi-asserted-by":"publisher","first-page":"489","DOI":"10.1002\/nur.20147","article-title":"The content validity index: are you sure you know what\u2019s being reported? Critique and recommendations","volume":"29","author":"Polit","journal-title":"Res Nurs Health"},{"issue":"4","key":"R31","doi-asserted-by":"publisher","first-page":"459","DOI":"10.1002\/nur.20199","article-title":"Is the CVI an acceptable indicator of content validity? Appraisal and recommendations","volume":"30","author":"Polit","journal-title":"Res Nurs Health"},{"issue":"1","key":"R32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/10705519909540118","article-title":"Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives","volume":"6","author":"Hu","journal-title":"Struct Equ Model"},{"key":"R33","doi-asserted-by":"publisher","DOI":"10.1007\/s11165-016-9602-2","article-title":"The use of Cronbach\u2019s alpha when developing and reporting research instruments in science education","volume":"48","author":"Taber","journal-title":"Res Sci Educ"},{"key":"R34","unstructured":"Nunnally JC Bernstein IH . Psychometric Theory. 3rd ed. McGraw Hill; 1994. ISBN: 978-0070478497"},{"issue":"2","key":"R35","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1007\/BF02289447","article-title":"A rationale and test for the number of factors in factor analysis","volume":"30","author":"HORN","journal-title":"Psychometrika"},{"issue":"1","key":"R36","doi-asserted-by":"publisher","first-page":"409","DOI":"10.3758\/s13428-018-1055-2","article-title":"RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: the story they tell depends on the estimation methods","volume":"51","author":"Xia","journal-title":"Behav Res Methods"},{"key":"R37","unstructured":"Kline RB . Principles and Practice of Structural Equation Modeling. Guilford Press; 2023:494. ISBN: 9781462551910"},{"issue":"1","key":"R38","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1177\/002224378101800104","article-title":"Evaluating structural equation models with unobservable variables and measurement error","volume":"18","author":"Fornell","journal-title":"J Mark Res"},{"issue":"6","key":"R39","doi-asserted-by":"publisher","first-page":"952","DOI":"10.1080\/10705511.2019.1704289","article-title":"semopy: a Python package for structural equation modeling","volume":"27","author":"Igolkina","journal-title":"Struct Equ Model"},{"issue":"1","key":"R40","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1037\/1082-989X.4.1.84","article-title":"Sample size in factor analysis","volume":"4","author":"MacCallum","journal-title":"Psychol Methods"},{"issue":"2","key":"R41","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1080\/00273170902794206","article-title":"Exploratory factor analysis with small sample sizes","volume":"44","author":"de Winter","journal-title":"Multivariate Behav Res"},{"issue":"7","key":"R42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.7275\/jyj1-4868","article-title":"Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis","volume":"10","author":"Costello","journal-title":"Pract Assess Res Eval"},{"issue":"6","key":"R43","doi-asserted-by":"publisher","first-page":"913","DOI":"10.1177\/0013164413495237","article-title":"Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety","volume":"76","author":"Wolf","journal-title":"Educ Psychol Meas"},{"issue":"5","key":"R44","doi-asserted-by":"publisher","first-page":"879","DOI":"10.1037\/0021-9010.88.5.879","article-title":"Common method biases in behavioral research: a critical review of the literature and recommended remedies","volume":"88","author":"Podsakoff","journal-title":"J Appl Psychol"}],"container-title":["JMIR Human Factors"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T22:08:49Z","timestamp":1792620529000},"score":1,"resource":{"primary":{"URL":"https:\/\/humanfactors.jmir.org\/2026\/1\/e102910"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,21]]},"references-count":44,"URL":"https:\/\/doi.org\/10.2196\/102910","relation":{},"ISSN":["2292-9495"],"issn-type":[{"value":"2292-9495","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,21]]},"article-number":"v13i4e102910"}}