{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T07:26:24Z","timestamp":1774596384591,"version":"3.50.1"},"reference-count":56,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2021,4,24]],"date-time":"2021-04-24T00:00:00Z","timestamp":1619222400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004329","name":"Javna Agencija za Raziskovalno Dejavnost RS","doi-asserted-by":"publisher","award":["P5-0399"],"award-info":[{"award-number":["P5-0399"]}],"id":[{"id":"10.13039\/501100004329","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004329","name":"Javna Agencija za Raziskovalno Dejavnost RS","doi-asserted-by":"publisher","award":["L5-9337"],"award-info":[{"award-number":["L5-9337"]}],"id":[{"id":"10.13039\/501100004329","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>This systematic review addresses problems identified in existing research on survey measurements of individuals\u2019 information privacy concerns in online contexts. The search in this study focused on articles published between 1996 and 2019 and yielded 970 articles. After excluding duplicates and screening for eligibility, we were left with 13 articles in which the investigators developed a total of 16 survey scales. In addition to reviewing the conceptualizations, contexts, and dimensionalities of the scales, we evaluated the quality of methodological procedures used in the scale development process, drawing upon the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) Risk of Bias checklist. The results confirmed that the breadth of conceptualizations and dimensions of information privacy concerns are constructed with a low emphasis on contextuality. Assessment of the quality of methodological procedures suggested a need for a more thorough evaluation of content validity. We provide several recommendations for tackling these issues and propose new research directions.<\/jats:p>","DOI":"10.3390\/informatics8020031","type":"journal-article","created":{"date-parts":[[2021,4,25]],"date-time":"2021-04-25T02:12:57Z","timestamp":1619316777000},"page":"31","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Should We Be Concerned about How Information Privacy Concerns Are Measured in Online Contexts? A Systematic Review of Survey Scale Development Studies"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7425-1644","authenticated-orcid":false,"given":"Jo\u0161t","family":"Bartol","sequence":"first","affiliation":[{"name":"Centre for Social Informatics, Faculty of Social Sciences, University of Ljubljana, 1000 Ljubljana, Slovenia"},{"name":"Faculty of Arts, University of Ljubljana, 1000 Ljubljana, Slovenia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3253-7959","authenticated-orcid":false,"given":"Vasja","family":"Vehovar","sequence":"additional","affiliation":[{"name":"Centre for Social Informatics, Faculty of Social Sciences, University of Ljubljana, 1000 Ljubljana, Slovenia"}]},{"given":"Andra\u017e","family":"Petrov\u010di\u010d","sequence":"additional","affiliation":[{"name":"Centre for Social Informatics, Faculty of Social Sciences, University of Ljubljana, 1000 Ljubljana, Slovenia"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1111\/1540-4560.00072","article-title":"Social and Political Dimensions of Privacy","volume":"59","author":"Westin","year":"2003","journal-title":"J. Soc. Issues"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"989","DOI":"10.2307\/41409970","article-title":"Information Privacy Research: An Interdisciplinary Review","volume":"35","author":"Smith","year":"2011","journal-title":"MIS Q."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Van Dijck, J. (2013). The Culture of Connectivity: A Critical History of Social Media, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780199970773.001.0001"},{"key":"ref_4","unstructured":"(2020, November 06). Regulation (EU) 2016\/679. The Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95\/46\/EC (General Data Protection Regulation), Available online: https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/PDF\/?uri=CELEX:32016R0679."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1017","DOI":"10.2307\/41409971","article-title":"Privacy in the Digital Age: A Review of Information Privacy Research in Information Systems","volume":"35","author":"Crossler","year":"2011","journal-title":"MIS Q."},{"key":"ref_6","unstructured":"Rohunen, A. (2019). Advancing Information Privacy Concerns Evaluation in Personal Data Intensive Services. [Ph.D. Thesis, University of Oulu]."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Musso, F., and Druica, E. (2020). An Empirical Investigation to Improve Information Sharing in Online Settings: A Multi-Target Comparison. Handbook of Research on Retailing Techniques for Optimal Consumer Engagement and Experiences, IGI Global.","DOI":"10.4018\/978-1-7998-1412-2"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1002\/asi.20459","article-title":"Development of Measures of Online Privacy Concern and Protection for Use on the Internet","volume":"58","author":"Buchanan","year":"2007","journal-title":"J. Am. Soc. Inf. Sci. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"275","DOI":"10.25300\/MISQ\/2013\/37.1.12","article-title":"Internet Privacy Concerns: An Integrated Conceptualization and Four Empirical Studies","volume":"37","author":"Hong","year":"2013","journal-title":"MIS Q."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"14","DOI":"10.36965\/OJAKM.2017.5(1)14-26","article-title":"Social Media Sites Privacy Concerns: Empirical Validation of an Instrument","volume":"5","author":"Koohang","year":"2017","journal-title":"Online J. Appl. Knowl. Manag."},{"key":"ref_11","first-page":"453","article-title":"Empirical Studies on Online Information Privacy Concerns: Literature Review and an Integrative Framework","volume":"28","author":"Li","year":"2011","journal-title":"Commun. Assoc. Inf. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1016\/j.im.2018.10.001","article-title":"A Chronological Review of Empirical Research on Personal Information Privacy Concerns: An Analysis of Contexts and Research Constructs","volume":"56","author":"Yun","year":"2019","journal-title":"Inf. Manag."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1287\/isre.1040.0032","article-title":"Internet Users\u2019 Information Privacy Concerns (IUIPC): The Construct, the Scale, and a Causal Model","volume":"15","author":"Malhotra","year":"2004","journal-title":"Inf. Syst. Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"167","DOI":"10.2307\/249477","article-title":"Information Privacy: Measuring Individuals\u2019 Concerns about Organizational Practices","volume":"20","author":"Smith","year":"1996","journal-title":"MIS Q."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1002\/ejsp.2049","article-title":"Is the Privacy Paradox a Relic of the Past? An in-Depth Analysis of Privacy Attitudes and Privacy Behaviors","volume":"45","author":"Dienlin","year":"2015","journal-title":"Eur. J. Soc. Psychol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1016\/j.cose.2018.04.002","article-title":"Explaining the Privacy Paradox: A Systematic Review of Literature Investigating Privacy Attitude and Behavior","volume":"77","author":"Gerber","year":"2018","journal-title":"Comput. Secur."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1133","DOI":"10.1016\/j.ijhcs.2013.09.002","article-title":"Guide to Measuring Privacy Concern: Review of Survey and Observational Instruments","volume":"71","author":"Preibusch","year":"2013","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1207\/s15327051hci2103_2","article-title":"Collective Information Practice: Exploring Privacy and Security as Social and Cultural Phenomena","volume":"21","author":"Dourish","year":"2006","journal-title":"Hum. Comput. Interact."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1016\/j.dss.2012.06.010","article-title":"Theories in Online Information Privacy Research: A Critical Review and an Integrated Framework","volume":"54","author":"Li","year":"2012","journal-title":"Decis. Support Syst."},{"key":"ref_20","first-page":"359","article-title":"An Instrument for Measuring Social Media Users\u2019 Information Privacy Concerns","volume":"6","author":"Osatuyi","year":"2014","journal-title":"J. Curr. Issues Media Telecommun."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1007\/s11136-018-1798-3","article-title":"COSMIN Guideline for Systematic Reviews of Patient-Reported Outcome Measures","volume":"27","author":"Prinsen","year":"2018","journal-title":"Qual. Life Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1171","DOI":"10.1007\/s11136-017-1765-4","article-title":"COSMIN Risk of Bias Checklist for Systematic Reviews of Patient-Reported Outcome Measures","volume":"27","author":"Mokkink","year":"2018","journal-title":"Qual. Life Res."},{"key":"ref_23","first-page":"378","article-title":"Explaining Diversity and Conflicts in Privacy Behavior Models","volume":"60","author":"Rohunen","year":"2020","journal-title":"J. Comput. Inf. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Acquisti, A. (2004, January 17\u201320). Privacy in electronic commerce and the economics of immediate gratification. Proceedings of the 5th ACM conference on Electronic commerce (EC 2004), New York, NY, USA.","DOI":"10.1145\/988772.988777"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Nissenbaum, H.F. (2010). Privacy in Context: Technology, Policy, and the Integrity of Social Life, Stanford University Press.","DOI":"10.1515\/9780804772891"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1108\/14684520710780395","article-title":"The Dimensionality of Customer Privacy Concern on the Internet","volume":"31","author":"Montoso","year":"2007","journal-title":"Online Inf. Rev."},{"key":"ref_27","unstructured":"DeVellis, R.F. (2012). Scale Development: Theory and Applications, Sage. [3rd ed.]."},{"key":"ref_28","unstructured":"Mokkink, L.B., Prinsen, C.A.C., Patrick, D.L., Alonso, J., Bouter, L.M., de Vet, H.C.W., and Terwee, C.B. (2020, November 06). COSMIN Methodology for Systematic Reviews of Patient-reported Outcome Measures (PROMs): User Manual. Available online: https:\/\/www.cosmin.nl\/wp-content\/uploads\/COSMIN-syst-review-for-PROMs-manual_version-1_feb-2018-1.pdf."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1136\/bmj.b2535","article-title":"Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement","volume":"339","author":"Moher","year":"2009","journal-title":"BMJ"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1159","DOI":"10.1007\/s11136-018-1829-0","article-title":"COSMIN Methodology for Evaluating the Content Validity of Patient-Reported Outcome Measures: A Delphi Study","volume":"27","author":"Terwee","year":"2018","journal-title":"Qual. Life Res."},{"key":"ref_31","unstructured":"PRISMA (2020, November 05). Available online: http:\/\/prisma-statement.org\/."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1080\/01449290410001715723","article-title":"Internet Privacy Concerns and Their Antecedents\u2014Measurement Validity and a Regression Model","volume":"23","author":"Dinev","year":"2004","journal-title":"Behav. Inf. Technol."},{"key":"ref_33","unstructured":"Taddicken, M.M.M. (2010, January 22\u201326). Measuring online privacy concern and protection in the (social) web: Development of the APCP and APCP-18 scale. Proceedings of the 60th Annual Conference of the International Communication Association (ICA 2010), Singapore."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zheng, S., Shi, K., Zeng, Z., and Lu, Q. (2010, January 7\u201310). The exploration of instrument of users\u2019 privacy concerns of social network service. Proceedings of the 2010 IEEE International Conference on Industrial Engineering and Engineering Management (IEEE 2010), Macao, China.","DOI":"10.1109\/IEEM.2010.5674165"},{"key":"ref_35","unstructured":"Xu, H., Gupta, S., Rosson, M.B., and Carroll, J.M. (2012, January 16\u201319). Measuring mobile users\u2019 concerns for information privacy. Proceedings of the International Conference on Information Systems (ICIS 2012), Orlando, FL, USA."},{"key":"ref_36","unstructured":"Borena, B., Belanger, F., Ejigu, D., and Anteneh, S. (2015, January 13\u201315). Conceptualizing information privacy concern in low-income countries: An Ethiopian language instrument for social networks sites. Proceedings of the Twenty-First Americas Conference on Information Systems (AMCIS 2015), Fajardo, Puerto Rico."},{"key":"ref_37","first-page":"611","article-title":"Adaptation of Online Privacy Concern Scale into Turkish Culture","volume":"7","author":"Alakurt","year":"2017","journal-title":"Pegem E\u011fitim ve \u00d6\u011fretim Derg."},{"key":"ref_38","unstructured":"Buck, C., and Burster, S. (2017, January 10\u201312). App information privacy concerns. Proceedings of the Twenty-Third Americas Conference on Information Systems (AMCIS 2017), Boston, MA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"341","DOI":"10.2307\/249775","article-title":"\u2018How Did They Get My Name\u2019: An Exploratory Investigation of Consumer Attitudes toward Secondary Information Use","volume":"17","author":"Culnan","year":"1993","journal-title":"MIS Q."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1287\/orsc.10.1.104","article-title":"Information Privacy Concerns, Procedural Fairness, and Impersonal Trust: An Empirical Investigation","volume":"10","author":"Culnan","year":"1999","journal-title":"Organ. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1287\/isre.1060.0080","article-title":"An Extended Privacy Calculus Model for E-Commerce Transactions","volume":"17","author":"Dinev","year":"2006","journal-title":"Inf. Syst. Res."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Xu, H., Dinev, T., Smith, J.H., and Hart, P. (2008, January 14\u201317). Examining the formation of individual\u2019s privacy concerns: Toward an integrative view. Proceedings of the Twenty-Ninth International Conference on Information Systems (ICIS 2008), Paris, France.","DOI":"10.1016\/j.jsis.2007.09.002"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Petronio, S. (2002). Boundaries to Privacy: Dialectics of Disclosure, State University of New York Press.","DOI":"10.1353\/book4588"},{"key":"ref_44","unstructured":"Donaldson, T. (1989). The Ethics of International Business, Oxford University Press."},{"key":"ref_45","unstructured":"Friend, C. (2020, November 06). Social Contract Theory. Available online: https:\/\/www.iep.utm.edu\/soc-cont\/."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1111\/j.1540-4560.1977.tb01880.x","article-title":"Privacy as a Concept and a Social Issue: A Multidimensional Developmental Theory","volume":"33","author":"Laufer","year":"1977","journal-title":"J. Soc. Issues"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1002\/(SICI)1522-7138(199722)11:3<44::AID-DIR7>3.0.CO;2-X","article-title":"Relationship Marketing in Consumer Markets: A Comparison of Managerial and Consumer Attitudes about Information Privacy","volume":"11","author":"Campbell","year":"1997","journal-title":"J. Direct Mark."},{"key":"ref_48","unstructured":"Westin, A. (1967). Privacy and Freedom, Atheneum."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1002\/j.1662-6370.1998.tb00239.x","article-title":"A Theoretical Note on the Differences between Attitudes, Opinions, and Values","volume":"4","author":"Bergman","year":"1998","journal-title":"Swiss Polit. Sci. Rev."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"25","DOI":"10.2979\/esj.2006.4.3.25","article-title":"Privacy Concerns and Levels of Information Exchange: An Empirical Investigation of Intended e-Services Use","volume":"4","author":"Dinev","year":"2006","journal-title":"e-Serv. J."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"7","DOI":"10.2753\/JEC1086-4415100201","article-title":"Internet Privacy Concerns and Social Awareness as Determinants of Intention to Transact","volume":"10","author":"Dinev","year":"2005","journal-title":"Int. J. Electron. Commer."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1207\/s15506878jobem4901_6","article-title":"Teenagers\u2019 Perceptions of Online Privacy and Coping Behaviors: A Risk\u2013Benefit Appraisal Approach","volume":"49","author":"Youn","year":"2005","journal-title":"J. Broadcast. Electron. Media"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1023\/B:MARK.0000012471.31858.e5","article-title":"A Model Integrating the Multidimensional Developmental Theory of Privacy and Theory of Planned Behavior to Examine Fabrication of Information Online","volume":"14","author":"Lwin","year":"2003","journal-title":"Mark. Lett."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Bollen, K.A. (1989). Structural Equations with Latent Variables, John Wiley & Sons.","DOI":"10.1002\/9781118619179"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"293","DOI":"10.2307\/23044045","article-title":"Construct Measurement and Validation Procedures in MIS and Behavioral Research: Integrating New and Existing Techniques","volume":"35","author":"Mackenzie","year":"2011","journal-title":"MIS Q."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1037\/1040-3590.16.3.231","article-title":"Focus Groups in Psychological Assessment: Enhancing Content Validity by Consulting Members of the Target Population","volume":"16","author":"Vogt","year":"2004","journal-title":"Psychol. Assess."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/8\/2\/31\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:52:26Z","timestamp":1760161946000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/8\/2\/31"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,24]]},"references-count":56,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["informatics8020031"],"URL":"https:\/\/doi.org\/10.3390\/informatics8020031","relation":{},"ISSN":["2227-9709"],"issn-type":[{"value":"2227-9709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,24]]}}}