{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T15:36:47Z","timestamp":1776094607717,"version":"3.50.1"},"reference-count":69,"publisher":"Privacy Enhancing Technologies Symposium Advisory Board","issue":"2","license":[{"start":{"date-parts":[[2022,3,3]],"date-time":"2022-03-03T00:00:00Z","timestamp":1646265600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/3.0"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Privacy tasks can be challenging for developers, resulting in privacy frameworks and guidelines from the research community which are designed to assist developers in considering privacy features and applying privacy enhancing technologies in early stages of software development. However, how developers engage with privacy design strategies is not yet well understood. In this work, we look at the types of privacy-related advice developers give each other and how that advice maps to Hoepman\u2019s privacy design strategies.<\/jats:p>\n               <jats:p>We qualitatively analyzed 119 privacy-related accepted <jats:italic>answers<\/jats:italic> on <jats:italic>Stack Overflow<\/jats:italic> from the past five years and extracted 148 pieces of advice from these answers. We find that the advice is mostly around compliance with regulations and ensuring confidentiality with a focus on the <jats:monospace>inform<\/jats:monospace>, <jats:monospace>hide<\/jats:monospace>, <jats:monospace>control<\/jats:monospace>, and <jats:monospace>minimize <\/jats:monospace>of the Hoepman\u2019s privacy design strategies. Other strategies, <jats:monospace>abstract<\/jats:monospace>, <jats:monospace>separate<\/jats:monospace>, <jats:monospace>enforce<\/jats:monospace>, and <jats:monospace>demonstrate<\/jats:monospace>, are rarely advised. Answers often include links to official documentation and online articles, highlighting the value of both official documentation and other informal materials such as blog posts. We make recommendations for promoting the under-stated strategies through tools, and detail the importance of providing better developer support to handle third-party data practices.<\/jats:p>","DOI":"10.2478\/popets-2022-0038","type":"journal-article","created":{"date-parts":[[2022,3,5]],"date-time":"2022-03-05T04:34:43Z","timestamp":1646454883000},"page":"114-131","source":"Crossref","is-referenced-by-count":26,"title":["Understanding Privacy-Related Advice on Stack Overflow"],"prefix":"10.56553","volume":"2022","author":[{"given":"Mohammad","family":"Tahaei","sequence":"first","affiliation":[{"name":"University of Bristol"}]},{"given":"Tianshi","family":"Li","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}]},{"given":"Kami","family":"Vaniea","sequence":"additional","affiliation":[{"name":"University of Edinburgh"}]}],"member":"35752","published-online":{"date-parts":[[2022,3,3]]},"reference":[{"key":"2022060207212690786_j_popets-2022-0038_ref_001","doi-asserted-by":"crossref","unstructured":"[1] Yasemin Acar, Michael Backes, Sascha Fahl, Doowon Kim, Michelle L Mazurek, and Christian Stransky. You Get Where You\u2019re Looking for: The Impact of Information Sources on Code Security. In 2016 IEEE Symposium on Security and Privacy (SP), pages 289\u2013305. IEEE, May 2016. 10.1109\/SP.2016.25.10.1109\/SP.2016.25","DOI":"10.1109\/SP.2016.25"},{"key":"2022060207212690786_j_popets-2022-0038_ref_002","doi-asserted-by":"crossref","unstructured":"[2] Nitin Agrawal, Reuben Binns, Max Van Kleek, Kim Laine, and Nigel Shadbolt. Exploring Design and Governance Challenges in the Development of Privacy-Preserving Computation. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI \u201921, New York, NY, USA, 2021. ACM. 10.1145\/3411764.3445677.10.1145\/3411764.3445677","DOI":"10.1145\/3411764.3445677"},{"key":"2022060207212690786_j_popets-2022-0038_ref_003","doi-asserted-by":"crossref","unstructured":"[3] Sami Alkhatib, Jenny Waycott, George Buchanan, Marthie Grobler, and Shuo Wang. Privacy by Design in Aged Care Monitoring Devices? Well, Not Quite Yet! In 32nd Australian Conference on Human-Computer Interaction, OzCHI \u201920, page 492\u2013505, New York, NY, USA, 2020. ACM. 10.1145\/3441000.3441049.10.1145\/3441000.3441049","DOI":"10.1145\/3441000.3441049"},{"key":"2022060207212690786_j_popets-2022-0038_ref_004","doi-asserted-by":"crossref","unstructured":"[4] Miltiadis Allamanis and Charles Sutton. Why, when, and what: Analyzing Stack Overflow questions by topic, type, and code. In 2013 10th Working Conference on Mining Software Repositories (MSR), pages 53\u201356. IEEE, May 2013. 10.1109\/MSR.2013.6624004.10.1109\/MSR.2013.6624004","DOI":"10.1109\/MSR.2013.6624004"},{"key":"2022060207212690786_j_popets-2022-0038_ref_005","doi-asserted-by":"crossref","unstructured":"[5] Majed Almansoori, Jessica Lam, Elias Fang, Kieran Mulligan, Adalbert Gerald Soosai Raj, and Rahul Chatterjee. How Secure Are Our Computer Systems Courses? In Proceedings of the 2020 ACM Conference on International Computing Education Research, ICER \u201920, page 271\u2013281, New York, NY, USA, 2020. ACM. 10.1145\/3372782.3406266.10.1145\/3372782.3406266","DOI":"10.1145\/3372782.3406266"},{"key":"2022060207212690786_j_popets-2022-0038_ref_006","unstructured":"[6] Jeff Atwood. Attribution Required, 2009. URL https:\/\/stackoverflow.blog\/2009\/06\/25\/attribution-required\/."},{"key":"2022060207212690786_j_popets-2022-0038_ref_007","unstructured":"[7] Jeff Atwood. Stack Overflow Creative Commons Data Dump, 2009. URL https:\/\/stackoverflow.blog\/2009\/06\/04\/stack-overflow-creative-commons-data-dump\/."},{"key":"2022060207212690786_j_popets-2022-0038_ref_008","unstructured":"[8] Jeff Atwood. Academic Papers Using Stack Overflow Data, 2010. URL https:\/\/stackoverflow.blog\/2010\/05\/31\/academic-papers-using-stack-overflow-data\/."},{"key":"2022060207212690786_j_popets-2022-0038_ref_009","doi-asserted-by":"crossref","unstructured":"[9] Anton Barua, Stephen W Thomas, and Ahmed E Hassan. What are developers talking about? An analysis of topics and trends in Stack Overflow. Empirical Software Engineering, 19(3):619\u2013654, 2014. 10.1007\/s10664-012-9231-y.10.1007\/s10664-012-9231-y","DOI":"10.1007\/s10664-012-9231-y"},{"key":"2022060207212690786_j_popets-2022-0038_ref_010","doi-asserted-by":"crossref","unstructured":"[10] Maia J. Boyd, Jamar L. Sullivan Jr., Marshini Chetty, and Blase Ur. Understanding the Security and Privacy Advice Given to Black Lives Matter Protesters. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI \u201921, New York, NY, USA, 2021. ACM. 10.1145\/3411764.3445061.10.1145\/3411764.3445061","DOI":"10.1145\/3411764.3445061"},{"key":"2022060207212690786_j_popets-2022-0038_ref_011","doi-asserted-by":"crossref","unstructured":"[11] Virginia Braun and Victoria Clarke. Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2): 77\u2013101, 2006. 10.1191\/1478088706qp063oa.10.1191\/1478088706qp063oa","DOI":"10.1191\/1478088706qp063oa"},{"key":"2022060207212690786_j_popets-2022-0038_ref_012","doi-asserted-by":"crossref","unstructured":"[12] Julio C. Caiza, Yod-Samuel Mart\u00edn, Danny S. Guam\u00e1n, Jose M. Del Alamo, and Juan C. Yelmo. Reusable Elements for the Systematic Design of Privacy-Friendly Information Systems: A Mapping Study. IEEE Access, 7:66512\u201366535, 2019. 10.1109\/ACCESS.2019.2918003.","DOI":"10.1109\/ACCESS.2019.2918003"},{"key":"2022060207212690786_j_popets-2022-0038_ref_013","unstructured":"[13] Ann Cavoukian. Privacy by Design: The 7 Foundational Principles. Information and privacy commissioner of Ontario, Canada, 5, 2009. URL https:\/\/iab.org\/wp-content\/IABuploads\/2011\/03\/fred_carter.pdf."},{"key":"2022060207212690786_j_popets-2022-0038_ref_014","doi-asserted-by":"crossref","unstructured":"[14] Ann Cavoukian, Scott Taylor, and Martin E. Abrams. Privacy by Design: essential for organizational accountability and strong business practices. Identity in the Information Society, 3(2):405\u2013413, August 2010. 10.1007\/s12394-010-0053-z.10.1007\/s12394-010-0053-z","DOI":"10.1007\/s12394-010-0053-z"},{"key":"2022060207212690786_j_popets-2022-0038_ref_015","doi-asserted-by":"crossref","unstructured":"[15] Mina Deng, Kim Wuyts, Riccardo Scandariato, Bart Preneel, and Wouter Joosen. A privacy threat analysis framework: supporting the elicitation and fulfillment of privacy requirements. Requirements Engineering, 16(1):3\u201332, 2011. 10.1007\/s00766-010-0115-7.","DOI":"10.1007\/s00766-010-0115-7"},{"key":"2022060207212690786_j_popets-2022-0038_ref_016","unstructured":"[16] Nick Doty and Mohit Gupta. Privacy Design Patterns and Anti-Patterns, 2013. URL https:\/\/cups.cs.cmu.edu\/soups\/2013\/trustbusters2013\/Privacy_Design_Patterns-Antipatterns_Doty.pdf."},{"key":"2022060207212690786_j_popets-2022-0038_ref_017","doi-asserted-by":"crossref","unstructured":"[17] Cynthia Dwork. Differential privacy: A survey of results. In International conference on theory and applications of models of computation, pages 1\u201319. Springer, 2008. 10.1007\/978-3-540-79228-4_1.10.1007\/978-3-540-79228-4_1","DOI":"10.1007\/978-3-540-79228-4_1"},{"key":"2022060207212690786_j_popets-2022-0038_ref_018","doi-asserted-by":"crossref","unstructured":"[18] Serge Egelman, Julia Bernd, Gerald Friedland, and Dan Garcia. The Teaching Privacy Curriculum. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education, SIGCSE \u201916, page 591\u2013596, New York, NY, USA, 2016. ACM. 10.1145\/2839509.2844619.10.1145\/2839509.2844619","DOI":"10.1145\/2839509.2844619"},{"key":"2022060207212690786_j_popets-2022-0038_ref_019","doi-asserted-by":"crossref","unstructured":"[19] Pardis Emami-Naeini, Yuvraj Agarwal, Lorrie Faith Cranor, and Hanan Hibshi. Ask the Experts: What Should Be on an IoT Privacy and Security Label? In 2020 IEEE Symposium on Security and Privacy (SP), pages 447\u2013464. IEEE, 2020. 10.1109\/SP40000.2020.00043.10.1109\/SP40000.2020.00043","DOI":"10.1109\/SP40000.2020.00043"},{"key":"2022060207212690786_j_popets-2022-0038_ref_020","doi-asserted-by":"crossref","unstructured":"[20] Yuanyuan Feng, Yaxing Yao, and Norman Sadeh. A Design Space for Privacy Choices: Towards Meaningful Privacy Control in the Internet of Things. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI \u201921, New York, NY, USA, 2021. ACM. 10.1145\/3411764.3445148.10.1145\/3411764.3445148","DOI":"10.1145\/3411764.3445148"},{"key":"2022060207212690786_j_popets-2022-0038_ref_021","doi-asserted-by":"crossref","unstructured":"[21] Felix Fischer, Konstantin B\u00f6ttinger, Huang Xiao, Christian Stransky, Yasemin Acar, Michael Backes, and Sascha Fahl. Stack Overflow Considered Harmful? The Impact of Copy Paste on Android Application Security. In 2017 IEEE Symposium on Security and Privacy (SP), pages 121\u2013136. IEEE, May 2017. 10.1109\/SP.2017.31.10.1109\/SP.2017.31","DOI":"10.1109\/SP.2017.31"},{"key":"2022060207212690786_j_popets-2022-0038_ref_022","doi-asserted-by":"crossref","unstructured":"[22] Imane Fouad, Cristiana Santos, Feras Al Kassar, Nataliia Bielova, and Stefano Calzavara. On Compliance of Cookie Purposes with the Purpose Specification Principle. In IWPE 2020 - International Workshop on Privacy Engineering, pages 1\u20138, Genova, Italy, September 2020. Inria. URL https:\/\/hal.inria.fr\/hal-02567022.10.1109\/EuroSPW51379.2020.00051","DOI":"10.1109\/EuroSPW51379.2020.00051"},{"key":"2022060207212690786_j_popets-2022-0038_ref_023","doi-asserted-by":"crossref","unstructured":"[23] Daniel Greene and Katie Shilton. Platform privacies: Governance, collaboration, and the different meanings of \u201cprivacy\u201d in iOS and Android development. New Media & Society, 20 (4):1640\u20131657, 2018. 10.1177\/1461444817702397.","DOI":"10.1177\/1461444817702397"},{"key":"2022060207212690786_j_popets-2022-0038_ref_024","doi-asserted-by":"crossref","unstructured":"[24] Kilem Li Gwet. Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology, 61(1):29\u201348, 2008. 10.1348\/000711006X126600.10.1348\/000711006X12660018482474","DOI":"10.1348\/000711006X126600"},{"key":"2022060207212690786_j_popets-2022-0038_ref_025","doi-asserted-by":"crossref","unstructured":"[25] Irit Hadar, Tomer Hasson, Oshrat Ayalon, Eran Toch, Michael Birnhack, Sofia Sherman, and Arod Balissa. Privacy by designers: software developers\u2019 privacy mindset. Empirical Software Engineering, 23(1):259\u2013289, February 2018. 10.1007\/s10664-017-9517-1.10.1007\/s10664-017-9517-1","DOI":"10.1007\/s10664-017-9517-1"},{"key":"2022060207212690786_j_popets-2022-0038_ref_026","doi-asserted-by":"crossref","unstructured":"[26] Thomas Heyman, Koen Yskout, Riccardo Scandariato, and Wouter Joosen. An analysis of the security patterns landscape. In Third International Workshop on Software Engineering for Secure Systems (SESS\u201907: ICSE Workshops 2007), pages 3\u20133. IEEE, 2007. 10.1109\/SESS.2007.4.10.1109\/SESS.2007.4","DOI":"10.1109\/SESS.2007.4"},{"key":"2022060207212690786_j_popets-2022-0038_ref_027","unstructured":"[27] Jaap-Henk Hoepman. Privacy Design Strategies. In Nora Cuppens-Boulahia, Fr\u00e9d\u00e9ric Cuppens, Sushil Jajodia, Anas Abou El Kalam, and Thierry Sans, editors, ICT Systems Security and Privacy Protection, pages 446\u2013459, Berlin, Heidelberg, 2014. Springer Berlin Heidelberg. 978-3-642-55415-5_38."},{"key":"2022060207212690786_j_popets-2022-0038_ref_028","unstructured":"[28] Jaap-Henk Hoepman. Privacy Design Strategies (The Little Blue Book). Radboud University, 2019. URL https:\/\/cs.ru.nl\/~jhh\/publications\/pds-booklet.pdf."},{"key":"2022060207212690786_j_popets-2022-0038_ref_029","doi-asserted-by":"crossref","unstructured":"[29] Bert-Jaap Koops and Ronald Leenes. Privacy regulation cannot be hardcoded. a critical comment on the \u2018privacy by design\u2019 provision in data-protection law. International Review of Law, Computers & Technology, 28(2):159\u2013171, 2014. 10.1080\/13600869.2013.801589.10.1080\/13600869.2013.801589","DOI":"10.1080\/13600869.2013.801589"},{"key":"2022060207212690786_j_popets-2022-0038_ref_030","unstructured":"[30] Blagovesta Kostova, Seda G\u00fcrses, and Carmela Troncoso. Privacy Engineering Meets Software Engineering. On the Challenges of Engineering Privacy By Design, 2020. URL https:\/\/arxiv.org\/abs\/2007.08613."},{"key":"2022060207212690786_j_popets-2022-0038_ref_031","doi-asserted-by":"crossref","unstructured":"[31] Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. Chapter 8 - Interviews and focus groups. In Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser, editors, Research Methods in Human Computer Interaction, pages 187\u2013228. Morgan Kaufmann, Boston, second edition edition, 2017. 10.1016\/B978-0-12-805390-4.00008-X.10.1016\/B978-0-12-805390-4.00008-X","DOI":"10.1016\/B978-0-12-805390-4.00008-X"},{"key":"2022060207212690786_j_popets-2022-0038_ref_032","doi-asserted-by":"crossref","unstructured":"[32] Tianshi Li, Yuvraj Agarwal, and Jason I. Hong. Coconut: An IDE Plugin for Developing Privacy-Friendly Apps. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(4), December 2018. 10.1145\/3287056.10.1145\/3287056","DOI":"10.1145\/3287056"},{"key":"2022060207212690786_j_popets-2022-0038_ref_033","doi-asserted-by":"crossref","unstructured":"[33] Tianshi Li, Elizabeth Louie, Laura Dabbish, and Jason I. Hong. How Developers Talk About Personal Data and What It Means for User Privacy: A Case Study of a Developer Forum on Reddit. Proc. ACM Hum.-Comput. Interact., 4 (CSCW3), January 2021. 10.1145\/3432919.10.1145\/3432919","DOI":"10.1145\/3432919"},{"key":"2022060207212690786_j_popets-2022-0038_ref_034","doi-asserted-by":"crossref","unstructured":"[34] Tianshi Li, Elijah B. Neundorfer, Yuvraj Agarwal, and Jason I. Hong. Honeysuckle: Annotation-guided code generation of in-app privacy notices. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 5(3), September 2021. 10.1145\/3478097.10.1145\/3478097","DOI":"10.1145\/3478097"},{"key":"2022060207212690786_j_popets-2022-0038_ref_035","doi-asserted-by":"crossref","unstructured":"[35] Ilaria Liccardi, Monica Bulger, Hal Abelson, Daniel Weitzner, and Wendy Mackay. Can apps play by the COPPA Rules? In 2014 Twelfth Annual International Conference on Privacy, Security and Trust, pages 1\u20139. IEEE, 2014. 10.1109\/PST.2014.6890917.10.1109\/PST.2014.6890917","DOI":"10.1109\/PST.2014.6890917"},{"key":"2022060207212690786_j_popets-2022-0038_ref_036","unstructured":"[36] Jialiu Lin, Shahriyar Amini, Jason I. Hong, Norman Sadeh, Janne Lindqvist, and Joy Zhang. Expectation and Purpose: Understanding Users\u2019 Mental Models of Mobile App Privacy through Crowdsourcing. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, Ubi-Comp \u201912, page 501\u2013510, New York, NY, USA, 2012. ACM. 10.1145\/2370216.2370290."},{"key":"2022060207212690786_j_popets-2022-0038_ref_037","unstructured":"[37] Matomo. Google Analytics alternative that protects your data, 2021. URL https:\/\/matomo.org."},{"key":"2022060207212690786_j_popets-2022-0038_ref_038","doi-asserted-by":"crossref","unstructured":"[38] Celestin Matte, Nataliia Bielova, and Cristiana Santos. Do Cookie Banners Respect my Choice? : Measuring Legal Compliance of Banners from IAB Europe\u2019s Transparency and Consent Framework. In 2020 IEEE Symposium on Security and Privacy (SP), pages 791\u2013809. IEEE, 05 2020. 10.1109\/SP40000.2020.00076.10.1109\/SP40000.2020.00076","DOI":"10.1109\/SP40000.2020.00076"},{"key":"2022060207212690786_j_popets-2022-0038_ref_039","unstructured":"[39] Aleecia M McDonald and Lorrie Faith Cranor. The Cost of Reading Privacy Policies. I\/S: A Journal of Law and Policy for the Information Society (ISJLP), 4:543, 2008. URL https:\/\/heinonline.org\/HOL\/P?h=hein.journals\/isjlpsoc4&i=563."},{"key":"2022060207212690786_j_popets-2022-0038_ref_040","unstructured":"[40] Abraham H. Mhaidli, Yixin Zou, and Florian Schaub. \u201cWe Can\u2019t Live Without Them!\u201d App Developers\u2019Adoption of Ad Networks and Their Considerations of Consumer Risks. In Fifteenth Symposium on Usable Privacy and Security (SOUPS 2019), Santa Clara, CA, August 2019. USENIX Association. URL https:\/\/www.usenix.org\/conference\/soups2019\/presentation\/mhaidli."},{"key":"2022060207212690786_j_popets-2022-0038_ref_041","doi-asserted-by":"crossref","unstructured":"[41] Michael Naehrig, Kristin Lauter, and Vinod Vaikuntanathan. Can homomorphic encryption be practical? In Proceedings of the 3rd ACM Workshop on Cloud Computing Security Workshop, CCSW \u201911, page 113\u2013124, New York, NY, USA, 2011. ACM. 10.1145\/2046660.2046682.10.1145\/2046660.2046682","DOI":"10.1145\/2046660.2046682"},{"key":"2022060207212690786_j_popets-2022-0038_ref_042","unstructured":"[42] Trung Tin Nguyen, Michael Backes, Ninja Marnau, and Ben Stock. Share first, ask later (or never?) studying violations of gdpr\u2019s explicit consent in android apps. In 30th USENIX Security Symposium (USENIX Security 21), pages 3667\u20133684. USENIX Association, August 2021. URL https:\/\/www.usenix.org\/conference\/usenixsecurity21\/presentation\/nguyen."},{"key":"2022060207212690786_j_popets-2022-0038_ref_043","unstructured":"[43] State of California Department of Justice. California Consumer Privacy Act (CCPA), 2018. URL https:\/\/oag.ca.gov\/privacy\/ccpa."},{"key":"2022060207212690786_j_popets-2022-0038_ref_044","unstructured":"[44] Information Commissioner\u2019s Office. Data protection impact assessments, 2021. URL https:\/\/ico.org.uk\/for-organisations\/guide-to-data-protection\/guide-to-the-general-data-protection-regulation-gdpr\/accountability-and-governance\/data-protection-impact-assessments\/."},{"key":"2022060207212690786_j_popets-2022-0038_ref_045","unstructured":"[45] Stack Overflow. What should I do when someone answers my question?, 2021. URL https:\/\/stackoverflow.com\/help\/someone-answers."},{"key":"2022060207212690786_j_popets-2022-0038_ref_046","unstructured":"[46] The European parliament and the council of the European union. General Data Protection Regulation (GDPR), 2018. URL https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/PDF\/?uri=CELEX:32016R0679."},{"key":"2022060207212690786_j_popets-2022-0038_ref_047","unstructured":"[47] Chris Parnin, Christoph Treude, Lars Grammel, and Margaret-Anne Storey. Crowd documentation: Exploring the coverage and the dynamics of API discussions on Stack Overflow. Georgia Institute of Technology, Tech. Rep, 11, 2012. URL http:\/\/citeseerx.ist.psu.edu\/viewdoc\/summary?doi=10.1.1.371.6263."},{"key":"2022060207212690786_j_popets-2022-0038_ref_048","unstructured":"[48] Elissa M. Redmiles, Noel Warford, Amritha Jayanti, Aravind Koneru, Sean Kross, Miraida Morales, Rock Stevens, and Michelle L. Mazurek. A Comprehensive Quality Evaluation of Security and Privacy Advice on the Web. In 29th USENIX Security Symposium (USENIX Security 20), pages 89\u2013108. USENIX Association, August 2020. URL https:\/\/www.usenix.org\/conference\/usenixsecurity20\/presentation\/redmiles."},{"key":"2022060207212690786_j_popets-2022-0038_ref_049","doi-asserted-by":"crossref","unstructured":"[49] Irwin Reyes, Primal Wijesekera, Joel Reardon, Amit Elazari Bar On, Abbas Razaghpanah, Narseo Vallina-Rodriguez, and Serge Egelman. \u201cWon\u2019t Somebody Think of the Children?\u201d Examining COPPA Compliance at Scale. Proceedings on Privacy Enhancing Technologies, 2018(3): 63\u201383, 2018. 10.1515\/popets-2018-0021.10.1515\/popets-2018-0021","DOI":"10.1515\/popets-2018-0021"},{"key":"2022060207212690786_j_popets-2022-0038_ref_050","unstructured":"[50] Neil Salkind. Encyclopedia of Research Design. SAGE Publications, Inc, June 2020. 10.4135\/9781412961288."},{"key":"2022060207212690786_j_popets-2022-0038_ref_051","unstructured":"[51] Florian Schaub, Rebecca Balebako, Adam L. Durity, and Lorrie Faith Cranor. A Design Space for Effective Privacy Notices. In Proceedings of the Eleventh USENIX Conference on Usable Privacy and Security, SOUPS \u201915, page 1\u201317, USA, 2015. USENIX Association. URL https:\/\/www.usenix.org\/system\/files\/conference\/soups2015\/soups15-paper-schaub.pdf."},{"key":"2022060207212690786_j_popets-2022-0038_ref_052","doi-asserted-by":"crossref","unstructured":"[52] Awanthika Senarath and Nalin A. G. Arachchilage. Why Developers Cannot Embed Privacy into Software Systems?: An Empirical Investigation. In Proceedings of the 22Nd International Conference on Evaluation and Assessment in Software Engineering 2018, EASE\u201918, pages 211\u2013216, New York, NY, USA, 2018. ACM. 10.1145\/3210459.3210484.10.1145\/3210459.3210484","DOI":"10.1145\/3210459.3210484"},{"key":"2022060207212690786_j_popets-2022-0038_ref_053","doi-asserted-by":"crossref","unstructured":"[53] Swapneel Sheth, Gail Kaiser, and Walid Maalej. Us and Them: A Study of Privacy Requirements Across North America, Asia, and Europe. In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, pages 859\u2013870, New York, NY, USA, 2014. ACM. 10.1145\/2568225.2568244.10.1145\/2568225.2568244","DOI":"10.1145\/2568225.2568244"},{"key":"2022060207212690786_j_popets-2022-0038_ref_054","doi-asserted-by":"crossref","unstructured":"[54] Katie Shilton and Daniel Greene. Linking Platforms, Practices, and Developer Ethics: Levers for Privacy Discourse in Mobile Application Development. Journal of Business Ethics, 155(1):131\u2013146, March 2019. 10.1007\/s10551-017-3504-8.10.1007\/s10551-017-3504-8","DOI":"10.1007\/s10551-017-3504-8"},{"key":"2022060207212690786_j_popets-2022-0038_ref_055","doi-asserted-by":"crossref","unstructured":"[55] Katie Shilton, Donal Heidenblad, Adam Porter, Susan Winter, and Mary Kendig. Role-Playing Computer Ethics: Designing and Evaluating the Privacy by Design (PbD) Simulation. Science and Engineering Ethics, PP(PP), July 2020. 10.1007\/s11948-020-00250-0.10.1007\/s11948-020-00250-0775562832613325","DOI":"10.1007\/s11948-020-00250-0"},{"key":"2022060207212690786_j_popets-2022-0038_ref_056","doi-asserted-by":"crossref","unstructured":"[56] Laura Shipp and Jorge Blasco. How private is your period?: A systematic analysis of menstrual app privacy policies. Proceedings on Privacy Enhancing Technologies, 2020(4): 491\u2013510, October 2020. 10.2478\/popets-2020-0083.10.2478\/popets-2020-0083","DOI":"10.2478\/popets-2020-0083"},{"key":"2022060207212690786_j_popets-2022-0038_ref_057","doi-asserted-by":"crossref","unstructured":"[57] Mohammad Tahaei and Kami Vaniea. A Survey on Developer-Centred Security. In 2019 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), pages 129\u2013138. IEEE, June 2019. 10.1109\/EuroSPW. 2019.00021.10.1109\/EuroSPW.2019.00021","DOI":"10.1109\/EuroSPW.2019.00021"},{"key":"2022060207212690786_j_popets-2022-0038_ref_058","doi-asserted-by":"crossref","unstructured":"[58] Mohammad Tahaei and Kami Vaniea. \u201cDevelopers Are Responsible\u201d: What Ad Networks Tell Developers About Privacy. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems Extended Abstracts, CHI \u201921 Extended Abstracts, pages 1\u201312, New York, NY, USA, 2021. ACM. 10.1145\/3411763.3451805.10.1145\/3411763.3451805","DOI":"10.1145\/3411763.3451805"},{"key":"2022060207212690786_j_popets-2022-0038_ref_059","doi-asserted-by":"crossref","unstructured":"[59] Mohammad Tahaei, Kami Vaniea, and Naomi Saphra. Understanding Privacy-Related Questions on Stack Overflow. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI \u201920, page 1\u201314. ACM, 2020. 10.1145\/3313831.3376768.10.1145\/3313831.3376768","DOI":"10.1145\/3313831.3376768"},{"key":"2022060207212690786_j_popets-2022-0038_ref_060","doi-asserted-by":"crossref","unstructured":"[60] Mohammad Tahaei, Alisa Frik, and Kami Vaniea. Privacy Champions in Software Teams: Understanding Their Motivations, Strategies, and Challenges. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI \u201921, pages 1\u201315. ACM, 2021. 10.1145\/3411764.3445768.10.1145\/3411764.3445768","DOI":"10.1145\/3411764.3445768"},{"key":"2022060207212690786_j_popets-2022-0038_ref_061","unstructured":"[61] Mohammad Tahaei, Alisa Frik, and Kami Vaniea. Deciding on Personalized Ads: Nudging Developers About User Privacy. In Seventeenth Symposium on Usable Privacy and Security (SOUPS 2021), pages 573\u2013596. USENIX Association, August 2021. URL https:\/\/www.usenix.org\/conference\/soups2021\/presentation\/tahaei."},{"key":"2022060207212690786_j_popets-2022-0038_ref_062","unstructured":"[62] Christine Utz, Martin Degeling, Sascha Fahl, Florian Schaub, and Thorsten Holz. (Un) Informed Consent: Studying GDPR Consent Notices in the Field. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS \u201919, page 973\u2013990. ACM, 2019. 10.1145\/3319535.3354212."},{"key":"2022060207212690786_j_popets-2022-0038_ref_063","doi-asserted-by":"crossref","unstructured":"[63] Daniel Votipka, Mary Nicole Punzalan, Seth M Rabin, Yla Tausczik, and Michelle L Mazurek. An Investigation of Online Reverse Engineering Community Discussions in the Context of Ghidra. In IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 2021.10.1109\/EuroSP51992.2021.00012","DOI":"10.1109\/EuroSP51992.2021.00012"},{"key":"2022060207212690786_j_popets-2022-0038_ref_064","unstructured":"[64] Richmond Y. Wong and Deirdre K. Mulligan. Bringing Design to the Privacy Table: Broadening \u201cDesign\u201d in \u201cPrivacy by Design\u201d Through the Lens of HCI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI \u201919, pages 262:1\u2013262:17. ACM, 2019. 10.1145\/3290605.3300492."},{"key":"2022060207212690786_j_popets-2022-0038_ref_065","doi-asserted-by":"crossref","unstructured":"[65] Nahathai Wongpakaran, Tinakon Wongpakaran, Danny Wedding, and Kilem L. Gwet. A comparison of Cohen\u2019s Kappa and Gwet\u2019s AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples. BMC Medical Research Methodology, 13(1):61, April 2013. 10.1186\/1471-2288-13-61.10.1186\/1471-2288-13-61364386923627889","DOI":"10.1186\/1471-2288-13-61"},{"key":"2022060207212690786_j_popets-2022-0038_ref_066","doi-asserted-by":"crossref","unstructured":"[66] Xin-Li Yang, David Lo, Xin Xia, Zhi-Yuan Wan, and Jian-Ling Sun. What Security Questions Do Developers Ask? A Large-Scale Study of Stack Overflow Posts. Journal of Computer Science and Technology, 31(5):910\u2013924, September 2016. 10.1007\/s11390-016-1672-0.10.1007\/s11390-016-1672-0","DOI":"10.1007\/s11390-016-1672-0"},{"key":"2022060207212690786_j_popets-2022-0038_ref_067","doi-asserted-by":"crossref","unstructured":"[67] Slavica Zec, Nicola Soriani, Rosanna Comoretto, and Ileana Baldi. High Agreement and High Prevalence: The Paradox of Cohen\u2019s Kappa. The open nursing journal, 11:211\u2013218, October 2017. 10.2174\/1874434601711010211.10.2174\/1874434601711010211571264029238424","DOI":"10.2174\/1874434601711010211"},{"key":"2022060207212690786_j_popets-2022-0038_ref_068","unstructured":"[68] Eric Zeng and Franziska Roesner. Understanding and improving security and privacy in multi-user smart homes: a design exploration and in-home user study. In 28th USENIX Security Symposium (USENIX Security 19), pages 159\u2013176, 2019."},{"key":"2022060207212690786_j_popets-2022-0038_ref_069","doi-asserted-by":"crossref","unstructured":"[69] Sebastian Zimmeck, Peter Story, Daniel Smullen, Abhilasha Ravichander, Ziqi Wang, Joel Reidenberg, N. Cameron Russell, and Norman Sadeh. MAPS: Scaling Privacy Compliance Analysis to a Million Apps. Proceedings on Privacy Enhancing Technologies, 2019(3):66\u201386, 2019. 10.2478\/popets-2019-0037.","DOI":"10.2478\/popets-2019-0037"}],"container-title":["Proceedings on Privacy Enhancing Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.sciendo.com\/pdf\/10.2478\/popets-2022-0038","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,20]],"date-time":"2022-07-20T16:32:01Z","timestamp":1658334721000},"score":1,"resource":{"primary":{"URL":"https:\/\/petsymposium.org\/popets\/2022\/popets-2022-0038.php"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,3]]},"references-count":69,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,3,3]]},"published-print":{"date-parts":[[2022,4,1]]}},"alternative-id":["10.2478\/popets-2022-0038"],"URL":"https:\/\/doi.org\/10.2478\/popets-2022-0038","relation":{},"ISSN":["2299-0984"],"issn-type":[{"value":"2299-0984","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,3]]}}}