{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T11:39:44Z","timestamp":1784288384913,"version":"3.55.0"},"reference-count":45,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2022,11,19]],"date-time":"2022-11-19T00:00:00Z","timestamp":1668816000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Mobile app developers are often obliged by regulatory frameworks to provide a privacy policy in natural comprehensible language to describe their apps\u2019 privacy practices. However, prior research has revealed that: (1) not all app developers offer links to their privacy policies; and (2) even if they do offer such access, it is difficult to determine if it is a valid link to a (valid) policy. While many prior studies looked at this issue in Google Play Store, Apple App Store, and particularly the iOS store, is much less clear. In this paper, we conduct the first and the largest study to investigate the previous issues in the iOS app store ecosystem. First, we introduce an App Privacy Policy Extractor (APPE), a system that embraces and analyses the metadata of over two million apps to give insightful information about the distribution of the supposed privacy policies, and the content of the provided privacy policy links, store-wide. The result shows that only 58.5% of apps provide links to purported privacy policies, while 39.3% do not provide policy links at all. Our investigation of the provided links shows that only 38.4% of those links were directed to actual privacy policies, while 61.6% failed to lead to a privacy policy. Further, for research purposes we introduce the App Privacy Policy Corpus (APPC-451K); the largest app privacy policy corpus consisting of data relating to more than 451K verified privacy policies.<\/jats:p>","DOI":"10.3390\/s22228964","type":"journal-article","created":{"date-parts":[[2022,11,21]],"date-time":"2022-11-21T04:39:59Z","timestamp":1669005599000},"page":"8964","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Do the Right Thing: A Privacy Policy Adherence Analysis of over Two Million Apps in Apple iOS App Store"],"prefix":"10.3390","volume":"22","author":[{"given":"Hamad","family":"Alamri","sequence":"first","affiliation":[{"name":"Warwick Manufacturing Group, The University of Warwick, Gibbet Hill Road, Coventry CV4 7AL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4715-212X","authenticated-orcid":false,"given":"Carsten","family":"Maple","sequence":"additional","affiliation":[{"name":"Warwick Manufacturing Group, The University of Warwick, Gibbet Hill Road, Coventry CV4 7AL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saad","family":"Mohamad","sequence":"additional","affiliation":[{"name":"School of Cellular & Molecular Medicine, University of Bristol, Bristol BS8 1TD, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gregory","family":"Epiphaniou","sequence":"additional","affiliation":[{"name":"Warwick Manufacturing Group, The University of Warwick, Gibbet Hill Road, Coventry CV4 7AL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2619091","article-title":"Taintdroid: An information-flow tracking system for realtime privacy monitoring on smartphones","volume":"32","author":"Enck","year":"2014","journal-title":"ACM Trans. Comput. Syst. (TOCS)"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Agarwal, Y., and Hall, M. (2013, January 25\u201328). ProtectMyPrivacy: Detecting and mitigating privacy leaks on iOS devices using crowdsourcing. Proceedings of the 11th Annual International Conference on Mobile Systems, Applications, and Services, Taipei, Taiwan.","DOI":"10.1145\/2462456.2464460"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Almuhimedi, H., Schaub, F., Sadeh, N., Adjerid, I., Acquisti, A., Gluck, J., Cranor, L.F., and Agarwal, Y. (2015, January 18\u201323). Your location has been shared 5398 times! A field study on mobile app privacy nudging. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Republic of Korea.","DOI":"10.1145\/2702123.2702210"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Balebako, R., Jung, J., Lu, W., Cranor, L.F., and Nguyen, C. (2013, January 24\u201326). \u201cLittle brothers watching you\u201d raising awareness of data leaks on smartphones. Proceedings of the Ninth Symposium on Usable Privacy and Security, Newcastle, UK.","DOI":"10.1145\/2501604.2501616"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/MS.2014.50","article-title":"What do mobile app users complain about?","volume":"32","author":"Khalid","year":"2014","journal-title":"IEEE Softw."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.1007\/s10664-015-9375-7","article-title":"Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews","volume":"21","author":"McIlroy","year":"2016","journal-title":"Empir. Softw. Eng."},{"key":"ref_7","unstructured":"(2022, January 17). General Data Protection Regulation. Available online: https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/HTML\/?uri=CELEX:32016R0679."},{"key":"ref_8","unstructured":"Pitofsky, R., Azcuenaga, M., Anthony, S., Thompson, M., and Swindle, O. (1998). Privacy Online: A Report to Congress, Federal Trade Commission."},{"key":"ref_9","unstructured":"McIlroy, S., Ali, N., Khalid, H., and Hassan, A.E. (2015). Mobile Privacy Disclosures: Building Trust through Transparency: A Federal Trade Commission Staff Report, Federal Trade Commission."},{"key":"ref_10","unstructured":"(2022, January 10). California Consumer Privacy Act of 2018, CAL. CIV. CODE \u00a7 1798.100. Available online: https:\/\/www.spirion.com\/wp-content\/uploads\/2020\/07\/Spirion_CCPA_v3.pdf."},{"key":"ref_11","unstructured":"(2022, January 15). App Store Review Guidelines\u2014Apple Developer. Available online: https:\/\/developer.apple.com\/app-store\/review\/guidelines\/#legal."},{"key":"ref_12","unstructured":"(2022, January 15). Prepare Your App for Review\u2014Play Console Help. Available online: https:\/\/support.google.com\/googleplay\/android-developer\/answer\/9815348?hl=en."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"e28","DOI":"10.1136\/amiajnl-2013-002605","article-title":"Availability and quality of mobile health app privacy policies","volume":"22","author":"Sunyaev","year":"2015","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"100243","DOI":"10.1016\/j.invent.2019.100243","article-title":"Availability, readability, and content of privacy policies and terms of agreements of mental health apps","volume":"17","author":"Robillard","year":"2019","journal-title":"Internet Interv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1051","DOI":"10.1001\/jama.2015.19426","article-title":"Privacy policies of android diabetes apps and sharing of health information","volume":"315","author":"Blenner","year":"2016","journal-title":"JAMA"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.invent.2018.12.001","article-title":"Reviewing the data security and privacy policies of mobile apps for depression","volume":"15","author":"Neary","year":"2019","journal-title":"Internet Interv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e4581","DOI":"10.2196\/jmir.4581","article-title":"Mobile apps for bipolar disorder: A systematic review of features and content quality","volume":"17","author":"Nicholas","year":"2015","journal-title":"J. Med. Internet Res."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"e3672","DOI":"10.2196\/mhealth.3672","article-title":"Exploring the far side of mobile health: Information security and privacy of mobile health apps on iOS and Android","volume":"3","author":"Dehling","year":"2015","journal-title":"JMIR mHealth uHealth"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1007\/s11747-016-0495-4","article-title":"The role of data privacy in marketing","volume":"45","author":"Martin","year":"2017","journal-title":"J. Acad. Mark. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1016\/j.ijlp.2019.04.002","article-title":"How private is your mental health app data? An empirical study of mental health app privacy policies and practices","volume":"64","author":"Parker","year":"2019","journal-title":"Int. J. Law Psychiatry"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e9871","DOI":"10.2196\/mhealth.9871","article-title":"The complexity of mental health app privacy policies: A potential barrier to privacy","volume":"6","author":"Powell","year":"2018","journal-title":"JMIR mHealth uHealth"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e35195","DOI":"10.2196\/35195","article-title":"Data management and privacy policy of COVID-19 contact-tracing apps: Systematic review and content analysis","volume":"10","author":"Bardus","year":"2022","journal-title":"JMIR mHealth uHealth"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"De, S.J., and Shukla, R. (2021). An analysis of privacy policies of public COVID-19 apps: Evidence from India. J. Public Aff., e2801.","DOI":"10.1002\/pa.2801"},{"key":"ref_24","unstructured":"Bowers, J., Reaves, B., Sherman, I.N., Traynor, P., and Butler, K. (2017, January 12\u201314). Regulators, mount up! analysis of privacy policies for mobile money services. Proceedings of the Thirteenth Symposium on Usable Privacy and Security (SOUPS 2017), Santa Clara, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"e7626","DOI":"10.2196\/mhealth.7626","article-title":"Privacy policies for apps targeted toward youth: Descriptive analysis of readability","volume":"6","author":"Das","year":"2018","journal-title":"JMIR mHealth uHealth"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zimmeck, S., Wang, Z., Zou, L., Iyengar, R., Liu, B., Schaub, F., Wilson, S., Sadeh, N., Bellovin, S., and Reidenberg, J. (2016, January 12\u201317). Automated analysis of privacy requirements for mobile apps. Proceedings of the 2016 AAAI Fall Symposium Series, Phoenix, AZ, USA.","DOI":"10.14722\/ndss.2017.23034"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Story, P., Zimmeck, S., and Sadeh, N. (2018, January 13\u201314). Which apps have privacy policies?. Proceedings of the Annual Privacy Forum, Barcelona, Spain.","DOI":"10.1007\/978-3-030-02547-2_1"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Costante, E., Sun, Y., Petkovi\u0107, M., and Den Hartog, J. (2012, January 15). A machine learning solution to assess privacy policy completeness: (Short paper). Proceedings of the 2012 ACM Workshop on Privacy in the Electronic Society, Raleigh, NC, USA.","DOI":"10.1145\/2381966.2381979"},{"key":"ref_29","unstructured":"Zimmeck, S., and Bellovin, S.M. (2014, January 20\u201322). Privee: An architecture for automatically analyzing web privacy policies. Proceedings of the 23rd {USENIX} Security Symposium ({USENIX} Security 14), San Diego, CA, USA."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3127519","article-title":"Privacycheck: Automatic summarization of privacy policies using data mining","volume":"18","author":"Zaeem","year":"2018","journal-title":"ACM Trans. Internet Technol. (TOIT)"},{"key":"ref_31","unstructured":"Harkous, H., Fawaz, K., Lebret, R., Schaub, F., Shin, K.G., and Aberer, K. (2018, January 15\u201317). Polisis: Automated analysis and presentation of privacy policies using deep learning. Proceedings of the 27th {USENIX} security symposium ({USENIX} security 18), Baltimore, MD, USA."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Liu, S., Zhao, B., Guo, R., Meng, G., Zhang, F., and Zhang, M. (2021, January 19\u201323). Have You been Properly Notified? Automatic Compliance Analysis of Privacy Policy Text with GDPR Article 13. Proceedings of the Web Conference 2021, Ljubljana, Slovenia.","DOI":"10.1145\/3442381.3450022"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"89248","DOI":"10.1109\/ACCESS.2022.3199882","article-title":"PermPress: Machine Learning-Based Pipeline to Evaluate Permissions in App Privacy Policies","volume":"10","author":"Rahman","year":"2022","journal-title":"IEEE Access"},{"key":"ref_34","unstructured":"Sadeh, N., Acquisti, R., Breaux, T.D., Cranor, L.F., Mcdonalda, A.M., Reidenbergb, J.R., Smith, N.A., Liu, F., Russellb, N.C., and Schaub, F. (2022, January 15). The usable privacy policy project: Combining crowdsourcing. In Machine Learning and Natural Language Processing to Semi-Automatically Answer Those Privacy Questions Users Care About, Available online: https:\/\/usableprivacy.org\/."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wilson, S., Schaub, F., Dara, A.A., Liu, F., Cherivirala, S., Leon, P.G., Andersen, M.S., Zimmeck, S., Sathyendra, K.M., and Russell, N.C. (2016, January 7\u201312). The creation and analysis of a website privacy policy corpus. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany.","DOI":"10.18653\/v1\/P16-1126"},{"key":"ref_36","unstructured":"Sathyendra, K.M., Wilson, S., Schaub, F., Zimmeck, S., and Sadeh, N. (2017, January 9\u201311). Identifying the provision of choices in privacy policy text. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark."},{"key":"ref_37","unstructured":"Liu, F., Ramanath, R., Sadeh, N., and Smith, N.A. (2014, January 23\u201329). A step towards usable privacy policy: Automatic alignment of privacy statements. Proceedings of the COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, Dublin, Ireland."},{"key":"ref_38","first-page":"66","article-title":"Maps: Scaling privacy compliance analysis to a million apps","volume":"2019","author":"Zimmeck","year":"2019","journal-title":"Proc. Priv. Enhancing Tech."},{"key":"ref_39","unstructured":"Lui, M., and Baldwin, T. (2012, January 10). langid. py: An off-the-shelf language identification tool. Proceedings of the ACL 2012 System Demonstrations, Jeju Island, Republic of Korea."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., and Manning, C.D. (2014, January 25\u201329). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1162"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., and Liu, C. (2018, January 8\u201310). A survey on deep transfer learning. Proceedings of the International Conference on Artificial Neural Networks, Shenzhen, China.","DOI":"10.1007\/978-3-030-01424-7_27"},{"key":"ref_42","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_43","unstructured":"Lin, Z., Feng, M., Santos, C.N.d., Yu, M., Xiang, B., Zhou, B., and Bengio, Y. (2017). A structured self-attentive sentence embedding. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014, January 25\u201329). Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1181"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Lai, S., Xu, L., Liu, K., and Zhao, J. (2015, January 25\u201330). Recurrent convolutional neural networks for text classification. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA.","DOI":"10.1609\/aaai.v29i1.9513"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/22\/8964\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:21:52Z","timestamp":1760145712000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/22\/8964"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,19]]},"references-count":45,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["s22228964"],"URL":"https:\/\/doi.org\/10.3390\/s22228964","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,19]]}}}