{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:18:45Z","timestamp":1760149125894,"version":"build-2065373602"},"reference-count":45,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,7,12]],"date-time":"2023-07-12T00:00:00Z","timestamp":1689120000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["62272024"],"award-info":[{"award-number":["62272024"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In traditional centralized Android malware classifiers based on machine learning, the training sample uploaded by users contains sensitive personal information, such as app usage and device security status, which will undermine personal privacy if used directly by the server. Federated-learning-based Android malware classifiers have attracted much attention due to their privacy-preserving and multi-party joint modeling. However, research shows that indirect privacy inferences from curious central servers threaten this framework. We propose a privacy risk evaluation framework, FedDroidMeter, based on normalized mutual information in response to user privacy requirements to measure the privacy risk in FL-based malware classifiers. It captures the essential cause of the disclosure of sensitive information in classifiers, independent of the attack model and capability. We performed numerical assessments using the Androzoo dataset, the baseline FL-based classifiers, the privacy-inferred attack model, and the baseline methodology of privacy evaluation. The experimental results show that FedDroidMeter can measure the privacy risks of the classifiers more effectively. Meanwhile, by comparing different models, FL, and privacy parameter settings, we proved that FedDroidMeter could compare the privacy risk between different use cases equally. Finally, we preliminarily study the law of privacy risk in classifiers. The experimental results emphasize the importance of providing a systematic privacy risk evaluation framework for FL-based malware classifiers and provide experience and a theoretical basis for studying targeted defense methods.<\/jats:p>","DOI":"10.3390\/e25071053","type":"journal-article","created":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T01:43:22Z","timestamp":1689212602000},"page":"1053","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["FedDroidMeter: A Privacy Risk Evaluator for FL-Based Android Malware Classification Systems"],"prefix":"10.3390","volume":"25","author":[{"given":"Changnan","family":"Jiang","sequence":"first","affiliation":[{"name":"Key Laboratory of Beijing Network Technology, Beihang University, Beijing 100191, China"}]},{"given":"Chunhe","family":"Xia","sequence":"additional","affiliation":[{"name":"Key Laboratory of Beijing Network Technology, Beihang University, Beijing 100191, China"},{"name":"Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, Guilin 541004, China"}]},{"given":"Zhuodong","family":"Liu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Beijing Network Technology, Beihang University, Beijing 100191, China"}]},{"given":"Tianbo","family":"Wang","sequence":"additional","affiliation":[{"name":"Shanghai Key Laboratory of Computer Software Evaluating and Testing, Shanghai 201112, China"},{"name":"School of Cyber Science and Technology, Beihang University, Beijing 100191, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,7,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3417978","article-title":"A Survey of Android Malware Detection with Deep Neural Models","volume":"53","author":"Qiu","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1007\/s42486-020-00041-3","article-title":"Demographics of mobile app usage: Long-term analysis of mobile app usage","volume":"3","author":"Tu","year":"2021","journal-title":"CCF Trans. Pervasive Comput. Interact."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3339474","article-title":"Federated machine learning: Concept and applications","volume":"10","author":"Yang","year":"2019","journal-title":"ACM Trans. Intell. Syst. Technol. (TIST)"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"G\u00e1lvez, R., Moonsamy, V., and Diaz, C. (2020). Less is More: A privacy-respecting Android malware classifier using federated learning. Proc. Priv. Enhancing Technol. arXiv.","DOI":"10.2478\/popets-2021-0062"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Melis, L., Song, C., De Cristofaro, E., and Shmatikov, V. (2019, January 19\u201323). Exploiting Unintended Feature Leakage in Collaborative Learning. Proceedings of the 2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA.","DOI":"10.1109\/SP.2019.00029"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Orekondy, T., Schiele, B., and Fritz, M. (2019, January 15\u201320). Knockoff nets: Stealing functionality of black-box models. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00509"},{"key":"ref_7","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). Advances in Neural Information Processing Systems, Curran Associates, Inc.. Available online: https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2019\/file\/60a6c4002cc7b29142def8871531281a-Paper.pdf."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Shokri, R., Stronati, M., Song, C., and Shmatikov, V. (2017, January 22\u201326). Membership Inference Attacks Against Machine Learning Models. Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA.","DOI":"10.1109\/SP.2017.41"},{"key":"ref_9","unstructured":"(2022, May 16). ICO Consultation on the Draft AI Auditing Framework Guidance for Organisations, 2020. Available online: https:\/\/ico.org.uk\/about-the-ico\/ico-and-stakeholder-consultations\/ico-consultation-on-the-draft-ai-auditing-framework-guidance-for-organisations\/."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1145\/3264948","article-title":"Your Apps Give You Away","volume":"2","author":"Tu","year":"2018","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1622","DOI":"10.1109\/COMST.2021.3075439","article-title":"Federated Learning for Internet of Things: A Comprehensive Survey","volume":"23","author":"Nguyen","year":"2021","journal-title":"IEEE Commun. Surv. Tutorials."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2031","DOI":"10.1109\/COMST.2020.2986024","article-title":"Federated learning in mobile edge networks: A comprehensive survey","volume":"22","author":"Lim","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"8442","DOI":"10.1109\/TII.2020.3043458","article-title":"Fed-IIoT: A Robust Federated Malware Detection Architecture in Industrial IoT","volume":"17","author":"Taheri","year":"2021","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_14","unstructured":"Singh, N., Kasyap, H., and Tripathy, S. (2020). PKDD\/ECML Workshops 2020, Springer."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Shukla, S., Manoj, P.S., Kolhe, G., and Rafatirad, S. (2021, January 5\u20139). On-device Malware Detection using Performance-Aware and Robust Collaborative Learning. Proceedings of the DAC 2021, San Francisco, CA, USA.","DOI":"10.1109\/DAC18074.2021.9586330"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Singh, A.K., and Goyal, N. (2022, January 4\u20138). Android Web Security Solution using Cross-device Federated Learning. Proceedings of the COMSNETS 2022, Bangalore, India.","DOI":"10.1109\/COMSNETS53615.2022.9668449"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"108693","DOI":"10.1016\/j.comnet.2021.108693","article-title":"Federated learning for malware detection in IoT devices","volume":"204","author":"Rey","year":"2022","journal-title":"Comput. Netw."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Salem, A., Zhang, Y., Humbert, M., Berrang, P., Fritz, M., and Backes, M. (2019, January 24\u201327). ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models. Proceedings of the NDSS, 2019, San Diego, CA, USA.","DOI":"10.14722\/ndss.2019.23119"},{"key":"ref_19","unstructured":"Leino, K., and Fredrikson, M. (2020). USENIX Security, 2020, USENIX."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Shafran, A., Peleg, S., and Hoshen, Y. (2021, January 10\u201317). Membership Inference Attacks are Easier on Difficult Problems. Proceedings of the ICCV 2021, Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.01455"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1504\/IJSN.2015.071829","article-title":"Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers","volume":"10","author":"Ateniese","year":"2015","journal-title":"Int. J. Secur. Netw."},{"key":"ref_22","unstructured":"Zhao, B., Mopuri, K.R., and Bilen, H. (2020). iDLG: Improved Deep Leakage from Gradients. arXiv."},{"key":"ref_23","unstructured":"Song, C., and Shmatikov, V. (2020, January 26\u201330). Overlearning Reveals Sensitive Attributes. Proceedings of the ICLR, 2020, Addis Ababa, Ethiopia."},{"key":"ref_24","unstructured":"Fredrikson, M., Lantz, E., Jha, S., Lin, S., Page, D., and Ristenpart, T. (2014, January 20\u201322). Privacy in Pharmacogenetics: An End-to-End Case Study of Personalized Warfarin Dosing. Proceedings of the USENIX Security, San Diego, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Fredrikson, M., Jha, S., and Ristenpart, T. (2015, January 12\u201316). Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures. Proceedings of the CCS, Denver, CO, USA.","DOI":"10.1145\/2810103.2813677"},{"key":"ref_26","unstructured":"Carlini, N., Liu, C., Erlingsson, \u00da., Kos, J., and Song, D. (2019, January 14\u201316). The Secret Sharer: Evaluating and Testing Unintended Memorizationin Neural Networks. Proceedings of the USENIX Security, Santa Clara, CA USA."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Nasr, M., Shokri, R., and Houmansadr, A. (2019, January 19\u201323). Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralizedand Federated Learning. Proceedings of the 2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA.","DOI":"10.1109\/SP.2019.00065"},{"key":"ref_28","unstructured":"Tram\u00e8r, F., Zhang, F., Juels, A., Reiter, M.K., and Ristenpart, T. (2016, January 10\u201312). Stealing machine learning models via prediction {APIs}. Proceedings of the 25th USENIX Security Symposium (USENIX Security 16), Austin, TX, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Oh, S.J., Schiele, B., and Fritz, M. (May, January 30). Towards Reverse-Engineering Black-Box Neural Networks. Proceedings of the ICLR, 2018, Vancouver, BC, Canada.","DOI":"10.1007\/978-3-030-28954-6_7"},{"key":"ref_30","unstructured":"Zhang, W., Tople, S., and Ohrimenko, O. (2021, January 11\u201313). Leakage of Dataset Properties in Multi-Party Machine Learning. Proceedings of the USENIX Security Symposium 2021, Virtual."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sun, J., Li, A., Wang, B., Yang, H., Li, H., and Chen, Y. (2021, January 20\u201325). Soteria: Provable defense against privacy leakage in federated learning from representation perspective. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 2021, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00919"},{"key":"ref_32","unstructured":"Murakonda, S.K., and Shokri, R. (2022, May 16). ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning. In Workshop on Hot Topics in Privacy Enhancing Technologies (HotPETs), 2020. Available online: https:\/\/arxiv.org\/abs\/2007.09339."},{"key":"ref_33","unstructured":"Liu, Y., Wen, R., He, X., Salem, A., Zhang, Z., Backes, M., Fritz, M., and Zhang, Y. (2022, January 10\u201312). ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models. Proceedings of the USENIX Security Symposium 2022, Boston, MA, USA."},{"key":"ref_34","unstructured":"Duddu, V., Szyller, S., and Asokan, N. (2021). SHAPr: An Efficient and Versatile Membership Privacy Risk Metric for Machine Learning. arXiv."},{"key":"ref_35","unstructured":"Song, L., and Mittal, P. (2021, January 11\u201313). Systematic evaluation of privacy risks of machine learning models. Proceedings of the 30th {USENIX} Security Symposium ({USENIX}Security 21), Virtual."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Hannun, A., Guo, C., and van der Maaten, L. (2021). Measuring data leakage in machine-learning models with fisher information. arXiv.","DOI":"10.24963\/ijcai.2022\/736"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"3096","DOI":"10.1109\/TIFS.2021.3073804","article-title":"Quantifying Membership Privacy via Information Leakage","volume":"16","author":"Saeidian","year":"2021","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1109\/TIFS.2019.2903658","article-title":"Optimal Utility-Privacy Trade-off with Total Variation Distance as a Privacy Measure","volume":"15","author":"Rassouli","year":"2019","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_39","unstructured":"Yu, D., Kamath, G., Kulkarni, J., Yin, J., Liu, T.Y., and Zhang, H. (2022). Per-instance privacy accounting for differentially private stochastic gradient descent. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Bai, Y., Fan, M., Li, Y., and Xie, C. (2022, January 16\u201320). Privacy Risk Assessment of Training Data in Machine Learning. Proceedings of the ICC 2022, Seoul, Republic of Korea.","DOI":"10.1109\/ICC45855.2022.9839062"},{"key":"ref_41","first-page":"1","article-title":"Technical privacy metrics: A systematic survey","volume":"51","author":"Wagner","year":"2018","journal-title":"Comput. Sci."},{"key":"ref_42","first-page":"1","article-title":"An Intrusion Detection System Based on Normalized Mutual Information Antibodies Feature Selection and Adaptive Quantum Artificial Immune System","volume":"18","author":"Ling","year":"2022","journal-title":"Int. J. Semant. Web Inf. Syst."},{"key":"ref_43","unstructured":"Andrew, G., Thakkar, O., and McMahan, B. (2021, January 6\u201314). Differentially Private Learning with Adaptive Clipping. Proceedings of the NeurIPS 2021, Virtual."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Allix, K., Bissyand\u00e9, T.F., Klein, J., and Le Traon, Y. (2016, January 14\u201315). AndroZoo: Collecting millions of Android apps for the research community. Proceedings of the 13th International Conference on Mining Software Repositories, Austin, TX, USA.","DOI":"10.1145\/2901739.2903508"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"227","DOI":"10.2478\/popets-2022-0043","article-title":"User-Level Label Leakage from Gradients in Federated Learning","volume":"2022","author":"Wainakh","year":"2022","journal-title":"Proc. Priv. Enhancing Technol."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/7\/1053\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:11:47Z","timestamp":1760127107000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/7\/1053"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,12]]},"references-count":45,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,7]]}},"alternative-id":["e25071053"],"URL":"https:\/\/doi.org\/10.3390\/e25071053","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2023,7,12]]}}}