{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:05:14Z","timestamp":1760058314284,"version":"build-2065373602"},"reference-count":40,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,3,30]],"date-time":"2025-03-30T00:00:00Z","timestamp":1743292800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Union","award":["101070222","ANR-22PESN-0006"],"award-info":[{"award-number":["101070222","ANR-22PESN-0006"]}]},{"name":"CYBAILE industrial chair","award":["101070222","ANR-22PESN-0006"],"award-info":[{"award-number":["101070222","ANR-22PESN-0006"]}]},{"name":"Agence Nationale de la Recherche","award":["101070222","ANR-22PESN-0006"],"award-info":[{"award-number":["101070222","ANR-22PESN-0006"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Due to their considerable costs, deep neural networks (DNNs) are valuable assets that need to be protected in terms of intellectual property (IP). From this statement, DNN watermarking gains significant interest since it allows DNN owners to prove their ownership. Various methods that embed ownership information in the model behavior have been proposed. They need to fill several requirements, among them the security, which represents an attacker\u2019s difficulty in breaking the watermarking scheme. There is also the robustness requirement, which quantifies the resistance against watermark removal techniques. The problem is that the proposed methods generally fail to meet these necessary standards. This paper presents RoSe-Mix, a robust and secure deep neural network watermarking technique designed for black-box settings. It addresses limitations in existing DNN watermarking approaches by integrating key features from two established methods: RoSe, which uses cryptographic hashing to ensure security, and Mixer, which employs image Mixup to enhance robustness. Experimental results demonstrate that RoSe-Mix achieves security across various architectures and datasets with a robustness to removal attacks exceeding 99%.<\/jats:p>","DOI":"10.3390\/make7020032","type":"journal-article","created":{"date-parts":[[2025,4,1]],"date-time":"2025-04-01T10:59:59Z","timestamp":1743505199000},"page":"32","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["RoSe-Mix: Robust and Secure Deep Neural Network Watermarking in Black-Box Settings via Image Mixup"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-5082-0342","authenticated-orcid":false,"given":"Tamara","family":"El Hajjar","sequence":"first","affiliation":[{"name":"IMT Atlantique, Inserm UMR 1101, 29200 Brest, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-8025-5587","authenticated-orcid":false,"given":"Mohammed","family":"Lansari","sequence":"additional","affiliation":[{"name":"IMT Atlantique, Inserm UMR 1101, 29200 Brest, France"},{"name":"CortAIx Labs, Thales, 91120 Palaiseau, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1131-4115","authenticated-orcid":false,"given":"Reda","family":"Bellafqira","sequence":"additional","affiliation":[{"name":"IMT Atlantique, Inserm UMR 1101, 29200 Brest, France"},{"name":"National Institute of Health and Medical Research (Inserm), UMR 1101 Latim, 29238 Brest, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5643-0224","authenticated-orcid":false,"given":"Gouenou","family":"Coatrieux","sequence":"additional","affiliation":[{"name":"IMT Atlantique, Inserm UMR 1101, 29200 Brest, France"},{"name":"National Institute of Health and Medical Research (Inserm), UMR 1101 Latim, 29238 Brest, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katarzyna","family":"Kapusta","sequence":"additional","affiliation":[{"name":"CortAIx Labs, Thales, 91120 Palaiseau, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kassem","family":"Kallas","sequence":"additional","affiliation":[{"name":"National Institute of Health and Medical Research (Inserm), UMR 1101 Latim, 29238 Brest, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Azzouzi, M.E., Coatrieux, G., Bellafqira, R., Delamarre, D., Riou, C., Oubenali, N., Cabon, S., Cuggia, M., and Bouzill\u00e9, G. (2024). Automatic de-identification of French electronic health records: A cost-effective approach exploiting distant supervision and deep learning models. BMC Med Informat. Decis. Mak., 24.","DOI":"10.1186\/s12911-024-02422-5"},{"key":"ref_2","first-page":"1","article-title":"Deep learning for time series classification and extrinsic regression: A current survey","volume":"56","author":"Miller","year":"2024","journal-title":"ACM Comput. Surv."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"48","DOI":"10.58496\/BJML\/2024\/004","article-title":"A Short Review on Supervised Machine Learning and Deep Learning Techniques in Computer Vision","volume":"2024","author":"Nafea","year":"2024","journal-title":"Babylon. J. Mach. Learn."},{"key":"ref_4","unstructured":"Buchholz, K. (2025, March 03). The Extreme Cost of Training AI Models. Available online: https:\/\/www.statista.com\/chart\/33114\/estimated-cost-of-training-selected-ai-models\/."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Uchida, Y., Nagai, Y., Sakazawa, S., and Satoh, S. (2017, January 6\u20139). Embedding watermarks into deep neural networks. Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, Bucharest, Romania.","DOI":"10.1145\/3078971.3078974"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sun, Y., Liu, L., Yu, N., Liu, Y., Tian, Q., and Guo, D. (2025, March 25). Deep Watermarking for Deep Intellectual Property Protection: A Comprehensive Survey. Available online: https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=4697020.","DOI":"10.2139\/ssrn.4697020"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1382","DOI":"10.3390\/make5040070","article-title":"When federated learning meets watermarking: A comprehensive overview of techniques for intellectual property protection","volume":"5","author":"Lansari","year":"2023","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Darvish Rouhani, B., Chen, H., and Koushanfar, F. (2019, January 13\u201317). Deepsigns: An end-to-end watermarking framework for ownership protection of deep neural networks. Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, Providence, RI, USA.","DOI":"10.1145\/3297858.3304051"},{"key":"ref_9","unstructured":"Fan, L., Ng, K.W., and Chan, C.S. (2019, January 8\u201314). Rethinking deep neural network ownership verification: Embedding passports to defeat ambiguity attacks. Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wang, T., and Kerschbaum, F. (2021, January 19\u201323). Riga: Covert and robust white-box watermarking of deep neural networks. Proceedings of the Web Conference 2021, Ljubljana, Slovenia.","DOI":"10.1145\/3442381.3450000"},{"key":"ref_11","unstructured":"Bellafqira, R., and Coatrieux, G. (2022). Diction: Dynamic robust white box watermarking scheme. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"5214","DOI":"10.1109\/TDSC.2023.3242737","article-title":"A robustness-assured white-box watermark in neural networks","volume":"20","author":"Lv","year":"2023","journal-title":"IEEE Trans. Dependable Secur. Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"103830","DOI":"10.1016\/j.csi.2023.103830","article-title":"When deep learning meets watermarking: A survey of application, attacks and defenses","volume":"89","author":"Chen","year":"2024","journal-title":"Comput. Stand. Interfaces"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Lansari, M., Bellafqira, R., Kapusta, K., Kallas, K., Thouvenot, V., Bettan, O., and Coatrieux, G. (2024). FedCrypt: A Dynamic White-Box Watermarking Scheme for Homomorphic Federated Learning. TechRxiv.","DOI":"10.36227\/techrxiv.172114666.63343276\/v1"},{"key":"ref_15","unstructured":"Adi, Y., Baum, C., Cisse, M., Pinkas, B., and Keshet, J. (2018, January 15\u201317). Turning your weakness into a strength: Watermarking deep neural networks by backdooring. Proceedings of the 27th USENIX Security Symposium (USENIX Security 18), Baltimore, MD, USA."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Guo, J., and Potkonjak, M. (2018, January 5\u20138). Watermarking deep neural networks for embedded systems. Proceedings of the 2018 IEEE\/ACM International Conference on Computer-Aided Design (ICCAD), San Diego, CA, USA.","DOI":"10.1145\/3240765.3240862"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhang, J., Gu, Z., Jang, J., Wu, H., Stoecklin, M.P., Huang, H., and Molloy, I. (2018, January 4\u20138). Protecting intellectual property of deep neural networks with watermarking. Proceedings of the 2018 on Asia Conference on Computer and Communications Security, Incheon, Republic of Korea.","DOI":"10.1145\/3196494.3196550"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"9233","DOI":"10.1007\/s00521-019-04434-z","article-title":"Adversarial frontier stitching for remote neural network watermarking","volume":"32","author":"Perez","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Yadollahi, M.M., Shoeleh, F., Dadkhah, S., and Ghorbani, A.A. (2021, January 25\u201328). Robust black-box watermarking for deep neural network using inverse document frequency. Proceedings of the 2021 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC\/PiCom\/CBDCom\/CyberSciTech), Virtual.","DOI":"10.1109\/DASC-PICom-CBDCom-CyberSciTech52372.2021.00100"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wang, Y., and Wu, H. (2022). Protecting the intellectual property of speaker recognition model by black-box watermarking in the frequency domain. Symmetry, 14.","DOI":"10.3390\/sym14030619"},{"key":"ref_21","unstructured":"Gloaguen, T., Jovanovi\u0107, N., Staab, R., and Vechev, M. (2024). Black-box detection of language model watermarks. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Leroux, S., Vanassche, S., and Simoens, P. (2024, January 16\u201322). Multi-bit Black-box Watermarking of Deep Neural Networks in Embedded Applications. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPRW63382.2024.00217"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Kallas, K., and Furon, T. (2022, January 12\u201316). Rose: A robust and secure dnn watermarking. Proceedings of the 2022 IEEE International Workshop on Information Forensics and Security (WIFS), Online.","DOI":"10.1109\/WIFS55849.2022.9975300"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhang, H. (2017). mixup: Beyond empirical risk minimization. arXiv.","DOI":"10.1007\/978-1-4899-7687-1_79"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kallas, K., and Furon, T. (2023, January 4\u201310). Mixer: Dnn watermarking using image mixup. Proceedings of the ICASSP 2023\u20142023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece.","DOI":"10.1109\/ICASSP49357.2023.10095332"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Boenisch, F. (2021). A systematic review on model watermarking for neural networks. Front. Big Data, 4.","DOI":"10.3389\/fdata.2021.729663"},{"key":"ref_27","unstructured":"Oh, G., Kim, S., Cho, W., Lee, S., Chung, J., Song, D., and Yu, Y. (2025). SEAL: Entangled White-box Watermarks on Low-Rank Adaptation. arXiv."},{"key":"ref_28","unstructured":"Downer, J., Wang, R., and Wang, B. (2025). Watermarking Graph Neural Networks via Explanations for Ownership Protection. arXiv."},{"key":"ref_29","unstructured":"Krizhevsky, A., and Hinton, G. (2025, March 25). Learning Multiple Layers of Features from Tiny Images. Available online: https:\/\/www.cs.utoronto.ca\/~kriz\/learning-features-2009-TR.pdf."},{"key":"ref_30","unstructured":"Liang, J., and Wang, R. (2023). Fedcip: Federated client intellectual property protection with traitor tracking. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_32","unstructured":"LeCun, Y., and Cortes, C. (2025, March 25). MNIST Handwritten Digit Database. Available online: https:\/\/www.semanticscholar.org\/paper\/The-mnist-database-of-handwritten-digits-LeCun-Cortes\/dc52d1ede1b90bf9d296bc5b34c9310b7eaa99a2."},{"key":"ref_33","unstructured":"Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms. arXiv."},{"key":"ref_34","unstructured":"Simonyan, K. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_36","unstructured":"Stallkamp, J., Schlipsing, M., Salmen, J., and Igel, C. (August, January 31). The German Traffic Sign Recognition Benchmark: A multi-class classification competition. Proceedings of the IEEE International Joint Conference on Neural Networks, San Jose, CA, USA."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Szyller, S., Atli, B.G., Marchal, S., and Asokan, N. (2021, January 20\u201324). Dawn: Dynamic adversarial watermarking of neural networks. Proceedings of the 29th ACM International Conference on Multimedia, Virtual.","DOI":"10.1145\/3474085.3475591"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Pascal, L., Michiardi, P., Bost, X., Huet, B., and Zuluaga, M.A. (2021, January 2\u20139). Maximum roaming multi-task learning. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.","DOI":"10.1609\/aaai.v35i10.17125"},{"key":"ref_40","unstructured":"Natarajan, N., Dhillon, I.S., Ravikumar, P.K., and Tewari, A. (2013, January 5\u201310). Learning with noisy labels. Proceedings of the 27th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/32\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:06:12Z","timestamp":1760029572000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/32"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,30]]},"references-count":40,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["make7020032"],"URL":"https:\/\/doi.org\/10.3390\/make7020032","relation":{},"ISSN":["2504-4990"],"issn-type":[{"type":"electronic","value":"2504-4990"}],"subject":[],"published":{"date-parts":[[2025,3,30]]}}}