{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T23:37:24Z","timestamp":1781998644396,"version":"3.54.5"},"reference-count":30,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2020,12,6]],"date-time":"2020-12-06T00:00:00Z","timestamp":1607212800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>As training Deep Neural Networks (DNNs) becomes more expensive, the interest in protecting the ownership of the models with watermarking techniques increases. Uchida et al. proposed a digital watermarking algorithm that embeds the secret message into the model coefficients. However, despite its appeal, in this paper, we show that its efficacy can be compromised by the optimization algorithm being used. In particular, we found through a theoretical analysis that, as opposed to Stochastic Gradient Descent (SGD), the update direction given by Adam optimization strongly depends on the sign of a combination of columns of the projection matrix used for watermarking. Consequently, as observed in the empirical results, this makes the coefficients move in unison giving rise to heavily spiked weight distributions that can be easily detected by adversaries. As a way to solve this problem, we propose a new method called Block-Orthonormal Projections (BOP) that allows one to combine watermarking with Adam optimization with a minor impact on the detectability of the watermark and an increased robustness.<\/jats:p>","DOI":"10.3390\/e22121379","type":"journal-article","created":{"date-parts":[[2020,12,6]],"date-time":"2020-12-06T22:27:12Z","timestamp":1607293632000},"page":"1379","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Adam and the Ants: On the Influence of the Optimization Algorithm on the Detectability of DNN Watermarks"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6914-9757","authenticated-orcid":false,"given":"Betty","family":"Corti\u00f1as-Lorenzo","sequence":"first","affiliation":[{"name":"Atlanttic Research Center, University of Vigo, 36310 Vigo, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0568-1373","authenticated-orcid":false,"given":"Fernando","family":"P\u00e9rez-Gonz\u00e1lez","sequence":"additional","affiliation":[{"name":"Atlanttic Research Center, University of Vigo, 36310 Vigo, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7\u201313). Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV \u201915), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.123"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"19143","DOI":"10.1109\/ACCESS.2019.2896880","article-title":"Speech Recognition Using Deep Neural Networks: A Systematic Review","volume":"7","author":"Nassif","year":"2019","journal-title":"IEEE Access"},{"key":"ref_3","unstructured":"Le Merrer, E., P\u00e9rez, P., and Tr\u00e9dan, G. (2017). Adversarial Frontier Stitching for Remote Neural Network Watermarking. arXiv."},{"key":"ref_4","unstructured":"Adi, Y., Baum, C., Ciss\u00e9, M., Pinkas, B., and Keshet, J. (2018). Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Uchida, Y., Nagai, Y., Sakazawa, S., and Satoh, S. (2017, January 6\u20139). Embedding Watermarks into Deep Neural Networks. Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval (ICMR \u201917), Bucharest, Romania.","DOI":"10.1145\/3078971.3078974"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s13735-018-0147-1","article-title":"Digital Watermarking for Deep Neural Networks","volume":"7","author":"Nagai","year":"2018","journal-title":"Int. J. Multimed. Inf. Retr."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1673","DOI":"10.1109\/83.650120","article-title":"Secure Spread Spectrum Watermarking for Multimedia","volume":"6","author":"Cox","year":"1997","journal-title":"IEEE Trans. Image Process."},{"key":"ref_8","unstructured":"Saad, D. (1998). Online Algorithms and Stochastic Approximations. Online Learning and Neural Networks, Cambridge University Press."},{"key":"ref_9","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations (ICLR \u201915), San Diego, CA, USA."},{"key":"ref_10","unstructured":"Rouhani, B.D., Chen, H., and Koushanfar, F. (2018). DeepSigns: A Generic Watermarking Framework for IP Protection of Deep Learning Models. arXiv."},{"key":"ref_11","unstructured":"Balles, L., and Hennig, P. (2018, January 10\u201315). Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients. Proceedings of the 2018 International Conference on Machine Learning (ICML \u201918), Stockholm, Sweden."},{"key":"ref_12","unstructured":"Wilson, A.C., Roelofs, R., Stern, M., Srebro, N., and Recht, B. (2017, January 4\u20139). The Marginal Value of Adaptive Gradient Methods in Machine Learning. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS \u201917), Long Beach, CA, USA."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On Information and Sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4608","DOI":"10.1109\/TIP.2018.2839891","article-title":"FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising","volume":"27","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1186\/s42492-019-0016-7","article-title":"Brief Review of Image Denoising Techniques","volume":"2","author":"Fan","year":"2019","journal-title":"Vis. Comput. Ind. Biomed. Art"},{"key":"ref_16","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on Machine Learning (ICML \u201915), Lille, France."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3668","DOI":"10.1109\/TCYB.2019.2950779","article-title":"A Survey of Optimization Methods From a Machine Learning Perspective","volume":"50","author":"Sun","year":"2020","journal-title":"IEEE Trans. Cybern."},{"key":"ref_18","first-page":"2121","article-title":"Adaptive Subgradient Methods for Online Learning and Stochastic Optimization","volume":"12","author":"Duchi","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_19","unstructured":"Tieleman, T., and Hinton, G. (2012). Lecture 6.5\u2014RMSProp, COURSERA: Neural Networks for Machine Learning, University of Toronto."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wang, T., and Kerschbaum, F. (2019, January 12\u201317). Attacks on Digital Watermarks for Deep Neural Networks. Proceedings of the 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP \u201919), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8682202"},{"key":"ref_21","unstructured":"Burden, R.L., and Faires, J.D. (2010). Numerical Analysis, Brooks\/Cole. [9th ed.]."},{"key":"ref_22","first-page":"493","article-title":"Practical Markov Chain Monte Carlo","volume":"7","author":"Geyer","year":"1992","journal-title":"Stat. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Aucsmith, D. (1998). An Information-Theoretic Model for Steganography. Information Hiding, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/3-540-49380-8"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Comesa\u00f1a, P. (2012, January 2\u20135). Detection and information theoretic measures for quantifying the distinguishability between multimedia operator chains. Proceedings of the IEEE Workshop on Information Forensics and Security (WIFS12), Tenerife, Spain.","DOI":"10.1109\/WIFS.2012.6412651"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1109\/TIFS.2012.2237397","article-title":"The Source Identification Game: An Information-Theoretic Perspective","volume":"8","author":"Barni","year":"2013","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.5201\/ipol.2019.231","article-title":"An Analysis and Implementation of the FFDNet Image Denoising Method","volume":"9","author":"Tassano","year":"2019","journal-title":"Image Process. Line"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TIP.2016.2631888","article-title":"Waterloo Exploration Database: New Challenges for Image Quality Assessment Models","volume":"26","author":"Ma","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_28","unstructured":"Franzen, R. (2020, September 22). Kodak Lossless True Color Image Suite. Available online: http:\/\/r0k.us\/graphics\/kodak."},{"key":"ref_29","unstructured":"Martin, D., Fowlkes, C., Tal, D., and Malik, J. (2001, January 7\u201314). A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. Proceedings of the 8th Int\u2019l Conf. Computer Vision (ICCV 2001), Vancouver, BC, Canada."},{"key":"ref_30","unstructured":"Wilson, S.G. (1995). Digital Modulation and Coding, Pearson."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/12\/1379\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:41:36Z","timestamp":1760179296000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/12\/1379"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,6]]},"references-count":30,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["e22121379"],"URL":"https:\/\/doi.org\/10.3390\/e22121379","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,6]]}}}