{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T12:36:12Z","timestamp":1780058172319,"version":"3.54.0"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2022,1,1]],"date-time":"2022-01-01T00:00:00Z","timestamp":1640995200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,1]],"date-time":"2022-01-01T00:00:00Z","timestamp":1640995200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002790","name":"Canadian Network for Research and Innovation in Machining Technology, Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002790","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2022,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>A crucial aspect of reliable machine learning is to design a deployable system for generalizing new related but unobserved environments. Domain generalization aims to alleviate such a prediction gap between the observed and unseen environments. Previous approaches commonly incorporated learning the invariant representation for achieving good empirical performance. In this paper, we reveal that merely learning the invariant representation is vulnerable to the related unseen environment. To this end, we derive a novel theoretical analysis to control the unseen test environment error in the representation learning, which highlights the importance of controlling the smoothness of representation. In practice, our analysis further inspires an efficient regularization method to improve the robustness in domain generalization. The proposed regularization is orthogonal to and can be straightforwardly adopted in existing domain generalization algorithms that ensure invariant representation learning. Empirical results show that our algorithm outperforms the base versions in various datasets and invariance criteria.<\/jats:p>","DOI":"10.1007\/s10994-021-06080-w","type":"journal-article","created":{"date-parts":[[2022,1,1]],"date-time":"2022-01-01T07:16:29Z","timestamp":1641021389000},"page":"895-915","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["On the benefits of representation regularization in invariance based domain generalization"],"prefix":"10.1007","volume":"111","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6447-6559","authenticated-orcid":false,"given":"Changjian","family":"Shui","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Boyu","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"Gagn\u00e9","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,1,1]]},"reference":[{"issue":"1","key":"6080_CR1","first-page":"1947","volume":"19","author":"A Achille","year":"2018","unstructured":"Achille, A., & Soatto, S. (2018). Emergence of invariance and disentanglement in deep representations. The Journal of Machine Learning Research, 19(1), 1947\u20131980.","journal-title":"The Journal of Machine Learning Research"},{"key":"6080_CR2","unstructured":"Albuquerque, I., Monteiro, J., Darvishi, M., Falk, T. H., & Mitliagkas, I. (2019). Generalizing to unseen domains via distribution matching. arXiv preprint arXiv:1911.00804."},{"key":"6080_CR3","unstructured":"Arjovsky, M., Bottou, L., Gulrajani, I., & Lopez-Paz, D. (2019). Invariant risk minimization. arXiv preprint arXiv:1907.02893."},{"key":"6080_CR4","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1613\/jair.731","volume":"12","author":"J Baxter","year":"2000","unstructured":"Baxter, J. (2000). A model of inductive bias learning. Journal of Artificial Intelligence Research, 12, 149\u2013198.","journal-title":"Journal of Artificial Intelligence Research"},{"issue":"1","key":"6080_CR5","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1007\/s10994-009-5152-4","volume":"79","author":"S Ben-David","year":"2010","unstructured":"Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., & Vaughan, J. W. (2010). A theory of learning from different domains. Machine Learning, 79(1), 151\u2013175.","journal-title":"Machine Learning"},{"key":"6080_CR6","first-page":"2178","volume":"24","author":"G Blanchard","year":"2011","unstructured":"Blanchard, G., Lee, G., & Scott, C. (2011). Generalizing from several related classification tasks to a new unlabeled sample. Advances in Neural Information Processing Systems, 24, 2178\u20132186.","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"3","key":"6080_CR7","first-page":"404","volume":"35","author":"P B\u00fchlmann","year":"2020","unstructured":"B\u00fchlmann, P., et al. (2020). Invariance, causality and robustness. Statistical Science, 35(3), 404\u2013426.","journal-title":"Statistical Science"},{"key":"6080_CR8","unstructured":"Deshmukh, A.A., Lei, Y., Sharma, S., Dogan, U., Cutler, J. W., & Scott, C. (2019). A generalization error bound for multi-class domain generalization. arXiv preprint arXiv:1905.10392."},{"key":"6080_CR9","unstructured":"Devroye, L., Mehrabian, A., & Reddad, T. (2018). The total variation distance between high-dimensional gaussians. arXiv preprint arXiv:1810.08693."},{"issue":"1","key":"6080_CR10","first-page":"2096","volume":"17","author":"Y Ganin","year":"2016","unstructured":"Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2016). Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 17(1), 2096\u20132030.","journal-title":"The Journal of Machine Learning Research"},{"key":"6080_CR11","unstructured":"Goodfellow, I.J., Shlens, J., & Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572."},{"key":"6080_CR12","unstructured":"Gulrajani, I., & Lopez-Paz, D. (2021). In search of lost domain generalization. In International conference on learning representations. https:\/\/openreview.net\/forum?id=lQdXeXDoWtI."},{"key":"6080_CR13","unstructured":"Ilse, M., Tomczak, J. M., Louizos, C., & Welling, M. (2019). Diva: Domain invariant variational autoencoders. arXiv preprint arXiv:1905.10427."},{"key":"6080_CR14","unstructured":"Kamath, P., Tangella, A., Sutherland, D. J., & Srebro, N. (2021). Does invariant risk minimization capture invariance? arXiv preprint arXiv:2101.01134."},{"key":"6080_CR15","doi-asserted-by":"crossref","unstructured":"Li, D., Yang, Y., Song, Y. Z., & Hospedales, T. M. (2017). Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision (pp. 5542\u20135550).","DOI":"10.1109\/ICCV.2017.591"},{"key":"6080_CR16","doi-asserted-by":"crossref","unstructured":"Li, D., Yang, Y., Song, Y. Z., & Hospedales, T. M. (2018). Learning to generalize: Meta-learning for domain generalization. In Thirty-second AAAI conference on artificial intelligence.","DOI":"10.1609\/aaai.v32i1.11596"},{"key":"6080_CR17","doi-asserted-by":"crossref","unstructured":"Li, Y., Gong, M., Tian, X., Liu, T., & Tao, D. (2018). Domain generalization via conditional invariant representations. In Proceedings of the AAAI conference on artificial intelligence (Vol. 32).","DOI":"10.1007\/978-3-030-01267-0_38"},{"key":"6080_CR18","unstructured":"Li, Y., Yang, Y., Zhou, W., & Hospedales, T. M. (2019). Feature-critic networks for heterogeneous domain generalization. arXiv preprint arXiv:1901.11448."},{"issue":"1","key":"6080_CR19","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1109\/18.61115","volume":"37","author":"J Lin","year":"1991","unstructured":"Lin, J. (1991). Divergence measures based on the Shannon entropy. IEEE Transactions on Information theory, 37(1), 145\u2013151.","journal-title":"IEEE Transactions on Information theory"},{"key":"6080_CR20","unstructured":"Liu, F., Xu, W., Lu, J., Zhang, G., Gretton, A., & Sutherland, D. J. (2020). Learning deep kernels for non-parametric two-sample tests. In International conference on machine learning (pp. 6316\u20136326.) PMLR."},{"key":"6080_CR21","unstructured":"Lu, C., Wu, Y., Hern\u00e1ndez-Lobato, J. M., & Sch\u00f6lkopf, B. (2021). Nonlinear invariant risk minimization: A causal approach. arXiv preprint arXiv:2102.12353."},{"key":"6080_CR22","doi-asserted-by":"crossref","unstructured":"Matsuura, T., & Harada, T. (2020). Domain generalization using a mixture of multiple latent domains. In AAAI.","DOI":"10.1609\/aaai.v34i07.6846"},{"key":"6080_CR23","unstructured":"Mirza, M., & Osindero, S. (2014). Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784."},{"key":"6080_CR24","unstructured":"Miyato, T., Kataoka, T., Koyama, M., & Yoshida, Y. (2018). Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957."},{"key":"6080_CR25","doi-asserted-by":"crossref","unstructured":"M\u00fcller, J., Schmier, R., Ardizzone, L., Rother, C., & K\u00f6the, U. (2020). Learning robust models using the principle of independent causal mechanisms. arXiv preprint arXiv:2010.07167.","DOI":"10.1007\/978-3-030-92659-5_6"},{"key":"6080_CR26","unstructured":"Polyanskiy, Y., & Wu, Y. (2019). Lecture notes on information theory."},{"key":"6080_CR27","unstructured":"Roberts, D. A. (2021). Sgd implicitly regularizes generalization error. arXiv preprint arXiv:2104.04874."},{"key":"6080_CR28","unstructured":"Sicilia, A., Zhao, X., & Hwang, S. J. (2021). Domain adversarial neural networks for domain generalization: When it works and how to improve. arXiv preprint arXiv:2102.03924."},{"issue":"5","key":"6080_CR29","first-page":"985","volume":"8","author":"M Sugiyama","year":"2007","unstructured":"Sugiyama, M., Krauledat, M., & M\u00fcller, K. R. (2007). Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 985\u20131005.","journal-title":"Journal of Machine Learning Research"},{"key":"6080_CR30","doi-asserted-by":"crossref","unstructured":"Venkateswara, H., Eusebio, J., Chakraborty, S., & Panchanathan, S. (2017). Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5018\u20135027).","DOI":"10.1109\/CVPR.2017.572"},{"key":"6080_CR31","unstructured":"Volpi, R., Namkoong, H., Sener, O., Duchi, J., Murino, V., & Savarese, S. (2018). Generalizing to unseen domains via adversarial data augmentation. arXiv preprint arXiv:1805.12018."},{"key":"6080_CR32","unstructured":"Wang, W., Liao, S., Zhao, F., Kang, C., & Shao, L. (2020). Domainmix: Learning generalizable person re-identification without human annotations. arXiv preprint arXiv:2011.11953."},{"key":"6080_CR33","unstructured":"Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017). mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412."},{"key":"6080_CR34","unstructured":"Zhang, K., Sch\u00f6lkopf, B., Muandet, K., & Wang, Z. (2013). Domain adaptation under target and conditional shift. In International conference on machine learning (pp. 819\u2013827). PMLR."},{"key":"6080_CR35","unstructured":"Zhao, S., Gong, M., Liu, T., Fu, H., & Tao, D. (2020). Domain generalization via entropy regularization. Advances in Neural Information Processing Systems, 33."},{"key":"6080_CR36","doi-asserted-by":"crossref","unstructured":"Zhou, K., Yang, Y., Hospedales, T., & Xiang, T. (2020). Learning to generate novel domains for domain generalization. In European conference on computer vision (pp. 561\u2013578). Springer.","DOI":"10.1007\/978-3-030-58517-4_33"},{"key":"6080_CR37","unstructured":"Zhou, K., Yang, Y., Qiao, Y., & Xiang, T. (2021). Domain generalization with mixstyle. arXiv preprint arXiv:2104.02008."}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-06080-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-021-06080-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-06080-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,21]],"date-time":"2023-01-21T11:59:30Z","timestamp":1674302370000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-021-06080-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,1]]},"references-count":37,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,3]]}},"alternative-id":["6080"],"URL":"https:\/\/doi.org\/10.1007\/s10994-021-06080-w","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,1]]},"assertion":[{"value":"17 May 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 August 2021","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 September 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 January 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The author declares that he has no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}]}}