{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T05:02:41Z","timestamp":1780981361540,"version":"3.54.1"},"reference-count":28,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2025,5,28]],"date-time":"2025-05-28T00:00:00Z","timestamp":1748390400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,5,28]],"date-time":"2025-05-28T00:00:00Z","timestamp":1748390400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2025,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>The Learning Using Statistical Invariants (LUSI) method is designed to integrate domain knowledge into machine learning models using structured elements referred to as \u201cpredicates.\u201dThis approach aims to improve model performance and reduce the amount of data required for training by leveraging the inherent properties of the domain. However, through a thorough theoretical analysis and numerical experiments, we demonstrate that LUSI often falls short of its intended goal. Our findings reveal significant flaws in the current formulation of LUSI; instead of effectively narrowing the set of admissible functions, the predicates act primarily as constraints on the output of the model, which can often lead to underwhelming performance and limited generalization. While LUSI may produce marginal improvements under specific conditions, its overall limitations and inability to consistently deliver intelligence-driven enhancements suggest a need for further refinement. Despite the identified limitations, the introduction of LUSI represents a timely and important step towards integrating domain-specific knowledge into machine learning, highlighting a promising direction for future research and development.<\/jats:p>","DOI":"10.1007\/s10994-025-06789-y","type":"journal-article","created":{"date-parts":[[2025,5,28]],"date-time":"2025-05-28T17:22:23Z","timestamp":1748452943000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Learning using statistical invariants: when they work and when they don\u2019t"],"prefix":"10.1007","volume":"114","author":[{"given":"Yang","family":"Liu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ernest","family":"Fokoue","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel E.","family":"Krutz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,5,28]]},"reference":[{"issue":"2","key":"6789_CR1","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1016\/0885-064X(90)90006-Y","volume":"6","author":"YS Abu-Mostafa","year":"1990","unstructured":"Abu-Mostafa, Y. S. (1990). Learning from hints in neural networks. Journal of Complexity, 6(2), 192\u2013198.","journal-title":"Journal of Complexity"},{"issue":"6","key":"6789_CR2","doi-asserted-by":"publisher","first-page":"783","DOI":"10.1016\/S0893-6080(99)00032-5","volume":"12","author":"S Amari","year":"1999","unstructured":"Amari, S., & Wu, S. (1999). Improving support vector machine classifiers by modifying kernel functions. Neural Networks: The Official Journal of the International Neural Network Society, 12(6), 783\u2013789.","journal-title":"Neural Networks: The Official Journal of the International Neural Network Society"},{"key":"6789_CR3","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1016\/j.tcs.2015.06.048","volume":"633","author":"F Anselmi","year":"2016","unstructured":"Anselmi, F., Leibo, J. Z., Rosasco, L., Mutch, J., Tacchetti, A., & Poggio, T. (2016). Unsupervised learning of invariant representations. Theoretical Computer Science, 633, 112\u2013121.","journal-title":"Theoretical Computer Science"},{"issue":"7","key":"6789_CR4","doi-asserted-by":"publisher","first-page":"0130140","DOI":"10.1371\/journal.pone.0130140","volume":"10","author":"S Bach","year":"2015","unstructured":"Bach, S., Binder, A., Montavon, G., Klauschen, F., M\u00fcller, K.-R., & Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE, 10(7), 0130140.","journal-title":"PLoS ONE"},{"key":"6789_CR5","unstructured":"Bock, R. (2004). MAGIC Gamma Telescope. UCI Machine Learning Repository."},{"key":"6789_CR6","unstructured":"Bronstein, M. M., Bruna, J., Cohen, T., & Veli\u010dkovi\u0107, P. (2021). Geometric Deep Learning: Grids, groups, graphs, geodesics, and gauges. arXiv:2104.13478"},{"issue":"4","key":"6789_CR7","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/MSP.2017.2693418","volume":"34","author":"MM Bronstein","year":"2017","unstructured":"Bronstein, M. M., Bruna, J., Lecun, Y., Szlam, A., & Vandergheynst, P. (2017). Geometric deep learning. IEEE Signal Processing Magazine, 34(4), 18\u201342.","journal-title":"IEEE Signal Processing Magazine"},{"issue":"6","key":"6789_CR8","doi-asserted-by":"publisher","first-page":"1662","DOI":"10.1109\/TIP.2007.896668","volume":"16","author":"T-J Chin","year":"2007","unstructured":"Chin, T.-J., & Suter, D. (2007). Incremental kernel principal component analysis. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society, 16(6), 1662\u20131674.","journal-title":"IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society"},{"key":"6789_CR9","unstructured":"Cohen, T. S., & Welling, M. (2016). Group equivariant convolutional networks. In ICML\u201916: Proceedings of the 33rd international conference on international conference on machine learning."},{"key":"6789_CR10","unstructured":"Crammer, K., Keshet, J., & Singer, Y. (2002). Kernel design using boosting. In Advances in neural information processing systems (Vol. 15)."},{"issue":"1","key":"6789_CR11","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1007\/s10994-007-5009-7","volume":"68","author":"B Haasdonk","year":"2007","unstructured":"Haasdonk, B., & Burkhardt, H. (2007). Invariant kernel functions for pattern analysis and machine learning. Machine Learning, 68(1), 35\u201361.","journal-title":"Machine Learning"},{"key":"6789_CR12","doi-asserted-by":"crossref","unstructured":"Haasdonk, B., Halawani, A., & Burkhardt, H. (2004). Adjustable invariant features by partial Haar-integration. In Proceedings of the 17th international conference on pattern recognition, 2004, ICPR 2004 (Vol. 2, pp. 769\u2013774).","DOI":"10.1109\/ICPR.2004.1334372"},{"key":"6789_CR13","doi-asserted-by":"crossref","unstructured":"Lohweg, V. (2012). Banknote authentication. UCI Machine Learning Repository","DOI":"10.1117\/12.2001444"},{"key":"6789_CR14","unstructured":"Mroueh, Y., Voinea, S., & Poggio, T. (2015). Learning with group invariant features: A kernel perspective. In NIPS\u201915: proceedings of the 29th international conference on neural information processing systems."},{"key":"6789_CR15","first-page":"1894","volume":"2","author":"D Pechyony","year":"2010","unstructured":"Pechyony, D., & Vapnik, V. (2010). On the theory of learning with Privileged Information. Neural Information Processing Systems, 2, 1894\u20131902.","journal-title":"Neural Information Processing Systems"},{"key":"6789_CR16","unstructured":"Petersen, K. B., & Pedersen, M. S. (2012). The Matrix Cookbook. Technical University of Denmark."},{"key":"6789_CR17","doi-asserted-by":"crossref","unstructured":"Rueden, L., Garcke, J., & Bauckhage, C. (2023). How does knowledge injection help in informed machine learning? In International joint conference on neural networks (IJCNN) (pp. 1\u20138). IEEE.","DOI":"10.1109\/IJCNN54540.2023.10191994"},{"key":"6789_CR18","doi-asserted-by":"crossref","unstructured":"Schulz-Mirbach, H. (1994). Constructing invariant features by averaging techniques. In Proceedings of the 12th IAPR international conference on pattern recognition, Vol. 3\u2014conference C: Signal processing (Cat. No.94CH3440-5) (Vol. 2, pp. 387\u20133902).","DOI":"10.1109\/ICPR.1994.576950"},{"issue":"3","key":"6789_CR19","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1002\/1098-1098(2000)11:3<181::AID-IMA1003>3.0.CO;2-E","volume":"11","author":"PY Simard","year":"2000","unstructured":"Simard, P. Y., Cun, Y. A. L., Denker, J. S., & Victorri, B. (2000). Transformation invariance in pattern recognition: Tangent distance and propagation. International Journal of Imaging Systems and Technology, 11(3), 181\u2013197.","journal-title":"International Journal of Imaging Systems and Technology"},{"key":"6789_CR20","unstructured":"Street, W. (1993). Breast Cancer Wisconsin (Diagnostic). UCI Machine Learning Repository."},{"issue":"3","key":"6789_CR21","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1007\/s10994-018-5742-0","volume":"108","author":"V Vapnik","year":"2019","unstructured":"Vapnik, V., & Izmailov, R. (2019). Rethinking statistical learning theory: Learning using statistical invariants. Machine Learning, 108(3), 381\u2013423.","journal-title":"Machine Learning"},{"issue":"5\u20136","key":"6789_CR22","doi-asserted-by":"publisher","first-page":"544","DOI":"10.1016\/j.neunet.2009.06.042","volume":"22","author":"V Vapnik","year":"2009","unstructured":"Vapnik, V., & Vashist, A. (2009). A new learning paradigm: Learning using privileged information. Neural Networks: The Official Journal of the International Neural Network Society, 22(5\u20136), 544\u2013557.","journal-title":"Neural Networks: The Official Journal of the International Neural Network Society"},{"issue":"11","key":"6789_CR23","doi-asserted-by":"publisher","first-page":"1949","DOI":"10.1134\/S000511791911002X","volume":"80","author":"VN Vapnik","year":"2019","unstructured":"Vapnik, V. N. (2019). Complete statistical theory of learning. Automation and Remote Control, 80(11), 1949\u20131975.","journal-title":"Automation and Remote Control"},{"key":"6789_CR24","doi-asserted-by":"crossref","unstructured":"Weiler, M., Hamnrecht, F. A., & Storath, M. (2018). Learning steerable filters for rotation equivariant CNNs. In 2018 IEEE\/CVF conference on computer vision and pattern recognition (pp. 849\u2013858).","DOI":"10.1109\/CVPR.2018.00095"},{"key":"6789_CR25","doi-asserted-by":"crossref","unstructured":"Wood, J., & Shawe-Taylor, J. (1996). Representation theory and invariant neural networks. Discrete Applied Mathematics (Amsterdam, Netherlands: 1988), 69 (1\u20132), 33\u201360.","DOI":"10.1016\/0166-218X(95)00075-3"},{"issue":"1","key":"6789_CR26","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1007\/s00365-021-09546-1","volume":"55","author":"D Yarotsky","year":"2022","unstructured":"Yarotsky, D. (2022). Universal approximations of invariant maps by neural networks. Constructive Approximation, 55(1), 407\u2013474.","journal-title":"Constructive Approximation"},{"key":"6789_CR27","unstructured":"Zhao, H., Combes, R. T., Zhang, K., & Gordon, G. J. (2019). On learning invariant representations for domain adaptation. In International conference on machine learning (Vol. 97, pp. 7523\u20137532)."},{"issue":"8","key":"6789_CR28","doi-asserted-by":"publisher","first-page":"6667","DOI":"10.1007\/s10489-024-05506-x","volume":"54","author":"T-T Zhu","year":"2024","unstructured":"Zhu, T.-T., Li, C.-N., Liu, T., & Shao, Y.-H. (2024). Learning using granularity statistical invariants for classification. Applied Intelligence, 54(8), 6667\u20136681.","journal-title":"Applied Intelligence"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-025-06789-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-025-06789-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-025-06789-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,6]],"date-time":"2025-09-06T16:10:29Z","timestamp":1757175029000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-025-06789-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,28]]},"references-count":28,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,7]]}},"alternative-id":["6789"],"URL":"https:\/\/doi.org\/10.1007\/s10994-025-06789-y","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,28]]},"assertion":[{"value":"19 October 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 April 2025","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 April 2025","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 May 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"154"}}