{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T00:50:12Z","timestamp":1760057412327,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,2,1]],"date-time":"2025-02-01T00:00:00Z","timestamp":1738368000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"JSPS KAKENHI","award":["JP20K19901"],"award-info":[{"award-number":["JP20K19901"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>This work presents a novel approach to handling epistemic uncertainty estimates with motivation from Bayesian linear regression. We propose treating the model-dependent variance in the predictive distribution\u2014commonly associated with epistemic uncertainty\u2014as a model for the underlying data distribution. Using high-dimensional random feature transformations, this approach allows for a computationally efficient, parameter-free representation of arbitrary data distributions. This allows assessing whether a query point lies within the distribution, which can also provide insights into outlier detection and generalization tasks. Furthermore, given an initial input, minimizing the uncertainty using gradient descent offers a new method of querying data points that are close to the initial input and belong to the distribution resembling the training data, much like auto-completion in associative networks. We extend the proposed method to applications such as local Gaussian approximations, input\u2013output regression, and even a mechanism for unlearning of data. This reinterpretation of uncertainty, alongside the geometric insights it provides, offers an innovative and novel framework for addressing classical machine learning challenges.<\/jats:p>","DOI":"10.3390\/e27020144","type":"journal-article","created":{"date-parts":[[2025,2,3]],"date-time":"2025-02-03T04:36:51Z","timestamp":1738557411000},"page":"144","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["The Epistemic Uncertainty Gradient in Spaces of Random Projections"],"prefix":"10.3390","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9725-6971","authenticated-orcid":false,"given":"Jeffrey F.","family":"Quei\u00dfer","sequence":"first","affiliation":[{"name":"Cognitive Neurorobotics Research Unit, Okinawa Institute of Science and Technology Graduate University, Onna 904-0495, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9131-9206","authenticated-orcid":false,"given":"Jun","family":"Tani","sequence":"additional","affiliation":[{"name":"Cognitive Neurorobotics Research Unit, Okinawa Institute of Science and Technology Graduate University, Onna 904-0495, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6738-9933","authenticated-orcid":false,"given":"Jochen J.","family":"Steil","sequence":"additional","affiliation":[{"name":"Institut f\u00fcr Robotik und Prozessinformatik, Technische Universit\u00e4t Braunschweig, 38106 Braunschweig, Germany"},{"name":"Theoretical Sciences Visiting Program (TSVP), Okinawa Institute of Science and Technology Graduate University, Onna 904-0495, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,2,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1007\/s10994-021-05946-3","article-title":"Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods","volume":"110","author":"Waegeman","year":"2021","journal-title":"Mach. Learn."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Hacking, I. (2006). Chap. 2. Duality. The Emergence of Probability: A Philosophical Study of Early Ideas About Probability, Induction and Statistical Inference, Cambridge University Press. [2nd ed.].","DOI":"10.1017\/CBO9780511817557"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/S0951-8320(96)00077-4","article-title":"Aleatory and epistemic uncertainty in probability elicitation with an example from hazardous waste management","volume":"54","author":"Hora","year":"1996","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.strusafe.2008.06.020","article-title":"Aleatory or epistemic? Does it matter?","volume":"31","author":"Kiureghian","year":"2009","journal-title":"Struct. Saf."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.ins.2013.07.030","article-title":"Reliable classification: Learning classifiers that distinguish aleatoric and epistemic uncertainty","volume":"255","author":"Senge","year":"2014","journal-title":"Inf. Sci."},{"key":"ref_6","unstructured":"Kendall, A., and Gal, Y. (2017, January 4\u20139). What uncertainties do we need in Bayesian deep learning for computer vision?. Proceedings of the 31st International Conference on Neural Information Processing Systems, Red Hook, NY, USA. NIPS\u201917."},{"key":"ref_7","unstructured":"Becker, A., and Liebig, T. (2022). Evaluating Machine Unlearning via Epistemic Uncertainty. arXiv."},{"key":"ref_8","unstructured":"Bishop, C.M. (2006). Pattern Recognition and Machine Learning, Springer. Chapter Graphical."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1109\/2.144401","article-title":"Functional-link net computing: Theory, system architecture, and functionalities","volume":"25","author":"Pao","year":"1992","journal-title":"Computer"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.neucom.2005.12.126","article-title":"Extreme learning machine: Theory and applications","volume":"70","author":"Huang","year":"2006","journal-title":"Neurocomputing"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Valdenegro-Toro, M., and Mori, D.S. (2022, January 19\u201320). A Deeper Look into Aleatoric and Epistemic Uncertainty Disentanglement. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Los Alamitos, CA, USA.","DOI":"10.52591\/lxai202206244"},{"key":"ref_12","unstructured":"DeVries, T., and Taylor, G.W. (2018). Learning Confidence for Out-of-Distribution Detection in Neural Networks. arXiv."},{"key":"ref_13","unstructured":"Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_14","unstructured":"Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Predictive Uncertainty Estimation via Prior Networks. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_15","unstructured":"Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Evidential Deep Learning to Quantify Classification Uncertainty. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum Likelihood from Incomplete Data via the EM Algorithm","volume":"39","author":"Dempster","year":"1977","journal-title":"J. R. Stat. Soc. Ser. (Methodol.)"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1007\/BF00337288","article-title":"Self-organized formation of topologically correct feature maps","volume":"43","author":"Kohonen","year":"1982","journal-title":"Biol. Cybern."},{"key":"ref_18","unstructured":"Touretzky, D., Mozer, M., and Hasselmo, M. (1995). Generalized Learning Vector Quantization. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_19","unstructured":"Touretzky, D., Mozer, M., and Hasselmo, M. (1995). Gaussian Processes for Regression. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2531","DOI":"10.1162\/089976602760407955","article-title":"Real-time computing without stable states: A new framework for neural computation based on perturbations","volume":"14","author":"Maass","year":"2002","journal-title":"Neural Comput."},{"key":"ref_21","unstructured":"Becker, S., Thrun, S., and Obermayer, K. (2002). Adaptive Nonlinear System Identification with Echo State Networks. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1109\/IJCNN.2004.1380039","article-title":"Backpropagation-decorrelation: Online recurrent learning with O (N) complexity","volume":"Volume 2","author":"Steil","year":"2004","journal-title":"Proceedings of the 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No. 04CH37541)"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1016\/j.neunet.2007.04.011","article-title":"Online reservoir adaptation by intrinsic plasticity for backpropagation\u2013decorrelation and echo state learning","volume":"20","author":"Steil","year":"2007","journal-title":"Neural Netw."},{"key":"ref_24","unstructured":"Schmidt, W., Kraaijveld, M., and Duin, R. (September, January 30). Feedforward neural networks with random weights. Proceedings of the 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems, The Hague, The Netherlands."},{"key":"ref_25","unstructured":"Huang, G.B., Zhu, Q.Y., and Siew, C.K. (2004, January 25\u201329). Extreme learning machine: A new learning scheme of feedforward neural networks. Proceedings of the 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541), Budapest, Hungary."},{"key":"ref_26","first-page":"230","article-title":"Regularization by intrinsic plasticity and its synergies with recurrence for random projection methods","volume":"4","author":"Neumann","year":"2012","journal-title":"J. Intell. Learn. Syst. Appl."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1197","DOI":"10.1109\/T-C.1972.223477","article-title":"Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements","volume":"C-21","author":"Amari","year":"1972","journal-title":"IEEE Trans. Comput."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2554","DOI":"10.1073\/pnas.79.8.2554","article-title":"Neural networks and physical systems with emergent collective computational abilities","volume":"79","author":"Hopfield","year":"1982","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_29","unstructured":"Ritter, H., and Schulten, K. (1987). Extending Kohonen\u2019s self-organizing mapping algorithm to learn ballistic movements. Neural Computers, Springer."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1016\/0925-2312(95)00117-4","article-title":"Rapid learning with parametrized self-organizing maps","volume":"12","author":"Walter","year":"1996","journal-title":"Neurocomputing"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1109\/2.30722","article-title":"Neutral learning of constrained nonlinear transformations","volume":"22","author":"Barhen","year":"1989","journal-title":"Computer"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1007\/s10514-014-9417-9","article-title":"Efficient policy search in low-dimensional embedding spaces by generalizing motion primitives with a parameterized skill memory","volume":"38","author":"Reinhart","year":"2015","journal-title":"Auton. Robot."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Yuen, K.V. (2010). Appendix A: Relationship between the Hessian and Covariance Matrix for Gaussian Random Variables. Bayesian Methods for Structural Dynamics and Civil Engineering, John Wiley & Sons, Ltd.","DOI":"10.1002\/9780470824566.app1"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1093\/biomet\/65.3.457","article-title":"Assessing the Accuracy of the Maximum Likelihood Estimator: Observed Versus Expected Fisher Information","volume":"65","author":"Efron","year":"1978","journal-title":"Biometrika"},{"key":"ref_35","unstructured":"Nguyen, Q.P., Kian, B., Low, H., and Jaillet, P. (2020, January 6\u201312). Variational Bayesian Unlearning. Proceedings of the 34th International Conference on Neural Information Processing Systems, Red Hook, NY, USA. NIPS\u201920."},{"key":"ref_36","unstructured":"Rawat, A., Requeima, J., Bruinsma, W.P., and Turner, R.E. (2022). Challenges and Pitfalls of Bayesian Unlearning. arXiv."},{"key":"ref_37","unstructured":"Fletcher, R. (1987). Practical Methods of Optimization, Wiley. Number v. 2 in a Wiley-Interscience Publication."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python","volume":"17","author":"Virtanen","year":"2020","journal-title":"Nat. Methods"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Han, S., Hu, X., Huang, H., Jiang, M., and Zhao, Y. (2022, January 28). ADBench: Anomaly Detection Benchmark. Proceedings of the Neural Information Processing Systems (NeurIPS), New Orleans, LO, USA.","DOI":"10.2139\/ssrn.4266498"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Reinhart, R.F., and Steil, J.J. (2011, January 26\u201328). Neural learning and dynamical selection of redundant solutions for inverse kinematic control. Proceedings of the 2011 11th IEEE-RAS International Conference on Humanoid Robots, Bled, Slovenia.","DOI":"10.1109\/Humanoids.2011.6100815"},{"key":"ref_41","first-page":"6840","article-title":"Denoising Diffusion Probabilistic Models","volume":"Volume 33","author":"Larochelle","year":"2020","journal-title":"Advances in Neural Information Processing Systems"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/2\/144\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:25:17Z","timestamp":1760027117000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/2\/144"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,1]]},"references-count":41,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["e27020144"],"URL":"https:\/\/doi.org\/10.3390\/e27020144","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2025,2,1]]}}}