{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T23:50:03Z","timestamp":1772841003667,"version":"3.50.1"},"reference-count":50,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,11,11]],"date-time":"2025-11-11T00:00:00Z","timestamp":1762819200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100007567","name":"City University of Hong Kong","doi-asserted-by":"publisher","award":["9610591"],"award-info":[{"award-number":["9610591"]}],"id":[{"id":"10.13039\/100007567","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007567","name":"City University of Hong Kong","doi-asserted-by":"publisher","award":["7006055"],"award-info":[{"award-number":["7006055"]}],"id":[{"id":"10.13039\/100007567","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007567","name":"City University of Hong Kong","doi-asserted-by":"publisher","award":["7020162"],"award-info":[{"award-number":["7020162"]}],"id":[{"id":"10.13039\/100007567","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007567","name":"City University of Hong Kong","doi-asserted-by":"publisher","award":["9680367"],"award-info":[{"award-number":["9680367"]}],"id":[{"id":"10.13039\/100007567","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010877","name":"Science, Technology and Innovation Commission of Shenzhen Municipality","doi-asserted-by":"publisher","award":["JCYJ20230807115001004"],"award-info":[{"award-number":["JCYJ20230807115001004"]}],"id":[{"id":"10.13039\/501100010877","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Unsupervised Hebbian learning is a biologically inspired algorithm designed to extract representations from input images, which can subsequently support supervised learning. It presents a promising alternative to traditional artificial neural networks (ANNs). Many attempts have focused on enhancing Hebbian learning by incorporating more biologically plausible components. Contrarily, we draw inspiration from recent advances in ANNs to rethink and further improve Hebbian learning in three interconnected aspects. First, we investigate the issue of overfitting in Hebbian learning and emphasize the importance of selecting an optimal number of training epochs, even in unsupervised settings. In addition, we discuss the risks and benefits of anti-Hebbian learning in model performance, and our visualizations reveal that synapses resembling the input images sometimes do not necessarily reflect effective learning. Then, we explore the impact of different activation functions on Hebbian representations, highlighting the benefits of properly utilizing negative values. Furthermore, motivated by the success of large pre-trained language models, we propose a novel approach for leveraging unlabeled data from other datasets. Unlike conventional pre-training in ANNs, experimental results demonstrate that merging trained synapses from different datasets leads to improved performance. Overall, our findings offer fresh perspectives on enhancing the future design of Hebbian learning algorithms.<\/jats:p>","DOI":"10.3390\/make7040143","type":"journal-article","created":{"date-parts":[[2025,11,11]],"date-time":"2025-11-11T14:28:59Z","timestamp":1762871339000},"page":"143","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Delving into Unsupervised Hebbian Learning from Artificial Intelligence Perspectives"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7109-3051","authenticated-orcid":false,"given":"Wei","family":"Lin","sequence":"first","affiliation":[{"name":"Department of Neuroscience, College of Biomedicine, City University of Hong Kong, Tat Chee Avenue, Kowloon Tong, Kowloon, Hong Kong 999077, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-8825-0829","authenticated-orcid":false,"given":"Zhixin","family":"Piao","sequence":"additional","affiliation":[{"name":"Department of Neuroscience, College of Biomedicine, City University of Hong Kong, Tat Chee Avenue, Kowloon Tong, Kowloon, Hong Kong 999077, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7780-8230","authenticated-orcid":false,"given":"Chi Chung Alan","family":"Fung","sequence":"additional","affiliation":[{"name":"Department of Neuroscience, College of Biomedicine, City University of Hong Kong, Tat Chee Avenue, Kowloon Tong, Kowloon, Hong Kong 999077, China"},{"name":"CityU Shenzhen Research Institute, 8 Yuexing 1st Road, Shenzhen Hi-Tech Industrial Park, Nanshan District, Shenzhen 518057, China"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,11]]},"reference":[{"key":"ref_1","unstructured":"Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., and Anadkat, S. (2023). Gpt-4 technical report. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"30033","DOI":"10.1073\/pnas.1907373117","article-title":"The unreasonable effectiveness of deep learning in artificial intelligence","volume":"117","author":"Sejnowski","year":"2020","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_4","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_5","unstructured":"Zeiler, M.D., and Fergus, R. (2014, January 6\u201312). Visualizing and understanding convolutional networks. Proceedings of the Computer Vision\u2014ECCV 2014: 13th European Conference, Zurich, Switzerland. Proceedings, Part I 13."},{"key":"ref_6","unstructured":"Bengio, Y., Lee, D.H., Bornschein, J., Mesnard, T., and Lin, Z. (2015). Towards biologically plausible deep learning. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.neunet.2019.06.001","article-title":"Biologically plausible deep learning\u2014But how far can we go with shallow networks?","volume":"118","author":"Illing","year":"2019","journal-title":"Neural Netw."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Hebb, D.O. (2005). The Organization of Behavior: A Neuropsychological Theory, Psychology Press.","DOI":"10.4324\/9781410612403"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"pgad161","DOI":"10.1093\/pnasnexus\/pgad161","article-title":"Competition on presynaptic resources enhances the discrimination of interfering memories","volume":"2","author":"Fung","year":"2023","journal-title":"PNAS Nexus"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"7723","DOI":"10.1073\/pnas.1820458116","article-title":"Unsupervised learning by competing hidden units","volume":"116","author":"Krotov","year":"2019","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"044017","DOI":"10.1088\/2634-4386\/aca710","article-title":"Softhebb: Bayesian inference in unsupervised hebbian soft winner-take-all networks","volume":"2","author":"Moraitis","year":"2022","journal-title":"Neuromorphic Comput. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2519","DOI":"10.1162\/089976600300014827","article-title":"On the computational power of winner-take-all","volume":"12","author":"Maass","year":"2000","journal-title":"Neural Comput."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Gupta, M., Ambikapathi, A., and Ramasamy, S. (2021, January 6\u201311). Hebbnet: A simplified hebbian learning framework to do biologically plausible learning. Proceedings of the ICASSP 2021\u20142021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.","DOI":"10.1109\/ICASSP39728.2021.9414241"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"127867","DOI":"10.1016\/j.neucom.2024.127867","article-title":"Scalable bio-inspired training of Deep Neural Networks with FastHebb","volume":"595","author":"Lagani","year":"2024","journal-title":"Neurocomputing"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1007\/BF02331346","article-title":"Forming sparse representations by local anti-Hebbian learning","volume":"64","year":"1990","journal-title":"Biol. Cybern."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1461","DOI":"10.1162\/NECO_a_00745","article-title":"A hebbian\/anti-hebbian neural network for linear subspace learning: A derivation from multidimensional scaling of streaming data","volume":"27","author":"Pehlevan","year":"2015","journal-title":"Neural Comput."},{"key":"ref_17","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel."},{"key":"ref_18","unstructured":"Glorot, X., Bordes, A., and Bengio, Y. (2011, January 11\u201313). Deep sparse rectifier neural networks. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA."},{"key":"ref_19","unstructured":"Hendrycks, D., and Gimpel, K. (2016). Gaussian error linear units (gelus). arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e2016239118","DOI":"10.1073\/pnas.2016239118","article-title":"Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences","volume":"118","author":"Rives","year":"2021","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022, January 18\u201324). A convnet for the 2020s. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01167"},{"key":"ref_22","unstructured":"Ramachandran, P., Zoph, B., and Le, Q.V. (2017). Searching for activation functions. arXiv."},{"key":"ref_23","unstructured":"Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013, January 16\u201321). Rectifier nonlinearities improve neural network acoustic models. Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_24","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.neuron.2004.09.012","article-title":"LTP and LTD","volume":"44","author":"Malenka","year":"2004","journal-title":"Neuron"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"100244","DOI":"10.1016\/j.patter.2021.100244","article-title":"The overfitted brain: Dreams evolved to assist generalization","volume":"2","author":"Hoel","year":"2021","journal-title":"Patterns"},{"key":"ref_27","unstructured":"Salman, S., and Liu, X. (2019). Overfitting mechanism and avoidance in deep neural networks. arXiv."},{"key":"ref_28","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 18\u201320). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 18\u201320). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Lin, W., and Fung, C.C.A. (2025). Utilizing Data Imbalance to Enhance Compound\u2013Protein Interaction Prediction Models. Adv. Intell. Syst., 2400985.","DOI":"10.1002\/aisy.202400985"},{"key":"ref_32","unstructured":"Dauphin, Y.N., Fan, A., Auli, M., and Grangier, D. (2017, January 6\u201311). Language modeling with gated convolutional networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_33","unstructured":"Shazeer, N. (2020). Glu variants improve transformer. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"He, K., Chen, X., Xie, S., Li, Y., Doll\u00e1r, P., and Girshick, R. (2022, January 18\u201324). Masked autoencoders are scalable vision learners. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"ref_35","unstructured":"Bao, H., Dong, L., Piao, S., and Wei, F. (2021, January 3\u20137). BEiT: BERT Pre-Training of Image Transformers. Proceedings of the International Conference on Learning Representations, Vienna, Austria."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"20832","DOI":"10.1021\/acsomega.3c09512","article-title":"Rethinking the Masking Strategy for Pretraining Molecular Graphs from a Data-Centric View","volume":"9","author":"Lin","year":"2024","journal-title":"ACS Omega"},{"key":"ref_37","unstructured":"Hu, W., Liu, B., Gomes, J., Zitnik, M., Liang, P., Pande, V.S., and Leskovec, J. (2020, January 26\u201330). Strategies for Pre-training Graph Neural Networks. Proceedings of the 8th International Conference on Learning Representations (ICLR 2020), Addis Ababa, Ethiopia."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"7112","DOI":"10.1109\/TPAMI.2021.3095381","article-title":"Prottrans: Toward understanding the language of life through self-supervised learning","volume":"44","author":"Elnaggar","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Luo, H., Xiang, Y., Fang, X., Lin, W., Wang, F., Wu, H., and Wang, H. (2022). BatchDTA: Implicit batch alignment enhances deep learning-based drug\u2013target affinity estimation. Briefings Bioinform., 23.","DOI":"10.1093\/bib\/bbac260"},{"key":"ref_40","unstructured":"Wortsman, M., Ilharco, G., Gadre, S.Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A.S., Namkoong, H., Farhadi, A., Carmon, Y., and Kornblith, S. (2022, January 17\u201323). Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. Proceedings of the International Conference on Machine Learning, Baltimore, MD, USA."},{"key":"ref_41","unstructured":"Krotov, D., and Hopfield, J.J. (2016, January 5\u201310). Dense associative memory for pattern recognition. Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS\u201916), Red Hook, NY, USA."},{"key":"ref_42","unstructured":"Ba, J., and Frey, B. (2013, January 5\u20138). Adaptive dropout for training deep neural networks. Proceedings of the Advances in Neural Information Processing Systems 26 (NIPS 2013), Lake Tahoe, NV, USA."},{"key":"ref_43","unstructured":"Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms. arXiv."},{"key":"ref_44","unstructured":"Clanuwat, T., Bober-Irizar, M., Kitamoto, A., Lamb, A., Yamamoto, K., and Ha, D. (2018). Deep learning for classical japanese literature. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wu, J., Pan, Y., Ye, Q., Zhou, J., and Gou, F. (2024). Intelligent cell images segmentation system: Based on SDN and moving transformer. Sci. Rep., 14.","DOI":"10.1038\/s41598-024-76577-6"},{"key":"ref_46","unstructured":"Jain, N., Chiang, P.y., Wen, Y., Kirchenbauer, J., Chu, H.M., Somepalli, G., Bartoldson, B.R., Kailkhura, B., Schwarzschild, A., and Saha, A. (2024, January 7\u201311). NEFTune: Noisy Embeddings Improve Instruction Finetuning. Proceedings of the ICLR, Vienna, Austria."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"2405145","DOI":"10.1002\/adma.202405145","article-title":"Development of bio-voltage operated humidity-sensory neurons comprising self-assembled peptide memristors","volume":"36","author":"Lv","year":"2024","journal-title":"Adv. Mater."},{"key":"ref_48","unstructured":"Journ\u00e9, A., Rodriguez, H.G., Guo, Q., and Moraitis, T. (2022). Hebbian deep learning without feedback. arXiv."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"021501","DOI":"10.1063\/5.0186054","article-title":"Brain-inspired learning in artificial neural networks: A review","volume":"2","author":"Schmidgall","year":"2024","journal-title":"APL Mach. Learn."},{"key":"ref_50","unstructured":"Hinton, G. (2022). The forward-forward algorithm: Some preliminary investigations. arXiv."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/4\/143\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T05:27:40Z","timestamp":1763011660000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/4\/143"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,11]]},"references-count":50,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["make7040143"],"URL":"https:\/\/doi.org\/10.3390\/make7040143","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,11]]}}}