{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T15:33:07Z","timestamp":1773761587162,"version":"3.50.1"},"reference-count":88,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T00:00:00Z","timestamp":1773446400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Biomimetics"],"abstract":"<jats:p>This paper introduces a biomimetic framework and novel brain-inspired AI (BIAI) models based on spiking neural networks (SNNs) for emotional state recognition from audio (speech), visual (face), and integrated multimodal audio\u2013visual data. The developed framework, named eXCube2, uses a three-dimensional SNN architecture NeuCube that is spatially structured according to a human brain template. The BIAI models developed in eXCube2 are trainable on spatio- and spectro-temporal data using brain-inspired learning rules. Such models are explainable in terms of revealing patterns in data and are adaptable to new data. The eXCube2 models are implemented as software systems and tested on speech and video data of subjects expressing emotional states. The use of a brain template for the SNN structure enables brain-inspired tonotopic and stereo mapping of audio inputs, topographic mapping of visual data, and the combined use of both modalities. This novel approach brings AI-based emotional state recognition closer to human perception, provides a better explainability and adaptability than existing AI systems. It also results in a higher or competitive accuracy, even though this was not the main goal here. This is demonstrated through experiments on benchmark datasets, achieving classification accuracy above 80% on single-modality data and 88.9% when multimodal audio\u2013visual data are used, and a \u201cdon\u2019t know\u201d output is introduced. The paper further discusses possible applications of the proposed eXCube2 framework to other audio, visual, and audio\u2013visual data for solving challenging problems, such as recognizing emotional states of people from different origins; brain state diagnosis (e.g., Parkinson\u2019s disease, Alzheimer\u2019s disease, ADHD, dementia); measuring response to treatment over time; evaluating satisfaction responses from online clients; cognitive robotics; human\u2013robot interaction; chatbots; and interactive computer games. The SNN-based implementation of BIAI also enables the use of neuromorphic chips and platforms, leading to reduced power consumption, smaller device size, higher performance accuracy, and improved adaptability and explainability. This research shows a step toward building brain-inspired AI systems.<\/jats:p>","DOI":"10.3390\/biomimetics11030208","type":"journal-article","created":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T08:41:26Z","timestamp":1773650486000},"page":"208","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["eXCube2: Explainable Brain-Inspired Spiking Neural Network Framework for Emotion Recognition from Audio, Visual and Multimodal Audio\u2013Visual Data"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4433-7521","authenticated-orcid":false,"given":"N. K.","family":"Kasabov","sequence":"first","affiliation":[{"name":"Knowledge Engineering and Discovery Research Institute (KEDRI), School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology (AUT), WZ Building, St. Paul Street, Auckland 1010, New Zealand"},{"name":"Institute for Information and Communication Technologies (IICT), Bulgarian Academy of Sciences, Acad. Georgi Bonchev Str., Bl. 2,25A, 1113 Sofia, Bulgaria"},{"name":"KEC Ltd., 47C Nihill Cr., Auckland 1071, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"A.","family":"Yang","sequence":"additional","affiliation":[{"name":"ManaBridge Ltd., Queenstown 9371, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Z.","family":"Wang","sequence":"additional","affiliation":[{"name":"Knowledge Engineering and Discovery Research Institute (KEDRI), School of Engineering, Computer and Mathematical Sciences, Auckland University of Technology (AUT), WZ Building, St. Paul Street, Auckland 1010, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"I.","family":"Abouhassan","sequence":"additional","affiliation":[{"name":"KEC Ltd., 47C Nihill Cr., Auckland 1071, New Zealand"},{"name":"Department of Industrial Applications of Computers, Technical University of Sofia, ul. \u201cProfessor Georgi Bradistilov\u201d 11, 1756 Sofia, Bulgaria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"A.","family":"Kassabova","sequence":"additional","affiliation":[{"name":"KEC Ltd., 47C Nihill Cr., Auckland 1071, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4669-4170","authenticated-orcid":false,"given":"T.","family":"Lappas","sequence":"additional","affiliation":[{"name":"Department of Marketing and Communication, School of Business Administration, Athens University of Economics and Business, 28is Oktovriou 76, 104 34 Athina, Greece"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,3,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chern, I.C., Hung, K.H., Chen, Y.T., Hussain, T., Gogate, M., Hussain, A., Tsao, Y., and Hou, J.C. (2023, January 4\u201310). Audio-visual speech enhancement and separation by utilizing multi-modal self-supervised embeddings. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), Rhodes Island, Greece.","DOI":"10.1109\/ICASSPW59220.2023.10193049"},{"key":"ref_2","unstructured":"Saraceno, G. (2017). Deep Learning and Memorizing of Spectro-Temporal Data (Music) in the Spatio-Temporal Brain. [Master\u2019s Thesis, University of Trento]."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1109\/TNN.2004.841811","article-title":"Gabor wavelet associative memory for face recognition","volume":"16","author":"Zhang","year":"2005","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Liu, W., Quan, Y., Liu, Y., and Yan, D.-M. (2022, January 23\u201327). Bi-directional modality fusion network for audio-visual event localization. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.","DOI":"10.1109\/ICASSP43922.2022.9746280"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lacheze, L., Guo, Y., Benosman, R., Gas, B., and Couverture, C. (2009, January 10\u201315). Audio\/video fusion for objects recognition. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO, USA.","DOI":"10.1109\/IROS.2009.5354442"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1702","DOI":"10.1109\/TASLP.2018.2842159","article-title":"Supervised Speech Separation Based on Deep Learning: An Overview","volume":"26","author":"Wang","year":"2018","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zheng, X., and Wei, Y. (2022, January 20\u201322). Audio-visual event and sound source localization based on spatial-channel feature fusion. Proceedings of the International Conference on Signal and Image Processing (ICSIP), Suzhou, China.","DOI":"10.1109\/ICSIP55141.2022.9886106"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/S0020-0255(99)00114-0","article-title":"AVIS: A connectionist-based framework for integrated auditory and visual information processing","volume":"123","author":"Kasabov","year":"2000","journal-title":"Inf. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"819","DOI":"10.1016\/j.neunet.2010.04.009","article-title":"Evolving spiking neural networks for audiovisual information processing","volume":"23","author":"Wysoski","year":"2010","journal-title":"Neural Netw."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"828","DOI":"10.1109\/TPAMI.2003.1206512","article-title":"A graphical model for audiovisual object tracking","volume":"25","author":"Beal","year":"2003","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yue, Q., Wu, X., and Gao, J. (2021, January 1\u20133). Audio-visual event localization based on cross-modal interacting guidance. Proceedings of the IEEE International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), Laguna Hills, CA, USA.","DOI":"10.1109\/AIKE52691.2021.00022"},{"key":"ref_12","first-page":"5467936","article-title":"A multichannel convolutional neural network architecture for the detection of the state of mind using physiological signals from wearable devices","volume":"2020","author":"Chakraborty","year":"2020","journal-title":"J. Healthc. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"155","DOI":"10.3233\/NRE-208011","article-title":"Neural plasticity: The substratum of music-based interventions in neurorehabilitation","volume":"48","author":"Chatterjee","year":"2021","journal-title":"NeuroRehabilitation"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Krautz, A.E., Langner, J., Helmhold, F., Volkening, J., Hoffmann, A., and Hasler, C. (2025). Bridging AI innovation and healthcare: Scalable clinical validation methods for voice biomarkers. Front. Digit. Health, 7.","DOI":"10.3389\/fdgth.2025.1575753"},{"key":"ref_15","first-page":"45105","article-title":"Neural correlates of auditory predictive timing are linked to human vocal pitch stability","volume":"9","author":"Rao","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_16","unstructured":"Reddy, V. (2022). PPINtonus: Early detection of Parkinson\u2019s disease using deep-learning tonal analysis. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kasabov, N. (2019). Time-Space, Spiking Neural Networks and Brain-Inspired AI, Springer Nature.","DOI":"10.1007\/978-3-662-57715-8"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.neunet.2014.01.006","article-title":"NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data","volume":"52","author":"Kasabov","year":"2014","journal-title":"Neural Netw."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1109\/TAFFC.2015.2457417","article-title":"The Geneva minimalistic acoustic parameter set (GeMAPS) for voice research and affective computing","volume":"7","author":"Eyben","year":"2016","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/S0167-6393(02)00084-5","article-title":"Vocal communication of emotion: A review of research paradigms","volume":"40","author":"Scherer","year":"2003","journal-title":"Speech Commun."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"101894","DOI":"10.1016\/j.bspc.2020.101894","article-title":"Speech emotion recognition with deep convolutional neural networks","volume":"59","author":"Issa","year":"2020","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"and Kwon, S. (2020). CLSTM: Deep feature-based speech emotion recognition using the hierarchical ConvLSTM network. Mathematics, 8.","DOI":"10.3390\/math8122133"},{"key":"ref_23","unstructured":"Pepino, L., Riera, P., and Ferrer, L. (September, January 30). Emotion recognition from speech using wav2vec 2.0 embeddings. Proceedings of the Interspeech, Brno, Czechia."},{"key":"ref_24","unstructured":"Ismail, S.M. (2024). Distilled HuBERT for mobile speech emotion recognition: A cross-corpus validation study. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Luna-Jim\u00e9nez, C., Griol, D., Callejas, Z., Kleinlein, R., Montero, J.M., and Fern\u00e1ndez-Mart\u00ednez, F. (2021). Multimodal emotion recognition on RAVDESS dataset using transfer learning. Sensors, 21.","DOI":"10.3390\/s21227665"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Luna-Jim\u00e9nez, C., Kleinlein, R., Griol, D., Callejas, Z., Montero, J.M., and Fern\u00e1ndez-Mart\u00ednez, F. (2022). A proposal for multimodal emotion recognition using aural transformers and action units on RAVDESS dataset. Appl. Sci., 12.","DOI":"10.3390\/app12010327"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1717","DOI":"10.1007\/s00500-020-05501-7","article-title":"Generalisation and robustness investigation for facial and speech emotion recognition using bio-inspired spiking neural networks","volume":"25","author":"Ye","year":"2021","journal-title":"Soft Comput."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Tan, C., Ceballos, G., Kasabov, N., and Subramaniyam, N. (2020). FusionSense: Emotion Classification using Feature Fusion of Multimodal Data and Deep learning in a Brain-inspired Spiking Neural Network. Sensors, 20.","DOI":"10.3390\/s20185328"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1077","DOI":"10.1109\/TCDS.2023.3327081","article-title":"Transformer-based spiking neural networks for multimodal audiovisual classification","volume":"16","author":"Guo","year":"2024","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"10028","DOI":"10.1109\/TNNLS.2022.3163771","article-title":"Multimodal Sparse Transformer Network for Audio-Visual Speech Recognition","volume":"34","author":"Song","year":"2023","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Singh, A., and Shukla, A.K. (2025, January 7\u20138). Transformer based Multimodal Mental Health Assessment Framework Using Cross-Attention Mechanisms on Audio-Visual Data. Proceedings of the 2025 IEEE International Conference on Intelligent Signal Processing and Effective Communication Technologies (INSPECT), Gwalior, India.","DOI":"10.1109\/INSPECT67393.2025.11350529"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Feng, W., Guan, N., Li, Y., Zhang, X., and Luo, Z. (2017). Audio visual speech recognition with multimodal recurrent neural networks. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), IEEE.","DOI":"10.1109\/IJCNN.2017.7965918"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Su, R., Wang, L., and Liu, X. (2017). Multimodal learning using 3D audio-visual data for audio-visual speech recognition. Proceedings of the 2017 International Conference on Asian Language Processing (IALP), IEEE.","DOI":"10.1109\/IALP.2017.8300541"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Padhira, K.R., Gundla, N.R., Kumar, K.V., Sarangi, P.K., and Jabbar, M.A. (2025). RBi-LSTM based CNN model for speech emotion recognition. Proceedings of the 12th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO), Noida NCR, India, 18\u201319 September 2025, IEEE.","DOI":"10.1109\/ICRITO66076.2025.11241506"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1162\/089976606775093882","article-title":"Polychronization: Computation with spikes","volume":"18","author":"Izhikevich","year":"2006","journal-title":"Neural Comput."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Abeles, M. (1991). Corticonics: Neural Circuits of the Cerebral Cortex, Cambridge University Press.","DOI":"10.1017\/CBO9780511574566"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kov\u00e1cs, L., Haidegger, T., and Szak\u00e1l, A. (2024). STAM-SNN: Spatio-temporal associative memory in brain-inspired spiking neural networks: Concepts and perspectives. Recent Advances in Intelligent Engineering, Springer.","DOI":"10.1007\/978-3-031-58257-8"},{"key":"ref_38","first-page":"56","article-title":"Life-long learning and evolving associative memories in brain-inspired spiking neural networks","volume":"8","author":"Kasabov","year":"2024","journal-title":"MOJ Appl. Bio. Biomech."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kasabov, N.K. (2023). Spatio-temporal associative memories in brain-inspired spiking neural networks: Concepts and perspectives. TechRxiv.","DOI":"10.36227\/techrxiv.23723208.v1"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kasabov, N., Bahrami, H., Doborjeh, M., and Wang, A. (2023). Brain inspired spatio-temporal associative memories for neuroimaging data: Classification EEG and fMRI. Bioengineering, 10.","DOI":"10.20944\/preprints202308.0333.v1"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"5574","DOI":"10.1093\/cercor\/bhac443","article-title":"Audiovisual integration in the human brain: A coordinate-based meta-analysis","volume":"33","author":"Gao","year":"2023","journal-title":"Cereb. Cortex"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Mana, N., Schwenker, F., and Trentin, E. (2012). Neucube evospike architecture for spatio-temporal modelling and pattern recognition of brain signals. Artificial Neural Networks in Pattern Recognition, Springer.","DOI":"10.1007\/978-3-642-33212-8"},{"key":"ref_43","first-page":"72","article-title":"Co-planar stereotaxic atlas of the human brain: 3-dimensional proportional system","volume":"104","author":"Talairach","year":"1988","journal-title":"J. Laryngol. Otol."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1038\/nature18933","article-title":"A multi-modal parcellation of human cerebral cortex","volume":"536","author":"Glasser","year":"2016","journal-title":"Nature"},{"key":"ref_45","unstructured":"Al-Tahan, H., Deb, M., Feather, J., and Murty, N.A.R. (2025). End-to-end topographic auditory models replicate signatures of human auditory cortex. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Moerel, M., De Martino, F., and Formisano, E. (2014). An anatomical and functional topography of human auditory cortical areas. Front. Neurosci., 8.","DOI":"10.3389\/fnins.2014.00225"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1016\/j.neunet.2012.11.014","article-title":"Dynamic evolving spiking neural networks for online spatio- and spectro-temporal pattern recognition","volume":"41","author":"Kasabov","year":"2013","journal-title":"Neural Netw."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1016\/j.neunet.2019.08.029","article-title":"Deep learning and deep knowledge representation in spiking neural networks for brain\u2013computer interfaces","volume":"121","author":"Kumarasinghe","year":"2020","journal-title":"Neural Netw."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"4542","DOI":"10.1109\/TFUZZ.2023.3292802","article-title":"Transfer learning of fuzzy spatio-temporal rules in the NeuCube brain-inspired spiking neural network","volume":"31","author":"Kasabov","year":"2023","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_50","unstructured":"Swanson, R., Livingstone, S.R., and Russo, F.A. (2019). RAVDESS Facial Landmark Tracking, Version 1.0.0, Affective Data Science Lab (ADSL)."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Livingstone, S.R., and Russo, F.A. (2018). The Ryerson audio-visual database of emotional speech and song (RAVDESS). PLoS ONE, 13.","DOI":"10.1371\/journal.pone.0196391"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1037\/0033-2909.129.5.770","article-title":"Communication of emotions in vocal expression and music performance: Different channels, same code?","volume":"129","author":"Juslin","year":"2003","journal-title":"Psychol. Bull."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1016\/S0896-6273(03)00669-X","article-title":"Mirror-symmetric tonotopic maps in human primary auditory cortex","volume":"40","author":"Formisano","year":"2003","journal-title":"Neuron"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"14067","DOI":"10.1523\/JNEUROSCI.2000-11.2011","article-title":"Human primary auditory cortex follows the shape of Heschl\u2019s gyrus","volume":"31","author":"Marques","year":"2011","journal-title":"J. Neurosci."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1523\/JNEUROSCI.0960-21.2021","article-title":"Distinct representations of tonotopy and pitch in human auditory cortex","volume":"42","author":"Allen","year":"2022","journal-title":"J. Neurosci."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1016\/S1364-6613(00)01482-0","article-title":"The distributed human neural system for face perception","volume":"4","author":"Haxby","year":"2000","journal-title":"Trends Cogn. Sci."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"119587","DOI":"10.1016\/j.neuroimage.2022.119587","article-title":"Let\u2019s face it: The lateralization of the face perception network as measured with fMRI is not clearly right dominant","volume":"263","author":"Thome","year":"2022","journal-title":"NeuroImage"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1016\/j.neuroimage.2018.08.025","article-title":"TMS demonstrates that both right and left superior temporal sulci are important for facial expression recognition","volume":"183","author":"Sliwinska","year":"2018","journal-title":"NeuroImage"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1093\/scan\/nsx119","article-title":"Functional organization of face processing in the human superior temporal sulcus: A 7T high-resolution fMRI study","volume":"13","author":"Schobert","year":"2018","journal-title":"Soc. Cogn. Affect. Neurosci."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"673","DOI":"10.1016\/j.patrec.2006.08.007","article-title":"Global, local and personalised modelling and profile discovery in Bioinformatics: An integrated approach","volume":"28","author":"Kasabov","year":"2007","journal-title":"Pattern Recognit. Lett."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"1675","DOI":"10.1007\/s11063-020-10322-8","article-title":"Spiking neural networks: Background, recent development and the NeuCube architecture","volume":"52","author":"Tan","year":"2020","journal-title":"Neural Process. Lett."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Chen, C., Al-Halah, Z., and Grauman, K. (2021, January 20\u201325). Semantic audio-visual navigation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01526"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"648","DOI":"10.1109\/TNN.2006.890804","article-title":"Sparse distributed memory using rank-order neural codes","volume":"18","author":"Furber","year":"2007","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"026019","DOI":"10.1088\/1741-2552\/aafabc","article-title":"Classification and regression of spatio-temporal signals using NeuCube","volume":"16","author":"Behrenbeck","year":"2019","journal-title":"J. Neural Eng."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"James, R., Garside, J., Hopkins, M., Plana, L.A., Temple, S., Davidson, S., and Furber, S. (2017, January 19\u201321). Parallel distribution of an inner hair cell and auditory nerve model. Proceedings of the IEEE BioCAS, Turin, Italy.","DOI":"10.1109\/BIOCAS.2017.8325171"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"652","DOI":"10.1109\/JPROC.2014.2304638","article-title":"The SpiNNaker project","volume":"102","author":"Furber","year":"2014","journal-title":"Proc. IEEE"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Paulun, L., Wendt, A., and Kasabov, N.K. (2018). A retinotopic spiking neural network system for accurate recognition of moving objects. Front. Comput. Neurosci., 12.","DOI":"10.3389\/fncom.2018.00042"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"799","DOI":"10.1109\/TFUZZ.2005.859311","article-title":"NFI: A neuro-fuzzy inference method for transductive reasoning","volume":"13","author":"Song","year":"2005","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s12530-024-09630-4","article-title":"NeuDen: A framework for the integration of neuromorphic evolving spiking neural networks","volume":"16","author":"AbouHassan","year":"2025","journal-title":"Evol. Syst."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"2486","DOI":"10.1038\/s41598-021-81805-4","article-title":"Brain-inspired spiking neural networks for decoding muscle activity","volume":"11","author":"Kumarasinghe","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1007\/s12530-024-09628-y","article-title":"ePAMeT: Evolving predictive associative memory for time series","volume":"16","author":"AbouHassan","year":"2025","journal-title":"Evol. Syst."},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Gong, Y., Chung, Y.-A., and Glass, J. (2021). AST: Audio spectrogram transformer. arXiv.","DOI":"10.21437\/Interspeech.2021-698"},{"key":"ref_73","unstructured":"(2026, March 05). NeuCubePy. Available online: https:\/\/github.com\/KEDRI-AUT\/NeuCube-Py."},{"key":"ref_74","unstructured":"Kalkhorani, A., Kumar, A., Tan, K., Buye Xu, B., and Wang, D.L. (2023, January 20\u201324). Time-domain Transformer-based Audiovisual Speaker Separation. Proceedings of the Vahid NTERSPEECH 2023, Dublin, Ireland."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"100768","DOI":"10.1016\/j.cosrev.2025.100768","article-title":"Transformers in speech processing: Overcoming challenges and paving the future","volume":"58","author":"Latif","year":"2025","journal-title":"Comput. Sci. Rev."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"104956","DOI":"10.1016\/j.dsp.2024.104956","article-title":"Transformers and audio detection tasks: An overview","volume":"158","author":"Zaman","year":"2025","journal-title":"Digit. Signal Process."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"124159","DOI":"10.1016\/j.eswa.2024.124159","article-title":"Audio\u2013visual speech recognition based on regulated transformer and spatio\u2013temporal fusion strategy for driver assistive systems","volume":"252","author":"Ryumin","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_78","doi-asserted-by":"crossref","unstructured":"Zhang, K., Zhao, K., and Tian, Y. (2024). Temporal\u2013Semantic Aligning and Reasoning Transformer for Audio-Visual Zero-Shot Learning. Mathematics, 12.","DOI":"10.3390\/math12142200"},{"key":"ref_79","unstructured":"Lee, J., Chang, J., Lee, D., and Choi, J. (2025). Cross-Attention in Audio, Space, and Time for Holistic Video Recognition. arXiv."},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Kim, J.-H., and Won, C.S. (2024). Audio-Visual Action Recognition Using Transformer Fusion Network. Appl. Sci., 14.","DOI":"10.3390\/app14031190"},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Che, N., Zhu, Y., Wang, H., Zeng, X., and Du, Q. (2025). AFT-SAM: Adaptive Fusion Transformer with a Sparse Attention Mechanism for Audio\u2013Visual Speech Recognition. Appl. Sci., 15.","DOI":"10.3390\/app15010199"},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Gwun, W., Choi, K., and Park, G.H. (2025). Multi-Scale Self-Attention-Based Convolutional-Neural-Network Post-Filtering for AV1 Codec: Towards Enhanced Visual Quality and Overall Coding Performance. Mathematics, 13.","DOI":"10.3390\/math13111782"},{"key":"ref_83","doi-asserted-by":"crossref","unstructured":"Wang, Y., Cheung, L., Ma, P., Lee, H., and Lau, A.S.M. (2025). An Emotional AI Chatbot Using an Ontology and a Novel Audiovisual Emotion Transformer for Improving Nonverbal Communication. Electronics, 14.","DOI":"10.3390\/electronics14214304"},{"key":"ref_84","doi-asserted-by":"crossref","unstructured":"Lee, J.-H., Kim, J.-Y., and Kim, H.-G. (2024). Emotion Recognition Using EEG Signals and Audiovisual Features with Contrastive Learning. Bioengineering, 11.","DOI":"10.3390\/bioengineering11100997"},{"key":"ref_85","doi-asserted-by":"crossref","unstructured":"Ma, F., Zhang, W., Li, Y., Huang, S.-L., and Zhang, L. (2020). Learning Better Representations for Audio-Visual Emotion Recognition with Common Information. Appl. Sci., 10.","DOI":"10.3390\/app10207239"},{"key":"ref_86","doi-asserted-by":"crossref","unstructured":"Moorthy, S., and Moon, Y.-K. (2025). Hybrid Multi-Attention Network for Audio\u2013Visual Emotion Recognition Through Multimodal Feature Fusion. Mathematics, 13.","DOI":"10.3390\/math13071100"},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1038\/s41531-025-00913-4","article-title":"Speech and language biomarkers for Parkinson\u2019s disease prediction","volume":"11","author":"Cao","year":"2025","journal-title":"npj Park. Dis."},{"key":"ref_88","unstructured":"(2026, March 05). NeuCom. Available online: https:\/\/kedri.aut.ac.nz\/research-groups\/data-mining-and-big-data-group\/neucom-a-neuro-computing-decision-support-enviroment."}],"container-title":["Biomimetics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-7673\/11\/3\/208\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T13:14:32Z","timestamp":1773753272000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-7673\/11\/3\/208"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,14]]},"references-count":88,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["biomimetics11030208"],"URL":"https:\/\/doi.org\/10.3390\/biomimetics11030208","relation":{},"ISSN":["2313-7673"],"issn-type":[{"value":"2313-7673","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,14]]}}}