{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T21:38:08Z","timestamp":1773524288517,"version":"3.50.1"},"reference-count":40,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2015,1,14]],"date-time":"2015-01-14T00:00:00Z","timestamp":1421193600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The classification of emotional speech is mostly considered in speech-related research on human-computer interaction (HCI). In this paper, the purpose is to present a novel feature extraction based on multi-resolutions texture image information (MRTII). The MRTII feature set is derived from multi-resolution texture analysis for characterization and classification of different emotions in a speech signal. The motivation is that we have to consider emotions have different intensity values in different frequency bands. In terms of human visual perceptual, the texture property on multi-resolution of emotional speech spectrogram should be a good feature set for emotion classification in speech. Furthermore, the multi-resolution analysis on texture can give a clearer discrimination between each emotion than uniform-resolution analysis on texture. In order to provide high accuracy of emotional discrimination especially in real-life, an acoustic activity detection (AAD) algorithm must be applied into the MRTII-based feature extraction. Considering the presence of many blended emotions in real life, in this paper make use of two corpora of naturally-occurring dialogs recorded in real-life call centers. Compared with the traditional Mel-scale Frequency Cepstral Coefficients (MFCC) and the state-of-the-art features, the MRTII features also can improve the correct classification rates of proposed systems among different language databases. Experimental results show that the proposed  MRTII-based feature information inspired by human visual perception of the spectrogram image can provide significant classification for real-life emotional recognition in speech.<\/jats:p>","DOI":"10.3390\/s150101458","type":"journal-article","created":{"date-parts":[[2015,1,14]],"date-time":"2015-01-14T10:33:15Z","timestamp":1421231595000},"page":"1458-1478","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Time-Frequency Feature Representation Using Multi-Resolution Texture Analysis and Acoustic Activity Detector for Real-Life Speech Emotion Recognition"],"prefix":"10.3390","volume":"15","author":[{"given":"Kun-Ching","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Information Technology & Communication, Shih Chien University,  200 University Road, Neimen, Kaohsiung 84550, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2015,1,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1109\/5254.867909","article-title":"Humanoid robots: A new kind of tool","volume":"15","author":"Adams","year":"2000","journal-title":"IEEE Intell. Syst. Their Appl."},{"key":"ref_2","unstructured":"Kim, E., Hyun, K., Kim, S., and Kwak, Y. (2007, January 4\u20137). Emotion interactive robot focus on speaker independently emotion recognition. Zurich, Switzerland."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/79.911197","article-title":"Emotion Recognition in Human-Computer Interaction","volume":"18","author":"Cowie","year":"2001","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_4","first-page":"222","article-title":"Emotion recognition in speech signal: Experimental study, development, and application","volume":"2000","author":"Petrushin","year":"2000","journal-title":"Proc. ICSLP"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Schuller, B., Rigoll, G., and Lang, M. (2003, January 6\u201310). Hidden Markov Model-based Speech Emotion Recognition. Hong Kong, China.","DOI":"10.1109\/ICME.2003.1220939"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kwon, O.W., Chan, K., Hao, J., and Lee, T.W. (2003, January 1\u20134). Emotion Recognition by Speech Signals. Geneva, Switzerland.","DOI":"10.21437\/Eurospeech.2003-80"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"122","DOI":"10.5391\/IJFIS.2002.2.2.122","article-title":"Emotion Recognition based on Frequency Analysis of Speech Signal","volume":"2","author":"Park","year":"2002","journal-title":"Int. J. Fuzzy Log. Intell. Syst."},{"key":"ref_8","unstructured":"Le, X.H., Quenot, G., and Castelli, E. (July, January 28). Recognizing Emotions for the Audio-Visual Document Indexing. Alexandria, Egypt."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1016\/S0167-6393(03)00099-2","article-title":"Speech Emotion Recognition Using Hidden Markov Models","volume":"41","author":"Nwe","year":"2003","journal-title":"Speech Commun."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Yacoub, S., Simske, S., Lin, X., and Burns, J. (2003, January 1\u20134). Recognition of Emotions in Interactive Voice Response Systems. Geneva, Switzerland.","DOI":"10.21437\/Eurospeech.2003-307"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Tato, R.S., Kompe, R., and Pardo, J.M. (2002, January 16\u201320). Emotional Space Improves Emotion Recognition. Denver, CO, USA.","DOI":"10.21437\/ICSLP.2002-557"},{"key":"ref_12","unstructured":"Nwe, T.L., Wei, F.S., and de Silva, L.C. (2001, January 19\u201322). Speech based emotion classification."},{"key":"ref_13","first-page":"316","article-title":"Speech Emotion Recognition using Hidden Markov Model and Support Vector Machine","volume":"1","author":"Ingale","year":"2012","journal-title":"Int. J. Adv. Eng. Res. Stud."},{"key":"ref_14","first-page":"416","article-title":"Methods for capturing spectrotemporal modulations in automatic speech recognition","volume":"8","author":"Kleinschmidt","year":"2001","journal-title":"Acta Acust."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/S0167-6393(02)00058-4","article-title":"Sub-band SNR estimation using auditory feature processing","volume":"39","author":"Kleinschmidt","year":"2003","journal-title":"Speech Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"887","DOI":"10.1121\/1.1945807","article-title":"Multiresolution spectrotemporal analysis of complex sounds","volume":"118","author":"Chih","year":"2005","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_17","unstructured":"Ezzat, T., and Tomaso Poggio, T. (2008, January 22\u201326). Discriminative Word-Spotting Using Ordered Spectro-Temporal Patch Features. Brisbane, Australia."},{"key":"ref_18","unstructured":"Bouvrie, J., Ezzat, T., and Poggio, T. (April, January 31). Localized Spectro-Temporal Cepstral Analysis of Speech. LasVegas, NV, USA."},{"key":"ref_19","unstructured":"Ezzat, T., Bouvrie, J., and Poggio, T. Spectro-Temporal Analysis of Speech Using 2-D Gabor Filters. Antwerp, Belgium."},{"key":"ref_20","unstructured":"Quatieri, T.F. (2002). Discrete-Time Speech Signal Processing, Prentice Hall PTR."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, L., Lech, M., Maddage, N., and Allen, N. (2009, January 11\u201313). Emotion Recognition in Speech of Parents of Depressed Adolescents. Beijing, China.","DOI":"10.1109\/ICBBE.2009.5162771"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"He, L., Lech, M., Memon, S., and Allen, N. (2008, January 22\u201326). Recognition of Stress in Speech Using Wavelet Analysis and Teager Energy Operator. Brisbane, Australia.","DOI":"10.21437\/Interspeech.2008-194"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Laws, K.I. (1980). Textured Image Segmentation. [Ph.D. Thesis, Image Processing Institute, University of Southern California].","DOI":"10.21236\/ADA083283"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1513","DOI":"10.1016\/S0167-8655(02)00390-2","article-title":"Texture classification using wavelet transform","volume":"24","author":"Arivazhagan","year":"2003","journal-title":"Pattern Recognit. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1109\/83.242353","article-title":"Texture analysis and classification with tree-structured wavelet transform","volume":"2","author":"Chang","year":"1993","journal-title":"IEEE Trans. Image Proc."},{"key":"ref_26","unstructured":"Rabiner, L., and Juang, B.-H. (1993). Fundamentals of Speech Recognition, Prentice Hall."},{"key":"ref_27","unstructured":"Yeh, J.H. (2004). Emotion Recognition from Mandarin Speech Signals. [Master's Thesis, Tatung University]."},{"key":"ref_28","first-page":"949","article-title":"A Wavelet-Based Voice Activity Detection Algorithm in Variable-Level Noise Environment","volume":"6","author":"Wang","year":"2009","journal-title":"WSEAS Trans. Comput."},{"key":"ref_29","unstructured":"Zwicker, E., and Fastl, H. (1990). Psychoacoustics: Facts and Models, Springer-Verlag."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2091","DOI":"10.1109\/29.45554","article-title":"Multifrequency channel decomposition of images and wavelet model","volume":"37","author":"Mallat","year":"1989","journal-title":"IEEE Trans. Acoust. Speech Signal Process."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1109\/89.861373","article-title":"Word boundary detection with mel-scale frequency bank in noise environment","volume":"8","author":"Wu","year":"2000","journal-title":"IEEE Trans. Speech Audio Process."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1109\/LSP.2010.2100380","article-title":"Spectrogram image feature for sound event classification in mismatched conditions","volume":"18","author":"Dennis","year":"2011","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_33","first-page":"2","article-title":"Using Fuzzy Inference and Cubic Curve to Detect and Compensate Backlight Image","volume":"8","author":"Lin","year":"2006","journal-title":"Int. J. Fuzzy Syst."},{"key":"ref_34","unstructured":"Berlin Database of Emotional Speech. Available online: http:\/\/pascal.kgw.tu-berlin.de\/emodb\/."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Martin, O., Kotsia, I., and Macq, B. (2006, January 3\u20137). The eNTERFACE'05 Audio-visual Emotion Database. Los Alamitos, CA, USA.","DOI":"10.1109\/ICDEW.2006.145"},{"key":"ref_36","unstructured":"Jiang, D.-N., and Cai, L.-H. (2004, January 27\u201330). Speech Emotion Classification with the Combination of Statistic Features and Temporal Features."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Schuller, B., Steidl, S., and Batliner, A. (2009, January 7). The INTERSPEESH 2009 Emotion Challenge.","DOI":"10.21437\/Interspeech.2009-103"},{"key":"ref_38","unstructured":"Eyben, F., Wollmer, M., and Schuller, B. (2009): Speech and Music Interpretation by Large-Space Extraction. Available online: http:\/\/sourceforge.net\/projects\/openSMILE."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2348","DOI":"10.1109\/TSP.2004.831018","article-title":"Application of Support Vector Machines to Speech Recognition","volume":"52","author":"Ganapathiraju","year":"2004","journal-title":"IEEE Trans, Signal Process."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Alexandre-Cortizo, E., Rosa-Zurera, M., and Lopez-Ferreras, F. (2005, January 21\u201324). Application of Fisher Linear Discriminant Analysis to Speech\/Music Classification. Belgrade, Yugoslavia.","DOI":"10.1109\/EURCON.2005.1630291"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/15\/1\/1458\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:41:36Z","timestamp":1760215296000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/15\/1\/1458"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,1,14]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2015,1]]}},"alternative-id":["s150101458"],"URL":"https:\/\/doi.org\/10.3390\/s150101458","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,1,14]]}}}