{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T06:30:49Z","timestamp":1760596249368,"version":"build-2065373602"},"reference-count":37,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2014,9,9]],"date-time":"2014-09-09T00:00:00Z","timestamp":1410220800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this paper, we present a novel texture image feature for Emotion Sensing in Speech (ESS). This idea is based on the fact that the texture images carry emotion-related information. The feature extraction is derived from time-frequency representation of spectrogram images. First, we transform the spectrogram as a recognizable image. Next, we use a cubic curve to enhance the image contrast. Then, the texture image information (TII) derived from the spectrogram image can be extracted by using Laws\u2019 masks to characterize emotional state. In order to evaluate the effectiveness of the proposed emotion recognition in different languages, we use two open emotional databases including the Berlin Emotional Speech Database (EMO-DB) and eNTERFACE corpus and one  self-recorded database (KHUSC-EmoDB), to evaluate the performance cross-corpora. The results of the proposed ESS system are presented using support vector machine (SVM) as a classifier. Experimental results show that the proposed TII-based feature extraction inspired by visual perception can provide significant classification for ESS systems. The two-dimensional (2-D) TII feature can provide the discrimination between different emotions in visual expressions except for the conveyance pitch and formant tracks.  In addition, the de-noising in 2-D images can be more easily completed than de-noising  in 1-D speech.<\/jats:p>","DOI":"10.3390\/s140916692","type":"journal-article","created":{"date-parts":[[2014,9,9]],"date-time":"2014-09-09T09:51:28Z","timestamp":1410256288000},"page":"16692-16714","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["The Feature Extraction Based on Texture Image Information for Emotion Sensing in Speech"],"prefix":"10.3390","volume":"14","author":[{"given":"Kun-Ching","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Information Technology & Communication, Shih Chien University, 200 University Road, Neimen, Kaohsiung 84550, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2014,9,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1109\/TPAMI.2013.134","article-title":"Neighborhood Repulsed Metric Learning for Kinship Verification","volume":"36","author":"Lu","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1559","DOI":"10.1109\/TPAMI.2013.2296528","article-title":"Large-Margin Multi-View Information Bottleneck","volume":"36","author":"Xu","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/TPAMI.2012.70","article-title":"Discriminative Multimanifold Analysis for Face Recognition from a Single Training Sample per Person","volume":"35","author":"Lu","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1109\/5254.867909","article-title":"Humanoid robots: A new kind of tool","volume":"15","author":"Adams","year":"2000","journal-title":"IEEE Intell. Syst. Their Appl."},{"key":"ref_5","unstructured":"Kim, E., Hyun, K., Kim, S., and Kwak, Y. (2007, January 4\u20137). Emotion interactive robot focus on speaker independently emotion recognition. Zurich, Switzerland."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/79.911197","article-title":"Emotion Recognition in Human-Computer Interaction","volume":"18","author":"Cowie","year":"2001","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Petrushin, V.A. (2000, January 16\u201320). Emotion recognition in speech signal: Experimental study, development, and application. Beijing, China.","DOI":"10.21437\/ICSLP.2000-791"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1243","DOI":"10.1109\/TASL.2009.2031793","article-title":"Speech Enhancement with Inventory Style Speech Resynthesis","volume":"18","author":"Xiao","year":"2010","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Schuller, B., Rigoll, G., and Lang, M. (2003, January 6\u201310). Hidden Markov Model-based Speech Emotion Recognition. Hong Kong, China.","DOI":"10.1109\/ICME.2003.1220939"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"122","DOI":"10.5391\/IJFIS.2002.2.2.122","article-title":"Emotion Recognition based on Frequency Analysis of Speech Signal","volume":"2","author":"Park","year":"2002","journal-title":"Int. J. Fuzzy Log. Intell. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yacoub, S., Simske, S., Lin, X., and Burns, J. (2003, January 1\u20134). Recognition of Emotions in Interactive Voice Response Systems. Geneva, Switzerland.","DOI":"10.21437\/Eurospeech.2003-307"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Tato, R.S., Kompe, R., and Pardo, J.M. (2002, January 16\u201320). Emotional Space Improves Emotion Recognition. Denver, CO, USA.","DOI":"10.21437\/ICSLP.2002-557"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kwon, O.W., Chan, K., Hao, J., and Lee, T.W. (2003, January 1\u20134). Emotion Recognition by Speech Signals. Geneva, Switzerland.","DOI":"10.21437\/Eurospeech.2003-80"},{"key":"ref_14","unstructured":"Le, X.H., Quenot, G., and Castelli, E. (July, January 28). Recognizing Emotions for the Audio-Visual Document Indexing. Alexandria, Egypt."},{"key":"ref_15","unstructured":"New, T.L., Wei, F.S., and De Silva, L.C. (2001, January 19\u201322). Speech based emotion classification. Hong Kong, China."},{"key":"ref_16","first-page":"416","article-title":"Methods for capturing spectrotemporal modulations in automatic speech recognition","volume":"8","author":"Kleinschmidt","year":"2001","journal-title":"Acta Acust."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/S0167-6393(02)00058-4","article-title":"Sub-band SNR estimation using auditory feature processing","volume":"39","author":"Kleinschmidt","year":"2003","journal-title":"Speech Commun."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"887","DOI":"10.1121\/1.1945807","article-title":"Multiresolution spectrotemporal analysis of complex sounds","volume":"118","author":"Chih","year":"2005","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_19","unstructured":"Ezzat, T., and Poggio, T. (2008, January 21). Discriminative Word-Spotting Using Ordered Spectro-Temporal Patch Features. Brisbane, Australia."},{"key":"ref_20","unstructured":"Bouvrie, J., Ezzat, T., and Poggio, T. (April, January 30). Localized Spectro-Temporal Cepstral Analysis of Speech. LasVegas, NV, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ezzat, T., Bouvrie, J., and Poggio, T. (2007, January 27\u201331). Spectro-Temporal Analysis of Speech Using 2-D Gabor Filters. Antwerp, Belgium.","DOI":"10.21437\/Interspeech.2007-236"},{"key":"ref_22","unstructured":"Vogt, T., and Andre, E. (2005, January 6). Comparing feature sets for acted and spontaneous speech in view of automatic emotion recognition. Amsterdam, The Netherlands."},{"key":"ref_23","unstructured":"Schuller, B., Wimmer, M., Mosenlechner, L., Kern, C., Arsic, D., and Rigoll, G. (April, January 30). Brute-forcing hierarchical functionals for paralinguistics: A waste of feature space. Las Vegas, NV, USA."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Pachet, F., and Roy, P. (2009). Analytical features: A knowledge-based approach to audio feature generation. EURASIP J. Audio Speech Music Process.","DOI":"10.1155\/2009\/153017"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Schuller, B., Reiter, S., and Rigoll, G. (2006, January 9\u201312). Evolutionary feature generation in speech emotion recognition. Toronto, Canada.","DOI":"10.1109\/ICME.2006.262500"},{"key":"ref_26","unstructured":"Marc, E.M. (2012). Emotion Recognition from Speech Signals: Erasmus Exchange Project Work. [Ph.D. Thesis, University of Ljubljana]."},{"key":"ref_27","unstructured":"Berlin Database of Emotional Speech. Available online: http:\/\/pascal.kgw.tu-berlin.de\/emodb\/."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Martin, O., Kotsia, I., and Macq, B. (2006, January 3\u20137). The eNTERFACE'05 Audio-visual Emotion Database. Atlanta, GA, USA.","DOI":"10.1109\/ICDEW.2006.145"},{"key":"ref_29","unstructured":"Quatieri, T.F. (2002). Discrete-Time Speech Signal Processing: Principles and Practice, Pearson Prentice Hall."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"He, L., Lech, M., Maddage, N., and Allen, N. (2009, January 11\u201313). Emotion Recognition in Speech of Parents of Depressed Adolescents. Beijing, China.","DOI":"10.1109\/ICBBE.2009.5162771"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"He, L., Lech, M., Maddage, N., Memon, S., and Allen, N. (2009, January 11\u201313). Emotion Recognition in Spontaneous Speech within Work and Family Environments. Beijing, China.","DOI":"10.1109\/ICBBE.2009.5162772"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"He, L., Lech, M., Memon, S., and Allen, N. (2008, January 22\u201326). Recognition of Stress in Speech Using Wavelet Analysis and Teager Energy Operator. Brisbane, Australia.","DOI":"10.21437\/Interspeech.2008-194"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1109\/LSP.2010.2100380","article-title":"Spectrogram image feature for sound event classification in mismatched conditions","volume":"18","author":"Dennis","year":"2011","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_34","first-page":"2","article-title":"Using Fuzzy Inference and Cubic Curve to Detect and Compensate Backlight Image","volume":"8","author":"Lin","year":"2006","journal-title":"Int. J. Fuzzy Syst."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Laws, K.I. (1980). Textured image segmentation. [Ph.D. Dissertation, University of Southern California].","DOI":"10.21236\/ADA083283"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2348","DOI":"10.1109\/TSP.2004.831018","article-title":"Application of Support Vector Machines to Speech Recognition","volume":"52","author":"Ganapathiraju","year":"2004","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_37","first-page":"316","article-title":"Speech Emotion Recognition using Hidden Markov Model and Support Vector Machine","volume":"1","author":"Dr","year":"2012","journal-title":"Int. J. Adv. Eng. Res. Stud."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/14\/9\/16692\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:15:42Z","timestamp":1760217342000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/14\/9\/16692"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,9,9]]},"references-count":37,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2014,9]]}},"alternative-id":["s140916692"],"URL":"https:\/\/doi.org\/10.3390\/s140916692","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2014,9,9]]}}}