{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:38:59Z","timestamp":1760236739699,"version":"build-2065373602"},"reference-count":57,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,12,22]],"date-time":"2021-12-22T00:00:00Z","timestamp":1640131200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program","doi-asserted-by":"publisher","award":["2019YFB1804003"],"award-info":[{"award-number":["2019YFB1804003"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Acoustic scene classification (ASC) tries to inference information about the environment using audio segments. The inter-class similarity is a significant issue in ASC as acoustic scenes with different labels may sound quite similar. In this paper, the similarity relations amongst scenes are correlated with the classification error. A class hierarchy construction method by using classification error is then proposed and integrated into a multitask learning framework. The experiments have shown that the proposed multitask learning method improves the performance of ASC. On the TUT Acoustic Scene 2017 dataset, we obtain the ensemble fine-grained accuracy of 81.4%, which is better than the state-of-the-art. By using multitask learning, the basic Convolutional Neural Network (CNN) model can be improved by about 2.0 to 3.5 percent according to different spectrograms. The coarse category accuracies (for two to six super-classes) range from 77.0% to 96.2% by single models. On the revised version of the LITIS Rouen dataset, we achieve the ensemble fine-grained accuracy of 83.9%. The multitask learning models obtain an improvement of 1.6% to 1.8% compared to their basic models. The coarse category accuracies range from 94.9% to 97.9% for two to six super-classes with single models.<\/jats:p>","DOI":"10.3390\/s22010036","type":"journal-article","created":{"date-parts":[[2021,12,23]],"date-time":"2021-12-23T02:02:57Z","timestamp":1640224977000},"page":"36","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Clustering by Errors: A Self-Organized Multitask Learning Method for Acoustic Scene Classification"],"prefix":"10.3390","volume":"22","author":[{"given":"Weiping","family":"Zheng","sequence":"first","affiliation":[{"name":"School of Computer Science, South China Normal University, Guangzhou 510631, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhenyao","family":"Mo","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gansen","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Computer Science, South China Normal University, Guangzhou 510631, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1109\/MSP.2014.2326181","article-title":"Acoustic Scene Classification: Classifying environments from the sounds they produce","volume":"32","author":"Barchiesi","year":"2015","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1733","DOI":"10.1109\/TMM.2015.2428998","article-title":"Detection and classification of acoustic scenes and events","volume":"17","author":"Stowell","year":"2015","journal-title":"IEEE Trans. Multimed."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/MCOM.2018.1700577","article-title":"Environment classification for urban big data using deep learning","volume":"56","author":"Hossain","year":"2018","journal-title":"IEEE Commun. Mag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1250\/ast.39.182","article-title":"Incorporating Intra-Class Variance to Fine-Grained Visual Recognition","volume":"39","author":"Imoto","year":"2018","journal-title":"Acoust. Sci. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Em, Y., Gag, F., Lou, Y., Wang, S., Huang, T., and Duan, L.-Y. (2017, January 10\u201314). Incorporating Intra-Class Variance to Fine-Grained Visual Recognition. Proceedings of the 2017 IEEE International Conference on Multimedia and Expo (ICME), Hong Kong, China.","DOI":"10.1109\/ICME.2017.8019371"},{"key":"ref_6","unstructured":"Mesaros, A., Heittola, T., Diment, A., Elizalde, B., Shah, A., Vincent, E., Raj, B., and Virtanen, T. (2017, January 16\u201317). DCASE 2017 challenge setup: Tasks, datasets and baseline system. Proceedings of the DCASE 2017-Workshop on Detection and Classification of Acoustic Scenes and Events, Munich, Germany."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1109\/TETC.2017.2700843","article-title":"Audio Data Mining for Anthropogenic Disaster Identification: An Automatic Taxonomy Approach","volume":"8","author":"Ye","year":"2017","journal-title":"IEEE Trans. Emerg. Top. Comput."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1385","DOI":"10.1109\/TMM.2019.2947199","article-title":"Acoustic Scene Clustering Using Joint Optimization of Deep Embedding Learning and Clustering Iteration","volume":"22","author":"Li","year":"2019","journal-title":"IEEE Trans. Multimed."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Tonami, N., Imoto, K., Niitsuma, M., Yamanishi, R., and Yamashita, Y. (2019, January 20\u201323). Joint analysis of acoustic events and scenes based on multitask learning. Proceedings of the 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA.","DOI":"10.1109\/WASPAA.2019.8937196"},{"key":"ref_10","first-page":"1964","article-title":"Learning Hierarchy Aware Embedding from Raw Audio for Acoustic Scene Classification","volume":"28","author":"Abrol","year":"2020","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"key":"ref_11","first-page":"142","article-title":"Histogram of gradients of time-frequency representations for audio scene classification","volume":"23","author":"Rakotomamonjy","year":"2014","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Geiger, J.T., Schuller, B., and Rigoll, G. (2013, January 20\u201323). Large-scale audio feature extraction and SVM for acoustic scene classification. Proceedings of the 2013 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, NY, USA.","DOI":"10.1109\/WASPAA.2013.6701857"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1149290.1149292","article-title":"Acoustic environment classification","volume":"3","author":"Ma","year":"2006","journal-title":"ACM Trans. Speech Lang. Process. TSLP"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Chakrabarty, D., and Elhilali, M. (2015, January 18\u201321). Exploring the role of temporal dynamics in acoustic scene classification. Proceedings of the 2015 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA.","DOI":"10.1109\/WASPAA.2015.7336898"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1109\/JAS.2017.7510508","article-title":"Recent progresses in deep learning based acoustic models","volume":"4","author":"Yu","year":"2017","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_16","unstructured":"Eghbal-Zadeh, H., Lehner, B., Dorfer, M., and Widmer, G. (2016, January 3). CP-JKU submission for DCASE-2016: A hybrid approach using binaural i-vectors and deep convolutional neural networks. Proceedings of the IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events (DCASE), Budapest, Hungary."},{"key":"ref_17","unstructured":"Weiping, Z., Jiantao, Y., Xiaotao, X., Xiangtao, L., and Shaohu, P. (2017, January 16\u201317). Acoustic scene classification using deep convolutional neural network and multiple spectrograms fusion. Proceedings of the Detection and Classification of Acoustic Scenes and Events (DCASE), Munich, Germany."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Xu, K., Feng, D., Mi, H., Zhu, B., Wang, D., Zhang, L., Cai, H., and Liu, S. (2018, January 21\u201322). Mixup-based acoustic scene classification using multi-channel convolutional neural network. Proceedings of the Pacific Rim Conference on Multimedia, Hefei, China.","DOI":"10.1007\/978-3-030-00764-5_2"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Hershey, S., Chaudhuri, S., Ellis, D.P., Gemmeke, J.F., Jansen, A., Moore, R.C., Plakal, M., Platt, D., Saurous, R.A., and Seybold, B. (2017, January 5\u20139). CNN architectures for large-scale audio classification. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.","DOI":"10.1109\/ICASSP.2017.7952132"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Phan, H., Koch, P., Katzberg, F., Maass, M., Mazur, R., and Mertins, A. (2017, January 20\u201324). Audio scene classification with deep recurrent neural networks. Proceedings of the INTERSPEECH, Stockholm, Sweden.","DOI":"10.21437\/Interspeech.2017-101"},{"key":"ref_21","unstructured":"Bae, S.H., Choi, I., and Kim, N.S. (2016, January 3). Acoustic scene classification using parallel combination of LSTM and CNN. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2016 Workshop (DCASE2016), Budapest, Hungary."},{"key":"ref_22","unstructured":"Xu, Y., Huang, Q., Wang, W., and Plumbley, M.D. (2016, January 3). Hierarchical learning for DNN-based acoustic scene classification. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2016 Workshop (DCASE2016), Budapest, Hungary."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Guo, J., Xu, N., Li, L.-J., and Alwan, A. (2017, January 20\u201324). Attention based CLDNNs for short-duration acoustic scene classification. Proceedings of the INTERSPEECH, Stockholm, Sweden.","DOI":"10.21437\/Interspeech.2017-440"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1109\/TASL.2010.2064307","article-title":"Front-end factor analysis for speaker verification","volume":"19","author":"Dehak","year":"2010","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhang, X., Zhou, F., Lin, Y., and Zhang, S. (2016, January 27\u201330). Embedding label structures for fine-grained feature representation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.126"},{"key":"ref_27","unstructured":"Sohn, K. (2016, January 5\u201310). Improved deep metric learning with multi-class n-pair loss objective. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Xie, S., Yang, T., Wang, X., and Lin, Y. (2015, January 7\u201312). Hyper-class augmented and regularized deep learning for fine-grained image classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298880"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wu, H., Merler, M., Uceda-Sosa, R., and Smith, J.R. (2016, January 15\u201319). Learning to make better mistakes: Semantics-aware visual food recognition. Proceedings of the 24th ACM International Conference on Multimedia, Amsterdam The Netherlands.","DOI":"10.1145\/2964284.2967205"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Imoto, K., Tonami, N., Koizumi, Y., Yasuda, M., Yamanishi, R., and Yamashita, Y. (2020, January 4\u20138). Sound event detection by multitask learning of sound events and scenes with soft scene labels. Proceedings of the ICASSP 2020\u20132020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9053912"},{"key":"ref_31","unstructured":"Mun, S., Park, S., Han, D.K., and Ko, H. (2017, January 16\u201317). Generative adversarial network based acoustic scene training set augmentation and selection using SVM hyper-plane. Proceedings of the DCASE 2017\u2013Detection and Classification of Acoustic Scenes and Events Workshop, Munich, Germany."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1109\/LSP.2017.2657381","article-title":"Deep convolutional neural networks and data augmentation for environmental sound classification","volume":"24","author":"Salamon","year":"2017","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lu, R., Duan, Z., and Zhang, C. (2017, January 15\u201318). Metric learning based data augmentation for environmental sound classification. Proceedings of the 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA.","DOI":"10.1109\/WASPAA.2017.8169983"},{"key":"ref_34","first-page":"2672","article-title":"Generative Adversarial Networks","volume":"3","author":"Goodfellow","year":"2014","journal-title":"Proc. Adv. Neural Inf. Process. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"13001","DOI":"10.1609\/aaai.v34i07.7000","article-title":"Random Erasing Data Augmentation","volume":"34","author":"Zhong","year":"2020","journal-title":"AAAI"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Gharib, S., Derrar, H., Niizumi, D., Senttula, T., Tommola, J., Heittola, T., Virtanen, T., and Huttunen, H. (2018, January 17\u201320). Acoustic scene classification: A competition review. Proceedings of the 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP), Aalborg, Denmark.","DOI":"10.1109\/MLSP.2018.8517000"},{"key":"ref_37","unstructured":"Zhang, H., Cisse, M., Dauphin, Y.N., and Lopez-Paz, D. (May, January 30). Mixup: Beyond empirical risk minimization. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"986","DOI":"10.1109\/TASSP.1983.1164162","article-title":"Signal Reconstruction from Short-Time Fourier Transform Magnitude","volume":"31","author":"Nawa","year":"1983","journal-title":"IEEE Trans. Acoust. Speech Signal Process."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2698","DOI":"10.1121\/1.404385","article-title":"An efficient algorithm for the calculation of a constant Q transform","volume":"92","author":"Brown","year":"1992","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_40","unstructured":"Logan, B. (2000, January 23\u201325). Mel frequency cepstral coefficients for music modeling. Proceedings of the In International Symposium on Music Information Retrieval, Montr\u00e9al, QC, Canada."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.dsp.2014.12.015","article-title":"Time-frequency features for pattern recognition using high-resolution TFDs: A tutorial review","volume":"40","author":"Boashash","year":"2015","journal-title":"Digit. Signal Process."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"107568","DOI":"10.1016\/j.apacoust.2020.107568","article-title":"Late fusion framework for Acoustic Scene Classification using LPCC, SCMC, and log-Mel band energies with Deep Neural Networks","volume":"172","author":"Paseddula","year":"2021","journal-title":"Appl. Acoust."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Mcdonnellk, M.D., and Gao, W. (2020, January 4\u20138). Acoustic Scene Classification Using Deep Residual Networks with Late Fusion of Separated High and Low Frequency Paths. Proceedings of the ICASSP 2020\u20132020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Virtual Conference.","DOI":"10.1109\/ICASSP40776.2020.9053274"},{"key":"ref_44","unstructured":"Ng, A., Jordan, M., and Weiss, Y. (2002). On spectral clustering: Analysis and an algorithm. Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_45","unstructured":"Gopal, S., and Yang, Y. (August, January USA). Recursive regularization for large-scale classification with hierarchical and graphical dependencies. Proceedings of the 19th ACM SIGKDD\u2013International Conference on Knowledge Discovery and Data Mining, Chicago, IL."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1253","DOI":"10.1109\/TASLP.2017.2690561","article-title":"Supervised representation learning for audio scene classification","volume":"25","author":"Rakotomamonjy","year":"2017","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"key":"ref_49","unstructured":"Abadi, M., Agarwal, A., Barham, P., Barham, P., Brevdo, E., Chen, Z., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2016, January 2\u20134). TensorFlow: Large-scale machine learning on heterogeneous systems. Proceedings of the Operating Systems Design and Implementation, Savannah, GA, USA."},{"key":"ref_50","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_51","unstructured":"Han, Y., Park, J., and Lee, K. (2017, January 16\u201317). Convolutional neural networks with binaural representations and background subtraction for acoustic scene classification. Proceedings of the DCASE 2017\u2013Detection and Classification of Acoustic Scenes and Events Workshop, Munich, Germany."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"107829","DOI":"10.1016\/j.apacoust.2020.107829","article-title":"A novel acoustic scene classification model using the late fusion of convolutional neural networks and different ensemble classifiers","volume":"175","author":"Alamir","year":"2021","journal-title":"Appl. Acoust."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Chen, H., Zhang, P., and Yan, Y. (2019, January 12\u201317). An audio scene classification framework with embedded filters and a DCT-based temporal module. Proceedings of the ICASSP 2019\u20132019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8683636"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Wu, Y., and Lee, T. (2019, January 12\u201317). Enhancing sound texture in CNN-based acoustic scene classification. Proceedings of the ICASSP 2019\u20132019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8683490"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"102943","DOI":"10.1016\/j.dsp.2020.102943","article-title":"Robust acoustic scene classification using a multi-spectrogram encoder-decoder framework","volume":"110","author":"Pham","year":"2021","journal-title":"Digit. Signal Process."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Lee, Y.J., and Grauman, K. (2011, January 20\u201325). Learning the easy things first: Self-paced visual category discovery. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995523"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1007379606734","article-title":"Multitask learning","volume":"28","author":"Caruana","year":"1997","journal-title":"Mach. Learn."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/1\/36\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:51:08Z","timestamp":1760169068000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/1\/36"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,22]]},"references-count":57,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["s22010036"],"URL":"https:\/\/doi.org\/10.3390\/s22010036","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,12,22]]}}}