{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:43:41Z","timestamp":1760179421924,"version":"build-2065373602"},"reference-count":37,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2020,9,25]],"date-time":"2020-09-25T00:00:00Z","timestamp":1600992000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Measuring the productivity of an agent in a call center domain is a challenging task. Subjective measures are commonly used for evaluation in the current systems. In this paper, we propose an objective framework for modeling agent productivity for real estate call centers based on speech signal processing. The problem is formulated as a binary classification task using deep learning methods. We explore several designs for the classifier based on convolutional neural networks (CNNs), long-short-term memory networks (LSTMs), and an attention layer. The corpus consists of seven hours collected and annotated from three different call centers. The result shows that the speech-based approach can lead to significant improvements (1.57% absolute improvements) over a robust text baseline system.<\/jats:p>","DOI":"10.3390\/s20195489","type":"journal-article","created":{"date-parts":[[2020,9,25]],"date-time":"2020-09-25T08:57:32Z","timestamp":1601024252000},"page":"5489","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Agent Productivity Modeling in a Call Center Domain Using Attentive Convolutional Neural Networks"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1024-5469","authenticated-orcid":false,"given":"Abdelrahman","family":"Ahmed","sequence":"first","affiliation":[{"name":"Department of Electronics Engineering, University of Seville, 41092 Seville, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2612-0388","authenticated-orcid":false,"given":"Sergio","family":"Toral","sequence":"additional","affiliation":[{"name":"Department of Electronics Engineering, University of Seville, 41092 Seville, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0823-8390","authenticated-orcid":false,"given":"Khaled","family":"Shaalan","sequence":"additional","affiliation":[{"name":"Faculty of Informatics, The British University in Dubai, Dubai 345015, UAE"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5516-7225","authenticated-orcid":false,"given":"Yaser","family":"Hifny","sequence":"additional","affiliation":[{"name":"Faculty of Computer Sciences and Information, Helwan University, Helwan 11795, Egypt"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1007\/s11846-011-0076-3","article-title":"Social ties and subjective performance evaluations: An empirical investigation","volume":"7","author":"Breuer","year":"2013","journal-title":"Rev. Manag. Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4102\/sajhrm.v16i0.905","article-title":"Exploring employee retention and intention to leave within a call center","volume":"16","author":"Dhanpat","year":"2018","journal-title":"SA J. Hum. Resour. Manag."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1016\/j.jebo.2016.12.016","article-title":"Subjective performance evaluations and employee careers","volume":"134","author":"Frederiksen","year":"2017","journal-title":"J. Econ. Behav. Organ."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"797","DOI":"10.1016\/j.indmarman.2005.01.002","article-title":"Cultural vs. operational market orientation and objective vs. subjective performance: Perspective of production and operations","volume":"34","year":"2005","journal-title":"Ind. Mark. Manag."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1080\/08911762.2018.1427293","article-title":"Emotional Exhaustion in Offshore Call Centers: A Comparative Study","volume":"32","author":"Echchakoui","year":"2019","journal-title":"J. Glob. Mark."},{"key":"ref_6","unstructured":"Cleveland, B. (2012). Call Center Management on Fast Forward: Succeeding in the New Era of Customer Relationships, ICMI Press."},{"key":"ref_7","first-page":"3","article-title":"Performance concepts and performance theory","volume":"23","author":"Sonnentag","year":"2002","journal-title":"Psychol. Manag. Individ. Perform."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ahmed, A., Hifny, Y., Toral, S., and Shaalan, K. (2018). A Call Center Agent Productivity Modeling Using Discriminative Approaches. Intelligent Natural Language Processing: Trends and Applications, Springer. Book Section 1.","DOI":"10.1007\/978-3-319-67056-0_24"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ahmed, A., Toral, S., and Shaalan, K. (2016, January 24\u201326). Agent productivity measurement in call center using machine learning. Proceedings of the International Conference on Advanced Intelligent Systems and Informatics, Cairo, Egypt.","DOI":"10.1007\/978-3-319-48308-5_16"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1142\/9789813229396_0011","article-title":"End-to-End Lexicon Free Arabic Speech Recognition Using Recurrent Neural Networks","volume":"4","author":"Ahmed","year":"2018","journal-title":"Comput. Linguist. Speech Image Process. Arab. Lang."},{"key":"ref_11","first-page":"1","article-title":"Feature extraction methods LPC, PLP and MFCC in speech recognition","volume":"1","author":"Dave","year":"2013","journal-title":"Int. J. Adv. Res. Eng. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Sainath, T.N., Vinyals, O., Senior, A., and Sak, H. (2015, January 19\u201324). Convolutional, long short-term memory, fully connected deep neural networks. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia.","DOI":"10.1109\/ICASSP.2015.7178838"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.specom.2015.12.003","article-title":"Maxout neurons for deep convolutional and LSTM neural networks in speech recognition","volume":"77","author":"Cai","year":"2016","journal-title":"Speech Commun."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mishne, G., Carmel, D., Hoory, R., Roytman, A., and Soffer, A. (2005, January 31). Automatic analysis of call-center conversations. Proceedings of the 14th ACM International Conference on Information and knowledge Management, Bremen, Germany.","DOI":"10.1145\/1099554.1099684"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"9939","DOI":"10.1016\/j.eswa.2011.11.126","article-title":"Job performance prediction in a call center using a naive Bayes classifier","volume":"39","author":"Valle","year":"2012","journal-title":"Expert Syst. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ahmed, A., Hifny, Y., Shaalan, K., and Toral, S. (2016, January 24\u201326). Lexicon free Arabic speech recognition recipe. Proceedings of the International Conference on Advanced Intelligent Systems and Informatics, Cairo, Egypt.","DOI":"10.1007\/978-3-319-48308-5_15"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.eswa.2004.08.008","article-title":"A web-based system for analyzing the voices of call center customers in the service industry","volume":"28","author":"Bae","year":"2005","journal-title":"Expert Syst. Appl."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Karakus, B., and Aydin, G. (2016, January 11\u201313). Call center performance evaluation using big data analytics. Proceedings of the 2016 International Symposium on Networks, Computers and Communications (ISNCC), Yasmine Hammamet, Tunisia.","DOI":"10.1109\/ISNCC.2016.7746116"},{"key":"ref_19","first-page":"1","article-title":"Automatic Evaluation Software for Contact Centre Agents\u2019 voice Handling Performance","volume":"5","author":"Perera","year":"2019","journal-title":"Int. J. Sci. Res. Publ."},{"key":"ref_20","first-page":"133","article-title":"Voice call analytics using natural language processing","volume":"4","author":"Sudarsan","year":"2019","journal-title":"Int. J. Stat. Appl. Math."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Neumann, M., and Vu, N.T. (2017). Attentive convolutional neural network based speech emotion recognition: A study on the impact of input features, signal length, and acted speech. arXiv.","DOI":"10.21437\/Interspeech.2017-917"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Hifny, Y., and Ali, A. (2019, January 12\u201317). Efficient Arabic Emotion Recognition Using Deep Neural Networks. Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8683632"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Cho, J., Pappagari, R., Kulkarni, P., Villalba, J., Carmiel, Y., and Dehak, N. (2019). Deep neural networks for emotion recognition combining audio and transcripts. arXiv.","DOI":"10.21437\/Interspeech.2018-2466"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M.A., Schuller, B., and Zafeiriou, S. (2016, January 20\u201325). Adieu features? end-to-end speech emotion recognition using a deep convolutional recurrent network. Proceedings of the 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP), Shanghai, China.","DOI":"10.1109\/ICASSP.2016.7472669"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.sigpro.2017.12.008","article-title":"Recurrent attention network using spatial-temporal relations for action recognition","volume":"145","author":"Zhang","year":"2018","journal-title":"Signal Process."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Broux, P.A., Desnous, F., Larcher, A., Petitrenaud, S., Carrive, J., and Meignier, S. (2018, January 2\u20136). S4D: Speaker Diarization Toolkit in Python. Proceedings of the Interspeech 2018, Hyderabad, India.","DOI":"10.21437\/Interspeech.2018-1232"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2157","DOI":"10.1016\/j.eswa.2011.07.065","article-title":"Classification of speech dysfluencies with MFCC and LPCC features","volume":"39","author":"Ai","year":"2012","journal-title":"Expert Syst. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"9799","DOI":"10.1016\/j.eswa.2009.02.040","article-title":"Unsupervised speaker segmentation with residual phase and MFCC features","volume":"36","author":"Jothilakshmi","year":"2009","journal-title":"Expert Syst. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Palaz, D., Magimai, M., and Collobert, R. (2015, January 6\u201310). Analysis of CNN-Based Speech Recognition System Using Raw Speech as Input. Proceedings of the 16th Annual Conference of International Speech Communication Association (Interspeech), Dresden, Germany.","DOI":"10.21437\/Interspeech.2015-3"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1142\/S0218488598000094","article-title":"The vanishing gradient problem during learning recurrent neural nets and problem solutions","volume":"6","author":"Hochreiter","year":"1998","journal-title":"Int. J. Uncertain. Fuzziness Knowl. Based Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1016\/j.neunet.2005.06.042","article-title":"Framewise phoneme classification with bidirectional LSTM and other neural network architectures","volume":"18","author":"Graves","year":"2005","journal-title":"Neural Netw."},{"key":"ref_32","unstructured":"Gehring, J., Auli, M., Grangier, D., Yarats, D., and Dauphin, Y.N. (2017, January 6\u201311). Convolutional sequence to sequence learning. Proceedings of the 34th International Conference on Machine Learning-Volume 70, Sydney, Australia."},{"key":"ref_33","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, NIPS."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Norouzian, A., Mazoure, B., Connolly, D., and Willett, D. (2019, January 12\u201317). Exploring attention mechanism for acoustic-based classification of speech utterances into system-directed and non-system-directed. Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8683565"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1080\/19312450709336664","article-title":"Answering the call for a standard reliability measure for coding data","volume":"1","author":"Hayes","year":"2007","journal-title":"Commun. Methods Meas."},{"key":"ref_36","unstructured":"Britto, A., Gouyon, F., and Dixon, S. (2013). Essentia: An audio analysis library for music information retrieval. Proceedings of the 14th Conference of the International Society for Music Information Retrieval (ISMIR), Curitiba, Brazil, 4\u20138 November 2013, International Society for Music Information Retrieval (ISMIR)."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1111\/1467-6486.00300","article-title":"Work relationships in telephone call centers: Understanding emotional exhaustion and employee withdrawal","volume":"39","author":"Deery","year":"2002","journal-title":"J. Manag. Stud."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5489\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:13:30Z","timestamp":1760177610000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/19\/5489"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,25]]},"references-count":37,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2020,10]]}},"alternative-id":["s20195489"],"URL":"https:\/\/doi.org\/10.3390\/s20195489","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,9,25]]}}}