{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T15:09:39Z","timestamp":1784560179828,"version":"3.55.0"},"reference-count":54,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2025,3,17]],"date-time":"2025-03-17T00:00:00Z","timestamp":1742169600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62376062"],"award-info":[{"award-number":["62376062"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Philosophy and Social Sciences 14th Five-Year Plan Project of Guangdong Province","award":["GD23CTS03"],"award-info":[{"award-number":["GD23CTS03"]}]},{"name":"Ministry of Education of Humanities and Social Science Project","award":["23YJAZH220"],"award-info":[{"award-number":["23YJAZH220"]}]},{"name":"Guangdong Basic and Applied Basic Research Foundation of China","award":["2023A1515012718"],"award-info":[{"award-number":["2023A1515012718"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,8,14]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Curriculum learning (CL) is a training strategy that imitates how humans learn, by gradually introducing more complex samples and information to the model. However, in multi-label emotion classification (MEC) tasks, using a traditional CL approach can result in overfitting on easy samples and lead to biased training. Additionally, the sample difficulty varies as the model trains. To address these challenges, we propose a novel CL framework for MEC tasks called CLF-MEC. Unlike traditional approaches that assess difficulty at the sample level, we utilize category-level assessment to determine the difficulty level of samples. As the model identifies a category well, the score for that category\u2019s samples is reduced, ensuring dynamic changes in the sample difficulty are accounted for. Our CL framework employs two training modes, namely \u201clearning\u201d and \u201ctackling.\u201d These two processes are trained alternatively to imitate the \u201clearning-tackling\u201d process in human learning. This ensures that samples from hard-to-learn categories receive more attention. During the \u201ctackling\u201d process, our method transforms the task of dealing with hard samples into an \u201ceasy\u201d learning task by utilizing contrastive learning to enhance the semantic representation of those hard samples. Experimental results demonstrate that our CLF-MEC framework has achieved significant improvements in MEC.<\/jats:p>","DOI":"10.1093\/comjnl\/bxaf022","type":"journal-article","created":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T15:23:00Z","timestamp":1740151380000},"page":"1050-1061","source":"Crossref","is-referenced-by-count":1,"title":["A novel curriculum learning framework for multi-label emotion classification"],"prefix":"10.1093","volume":"68","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2838-8273","authenticated-orcid":false,"given":"Nankai","family":"Lin","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, Guangdong University of Foreign Studies , Guangzhou 510006, Guangdong ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-9976-8241","authenticated-orcid":false,"given":"Hongyan","family":"Wu","sequence":"additional","affiliation":[{"name":"College of Computer, National University of Defense Technology , Changsha 410073, Changsha ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-3352-3565","authenticated-orcid":false,"given":"Peijian","family":"Zeng","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Guangdong University of Technology , Guangzhou 510006, Guangdong ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-9663-5834","authenticated-orcid":false,"given":"Qifeng","family":"Bai","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Guangdong University of Technology , Guangzhou 510006, Guangdong ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3310-8347","authenticated-orcid":false,"given":"Dong","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, Guangdong University of Foreign Studies , Guangzhou 510006, Guangdong ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7671-6150","authenticated-orcid":false,"given":"Aimin","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Guangdong University of Technology , Guangzhou 510006, Guangdong ,","place":["China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,3,17]]},"reference":[{"key":"2025081702470703500_ref1","doi-asserted-by":"publisher","first-page":"2935","DOI":"10.1109\/TASLP.2023.3297964","article-title":"CL-XABSA: contrastive learning for cross-lingual aspect-based sentiment analysis","volume":"31","author":"Lin","year":"2023","journal-title":"IEEE\/ACM Transactions on Audio, Speech, and Language Processing"},{"key":"2025081702470703500_ref2","doi-asserted-by":"publisher","first-page":"351","DOI":"10.1093\/comjnl\/bxz032","article-title":"Emotion recognition by a hybrid system based on the features of distances and the shapes of the wrinkles","volume":"63","author":"Afdhal","year":"2019","journal-title":"The Computer Journal"},{"key":"2025081702470703500_ref3","doi-asserted-by":"publisher","first-page":"3107","DOI":"10.1093\/comjnl\/bxac153","article-title":"Chinese RoBERTa distillation for emotion classification","volume":"66","author":"Liu","year":"2022","journal-title":"The Computer Journal"},{"key":"2025081702470703500_ref4","doi-asserted-by":"publisher","first-page":"2403","DOI":"10.1093\/comjnl\/bxae061","article-title":"Thematic editorial: sentiment analysis","volume":"67","author":"Manolopoulos","year":"2024","journal-title":"The Computer Journal"},{"key":"2025081702470703500_ref5","doi-asserted-by":"publisher","first-page":"103097","DOI":"10.1016\/j.ipm.2022.103097","article-title":"Multi-label emotion classification based on adversarial multi-task learning","volume":"59","author":"Lin","year":"2022","journal-title":"Information Processing and Management"},{"key":"2025081702470703500_ref6","doi-asserted-by":"crossref","first-page":"8730","DOI":"10.18653\/v1\/2023.findings-acl.556","article-title":"An effective deployment of contrastive learning in multi-label text classification","volume-title":"Findings of the Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023)","author":"Lin","year":"2023"},{"key":"2025081702470703500_ref7","doi-asserted-by":"publisher","first-page":"3874","DOI":"10.1109\/TNSRE.2023.3314642","article-title":"Aligning semantic in brain and language: a curriculum contrastive method for electroencephalography-to-text generation","volume":"31","author":"Feng","year":"2023","journal-title":"IEEE Trans Neural Syst Rehabil Eng"},{"key":"2025081702470703500_ref8","doi-asserted-by":"publisher","first-page":"11595","DOI":"10.1609\/aaai.v36i10.21413","article-title":"Hybrid curriculum learning for emotion recognition in conversation","volume-title":"Proceedings of the 32nd Innovative Applications of Artificial Intelligence Conference (AAAI 2022)","author":"Yang","year":"2022"},{"key":"2025081702470703500_ref9","doi-asserted-by":"publisher","first-page":"107585","DOI":"10.1016\/j.patcog.2020.107585","article-title":"Low: training deep neural networks by learning optimal sample weights","volume":"110","author":"Santiago","year":"2021","journal-title":"Pattern Recognition"},{"key":"2025081702470703500_ref10","doi-asserted-by":"crossref","first-page":"979","DOI":"10.1145\/3503161.3548289","article-title":"Angular gap: Reducing the uncertainty of image difficulty through model calibration","volume-title":"Proceedings of the 30th ACM International Conference on Multimedia (MM 2022) MM \u201822","author":"Peng","year":"2022"},{"key":"2025081702470703500_ref11","first-page":"17871","article-title":"Semifl: Semi-supervised federated learning for unlabeled clients with alternate training","volume-title":"Proceedings of 36th Annual Conference on Neural Information Processing Systems (NeurIPS 2022)","author":"Diao","year":"2022"},{"key":"2025081702470703500_ref12","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1109\/MIS.2023.3255591","article-title":"On the persistence of multilabel learning, its recent trends, and its open issues","volume":"38","author":"Mylonas","year":"2023","journal-title":"IEEE Intelligent Systems"},{"key":"2025081702470703500_ref13","first-page":"121","article-title":"Inferring your expertise from twitter: Integrating sentiment and topic relatedness","volume-title":"Proceedings of 2016 IEEE\/WIC\/ACM International Conference on Web Intelligence (WI 2016)","author":"Xu","year":"2016"},{"key":"2025081702470703500_ref14","first-page":"417","article-title":"Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL 2002)","author":"Turney","year":"2002"},{"key":"2025081702470703500_ref15","doi-asserted-by":"crossref","first-page":"1631","DOI":"10.18653\/v1\/D13-1170","article-title":"Recursive deep models for semantic compositionality over a sentiment treebank","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP 2013)","author":"Socher","year":"2013"},{"key":"2025081702470703500_ref16","first-page":"312","article-title":"SemEval-2013 task 2: Sentiment analysis in twitter","volume-title":"Proceedings of the 7th International Workshop on Semantic Evaluation (SemEval 2013)","author":"Nakov","year":"2013"},{"key":"2025081702470703500_ref17","first-page":"8","article-title":"Exposing a set of fine-grained emotion categories from tweets","volume-title":"Proceedings of 25th International Joint Conference on Artificial Intelligence (IJCAI 2016)","author":"Yan","year":"2016"},{"key":"2025081702470703500_ref18","first-page":"395","article-title":"Happy dance, slow clap: Using reaction GIFs to predict induced affect on twitter","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021)","author":"Shmueli","year":"2021"},{"key":"2025081702470703500_ref19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18653\/v1\/S18-1001","article-title":"SemEval-2018 task 1: Affect in tweets","volume-title":"Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval 2018)","author":"Mohammad","year":"2018"},{"key":"2025081702470703500_ref20","first-page":"1573","article-title":"SpanEmo: Casting multi-label emotion classification as span-prediction","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2021)","author":"Alhuzali","year":"2021"},{"key":"2025081702470703500_ref21","doi-asserted-by":"publisher","first-page":"2257","DOI":"10.1093\/comjnl\/bxae004","article-title":"Enhancing aspect category detection through hybridised contextualised neural language models: a case study In multi-label text classification","volume":"67","author":"Karaoglan","year":"2024","journal-title":"The Computer Journal"},{"key":"2025081702470703500_ref22","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1007\/978-981-97-7007-6_8","article-title":"Emobart: Multi-label emotion classification method based on pre-trained label sequence generation model","volume-title":"Proceedings of 5th International Conference on Neural Computing for Advanced Applications (NCAA 2024)","author":"Chen","year":"2025"},{"key":"2025081702470703500_ref23","doi-asserted-by":"publisher","first-page":"118534","DOI":"10.1016\/j.eswa.2022.118534","article-title":"Multi-label emotion classification in texts using transfer learning","volume":"213","author":"Ameer","year":"2023","journal-title":"Expert Systems with Applications"},{"key":"2025081702470703500_ref24","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1016\/0010-0277(93)90058-4","article-title":"Learning and development in neural networks: the importance of starting small","volume":"48","author":"Elman","year":"1993","journal-title":"Cognition"},{"key":"2025081702470703500_ref25","doi-asserted-by":"publisher","first-page":"2581","DOI":"10.1109\/JBHI.2023.3274486","article-title":"Graph attention-based curriculum learning for mental healthcare classification","volume":"28","author":"Ahmed","year":"2024","journal-title":"IEEE J Biomed Health Inform"},{"key":"2025081702470703500_ref26","doi-asserted-by":"publisher","first-page":"2259","DOI":"10.1109\/TASLP.2023.3282105","article-title":"Obscurity-quantified curriculum learning for machine translation evaluation","volume":"31","author":"Zhang","year":"2023","journal-title":"IEEE\/ACM Transactions on Audio, Speech, and Language Processing"},{"key":"2025081702470703500_ref27","doi-asserted-by":"crossref","first-page":"206","DOI":"10.18653\/v1\/2021.iwslt-1.25","article-title":"Self-guided curriculum learning for neural machine translation","volume-title":"Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021)","author":"Zhou","year":"2021"},{"key":"2025081702470703500_ref28","first-page":"2054","article-title":"Reinforcement learning based curriculum optimization for neural machine translation","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2019)","author":"Kumar","year":"2019"},{"key":"2025081702470703500_ref29","doi-asserted-by":"crossref","first-page":"4922","DOI":"10.18653\/v1\/P19-1486","article-title":"Simple and effective curriculum pointer-generator networks for reading comprehension over long narratives","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019)","author":"Tay","year":"2019"},{"key":"2025081702470703500_ref30","doi-asserted-by":"crossref","first-page":"6095","DOI":"10.18653\/v1\/2020.acl-main.542","article-title":"Curriculum learning for natural language understanding","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020)","author":"Xu","year":"2020"},{"key":"2025081702470703500_ref31","first-page":"11937","article-title":"In-sample curriculum learning by sequence completion for natural language generation","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023)","author":"Jia","year":"2023"},{"key":"2025081702470703500_ref32","doi-asserted-by":"publisher","first-page":"3307","DOI":"10.1109\/TASLP.2021.3121986","article-title":"Review and arrange: curriculum learning for natural language understanding","volume":"29","author":"Zhang","year":"2021","journal-title":"IEEE\/ACM Transactions on Audio, Speech, and Language Processing"},{"key":"2025081702470703500_ref33","doi-asserted-by":"publisher","first-page":"11595","DOI":"10.1609\/aaai.v36i10.21413","article-title":"Hybrid curriculum learning for emotion recognition in conversation","volume-title":"Proceedings of the 34th Innovative Applications of Artificial Intelligence Conference (AAAI 2022)","author":"Yang","year":"2022"},{"key":"2025081702470703500_ref34","doi-asserted-by":"publisher","first-page":"815","DOI":"10.1109\/TASLP.2019.2898816","article-title":"Curriculum learning for speech emotion recognition from crowdsourced labels","volume":"27","author":"Lotfian","year":"2019","journal-title":"IEEE\/ACM Transactions on Audio, Speech, and Language Processing"},{"key":"2025081702470703500_ref35","doi-asserted-by":"publisher","first-page":"111434","DOI":"10.1016\/j.knosys.2024.111434","article-title":"Ernetcl: a novel emotion recognition network in textual conversation based on curriculum learning strategy","volume":"286","author":"Li","year":"2024","journal-title":"Knowledge-Based Systems"},{"key":"2025081702470703500_ref36","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1109\/MIS.2024.3411369","article-title":"Adaclf: an adaptive curriculum learning framework for emotional support conversation","volume":"39","author":"Tu","year":"2024","journal-title":"IEEE Intelligent Systems"},{"key":"2025081702470703500_ref37","doi-asserted-by":"crossref","first-page":"167","DOI":"10.18653\/v1\/S18-1024","article-title":"Tw-StAR at SemEval-2018 task 1: Preprocessing impact on multi-label emotion classification","volume-title":"Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval 2018)","author":"Mulki","year":"2018"},{"key":"2025081702470703500_ref38","doi-asserted-by":"crossref","first-page":"236","DOI":"10.18653\/v1\/S18-1036","article-title":"EMA at SemEval-2018 task 1: Emotion mining for Arabic","volume-title":"Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval 2018)","author":"Badaro","year":"2018"},{"key":"2025081702470703500_ref39","doi-asserted-by":"publisher","first-page":"37843","DOI":"10.1109\/ACCESS.2020.2975906","article-title":"Hybrid feature model for emotion recognition in arabic text","volume":"8","author":"Alswaidan","year":"2020","journal-title":"IEEE Access"},{"key":"2025081702470703500_ref40","doi-asserted-by":"publisher","first-page":"250","DOI":"10.1007\/978-3-319-99495-6_21","article-title":"Joint binary neural network for multi-label learning with applications to emotion classification","volume-title":"Proceedings of 7th CCF International Conference on Natural Language Processing and Chinese Computing (NLPCC 2018)","author":"He","year":"2018"},{"key":"2025081702470703500_ref41","doi-asserted-by":"crossref","first-page":"245","DOI":"10.18653\/v1\/S18-1037","article-title":"NTUA-SLP at SemEval-2018 task 1: Predicting affective content in tweets with deep attentive RNNs and transfer learning","volume-title":"Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval 2018)","author":"Baziotis","year":"2018"},{"key":"2025081702470703500_ref42","doi-asserted-by":"crossref","first-page":"565","DOI":"10.18653\/v1\/S18-1092","article-title":"ELiRF-UPV at SemEval-2018 tasks 1 and 3: Affect and irony detection in tweets","volume-title":"Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval 2018)","author":"Gonz\u00e1lez","year":"2018"},{"key":"2025081702470703500_ref43","doi-asserted-by":"crossref","first-page":"1097","DOI":"10.18653\/v1\/D18-1137","article-title":"Improving multi-label emotion classification via sentiment classification with dual attention transfer network","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018)","author":"Yu","year":"2018"},{"key":"2025081702470703500_ref44","doi-asserted-by":"crossref","first-page":"316","DOI":"10.18653\/v1\/D19-5541","article-title":"Improving multi-label emotion classification by integrating both general and domain-specific knowledge","volume-title":"Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019)","author":"Ying","year":"2019"},{"key":"2025081702470703500_ref45","article-title":"Emograph: capturing emotion correlations using graph networks","volume":"abs\/2008.09378","author":"Xu","year":"2020","journal-title":"CoRR"},{"key":"2025081702470703500_ref46","first-page":"7692","article-title":"Latent emotion memory for multi-label emotion classification","volume-title":"Proceedings of the 32nd Innovative Applications of Artificial Intelligence Conference (AAAI 2022)","author":"Fei","year":"2020"},{"key":"2025081702470703500_ref47","first-page":"561","article-title":"Relevant emotion ranking from text constrained with emotion relationships","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2018)","author":"Zhou","year":"2018"},{"key":"2025081702470703500_ref48","doi-asserted-by":"publisher","first-page":"108114","DOI":"10.1016\/j.knosys.2021.108114","article-title":"Elementary discourse units with sparse attention for multi-label emotion classification","volume":"240","author":"Zhu","year":"2022","journal-title":"Knowledge-Based Systems"},{"key":"2025081702470703500_ref49","first-page":"1","article-title":"Leveraging label correlations in a multi-label setting: A case study in emotion","volume-title":"Proceedings of 48th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023)","author":"Chochlakis","year":"2023"},{"key":"2025081702470703500_ref50","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2019)","author":"Devlin","year":"2019"},{"key":"2025081702470703500_ref51","first-page":"1218","article-title":"A robustly optimized BERT pre-training approach with post-training","volume-title":"Proceedings of the 20th Chinese National Conference on Computational Linguistics (CCL 2021)","author":"Zhuang","year":"2021"},{"key":"2025081702470703500_ref52","doi-asserted-by":"crossref","first-page":"8440","DOI":"10.18653\/v1\/2020.acl-main.747","article-title":"Unsupervised cross-lingual representation learning at scale","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020)","author":"Conneau","year":"2020"},{"key":"2025081702470703500_ref53","first-page":"7088","article-title":"ARBERT & MARBERT: Deep bidirectional transformers for Arabic","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021)","author":"Abdul-Mageed","year":"2021"},{"key":"2025081702470703500_ref54","first-page":"878","article-title":"Language-agnostic BERT sentence embedding","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022)","author":"Feng","year":"2022"}],"container-title":["The Computer Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/68\/8\/1050\/62421844\/bxaf022.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/68\/8\/1050\/62421844\/bxaf022.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,17]],"date-time":"2025-08-17T06:47:22Z","timestamp":1755413242000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/comjnl\/article\/68\/8\/1050\/8080214"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,17]]},"references-count":54,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,3,17]]},"published-print":{"date-parts":[[2025,8,14]]}},"URL":"https:\/\/doi.org\/10.1093\/comjnl\/bxaf022","relation":{},"ISSN":["0010-4620","1460-2067"],"issn-type":[{"value":"0010-4620","type":"print"},{"value":"1460-2067","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,8]]},"published":{"date-parts":[[2025,3,17]]}}}