{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,2]],"date-time":"2026-08-02T04:31:37Z","timestamp":1785645097946,"version":"3.56.0"},"reference-count":30,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2022,11,10]],"date-time":"2022-11-10T00:00:00Z","timestamp":1668038400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,12,14]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Through knowledge distillation method, a student model can imitate the output of a teacher model to improve its generalization ability without changing the computational complexity. However, in existing knowledge distillation research, the efficiency of knowledge transfer is still not satisfactory, especially from pre-trained language models (PTMs) like Robustly optimized BERT approach (RoBERTa) to another structure student model. To address this issue, this paper proposes a prediction framework (RTLSTM) for Chinese emotion classification based on knowledge distillation. In RTLSTM, a new triple loss strategy is proposed for training a student \u2018BiLSTM\u2019, which combines supervised learning, distillation and word vector losses. This strategy enables the student to learn more fully from a teacher model RoBERTa and retains 99% of the teacher models\u2019 language understanding capability. We carried out emotion classification experiments on five Chinese datasets to compare RTLSTM with baseline models. The experiment results show that RTLSTM outperforms the baseline models belonging to the RNN group in terms of prediction performance under similar numbers of parameters. Moreover, RTLSTM is superior to the PTMs group baseline models through 92% fewer parameters and 83% less prediction time under comparable prediction performance.<\/jats:p>","DOI":"10.1093\/comjnl\/bxac153","type":"journal-article","created":{"date-parts":[[2022,11,11]],"date-time":"2022-11-11T10:39:18Z","timestamp":1668163158000},"page":"3107-3118","source":"Crossref","is-referenced-by-count":12,"title":["Chinese RoBERTa Distillation For Emotion Classification"],"prefix":"10.1093","volume":"66","author":[{"given":"Pingshan","family":"Liu","sequence":"first","affiliation":[{"name":"School of Business, Guilin University of Electronic Technology , Guilin, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuyue","family":"Lv","sequence":"additional","affiliation":[{"name":"School of Business, Guilin University of Electronic Technology , Guilin, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2022,11,10]]},"reference":[{"key":"2023121811455376800_ref1","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1109\/MIS.2016.31","article-title":"Affective computing and sentiment analysis","volume":"31","author":"Cambria","year":"2016","journal-title":"IEEE Intell. Syst."},{"key":"2023121811455376800_ref2","doi-asserted-by":"crossref","first-page":"1872","DOI":"10.1007\/s11431-020-1647-3","article-title":"Pre-trained models for natural language processing: A survey","volume":"63","author":"Qiu","year":"2020","journal-title":"Sci. China Technol. Sci."},{"key":"2023121811455376800_ref3","first-page":"1746","article-title":"Convolutional neural networks for sentence classification","volume":"1","author":"Kim","year":"2014","journal-title":"Proc. EMNLP\u201914"},{"key":"2023121811455376800_ref4","doi-asserted-by":"crossref","first-page":"2266","DOI":"10.3390\/s21072266","article-title":"Building a twitter sentiment analysis system with recurrent neural networks","volume":"21","author":"Nistor","year":"2021","journal-title":"Sensors\u201921"},{"key":"2023121811455376800_ref5","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"2023121811455376800_ref6","first-page":"747","volume-title":"Proc. SemEval\u201917","author":"Baziotis","year":"2017"},{"key":"2023121811455376800_ref7","first-page":"1724","article-title":"Aspect-based sentiment analysis based on multi-attention CNN","volume":"54","author":"Liang","year":"2017","journal-title":"J. Comp. Res. Dev"},{"key":"2023121811455376800_ref8","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.ins.2022.03.082","article-title":"Aggregated graph convolutional networks for aspect-based sentiment classification","volume":"600","author":"Zhao","year":"2022","journal-title":"Inform. Sci."},{"key":"2023121811455376800_ref9","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/j.future.2020.08.005","article-title":"ABCDM: An attention-based bidirectional CNN-RNN deep model for sentiment analysis","volume":"115","author":"Basiri","year":"2021","journal-title":"Future Gener. Comput. Syst."},{"key":"2023121811455376800_ref10","first-page":"165","article-title":"A deep learning approach combining CNN and Bi-LSTM with SVM lassifier for arabic sentiment analysis","volume":"12","author":"Alharbi","year":"2021","journal-title":"Int. J. Adv Comput. Sci. Appl."},{"key":"2023121811455376800_ref11","doi-asserted-by":"crossref","DOI":"10.1016\/j.artint.2021.103477","article-title":"Enhanced aspect-based sentiment analysis models with progressive self-supervised attention learning","volume":"296","author":"Su","year":"2021","journal-title":"Artif. Intell."},{"key":"2023121811455376800_ref12","first-page":"4171","volume-title":"Proc. NAACL-HLT\u201919","author":"Devlin","year":"2019"},{"key":"2023121811455376800_ref13","first-page":"2978","volume-title":"Proc. ACL\u201919","author":"Dai","year":"2019"},{"key":"2023121811455376800_ref14","article-title":"Roberta: A robustly optimized BERT pretraining approach","volume":"abs\/1907.11692","author":"Liu","year":"2019","journal-title":"CoRR"},{"key":"2023121811455376800_ref15","first-page":"194","volume-title":"Proc. CCL\u201919","author":"Sun","year":"2019"},{"key":"2023121811455376800_ref16","doi-asserted-by":"crossref","first-page":"3504","DOI":"10.1109\/TASLP.2021.3124365","article-title":"Pre-training with whole word masking for chinese bert","volume":"29","author":"Cui","year":"2021","journal-title":"IEEE\/ACM Trans. Audio, Speech, Language Process"},{"key":"2023121811455376800_ref17","first-page":"8726621","article-title":"A BERT-Based aspect-level sentiment analysis algorithm for cross-domain text","volume":"2022","author":"Liu","year":"2022","journal-title":"Comput. Intell. Neurosci."},{"key":"2023121811455376800_ref18","doi-asserted-by":"crossref","first-page":"133","DOI":"10.3390\/s21010133","article-title":"An effective bert-based pipeline for twitter sentiment analysis: a case study in italian","volume":"21","author":"Pota","year":"2020","journal-title":"Sensors"},{"key":"2023121811455376800_ref19","first-page":"194","volume-title":"Proc. EMC2-NIPS Workshop","author":"Sanh","year":"2019"},{"key":"2023121811455376800_ref20","first-page":"38","article-title":"Distilling the knowledge in a neural network","volume":"14","author":"Hinton","year":"2015","journal-title":"Comput. Sci."},{"key":"2023121811455376800_ref21","volume-title":"proc. ICLR\u201915","author":"Romero","year":"2015"},{"key":"2023121811455376800_ref22","volume-title":"proc.ICLR\u201917","author":"Zagoruyko","year":"2017"},{"key":"2023121811455376800_ref23","first-page":"3712","volume-title":"Proc. ICCV\u201919","author":"Zhang","year":"2019"},{"key":"2023121811455376800_ref24","first-page":"4163","volume-title":"Proc. Findings of ACL: EMNLP\u201920","author":"Jiao","year":"2020"},{"key":"2023121811455376800_ref25","first-page":"2158","volume-title":"Proc. ACL\u201920","author":"Sun","year":"2020"},{"key":"2023121811455376800_ref26","first-page":"5776","volume-title":"Proc. NIPS\u201920","author":"Wang","year":"2020"},{"key":"2023121811455376800_ref27","article-title":"Distilling task-specific knowledge from bert into simple neural networks","volume":"abs\/1903.12136","author":"Tang","year":"2019","journal-title":"CoRR"},{"key":"2023121811455376800_ref28","article-title":"BERTtoCNN: Similarity-preserving enhanced knowledge distillation for stance detection","volume":"16","author":"Li","year":"2021","journal-title":"PLoS One"},{"key":"2023121811455376800_ref29","first-page":"212","volume-title":"Proc. ICDM\u201920","author":"Jiang","year":"2020"},{"key":"2023121811455376800_ref30","doi-asserted-by":"crossref","DOI":"10.1016\/j.knosys.2021.106837","article-title":"Improving knowledge distillation via an expressive teacher","volume":"218","author":"Tan","year":"2021","journal-title":"Knowl. Based. Syst."}],"container-title":["The Computer Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/66\/12\/3107\/54446737\/bxac153.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/66\/12\/3107\/54446737\/bxac153.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,18]],"date-time":"2023-12-18T11:47:04Z","timestamp":1702900024000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/comjnl\/article\/66\/12\/3107\/6821147"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,10]]},"references-count":30,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,11,10]]},"published-print":{"date-parts":[[2023,12,14]]}},"URL":"https:\/\/doi.org\/10.1093\/comjnl\/bxac153","relation":{},"ISSN":["0010-4620","1460-2067"],"issn-type":[{"value":"0010-4620","type":"print"},{"value":"1460-2067","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,12]]},"published":{"date-parts":[[2022,11,10]]}}}