{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T00:54:50Z","timestamp":1771548890812,"version":"3.50.1"},"reference-count":40,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,3,23]],"date-time":"2022-03-23T00:00:00Z","timestamp":1647993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2021YFF1201200"],"award-info":[{"award-number":["2021YFF1201200"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61632014 and 61802158"],"award-info":[{"award-number":["61632014 and 61802158"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","award":["lzujbky-2021-66"],"award-info":[{"award-number":["lzujbky-2021-66"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>As a serious worldwide problem, suicide often causes huge and irreversible losses to families and society. Therefore, it is necessary to detect and help individuals with suicidal ideation in time. In recent years, the prosperous development of social media has provided new perspectives on suicide detection, but related research still faces some difficulties, such as data imbalance and expression implicitness. In this paper, we propose a Deep Hierarchical Ensemble model for Suicide Detection (DHE-SD) based on a hierarchical ensemble strategy, and construct a dataset based on Sina Weibo, which contains more than 550 thousand posts from 4521 users. To verify the effectiveness of the model, we also conduct experiments on a public Weibo dataset containing 7329 users\u2019 posts. The proposed model achieves the best performance on both the constructed dataset and the public dataset. In addition, in order to make the model applicable to a wider population, we use the proposed sentence-level mask mechanism to delete user posts with strong suicidal ideation. Experiments show that the proposed model can still effectively identify social media users with suicidal ideation even when the performance of the baseline models decrease significantly.<\/jats:p>","DOI":"10.3390\/e24040442","type":"journal-article","created":{"date-parts":[[2022,3,23]],"date-time":"2022-03-23T12:20:25Z","timestamp":1648038025000},"page":"442","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Deep Hierarchical Ensemble Model for Suicide Detection on Imbalanced Social Media Data"],"prefix":"10.3390","volume":"24","author":[{"given":"Zepeng","family":"Li","sequence":"first","affiliation":[{"name":"School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China"}]},{"given":"Jiawei","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China"}]},{"given":"Zhengyi","family":"An","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China"}]},{"given":"Wenchuan","family":"Cheng","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China"}]},{"given":"Bin","family":"Hu","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Lanzhou University, Lanzhou 730000, China"},{"name":"Institute of Engineering Medicine, Beijing Institute of Technology, Beijing 100081, China"},{"name":"CAS Center for Excellence in Brain Science and Institutes for Biological Sciences, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai 200000, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,23]]},"reference":[{"key":"ref_1","unstructured":"World Health Organization (2021). Suicide Worldwide in 2019: Global Health Estimates, World Health Organization."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"637","DOI":"10.2466\/pr0.1998.83.2.637","article-title":"The suicide probability scale: Norms and factor structure","volume":"83","author":"Bagge","year":"1998","journal-title":"Psychol. Rep."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1037\/1040-3590.19.4.422","article-title":"Predictive validity of the Chinese version of the Adult Suicidal Ideation Questionnaire: Psychometric properties and its short version","volume":"19","author":"Fu","year":"2007","journal-title":"Psychol. Assess."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Harris, K.M., Syu, J.J., Lello, O.D., Chew, Y.E., Willcox, C.H., and Ho, R.H. (2015). The ABC\u2019s of suicide risk assessment: Applying a tripartite approach to individual evaluations. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0127442"},{"key":"ref_5","unstructured":"Zogan, H., Razzak, I., Jameel, S., and Xu, G. (2021). DepressionNet: A Novel Summarization Boosted Deep Framework for Depression Detection on Social Media. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Shing, H.C., Resnik, P., and Oard, D.W. (2020, January 5\u201310). A prioritization model for suicidality risk assessment. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.723"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1109\/TMM.2020.3046867","article-title":"Building and using personal knowledge graph to improve suicidal ideation detection on social media","volume":"24","author":"Cao","year":"2020","journal-title":"IEEE Trans. Multimed."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"193248","DOI":"10.1109\/ACCESS.2020.3030468","article-title":"Fine-Tuning BERT for Multi-Label Sentiment Analysis in Unbalanced Code-Switching Text","volume":"8","author":"Tang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.knosys.2018.06.019","article-title":"Imbalanced text sentiment classification using universal and domain-specific knowledge","volume":"160","author":"Li","year":"2018","journal-title":"Knowl.-Based. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1968","DOI":"10.1109\/TCBB.2018.2827029","article-title":"Predicting hospital readmission via cost-sensitive deep learning","volume":"15","author":"Wang","year":"2018","journal-title":"IEEE ACM Trans. Comput. Biol. Bioinform."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1080\/15265161.2021.1928793","article-title":"Artificial intelligence, social media, and suicide prevention: Principle of beneficence besides respect for autonomy","volume":"21","author":"Zhang","year":"2021","journal-title":"Am. J. Bioeth."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"104499","DOI":"10.1016\/j.compbiomed.2021.104499","article-title":"A textual-based featuring approach for depression detection using machine learning classifiers and social media texts","volume":"135","author":"Chiong","year":"2021","journal-title":"Comput. Biol. Med."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"6157249","DOI":"10.1155\/2018\/6157249","article-title":"Supervised learning for suicidal ideation detection in online user content","volume":"2018","author":"Ji","year":"2018","journal-title":"Complexity"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Shing, H.C., Nair, S., Zirikly, A., Friedenberg, M., Daum\u00e9, H., and Resnik, P. (2018, January 5). Expert, crowdsourced, and machine assessment of suicide risk via online postings. Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic, New Orleans, LA, USA.","DOI":"10.18653\/v1\/W18-0603"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1147\/JRD.2017.2768678","article-title":"Detection of suicide-related posts in Twitter data streams","volume":"62","author":"Vioules","year":"2018","journal-title":"IBM J. Res. Dev."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1109\/TCSS.2020.3021467","article-title":"Suicidal ideation detection: A review of machine learning methods and applications","volume":"8","author":"Ji","year":"2020","journal-title":"IEEE T. Comput. Soc. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Masuda, N., Kurahashi, I., and Onari, H. (2013). Suicide ideation of individuals in online social networks. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0062262"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"6351","DOI":"10.1016\/j.eswa.2013.05.050","article-title":"Emotion detection in suicide notes","volume":"40","author":"Desmet","year":"2013","journal-title":"Expert Syst. Appl."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Huang, X., Zhang, L., Chiu, D., Liu, T., Li, X., and Zhu, T. (2014, January 9\u201312). Detecting suicidal ideation in Chinese microblogs with psychological lexicons. Proceedings of the 2014 IEEE 11th International Conference on Ubiquitous Intelligence and Computing and 2014 IEEE 11th International Conference on Autonomic and Trusted Computing and 2014 IEEE 14th International Conference on Scalable Computing and Communications and Its Associated Workshops, Bali, Indonesia.","DOI":"10.1109\/UIC-ATC-ScalCom.2014.48"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014, January 25\u201329). Convolutional neural networks for sentence classification. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Doha, Qatar.","DOI":"10.3115\/v1\/D14-1181"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Joulin, A., Grave, E., Bojanowski, P., and Mikolov, T. (2016). Bag of tricks for efficient text classification. arXiv.","DOI":"10.18653\/v1\/E17-2068"},{"key":"ref_22","unstructured":"Johnson, R., and Zhang, T. (August, January 30). Deep pyramid convolutional neural networks for text categorization. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, BC, Canada."},{"key":"ref_23","unstructured":"Ji, S., Long, G., Pan, S., Zhu, T., Jiang, J., Wang, S., and Li, X. (2019). Knowledge transferring via model aggregation for online social care. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Benton, A., Mitchell, M., and Hovy, D. (2017). Multi-task learning for mental health using social media text. arXiv.","DOI":"10.18653\/v1\/E17-1015"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sinha, P.P., Mishra, R., Sawhney, R., Mahata, D., Shah, R.R., and Liu, H. (2019, January 3\u20137). # suicidal-A multipronged approach to identify and explore suicidal ideation in twitter. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China.","DOI":"10.1145\/3357384.3358060"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1384","DOI":"10.1109\/JBHI.2015.2459683","article-title":"Examining accumulated emotional traits in suicide blogs with an emotion topic model","volume":"20","author":"Ren","year":"2015","journal-title":"IEEE J. Biomed. Health"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhao, X., Lin, S., and Huang, Z. (2018, January 21\u201323). Text Classification of Micro-blog\u2019s \u201cTree Hole\u201d Based on Convolutional Neural Network. Proceedings of the 2018 International Conference on Algorithms, Computing and Artificial Intelligence, Sanya, China.","DOI":"10.1145\/3302425.3302501"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Matero, M., Idnani, A., Son, Y., Giorgi, S., Vu, H., Zamani, M., Limbachiya, P., Guntuku, S., and Schwartz, H.A. (2019, January 6). Suicide risk assessment with multi-level dual-context language and bert. Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology, Minneapolis, MS, USA.","DOI":"10.18653\/v1\/W19-3005"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Sawhney, R., Joshi, H., Gandhi, S., and Shah, R.R. (2021, January 8\u201312). Towards Ordinal Suicide Ideation Detection on Social Media. Proceedings of the 14th ACM International Conference on Web Search and Data Mining, Virtual.","DOI":"10.1145\/3437963.3441805"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1613\/jair.953","article-title":"SMOTE: Synthetic minority over-sampling technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J. Artif. Intell. Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1016\/j.procs.2019.09.229","article-title":"Dealing with data imbalance in text classification","volume":"159","author":"Padurariu","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Lemnaru, C., and Potolea, R. (2018). Evolutionary cost-sensitive balancing: A generic method for imbalanced classification problems. EVOLVE-A Bridge between Probability, Set Oriented Numerics, and Evolutionary Computation VI, Springer.","DOI":"10.1007\/978-3-319-69710-9_14"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Lodkaew, T., and Pasupa, K. (2020, January 23\u201327). Hybrid Loss for Improving Classification Performance with Unbalanced Data. Proceedings of the International Conference on Neural Information Processing, Bangkok, Thailand.","DOI":"10.1007\/978-3-030-63820-7_92"},{"key":"ref_34","first-page":"703","article-title":"An embedded feature selection method for imbalanced data classification","volume":"6","author":"Liu","year":"2019","journal-title":"IEEE-CAA J. Autom."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1108\/SRT-01-2021-0003","article-title":"A K-means clustering method with feature learning for unbalanced vehicle fault diagnosis","volume":"3","author":"Wang","year":"2021","journal-title":"Smart Resilient Transp."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1016\/j.ipm.2018.09.004","article-title":"Supervised Hebb rule based feature selection for text classification","volume":"56","author":"Wang","year":"2019","journal-title":"Inform. Process. Manag."},{"key":"ref_37","first-page":"688","article-title":"Detection of suicidal intent in Spanish language social networks using machine learning","volume":"11","author":"Valeriano","year":"2020","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Gao, J., Cheng, Q., and Philip, L.H. (2018, January 15). Detecting comments showing risk for suicide in YouTube. Proceedings of the Future Technologies Conference, Vancouver, BC, Canada.","DOI":"10.1007\/978-3-030-02686-8_30"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Preotiuc-Pietro, D., Gaman, M., and Aletras, N. (2019). Automatically identifying complaints in social media. arXiv.","DOI":"10.18653\/v1\/P19-1495"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Cao, L., Zhang, H., Feng, L., Wei, Z., Wang, X., Li, N., and He, X. (2019, January 3\u20137). Latent Suicide Risk Detection on Microblog via Suicide-Oriented Word Embeddings and Layered Attention. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.","DOI":"10.18653\/v1\/D19-1181"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/4\/442\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:41:23Z","timestamp":1760136083000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/4\/442"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,23]]},"references-count":40,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["e24040442"],"URL":"https:\/\/doi.org\/10.3390\/e24040442","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,23]]}}}